Part1: Part1: #dailyreport #negativesampleing #sampling #llm #recsys
Negative sampling used in NLP, RecSys,retrival and
classification tasks to address the computational
challenges associated with large vocabularies or item
sets. It modifies the training objective: Instead of
computing the softmax over the entire vocabulary, it
focuses on distinguishing the target word from a few
randomly selected "noise" or "negative" words.
Instead of loss:
: softmax(x_i) = e^(x_i) / (sum of e^(x_j) for all j from 1 to n)
: L = -log(p(w | c)) = -log(softmax(x_i))
Anoncheg
npub1pe…0584q
2024-11-03 20:52:45 UTC
Author Public Key
npub1peq6e8s498rp8ts36c6zfedmdmf7v8wvfjc4dn9a337ml3ntvugsv0584qSeen on
wss://relay.mostr.pubPublished at
2024-11-03 20:52:45 UTCEvent JSON
{
"id": "a07df802e574d5cc12b4bf6732d365b3e1c110921b8280d10f342a2c32f24e3d",
"pubkey": "0e41ac9e1529c613ae11d63424e5bb6ed3e61dcc4cb156ccbd8c7dbfc66b6711",
"created_at": 1730667165,
"kind": 1,
"tags": [
[
"t",
"dailyreport"
],
[
"t",
"negativesampleing"
],
[
"t",
"sampling"
],
[
"t",
"llm"
],
[
"t",
"recsys"
],
[
"proxy",
"https://techhub.social/users/Anoncheg/statuses/113421003333939568",
"activitypub"
]
],
"content": "Part1: Part1: #dailyreport #negativesampleing #sampling #llm #recsys\nNegative sampling used in NLP, RecSys,retrival and\n classification tasks to address the computational\n challenges associated with large vocabularies or item\n sets. It modifies the training objective: Instead of\n computing the softmax over the entire vocabulary, it\n focuses on distinguishing the target word from a few\n randomly selected \"noise\" or \"negative\" words.\n\nInstead of loss:\n: softmax(x_i) = e^(x_i) / (sum of e^(x_j) for all j from 1 to n)\n: L = -log(p(w | c)) = -log(softmax(x_i))",
"sig": "d40e338af8a9a454edaa81a39aea8103d43194d41da845f5b36004f48b8d7b368fc87fdb0593e5871f25ac1e8b74d50eda4f095ce5f2ebd8349568d75a05a0f5"
}