Yann LeCun
@ylecun · Mar 9, 2024
* Language is low bandwidth: less than 12 bytes/second. A person can read 270 words/minutes, or 4.5 words/second, which is 12 bytes/s (assuming 2 bytes per token and 0.75 words per token). A modern LLM is typically trained with 1x10^13 two-byte tokens, which is 2x10^13 bytes.
Parmita Mishra@parmita· Mar 8, 2024This is an essential point people seem to misrepresent.
Elon Musk
@elonmusk
Yes
08:28 PM · March 9, 2024 · 44.8K views
165
22
620