Jim Fan
@DrJimFan · Jul 10, 2023
I'm calling the Myth of Context Length:
Don't get too excited by claims of 1M or even 1B context tokens. You know what, LSTMs already achieve infinite context length 25 yrs ago!
What truly matters is how well the model actually uses the context. It's easy to make seemingly wild
Elon Musk
@elonmusk
06:56 PM · July 10, 2023 · 784.6K views
210
151
2K