John Carmack
@ID_AA_Carmack · Sep 20, 2020
With the success of FP16 data types in machine learning, there is talk about FP8 data types. I suspect that having a few different curves, akin to sRGB in graphics, would be better. An 8 bit Gaussian distribution, in particular, seems likely to be useful.
Elon Musk
@elonmusk
For Dojo, we’re betting on FP16 with floating floating point (essentially, truncating mostly zeroes), so it feels like FP32 & existing software just works. We considered FP8 for a while, but everyone hated it. I even asked Twitter! For inference, int8 works great.
11:21 PM · September 20, 2020 · 0 views
81
102
2.4K