jimmah
@jamesdouma · Aug 30, 2023
Should at least 4x their training speed once the H100 cluster is fully operational. Quite a bit more if there's a lot of Dojo online - that part we don't know. Their goal is 10x by end of this year and 100x by end of 24 so this is in line with that.
Elon Musk
@elonmusk
Training compute should soon not be so much of a limiting factor.
Very difficult bringing the 10k H100 cluster online btw.
Similar experience to bringing our now 16k A100 cluster online. Uptime & performance are low at first, then improve with lots of work by Tesla & Nvidia.
01:41 AM · August 31, 2023 · 462.1K views
137
181
2.3K