Twitter Spaces: Twitters Software-Stack mit George Hotz 21. Dezember 2022 Musk schaltete sich in einen vom Programmierer George Hotz moderierten Twitter Space, um über Twitters Software-Stack, die Gründe für einen Neuaufbau und die Finanzen des Unternehmens zu sprechen — samt eines hitzigen Wortwechsels mit einem ehemaligen Mitarbeiter.
Musk schaltete sich in einen vom Programmierer George Hotz moderierten Twitter Space, um über Twitters Software-Stack, die Gründe für einen Neuaufbau und die Finanzen des Unternehmens zu sprechen — samt eines hitzigen Wortwechsels mit einem ehemaligen Mitarbeiter.
Transkript George Hotz
No, no, no. I mean, like, what do you mean by. Are you crazy?
Co-host
No, no, seriously, come on, buddy.
Audience Member
Come on. Who.
George Hotz
What do you mean, who am I? I don't know. You gave me the mic.
George Hotz
No, I didn't give you the mic. I'm doing the mic. And let's. Let's keep it. Let's keep it civil in my space.
George Hotz
Yes, like what? No, no, man.
Co-host
You're in charge of the servers and the programming.
George Hotz
Whatever. Like what? Keep things technical in my space. Please. Take me from top to bottom. What does the stack look like right now?
Co-host
What's so crazy about it?
George Hotz
What's so abnormal about this stack versus
Co-host
every other large scale system on the planet?
George Hotz
Buddy, come on, give it to me. All right, so first off, amazing. Wow.
Elon Musk
You're a jackass.
George Hotz
Okay, okay, okay. All right. No, no, no.
George Hotz
I got no credibility. Everybody, I got no idea. First off, my state.
Audience Member
Sybil, you know, like, you removed him from speakers.
George Hotz
Can I go into it?
George Hotz
Okay. One of the biggest problems is that you can't run Twitter locally. So when I was at Facebook, you could run all of Facebook on a laptop. There is no way to run Facebook to run Twitter outside of a very bespoke configuration in a data center. And it makes it nearly impossible to build anything new on it.
Elon Musk
Yeah, I mean, there's 20 million lines of Scala. Think about that. That's insane. Did you post that diagram?
George Hotz
No, I didn't post the diagram. But the joke of the diagram isn't that there's that many pieces, it's that when people think of a microservice, they think of 200 lines of code, not 200,000. Yes, and it's 200,000 for most.
Elon Musk
It should be trivial to do something like add View Count. It should be like literally another sort of line in a database. It should be trivial, but it's not trivial because of the insane complexity of the Twitter software stack.
George Hotz
I disagree about view count. I think view count is actually inherently a hard feature. But, for example, right now there's a service that runs a model, and there's a service that calls that like, there's a RPC to run a model. Basically, at comma, it's one line. If you want to put the model inside the process at Twitter, it's two weeks of the best engineer's time.
George Hotz
That's something that should actually be one line. I disagree that view counts should be easy, even on the best stack in the world. But there are some things that are the likes feature that I wanted to ship in search. Should have been five lines.
Elon Musk
Yeah, I know. I. Well, so I think if you. We do count view counts, we just batch them. So if you reduce the requirement on view count to say like, look, we don't care about the view count being real time, it's okay that it be every hundred seconds instead of every one second. You reduce the load by two orders of magnitude, obviously.
George Hotz
sort of. You definitely reduce the load two orders of magnitude for tweets that are viewed all the time. Right. But for tweets that are viewed every 200 seconds, it's the same load. It's. If you're basically saying you're going to read every tweet from the table to get the views every hundred seconds, that's a lot of reads.
Elon Musk
Look, the point I'm trying to make here is view count is already counted. Like it's already a thing that is happening. But view count, currently the issue is that it's going to Google Cloud. Instead of the On Prem stuff, the view count is making a call to Google cut in batches at a rate that is towards the magnitude slower than what is needed to render it with the current pipe. That's the actual issue. But if you then say, okay, look, we don't care about it being real time because something like a like is being updated about every second or so. But if we said on your tweets.
George Hotz
Yes, on most people's tweets.
Elon Musk
No, no, I'm just saying like the update frequency, it's near real time. But the point I'm trying to make is view counts are already counted. They're just being batched and being sent over a slowpipe to an analytics software on Google Cloud. Whereas the likes are using the On Prem database. But you could, that's like an arbitrary decision. It could use the On Prem database as well.
Elon Musk
So if it had not been an arbitrary decision to go off prem to Google Cloud, then it would be trivial to add likes.
George Hotz
It would be considerably easier.
Elon Musk
It wouldn't be onerous to add views. So if you're not like too precise about exactly how many views there were, if you like, let it be a little, let it float a little bit and you occasionally drop a view count that. So it's because like, we're not like trying to balance a bank book here, then I think views actually would be relatively easy
George Hotz
if views were in the same database as likes. Yes, it would be easy.
Elon Musk
Yeah, that's what I'm saying. And it should be it's absurd that views are on a totally different database in Google Cloud instead of the on PREM
Elon Musk
So anyway, that's sort of an unnecessary complexity. I mean the Twitter stack right now is like basically combining the difficulties of three different worlds, which is running a complex real time on PREM database that's in three data centers in Atlanta,
Elon Musk
year in Oregon and one in Sacramento. Sacramento is like possibly the worst place to have a data center because it's hot. And then like the summer it got so hot that the data center failed because the H Vac didn't work. But then in addition to that, you've
Elon Musk
also using aws, so using Amazon and using Google Cloud. So all three are required to operate Twitter fleets.
George Hotz
Fleets uses aws. They built all the fleets in aws. Yeah, I read the Google Docs for why too, for why they put fleets in aws. Basically they couldn't get enough provisioning for the internal data center.
Elon Musk
Yeah, I mean like Sam Pilar has been helping out and like I think Sam was saying there was like some pretty nutty amount of data storage is actually analytics for app versions that don't exist anymore. Yeah, like I think it was like 15%, maybe 20% is like, was literally logging data from apps that don't exist. This is like, okay, that's. Wow. Anyway, there's a lot of crazy stuff going on.
George Hotz
25% of the CPUs are running the ads prediction model. It's the tiniest model those CPUs. Only 30% is actually running the model. 70% is serializing and deserializing the data into four different formats.
Elon Musk
Yeah, and here's the crazy thing. So which is like really kind of mind blowing to think about it, but it took me a few weeks to actually even wrap my head around it. The Google Twitter's advertising is actually for the most part irrelevance optimizing. And so it's like, wait a second, like, what do you mean irrelevance optimizing? Well, when I ask people who have used Twitter for a while, it's like, how many things have you bought on Twitter? The answer is usually nothing. But it's like, well, how many things did you buy on Instagram? Oh, I bought a whole bunch of stuff last week. It's like, well, don't you think it's interesting that you wouldn't have accidentally bought something on Twitter after all this time? And the reason you haven't is because the ads are maximizing irrelevance. I feel like I'm stuck in a Douglas Adams novel. It's like the spaceship was powered by maximum improbability. And so you're like, wait a second, why on earth would you maximize irrelevance? Well, because the team was told to maximize the impression, the number of advertising impressions. So what happens if you try to maximize advertising impressions? Well, you actually are going to maximize irrelevance because if you. If somebody is spending five minutes on Twitter and two of those five minutes are spent clicking on an ad, you've now lost 40% of your impressions. Like head exploding emoji.
George Hotz
Yeah, Dwell time is probably a much better signal for ads than impressions. I mean, this is what made TikTok. What makes TikTok. What makes TikTok so good? They have dwell time on all the videos. The time you spend on an ad is probably better than whether you saw the ad or not, though I somewhat disagree. In the limit, if you had an ideal ML model trying to maximize impressions, I think you'd end up with someone who just scrolls Twitter and looks at the ads all day.
Elon Musk
Look, I'm just saying that, like, when I was going through this with some of the engineers a few weeks ago, it took me a moment. I just. It was so insane that I couldn't comprehend it. It took me a while to actually understand that because I was saying that, like, no, we want people to click on the ads. That's the whole point of having an ad is so that you might buy something. That's the reason for advertising. And so, you know, an ad that is highly relevant is content. An ad that is irrelevant is spam. So we want ads that are as relevant to people's needs as possible. And yet I see many ads that, in fact, most ads are not relevant. And when I ask people, what are the concerns about Twitter, it's like, well, Twitter keeps showing me the same ad that I don't care about all the time. I'm like, well, that seems crazy. But then I found the reason, which is that they were being asked to maximize impressions instead of maximizing actual clicks and buying things, which is what they should be doing.
George Hotz
When you run an ad campaign on Twitter, you can choose, I think. I'm not really sure what the back end does. I heard the backend tries to maximize money, but you can choose whether you want to run an impressions campaign or whether you want to run a clicks campaign.
Elon Musk
Yes, things have changed recently. So we have, in the, like, you know, basically since the acquisition, I was like, no, guys, we need to maximize relevance. And there actually has been good work in the direction of relevance optimization.
Elon Musk
which, you know, so that, you know, I don't mean to be sort of all negative here, but there's a lot of, you know, there has been good work recently in the direction of actually having ads that are relevant for questions on this.
Co-host
I am super nerdy about ad tech stuff. I'm super curious how Twitter can compete with like Google and Facebook just in terms of their identity network and how well they have like bucketed groups of interest to target. Like, how does Twitter compete?
Elon Musk
Actually, I think Twitter can actually do arguably better for someone that has used Twitter for some period of time. So if you have used Twitter for, let's say, I don't know, a few weeks, and if you have like, you're several hours in and it's, it's the, the signal, the amount of signal that you have is high. You know, what people, what, what, what topics people followed, what, what, what accounts people followed, what they liked, what they, you can say, you know, certainly view count. There's. And a bunch of the stuff is actually used at Twitter for recommended tweets, what is called sometimes the home timeline. You know, if you like, there's like a little stars icon in the upper right of the Twitter interface and it'll say like, for your or latest. So for your is recommended tweets. A lot of people actually don't even realize that the upper right stars icon is a way that you can switch between recommended tweets and accounts that you actually follow. I bet a lot of people on this call currently do not know that that is what that stars icon is.
Elon Musk
But the recommended tweets are actually pretty good.
George Hotz
Home ML is much better than adsml. Home ML is using much larger models, many more features. The adsml is very rudimentary.
Elon Musk
Yes. But essentially they should be going through the same pipeline.
Elon Musk
If Twitter can accurately recommend tweets to you that you don't explicitly follow, it can by the same token, also show ads that are relevant to you.
Co-host
So the way that a lot of platforms do ad sales and a lot of purchasers expect ads is, isn't necessarily, I'm going to give you these ads and I trust you to serve them to the right people. It's usually almost the opposite where it's, I want to target this group of people in this region who went to a Jiffy Lube in the last 72 days. That level of micro targeting is something a lot of ad buyers now expect. And like, it's a huge part of how CPMs can get so high on platforms.
George Hotz
Like YouTube, regardless of whether they're targeted or not. If your goal is C cpms, you should have a very good idea that someone's going to click on an ad before you show it to them. And you can do this with GoodML,
Co-host
but sometimes these advertisers will serve entirely different versions of the same app.
George Hotz
Turn this guy off, please. He's just talking nonsense.
Elon Musk
I've actually talked to the advertisers. I talked to some major advertisers just today, this morning, and they were looking for, as you would expect, return on investment. So that if they spend a certain amount of money, are they going to get a return in excess of the amount of money spent? Quite rational, not arbitrary or random. So, and I won't say who it is, but there are major Advertisers who spent $60 million on Twitter this year. And they were like, we calculated the roi. Actually, another company helped them calculate the ROI and Twitter was the lowest. And this is why they're having difficulty spending additional money on Twitter, because they're literally looking at the lowest roi. I'm like, yeah, okay, well, that's quite sensible. And then they were like, well, you know, when this particular product is discussed in replies, wouldn't it make sense to at that point provide an ad that perhaps since you are discussing this product, perhaps you would like to buy it? I'm like, yes, that would make total sense. We should of course do that. They were like, yeah, that would be awesome. If you do that, then we will be glad to spend more advertising money. They were very rational and not arbitrary and random.
George Hotz
I'll stand up for Theo, worked at Twitch. He's provided some good stuff on this space so far. While in the limit, I agree with you, it would require retraining advertisers. And I'm not saying you can't do it, but advertisers have come to expect a lot of this micro targeting stuff. Facebook pioneered a lot of it.
Elon Musk
Yeah, look, I think at the end of the day, if you can say like, if you can make a clear equivalence, like, just say like, for. If you spend amount X to generate profit, Y, provided Y is greater than X, you should continue. And provided if you've got good analytics that demonstrate that whoever the advertiser is, you will succeed. And that is not currently the case for Twitter, but it will be.
George Hotz
Do you want to fight every battle or do you want to like, Like. I mean, is this the battle Twitter should be fighting or should we just provide the most standard boring ads interface and just make it good. Like Instagram's.
Elon Musk
Instagram has an excellent advertising interface as, as literally talk to anyone and they probably, if they're using Instagram at all, they probably bought something.
George Hotz
Last week I bought stuff off Instagram ads. Yeah, Facebook has GPUs in the data center too. They're running decent sized models on people.
Elon Musk
Yeah, it's, it's. I don't think we're trying to split the atom here. You know, we're just literally trying to show people ads for something they might actually want to buy. You know, this is not the Manhattan Project, even though the database weirdly is called Manhattan.
Elon Musk
so it's just, I'm just, this is common sense stuff. So like, yeah, like sound like this is a monumentally difficult to implement. It will get implemented and it will get done fast.
George Hotz
Elon, would it be possible to offer
Elon Musk
like a more expensive $100 a month version with zero advertisements?
George Hotz
I mean 1% of people might be
Elon Musk
willing to pay that. Yeah, absolutely. I think it's a no brainer. This is just a question of what complexity do we add when. Okay, so like it's difficult to, frankly it's difficult to overstate the difficulty of adding stuff to the adding features. Features that should be easy to add are not easy to add because the stack complexity is extremely complex.
George Hotz
So can we do refactors? Forgot.
Elon Musk
So we've made some simplifications even in the last few weeks, but this is, it will explode your mind to go through the entire stack. It's like wow.
George Hotz
Well, you have to sort of make a trade off between features and refactoring.
George Hotz
Like if you have X amount of engineering team, you can decide how you want to allocate that.
Elon Musk
Yeah, I mean basically we're going to make you know, a series of I think significant short term product improvements. But then I think we'll need to rewrite in order to have a high velocity of improvement. Unfortunately I think a rewrite is needed. Like say like look back at old school Apple, you know, when they had their old operating system and then Jobs came along with osx, you know, it was a new stack. It wasn't like let's like you know, fool around with the old Apple os. It was like he just took the Next OS and adapted that to Apple and said yeah, your old software just doesn't work anymore. Too bad.
George Hotz
But an operating system doesn't run at scale. Like this is doable, but it'd be, I think it'd be pretty hard to do with continuity to roll out this New stack, promising, no downtime on a clean rewrite. Seems very, very hard. Yeah.
Elon Musk
Nonetheless, I think that's what'll be needed long term.
Audience Member
I actually have a question about the stack.
Audience Member
So you brought up the 20 million lines of Scala before, and you mentioned that the microservices have a lot of the complexity of the Twitter system. So I think microservices to like. I believe microservices for complexity, like a system at that scale won't really microservices land.
George Hotz
Sorry, you're breaking up now. You think microservice yes or microservices no.
Audience Member
I'm fine with the microservices. My concern is the infrastructure or the tech stack that you guys are using, because instead of using commercial or open source or any standardized products, you guys have inbuilt or in house, your own implementations of everything. Sure, you're using Kafka, but your RPC layer is Thrift, which is developed internally. Your database is Manhattan.
George Hotz
Wait, wait, wait, wait, wait, wait. Thrift is a Facebook product?
Audience Member
Which one?
George Hotz
Thrift is a Facebook product.
Audience Member
My point is that
Elon Musk
we got the TFE, we got the GraphQL, we got the Product Mixer, we got the Ranker, we got the Manhattan database, we got the Strato, we got the Tweety Pie, we got the Gizmo Duck, we got the Twitter memcache, and then wait till
George Hotz
you find out they're all built on top of Finatra and Finagle, which are entirely custom Twitter open source Scala projects.
Audience Member
Yeah, I get. My point is that if you have all of this custom and not like super standardized products, when 80% of your workforce happens to leave now, how much more difficult is it for new people or like even existing people to work with that infrastructure and be able to be able to iterate on it, much less like refactor it?
George Hotz
I mean, I think I could do it, but I think it would take me six months.
Audience Member
So, George, you mentioned that at Facebook everything could run locally. I've worked at companies where we had a dedicated team just for developer experience, and their job was literally to Dockerize and containerize everything and just make the app run so people that can code, can code. And it just sounds like Twitter's lacking that.
George Hotz
Yeah, you want to hire those people for Twitter. I think Twitter's definitely lacking that. So, I mean, Elon, how far, how
Audience Member
far are you guys away from having an engineer, like a new engineer on day one? Just, you know, walk in, go through quick orientation, and then get like a single dev instance running and just being able to submit a PR on the same day, or maybe the same.
George Hotz
A dev instance. No, Twitter doesn't believe in that. We have produced and sometimes.
Audience Member
Or local. Any kind of local.
Elon Musk
No, I think to your point, it takes a while. There's so much custom stuff that just understanding where everything is and how it all connects, I think is a matter takes at least a few weeks. So I think it would be difficult to be productive, you know, without at least a month of intense effort. I mean, that. Yeah. And that would be like, you're really good at software, I'd say, if you can do something useful in a month.
George Hotz
I think I understand most of it, but I think a lot of the problems are not a question of understanding. I think they're of just toolings and workflows. The fact that I can't stand up Twitter anywhere except for the data center, for example, some services, when you develop them, you have to. They pack. It's a 1.6 gigabyte binary. It has to upload to the data center. That takes 20 minutes and then you unpack it, and then, okay, 30 minutes later, you've tested your one line change and. Oh, you made a typo.
Elon Musk
Yeah, I'm just saying, basically, to George's point, it's a good point, is that. There's so much complexity and so much interaction across so many different levels that to make progress is very difficult. Progress is being made, but it is very difficult to make progress. When you touch, like one element of the system, it affects like 17 other elements. So, you know, like one argument in favor of, say, a whole bunch of microservices is if they are truly independent, you can add additional microservices without affecting the rest of the system. But that is not the case at Twitter. Like I said, we have this challenge of trying to add view count to tweets, which I think is really going to make the site pop, because you'll see just how many people actually view your tweets. This is completely normal for video. So if you look at even a video on Twitter, it will actually show you how many views the video has gotten. And that gives you some indication of the popularity of the video, how interested are people in the video. So Twitter has views, poor videos, but does not have views for tweets. So you can tweet something and think, man, did anyone. Was it like a tweet fell in the forest and nobody heard it? Did anyone see my tweet? Actually, lots of people saw your tweet, but you don't know, because they didn't want to put themselves out there by. By actually liking it or retweeting or replying. So the site will pop in a big way. Just showing view count of tweets and people realize, holy smoke. A lot of people actually seeing my tweet. That's cool. And then, like I said earlier, we need to add functionality where you can like a tweet without it becoming. Without advertising the fact that you liked it to the world or being able to say that maybe you didn't like it in a. I love that that thing happened. Like the earthquake example. You don't love earthquakes, but it's noteworthy, certainly. And it's putting a heart next to an earthquake. That's weird. So you want to have more fidelity in how you. What you think of a tweet, and you want to have the option of making that how you feel about that tweet, public or not public. That's like, pretty obvious stuff. But making those changes to the existing Twitter system is very difficult.
George Hotz
Why not refactor and make the changes later? Why do they need to be done now?
Elon Musk
I think we need to make some improvement in the near term. And I think we can make some improvement in the near term.
George Hotz
So I ran Git log this morning and I looked at how many commits are going into the code base and what they actually are. Huge green diffs
Elon Musk
for view count or for something else?
George Hotz
All sorts of things. Maybe 10% of them are view count related.
George Hotz
But all of these other ones are still. They're still green diffs. There are a lot of people writing a lot of code today. Timeline Mixer was deprecated for Home Mixer. Great. It's deprecated for Home Mixer. Was Timeline Mixer deleted? Oh, no. Now we have both Timeline Mixer and Home Mixer.
Elon Musk
They're running simultaneously.
Elon Musk
Well, it's not very meaningful to deprecate something if it's still running, obviously.
George Hotz
Yeah, exactly what I'm saying. Right. Nothing gets actually deprecated to Twitter. So Twitter has three APIs, and you can find this on public Twitter if you go to Chrome Inspector. Twitter has a 1.1 API, a 2.0 API, a GraphQL API. It also has 1.1 developer and 2.0 developer. It has five APIs that I could find. And then there's other ones people have talked about too, that I don't know how to find because I don't have Android Emulator on my computer. This stuff needs to be cleaned up and no one's doing it.
Audience Member
Isn't it, it's pretty common.
George Hotz
Hey guys, do you mind if I ask a.
George Hotz
Yeah. So I just wanted to know.
George Hotz
So obviously, as you know, I think with the, the Tesla, you know, and
Elon Musk
what you guys do on the autopilot,
George Hotz
you guys are recreating and redoing a lot of the code. And it sounds like with Twitter it sounds like it's kind of a nightmare.
Elon Musk
So how are you prioritizing what you fix, what you're going to implement, like the views, how are you stack ranking everything, especially if things are taking a lot of extra time to get done. Well, bear in mind like, you know, the acquisition only closed like whatever, like six weeks ago, 5ish. So this is probably like the biggest amount of change that has happened in an acquisition possibly in history. You know, we're at a little over 2,000 people. We were at close to 8,000 people. The site is still running as evidenced by us being on it. So, you know, this is reasonably good progress. The most essential thing to be done at Twitter was to cut the burn rate so that Twitter does not just go bankrupt next year. That was like the, you know, task number one. And there's still work to do on that front, but it's mostly done. Then there's also, it's also important to move the revenue base from being essentially entirely advertiser dependent on advertisers, and not just dependent on advertisers, but dependent on fuzzy brand advertising as opposed to advertising that has a clear roi. And as we're heading into what appears to be a recession, the first thing that companies will cut is discretionary brand advertising. So it is important for Twitter to therefore have a subscriber base. This is the reason for the Twitter Verified. The reason we're able to make progress with Twitter Verified is because it makes use of a product that sort of was pre existing but not very useful, called Twitter Blue. So being able to move to a. Have subscriber revenue be a meaningful part of Twitter revenue is going to be essential for the prosperity survival of Twitter. So that's why that's actually the number one priority is getting the Twitter Verified to work and then also getting Twitter verification for businesses and organizations to work so that you're starting to see that rollout now where an organization can actually identify who belongs to that organization. So like who is actually, which accounts actually belong to Disney and can speak on behalf of Disney or on behalf of say Stanford or any organization. Like if someone says they're affiliated with an organization, how do you know that besides them Saying it in their bio. Well, we're now linking that on the back end so that someone organizations can say, yes, these Twitter handles actually do belong to the company and are authenticated. And this is also, you know, obviously an avenue for, you know, business revenue, business to business revenue. So the subscriber revenue and sort of back end business revenue is really important as we see a rapid decline in brand advertising.
George Hotz
But these are all features and they're all at the expense and refactors.
Elon Musk
Yeah, no, they are.
George Hotz
Okay, thank you for acknowledging.
Elon Musk
I think we won't get to the refactor if we don't make these changes.
Elon Musk
Because we used to be bankrupt.
George Hotz
What's the Runway?
Audience Member
Could I jump in with a quick question to cut the Runway silence there?
Audience Member
Yeah. So I feel like, you know, Elon, you're our philosopher king. Plato would be very proud of you. You know, you're extremely technically competent and, you know, you, I think, really understand how to drive things in a good direction. But, you know, I really love how you're also leaning into the voice of the people and letting them have a say as well. But you can't have a million backseat drivers. So I guess my question to you is, how do you integrate the voice of the people when there are a lot of people that just don't have the right information to make decisions? And how do you create the right governance systems to really find the signal from all of the noise to get Twitter to become something more like a republic rather than a dictatorship, you know? Curious what you think.
Elon Musk
Yeah, well, I mean, you know, in the, in the relatively near term, like, these things are all kind of moot if Twitter is already into the ground financially. So I was thinking about the Runway question, like just to sort of, you know, give people some perspective here. Twitter was tracking to spend about $5 billion next year.
George Hotz
We needed the trip to Disneyland.
Elon Musk
Yeah, exactly. Then because of the nature of this transaction, where there's $12.5 billion of debt and the Fed rate has been going crazy, there's on the order of a billion and a half ish of debt servicing that's required. So we're talking about like net cash outflow if you didn't make any changes of on the order of like 6, 6.5 billion next year with revenue probably tracking to 3. So that's like a negative cash flow situation of $3 billion a year. Not good since Twitter has 1 billion in cash. So that's why I spent the last five weeks cutting costs like crazy.
George Hotz
I think we also got to get the advertisers.
Elon Musk
Yeah, yeah, we do need to get the advertisers. But like I said, I've spoken to a number of the advertisers. They're. Their requests are not fuzzy or irrational or anything. They're like, quite reasonable. They're like, just show us an ROI that makes sense. And I'm like, yes, I agree. If I were in their position, I would also want an ROI that makes sense. Especially when headed into difficult economic times where these questions are asked a lot more than in prosperous times. In prosperous times, there is plenty of budget for advertising and you can get away with unclear roi. But when times are tough, then the hard questions of return on investment are asked. And when you do not have a clear answer, then advertisers don't want to advertise because they're being sane. That's the main issue. So this is important. So now with the changes that we're making here on massively reducing the burn rate and building subscriber revenue, I now think that Twitter will in fact be okay next year. I think we will be about, hopefully, sort of roughly cash flow break even. That's what I expect for next year. But this will be quite. This will be difficult, but I think that's what will happen is roughly cash flow break even, as opposed to minus 3 billion ish on worth 1 billion of cash, which would be therefore dead. So that's the reason for my actions. They may seem sometimes spurious or odd or whatever. It's because this is not. We have an emergency fired roll on our hands. That's the reason. Not because I am like naturally capricious or at least aspirationally, I'm not naturally capricious. So. But if you sort of, if you say you're looking at it from my standpoint and you're saying, okay, wow, this company is like, like basically you're in a plane that is headed towards the ground at high speed with the engines on fire and the controls don't work.
George Hotz
I use the same analogy at comma, can we afford good food?
George Hotz
it's really important to me.
Elon Musk
No, I think we can afford good food. The madness of, for example, the headquarters food budget was that it was $13 million a year of fixed rate, no matter how many people showed up. So if the normal meal price would have been, if the building was fully occupied, $20 for actually quite a good lunch, the food was good. But if you drop to 5% occupancy, you've now multiplied the effective meal cost by 20 and that's where you get a $400 lunch, literally.
George Hotz
Absolutely. The food was great. I loved the chia seed pudding and the smoothies in the morning. But yeah, you got to tell everyone to come into the office. It's crazy to me that people think they can have a job where they don't show up to an office.
Elon Musk
Right. And it seems like a logical thing if, if you have a building where there's literally 5% occupancy that you would seek to renegotiate the food budget and not, you know, have it be 20 times more than it should be. But that was literally the case. So I, you know, the food's a
George Hotz
lot worse now than it was. A lot worse.
Elon Musk
Well, we can definitely improve that. That's not like, I think some impossible thing to overcome.
Elon Musk
this is not, you know, it's not going to break the bank. The issue is more you can't pay like $13 million a year for a mostly empty building for lunch and not even including dinner.
George Hotz
I could save $13 million by switching ads prediction from TF serving to ONNX runtime alone.
Elon Musk
Yeah, I'm saying we can totally have a good lunch. We're not in like lunch purgatory for the rest of time. We may have gone too far is what I'm saying. I admit we may have gone too far in cost cutting with respect to the lunch that, that could be the case. And so we'll have to sort of. We may have overcorrected in that regard and we can fix it. This is not an insurmountable problem. So.
George Hotz
Yeah, so I'm like, refactors food. How about free speech?
Elon Musk
Well, I mean, free speech, let me tell you that if somebody's got to pay the servers, okay, so the speech has got to cost at least $8 because otherwise how do we pay the friggin server bill? You know, there's like a billion and a half ish, you know, of all in server related costs and you know, we're trying to get that lower, but it's a lot, it's like not trivial and a minimum, you know, like there's like somebody's got to pay the bills. So advertising for sure, but there's some degree of subscriptions. And then on the, on the speech front, like my general philosophy on that is that we should hew close to the law in any given country, you know, so
George Hotz
yeah, Elon, that makes me so happy. If that's actually what the policy is. I'm upset when I see Paul Graham get banned for mentioning Mastodon.
Elon Musk
Yeah, that was a mistake.
George Hotz
Okay. I mean, yeah, I think there's a far cry, you know, look, you can make arguments for Kanye west, you can make arguments for Elon Jack, but you can't make an argument for Paul Gray.
Elon Musk
No, I agree, that was a mistake.
George Hotz
Okay? Can people post Mastodon now?
Elon Musk
Yeah, Fucking post Mastodon all goddamn day long. I don't care.
George Hotz
Yeah, because we're gonna build something way better than Mastodon, right? Like, come on, how hard could it be to beat these guys?
Elon Musk
You know, from an evolutionary standpoint, how'd that work out for the Mastodons?
George Hotz
I don't know what a mastodon was.
Audience Member
I still feel like there's something to be said about like Open source projects though. You know, like you have Linux that you know, out competed windows. Like you like big, big networks of like armies of volunteers can maintain, you know, pretty complex things like yeah, like sure, maybe Linux is run a little bit like a monarchy, like George said earlier in the space. But you know, like I do think that there's something to be said about Open source software in general, you know, and it's, it's like anti fragility.
George Hotz
I mean Linux is absolutely run like a monarchy, but Linux is not Twitter. Linux has a large code base because it needs to support many different device drivers and many different devices. Twitter needs to support basic core functionality and I think open sourcing it. There's no value in the code, right? The operating system has value in the code. The value of Twitter's in the network,
Audience Member
I suppose, but there's still like, you know, composability arguments, right? Like you guys could reuse and other people could build on top of.
George Hotz
If Twitter, if Twitter was open source tomorrow, what would you build?
Elon Musk
Honestly, I think we should just post the source somewhere. Frankly.
Audience Member
That would be sick, Elon.
George Hotz
No one would ever get it to run.
Elon Musk
Well, I mean listen, that would just, I think, finally explain.
Elon Musk
That is explained with words because you can just look at it.
George Hotz
Dump a tarball Twitter source. It's unbelievable.
Audience Member
So I have two ideas that I would use.
Elon Musk
20 million lines of Scala and you can just watch and read to your heart's content because.
George Hotz
I think it's actually, I think those are just the used lines.
Elon Musk
Round numbers, round numbers, you know.
Elon Musk
We'll take a million.
George Hotz
Oh yeah, well that's just Scala, you know, we have a lot of Python as well.
Elon Musk
Yeah, but actually there's a lot of,
Audience Member
a lot of most open source code quality is like really poor and On Linux, most of the contributors are paid by like intel and other big firms. So it's not really a fair comparison at all. Yeah, sure, but you still have like composability though, right? Like a lot of people can build on top of what they're building and then that makes like, like, I don't know.
George Hotz
Yeah, yeah, I mean that's absolutely true,
Audience Member
but who's gonna sponsor it? And no one's gonna sponsor open source engineers unless.
Co-host
And composability is not inherent to open source. Composability is a thing that takes a lot of effort and money and time as well. Like there's a lot of code bases that are open source that aren't composable at all. The theoretical potential is there, but it's not like a thing almost always.
George Hotz
No, no one is going to volunteer to your open source project to come in and do meaningful refactors that reduce the code. They're going to write a feature, be like, please merge it so I don't have to maintain it. And then.
Elon Musk
Yeah, that sounds about right. Still though, I have a quick idea.
George Hotz
Elon, is there any way to like
Elon Musk
gamify getting more people to use Twitter?
George Hotz
Because I think a lot of people hear about Twitter in the news, they
Elon Musk
don't really understand how to use it or get the app and use it. But maybe if there's a way to encourage other people to show people how to use it, it could help get more daily active users eventually.
Elon Musk
well, so daily active users have been growing reasonably well, especially if you count real users and not like bots and trolls. So I mean like one of the super crazy things that I discovered, I only discovered this like basically about 10 days ago, was that Twitter was being scammed to the tune of $60 million a year for SMS texts, not counting North America. So I'm like, what do you mean? I was like, what do you mean? Like there's $60 million of SMS text.
Elon Musk
Twitter is paying telcos basically. There are telcos who are not being super honest out there in other parts of the world who were basically gaming the system and running like two factor authentication SMS techs over and over again and just creating a zillion bot accounts to literally run up the tab so that Twitter would SMS text them and Twitter would just pay them millions of dollars without even asking about it. And now this created some problems because I said, well, we got to stop that. I said, like, you know, cut off any telco that's got fraud above 10% now that did cause some havoc in many parts of the world. And then people were like imputing like bad motives to it, but it was literally I just said to the team, hey, cut off any telco who's where. The fraud's above 10%. That turned out to be 390 telcos. I'm like, dan, that's hot. Oh my gosh. Outside of North America, you know, so I'm like, okay, I didn't know there were that many telcos in existence, but turns out there's some. You know, you get to like the fourth tier telco in Bilo, Russia and you're like, you got some sketchy things going on.
George Hotz
Now you have people complaining that they can't log into Twitter because their SMS doesn't work.
Elon Musk
Yes. And so like now we're going back to the telcos and saying, listen, if you, if you stop scamming us, then we will gladly pay you some amount of money in SMS to texts, but you can't turn a blind eye to relentless bogus SMS texts. So then we're having that conversation with the telcos and then we can turn them back on and say, listen, you just be reasonable, we'll take 10% of fraud, it's fine. But we're not going to take 90% fraud. So we're not going to spend $60 million a year on literally SMS texts outside of North America. So that. So a bunch of those users were just fake and like literally for the purpose of running up Twitter's tab on SMS texts.
Elon Musk
This is insane. So now, but to your point of making Twitter easier to use, like the obvious move I think is like, if you download the Twitter app or you go to Twitter, it should, it should put you into the search page and immediately show you like what's trending. So it's like, as opposed to saying, hey, you need to download the app, you need to install the app, you need to give us a username, you need to give us a friggin phone number and making it like an escape room to open a Twitter account. It's like we're going to give you a bunch of math puzzles basically to open a Twitter account. And then once you're in the Twitter account, you're sitting there with nothing to do. This is insane. It's not clear to me you should actually have a Twitter account at all. Unless you need to make a write action. Unless you need to write something. So if you want to like or retweet or tweet or something like that, okay, at that point, yes, you need an account.
George Hotz
Then why did people put the pop up back? They said you said they could do it.
George Hotz
The pop up that I emailed you the first week that I wanted to remove. The one that when you scroll on Twitter when you're logged out.
Elon Musk
I didn't say we should put the. Okay, that's the miscommunication.
George Hotz
Oh great, let's get rid of it again. Because I got rid of it entirely. I got rid of it entirely. They added it back and they messed up adding it back so it wasn't dismissible and then they fixed that five days later. But like they didn't mean to make it non dismissible and it was non dismissible and I'd love to remove it altogether. Yeah.
Elon Musk
It's not clear to me why unless there's some reason that I'm not aware of, we should not be prohibiting just a read only scroll.
George Hotz
No, there's no reason except that people are scared to lose mdao. Yeah.
Elon Musk
This optimization of MDAO is like I think creating a bizarre incentive structure that doesn't make sense.
George Hotz
It's endemic in the organization. They love mdao.
Elon Musk
Yeah. I mean the thing that actually matters is like true user minutes, not like fake user minutes. But we're confident this is a real user. We're confident this user actually spent time on the app as measured by screen time on the phone, which is since the phone is the primary user use case and then, and then to get like another layer deeper is what we said the thing to optimize is unregretted user minutes. So I've heard like a lot of people say like, hey, I spent two hours on TikTok, but I regret those two hours. I'm not saying I want to trash TikTok because it's quite compelling to a lot of people, but. But I do often hear that they. People regret the amount of time they spent spent on it. So I think the thing we'd want to optimize for is unregretted true human user minutes. That's the thing to optimize for, not mdao. And then with respect to advertising, as I said earlier, you want advertising that is as close to content as possible where it's something where it's a product or service that you want when you want it, then that's actually useful. It's actually at that point becomes indistinguishable from content. So these are the things to aim for. We're going to get rid of this MDOT bullshit and we're going to get.
George Hotz
Yeah, let's get rid of all the pop ups. Let's get rid of when you click likes and quote tweets. Because fundamentally what these things are doing is they're lowering total minutes on the website. People see that pop up and they
Elon Musk
click the X. Yeah. I mean the thing to that it is important to sort of have some authentication mechanism so people can't see. You can't just create 100,000 instances of virtual machines. Just create 100,000 virtual machines and just create and have them all behave like users. Although at this point with ChatGPT you could probably just crush frankly, because ChatGPT would exceed. I think the.
George Hotz
Oh, it's about to be impossible. Capture is about to be impossible. I don't know what we're going to do.
Elon Musk
Yeah, but I'm like, at least we can try. At least we can make it like. So it's not trivial to have like 100,000 Bard accounts, which it currently is.
George Hotz
The best things, the best things to do are like social proof kind of metrics. Gmail. Remember OG Gmail with the invite?
Elon Musk
Yeah, Just follow. You follow the tree. So if like, if one. The referral basis and if one put. If there's one referral that goes sideways, you just look at that and you cut off that entire referral tree. That's how beginning.
Audience Member
George, can I get two questions in or two thoughts?
George Hotz
Yeah, what do you got?
Audience Member
So I'm in San Diego on my Starlink right now, so this is pretty awesome. I. So I'm a father of four and you want to have two ideas to help boost revenue. So my kids, they're going to be exposed to advertisements no matter what. Could we have like a Twitter nest feature where parents can like let their kids have accounts and the parents can control what advertisers their kids see. Because my kids are going to buy Robux or you know, they're going to be buying stuff no matter what and I'd like to choose what they're exposed to.
George Hotz
Refactors first.
Audience Member
I agree with that as a software developer. Then the next thing is there are tweets that I want to boost that are other people's tweets because I just.
George Hotz
Whether it's a political party or you
Audience Member
know, some move, just. Yeah. Could we see like a paper trail of where the money comes from? So going back to what Pablo said. Pablo? Yeah, Pablo said earlier how to see through the noise. I think it's a lot easier to see through the noise when we can see the paper trail, the money that it's influencing the tweets and I think that's what as an older user I
George Hotz
was frustrated with the bots and I'm fine with people paying to get things
Audience Member
boosted, but I'd like to see where, what the source is, where it's coming from. That's it.
Elon Musk
Okay. Yeah, agreed.
Co-host
I did see a post a while back about video monetization. Is that a thing you're down to talk more on? Because I have a lot of thoughts after my time at Twitch.
Elon Musk
Twitter certainly needs to enable longer videos. I wasn't clear on what the limitation on the Twitter video length was, but I was recently told that it was a 2 gigabyte file size limitation, at least in the current stack. So like it's difficult to have video monetization if the videos are like extremely short. So. But if you have a, you know, better codec than is currently used at Twitter and resolution that is reasonable for a phone, you can have some pretty long videos in 2 gigabytes. So like the first approximation, I feel like the first order is like you need to actually be able to post a sufficiently long video. If you cannot do that, then nothing else matters.
George Hotz
Yeah, I don't think most people have access to that. So when I did that live stream before I joined Twitter, I had to get Media Studio and I actually got this. My friend was like, oh yeah, I know a guy in a telegram group and he PayPal them 30 bucks and got access to it. So that's what it takes to get access to long video on Twitter today.
Elon Musk
Yeah, well it's my again, I have to like sort of a lot of things I should say. I should preface by this is like what I think someone told me as opposed to like this is the God's honest truth. So that's an important preface for anything I say it. What I say may not be true. At least it's my recollection of what I think someone told me. So take anything I say with a grain of salt. But Anyway, if the 2 gigabyte thing is real, then we can actually have long videos with an improved codec. The current codec I think is quite old and then having a resolution that's manageable on a phone, not super high def, especially if it's something like a podcast where you don't need flexi every detail in high res, then it's just like basic stuff like having a video ad attached that is at least reasonably relevant to the material. So if it's a video about technology, that the ad is a technology related ad, it's a video about gardening. It's a gardening related ad. It's sort of stuff that would make sense. Like I said, not splitting the atom. It's pretty basic and that's what needs to happen on that front.
Co-host
The main problem that we saw was getting good enough ad inventory for video ad. We found it was a lot harder to build up a large pile of ads to serve a distributed enough group of people. And it took a while for Twitch to even have bucketing of ads for users in the first place. Are you confident in the ability for Twitter to acquire video ad buckets and targets?
Elon Musk
No, I think that is a legitimate issue. And actually if I look at YouTube, because I actually watch YouTube without YouTube Premium just to kind of see what kind of ads are being served here and the like, very often we'll see sort of nonsense ads that are like quite scammy and irrelevant to the subject. So if YouTube is even having trouble getting ads that are relevant to the subject, then probably Twitter will have some challenges that are significant. But I think part of this is also just like reaching out proactively to companies that do things in a particular arena and say we have inventory that is relevant. Why don't you try advertising with something that's in this genre and see how it goes? And then like I said, if you can actually have a logical return on investment for advertisers, I think en masse they will be logical. You might have some irrational actors, but for the most part they will be logical. So yeah, but I do think that the ad inventory thing is an important problem. If you only have ads of a particular type, then you're going to have trouble achieving relevancy because you simply lack an ad that is relevant.
Co-host
It also has to be a diverse enough set within that because video ads are much more frustrating as a user to have as repeats. So if you see the same video ad three times, it's significantly more frustrating than seeing the same suggested tweet.
George Hotz
I feel like they're so bad everywhere. I feel like I get that all the time. Like when I'm watching, like I have a cheap Hulu account with ads, it's the same ad three times. And I'm like, this is terrible.
Co-host
Google's the only platform that's really nailed this. Instagram's doing better here because they're just their ad platforms. Incredible. Nowadays, like meta's been killing it. But the inventory problem for a diverse set of video ads is really tough.
George Hotz
Yeah. Doesn't take talk.
Audience Member
Build a model per user and they spend something like 10 kilowatt hour, like
George Hotz
per year, per user. No, I might have spread some of that misinformation. No, you can read TikTok's paper. You can read it Monolith. It's pretty normal content recommendation engine stuff. They just like do it well, the papers.
Audience Member
Is it actually crazy to do that though?
George Hotz
No. And we're gonna start to see that. Like, I think that the. Look, eventually you're going to build the Infinite Jazz videotape. Eventually you're going to build content that is so compelling that you'll find people just stare at this stuff for hours
Co-host
and hours and hours and hours.
George Hotz
And you can, you can do that as the ML gets better and better and better. But you know, unregretted user minutes, right?
Elon Musk
Yeah, I think that's the right thing to aim for. Like, I think it'd be hard to argue, like, okay, we improved unregretted user minutes. That is definitely a good thing and not a bad thing. So anyway, I gotta throw in the towel here, guys, but I do want to sort of leave off saying that there are a lot of people at Twitter doing great work and it is super appreciated. So this is like, you know, definitely not meant to be. Like, let's trash Twitter nonstop. It's like, you know, at least aspirationally meant to be self critical and to say like, okay, what, what can we do to make things better? So, you know, I just do want to make sure that this does not come across as being unreasonably critical of those who are actually working hard to make Twitter better. So with that, good night guys.
George Hotz
Very supportive. Thanks, Elon. See where Twitter 2.0 goes.
Audience Member
Hey George, can I ask you a question about engineer numbers?
Audience Member
Originally why I joined, I didn't think Elon was going to join.
George Hotz
Well, I mean, Elon joined. So after Elon joins, you know, he's like, he, he, he, he's the only one who's allowed to say anything about Twitter. You know, I'm not like, I can't, I can't. I'm not going to reveal how like Twitter stuff works. I signed an NDA.
Audience Member
Exactly. So I'm, I'm a software engineer. I similar situation of.
George Hotz
On a project where there's been a lot of turnover and the skeletal skeleton crew.
Audience Member
I feel like I'm being pulled between, like, I want to be doing software, but I also have to understand the business better too. And do you think that's like a bad position to be in when you, when you as an engineer, you got to Start trying to innovate the business.
George Hotz
And let's, let's, let's bring this space back to. I mean, you know, Elon shows up, he'll take any conversation, any direction he wants. But yeah, let's bring this back to, you know, the question of the space, which is scale, complexity and number of engineers. No, and look, it's great to have Elon join. You know, it's so nice to think Mark Zuckerberg would show up in the space. And, you know, Mark Zuckerberg is not even the worst. You think Sundar Pichai would ever come talk informally about Google? This is something right here. This is something I absolutely love about the new Twitter.
Co-host
Like that YouTube actually does this, which is really cool. Huge shout out to her.
George Hotz
Wait, YouTube does this? Who?
Co-host
Yep, the CEO of YouTube. I, Susan Wojcicki, believes her.
George Hotz
Susan Wojcicki has like. I don't know how to pronounce her last name. She has like Twitter spaces where she just casually talks about YouTube, not Twitter
Co-host
Spaces, but she hops into the conversation on Twitter in the replies all the time. And she shows up on like, really, like small people podcasts on people who are known for pushing back on the YouTube platform, like regularly. She's been surprisingly transparent. Like, I've been super impressed with the leadership at YouTube.
George Hotz
Then why'd we get rid of dislikes?
Co-host
Because of the. The net use of them was harmful enough that they decided it wasn't worth maintaining. It was a long battle internally. Multiple interviews where she talks in depth about this with both Marques Brownlee and with Ludwig.
George Hotz
I appreciate that. You know, I haven't seen these interviews. I haven't looked. I've just complained about dislikes. So, look, I'm. Personally, I'm a huge fan of transparency building in public. You know, it's just, it's cool to see.
Audience Member
Hey, George, was this space inspired by John Carmack leaving Facebook?
George Hotz
I mean, that's one of the things too, right? Like, this is really not all about Twitter. We can discuss John Carmack leaving Meta. Yeah, I mean, look at what he said, right? Like, you know.
Audience Member
Yeah, something like 5% GPU usage. Oh.
George Hotz
I feel like I can say absolutely anything I want because I don't work there and I have no NDAs in place. Place with there. Yeah. I mean, you know, look how much they spent on Horizon Worlds.
George Hotz
Have you actually used Horizon Worlds? Horizon Worlds is the most expensive video game in history.
Audience Member
I've never heard of it.
George Hotz
Exactly. It's their metaverse thing.
Audience Member
I've never.
George Hotz
I'VE never actually used it. I've only seen, seen videos of it.
Co-host
This point, I'm convinced every social media company needs to, like, spend too much money to build a game at their peak. I watched the same thing happen at Twitch. They built a karaoke game. They tried to con or contract Harmonix to do it.
George Hotz
They're like, wait, they.
Co-host
Nope, the scope here is absurd. We're not doing that. And they hired a whole game dev team and I had to teach them react after when it was a disaster. Good times.
George Hotz
Yeah, I think, you know, the metaverse could be so good, but you need a John Carmack, like, character driving it forward.
George Hotz
Because otherwise you're going to end up with the same sort of blow. You end up with all these companies and, you know, it'll be terrible. We got some new speakers.
Audience Member
Hey, George. Thanks, man. Hey, I was impressed with the. With the way everyone handled that Runway question that was. That really stuck out. And so I was curious just to like, kind of download those numbers real quick. So it sounded like he was saying 3 billion a year in cash burn was what he was expecting with like a billion in the bank. I'm just trying to, like, track through all that because, like, I don't know if anybody knows, but I would imagine the payroll number previously was probably like 1.5 billion cut to like 0.5 or something like that. So I'm just trying to figure out.
George Hotz
I have Dan. I don't think he's supposed to talk.
Elon Musk
I don't think he wants to talk about Twitter stuff on this without.
George Hotz
Yeah, no, I'll talk about that quickly. I will. I will talk about that for a minute. I mean, I did. I did actually really appreciate that Elon answered it. I. Again, I know nothing about this. Everything I know about Twitter's financials comes from public disclosures beforehand. And I'm not just saying that I actually know nothing about it. So, you know, that's. I believe what he said was, Twitter's burn is 3 billion and they have 1 billion in the bank. And that's not a long run.
Elon Musk
It's like four months or something.
George Hotz
Yeah. So, again, things all are above my pay grade. I don't work at Twitter anymore.
Audience Member
Just the one thing I think to be aware of and maybe to factor in is just that I believe that the acquisition put a billion dollars of debt expense onto the company.
George Hotz
And Elon did say, yeah, that's correct.
Audience Member
And so, like, I kind of pressed him in a previous space, and I actually really appreciated him engaging with me on it just like you're saying. And he, you know, he. I think, I think if I were to summarize sort of like his answer, I think he kind of feels like, well, Twitter was going to get run the, the ground if he didn't buy it and stuff. And I don't know if everyone else agrees with that, but I think he kind of sort of insinuated that he was worth it for that billion year of expenses. But that's like the equivalent of like probably like the entire company payroll now or something like that, like for one guy. So it's just like I'm just trying to like wrap my head around it.
George Hotz
Look, yeah, I mean, look, this is. I, I can't really speak to that stuff. I know the Runway of, I know the Runway of my startup and it's considerably longer than, than four months.
Audience Member
Yeah, well, I'm, I'm glad to hear that and I appreciate you having me on and thanks.
George Hotz
Cool. Yeah, maybe we'll keep this going for another like five, ten minutes.
Audience Member
Okay. So if you can talk about Twitter's financials, let's talk about the Twitter's technology instead. I feel like I didn't really get to phrase my question correctly when I asked it last time when Elon was up. So I'll try to phrase it again. I think having all of this custom, non standards, tech, tech stack, a lot of it, a lot of which is like invented in house by Twitter employees, Twitter developers, doesn't that like slow down onboarding or working on new features significantly? Why not use GRPC that a million people out there know how to deal with and instead using thrift RPC, which is something that maybe 500 or 1,000 people in this world know.
George Hotz
Thrift is not at all. This is straight up engineering fact. Thrift was invented at Facebook. I used it while I was at Facebook. Thrift is not the problem. And so also open source Twitter, you can find Finatra and Finagle. So Finagle is a RPC server. Finotra is like a routing engine on top of it. Finagle can do both HTTP and Thrift. I know these are open.
Co-host
People are pinning random shit in the on top. Now that has nothing to do with what we're talking about.
George Hotz
Oh, I don't care. Just ignore it. How do I.
Audience Member
My point was not that it's not, it's. Oh, it's not open source. My point is that how many people out there the. The.
George Hotz
Yeah, okay, so you can say yeah, there's a developer pipeline problem for a language Like Scala. Again, technical debt is technical debt, right? I've never coded a line of Scala in my life and I can watch my last few streams.
George Hotz
Advent of code and Scala. Look, am I an expert scholar engineer? Not at all. But can I make decent pull requests? Yeah, I think now I can. I don't think that's the problem. I think the problem is technical debt. It's just like technical debt can appear in any language, in any environment. And I think the largest, I mean, you know, I have. I have a neural network framework called Tiny Grad. It's a thousand lines and, well, it's a little more than a thousand lines. But the goal is to keep it at a thousand lines because lines are one of the only reliable proxies I know for technical debt. Now, of course, you can game this, right? Any, anything. Once it becomes an optimization target, it ceases to be a metric. This is true, but at the same time, you can only game it. So far, Elon was saying 20 million lines of Scala. What's 20 million lines? You know what on this website really should take 20 million lines.
Elon Musk
Mind if I ask a question?
Audience Member
So I've worked on a lot of big tech companies. Amazon, Google. I worked at Google for like 10 years and a lot of teams and
Co-host
I also worked at a few startups.
Audience Member
So, like, one thing I noticed is with the JVM languages like Scala, I've worked with a few teams that have used Scala. Is that. Or just any JVM language is that there's this additive engineering problem. Like they keep adding code. It seems like the JVM languages in particular, whoever, and sometimes I think the JVM languages, they almost encourage, I feel like code bloat and lines of code and it's like, no big deal. There's so many advanced tooling. There's so many tooling around these languages is so good that they can just pump out. The IDEs can just pump out code. And then it gets to the point where it's not readable. It's optimized for writing, all the refactoring and stuff. So they're not optimized for reading because the tooling is so good, right? And so you end up with like tons of code. And then when people want to add features, they instead of reading the code that's already there, which is enormous, they just, like, they just write it again.
George Hotz
Yeah. So I was gonna, I was gonna first say, like, why would the VM have anything to do with. Right. I can run Python on the JVM if I wanted go get Jython. So I don't think it's the J. It's mostly the libraries.
Audience Member
The libraries. Like, like there's so many libraries and there's so much tooling that you end just with like enormous amount of code.
George Hotz
I, I think it is keep like standing around.
Co-host
Points I really agree with, but then the details are nonsense. Oh, it's like I largely agree with you here. I just, I don't think any one of these things is the problem. It's not libraries, it's not the jvm. But like absolutely. We're at a point now where things like you can write much more code much more easily because the tools are getting better every day. I think we as like an engineering like community need to figure out how to deal with that reality where the average engineer isn't as talented because the bar for engineering has gotten so much lower because the tools are so much better.
Audience Member
But I think what I was saying earlier is kind of right though is that like every company just has all these bespoke different tools and all this like complex little like onboarding thing that takes, you know, like it probably takes a month for someone to ramp up into any.
George Hotz
Not Facebook. Not. Not.
Co-host
Yeah, Facebook held it with their thoughts.
George Hotz
Would you agree? Facebook killed it. Exactly how good I worked. I was full time at Facebook in 2011. How good was Facebook's tech?
Co-host
I was genuinely surprised they had to do layoffs because they've been able to hire so lean compared to the amount of like product surface area they cover, it's nuts. Like they. The way they architect everything is incredible.
George Hotz
These are the arguments I kept making inside Twitter. I'm like, guys, it's. And people are like, George. George has no idea what he's talking about. I'm like, I've seen Facebook. It's beautiful.
Audience Member
CSH alum here from 0508. Long time man. I was wondering if you would agree that with the direction AI is going, I just feel like, I feel like with how much code exists open source and accessible to AI, I feel like code monkeying work of computer scientists and software engineers is going to just continue to get less and less and it's going to start to become more of a. Who can communicate with the engine the best and the most creatively.
George Hotz
It already is if you think about it, right? So programming, unlike say truck driving, is already tool complete. So I mean to specifically talk about the JVM thing, it may very well be. And you can like do this as a Conversation controlled experiment that like do ides like intellij which have all this autocomplete boilerplate functionality. I think it's actually IntelliJ.
Elon Musk
Yeah, I think it is.
Audience Member
I think you're right. It's IntelliJ and it's going to get worse. With Chat GPT and all this code generation, it's kind of, we're like it's not going to be good.
George Hotz
I, I, I mean, yeah, I, I don't even think it depends on your perspective.
Audience Member
Worse, right?
George Hotz
No, the AI, the AI code stuff is ridiculous. AI code generation is ridiculous. Ridiculous. And like it's just the whole thing is laughable. And here's why. Programming is tool complete. Meaning what happens Accounting is also tool complete. When you build better tools for accounting or programming, you end up with less accountants and less programmers having leverage to do more work. Right. This is in contrast to something like Walmart cashiers or, or truck drivers. Right.
Audience Member
No matter.
George Hotz
The truck might get better but you still need the driver to log the hours. All right, fine. You can say Walmart cashiers is a bad example because okay, now you have one person overseeing the four self checkouts. Maybe that is the same thing with technology. So we'll stick with truck drivers, which is not tool complete. But the AI code generation in of itself is like why do you need the extra step? Right? What do you want the AI to do?
Audience Member
Hey George. Hey George. One, one question right on, on the topic I think that you are trying to address is where are you at on, in terms of how many employees or engineers are ultimately needed just to sort of get the job done or kind of have like a good healthy work life balance, like that sort of thing.
George Hotz
Like in terms of work, work, work. I, I think, I don't like work life balance. I think work life balance or whatever it is you want.
Audience Member
Yeah. And the people there want.
George Hotz
No, I don't think, I don't, I think work life balance is stupid to concept. I think that you know, work is life.
Audience Member
Take that part out of the question. Just you know, kind of like
George Hotz
how many, how many engineers do you need to do what?
George Hotz
It all depends on what the task is. Right.
Audience Member
Yeah, I'm trying to think through like sort of what's, what's kind of the unifying vision that that's going to bring all the engineers at Twitter together. Like I like some of the feature points Elon made about like showing the views and how that would raised a lot of interest. I thought he had some like interesting ideas. But like what's the sort of vision and then like, what. What support do you need from, you know, whatever the powers that be are above you as teams to get the
George Hotz
job for the folks at Twitter. I don't work at Twitter. I don't know anything about Twitter. I resigned from Twitter today to talk about AI and for various reasons, I'm certainly down to talk about AI and AI coding.
Audience Member
I misunderstood. Thank you.
George Hotz
If you're not. If you're a speaker right now and you're not speaking, please step down and we'll let some new people.
Elon Musk
Hey, George, I'm sorry if I wasn't
Audience Member
supposed to pin up top, but. But I pinned a little bit about the Amber Alerts. If an Amber Alert goes off in an area and a car happens to drive by a Tesla, what are the odds it doesn't? Can the camera scan its license plate, make a model, and beam that instantly to authorities? And I figure this would be a really great application of Tesla.
George Hotz
Who's writing that? Oh, we got to write code. We got to test it. Ask ChatGPT maybe ChatGPT.
Audience Member
The reason why I even float that idea, though, is because, of course, I think there's something bigger here that Elon might be playing at. Right? His 5D chess, which is he has all these cars mapping the roadways and. And noting the cars that are driving by him. So now that he has Twitter with phones, with geolocation and other drivers pockets, Honda Accord drivers, is he gonna be able to pinpoint when the Honda Accord passes by, the exact location of a Tesla?
George Hotz
You know, what features are really easy to imagine until you have to write them, right? Until you actually.
Co-host
Ideas are so cheap.
George Hotz
Ideas cheap. Until you have to, like, make these things sort of work at scale. So you say, like, is this what someone's planning? Like, it doesn't matter what someone's planning. It matters, like, what's actually going to get built and. Yeah, you know, look. You know, any. Look, I've been working at Common now for six years. You know, like easy it sounds in theory, to make maps from our data. Oh, we have all the data. We know where all the lanes are. We have such an accurate gps. Yeah. Good luck making maps.
Audience Member
Productizing a good idea is the hardest part.
Audience Member
But I do think that Tesla is particularly good at making awesome product. Unlike Twitter was.
George Hotz
Well, yeah, I think that you can do certain things with organization, but it really is a different product, right? It is a different product. And there are some challenges. I mean, Twitter is not perfectly horizontal scale. Right. Tesla is Pretty much perfect horizontal scale. Not entirely perfect or, you know, I'll say, comma, comma, is pretty much perfect horizontal scale. We have some things where all the cars connect to the server, but all the driving is done locally on the car. So that scales perfectly. You know, like, there's no. There's no complexity to deal with there. If you make it work on one car, it's easy to make it work on all the cars, but if you make it work on one Twitter app, making it work on all the Twitter apps might be.
Audience Member
I think Elon kind of brought up a really good solution, though, which is like, just incentivize people to delete code. Like. Like refactoring and deleting code and simplifying isn't sexy work. And no one really wants to do it, but, I mean, pay them more for deleting code.
George Hotz
That is all I wanted to do with Twitter. That's all I wanted to do. I was there to refactor and make it beautiful. I don't know. But you see the problem with refactoring and making beautiful is you got to do that instead of features.
Audience Member
Yeah, exactly. I guess just pay the people more who are refactoring.
George Hotz
No, it's not a question of paying the people more. It's a question of if you have 400 people dumping shit into a pool and a hundred people shoveling shit out of the pool, the shit still becomes. You know, there's still so much shit in the pool, but at what point
Co-host
do you say, fuck it and just build it?
George Hotz
Well, and you know what I mean, this is one of those remind me in five years kind of things. Remind me in five years. And let's see. I'm so curious to see where Twitter will be and what their infrastructure will look like.
Audience Member
I mean, that's essentially what ended up happening, right? The timeline mixer was deprecated and the new rewrite of the home mixer was implemented.
George Hotz
Did it work? Well, did it work? What do you mean, did it work? There's now both of them.
Audience Member
Yeah, it increased the cost. But did it help the technique depth?
George Hotz
No, you're just saying, you have more shit, there's more shit in the pool. You didn't take any shit out of the pool. I.
Co-host
This is like a slight rap. Back to a topic earlier around, like, efficiency and, like, the AI stuff. I think that the problem we see in a lot of these larger companies, especially the ones that, like, do lots of different features, is how they transition from, like. I noticed a pattern. I think there's A term for this, but the similar to the Moore's Law where every two years processors get like twice as good. Every three years, any given piece of software becomes two to three times easier to write. And if your company's been around for 17 years, a lot of efficiency has been unlocked by us improving the tooling that. George, you and I, as younger guys, we look at these services, we're like, what the fuck? This could be like 100 lines of code instead of 10 services that are thousands of lines of code. A lot of that is a shift in the industry based on all the tools that we use now every day. For me, AI is another one of those where certain languages and certain architectures that right now are inefficient to write because of just raw amount of code. For something like Golang, for example, where you're copy pasting half the code you write, something like AI, specifically like Copilot can make you much more efficient as a developer. I think what we're seeing is one of those 2-3x moments in real time. But I don't think we're seeing developers die. I think we're seeing yet another advantage to the people who are willing to start from scratch and take advantage of the new way to build a pool.
George Hotz
So with respect to Golang, I mean that's a very deliberate choice on the language designers. It's like, you know, well, we have all these people and like, let's be honest, they're not smart enough to code C, so we'll give them Golang and like that's the real, that's the real justification for it, you know, and I consider myself one of those people not smart enough to code C. And that's why Golang is nice. But that verbosity, like, you know, the very explicit painful error handling and Go having to actually check each error code, it's there to encourage people to think right. So you can go and pave over that with chat GPT, but then you just get people who don't think right and then the whole point is kind of ruined. Like Go was very deliberately designed that way. I can't speak to how Java was designed because that was all before my time. But I remember when Go came out, I like watched all this happen.
Audience Member
Sorry if this is a Jason topic. Is Copilot's biggest barrier to adoption just installation, the, the intimidation of installation. And with the price of Copilot, would customers, would they be price elastic enough to like pay an extra 99 bucks to have like an Uber like service nearby?
George Hotz
Copilot I didn't find it great. The same thing with ChatGPT.
Co-host
Like I got multiple top 100 in the world placements this year in Advent of Code Using Copilot once you learn how to work with it as a tool, it's kind of like vim, where you have to learn to train it. Like know how to write, name a variable, leave a comment and trick it to writing the right code. But it's a tool in that way, like you have a relationship with it.
George Hotz
And if you're trying to play Advent of Code where you're trying to do it as fast as possible, sure, Copilot might help you.
Co-host
I also worked at Twitch for half a decade and run a company and wrote half the code for that too. It works really well in all these environments. From my experience.
George Hotz
You really you find Copilot helps you because I don't.
Co-host
Yes, it's when it conflicts with other tools that I find as if not more helpful. Like when it gets in the way of typescript, type definition, autocomplete and stuff like that, it can get annoying. But I do often find it improves my velocity and experience as a developer. When it hits, it hits. And if you know how it's going in this and you'll learn quick when you use it, I'm going to have it write this algorithm. I know it's going to put these two things in the wrong place. I'm going to delete this one and move this one somewhere else. It makes me feel better as a developer the same way I did when I first got really into vim. But it takes a bit to really like feel that.
George Hotz
So do you like Autocomplete in general?
Co-host
Yes, when it's guided and you have a good type safe system that is defining the autocomplete. I'm really big on full stack type safety and autocomplete. Intellisense, like the Microsoft way where it guides you to the right line of code, is something I'm really big on.
George Hotz
I find. I hate autocomplete. I never use any autocomplete for anything. The things that I like with IntelliSense.
Audience Member
Maybe mobile texting.
Audience Member
Maybe mobile texting because it's so annoying to try to get. That's the only time I use autocomplete.
George Hotz
Oh. Oh yeah. No, this is. No, specifically for programming. I like that I can right click on a function. It takes me to the definition. Even if that's only correct 90% of the time or 80% of the time, it's still a lot better than having to grep for it. But I don't know. I find most autocomplete to like.
Co-host
So I run a podcast where I talk about all of this type of stuff. I've had the creator of React JS on. I'm going to have the creator of GraphQL on soon. If you're down George, I'd love to have you on to show you how my stack works and how the T3 stack stuff goes specifically because I think you'll see when you have full stack type safety implemented from your database to your user, autocomplete stops being a thing that annoys you and starts being a thing that guides you to writing the right code the first time. Always. It's so powerful when you have it set up right.
George Hotz
Let's set it up. Let's set up a live stream and I will give this stuff a try. Look, it's the kind of thing also where I'm sure if I. I don't know, I have like. I don't really like tooling. I like absolutely minimal tooling. I used vim for a long time with nothing. No plugins, no nothing. And now I use VS code pretty much the most bare bones you can make it but.
Audience Member
So I'm a little embarrassed. I did not. I was using the. I said copilot earlier. I meant to say open pilot. We were just talking about copilot. I used the wrong word when I
Elon Musk
was trying to ask ox is if
Audience Member
it was with open pilot was the like with the cost of the system in general and if installation is intimidating for it is like a barrier to entry for adoption then like would increment of 99 extra dollars for like an uber like installation service.
George Hotz
Yeah, yeah, you know what? You were on that service. You run that service. I'm bored thinking about it. No, really. So. Oh by the. By the way, by the way we are dropping now that I got. I got 14k people here. We are dropping an incredible video of the year production car comma 3 Look, I don't want to run a business. I don't give a about any of that. I want to solve self driving cars, I want to beat Tesla and you hit a deadline.
Audience Member
Look at this.
Co-host
Where are you posting that video? George?
Co-host
Where are you posting the video?
George Hotz
Oh, right here on Twitter. I mean it's the best.
Co-host
Do you want any help with the YouTube strategy? I'm a. I'm a big fan of helping with YouTube strategy stuff. I help with bunch of like Y combinator companies. It's Something that's like, I'm nerdy about the YouTube algorithm and how to make it tick. So if you want to chat about that. I just had a video go viral about AI recently.
George Hotz
Yeah, it's not, it's not, it's not that. It's not that good. I don't really care. Like, it's more of a. Like a lot of things that comma are. We have a thing where it's like, I don't really care what people think of me. Perception is so meaningless.
George Hotz
Like, let's actually just solve self driving cars and let's build something that's 10x better. And then like, I don't care what you think of me, it's just 10x better. Like, you have to use it. I'm sorry. And we think of a lot of, like, the program and stuff like that. Like open pilot. Open pilot. Their entire open pilot, which can drive a car with zero disengagements to Taco Bell, is smaller than most of Twitter's microservices.
Audience Member
Well, Twitter might be a bad bar
Elon Musk
to compare with, but.
Audience Member
Yeah, I get the point.
Elon Musk
That's incredible.
George Hotz
Yeah. But it comes down to it. It's just the same number of engineers work on both, Right.
George Hotz
It's just the thing that will determine how many lines of code your thing will have is strictly a function of your number of engineers and a function of nothing else. And I think that kind of brings the space full circle. Does everyone kind of pretty much agree with that? You know, give or take.
Audience Member
Why did Twitter need that many engineers?
Co-host
I. I largely agree. I think that it. The way things were built when Twitter started and the way things are built now are different. And a lot of the chaos that we now see as obvious is like, obvious in hindsight.
George Hotz
Very much agree. Yeah. I think that I know that whenever I'm writing something, the first time I write it, it's terrible. The second time I write it, it's like kind of okay. You can sort of see that it's
Audience Member
going to be good.
George Hotz
And then the third time I write it, it's actually good.
Co-host
I have one challenge for this now that I'm like really deep on the product side. The first version's often enough to figure out if you have product market fit. And it's often not worth me paying you as an engineer to build the second and third version until we know that the product has some market fit. If you have a system where you can identify that and then Go build version 3 when like a very small subset of users are on that early V0. Awesome. Do that. But I think the problem is we're trying to find product market fit as cheap as possible. And when we find it, we keep building on that awful V0.
George Hotz
I think nobody has doubt that Twitter has product market fit.
Co-host
I think we can afford if you add new features. We don't know. We don't know if Twitter Blue is going to have product market fit. And I know I can't change my pro profile picture right now because I'm a Twitter Blue user. I had an automation set up to change my profile picture whenever I went live and it didn't error out when I had changed when I became verified through Blue. And now I can't change my picture anymore. My account's in this weird locked state. Just Growing Pains Rush the feature. It's not. No, it's not Growing Pains. It's experimenting in favor of product market fit.
Audience Member
Yeah, I've got the tweet pinned up here on top. Sorry if that's, if that's not okay. But I just thought I would. But like, I think that it's true though. Like, it would be awesome to have more feedback mechanisms into the core teams at Twitter. So like, if, like there's a feature that just makes a lot of fucking sense. Like, like you could just have that feedback directly from the users or something is broken. And then getting those information systems into the core team so that they can really iterate on that, on that product market fit, you know, I mean, obviously Twitter already has it, but, you know,
George Hotz
you can, you can add anybody at comma and you can discuss their features with them. I can't speak to Twitter, but I could say, you know, comma, everyone's pretty much on Twitter and you know, you can, you can, you can at them and ask for features. I don't know, like, I love building in public. I think that so much secrecy is incredibly stupid. I think it would be so cool if Elon actually dropped a 3 gigabyte tarball.
Audience Member
If he did that, that would actually be insane.
George Hotz
You want to sit there and try to read it? Welcome to my last four weeks.
Audience Member
I mean, it was just, it would actually like, like that would be mind blowing, you know, if he just was like, yeah, here, you solved it.
George Hotz
I mean, no, you're like, okay, I hope he actually doesn't do that, because who knows how many secrets are in that code base. I don't mean secrets like, like, like, like, like IP secrets. I mean secrets, like a literal like AWS
George Hotz
Everyone's got those in Their private business. All right, Ashok's got his hand up.
Elon Musk
Yes, hi. Is there any ideas and possibilities on having unique profile pictures to reduce fake profiles or reduce abuse?
George Hotz
Wait, the last thing that you can possibly ask me about is Twitter features. By the way, I'm in the Twitter should build zero features camp and should just do refactors for the next six months. So that's my take on any asks for Twitter features. But look, I'm not even at Twitter anymore. I don't even work there. I resigned.
Audience Member
Were you joking? So you actually worked there?
George Hotz
What were you doing? I tried to fix all this shit.
Co-host
I literally live across the street. How do I go to help fix all the react. Bullshit.
George Hotz
All the react bullshit. Look, I don't work there. I don't, I don't know about any of this.
Audience Member
I mean, this is. George, I got a question about comma, actually.
George Hotz
All right, all right, so let's say
Audience Member
the best case scenario is comma turns into like a 500 billion dollar company, right? How many?
George Hotz
I don't know if that's the best case scenario. That sounds like people want to come take the from me.
Audience Member
What do you mean by that?
George Hotz
I don't give a about the money. The money's fake. Guys, do you believe in the singularity? The money does not matter. The singularity.
Audience Member
Yeah. Okay, conspiracy aside, right, but how, how, how?
Audience Member
The money fake. I, I, I know, I know. All right.
George Hotz
Get the money's back. Yeah. All right, go on.
Audience Member
How, how big would comma grow? Because you can't operate that. Like, you would need at least a team for compliance alone.
George Hotz
Compliance with what?
Audience Member
I don't know, like people getting car crashes.
George Hotz
What if people get in car crashes? What does that have to do with me?
Audience Member
Because you're the CEO and you'd be responsible, but that, that's, that's not
George Hotz
comma system and it's open source.
Audience Member
Well, the goal is to be level four, right? Or full.
George Hotz
The goal is definitely not to be level four. I have no idea. Oh, level two forever. Actually, you know what the long term goal of comma is? The long term goal of comma is to have a shop in the mall where you go in and you buy your new comma body and you bring your comma body home with you and it cleans your house and it cooks you some food and you know, you can even have sex with it if you want. Now that is the goal of comma, right? Compliance. Go to my store at the mall, drop 20 grand, walk home with your comma body hand in hand with helmet shade. You're still so.
Audience Member
I'm surprised Elon came into your space.
Elon Musk
Dude, it's so funny.
George Hotz
What the. Elon came into my space.
George Hotz
you're chill, man.
Audience Member
He's a draw too. I get it.
George Hotz
He was not happy with the asking
Co-host
about ad tech initially. Thank you for defending me. Yeah, I appreciate. I got mad respect for you for that. I did not expect that at all.
George Hotz
Thank you. Thank you. No, look, I think. I think he was pissed at the other guy and I think he thought everyone else was like. That the other guy was literal, like, just. Just annoying him. But no, I mean, look, what you said about ad tech was like, absolutely right. And anyone who works on ads, like, knows us.
Co-host
Which is. Yeah, I was surprised.
Co-host
It sucks because, like, I really want Twitter to survive and ads are how you survive in this economy. And he just doesn't get a lot of it.
George Hotz
Yeah, look, you know what? Whenever I think about. Whenever I think about a decision that Elon's making, I remind myself that he's a billionaire and I'm not. So whatever his process is for making decisions, it's better than mine.
Co-host
So I have a thing I call reset theory, where it's very easy when you go from one place to another in terms of, like, areas of expertise, to think that your current expertise carries over because you're an expert, you're on top of your shit. And I made this mistake when I was on top of my game as an engineer thinking, oh, I can move over to running a company. No, I can't. It's not that easy. It's really hard to do. I thought when I was popular on YouTube, I could just move that over to TikTok. No, I absolutely can't. It's really hard to do. Going from one type of company to another, from a space where customers are paying you money to a space where advertisers are paying you money to have customers watch their stuff is such a huge shift that it's easy to see, like, certain patterns and, like, the fundamental truths, the one dimensional solutions that you're familiar with, which in Elon's case are people like your thing, people buy your thing. Thing will be successful. That's not true in this space. And like, the axes you've defined yourself on aren't even, like, the same here.
George Hotz
I mean, I think it can be true. Like, I think that one of the things is baffling to me. You look at Twitter's old things. How is this company possibly spending $5.5 billion.
Co-host
Yeah, there's like, like there's two extremes here and they're way past the like, too big extreme. But there are and like, I agree, Twitter spending way too much money. They're way too big. They had a lot of debt from both technically and like the architecture of like the company itself, financially. And financially they're, they were fucked on all of these dimensions. But that doesn't mean Elon's knowledge of how to start a business successfully necessarily carries over to how to make a successful advertising platform on a given social media product. And a lot of the things he's used for his successful career are specifically like harmful in this space. Like those truths don't exist here.
Audience Member
But you're talking about somebody who sold flamethrowers out and burnt hairspray. I think like out of anything. He does know how to generate money. And this is just, it's almost like thinking outside of the box where you just have to apply it in different ways and obviously limit your, your spending in some ways. But I think it's just, you know, fine tuning his expertise in other companies here and really learning as he goes. Like, he's not an expert there. He didn't really understand the code base or how chaotic it is. You know, somebody who, like I worked at GM and you have a lot of legacy where they keep stacking shit and they get some new integration tool and everything's just this big ball of different technologies that don't really even connect well, but they just work. And once they work, people just don't want to dive in there and refactor it. So I think it's just more so of, you know, getting the right guys behind it that can connect the world a little bit better than it already is.
George Hotz
There definitely are some truths to running an advertising based business business, but at the same time, this business is not being built from scratch, right? Like Twitter was bought for $44 billion. You only need to make this thing profitable to stay alive, right? There are a huge number of users on the platform that even though you might not be doing ads as efficiently as, as, as meta, it may not matter if all you're trying to do is stay alive. Because the first fundamental cost to run something like Twitter, this isn't a car company. At a car company you have margins and your margins are kind of razor thin, right? Like you got to buy all these parts from somewhere, right? You don't make the wheels. You buy the wheels, you don't make the engine. You buy the engine. Maybe Tesla makes the engine, but GM don't make the engine. Actually, maybe they do make the engine, but they don't make the brakes, they don't make the steering wheel. So there's no real. What's the actual cost to running Twitter.
Audience Member
I also want to do that. Elon's mentioned this before, and I think this is like, huge. Is that he's going to start incentivizing content creators like on Twitter. Yeah. But I mean, I think this is a game changers.
George Hotz
Who's gonna build them?
Audience Member
All the, like, substack people, all of
Elon Musk
the, like, YouTubers, all these people that
Audience Member
can make money, they can do it on Twitter.
Co-host
So right now, Twitter doesn't have a path to profitability. I. We've said a couple things throughout the space, like, Twitter has product market fit. It has product market fit for a product and a market, but we don't know how much money it can make. And there's two paths. We can try and raise the potential for money to be made so that it can either raise more money so it can get more Runway and be bigger, or we can pick. We can say, okay, Twitter is good enough. Today we're going to trim everything from here, from the waist down so that we know we can fit it under the amount of money it can make at this moment. It is hard to do both at the same time. And when we choose to refactor, now we are choosing to do the latter.
Elon Musk
One of the things from a feature
Audience Member
standpoint that you got to realize is like with Twitch, for example, right?
Elon Musk
Twitter Spaces is one of the reasons
Audience Member
why Twitter Spaces is so special is because you're not. There is no video. You are just doing voice. You're able to focus on that conversation.
George Hotz
And when you add Copy Clubhouse, when
Audience Member
you add in video, right, you're creating an arms race. Now it's like with Twitch, right?
Elon Musk
Like, as soon as you have video
Audience Member
now as soon as somebody starts getting like a green screen, it starts doing picture overlays, it starts doing animations and starts doing. Getting a ring light. It levels up the quality bar. And now there's like the barrier to entry starts to get. Get larger over time. It creeps up.
Elon Musk
And this is what happened with Twitch.
Audience Member
And now, like, if you were to start streaming on Twitch and be competitive,
Co-host
if you think that's what happened, like, you've said some stupid shit, but saying that's what happened to Twitch, really, like, icing on the cake.
Audience Member
God, yeah, that's exactly what happened. And it's also why?
George Hotz
Like, I just, I disagree. That's happened with Stretch. I, I stream on Twitch with the absolute lowest budget and I get a lot of viewers.
Co-host
I'm literally streaming on Twitch right now to a thousand viewers and I'm using a $300.
Audience Member
But you can't make a living off
Co-host
I. Yeah, I know you can't. I can. Especially since my YouTube gets 2 million views. I produce on Twitch. I worked at Twitch for half a decade. I know how the shit works, man.
George Hotz
You, yeah, you can, you can be successful those things. Yeah. This is like, I think this is the same people who think you need a fancy keyboard in order to program. I think these things, the high end people get them but I don't think they really have much to do with
Co-host
your success at all.
Audience Member
They don't have to do success, but it absolutely limits the pool of people who can find success. Right. There's always going to be the outliers, there's always going to be the exceptional people who find something creative to do. They do.
Co-host
You can buy a $600 phone from Apple, you can buy a $600 phone from Apple and be a top YouTuber with it right now, no questions asked. Like the quality you can get of like recording and quality both video and audio on a cheap Apple device from the store day one is insane.
George Hotz
I was actually looking into, I was looking into buying like a real high end video camera and then I like was watching some stuff and then I was shooting some stuff with my, my iPhone 13 and I'm like, it's not really that different.
Audience Member
No. But good luck being successful without being
Co-host
able to graphic edit your thumbnails without
Audience Member
being having title cards and these things matter tangibly.
George Hotz
To watch, watch my stream.
Co-host
You have no idea how. Again, discoverability.
Elon Musk
This is so fun for making a living.
George Hotz
It's all about that. Following off topic, do we want to end it with one key thing? The scale complexity, number of engineers. We all agreed that we only need 20 engineers. The trick is finding the right 20 engineers. Everything can be built with 20 engineers except rockets. They might need 50.
Co-host
The framing I want to offer you for this, George. This is the thing I try to talk a lot about. I think about problems in dimensions and it should be our goal as engineers not to make the dimension we're working within as rigid as possible. It should be to have as few dimensions as possible. So things like auto scaling, that's a dimension you have to work to maintain. Things like global distribution, that's a dimension you've added of complexity you can make those rock hard, rock solid and hard, keep them in good places, but you're fucked as soon as you have to make changes. My goal is almost always to reduce those dimensions and make it as simple as possible. And tools like serverless are a great example of how to do that. I think that that's the future we're going to see that's really exciting is a future where we're able to reduce complexity not by picking better languages or writing better code with better engineers, but simply fucking like building things that don't have those problems in the first place.
George Hotz
Yeah, and I think that's the real truth of the ChatGPT revolution. It's not going to be whether you ask your ChatGPT to write Python or Scala. Like, I think we can almost abstract away the programming language. When can the AI just directly do the compute?
Audience Member
I think the issue with ChatGPT is that, you know, there are certain things that it won't be able to know, like the integration or the creativity that comes from the engineers, or like certain times where they asked it to.
George Hotz
Total bullshit. Total bullshit. I promise you that creativity is, is absolutely Nothing. And the AIs will blow out of the water any human creativity.
Audience Member
I won't say creativity, but more so of like asking it to write a REACT application. It's using deprecated libraries. So staying up to speed or depending on how many things you actually want integrated with it, like how, how in depth will those. Will there be like a questionnaire to be able to build everything or kind of like a, I guess a build. I don't know.
George Hotz
As far as staying up to date goes, those things can read the Internet so much faster. I can retrain the model so much faster than a human could read a book.
George Hotz
Well, actually that's not exactly true, but like you could imagine this kind of being true in a limit. The new, the new systems that I think are going to move to right now, they store all the information in the weights, but you could store all the information like kind of like a memory bank and you can do retrieval systems. Facebook just came out with a paper on this. But I don't imagine it being a questionnaire, I imagine it being a conversation.
Audience Member
But I think that there's something to this though, that AI can't really. There are different kinds of creativity. AI can take different ideas and synthesize from different domains way better than any human could. Right. Because it just has such an insane breadth. But that breadth is composed of all stuff that humans already know running on top of like there's some types of creativity that AI I think will never really achieve. It's more a tool. But there are some domains and some things that it will create, obviously.
George Hotz
I promise you guys that this stuff is not true. There is no type of creativity that the humans have that is unique to them. You say the AI is just a synthesis of already existing ideas. That is all humans are.
George Hotz
There is absolutely no difference. And you know what? Everyone who. Everyone who doesn't believe this is going to be in for a sad awakening.
Co-host
I think I'm almost entirely with you. My one piece of pushback is that AI has these levels of creativity. The thing it doesn't have is understanding. And I think that is where humans excel. What their ability to under.
George Hotz
What is understanding?
Co-host
When something's working, why is it working? That's something you can use AI to help it give you additional insight on. But the decision, like there will always be a level of decision making that comes down to a human and we will keep moving that higher and higher up and keep getting like to do cooler things as a result. But we're always moving layers. We're not necessarily removing humans.
George Hotz
This is the. This is a smarter take on it. I still think it's wrong, but I think it's less obvious that it's wrong.
Audience Member
I think that it's. That AI I don't think will create new paradigms of thought. You know, like AI will incrementally improve and synthesize a bunch of different things, but we're not going to get a whole new paradigm of green open field. Does that make sense?
George Hotz
Both of you are wrong. We've already passed this point.
Audience Member
OpenAI wrote that Dota bot a while back and it was inventing new strategies and metas in the game that like pro players, I've never seen. And then they were trying to play against the open AI bot to get better at it. We're like at the point where we're learning from AI. It's. It's like the humans wrong.
George Hotz
Yeah, there's definitely no paradigm type of creativity. Any of this. None of that's true. There is a question of the agency of AI Theo. Maybe that's what you were getting at. You're like, well, so the AI is always going to sit below humans. It's just going to be like, does it sit right below executives? Right. Like as you kind of move up in an org chart.
Co-host
Yes, to an extent. There's a level of like, which product should we Build next. And when different products have different responses from users, which should we double down on? Which engineers or which people don't necessarily make sense to keep. But AI can help guide us to the right decisions for a lot of these things.
George Hotz
But if you. Okay, go ahead.
Co-host
My concern is the cost of certain, like AI making expensive decisions easier to make is different from AI making expensive decisions. And it will be a long time, if ever, where we should let AI regularly make the expensive decisions for us when we could instead use its insight. And if we disagree with the AI's decisions and make different ones and up as a result, there is a person who can hold accountable for that.
Audience Member
And I think that.
George Hotz
Did you see that on Twitter? The IBM memo from 1970. Computers can never make decisions because they can't be held accountable.
Co-host
Yes, and I do think that's a real problem outside of the dev world.
Elon Musk
I think one of the things AI
Audience Member
will never have is experience. Learned experience.
Audience Member
I can read all I want about childbirth. I'm never going to fully understand that experience. And AI will lack that for basically every human experience.
Co-host
And I don't after he said that. That way I understand why you were so against my take.
George Hotz
Okay, we're, we're the, the, the, the people with takes on AI that really make no sense. I've just remember speakers. I mean like I look, I, I can. The problem with these things is you get at the bottom of what these people are generally saying and they eventually get to something like AI can't experience qualia and you can. The AI will never know the touch of human skin and this is just completely meaningless. There is no such thing as consciousness. There's no such thing as qualia. Now to talk about the difference between tools and agents, that is a real thing. That is a real thing that has a real science. The loss functions look different, right? So agents usually come when the loop is closed. And to come to your exact example about products, you're like, well, the AI is not going to choose the product, right? The AI is going to help the humans. The humans are going to look at how the product's performing. The humans are going to tweak the product, put the new one out, maybe with the help of the AI. Yeah, but why not just close the loop entirely? And if your worry is because you're afraid, well, the computers can't be held accountable when it's terrible. I mean, maybe today where humans and AIs are on kind of similar ish playing fields, humans are actually still better. Sure. But someone's eventually not going to have that taboo and that company is going to crush you.
Audience Member
Hey, George, I have a question about modern security. What do you think about the new iPhones and the new consoles? And they're much more difficult to hack compared to back in 2017 when there was the iPhone3GS, unlike the Xbox.
George Hotz
That was a long time before 2017.
George Hotz
So when you say it gotten harder to hack, you can quantify this. You can talk about how many dollars an exploit costs.
George Hotz
And the price of exploits has gone up a lot. Yeah, yeah.
Audience Member
And you have to be a near expert. Like, you can't be a teenager doing this, right?
George Hotz
Yeah. The best people are probably still teenagers or close to teenagers. They always are. It's just one of those things that selects for. I promise you, the best exploiter in the world today is probably like 24. You know, it's just. I don't know who they are. I haven't kept up with the scene, but I bet you that's true. Because it's just like the way your mind works. But, yeah, the cost has gone up a lot. But I don't know, that actually might not be because it's harder. It's harder to say. It's actually more difficult. Like, I'm not sure I could really do it anymore. But it's like, you know, then I watched the generation before me, I watched, like, Charlie Miller be like, oh, there's ASLR now. It's hard. And, you know, ASLR was never a thing for me. And now you have these, like, new kids who are like, yeah, of course. It takes nine exploits and we have to break out of three sandboxes. No big deal, bro. Yeah, yeah. No, I was just curious.
Audience Member
The other thing too is I. I was just reading that the, like, Gen Z is less technical, technically advanced than, like, millennials, if that makes sense.
George Hotz
The youth. The youth, man. The world is falling. The youth are.
Audience Member
What do you think?
George Hotz
Socrates said that. You think so less technically, the Greeks were complaining about how the youth were corrupt and couldn't do X and they didn't know the virtue. Whatever. Whatever this is. This is a meaningless thing.
Audience Member
Hey, George.
George Hotz
All right, cool.
Elon Musk
Hey, George, I just have a question that you have any unfinished dream ideas for Twitter or weren't they listened or anything that you still feel that is pending with Twitter?
George Hotz
No, look, I don't work at Twitter anymore. I don't know. I think one of the main things it comes down to is Elon wants to build features. And I would do refactors But I did like that Elon said that. Even if it's not me, I do hope the Twitter employees get some good, quality food. The food matters a lot to me. It's important to me. Come on, Jordan, you might need to
Audience Member
change your profile picture to I do not work at Twitter anymore. Don't ask me Twitter questions.
George Hotz
Yeah, we're gonna get a few people who ask me Twitter questions. I'll start coming up with good, snippy responses. Maybe I'll start spreading misinformation. I spreading misinformation. All right, cool. Thank you, everyone, for being in my space. Check out Comma's Twitter tomorrow. We got a fire video dropping. It's going to be good.
Co-host
DM me so we can schedule that podcast.
George Hotz
All right, sounds good. I'll follow you.