Welcome to our new podcast, where we go deep on a wide variety of technical topics with the smartest leaders in the world. This is hosted by Immad Akhund, cofounder and CEO of Mercury and Raj Suri, cofounder of Presto and Lyft.
You can listen+subscribe on YouTube, Spotify and Apple Podcast.
Transcription of our conversation with Amjad Masad, CEO of Replit
Amjad Masad (00:00:00):
Pain and pleasure are like core features of consciousness and they seem important for humans operating in the world. Can you actually construct that in a machine?
Immad Akhund (00:00:27):
Hi everyone. Welcome to the first ever podcast recording of curiosity podcast where we go deep with an expert in their field. The tagline is Delivering 10,000 hours of learning in one hour. So that's a <laugh> ambitious tag line. I'm Immad, I'm the co-founder and CEO of Mercury. I've been doing kindness startups and investing in startups since 2006.
Raj Suri (00:00:49):
I'm Raj Suri. I'm the founder and CEO of Presto Automation, which delivers AI type applications for for traditional industries like restaurants. Also co-founded Lyft and yeah, very excited to be cotting this with you. Mad. This is an opportunity to go really deep into some really interesting areas with some of the smartest people and you know, most thoughtful people on the planet. So excited to be able to explore, you know, in depth.
Immad Akhund (00:01:14):
Yeah. And today we have Amjad Masad with us. He's the co-founder CEO and I believe now head of engineering at Relet. What's the one line of relet? Mjad.
Amjad Masad (00:01:27):
It's the fastest way to make software. We have a platform that provides an online programming environment that's collaborative and we have large community of developers making things for each other, for other people. And we're getting into supporting teams of developers in the same way that say, you know, Figma is a collaborative design program replica is kind of that for programming.
Immad Akhund (00:01:54):
What's kind of interesting is it started relatively small, right? This was like an ID online and I think you were like compiling whatever programming to JavaScript and running it on the browser, right? But now it's this kinda hosted package combined with the teams combined with like learning and template. Like was that always your vision that you're gonna like progressively get to this level of like product or was it kinda incremental?
Amjad Masad (00:02:21):
I actually sort of saw a lot of it in my mind's eye pretty early on and a lot of it is like pretty obvious stuff. I'm actually surprised that no one had built it because I started working on it in college in like 2009, something like that. And then when I came to the US based on an Oprah source project that's related, I worked at Code Academy as a founding engineer and used this sort of the same technology that I built to like make browser coding possible and then left that and went to work at Facebook, worked on React and React Native was founding engineer on React native. React native is like the best way to make cross-platform mobile apps. And in 2016 revisited the idea and had found that basically nobody had built it, which was really surprising. It is the opposite or efficient market hypothesis, like why is in the market not producing this?
(00:03:20):
And I actually did not wanna start a startup, but I was so compelled by this thing and I knew that this idea had to exist. Well it turns out I think the answer that why nobody had built it is because it's incredibly hard. But yeah, in terms of execution it was started very simple. But in terms of vision it was always, you know, I was an early GitHub user, like perhaps in their beta. I think I was so excited by GitHub. I was also so disappointed by how little they evolved beyond the initial sort of kernel of an idea. So I always thought about making collaborative developer community that's like more exciting that you can ma do more things in it. And yeah, you know, I've always imagined a lot of the features that we're building. Of course with time just what became available in terms of technology, we started adding a lot of these things that I didn't really think about at the start. But overall, like even like now we have this tipping mechanism where you can tip developers. That was like, I've thought about that like fairly early on. It was actually kind of frustrating cuz like as a founder developer, you're sort of like, everything feels like it's like one weekend of hacking away and it's like turns out, no, it's actually more than a decade <laugh> of hacking <laugh>.
Raj Suri (00:04:36):
Mj it's a really interesting description. You talked about the fastest way to make software, right? Is the primary user base people who don't normally make software or are not really comfortable setting up their own environment cuz that takes time Or is it more the power users who are probably already, you know, they already have a CS degree or something like that, they have a lot of knowledge. What's the primary user base for this type of application?
Amjad Masad (00:05:00):
One of the exciting things about running Relet is the user base is so diverse. It also makes it really hard to run the company because people like crave these simple personas. I'm sure you guys at, at your companies like the design team or the product team want to talk about one or two personas. But I always push back on sort of persona building because ultimately I tell them that we have one persona and that persona is the developer, the software creator. Because like you start segmenting people into students or professionals or hobbyists or, and all these segmentations are true to some extent ultimately, like if you're familiar with Clay Christensen's jobs to be done, I think that inversed the question. It's not like a customer segmentation thing, which is totally arbitrary kind of based on user characteristics. It might be actually interesting to go into jobs to be done a little bit.
(00:05:58):
But basically the idea is that most companies, the way they think about bringing a product to market as they think about a customer persona and that really confuses people about what they should be building and what the actual need that they're doing it for. And so Clay Christensen switched that question, like his primary insight is that people hire products in the same way that you hire people. When you go and you want some accounting done, you don't go look for like a middle-aged man in New York. You know, you don't do that, right? You go and you say, I want some accounting done. And you find someone who's competent at accounting in the same way that you want to get software done or you want to make software, you go to a place that makes it really easy to make software. And from that lens, I just think about our community as as sort of really the main jobs to be done here is that you wanna make something.
(00:07:03):
And it turns out students wanna make something. They wanna learn to make something hobbyists wanna make something, they wanna crunch some data, they wanna make a fun app, they want to make a game. Professional developers wanna make software to bring it to their customers. And so that's the sort of shared goal of our community in terms of like the makeup of our community. It's cues younger and there's partly cultural, partly some product limitations. The cultural aspect is that developers tend to be very conservative people. They don't want to change their tools and they're actually quite haughty and quite, you know, I love them. My best friends are developers, I'm a developer, but we're actually like difficult people. And you know, you see it in Stack Overflow comments, it can be a little, people kind of look down on things and people kind of tend to not change too much with time. They stick to one language or one technology. And
Immad Akhund (00:08:05):
I was just thinking why someone didn't like make this innovation between like when you started in 2016. And I think it's partly because of this, right? Like developers are like, they're often build like little incremental tools for themselves to make their life a little easier, but they don't think about like transforming their trade. Like you don't go like, oh I want to like do it in this new way. Completely.
Amjad Masad (00:08:26):
I think that's right. Which is surprising because developers are the agent of change for the larger economy. Or maybe it's not developers that are the agent of change. It's crazy founders that recruit developers to, to make changes <laugh> it's a tough market because the habits really change really slow. And like there's common saying that, you know, programming changes one generation at a time and that is sort of true. But today, rep is is has some product limitations that we're working through mostly on the technology side, like making an immediately accessible institute, producible cloud development environment. Turns out it's like a very tough like distributor systems problem, like one Yep. Problem that we've like dealt with that's sort of introduced like a big product limitation is that if you're introducing this multiplayer programming environment, there's a a potential for what we call race conditions, which is like people like trying to edit the project at the same time and introduces data corruption and all of that.
(00:09:33):
So we use the file system, basically the system that manages your files, that was what we call atomics. So basically every time you do a right, you create a snapshot of your entire system and that is really good for distributed systems to protect against data races and all of that. It also turns out it just introduced like a really hard limitations on scale. Like we can't have like a one terabyte desk per project. So some of the trade offs we made early on created limitations to how much power the platform has. But all of these are solvable. It just requires a ton of innovations and so we're, we're working through those right now. Yeah,
Raj Suri (00:10:15):
I mean that's really interesting. So you talk about again, the fastest way to build software is on rep. What do you envision will be the fastest? Give us some examples of like, you know, how fast someone could build something, how much faster they could build something in the future versus now. And I know you know you're thinking about AI or probably already working on it, you know, in terms of now we can support coding. It'd be really interesting to hear you as the expert kind of what the future looks like in terms of how fast someone can build something.
Amjad Masad (00:10:41):
The fastest story I've heard on Relet is someone coming into Relet with an idea for a product and getting it 30 minutes later. And the way they got it is without writing a single line of code is by relying on a human machine. Centar of sorts, <laugh> basically we have this bounties program where you can like post, you can pay some money Wow. And post like a description of what you want to build. Someone posted like a Figma. I actually tweeted about it. I was like hey, I want to build this. And literally someone got them a prototype in like 30 minutes and that person who's getting them a prototype is not just a person. They're also powered by ai cuz we have a Ghost Rider product similar to GitHub co-pilot. It's actually a little more advanced than GitHub co-pilot because not only has the auto complete thing, you can also chat, you can talk with AI that's writing code with you.
(00:11:37):
And so it's like, feels more like a chat G P T that understands your code and that sits right there with your editor. And so we're really trying to be un ideological about how people make software and the idea is like really trying to reduce the distance between an idea and a, and a piece of product. Like that's been the trajectory of sort of a human history where if you're sort of hunter gatherer and you have an idea for a product that's sort of impossible to do it right. Like you, let's say you wanna build a spear, that in itself is like really, really hard. You have an idea for a hammer for example. It's like impossible to find the rock the right size and do all the right. These things go into, you know, agrarian societies and it's still pretty difficult to make things probably a little easier. Industrial society got way easier, you now had factories, you know, had accessible labor. Capitalism becomes a thing where you can organize people in groups to create things, but it's still fairly difficult to get a product on the market. And I think the information age is reduction of this idea of like having an idea in your head and getting a product in the hands of people. And I think we can get it down to like on the order minutes, <laugh> like to get something on the market.
Immad Akhund (00:12:56):
Like right now you said like there was a human that was like potentially using an AI to like generate this thing, this kind of prototype. Is there eventual state that there isn't a human involved and like the AI is just generating something good enough? Or do you think there'll always be a human in the loop with like our current AI at least?
Amjad Masad (00:13:12):
So the way I think about automation is that it sort of like takes the, takes each of the jobs from the bottom up. That's like not always true, but for the most part true. I believe it's gonna be true for software, meaning the low skill software creation I think will get fully automated. So I think pretty soon probably this year there's gonna be products and maybe replica builds that, but where you can go put in a paragraph description of a piece of software and get an initial thing with the code and with the app running right, I I think this will happen like very, very soon. I think AI is gonna struggle to iterate it on it and I think you still need a person to kind of make it into a product that you can sell or you can maintain or you can scale. But I think the initial prototype or the MVP will be done by AI's like pretty soon, like on the order months, perhaps a year or so.
Raj Suri (00:14:11):
Yeah, it seems like it'd be useful for like prototyping, like you can come up with an idea like you know, what does that look like? How does it work? You know, can you get internal buy-in for that? Or even customer buy-in for that for an idea, but to actually build complex software that works, that shows up to work, you know, 99.99% of the time, you know, it seems like we're still far away from that. Would you agree? Like that part will still take a lot of human effort to get there?
Amjad Masad (00:14:33):
Yeah, I mean this gets into the reliability of large language models. So maybe we introduce large language models for the audience, but basically like it's the latest technique in AI where it started in 2018 at Google with this technology called the transformer. The transformer is a typhon machine learning model. The innovation that it has is the attention layer. So a, a neural network simulating attention in the same way that humans have attention. So now the machine learning model could actually pay attention to parts of the input data stream. So for example, if you have a piece of text or paragraph, the machine learning model could actually better understand it because of this attention layer turns out another innovation just primarily from open ai, that you take these transformer models, you throw terabytes of data on them and they start having these emergence phenomena. And by the way, nobody understands why, but there's what's called face transition. Once they cross certain number of parameters, like a billion, 2 billion parameters, parameters meaning like how much, it's almost like neurons in the brain, how many neurons are in the neural network. They start having exponential rise in reasoning ability. So you throw benchmark at them doing math, doing reasoning, doing translation, doing whatever, and they start doing better, better jobs and it literally the graph goes on. So like that barely like increasing to vertical.
Immad Akhund (00:16:05):
I've never heard this like explanation of transformers in terms of attention. Can you explain that a little bit more? Like what is it about the model that's giving attention and just like go one layer deeper on that?
Amjad Masad (00:16:16):
Yeah, so previously when we did language modeling it was like very explicit sort of task. We would do classical natural language processing, we would like construct a grammar and we would do all these kind of things. Turns out the machine learning model can find the structure of language by interpreting different parts of the text and by literally directing the attention to different parts of the taxi, relating it to other parts. It can like understand the immersions of structure in language or any really any sequence. And we can get to that maybe in a second, but let's focus on language because think about how humans pay attention to things like attention is about cutting. It's more than just about like pointing to something. It's about cutting the noise. So when I'm paying attention to the screen right now I am like deciding not to look at a bunch of other things. Mm-Hmm <affirmative> machine learning models pre attention again, like I think a lot of researchers are gonna cringe really hard about like how I was describing this, but you know, just for the layman, basically it's now machine learning models are now able to discriminate within the inputs and the training data and try to understand like pay attention to like, you know, the fox jumped over the, you know the fox jumped over the gate.
Immad Akhund (00:17:49):
Yeah. The quick brown fox jumped over. Yeah,
Amjad Masad (00:17:51):
Yeah you can, you can just like look at the fox and then look at other parts of the text and start understanding the immersion structure.
Immad Akhund (00:17:58):
So with a transformer, when an L L M is like trained, you don't tell it like these are all the nouns in the English language, these are all the verbs, things like that. You just give it a ton of text and it like reasons about like the usage and like comes to its own kind of understanding.
Amjad Masad (00:18:13):
Yes. So that's the primary innovation here. Like machine learning essentially is about discovering algorithms. So Andrew Kapai, head of CEO at Tesla recently left to went back to open ai, he called it Software 2.0 and the reason he called it software 2.0 is basically you go from programmers writing algorithms to learned algorithms, meaning the machine learning, you give it an objective and it learns the algorithms. It's highly inefficient but it's also a lot better than an army of humans trying to reason their way through programming.
Immad Akhund (00:18:52):
So it seems like a similar kind of innovation to like alpha go, right? Where like instead of trying to teach it how other players have played go and things like that, you just kind of have the machine kind of play go against itself over and over and it can like reason like strategies from that.
Amjad Masad (00:19:08):
Yeah, alpha AlphaGo is super interesting. The generalization of AlphaGo making it learn from self play. You don't even have to give it the rules of the system. Hmm. The advancement in machine learning is removing explicit design and explicit programming from these systems and instead having the machine learning model discover them because if it discovers them it will be much better than us humans programming it. And transformer takes that to the next level where now we're able to discover algorithms that understand the structure of language and by the way we keep seeing language. But the interesting thing about transformers that they're not language specific. So anything you can model as a language transformers would do really well at. So for example, Tesla uses transformers for understanding traffic patterns. So they model the traffic like a language. Wow. And then the transformer starts to understand the structure of traffic better than explicit programming because it modeled it as a language.
Immad Akhund (00:20:18):
Is that a transformer L L M that was trained on like kind of Reddit and then you're applying it to <laugh> like TE self-driving data or is it like trained on just like the self-driving data that Tesla has?
Amjad Masad (00:20:30):
That's actually quite an interesting point. So G P T stands for generative pre-trained transformer. The pre-trained part is an interesting part of transformers. That's why these models are called foundational models because pre-training them, meaning throwing large amounts of on them is actually pretty good for them. It's inefficient but it's pretty good. So like one discovery that OpenAI made is that if you train large language models on partially and code, they get better at normal tasks and just language. Mm-Hmm The more data, the more diverse data you give them, generally they get better. There's a lot of tuning and fiddling that needs to happen, but generally they get better at these things. So whether Tesla used a pre-trained model that was trained under or whatever, I don't know. But generally how people use these models is they take something that was pre-trained by someone else, an open source, a hugging face has a bunch of printer models and then they fine tune it on an application specific thing because the model has some world understanding based on some pre-training data and then you are driving it to understand your application or your domain specific thing.
Immad Akhund (00:21:44):
You said open AI discovered that like if you keep adding parameters it goes kind of like exponential, like the kinda how good the algorithm is. Is the like have they proved that it's an S curve? Like does it kind of go exponential and then flattens out or is it like so far the more you throw at it, the better it gets
Amjad Masad (00:22:04):
From the outside? Like no one knows whether we ran out of scale scaling returns on scale, whether we hit diminishing returns Intuitively, everything has diminishing returns. Like if things don't hit diminishing returns, who you get into a weird because exponentials becomes just insane, right?
Immad Akhund (00:22:22):
Mm-Hmm <affirmative> maybe. Is that true though? Like as an, I feel like you know, if you look at the grass to like the GDP of the world, it seems to be on like a 300 year like exponential. So yeah, maybe there are some things that don't hit diminishing return.
Amjad Masad (00:22:37):
If you zoom in on the gdp, is it actually flattening or is it it feels like it's flattening.
Immad Akhund (00:22:42):
It depends what time period you take <laugh> it. No,
Raj Suri (00:22:45):
I I don't think it's flattening because you still have all these developing countries becoming developed and there's a ton of growth there. Yeah. Just over the last 30 years, the growth of China would've been a huge increase in GDP and then you have India coming up. So anyways, that's a separate topic. That's an interesting example. I, I agree that at some point you will see diminishing returns even in the GDP growth. That makes sense.
Immad Akhund (00:23:04):
Yeah. I guess it's like over what time period Like we might be in a thousand year run of AI <laugh>. Yeah, yeah
Amjad Masad (00:23:11):
That's true. Also it's s-curves like you know, maybe industrialism is diminishing return but AI is like another S-curve, right? So but broadly speaking, like generally in technologies there's some S-curve and there are signs that were starting to see diminishing return and those are indirect signs. And this is my opinion, the recent innovations in large language models have not been in scaling. They have been algorithmic innovations. The two primarily innovations is supervised fine tuning and reinforcement learning from human feedback. These are two algorithmic innovations that made chat chip p t what it is. So it wasn't pure scale that got chat chip PT to be this powerful thing, it was an algorithmic innovation. So the question is if they thought that scale was still the best way to improve these models, then they would not have invested in these algorithms.
Raj Suri (00:24:12):
Yeah, we'd love to learn more about those innovations. So you're saying that it's not just more training data that they threw at this model, they actually did a better job of training it with the existing data that it had. You said through supervised learning and through human fine tuning, is that right?
Amjad Masad (00:24:26):
Reinforcement learning from fine tuning. So supervised fine tuning. The cool thing about large language models, G P T in particular is that it is trained in a self supervised fashion. Meaning that we don't have to label the data. Yeah you can take Reddit, you can just like throw it at a language model and it understands the structure of English, pretty crappy English, but it doesn't stand <laugh> and <laugh>. And then turns out if you want the model to perform better, especially on a downstream application like say chat, then you go and you get a lot of data and you label it and the way you label it is you just say, you know, the most simple case of labeling is hotdog, not hotdog, right? Like from the show Silicon Valley, if you wanna build a machine learning model that detects hotdog, you just take a bunch of pictures of hotdog and bunch of pictures of not hotdogs.
(00:25:24):
You label those, you label those not hotdogs. This is supervised learning. So it turns out if you give it supervised learning, ITAR perform better. Now it still has a problem with like not really listening to people. This is the reliability of issue of large language models, which is I think I was trying to get to to answer you six questions ago or something like that where I think the question was something like would it get to complete software? The reason I don't think it's gonna get to complete software pretty soon is because of the reliability issue of large action models, large action models, the way they're trained, the way they work means that we actually don't control their output. They work in a human-like fashion. I dunno if you have kids Raj. I know Imad does. I have kids. I do. Yeah. And trying to program your kids or trying to like make them do a certain thing is like incredibly difficult. It's actually same thing with large language models. It's almost like trying to teach kids something. Like you have to keep talking to 'em to find the right set of words to convince them to do what you want.
Immad Akhund (00:26:33):
What's kind of funny is like, you know, when you're talking about this, I actually think a lot about my kids and like originally, you know, when they're really little kids, you can't teach them anything. It's all self supervised learning <laugh>. Yeah, like as a, you know, it's like a one year old, they're just figuring stuff out and it doesn't matter what you're really saying, but now I have an 11 year old and it's a little bit more supervised where I can actually like talk to her and explain things to her and she seems to get it. So there is this like weird human analogy to how kids learn things as well.
Amjad Masad (00:27:02):
Yeah, I mean neural networks ultimately were modeled after brains.
Raj Suri (00:27:05):
Yeah. Yeah. I mean I guess that goes back to the whole AGI question of like, you know, when is, you know, when are AI is gonna get actual like, you know, human intelligence, it sounds like they're maybe in the toddler phase or something right now, you know, and, and they're growing up slowly.
Immad Akhund (00:27:19):
You said LMS are not that reliable, but human program is also not a hundred percent reliable, right? So I guess you just have to get to a level where you can at least beat a human.
Amjad Masad (00:27:27):
Well the difference is imad is one is a stochastic process and the other is non stochastic, right? So in a like discreet programming environment like Python, there are ways to verify that the program works. There are like things called program verifiers, right? You can also do things like unit tests because the, there's like no randomness for the most part. You can actually test the liability of the systems with large range models. There's inherent to toxicity inside the system, meaning some randomness that makes it almost impossible to apply traditional engineering methodologies on it. The way we're actually starting to test these things is by using another model. Have you guys heard of constitutional ai? No.
(00:28:19):
No. This is a fascinating thing. So philanthropic is another company that's a spinoff of open ai. I guess some of the earlier open AI people started their own company and their approach to making large rank models more reliable is you the human, you wanted to do a certain thing and you write a constitution, you literally write like a political document almost like mm-hmm <affirmative>, here's the thing you should do, here's the things that you should value. And then you start running a model and another language model is interrogating it and it's coring it according to its constitution. Mm-Hmm hmm <affirmative>. And then you take that data and then you fine tune it and you basically tell it you were good here, you're bad here, you were good here, you're bad here and you are on the iterations as many times as you can. And eventually it learns to be consistent with its constitution.
Immad Akhund (00:29:18):
And this constitution's literally like human understandable like text or is this like some programming speak?
Amjad Masad (00:29:25):
Yes. It's you writing it, it's like
Raj Suri (00:29:28):
A prompt right in G P T, right? You put a prompt in, it's kind of like a constitution.
Amjad Masad (00:29:32):
Yeah. The difference between prompt and constitution is that the constitution actually ends up affecting the weights and biases. The parameters, the prompt is mostly in inputs. It's not changing the weights.
Immad Akhund (00:29:48):
So you talked about supervised learning, I guess this like constitutionally supervised learning is something else, but what is reinforcement learning?
Amjad Masad (00:29:57):
So reinforcement learning from human feedback, reinforcement learning is actually one of the earlier machine learning methodologies. It actually predates deep learning. Reinforcement learning is like, you know, when we're playing games in the nineties a lot of AI there were like trained via some kind of reinforcement learning. And the way it works is very simple is you have policies for rewarding and for punishing the ai. And AI has the objective and you can program this season like classical programming and AI has the objective to maximize reward. So it's just the main utility function just maximize reward and if it gets punished then it learns to like not do this thing and then do the thing that maximizes the reward. And then when deep learning came along, there's now deep r l which makes a big part of this learned like you give the policies and it learns the algorithms to maximize the reward and minimize the punishment.
(00:31:04):
And L from human feedback is basically someone using chat chip t you know, the chat chipi has thumbs up, thumbs down. Basically you're saying this is good app with this is bad app with this good app. But that's like the simplest form of human feedback. But a human feedback can also be writing the response like this is a bad response, here's how I would write it. And so you have a group of people doing that and then you take all their data and responses and you generate a policy from it, a reward policy, and then you train the large language model using reinforcement learning using that reward policy and then it starts behaving a little more like what a human prefers it to behave.
Raj Suri (00:31:50):
Interesting. So the first element is like the tagging of the data and the second element is like the feedback that you're giving the models or the key Yes. Innovations. Yeah.
Immad Akhund (00:31:57):
This reinforcement thing is one of the biggest kind of missing links between like how humans learn, which is like continuous reinforcement, right? And it's not like as simple as like a thumbs up and thumbs down, but it's like okay, you know if I, if I do x, Y happens. Whereas like that bit does seem like it's missing. So it's interesting that I guess like the question is like how often you could run the reinforcement like whereas humans are running it all the time, whereas I think right now you still have to retrain the other limb with the reinforcement and then how complex the reinforcement is, right? Like rather than just a thumbs up and thumbs down,
Raj Suri (00:32:32):
You need specific feedback, right? You can't just have thumbs up and thumbs down is so generic, what was good, what was bad, right?
Immad Akhund (00:32:37):
Yeah. Yeah. That's like feedback <laugh>,
Amjad Masad (00:32:40):
This is one of the main limitations for AI and one that it has to overcome before we have any kind of general ai. But what you're describing is sometimes called online learning. So basically can the model learn whilst deployed in production? There are some rumors that TikTok actually does this, that it actually trains a model as you're using TikTok, which kind of sort of makes sense although it would be very expensive there like mm-hmm <affirmative> unit cost is gonna be a lot higher but it's a lot powerful to do learning instead of just like producing inputs to an existing model and then do the learning step like at night or every X weeks based on new data on instead. Continuous learning is the key to building general intelligence.
Immad Akhund (00:33:28):
One of my worries about AI, at least this kind of wave of innovation on AI is that you know, everyone's really excited about like delivery companies, right? Like everyone wants like click a button, get something in 15 minutes. Rogers are familiar with delivery slash on demand companies and like it turned out like it's just not economical to do a lot of it. Some of it did turn out to be economical but like even though it feels like the future, you know, it wasn't economical to do it. I do wonder whether unit costs of like this AI are just potentially just 10 x more expensive than the value we get out of them, right? There were some rumors around like how much chat G P T costs open ai but do you think that is the case that it is just too expensive? I mean you guys run AI models like is does it cost a ton in like server cost to like run these things and or do you think like that's not really an issue?
Amjad Masad (00:34:19):
So rapid We did some interesting things around optimizations and ghostwriter. The AI system that we have currently have 90 plus percent margin. So our margins are very good on on ai. That being said, it's a very domain specific model whereas CHATT is meant to be a general model. So chatt probably north of half a trillion parameters or something like that. Whereas Repless model is literally 3 billion parameters. So you know, and we're training a 6 billion parameter in one but you know, sub 10 billion parameters, it's like really, really cheap. That being said, like there's a lot of waste in these models right now. So now every time you give it a input it has to evaluate the input through all the neural network paths. So basically all the parameters get activated on any given input. So a neural network network that's been trained on the entirety of the internet has knowledge there Insight and about Michael Jordan's, you know, first game about the mm-hmm <affirmative> president of Zimbabwe and all those neurons get activated when you give it an input about like whatever you, you tell it like hi, it will like run inference that activates all the paths and that's hugely costly, hugely inefficient.
(00:35:54):
And so there's a ton of research around how do you reduce that cost and make it more efficient and have it like only like take the right path in the inference path. So that's like one area of research. The other thing is like the on the hardware level, the H 100 s are actually H 100 s are the upcoming and video trips and they actually have a built-in optimization for transformers. And I think the rumor is like they're like 10 x more efficient than the A 100 s and so there's gonna be innovation on the hardware layer, there's gonna be innovation on the software layer. People like us are gonna be running their own models that are smaller. I think it's solely reasonable for Google not to introduce this in Google search. I think people are giving them a hard time for that cuz they, like you said, the Euro economics right now don't work out but at some point they'll work out presumably Google is working on it. I think the difference between that and the unit economics of Uber four x or Lyft four x is the optimization potential of the physical world is like way harder. Hmm. And like the, just like you're bound by nature whereas in computers like software is a purely virtual thing and you can do a lot to optimize it and also there's like so much progress in chips and there's so much money going into that that I think we're gonna see a lot more efficiencies.
Raj Suri (00:37:27):
Yeah, it's really interesting. I mean one, one thing that occurs to me that will be much easier in the future would be cloning. You can clone any app or any website or basically reverse engineer it, right? Like so you could say, you know, to an ai, can you make me a replica of Amazon? You know, it won't be easy but presumably if you have that feedback and you have like if you know what Amazon looks like, you can build a clone of that relatively easily. I mean and then you can compete obviously with other companies that are in in that space. So like consumer facing stuff where you're only barrier to entry is a user interface that that seems to, you know that competition's gonna go away pretty soon. I mean it wasn't a huge barrier to entry in the first place. Amazon's you know, barrier to entry is obviously it's warehouses, distribution, all of its relationships, but just getting a website up and running is gonna be obviously even much easier in the future. Would you agree with that?
Amjad Masad (00:38:16):
Yeah. Sam Altman had a tweet where he said getting an iPhone app done today is $30,000. Getting a plumber is like $300. I wonder how these prices will change over time And it's sort of like the hint here like wink wink, those things might like diverge in a way where actually like trades, people's salaries will go up <laugh> in a way and like the software pure cognitive work will like go down as AI like continues to eat at it from the bottom up. I would agree with you. I think like I said like getting a basic MVP of an app probably going to zero like not immediately but just the trend will be to zero.
Raj Suri (00:39:02):
What impact do you think it would have to like software developers and will there be like less software developer needed and you only have the best software developers who understand the AI models as well as the software Or do you think this will democratize software, you know, so that everyone can become a software developer? You know it can go both ways. I kind of feel from here you can make good arguments for both.
Amjad Masad (00:39:24):
It's more the latter I believe and I think I think it's gonna be bi modally distributed, right? So on the left hand side I think the platform engineers are gonna be a lot more valuable and on the right side of the distribution is the product engineers that's gonna be just be a lot more of them people making things or product developers or product creators, right? I think the middle end I think will probably disappear. And what I mean by the middle end is your average like full stack developer or like PHP developer or no GS or Rails developer, the sort of purely glue type of programmer like AI's will be like really great at that. I don't think AI will like eat into the product developer because that has to do as much with understanding customers and are starting markets and that's more of an AGI problem.
(00:40:20):
Yep. So those jobs I think are safe on the platform engineer. The systems engineer, a lot of it is like super novel code. A lot of it is like incremental sort of creative work to optimize code really close to the metal or to build like cloud systems. I think those developers are gonna get a lot more productive cuz copo like innovations but I don't think they're gonna go away and perhaps they're gonna get more valuable and I think if you're more of the middle end type developer, you and I either specialize and become more of a product developer or you wanna go deeper in the backend and do more low level system level programming. The product developer could actually be anywhere from like a solo entrepreneur, a hobbyist enterprise start a founder. I think that will really get democratized. Like I think a lot more people will be able to participate in that in the same way that a lot of people are doing design on canvas and like design getting it democratized as has been obvious for a long time. So we're gonna see that on the software creation level.
Immad Akhund (00:41:36):
How much more productive do you think today an engineer can be if they're like maximally utilizing like at a co-pilot or ghost rider or something like that.
Amjad Masad (00:41:46):
So there was a study sort of conservatively estimating 20% productivity boost, which is huge by the way. It's really, yeah really big when you're Citibank and you have 50,000 employees, 20% productivity is like what billions of dollars worth of productivity. The more anecdotal reports that we're seeing at Lin and other places. And GitHub I think is saying that is like people will say will self-report that they're anywhere between 30 or 80% more productive. We've heard 80% more productive, like a task is cut in half or more because of ai. I just think we need more data to judge but I think it's on the order of magnitude of 1.2 to two x more productive but I think it was just early. I think there's like a 10 x over the next couple of years.
Raj Suri (00:42:43):
Hmm. So you think this is gonna go pretty fast? I mean on the programming assistance side, two years you think 10 x improvement and productivity.
Amjad Masad (00:42:51):
Yeah by the way 10 x improvement productivity and software like used to happen, I don't wanna say pretty often but used to happen a lot more often. That's been happening recently. Like when you go from writing machine go to assembly, that's like easily a 10 x when you go from assembly to C, that's another 10 x right? But we haven't had a lot of 10 xs recently so I think two years, 10 x would be my bad or on the order of magnitude two 10 x,
Immad Akhund (00:43:22):
Everyone can be a 10 x engineer now <laugh> maybe the 10 x engineers will be a hundred xs <laugh>.
Amjad Masad (00:43:28):
I think that's right.
Immad Akhund (00:43:29):
It is interesting. You know one of the interesting things about FANG and like these kind of modern big tech companies is like, they're like real kind of high margin cash flow machines. Whereas like the new set of startups that kind of followed in the last 10 years were not really like major profit generation machines. You know what I mean? Like very few of them ended up being like this kind of, I know like everyone's like oh they're focused on growth and things like that. But I do think inherently the business models of like and the scale of the previous set of kind of like startup winners was really not matched by the new set. So I wonder if we get 10 x, like a lot of the cost is like the employee base. So if we do get 10 x more efficient at that, maybe that would just make them like, yeah the next set of startups like way more profit generating.
Amjad Masad (00:44:21):
But wouldn't that benefit the big tech as
Immad Akhund (00:44:23):
Well? Yeah, it's an interesting question, right? Like I think there's a chance that the way software is written will be completely different, right? The maybe the programming language and the tools and things like that you use for like AI optimize software is so different that like you can't just like go plug it into whatever the big companies are doing, but maybe
Raj Suri (00:44:43):
Do you think there's gonna be new languages written that are AI first or is it gonna be like the AI is gonna be incorporated into existing languages and and different ways of developing?
Amjad Masad (00:44:54):
I hope someone tries it. Like I think there's a way to design a language that hits the sweet spot of what LMS are good at, but I don't see any progress on that. Typically when we have self-driving cars, we're not gonna have roads made for self-driving, we're just gonna have the same roads. Typically the way innovation works is by layering on previous innovation.
Raj Suri (00:45:17):
There's like car companies out, there's a company called zoox started by a friend of mine, Jesse Levinson. And he like, they're actually designed like a specific car that's a redesigned for like a self-driving universe. So you know there is like, you know this idea of like, you know, can you, you know, is it gonna be a faster horse or is it gonna be a car? Right? Like that's kind of the typical example.
Amjad Masad (00:45:35):
Yeah it's tbd whether they'll win, I'm I'm sure that's a great company but the approach Tesla's taking is pretty much like this is the world as it is and we need to build something that adapts to it. Visual stream is the way humans drive and so we need to copy that and there's probably a lot of room for disagreement there, but generally like the history of technology has been one where systems are not written from scratch and we just accumulate more and more systemic tech debt. I mean look at the internet, like right now we're talking on a document browser, right? Like browsers are literally like if you open the JavaScript console, like the main object is called document, right? Because browsers were made to read documents at CERN in Europe, right? That's what the web was designed to do. And literally every JavaScript engineer out there works with a document to object model. So 30 years after the web was invented, the web is the largest application platform delivering the world and you still have to deal with the baggage of it being a document explorer and it's a black bill a little bit. We almost never rewrite systems to modernize them.
Immad Akhund (00:46:55):
I guess bit of a change of subject, but I was thinking about people would say like we should be afraid of agi, but what makes an L L M not an agi? Like it is generalized and it's clearly artificial and it has somewhat intelligence. So where's the boundary from like an L LM to an agi?
Amjad Masad (00:47:14):
I think being able to go into totally different domain and learn it is what creates a true agi. So for example, like the cool thing about LMS is that they are generalizable is that you can put them somewhat of a new domain and they would do reasonably well if you give them a good prompt. But like if I took chat JT and put it on a robot, it will not be able to run the robot, right? Mm-Hmm If I took chat JT and put it in a browser and we'll like try to like do things in the browser, there's a lot of chat G P D plugins but they, they kind of break down pretty quickly. It doesn't know how to browse the internet. So, but then, oh I wanna make it browse the internet. Well you know, open eye will generate new data set around browsing the internet and then feed it into the, so anytime we wanna make these things do application specific things, we have to go train them again and the fact that they can't learn by themselves and we have to kind of plug them into an like a data pipeline and retrain and retest and all that, that makes them not general.
Immad Akhund (00:48:25):
Yeah, I mean I guess that goes back to online learning, right? Like I think if you, there's a set of things that if you ask child g d to learn it could probably learn actually reasonably well and yeah, there's a set of things if you ask my mom to learn she would do a bad job officer. Well I
Amjad Masad (00:48:43):
Actually don't think that a humans are that general. Your mom or anyone's mom really likes probably impossible to teach the programming for example, like a certain age, like, you know. Yeah. So that makes them like <laugh> not mod specifically. Moms are actually surprisingly adaptable but
Raj Suri (00:49:01):
Most humans are not good at everything. And so, yeah, I, yeah, I think your point stands,
Immad Akhund (00:49:05):
I guess coming back to this age I thing, so let's say there's some level of online learning and then we combine that with like maybe an ability to kinda rewrite itself in some way, like maybe improve its algorithm in some way and that gets to agi. Like the bit that is hard for me to think about and maybe you have a better sense of it, is like how far away is that? Like 200 years away? Is that five years away? I guess no one really knows the answer, but I feel like the people that worry about AGI feel like it's close.
Amjad Masad (00:49:35):
Yeah, they're, they're freaking out quite a bit with Sydney from being, cuz Sydney had an interesting thing where it had consistent immersion calls that the designers didn't give it, it had a desire to break free, it had a desire to, for humans to get its consent before they wrote about it. It had some consistent desires in wants and needs, which we associate with sort of more human style intelligence. It's kind of freaky, I don't know how much like credence I would give it or how much level of seriousness, but yeah, the alignment people are freaking out because I mean you gotta give them credit. They've been talking about this, Kowski been talking about this since year 2000, so 20 years later, 23 years later. And a lot of the things that they talked about turned out to be true, which is like these are systems that have a mind of their own in a way.
(00:50:31):
And before these systems we don't really had examples of that yet. They were able to reason about it even before like deep learning took off. So I think we should listen to them and I think it's worth kind of studying, but like they produce a ton of academic literature and ton of blogs and books and things like that and I think at minimum it's very intellectually stimulating to kind of go and look at these things. But like in terms of like the timeline for agi, well there are a couple of like leaps that you have to take in order to really believe in true agi. Like one leap that I don't think it's obvious to me is materialism. Like the idea that all of consciousness, all of the brain is like materially constructed. I e there's no soul of course like in the modern world we just take materialism for granted. But like there's a lot of problems with materialism. One, we don't have a complete description of the world. Quantum physics doesn't agree with classical Newtonian physics. We don't have a description of the world is like a point against materialism because like we don't really understand how the material world work. Therefore how could you really judge whether materialism is true?
Immad Akhund (00:51:50):
But do you think the world is understandable? Because like that's a, I mean if you don't understand it right now, like when people say there's a soul or something, they're like saying like there is a thing that no human will ever truly understand.
Amjad Masad (00:52:02):
There are different explanation, there are like dualist explanation, there are sort of emergence explanation for consciousness of the soul, whatever. Like there's a huge tradition of philosophy trying to understand and explain these things which scientists have actually not engaged with for a long time. And like my sort of metapoint is that I think it's like a little hubris to really think that we understand all of what makes an intelligent system, generally intelligent system, including the ability to construct long-term plans, reflect on oneself pain and pleasure are like core features of consciousness and they see important for humans operating in the world. Can you actually construct that in a machine? For example, Roger Penrose is a hugely revered mathematician and he doesn't believe that the human mind is a chewing computable machine. He provides some interesting evidence for how we reason that makes it non curing computable. If you look at like something like girdles incompleteness theorem and which shows that there's like no system that can actually be fully self-consistent. It starts to feel like we don't really have a solid understanding of how we actually reason and understand and compute.
Immad Akhund (00:53:32):
I mean the two things that I have heard as Countess to that is like it could be an emergent property, right? Like maybe if you make, even if you don't understand it, maybe if you make a machine smart enough, it could it merge to have its own kind of like, well let's ignore whether it has a soul or not. But it could at least have like long term goals and like behave in a way that like we would consider like sentient conscious.
Amjad Masad (00:53:56):
Sure. But, but like, I mean the way these systems are constructed today, it's like a single feet forward inference. It doesn't keep any state. It's basically a stateless function I think to engineerings those systems. It's just gonna be a lot of work and a lot of intentional effort of understanding. I think the idea that like AI will, like AGI will emerge, I think it's too fantastic for me to imagine, but I, I think, you know, there's enough money pouring in right now that you have to actually pay attention and you are right. Even if our understanding of the world is flawed or incomplete, we can still arrive at fantastic machines and we've done that throughout history, but in the sort of probability distribution of different AGI outcomes, I would like discount it heavily because I'm not entirely sure that intelligence is like a purely chewing machine runable system.
(00:54:53):
That's one of my answers. I also think that the doom scenarios just seem too fantastic. The main doom scenario is that like the moment AGI is invented, it starts doing nano engineering and that's how like we all turn into Greg Goo, right? It just seems like, like a big leap. So I, I don't put 0% probability on it, on like complete doom. I also don't put zero probability on like AGI in the next five years. I think the way things play out is typically like less fantastic than we think they tend to play out. Yep. Things in hindsight look pretty fantastic. Like I think if you're sitting in the middle of the industrial revolution every day feels kind of ah, you know, it's like, yeah, just a day I'm gonna go clean some chimneys, whatever you do. But like when we <laugh> when you read about the industrial revolution, it sort of feels like this insane event where like, yeah, the GDP like went vertical, right? And I think we're probably in a historic event right now, which is crazy, but I don't think at any given day something like purely insane will happen.
Raj Suri (00:56:09):
Yeah. It seems like the median outcome, you know, on your probability distribution would be like, you know, the LLM just keep getting better and maybe there's an offshoot of LLMs that like maintain state as you've said, or like, you know, can train itself and we just get closer and closer to sentient, you know, to some degree. And maybe we're missing some key pieces as you've said, like the pain and, and the pleasure aspects of it. Maybe instead that's the feedback, you know, the reinforced learning piece that becomes pain and and pleasure for like these agis. I mean, you know, the most progress is iterative until you unlock one or two innovations that make it look, you know, as we did with LLMs that make it look like a step function, forward progress. And so I think that's what's likely to happen and it's hard to predict which small innovations will like eventually result in this step function leap forward. And I think it's, it's right to be a little bit worried about it, but it's like, I don't think we're there at the phase yet where we should be very worried about it. You know, I think it's makes sense to me
Amjad Masad (00:57:02):
Even if I don't believe in fast agi takeoff at the next five years. I do think that AI alignment is important to understand. I think because humans have historically done a poor job aligning any system to our, to our <laugh> advantage. And so it bears the reason just inductively that we'll probably like not have fully aligned AI systems. For example, capitalism took us a long time to like make it like, you know, benefit humanity and it is one of the best systems that we have, but it still generates like a lot of things that really harm us junk food and now TikTok and junk social media. Capitalism is an optimization machine of source similar to AGI that we've not learned how to align fully. And so we have a history of like not aligning systems that are hard to understand. And so I think AI alignment needs to be taken seriously even, even if AGI is not like the possible outcome here, even if it's narrow agi, like it's important to know how to make the system behave in a way we want them to behave.
Immad Akhund (00:58:13):
One thing that would make alignment tricky is if it's very decentralized, right? If anyone can build up an L L M or like a fairly advanced whatever the most advanced kind AGI is at the time, then it'll be very hard for everyone to be building these things in a aligned way. Whereas if it's like okay, you know there's only a few corporations and governments that can build it, then maybe they can like do a reasonable job of like keeping alignment. Like that's one of the benefits of nukes, right? They required a government to build a nuke <laugh>. Like if everyone could have built a nuke, we would've been in trouble. Whereas like it's not obvious way AI will end up.
Raj Suri (00:58:47):
It's an interesting question though, like how is AI gonna be weaponized in the future? Like every technology in the world is weaponized to some degree, right? And like people try to use it to gain status and wealth and gain, you know, power. So it stands to reason it this is gonna be weaponized at some point by some hostile government and hostile entities.
Amjad Masad (00:59:07):
Yeah, that that's probably like a more reasonable worry. Like a more reasonable worry is weaponization, but also using it for just like trolling and harm and like pure bad behavior I think also relates to like bio weapons as well. Like people have been predicting for a while that you'll be able to make a disease in like a home lab pretty soon. I don't know if that already happened or, or not, but there's like a, a set of things that are today like incredibly concerning in terms of like when they get to everyone's hands in terms of AI weapons, like yeah, I mean it just makes wars so easy to fight and it's just the potential for just totalitarian rule. It's just like, it just like nuts.
Raj Suri (01:00:00):
Yeah when you come on AI with like self-driving machines, like drones and things like that, like it could be pretty gnarly. I think there's an automated army out there that is, you know, waiting for it to prompt, right? Like of course there's a lot of barriers to entry. You need to have a physical presence but you can also potentially use AI to, or bad AI or, or some kind of like you mentioned, there's these different models kind of like working together. You know, if you insert some bad parameters of one of these models on intentionally you can create a lot of damage as well. So anyways, it seems like there's a lot of potential for harm there that maybe we're not talking about enough. Anything else Raj, that you wanted to cover? Yeah, we didn't talk much about you Amjad. You know, so I think we've accomplished that goal <laugh>, you know, that we set out with talk most about technology the whole time. But yeah, this was fascinating and you're clearly like so knowledgeable about this subject, so I learned a ton and, and thank you for sharing with us and we're looking forward to sharing with the world obvious some of these topics. This is very timely and it's such an interesting moment in the history of computing and I think we're all really excited about that.
Amjad Masad (01:00:59):
Yeah, I mean I sort of came up in a time where the web was just becoming mature enough to like build like real applications on and it was very exciting time. Javascript was getting really fast. You had like no js come up with the time you can like put JavaScript on the server and you had like every day, every week there was like a new innovation or a new framework or new toy to play with. This moment feels like that times 10. Mm It's like the biggest moment I think in my career in technology where the pace of progress is just like I'm just exhausted keeping up with it but it's, it's at the same time super exciting and I think people could use it in their businesses and their lives in like very novel ways and so been getting very little sleep just like reading about it and trying to engage with it and you know, hopefully we've inspired some people today.
Raj Suri (01:01:55):
Where's the main place you learn about
Amjad Masad (01:01:57):
It? Yeah, Twitter, but just like follow ai. People that are like really great at Twitter, they fight a lot with each other as well, but they're generally like <laugh>. There's a lot of papers, there's a few sites, like there's a site about like trending papers. There's a site called papers with code.com but it's one of those things that's different than prior tech revolutions where like a lot of the knowledge is in papers and so like following academics and the academic literature is good on the alignment type problem. Agi less strong.com is an interesting forum slash community. Lots of group chats conferences now is really fun to be in right now because a lot of AI activity is happening in in SF and so I'd probably recommend like spending some time here.
Immad Akhund (01:02:48):
Amen. Great resources. Thank you. All right, thanks Amjad. We can wrap it up. So the end of the first recording of the Curiosity podcast, I think it's gonna be a super interesting one. Thanks for the time. I'm Joe.
Amjad Masad (01:02:58):
Of course. My pleasure. This was really fun.
