Is AI productivity as simple as using more tokens?

00:00:00,080 --> 00:00:30,360 [Eric] [upbeat music] Welcome back to the Token Intelligence Show, where John and I are trying to figure out AI and life, and you get to follow along with our successes and failures. We have a great topic today. John, I'm amazed at how the productivity hacks with AI 00:00:30,360 --> 00:00:45,320 [Eric] have changed so unbelievably fast over the last couple years. And I think, correct me if I'm wrong, but I think the very first iteration of this was prompt engineering. 00:00:45,320 --> 00:00:46,080 [John] Yes. 00:00:46,080 --> 00:01:01,600 [Eric] And it eventually became ca- it was eventually called prompt engineering. But even before that, I remember it was like in a past episode, we mentioned sort of, uh, hacks for Google search, where you could do site colon and it would- 00:01:01,600 --> 00:01:01,610 [John] Yeah 00:01:01,610 --> 00:01:03,100 [Eric] ... you know, Google would search a particular site or whatever. 00:01:03,100 --> 00:01:03,400 [John] Right. Right. 00:01:03,400 --> 00:01:07,460 [Eric] And so the AI version of this was 00:01:07,460 --> 00:01:17,000 [Eric] you give the model a persona before asking your question, right? So you are a, you know, technical writer at Vercel- 00:01:17,000 --> 00:01:17,060 [John] Right 00:01:17,060 --> 00:01:18,390 [Eric] ... right? And that sort of- 00:01:18,390 --> 00:01:18,390 [John] Right 00:01:18,390 --> 00:01:25,050 [Eric] ... you know, will warm the model up to, to operate in a, you know, with that persona as the, as the pervading context, right? 00:01:25,050 --> 00:01:26,040 [John] Right. 00:01:26,040 --> 00:01:42,220 [Eric] And then, of course, it went on to context engineering, you know, p- you know, beyond prompt engineering, which is where you're loading the context window with a bunch of relevant data. And then even the way that you do that changed dramatically in the last couple months- 00:01:42,220 --> 00:01:42,230 [John] Right 00:01:42,230 --> 00:01:56,880 [Eric] ... where you used to do that by actually loading the co- you know, loading a bunch of stuff into the context window in the context of an API call. And now the prevailing architecture is doing that with a bunch of markdown files, you know, in a sandbox or [chuckles] locally on your computer- 00:01:56,880 --> 00:01:56,890 [John] Right 00:01:56,890 --> 00:01:57,560 [Eric] ... which is crazy. 00:01:57,560 --> 00:01:57,580 [John] Right. 00:01:57,580 --> 00:02:06,560 [Eric] So it's been hard to keep up with... Oh, and then skills is another, another major iteration where you're actually 00:02:06,560 --> 00:02:13,220 [Eric] manually loading skills that your agent can use and reach for 00:02:13,220 --> 00:02:14,920 [Eric] while it's executing various tasks. 00:02:14,920 --> 00:02:15,060 [John] Right. 00:02:15,060 --> 00:02:21,160 [Eric] So productivity with AI has been kind of hard to follow. 00:02:21,160 --> 00:02:21,580 [John] Right. 00:02:21,580 --> 00:02:32,560 [Eric] And earlier this week, you and I were talking about maybe the simplest way to get more out of AI. And so do you wanna tell us your deep, dark secret about- 00:02:32,560 --> 00:02:42,280 [John] [laughs] Yeah. I mean, I'll s- I think I'll start this by, like, my stance maybe late last year, or focus- 00:02:42,280 --> 00:02:42,370 [Eric] Yep 00:02:42,370 --> 00:02:58,590 [John] ... late last year. So late last year, I was spending a, a ton of time doing research and, and optimization basically around the, the vectors you just described. So number one, prompt engineering, trying to understand it, trying to get good at it. Number two, context engineering, trying to understand it, [chuckles] trying to get good at- 00:02:58,590 --> 00:02:58,590 [Eric] Yep 00:02:58,590 --> 00:03:08,480 [John] ... you know, um, researching tools, trying tools out that, that essentially, like, e- each uniquely combine those two things in, in some way, right? 00:03:08,480 --> 00:03:09,440 [Eric] Yep. 00:03:09,440 --> 00:03:22,880 [John] Um, and, like, learned a lot and, and there's a lot there, but I was pretty focused on that and really felt like, oh, there's just a ton of value that's gonna be delivered in this layer, and then up in the, in the, the application layer kind of on top. 00:03:22,880 --> 00:03:23,820 [Eric] Right. 00:03:23,820 --> 00:03:31,420 [John] And, um, I think there's still some value there, but let's just say if I was an investor, I would be nervous investing in that- 00:03:31,420 --> 00:03:31,650 [Eric] [laughs] 00:03:31,650 --> 00:03:33,940 [John] ... in that layer. [laughs] 00:03:33,940 --> 00:03:36,280 [Eric] Can I give an, can I give a really specific example of that? 00:03:36,280 --> 00:03:37,380 [John] Yeah. 00:03:37,380 --> 00:03:38,740 [Eric] So 00:03:38,740 --> 00:03:41,540 [Eric] prompt engineering has already been productized. 00:03:41,540 --> 00:03:42,420 [John] Okay. How so? 00:03:42,420 --> 00:03:47,120 [Eric] In that the... I think in two ways. So 00:03:47,120 --> 00:03:51,780 [Eric] we're gonna release an episode about, uh, using a tool called vZero, which is- 00:03:51,780 --> 00:03:51,790 [John] Mm-hmm 00:03:51,790 --> 00:03:52,640 [Eric] ... a vibe coding tool. 00:03:52,640 --> 00:03:53,560 [John] Right. Yep. 00:03:53,560 --> 00:03:59,200 [Eric] And I don't know if you r- remember when we were recording that, but I 00:03:59,200 --> 00:04:01,000 [Eric] put a prompt in, and then they have- 00:04:01,000 --> 00:04:01,120 [John] Yeah 00:04:01,120 --> 00:04:02,140 [Eric] ... a prompt enhancer. 00:04:02,140 --> 00:04:02,380 [John] Yep. 00:04:02,380 --> 00:04:02,799 [Eric] Right? 00:04:02,800 --> 00:04:03,549 [John] Yep. 00:04:03,549 --> 00:04:09,770 [Eric] And, um, it just, it just really made the... It, it did prompt engineering- 00:04:09,770 --> 00:04:09,770 [John] Yeah 00:04:09,770 --> 00:04:11,670 [Eric] ... on my behalf by interpreting my, my- 00:04:11,670 --> 00:04:11,720 [John] Right 00:04:11,720 --> 00:04:23,370 [Eric] ... intent. And then in more advanced tooling like Claude or Cursor, et cetera, you actually have a deliberate plan mode, which- 00:04:23,370 --> 00:04:23,840 [John] Yes 00:04:23,900 --> 00:04:29,340 [Eric] ... is a way better way to do prompt engineering, is to actually build a plan- 00:04:29,340 --> 00:04:29,600 [John] Right 00:04:29,600 --> 00:04:30,460 [Eric] ... and [chuckles] then have the- 00:04:30,460 --> 00:04:31,200 [John] Wh- 00:04:31,200 --> 00:04:31,280 [Eric] You know, the model- 00:04:31,280 --> 00:04:35,240 [John] Which I don't know if you heard this episode, um, 00:04:35,240 --> 00:04:37,360 [John] um, the guy who created CloudCode- 00:04:37,360 --> 00:04:37,390 [Eric] Mm-hmm 00:04:37,390 --> 00:04:37,940 [John] ... Boris- 00:04:37,940 --> 00:04:38,980 [Eric] Mm-hmm 00:04:38,980 --> 00:04:42,600 [John] ... um, was on Lenny's podcast last, in the last couple weeks. 00:04:42,600 --> 00:04:42,659 [Eric] Yep. 00:04:42,660 --> 00:04:43,390 [John] Maybe last week. 00:04:43,390 --> 00:04:44,150 [Eric] Yep. 00:04:44,150 --> 00:04:53,200 [John] He was so funny. He's like, "Plan mode is great, like you should use plan mode," well, you know, et cetera. He said, "It's, all plan mode is, is a prompt to tell it to like plan and not write code." 00:04:53,200 --> 00:04:53,599 [Eric] Yes. 00:04:53,600 --> 00:04:54,360 [John] It's like, that's all it is. 00:04:54,360 --> 00:04:55,340 [Eric] It's prompt engineering. 00:04:55,340 --> 00:05:03,560 [John] Yeah. And then another example of that, um, but prompt engineering, like engineering is like, uh, I mean, that's, that's an exaggeration, like- 00:05:03,560 --> 00:05:05,840 [Eric] That's a, that's a, is a... It was the wrong- 00:05:05,840 --> 00:05:05,850 [John] It- 00:05:05,850 --> 00:05:06,220 [Eric] ... word 00:05:06,220 --> 00:05:07,500 [John] ... anybody... Yeah. 00:05:07,500 --> 00:05:07,560 [Eric] Mm. 00:05:07,560 --> 00:05:18,180 [John] That's what it's called, but anybody could tell it, "Hey, plan and don't write code," and then ask it like, "Hey, this is what I want." Like do a little meta engineering, like, "Tell it to plan and don't write code." 00:05:18,180 --> 00:05:18,400 [Eric] Right. 00:05:18,400 --> 00:05:19,760 [John] Like, how would you best say that? 00:05:19,760 --> 00:05:19,780 [Eric] Right. 00:05:19,780 --> 00:05:21,760 [John] And then it gives you like two sentences, and you have plan mode- 00:05:21,760 --> 00:05:21,920 [Eric] Yep 00:05:21,920 --> 00:05:22,540 [John] ... basically. 00:05:22,540 --> 00:05:23,020 [Eric] Yep. 00:05:23,020 --> 00:05:23,039 [John] Um- 00:05:23,040 --> 00:05:26,120 [Eric] That wasn't intuitive, but it was absolutely possible. 00:05:26,120 --> 00:05:26,240 [John] Right. 00:05:26,240 --> 00:05:28,040 [Eric] And now it's just been productized into- 00:05:28,040 --> 00:05:28,660 [John] Yeah, right 00:05:28,660 --> 00:05:29,099 [Eric] ... the form factor. 00:05:29,100 --> 00:05:39,479 [John] So, so you can... Yeah. So exactly. So there's some meta prompting that happens like in vZero and other tools where you click, this is what I think I want, and you click a button, and it like kinda rewrites it. 00:05:39,480 --> 00:05:39,630 [Eric] Yep. 00:05:39,630 --> 00:05:41,120 [John] And you're like, "Oh yeah, that is what I want." 00:05:41,120 --> 00:05:41,360 [Eric] Yeah. 00:05:41,360 --> 00:05:42,840 [John] Um, yeah. It's productized- 00:05:42,840 --> 00:05:42,930 [Eric] Yeah 00:05:42,930 --> 00:05:43,320 [John] ... for sure. 00:05:43,320 --> 00:05:47,280 [Eric] Yeah. Okay. Well, the other thing which 00:05:47,280 --> 00:05:53,979 [Eric] i- which I think gets closer to your stance is that it's actually 00:05:53,980 --> 00:06:06,640 [Eric] way more cost-effective to do that upfront because the model is putting together a plan which is way cheaper than actually building something or doing more complex reasoning around, you know, how to actually create something- 00:06:06,640 --> 00:06:07,040 [John] Right 00:06:07,040 --> 00:06:13,520 [Eric] ... is just reasoning about what to do and the approach that should be taken, you know, which you then validate. 00:06:13,520 --> 00:06:13,900 [John] Right. 00:06:13,900 --> 00:06:25,546 [Eric] And so you not only save cycles on, you know, going back and forthWith additional prompts to fix things that aren't necessarily right. But I think it's literally just cheaper for the model- 00:06:25,546 --> 00:06:25,546 [John] Right 00:06:25,546 --> 00:06:30,036 [Eric] ... to do that kind of work than it is to actually generate net new assets. 00:06:30,036 --> 00:06:46,215 [John] Yeah. And, and I, and I think actually part of this is, like, a little bit of a change of viewpoint. Like, I still think that's true, but... And, and actually this is a perfect segue. So we've talked about this on previous episodes, there's this new open source 00:06:46,216 --> 00:06:49,556 [John] tool called OpenClaw, and then, like, a million clones of it- 00:06:49,556 --> 00:06:49,776 [Eric] Mm-hmm 00:06:49,776 --> 00:07:10,576 [John] ... out there. And the idea behind it was, like, this is a, uh, a tool that actually does things. So you're used to chatting and, like, getting information and having it, like, prototype some code or maybe write some real code. This thing will actually do things like read your email and then you can tell it to send email. Haven't done [chuckles] that, don't recommend that, but- 00:07:10,576 --> 00:07:10,586 [Eric] Yeah 00:07:10,586 --> 00:07:18,476 [John] ... you know, you get the point. Um, and then a million other things with personal productivity kind of being, like, the bent, but, like, some other creative things too. 00:07:18,476 --> 00:07:19,176 [Eric] Yeah. 00:07:19,176 --> 00:07:20,996 [John] Um, so that's O- OpenClaw. 00:07:20,996 --> 00:07:37,216 [Eric] It, it... Yeah. I would describe it as, you know, the, an area where you and I see, and, and you know, it's interesting, I think a lot of the world who's not directly involved in software engineering doesn't necessarily see how much the software engineering role has changed- 00:07:37,216 --> 00:07:37,276 [John] Right 00:07:37,276 --> 00:07:38,796 [Eric] ... fundamentally forever- 00:07:38,796 --> 00:07:39,236 [John] Right 00:07:39,236 --> 00:07:40,976 [Eric] ... because 00:07:40,976 --> 00:07:48,296 [Eric] AI is doing things that a human once did. The, the really big paradigm shift for OpenClaw 00:07:48,356 --> 00:07:56,836 [Eric] is that, or one of them, there are many, is that it runs on your computer, and it can do, it can do normal, quote unquote, normal things, right? 00:07:56,836 --> 00:07:57,096 [John] Right. 00:07:57,096 --> 00:07:57,956 [Eric] It's not building software. 00:07:57,956 --> 00:07:58,106 [John] Yeah. 00:07:58,106 --> 00:07:59,116 [Eric] It can check your email. 00:07:59,116 --> 00:07:59,816 [John] Exactly. 00:07:59,816 --> 00:08:05,536 [Eric] It can research hotels, it can, you know, it can use a browser. 00:08:05,536 --> 00:08:06,166 [John] Right. It, it- 00:08:06,166 --> 00:08:07,386 [Eric] It can click, it can navigate 00:08:07,386 --> 00:08:10,176 [John] ... make a PowerPoint file, Excel file, Word doc. 00:08:10,176 --> 00:08:10,856 [Eric] Exactly. 00:08:10,856 --> 00:08:10,976 [John] Yeah. 00:08:10,976 --> 00:08:21,016 [Eric] And so it very much is sort of bringing that, like, software, you know, the, the fundamental change of software engineering into more normal computer tasks. 00:08:21,016 --> 00:08:30,876 [John] Yes. Right. Yeah. And, and then there's been products on top of that. I think Cowork is a product that, that kind of sits on top of this layer that, that makes it a little bit more friendly- 00:08:30,876 --> 00:08:30,886 [Eric] Yep, yep 00:08:30,886 --> 00:08:50,376 [John] ... more of a friendly, um, UI. Anyways, um, but, like, back to the OpenClaw thing. This, the, the, like the Cowor- Cowork as an example has... You can chat with it and it can do the things. The, like, extra level that this open source project has is it can do things more autonomously. 00:08:50,376 --> 00:08:50,796 [Eric] Yes. 00:08:50,796 --> 00:09:00,716 [John] So you can, like, leave it a set of instructions or leave it a to-do list, and it can kinda check the to-do list throughout the day, do some work on the to-do list, and kinda update you as it works. Like that's- 00:09:00,716 --> 00:09:00,786 [Eric] Yep 00:09:00,786 --> 00:09:03,236 [John] ... the most kind of unique piece of it. 00:09:03,236 --> 00:09:03,976 [Eric] Yep. 00:09:03,976 --> 00:09:24,456 [John] Um, anyways, so the guy that built that, I'd, I checked, checked on some stats here this week. Um, it is an open source project. What that means is all the source code is visible, auditable, readable, and usable by anyone for free. Um, it is on GitHub, which is a super common place to put code, especially for open source projects. 00:09:24,456 --> 00:09:25,036 [Eric] Yep. 00:09:25,036 --> 00:09:36,116 [John] It is either number two or three all time on all of GitHub as far as the number of people who have starred it or, like, favorited it in, like, three months. 00:09:36,116 --> 00:09:36,816 [Eric] What? 00:09:36,816 --> 00:09:43,076 [John] React is, React, I think is, uh, ahead of it, and there may be one other repo. That's it. All time. 00:09:43,076 --> 00:09:46,516 [Eric] That's unbelievable. Wow. 00:09:46,516 --> 00:09:47,416 [John] Yeah. 00:09:47,416 --> 00:09:48,696 [Eric] I'm clearly speechless. 00:09:48,696 --> 00:09:53,756 [John] Yeah. Yeah. So very popular in the developer community, obviously. 00:09:53,756 --> 00:09:54,275 [Eric] Right. 00:09:54,276 --> 00:10:16,456 [John] Um, and this guy Peter Steinberger started it. He had, he had sold a company a few, a few years ago, um, like, then that was just kind of a hobby project, and then he was talking with Meta and OpenAI, I think, and then ended up deciding to join OpenAI, and then this OpenClaw thing is in, like, some kind of foundation. 00:10:16,456 --> 00:10:16,616 [Eric] Right. 00:10:16,616 --> 00:10:18,156 [John] And OpenAI is, like, sponsoring the foundation. 00:10:18,156 --> 00:10:18,456 [Eric] Yeah, yeah. 00:10:18,456 --> 00:10:18,816 [John] So. 00:10:18,816 --> 00:10:19,176 [Eric] Yeah. 00:10:19,176 --> 00:10:20,836 [John] I don't know. I don't know what else is happening there. 00:10:20,836 --> 00:10:21,006 [Eric] We'll see what happens. We'll see. 00:10:21,006 --> 00:10:28,616 [John] We'll see what happens there. Um, but all that to say, I listened to the podcast with Peter Steinberger- 00:10:28,616 --> 00:10:28,626 [Eric] Mm-hmm 00:10:28,626 --> 00:10:43,736 [John] ... the founder there, and Lex Fridman. It's like [chuckles] a four-hour podcast. Um, but the things that were striking is, number one, which, which w- what's, what's talking between Lex and Peter, Peter's like, "I don't wanna run... I don't wanna do this again. Like, I don't wanna start another company." Like- 00:10:43,736 --> 00:10:43,976 [Eric] Yeah 00:10:43,976 --> 00:10:44,956 [John] ... "I don't wanna do this." 00:10:44,956 --> 00:10:45,135 [Eric] Yeah. 00:10:45,136 --> 00:10:47,276 [John] Um, this is before he announced what he was gonna do. 00:10:47,276 --> 00:10:47,716 [Eric] Okay. Interesting. 00:10:47,716 --> 00:10:49,316 [John] But even on that podcast- 00:10:49,316 --> 00:10:49,776 [Eric] Interesting 00:10:49,776 --> 00:10:52,136 [John] ... it was, like, clear he was gonna partner up with somebody. 00:10:52,136 --> 00:10:53,476 [Eric] Yeah. 00:10:53,476 --> 00:11:06,396 [John] So that was interesting. Number two is his draw to, like, who to join was basically, like, who's gonna give me the most tokens and who has, like, the best models, even, like, the unreleased models. Like, that's all he cared about. 00:11:06,396 --> 00:11:06,936 [Eric] Really? 00:11:06,996 --> 00:11:10,796 [John] Yeah. The, that's what it, what it seems from the- 00:11:10,796 --> 00:11:11,046 [Eric] Wow 00:11:11,046 --> 00:11:12,996 [John] ... from the podcast. Um- 00:11:12,996 --> 00:11:14,576 [Eric] So just more tokens. 00:11:14,576 --> 00:11:17,996 [John] Yeah. And, and then of course he wants the, like, smarter mo- 00:11:17,996 --> 00:11:18,126 [Eric] Yeah, yeah. Of course 00:11:18,126 --> 00:11:19,256 [John] ... like, you know, pre-release models or whatever. 00:11:19,256 --> 00:11:20,496 [Eric] Of course. 00:11:20,496 --> 00:11:35,496 [John] Um, and then it just got me thinking, like... And, and the, and the, and, uh, Lex asked him on the podcast, like, "Well, I mean, how much are you spending a month, like, now on this open source project?" 'Cause he's got, like, a Discord and he has some of his personal, like, bots or claws or whatever you wanna call them- 00:11:35,496 --> 00:11:36,076 [Eric] Mm-hmm 00:11:36,076 --> 00:11:48,716 [John] ... in the, um, in this community that people can talk to. And he was like, I don't remember the exact number, but he was like, "Oh, may- like 20 grand a month." 00:11:48,716 --> 00:11:57,126 [Eric] Okay, let me... Just on him running the, like, his own bots or on him continuing to develop out OpenClaw as a project? 00:11:57,126 --> 00:11:57,726 [John] I think it was both. 00:11:57,726 --> 00:11:57,776 [Eric] Okay. 00:11:57,776 --> 00:11:59,176 [John] I think it was, it was a combination of- 00:11:59,176 --> 00:12:00,046 [Eric] So that's his all in- 00:12:00,046 --> 00:12:00,706 [John] ... bots consuming- 00:12:00,706 --> 00:12:11,456 [Eric] ... running the whole OpenClaw empire, continuing development on the open source product and running the community and all the bots associated with that. 00:12:11,456 --> 00:12:26,515 [John] He, he didn't give, like, specific breakdown, but my impression was that, like, 20, 30-ish grand burn per month was, it was, like, just him. Didn't involve, like, employees or paying salaries or anything, I don't think, and then essentially was all, like, token budget. 00:12:26,904 --> 00:12:28,544 [Eric] [laughs] That's great. 00:12:28,544 --> 00:12:29,884 [John] So 00:12:29,884 --> 00:12:34,604 [John] even if I'm, like, half wrong and, like, let's say half of that was, was the tokens and, and he had some other expenses I'm unaware of. 00:12:34,604 --> 00:12:36,524 [Eric] Sure, let's just say it's 15 grand a month- 00:12:36,524 --> 00:12:36,844 [John] Yeah 00:12:36,844 --> 00:12:38,384 [Eric] ... on tokens. 00:12:38,384 --> 00:12:44,224 [John] It got me thinking, like, how is he spending that much money a month on tokens, and what is he doing with it? 00:12:44,224 --> 00:12:44,884 [Eric] Right. 00:12:44,884 --> 00:12:54,644 [John] Well, and then, and then I, I read another article or heard another podcast where somebody was estimating how many tokens their top engineers were spending. 00:12:54,644 --> 00:12:55,644 [Eric] Mm-hmm. 00:12:55,644 --> 00:13:11,714 [John] And they said ... I think this was the Boris podcast. They said more than their salaries. So there, there's engineers, I guess this is at, like, Anthropic, that are spending hundreds of thousands of dollars in tokens. But I don't know what salaries- 00:13:11,714 --> 00:13:11,714 [Eric] Per year. Like, that's their run rate 00:13:11,714 --> 00:13:36,464 [John] ... I mean, some of the salaries are pretty high there, so, like, hundreds of thousands of dollars is the run rate of their spend. Um, so it just got me thinking, like, that I could not if I tried, even if I worked all the time, have a terminal, even multiple terminals open and working and switching between things and prompting, and I don't th- and get to $100,000. 00:13:36,464 --> 00:13:37,034 [Eric] In a year? 00:13:37,034 --> 00:13:38,104 [John] In spend. 00:13:38,104 --> 00:13:39,443 [Eric] Or in a month? 00:13:39,444 --> 00:13:42,804 [John] Um, hund- hundreds of thousands of dollars a year. 00:13:42,804 --> 00:13:43,163 [Eric] Yeah. 00:13:43,164 --> 00:13:45,144 [John] Um, 00:13:45,144 --> 00:13:49,744 [John] yeah. Or, or let's say 10 or 15 grand a month. Like, I don't, I don't know, I don't know how I would do that. 00:13:49,744 --> 00:13:50,864 [Eric] Okay. Can I- 00:13:50,864 --> 00:13:51,924 [John] [laughs] 00:13:51,924 --> 00:13:53,684 [Eric] Can I challenge that a little bit? 00:13:53,684 --> 00:13:54,404 [John] Okay. 00:13:54,404 --> 00:13:56,484 [Eric] Because 00:13:56,484 --> 00:14:01,644 [Eric] ... So number one, this is, this makes me think about how many tokens I'm using at- 00:14:01,644 --> 00:14:01,674 [John] Right 00:14:01,674 --> 00:14:02,324 [Eric] ... Vercel. 00:14:02,324 --> 00:14:02,484 [John] Right. 00:14:02,484 --> 00:14:08,594 [Eric] Because on the marketing team, I don't know the exact numbers, b- and this is just on the marketing team. 00:14:08,594 --> 00:14:08,624 [John] Right. 00:14:08,624 --> 00:14:11,044 [Eric] We have an enormous token budget. 00:14:11,044 --> 00:14:11,634 [John] Yeah, sure. 00:14:11,634 --> 00:14:23,004 [Eric] And we, you know, we, we use it very, very heavily. Um, I mean, to put it in context, the conversations that I'm having with our head of growth, which we collaborate on a number of things- 00:14:23,004 --> 00:14:23,184 [John] Right 00:14:23,184 --> 00:14:27,204 [Eric] ... we talk a lot about our, like, Claude setup, right? 00:14:27,204 --> 00:14:27,924 [John] Okay. Sure. 00:14:27,924 --> 00:14:28,194 [Eric] And, you know- 00:14:28,194 --> 00:14:28,194 [John] Yeah 00:14:28,194 --> 00:14:28,934 [Eric] ... the skills and- 00:14:28,934 --> 00:14:29,824 [John] Yeah 00:14:29,824 --> 00:14:35,424 [Eric] ... Claude MD and all that sort of stuff, right? So it's a, you know, I would say, like, an atypical marketing team. 00:14:35,424 --> 00:14:35,704 [John] Right. 00:14:35,704 --> 00:14:37,324 [Eric] Um, 00:14:37,324 --> 00:14:38,274 [Eric] not for long, but- 00:14:38,274 --> 00:14:39,664 [John] Yeah, right 00:14:39,664 --> 00:15:02,724 [Eric] ... so I wanna ... Like, I wanna think about that 'cause I do think we probably, like, over-index on that. But the other thing that I would say is, do you think that you're limited by y- the fact that you actually have budget constraints? Like, you couldn't spend, you know, half a million dollars on 00:15:02,724 --> 00:15:06,984 [Eric] tokens in a year in a way that produced 00:15:06,984 --> 00:15:09,123 [Eric] return on investment or even broke even for- 00:15:09,124 --> 00:15:09,134 [John] Right 00:15:09,134 --> 00:15:09,834 [Eric] ... your business, right? 00:15:09,834 --> 00:15:10,384 [John] Right. 00:15:10,384 --> 00:15:17,734 [Eric] But if you had unlimited tokens, would that change the way that you think about what you would even try? 'Cause I think about- 00:15:17,734 --> 00:15:17,734 [John] Well- 00:15:17,734 --> 00:15:21,144 [Eric] ... someone at Anthropic, and it's like, 00:15:21,204 --> 00:15:28,244 [Eric] if you have a literally unlimited budget in tokens, you're gonna think way ... You're gonna think about crazier things to build. 00:15:28,244 --> 00:15:37,724 [John] Yeah. W- well, and the answer... And again, maybe these numbers are a little bit inflated, and they're, like, kinda actual value versus, like, consumer value. 00:15:37,724 --> 00:15:38,284 [Eric] Yeah. 00:15:38,284 --> 00:15:49,183 [John] 'Cause I currently, let's say, and I don't have this currently, but I'm considering it. Let's say I had two plans, $200 a month for OpenAI's, like, top plan, $200 a month for- 00:15:49,184 --> 00:15:49,304 [Eric] Yep 00:15:49,304 --> 00:15:58,444 [John] ... Claude's. Real, real dollars, that's, I don't know, five grand a month of tokens, maybe 10, in, like, real dollars if you're using the API. 00:15:58,444 --> 00:15:59,484 [Eric] Hmm. 00:15:59,484 --> 00:16:04,104 [John] So I, I think I would struggle to spend every one of those tokens today. 00:16:04,104 --> 00:16:05,784 [Eric] Interesting. 00:16:05,784 --> 00:16:08,754 [John] Because the... I mean, the plans are pretty generous- 00:16:08,754 --> 00:16:08,754 [Eric] Yeah 00:16:08,754 --> 00:16:32,644 [John] ... especially at the top tier. So because I... And the reason I, I say it would struggle, because I had kind of a, a little emergency, uh [laughs] in, like, previous week, and, um, ended up d- doing, using for a ton of research and validation, all sorts of things, and, like, hours and hours and hours, and, like, almost ran out of my weekly limit. But I would never wanna work like that on a regular week. 00:16:32,644 --> 00:16:33,964 [Eric] Oh, interesting. 00:16:33,964 --> 00:16:35,764 [John] So- 00:16:35,764 --> 00:16:37,284 [Eric] But that's also emergency- 00:16:37,284 --> 00:16:37,294 [John] Yeah 00:16:37,294 --> 00:16:39,804 [Eric] ... response versus, like, building- 00:16:39,804 --> 00:16:40,594 [John] Like net new building 00:16:40,594 --> 00:16:41,184 [Eric] ... on this idea. 00:16:41,184 --> 00:16:43,223 [John] Yeah. But okay, so here's where I'm going with this. 00:16:43,223 --> 00:16:44,004 [Eric] Okay. 00:16:44,004 --> 00:16:57,863 [John] And what changed my thinking on it and, and how Peter could, could be spending, you know, that much a month. And the answer is, is, um, is in the product, is in his OpenCloud product, is the thing's autonomously doing things. Like, that's how you can spend that much. 00:16:57,864 --> 00:16:58,344 [Eric] Hmm. 00:16:58,344 --> 00:17:04,543 [John] So the product has a thing which is super weird. It's called a heartbeat file. 00:17:04,544 --> 00:17:05,524 [Eric] [laughs] 00:17:05,524 --> 00:17:06,264 [John] Let that sink in. [laughs] 00:17:06,264 --> 00:17:09,484 [Eric] We're totally gonna d- W- well, we've already planned an episode on- 00:17:09,484 --> 00:17:10,024 [John] Yeah 00:17:10,024 --> 00:17:12,414 [Eric] ... like, a personified relationship with AI. 00:17:12,414 --> 00:17:12,684 [John] Yeah. It was- 00:17:12,684 --> 00:17:13,384 [Eric] But we'll save that for later. [laughs] 00:17:13,384 --> 00:17:18,144 [John] Yeah. Um, and it just, it kinda woke me up to, like, how you could spend that much. 00:17:18,144 --> 00:17:18,384 [Eric] Mm-hmm. 00:17:18,384 --> 00:17:21,064 [John] And, and a heartbeat file is just a text file- 00:17:21,064 --> 00:17:21,624 [Eric] Mm-hmm 00:17:21,624 --> 00:17:25,204 [John] ... with a list of instructions, and then there is a 00:17:25,204 --> 00:17:40,944 [John] job, a scheduled job that wakes up the AI on a schedule, 30 minutes, I think, for his product as a default. You can set it to whatever you want. And it reads the instructions, and it does things, and it goes back to sleep and wakes up again in 30 minutes. So you imagine you had a computer running all day, and it's gonna wake up- 00:17:40,944 --> 00:17:40,964 [Eric] Mm 00:17:40,964 --> 00:17:41,844 [John] ... every 30 minutes- 00:17:41,844 --> 00:17:42,424 [Eric] Mm-hmm 00:17:42,424 --> 00:17:44,744 [John] ... and work through what's called a task list. 00:17:44,744 --> 00:17:45,423 [Eric] Yep. 00:17:45,424 --> 00:17:58,164 [John] And, um, for sake of argument, like, i- if you kept a task list fed where every time it wakes up, there's actually, like, there's comments from what it did before- 00:17:58,164 --> 00:17:58,264 [Eric] Hmm 00:17:58,264 --> 00:17:59,724 [John] ... or comments from you. 00:17:59,724 --> 00:18:00,394 [Eric] Yep. 00:18:00,394 --> 00:18:09,664 [John] And then there's a ton of YouTube videos about this, like, now, where people are doing stuff like this. Um, that's what got me thinking like, "Oh, okay. That's how you spend 15 grand." 00:18:09,664 --> 00:18:10,524 [Eric] Yep. 00:18:10,524 --> 00:18:34,244 [John] Um, and that's, like, native to his product and a kind of a unique thing. And not... And, and none of the, none of the major labs like OpenAI or, or, um, you know, whatever, whatever, whatever tool you use, Claude, whatever, like, have a feature like that, partially because of the risk, I think. [laughs] Um, 'cause it's acting, you know, it's, it's running f- it is unmoni- it's autonomous, it's unmonitored. 00:18:34,244 --> 00:18:35,264 [Eric] Right. Right. 00:18:35,264 --> 00:18:39,164 [John] Um, which has a lot of inherent risk in it. So that's why they're not doing it yet. 00:18:39,608 --> 00:18:39,768 [Eric] Yep. 00:18:39,768 --> 00:18:46,948 [John] And that's probably why it's wrapped in a foundation. When they, like, bought OpenCloud, they were like, "That can't be a first-class product right now. We need to wrap that in a foundation so [chuckles]-" 00:18:46,948 --> 00:18:46,958 [Eric] Yeah 00:18:46,958 --> 00:18:48,008 [John] ... so we don't get sued." [chuckles] 00:18:48,008 --> 00:18:48,578 [Eric] Yeah. Yeah, totally. 00:18:48,578 --> 00:18:52,948 [John] Um, but that, that was the realization of like, oh, that's how you spend that much money. 00:18:52,948 --> 00:18:53,608 [Eric] Right. 00:18:53,608 --> 00:18:54,788 [John] Um. 00:18:54,788 --> 00:18:58,308 [Eric] Have you done the math on that? I'm trying to back into it a little bit right now. 00:18:58,308 --> 00:19:02,668 [John] Like, like if the thing basically works every 30 minutes for like some- 00:19:02,668 --> 00:19:03,308 [Eric] Yeah 00:19:03,308 --> 00:19:06,188 [John] ... like how many tokens you would spend 00:19:06,188 --> 00:19:07,548 [John] in like a year. 00:19:07,548 --> 00:19:10,188 [Eric] Yeah. Sorry, I'm looking at it right now. Okay. 00:19:10,188 --> 00:19:10,558 [John] I don't know. I haven't done the math. 00:19:10,558 --> 00:19:11,588 [Eric] Okay. So- 00:19:11,588 --> 00:19:13,408 [John] But that's the only way I think I could get to the math. 00:19:13,408 --> 00:19:15,688 [Eric] Here, I have some math. You ready? 00:19:15,688 --> 00:19:16,708 [John] Yes. 00:19:16,708 --> 00:19:24,768 [Eric] Uh, so I'm just gonna back into this. So if you run Opus 4.6 on standard pricing, and you just run it- 00:19:24,768 --> 00:19:24,908 [John] Okay 00:19:24,908 --> 00:19:27,768 [Eric] ... you know, at 75% utilization. 00:19:27,768 --> 00:19:27,988 [John] Okay. 00:19:27,988 --> 00:19:28,568 [Eric] So let's just say- 00:19:28,568 --> 00:19:28,638 [John] Sure 00:19:28,638 --> 00:19:30,028 [Eric] ... if it's doing a task, right? 00:19:30,028 --> 00:19:30,268 [John] Yeah, yeah. 00:19:30,268 --> 00:19:31,528 [Eric] It's, it's sort of gonna oscillate. 00:19:31,528 --> 00:19:31,868 [John] Right. 00:19:31,868 --> 00:19:35,308 [Eric] But let's just say, like it executes a task, we'll say 30 minutes. 00:19:35,308 --> 00:19:36,148 [John] Okay. 00:19:36,148 --> 00:19:37,768 [Eric] Um, 00:19:37,768 --> 00:19:46,107 [Eric] that is going to be, uh, like $3, 00:19:46,108 --> 00:19:46,708 [Eric] um- 00:19:46,708 --> 00:19:47,528 [John] Every 30 minutes 00:19:47,528 --> 00:19:54,668 [Eric] ... $3 every 30 minutes, and so if you do that for a full day, you're gonna be at like 130 to 150 bucks. 00:19:54,668 --> 00:19:55,488 [John] Okay. 00:19:55,488 --> 00:20:01,848 [Eric] And so if you're doing that every single day, you're over $4,000. 00:20:01,848 --> 00:20:03,728 [John] For 00:20:03,728 --> 00:20:04,808 [John] what time period? 00:20:04,808 --> 00:20:07,808 [Eric] If you're doing it 24 hours a day for 30 days- 00:20:07,808 --> 00:20:08,038 [John] Oh, for a month 00:20:08,038 --> 00:20:09,548 [Eric] ... straight. For a month. 00:20:09,548 --> 00:20:10,508 [John] Yeah. So you still... But the- 00:20:10,508 --> 00:20:11,618 [Eric] Yeah. So, so you're at 4 grand 00:20:11,618 --> 00:20:12,458 [John] ... but then imagine you had multiple of them, right? 00:20:12,458 --> 00:20:13,448 [Eric] Then you have multiple, right? 00:20:13,448 --> 00:20:13,467 [John] Yeah. 00:20:13,468 --> 00:20:13,638 [Eric] So then you multiply by- 00:20:13,638 --> 00:20:14,468 [John] Yeah, that's how you get there 00:20:14,468 --> 00:20:15,107 [Eric] ... right. Exactly. 00:20:15,108 --> 00:20:15,218 [John] Yeah. 00:20:15,218 --> 00:20:15,228 [Eric] Yeah. 00:20:15,228 --> 00:20:15,818 [John] Well, that was- 00:20:15,818 --> 00:20:15,818 [Eric] So it's, it's- 00:20:15,818 --> 00:20:16,688 [John] ... that was my math. 00:20:16,688 --> 00:20:16,828 [Eric] Yeah. 00:20:16,828 --> 00:20:29,188 [John] And that like if I'm a human, like... Yeah. I didn't do like the, the like math like you just did it, but my rough math was like there's no way, like it's a few thousand dollars. But if you run multiple of them- 00:20:29,188 --> 00:20:29,348 [Eric] Right 00:20:29,348 --> 00:20:31,948 [John] ... and they're like auto-waking up or whatever, like that's how you get to like- 00:20:31,948 --> 00:20:32,058 [Eric] 100% 00:20:32,058 --> 00:20:33,148 [John] ... 15 or 20 grand. 00:20:33,148 --> 00:20:33,308 [Eric] 100%, right. 00:20:33,308 --> 00:20:37,127 [John] Yeah. So then the question is like 00:20:37,128 --> 00:20:40,948 [John] what can I do? How... What things can you work that way and what things can you not? So- 00:20:40,948 --> 00:20:43,628 [Eric] Yeah. Like is that, is that even useful? 00:20:43,628 --> 00:20:44,228 [John] Yes. Exactly. 00:20:44,228 --> 00:20:45,288 [Eric] Like it's, it's possible- 00:20:45,288 --> 00:20:45,338 [John] Right 00:20:45,338 --> 00:20:46,368 [Eric] ... but is it useful? 00:20:46,368 --> 00:20:50,498 [John] Right. And, and like us, us nerds are famous for doing things possible and not useful. 00:20:50,498 --> 00:20:50,508 [Eric] [laughs] 00:20:50,508 --> 00:20:53,778 [John] Like it's [chuckles] it goes back a long way. 00:20:53,778 --> 00:20:55,868 [Eric] It goes back... [laughs] Yes. 00:20:55,868 --> 00:21:07,528 [John] So, so I've got two categories. I've got one, which is like creative, exploratory, like work, thinking work almost. So let's say I wanted to like update the messaging on my w- website to like add more AI stuff. 00:21:07,528 --> 00:21:08,068 [Eric] Right. 00:21:08,068 --> 00:21:13,598 [John] Like there's a... That one I probably want to do interactively- 00:21:13,598 --> 00:21:13,848 [Eric] Mm-hmm 00:21:13,848 --> 00:21:15,908 [John] ... and like think as part of the process. 00:21:15,908 --> 00:21:16,728 [Eric] Yep. 00:21:16,728 --> 00:21:27,608 [John] And, and, and I would think that any, any sort of content, like that's how I'd wanna approach it. Or, or even cer- certain content I might wanna not use it at all and just use like voice to text or just like plain write. 00:21:27,608 --> 00:21:28,007 [Eric] Yep. 00:21:28,008 --> 00:21:39,468 [John] Um, 'cause it's part of... The thinking is a core part of it. But from a dev workflow, so say I've got some like really clear outcome, but to get there, there's like complicated orchestration and workflow and data modeling- 00:21:39,468 --> 00:21:39,548 [Eric] Yeah 00:21:39,548 --> 00:21:57,008 [John] ... like and et cetera, that one the question is, and I'm trying to pay attention like, uh, as I like test things right now. Okay. It... Like here's me, and here's my prompting, and then like what value am I adding versus if I had this thing on a loop and it just like 00:21:57,008 --> 00:22:01,268 [John] looped every 30 minutes and like fed what it did last time back into itself. 00:22:01,268 --> 00:22:01,468 [Eric] Mm-hmm. 00:22:01,468 --> 00:22:06,128 [John] Like what are the comparable outcomes if I had a really crisp like outcome of what I wanted? 00:22:06,128 --> 00:22:07,628 [Eric] Yeah. So it's- 00:22:07,628 --> 00:22:11,388 [John] And that's, I think, the more tokens come into that. 00:22:11,388 --> 00:22:15,348 [Eric] That would be a shift from 00:22:15,348 --> 00:22:20,608 [Eric] I start with... So you're really overindexing for the outcome. 00:22:20,608 --> 00:22:20,888 [John] Right. 00:22:20,888 --> 00:22:22,808 [Eric] For your definition of the outcome. 00:22:22,808 --> 00:22:24,088 [John] Right. 00:22:24,088 --> 00:22:30,268 [Eric] Instead of, you know, sort of having a s- more ambiguous, 00:22:30,328 --> 00:22:33,398 [Eric] having a less defined outcome and then guiding- 00:22:33,398 --> 00:22:33,398 [John] Yes 00:22:33,398 --> 00:22:34,888 [Eric] ... guiding along the way. 00:22:34,888 --> 00:22:34,928 [John] Right. 00:22:34,928 --> 00:22:42,168 [Eric] So you have a super specific outcome, and the model is looping, um, 00:22:42,168 --> 00:22:47,528 [Eric] which the, uh, the [chuckles] industry term for this is Wal- Ralph Wiggum. 00:22:47,528 --> 00:22:49,348 [John] Yeah. That's right. [laughs] 00:22:49,348 --> 00:22:49,368 [Eric] [laughs] 00:22:49,368 --> 00:22:49,708 [John] Yeah. 00:22:49,708 --> 00:22:51,738 [Eric] And there are even tools called Ralph, which is a- 00:22:51,738 --> 00:22:51,738 [John] Yeah 00:22:51,738 --> 00:22:52,868 [Eric] ... you know, hilarious- 00:22:52,868 --> 00:22:52,878 [John] Right 00:22:52,878 --> 00:22:53,848 [Eric] ... sort of- 00:22:53,848 --> 00:22:54,778 [John] Internet meme [chuckles] 00:22:54,778 --> 00:22:55,468 [Eric] ... yeah, deep- 00:22:55,468 --> 00:22:55,538 [John] Basically 00:22:55,538 --> 00:22:57,197 [Eric] ... Simpsons internet, uh- 00:22:57,197 --> 00:22:57,197 [John] Right 00:22:57,197 --> 00:22:58,147 [Eric] ... internet meme. 00:22:58,148 --> 00:22:58,768 [John] Right. 00:22:58,768 --> 00:23:13,128 [Eric] But the... Instead of you guiding the m- the model as it executes autonomous work, you just have it reference the super crisp outcome that you've defined. 00:23:13,128 --> 00:23:13,448 [John] Right. Right. 00:23:13,448 --> 00:23:19,628 [Eric] And it evaluates deviation from that and then loops again until it gets there. 00:23:19,628 --> 00:23:21,588 [John] Yeah, exactly. Yeah. Right. 00:23:21,648 --> 00:23:27,838 [Eric] And so essentially you're just y- you could perform the same task over and over and over and over and over. 00:23:27,838 --> 00:23:27,848 [John] Right. 00:23:27,848 --> 00:23:29,708 [Eric] It doesn't matter. It's just gonna get- 00:23:29,708 --> 00:23:30,058 [John] Right 00:23:30,058 --> 00:23:31,628 [Eric] ... closer and closer with each iteration. 00:23:31,628 --> 00:23:45,828 [John] Well, and the full loop, the full loop to where we started is prompt engineering, context engineering, et cetera, et cetera. And then the open question in my mind is like if I'm the one doing the prompting 00:23:45,828 --> 00:23:57,748 [John] and like actively working a problem, that's a lot... That's valuable to me. But if I can get the same result and just have the loop run twice as much, 00:23:57,748 --> 00:23:59,188 [John] do I care? 00:23:59,188 --> 00:24:00,208 [Eric] Hmm. 00:24:00,208 --> 00:24:05,728 [John] That... See, because the one, the one is actively like using my time 'cause I'm reading it and like directing or whatever. 00:24:05,728 --> 00:24:05,967 [Eric] Sure. Sure. 00:24:05,968 --> 00:24:13,048 [John] But if it would've gotten to that same destination, which was clearly defined, and autonomously it needed, say like 10 more reps- 00:24:13,048 --> 00:24:13,058 [Eric] Yeah 00:24:13,058 --> 00:24:22,928 [John] ... then I could've gotten there in 10 reps fewer, which is less token spend. But if it could do it autonomously with 10 extra reps or 15 extra reps, then like that's a really interesting trade-off. 00:24:22,928 --> 00:24:23,218 [Eric] Totally. 00:24:23,218 --> 00:24:27,628 [John] And like probably my time is more valuable than like sitting there and directing it to get their 10 reps faster- 00:24:27,628 --> 00:24:27,708 [Eric] Totally 00:24:27,708 --> 00:24:28,208 [John] ... or 15 reps faster. 00:24:28,208 --> 00:24:29,068 [Eric] Totally. 00:24:29,068 --> 00:24:35,148 [John] And I don't know the answer to this, but like I think people seem to be leaning toward like just give it the extra reps. 00:24:35,148 --> 00:24:54,832 [Eric] Yeah. Well, two, two initial thoughts come to mind. One is-At what point does the decrease in cost make that more of a default way of solving problems? 00:24:54,832 --> 00:24:56,792 [Eric] Right. Because one of the... 00:24:56,792 --> 00:24:57,031 [John] Yeah. 00:24:57,032 --> 00:25:01,272 [Eric] It's, it's, it is cost-prohibitive today, so not everyone can- 00:25:01,272 --> 00:25:11,292 [John] Well, it, it only sort of is. Like, again, for four hundred dollars a month, which is... Not everybody wants to spend that. You can get basically an unlimited budget from the two major providers- 00:25:11,292 --> 00:25:12,052 [Eric] Yeah, that's true. That's true 00:25:12,052 --> 00:25:20,852 [John] ... and use them together, and that's worth, say, five grand. And based on your math, like, it works for, like, four hundred bucks. 00:25:20,852 --> 00:25:22,672 [Eric] That is true. 00:25:22,672 --> 00:25:23,332 [John] But and, and- 00:25:23,332 --> 00:25:25,812 [Eric] That can-- But there's no way that that lasts forever- 00:25:25,812 --> 00:25:26,162 [John] Yeah, no, I agree 00:25:26,162 --> 00:25:28,962 [Eric] ... because they're, because that's essentially taking advantage. So the punchline here is- 00:25:28,962 --> 00:25:37,492 [John] Well, and honestly, it probably works for two hundred bucks because, because, um, OpenAI is, like, so aggressively pursuing devs right now. Like, you get a- 00:25:37,492 --> 00:25:37,902 [Eric] A huge amount, sure 00:25:37,902 --> 00:25:40,921 [John] ... so many tokens, and you can use them however they want, and Anthropic's- 00:25:40,921 --> 00:25:40,992 [Eric] Yeah 00:25:40,992 --> 00:25:43,812 [John] ... a little like you can only use them in our tools. Like, but- 00:25:43,812 --> 00:25:46,292 [Eric] So the punchline is get while the getting's good. 00:25:46,292 --> 00:25:47,432 [John] Yeah. [chuckles] Well, but- 00:25:47,432 --> 00:25:48,502 [Eric] Go, go build something awesome for two hundred bucks a month. 00:25:48,502 --> 00:26:00,432 [John] But think about it. They, um... I don't know. I don't-- I mean, the-- You said, like, I don't know if that lasts forever, and, like, maybe not, but, like, your alternar- alternative right now is to use something downstream from that. 00:26:00,432 --> 00:26:00,542 [Eric] Mm-hmm. 00:26:00,542 --> 00:26:02,572 [John] And then you've got two layers of, like- 00:26:02,572 --> 00:26:02,932 [Eric] Right 00:26:02,932 --> 00:26:05,292 [John] ... of not lasting forever. 00:26:05,292 --> 00:26:05,532 [Eric] Yeah. 00:26:05,532 --> 00:26:05,952 [John] You know? 00:26:05,952 --> 00:26:06,612 [Eric] Yeah. 00:26:06,612 --> 00:26:08,652 [John] Or, or open source, or you can use, like, an open source- 00:26:08,652 --> 00:26:08,662 [Eric] Right 00:26:08,662 --> 00:26:22,492 [John] ... model, which, which, I mean, there are some really good ones, so that, that might be a good long-term play. But the other argument, however, is I do think there's value in the, like, for now, better general intelligence in the frontier models. 00:26:22,492 --> 00:26:23,032 [Eric] Hmm. 00:26:23,032 --> 00:26:28,092 [John] And I think there's value in the, all the, like, security stuff they're gonna work on. 00:26:28,092 --> 00:26:29,532 [Eric] Yeah. Yeah. 00:26:29,532 --> 00:26:36,152 [John] The, the, like, if you're running a business, like, something weird happens 'cause [chuckles] you're using a Chinese model. Like, people aren't gonna be super sympathetic for that. 00:26:36,152 --> 00:26:37,112 [Eric] Right. Right. Exactly. 00:26:37,112 --> 00:26:37,412 [John] Yeah. 00:26:37,412 --> 00:26:37,912 [Eric] Yeah, yeah, yeah. 00:26:37,912 --> 00:26:39,872 [John] Because you decided to save a couple bucks. Um- 00:26:39,872 --> 00:26:49,892 [Eric] So what are the areas? You mentioned a couple. So if you were doing website content, you would wanna be there, you would wanna be more involved in the process so that- 00:26:49,892 --> 00:26:49,902 [John] Right 00:26:49,902 --> 00:26:54,372 [Eric] ... your human input is guiding the process. 00:26:54,372 --> 00:26:59,812 [John] Actually, I mean, yes, but no. That actually isn't the primary reason. It's just 00:26:59,812 --> 00:27:07,572 [John] more of a, like, thinking thing of, like, making sure I got to the outcome I wanted. 'Cause I'm only, I w- maybe I'm not even start... I'm starting with such a vague outcome, let's say. 00:27:07,572 --> 00:27:07,832 [Eric] Right. 00:27:07,832 --> 00:27:12,092 [John] Just for sake of argument, of, like, "I think the website needs to have more AI stuff on it." Like- 00:27:12,092 --> 00:27:12,392 [Eric] Mm 00:27:12,392 --> 00:27:16,892 [John] ... and we're just gonna, like, kinda walk through what I think that means and tweak it and- 00:27:16,892 --> 00:27:17,032 [Eric] Right 00:27:17,032 --> 00:27:17,412 [John] ... whatever. 00:27:17,412 --> 00:27:24,701 [Eric] So it's actually part of the, your... It, it's augmenting your, a creative process that you're gonna go through- 00:27:24,701 --> 00:27:24,701 [John] Right 00:27:24,701 --> 00:27:25,492 [Eric] ... either way. 00:27:25,492 --> 00:27:26,492 [John] Right. Right. 00:27:26,492 --> 00:27:36,572 [Eric] As opposed to, uh, you know, I think about, was it OpenAI that released a bunch of agents and built a browser? No, that was Anthropic. 00:27:36,572 --> 00:27:39,272 [John] Bunch of agents. Oh, right. They tried to rebuild Chrome, right? 00:27:39,272 --> 00:27:40,512 [Eric] They, they, they built... Yeah. They rebuilt- 00:27:40,512 --> 00:27:40,522 [John] Yeah 00:27:40,522 --> 00:27:41,352 [Eric] ... a web browser. 00:27:41,352 --> 00:27:41,442 [John] Right. 00:27:41,442 --> 00:27:43,692 [Eric] And then, um, 00:27:43,692 --> 00:27:47,412 [Eric] the-- What was the other one? A C compiler? 00:27:47,412 --> 00:27:48,252 [John] Yeah. 00:27:48,252 --> 00:27:48,432 [Eric] Um- 00:27:48,432 --> 00:27:51,272 [John] And I think there was, like, a bunch of hard-coded stuff or something in the C compiler. 00:27:51,272 --> 00:27:52,212 [Eric] Right. But- 00:27:52,212 --> 00:27:52,712 [John] But- 00:27:52,712 --> 00:27:53,792 [Eric] Y- you know- 00:27:53,792 --> 00:27:54,672 [John] For sake of argument. 00:27:54,672 --> 00:28:06,372 [Eric] For-- Yeah. To, to draw the comparison, you have, um, you have a process where your outcome-- It's really sort of starting with the outcome and determining- 00:28:06,372 --> 00:28:06,481 [John] Right 00:28:06,481 --> 00:28:10,392 [Eric] ... whether or not throwing more tokens at it and creating a loop 00:28:10,392 --> 00:28:11,511 [Eric] is the best way to handle it. 00:28:11,512 --> 00:28:11,672 [John] Right. 00:28:11,672 --> 00:28:14,832 [Eric] Because even with the browser situation, 00:28:14,832 --> 00:28:23,512 [Eric] there are, there's an unbelievable amount of documentation on browser speck, browser specs. You know, you can fork Chrome. Um- 00:28:23,512 --> 00:28:24,252 [John] Yeah. Right 00:28:24,252 --> 00:28:27,672 [Eric] ... you know, and so the outcome is very, very clearly defined. 00:28:27,672 --> 00:28:27,812 [John] Yes. 00:28:27,812 --> 00:28:30,042 [Eric] And so that's actually a great use case for a loop- 00:28:30,042 --> 00:28:30,042 [John] Right 00:28:30,042 --> 00:28:31,862 [Eric] ... where you have the outcome. 00:28:31,862 --> 00:28:31,892 [John] Right. 00:28:31,892 --> 00:28:33,792 [Eric] You can measure deviation from the outcome- 00:28:33,792 --> 00:28:33,802 [John] Right 00:28:33,802 --> 00:28:35,412 [Eric] ... and just loop it until- 00:28:35,412 --> 00:28:35,852 [John] Right 00:28:35,852 --> 00:28:37,392 [Eric] ... uh, you know, you sort of get there. 00:28:37,392 --> 00:28:47,372 [John] Right. Which it'd be interesting to see that browser, like, experimentation done again now because I think when it was done before, it was done with some models that were, like, not as good. 00:28:47,372 --> 00:28:48,312 [Eric] Right. Right. 00:28:48,312 --> 00:28:57,852 [John] Um, okay, last thing for this topic. So it's like, where do we go from here? Like, say there are, like, a number of things that make sense to essentially have on a loop- 00:28:57,852 --> 00:28:57,872 [Eric] Mm-hmm 00:28:57,872 --> 00:28:59,952 [John] ... and you can get to good outcomes. 00:28:59,952 --> 00:29:09,492 [John] Then, like, what's the skill set, um, here for engineering? And I'm... And again, I'm kinda stealing this. I think the answer is long-horizon planning. 00:29:09,492 --> 00:29:11,212 [Eric] Hmm. 00:29:11,212 --> 00:29:17,852 [John] So how good can you get at, like, planning, like, way higher level? 00:29:17,852 --> 00:29:17,862 [Eric] Hmm. 00:29:17,862 --> 00:29:31,192 [John] And because before, like, b- when things were so, like, significantly slower, that kinda didn't, like, mat... I mean, you had to have a rough, rough i- everybody, like, had to have a rough idea of where they wanted to go, what they wanted to build. 00:29:31,192 --> 00:29:31,552 [Eric] Yep. 00:29:31,552 --> 00:29:32,952 [John] But 00:29:32,952 --> 00:29:44,432 [John] I think the long-horizon planning, like, just y- like, you wouldn't plan, like, three years in advance or whatever. Like, nobody's used to planning on the right time horizon. They're used to planning, like, quarterly- 00:29:44,432 --> 00:29:44,552 [Eric] Mm-hmm 00:29:44,552 --> 00:29:46,892 [John] ... and, like, know what, like, a quarterly amount of work is. 00:29:46,892 --> 00:29:47,212 [Eric] Right. 00:29:47,212 --> 00:29:51,952 [John] But then it turns into, like, "I need to think differently in planning," and, like, what maybe used to be- 00:29:51,952 --> 00:29:52,192 [Eric] Hmm 00:29:52,192 --> 00:29:57,232 [John] ... like, three-year planning is now, like, a quarterly plan. But I've never really done a three-year plan 'cause it wasn't worth- 00:29:57,232 --> 00:29:57,392 [Eric] Right 00:29:57,392 --> 00:29:57,892 [John] ... the effort. 00:29:57,892 --> 00:30:06,762 [Eric] Or, or, or the three-year plan is, uh, is significantly distilled. Like, there's not a lot of detail in it. It's just sort of- 00:30:06,762 --> 00:30:07,182 [John] Oh, of course 00:30:07,182 --> 00:30:07,992 [Eric] ... directionally- 00:30:07,992 --> 00:30:15,372 [John] Yeah. That's, that's what I mean. It's almost like you're gonna have to get good at, like, detailed three-year plans, which are now accomplishable in a quarter, potentially. 00:30:15,372 --> 00:30:15,912 [Eric] Right. Right. Right. 00:30:15,912 --> 00:30:18,842 [John] Which essentially nobody's done that. Like, as far- 00:30:18,842 --> 00:30:18,842 [Eric] Right 00:30:18,842 --> 00:30:20,232 [John] ... as a skill set. Like, where do you find that skill set? 00:30:20,232 --> 00:30:21,492 [Eric] Totally. But then- 00:30:21,492 --> 00:30:21,672 [John] Um- 00:30:21,672 --> 00:30:27,322 [Eric] Okay, so you're doing all of this... Okay, so here's... Right. 00:30:27,322 --> 00:30:30,712 [John] And I don't think that's a today problem. I, I think that's coming. 00:30:30,712 --> 00:30:46,452 [Eric] I agree. So let's divide this into two areas. So one area where I immediately see that providing utility, and I think already is, although I haven't, I haven't actually researched this topic, but things like scientific research- 00:30:46,452 --> 00:30:46,602 [John] Hmm 00:30:46,602 --> 00:30:48,512 [Eric] ... you know, new discoveries and other things like that. 00:30:48,512 --> 00:30:48,912 [John] Yeah. 00:30:48,912 --> 00:30:52,312 [Eric] A hundred percent compressing the three-year plan into three months. 00:30:52,312 --> 00:30:53,251 [John] Yep. 00:30:53,252 --> 00:30:55,112 [Eric] I think there's immediate value today for that. 00:30:55,112 --> 00:30:55,932 [John] Right. 00:30:55,932 --> 00:30:59,992 [Eric] With other things, though, especially if you think about building a product, 00:30:59,992 --> 00:31:10,432 [Eric] the, the big question in my mind is, and even if you think about OpenClaw, you, you compress three years of product development into three months. 00:31:10,432 --> 00:31:11,312 [John] Okay. Yep. 00:31:11,312 --> 00:31:16,572 [Eric] People can't consume that much product. 00:31:16,572 --> 00:31:17,172 [John] Sure. 00:31:17,172 --> 00:31:17,332 [Eric] Right? 00:31:17,332 --> 00:31:19,672 [John] Like, the, the speed of change is too rapid- 00:31:19,672 --> 00:31:20,182 [Eric] You just- 00:31:20,182 --> 00:31:21,132 [John] ... for, like, your users 00:31:21,132 --> 00:31:24,502 [Eric] ... like, y- yeah, your users, like, literally can't consume at that rate. 00:31:24,502 --> 00:31:24,552 [John] Right. 00:31:24,552 --> 00:31:33,072 [Eric] And so there's almost this physical ceiling of, you know, sort of what, what is the limit of where that has utility- 00:31:33,072 --> 00:31:33,142 [John] Right 00:31:33,142 --> 00:31:36,112 [Eric] ... in terms of building products, if that makes sense. 00:31:36,112 --> 00:31:52,612 [John] Yeah. That's a... Yeah. That is a great point, 'cause, because let's... The, yeah, the assumption is that, like, there's utility in that, and then it becomes... Then it, then the true bottleneck, like, emerges, which is, like, user adoption, p- like, end customer- 00:31:52,612 --> 00:31:52,622 [Eric] Right 00:31:52,622 --> 00:31:57,332 [John] ... bottleneck that, that basically, like, there's no bottleneck before then. 00:31:57,332 --> 00:31:58,352 [Eric] Yes. 00:31:58,352 --> 00:32:00,392 [John] Which has, which has never been true. 00:32:00,392 --> 00:32:04,012 [Eric] Yeah. Yeah. That's interesting. All right. 00:32:04,012 --> 00:32:04,342 [John] Yeah. 00:32:04,342 --> 00:32:06,402 [Eric] I'm gonna go look up how many tokens I'm burning on a daily basis. [chuckles] 00:32:06,402 --> 00:32:08,252 [John] [chuckles] Please do. 00:32:08,252 --> 00:32:08,462 [Eric] [chuckles] All right. 00:32:08,462 --> 00:32:08,772 [John] All right. 00:32:08,772 --> 00:32:13,992 [Eric] Thanks for joining the show. We'll catch you on the next one.