7 questions with Trey Grainger: AI is an alien intelligence, not a computer

00:00:00,120 --> 00:00:31,400 [Eric Dodds] [upbeat music] Welcome back to Token Intelligence. We are going to talk with our 3rd guest on the show, so 3rd time's a charm. And you had a chance to catch up with Trey Grainger, and I'm so interested in this conversation because Trey is an expert in AI search. And we've talked on the show before about how AI has really changed the way a lot of- 00:00:31,400 --> 00:00:31,410 [John Wessel] Yeah 00:00:31,410 --> 00:00:40,480 [Eric Dodds] ... people search, right? So, I know for you and I, instead of just going straight to Google, we often start with GPT or Claude as our entry to search. 00:00:41,560 --> 00:00:46,100 [Eric Dodds] I still use Google because I think the art of Googling is still very [laughing] relevant. 00:00:46,100 --> 00:00:46,900 [John Wessel] Yeah, sure. 00:00:46,900 --> 00:00:55,320 [Eric Dodds] But agents are actually better at Googling than almost all humans. And so it's a very interesting space because they can execute a lot of search- 00:00:55,320 --> 00:00:55,460 [John Wessel] Yeah 00:00:55,460 --> 00:00:56,340 [Eric Dodds] ... very quickly- 00:00:56,340 --> 00:00:57,240 [John Wessel] Yeah 00:00:57,240 --> 00:01:06,200 [Eric Dodds] ... and summarize the results for you. So, Trey is actually exploring even new ways of searching, uh, in the AI era that we live in. 00:01:06,200 --> 00:01:10,720 [John Wessel] Yep. So he's worked with... Uh, he's, he's a local. He's a Furman, Furman grad- 00:01:10,720 --> 00:01:11,040 [Eric Dodds] Okay 00:01:11,040 --> 00:01:13,740 [John Wessel] ... as am I. Um, and he's a local Greenvillian- 00:01:13,800 --> 00:01:14,260 [Eric Dodds] Mm-hmm 00:01:14,260 --> 00:01:38,260 [John Wessel] ... as Eric and I are as well. And, um, yeah. So we'll, we'll post a link to his talk, but he's doing some really innovative stuff with search. He's working with top comp- top companies like Amazon and NVIDIA and Apple, um, on various search projects and training their teams. Um, so we'll post to his talk, but a little spoiler, um, he's... part of his talk, and he's... we're gonna cover it in the, um, interview as well, is wormhole vector search. 00:01:38,260 --> 00:01:38,640 [Eric Dodds] Wow. 00:01:38,640 --> 00:01:39,840 [John Wessel] So if that piques your interest, uh- 00:01:39,840 --> 00:01:42,900 [Eric Dodds] Wormhole [chuckling] vector search. Okay. 00:01:42,900 --> 00:01:43,340 [John Wessel] Yeah. 00:01:43,340 --> 00:01:45,340 [Eric Dodds] All right. Let's go, uh, talk with Trey. 00:01:45,340 --> 00:01:53,060 [John Wessel] Yeah, let's dive in. All right. We're wrapping up the day here at the Carolina Code Conference with Trey Grainger. Trey, welcome to the show. 00:01:53,060 --> 00:01:54,580 [Trey Grainger] Thanks, John. Appreciate it. 00:01:54,580 --> 00:01:59,000 [John Wessel] Awesome. So just give us a little bit of, little bit of background on you, and we'll dive into some questions. 00:01:59,000 --> 00:02:34,060 [Trey Grainger] Sure. So my name's Trey Grainger. I am the founder of Search Kernel, which is an AI search, uh, consultancy company. Uh, also the author of the, the book AI-Powered Search with 2 great co-authors, Doug Turnbull and Max Irwin. Uh, and I've basically spent, uh, my entire career working, uh, in search and information retrieval, kind of the intersection of, uh, AI, search, relevance, matching, ranking, things like that. I've been, you know, uh, CTO, SVP of engineering, uh, you know, CTO a few times, was chief algorithms officer at one point. 00:02:34,060 --> 00:02:34,100 [John Wessel] Okay. 00:02:34,100 --> 00:02:44,240 [Trey Grainger] You know, San Francisco-based software companies, uh, you know, uh, CTO of a decentralized web search engine at one point. So I've been kind of up and down the, the startup, uh- 00:02:44,240 --> 00:02:44,480 [John Wessel] Yeah 00:02:44,480 --> 00:02:50,380 [Trey Grainger] ... spectrum. And, uh, now, yeah, I run my own thing, uh, working with some of the best engineers in the world, some of the- 00:02:50,380 --> 00:02:50,390 [John Wessel] Awesome 00:02:50,390 --> 00:02:52,140 [Trey Grainger] ... coolest companies in the world. So I'm enjoying it. 00:02:52,140 --> 00:03:05,720 [John Wessel] Yeah. That's awesome. Couple questions for you today, um, all of them related to AI, as maybe you're not surprised. But this is one of my favorite ones. Um, we've actually had kind of a standard set of questions today and gotten some really divergent answers. 00:03:05,720 --> 00:03:05,920 [Trey Grainger] Okay. 00:03:05,920 --> 00:03:13,109 [John Wessel] So for this first one, um, so imagine you time traveled back to 1900. So this is the Gilded Age, turn of the century. 00:03:13,109 --> 00:03:13,440 [Trey Grainger] Okay. 00:03:13,440 --> 00:03:21,000 [John Wessel] How would you describe the AI to s- somebody that lived there? So this is pre-computer, you know. I think pre-electricity would be, you know, around 1900 too. 00:03:21,000 --> 00:03:22,580 [Trey Grainger] This is like... Tesla was like right- 00:03:22,580 --> 00:03:23,400 [John Wessel] Yeah. Uh-huh. 00:03:23,400 --> 00:03:23,470 [Trey Grainger] Yeah, yeah. 00:03:23,470 --> 00:03:23,540 [John Wessel] Yeah, yeah. 00:03:23,540 --> 00:03:23,900 [Trey Grainger] Okay. 00:03:23,900 --> 00:03:24,980 [John Wessel] Right. Yeah. 00:03:24,980 --> 00:03:36,000 [Trey Grainger] Uh, [hissing] I th- if I recall, uh, Tesla thought he was intercepting messages from aliens from, like, Mars or Venus or somewhere- 00:03:36,000 --> 00:03:36,650 [John Wessel] Uh-huh. Yeah 00:03:36,650 --> 00:03:40,420 [Trey Grainger] ... at one point. I think there was a general belief back then in aliens- 00:03:40,420 --> 00:03:40,760 [John Wessel] Mm-hmm 00:03:40,760 --> 00:03:45,150 [Trey Grainger] ... um, that they lived on other planets. Like, why not? So within our solar system. So- 00:03:45,150 --> 00:03:45,940 [John Wessel] Yeah 00:03:45,940 --> 00:03:52,140 [Trey Grainger] ... I think I would probably go with something along the lines of we discovered 00:03:53,900 --> 00:03:55,799 [Trey Grainger] how to build a brain- 00:03:55,800 --> 00:03:56,080 [John Wessel] Mm-hmm 00:03:56,080 --> 00:03:59,500 [Trey Grainger] ... um, and it acts like an alien intell- intelligence or something like that. 00:03:59,500 --> 00:03:59,670 [John Wessel] Uh-huh. Yeah. 00:03:59,670 --> 00:04:02,760 [Trey Grainger] Um, probably, like, I don't... I, I mean, there's other metaphors you could- 00:04:02,760 --> 00:04:03,079 [John Wessel] Sure 00:04:03,079 --> 00:04:05,850 [Trey Grainger] ... go with. Um, but I feel like, 00:04:07,300 --> 00:04:09,640 [Trey Grainger] yeah, with- without the notion of a computer, I feel like- 00:04:09,640 --> 00:04:10,660 [John Wessel] Right. Yeah 00:04:10,660 --> 00:04:13,650 [Trey Grainger] ... I, I would wanna lean on the notion of some other intelligence that's not human. 00:04:13,650 --> 00:04:15,000 [John Wessel] A foreign intelligence. Yeah. 00:04:15,000 --> 00:04:16,880 [Trey Grainger] Um, and it thinks differently. 00:04:16,880 --> 00:04:17,089 [John Wessel] Mm-hmm. 00:04:17,089 --> 00:04:20,620 [Trey Grainger] Um, very intelligent, but doesn't, you know, do things the same way we do. I think I would go with- 00:04:20,620 --> 00:04:20,730 [John Wessel] Yeah 00:04:20,730 --> 00:04:25,900 [Trey Grainger] ... that analogy, and I don't think it would surprise anyone at that point. I mean, it wouldn't surprise everyone at that point- 00:04:25,900 --> 00:04:25,980 [John Wessel] Mm-hmm 00:04:25,980 --> 00:04:28,600 [Trey Grainger] ... to learn that aliens existed in some form. 00:04:28,600 --> 00:04:29,120 [John Wessel] [laughs] Okay. Yeah. 00:04:29,120 --> 00:04:30,620 [Trey Grainger] Just a different form than they thought. That'd probably- 00:04:30,620 --> 00:04:30,980 [John Wessel] Yeah 00:04:30,980 --> 00:04:31,340 [Trey Grainger] ... be my, my go-to. 00:04:31,340 --> 00:04:34,920 [John Wessel] Interesting. Okay. I like that. Yeah. That's awesome. 00:04:34,920 --> 00:04:36,220 [Trey Grainger] I don't think I'd try to explain- 00:04:36,220 --> 00:04:36,230 [John Wessel] Um- 00:04:36,230 --> 00:04:37,690 [Trey Grainger] ... the intricacies of a computer at any point [laughs] in history. 00:04:37,690 --> 00:04:39,740 [John Wessel] [laughs] Yeah. Sure. 00:04:39,740 --> 00:04:42,100 [Trey Grainger] May- maybe after they buy the alien thing. [laughs] 00:04:42,100 --> 00:04:49,900 [John Wessel] Yeah. The, the other common one I got today, um... So that, yeah, you're the 1st alien one, which is awesome. Somebody mentioned Orwell, but didn't go- 00:04:49,900 --> 00:04:50,090 [Trey Grainger] Mm-hmm 00:04:50,090 --> 00:05:03,780 [John Wessel] ... like, toward, you know, kind of that, like, science fiction of the day, but didn't go toward aliens, so that's awesome. My other ones have gone more toward, like, wizardry or sorcery or, like, imagine you're writing a letter and there's an automatic response to your letter. So that- 00:05:03,780 --> 00:05:03,890 [Trey Grainger] Yeah 00:05:03,890 --> 00:05:04,790 [John Wessel] ... so that's interesting. 00:05:04,790 --> 00:05:06,810 [Trey Grainger] I, I think the, the tricky thing is 00:05:08,200 --> 00:05:12,060 [Trey Grainger] we essentially... We either created it or we discovered it- 00:05:12,060 --> 00:05:12,240 [John Wessel] Right 00:05:12,240 --> 00:05:13,039 [Trey Grainger] ... as, like, the algorithm- 00:05:13,040 --> 00:05:13,180 [John Wessel] Yeah 00:05:13,180 --> 00:05:14,300 [Trey Grainger] ... can learn whatever. 00:05:14,300 --> 00:05:14,400 [John Wessel] Right, right. 00:05:14,400 --> 00:05:17,760 [Trey Grainger] And so depending on your perspective. And so I think, uh, 00:05:18,800 --> 00:05:21,760 [Trey Grainger] people will buy pretty much anything, but when you say we created it- 00:05:21,760 --> 00:05:21,920 [John Wessel] Uh-huh 00:05:21,920 --> 00:05:25,120 [Trey Grainger] ... and without them being able to understand what we did, it will sound like wizardry. 00:05:25,120 --> 00:05:25,620 [John Wessel] Yeah. Sure. 00:05:25,620 --> 00:05:30,860 [Trey Grainger] Maybe, maybe, you know, w- a wizard created a new species and... [laughs] 00:05:30,860 --> 00:05:31,140 [John Wessel] Yeah. Yeah. 00:05:31,140 --> 00:05:31,840 [Trey Grainger] I think I could see that. 00:05:31,840 --> 00:05:44,700 [John Wessel] Yeah. Cool. All right. So speaking of AI, what, what kind of in your estimation... You can take this personal or professional. Um, what are the biggest, like, most positive impacts you've seen and the most negative impacts you've seen from AI? 00:05:45,920 --> 00:05:46,240 [Trey Grainger] Oh. 00:05:47,780 --> 00:05:49,260 [Trey Grainger] I'll start with the positive. Uh, 00:05:50,660 --> 00:05:52,539 [Trey Grainger] so everyone's got their own strengths and weaknesses. 00:05:52,540 --> 00:05:52,960 [John Wessel] Mm-hmm. 00:05:52,960 --> 00:06:01,060 [Trey Grainger] Uh, I, strength-wise, um, from, like, middle school, maybe even elementary school and beyond, was always a really good writer. 00:06:01,060 --> 00:06:01,540 [John Wessel] Okay. 00:06:01,540 --> 00:06:02,390 [Trey Grainger] Um, and I prided- 00:06:02,390 --> 00:06:02,390 [John Wessel] Very cool 00:06:02,390 --> 00:06:03,330 [Trey Grainger] ... myself on- 00:06:03,330 --> 00:06:03,330 [John Wessel] Yeah 00:06:03,330 --> 00:06:10,000 [Trey Grainger] ... like, you know, being able to write, you know, whether it was papers or emails or whatever. Um, I mean, I misspell things from time to time, but- 00:06:10,000 --> 00:06:10,220 [John Wessel] Right 00:06:10,220 --> 00:06:11,020 [Trey Grainger] ... generally pretty good. 00:06:11,020 --> 00:06:11,640 [John Wessel] Right. 00:06:11,640 --> 00:06:16,010 [Trey Grainger] And, uh... But a lot of people have... don't have that skill at all. 00:06:16,010 --> 00:06:16,020 [John Wessel] Yeah. 00:06:16,020 --> 00:06:17,260 [Trey Grainger] They're really good at other things. 00:06:17,260 --> 00:06:17,860 [John Wessel] Yeah. 00:06:17,860 --> 00:06:22,752 [Trey Grainger] And- Historically, I have perceived those people to always struggle- 00:06:22,752 --> 00:06:22,912 [John Wessel] Mm-hmm 00:06:22,912 --> 00:06:24,032 [Trey Grainger] ... uh, professionally- 00:06:24,032 --> 00:06:24,472 [John Wessel] Mm-hmm 00:06:24,472 --> 00:06:26,572 [Trey Grainger] ... by, you know, having to, like, communicate and write emails. 00:06:26,572 --> 00:06:26,772 [John Wessel] Yeah. 00:06:26,772 --> 00:06:31,572 [Trey Grainger] And people read the emails or the message, and they perceive them as maybe less intelligent- 00:06:31,572 --> 00:06:31,792 [John Wessel] Right 00:06:31,792 --> 00:06:32,232 [Trey Grainger] ... or- 00:06:32,232 --> 00:06:32,372 [John Wessel] Yeah 00:06:32,372 --> 00:06:35,712 [Trey Grainger] ... you know, not as professional or something, even though, you know, it's not... that's just what- 00:06:35,712 --> 00:06:35,992 [John Wessel] Yeah 00:06:35,992 --> 00:06:37,111 [Trey Grainger] ... what they've been gifted with. 00:06:37,112 --> 00:06:37,612 [John Wessel] Right. 00:06:37,672 --> 00:06:43,292 [Trey Grainger] Uh, and I've seen lots of those people, like overnight, their writing is amazing. 00:06:43,292 --> 00:06:43,702 [John Wessel] Interesting. Yeah. 00:06:43,702 --> 00:06:49,932 [Trey Grainger] Uh, and, and it's, and it's-- for, for, like, normal day-to-day things, it, it's not like the AI is just doing the thinking for them. 00:06:49,932 --> 00:06:50,152 [John Wessel] Right. 00:06:50,152 --> 00:06:55,031 [Trey Grainger] It's just that they're using it to make themselves not have to spend as much time and struggle- 00:06:55,032 --> 00:06:55,062 [John Wessel] Mm-hmm 00:06:55,062 --> 00:06:59,772 [Trey Grainger] ... as much to communicate, like, a basic email or a basic message- 00:06:59,772 --> 00:06:59,832 [John Wessel] Yeah 00:06:59,832 --> 00:07:05,912 [Trey Grainger] ... in a professional way. And so I- I've seen a lot of people basically with, with weaker skills- 00:07:05,912 --> 00:07:06,312 [John Wessel] Mm-hmm 00:07:06,312 --> 00:07:07,932 [Trey Grainger] ... um, not need those skills anymore. 00:07:07,932 --> 00:07:08,352 [John Wessel] Yeah. 00:07:08,352 --> 00:07:11,652 [Trey Grainger] And, like, for all practical purposes, I don't even need those [laughs] skills anymore. 00:07:11,652 --> 00:07:12,332 [John Wessel] Yeah. Yeah. 00:07:12,332 --> 00:07:13,872 [Trey Grainger] Um, even though I've had them. It's like- 00:07:13,872 --> 00:07:14,072 [John Wessel] Right 00:07:14,072 --> 00:07:20,232 [Trey Grainger] ... you know, a, a lot of long, hard-fought battles over my lifetime at this point, uh, were kind of waste- 00:07:20,232 --> 00:07:20,462 [John Wessel] Yeah 00:07:20,462 --> 00:07:22,372 [Trey Grainger] ... um, because it's like you don't need those skills anymore. 00:07:22,372 --> 00:07:22,972 [John Wessel] Yeah. 00:07:22,972 --> 00:07:36,252 [Trey Grainger] Um, e- even in, like, my day-to-day work, there's things I, you know, have been working, um, in my space for like 18, 19 years, and probably I could go and delete about 5 years of my career- 00:07:36,252 --> 00:07:36,912 [John Wessel] [laughs] Right 00:07:36,912 --> 00:07:38,842 [Trey Grainger] ... and it would be totally fine 'cause it's not- 00:07:38,842 --> 00:07:39,042 [John Wessel] Right 00:07:39,042 --> 00:07:39,792 [Trey Grainger] ... relevant anymore. [laughs] 00:07:39,792 --> 00:07:40,562 [John Wessel] Right. Yeah. 00:07:40,562 --> 00:07:47,411 [Trey Grainger] And so I, I, I think... Yeah, I think giving people tools to do things that they don't really need to do- 00:07:47,412 --> 00:07:47,832 [John Wessel] Mm-hmm 00:07:47,832 --> 00:07:49,652 [Trey Grainger] ... but that were holding them back, I think that's the biggest- 00:07:49,652 --> 00:07:49,912 [John Wessel] Yeah 00:07:49,912 --> 00:07:51,592 [Trey Grainger] ... like, kind of across the board, the biggest thing. 00:07:51,592 --> 00:07:52,232 [John Wessel] I like that. 00:07:52,232 --> 00:07:53,172 [Trey Grainger] Uh, I mean, for me, 00:07:54,372 --> 00:07:54,592 [Trey Grainger] uh, 00:07:55,652 --> 00:07:57,832 [Trey Grainger] I would say that, like, I run my own business. 00:07:57,832 --> 00:07:58,002 [John Wessel] Mm-hmm. 00:07:58,002 --> 00:08:04,172 [Trey Grainger] I've had people work for me, you know, engineers, marketing people, like it... all, all across the board, um, and it costs money, obviously. 00:08:04,172 --> 00:08:04,912 [John Wessel] Yeah. Yeah. 00:08:04,912 --> 00:08:09,252 [Trey Grainger] Um, and at times it was needed, it was necessary. Right now, it's just me. 00:08:09,252 --> 00:08:09,652 [John Wessel] Mm-hmm. 00:08:09,652 --> 00:08:15,152 [Trey Grainger] Um, and I am able to be more efficient and effective with my time- 00:08:15,152 --> 00:08:15,672 [John Wessel] Mm-hmm 00:08:15,672 --> 00:08:19,652 [Trey Grainger] ... than, in a lot of cases, the overhead of managing other people- 00:08:19,652 --> 00:08:19,882 [John Wessel] Yeah 00:08:19,882 --> 00:08:20,712 [Trey Grainger] ... and paying to do things. 00:08:20,712 --> 00:08:20,992 [John Wessel] Interesting. Right. 00:08:20,992 --> 00:08:22,142 [Trey Grainger] And that won't always be the case. 00:08:22,142 --> 00:08:22,152 [John Wessel] Yeah. 00:08:22,152 --> 00:08:23,952 [Trey Grainger] And it's, it's kind of unique to my current- 00:08:23,952 --> 00:08:24,052 [John Wessel] Mm-hmm 00:08:24,052 --> 00:08:27,592 [Trey Grainger] ... like, setup and how I'm... Yeah, I'm doing more advisory type work. 00:08:27,592 --> 00:08:27,612 [John Wessel] Yeah. 00:08:27,612 --> 00:08:35,232 [Trey Grainger] But I, I think the notion of just general efficiency. Like I, I used to run teams of, you know, 30 to 40 people- 00:08:35,232 --> 00:08:35,242 [John Wessel] Mm-hmm 00:08:35,242 --> 00:08:36,932 [Trey Grainger] ... engineering teams, companies, what have you- 00:08:36,932 --> 00:08:37,352 [John Wessel] Mm-hmm 00:08:37,352 --> 00:08:40,452 [Trey Grainger] ... um, building things that today could be done with, like, 3 people and AI. 00:08:40,452 --> 00:08:41,272 [John Wessel] Yeah. Wow. 00:08:41,272 --> 00:08:42,072 [Trey Grainger] And I mean, you, 00:08:43,112 --> 00:08:45,372 [Trey Grainger] you still need good, solid engineers and good people- 00:08:45,372 --> 00:08:45,382 [John Wessel] Mm-hmm 00:08:45,382 --> 00:08:50,452 [Trey Grainger] ... but it's just the, the speed at which an individual can ideate and build- 00:08:50,452 --> 00:08:51,032 [John Wessel] Mm-hmm 00:08:51,032 --> 00:09:01,372 [Trey Grainger] ... is so much accelerated now versus, you know, what you used to just have to hire a bunch of people to do. So I think overall efficiency, particularly in, like, software development- 00:09:01,372 --> 00:09:01,512 [John Wessel] Mm-hmm 00:09:01,512 --> 00:09:03,572 [Trey Grainger] ... I, you know, and I think it'll spread across- 00:09:03,572 --> 00:09:03,672 [John Wessel] Yeah 00:09:03,672 --> 00:09:12,592 [Trey Grainger] ... all industries pretty much. But, uh, I, I think those are big wins. In terms of, uh, detriment and downside, uh, I, 00:09:14,972 --> 00:09:20,972 [Trey Grainger] I've had my fair share of people, uh, who have what I can effectively just call AI psychosis. 00:09:20,972 --> 00:09:21,352 [John Wessel] Yeah. 00:09:21,352 --> 00:09:22,082 [Trey Grainger] Um, and there's, there's different- 00:09:22,082 --> 00:09:23,692 [John Wessel] I've heard that term a lot in the last couple of months. Yeah. 00:09:23,692 --> 00:09:24,712 [Trey Grainger] There's different, uh, 00:09:25,732 --> 00:09:35,792 [Trey Grainger] meanings and definitions of it, but, uh, for me, and particularly in software engineering, I've, I've had people come to me who actually weren't in software engineering, and they talk to Claude or talk to- 00:09:35,792 --> 00:09:36,252 [John Wessel] Mm-hmm 00:09:36,252 --> 00:09:40,412 [Trey Grainger] ... OpenAI or whatever, and they, they start to ideate on something and build it. 00:09:40,412 --> 00:09:40,872 [John Wessel] Mm-hmm. 00:09:40,872 --> 00:09:47,272 [Trey Grainger] And they're having these conversations with it, and they're coming up with terminology and language and concepts. 00:09:47,272 --> 00:09:47,481 [John Wessel] Mm-hmm. 00:09:47,481 --> 00:09:49,992 [Trey Grainger] And, and they, they do this for weeks or months. 00:09:49,992 --> 00:09:50,302 [John Wessel] Mm-hmm. 00:09:50,302 --> 00:09:57,902 [Trey Grainger] And they build something, but the thing they built doesn't make any sense. But the, the language that they're using to describe it makes sense to them- 00:09:57,902 --> 00:09:57,902 [John Wessel] Mm-hmm 00:09:57,902 --> 00:09:58,932 [Trey Grainger] ... in their brain- 00:09:58,932 --> 00:09:58,981 [John Wessel] Yeah 00:09:58,981 --> 00:10:01,132 [Trey Grainger] ... because Claude or ChatGPT- 00:10:01,132 --> 00:10:01,332 [John Wessel] Yeah 00:10:01,332 --> 00:10:06,842 [Trey Grainger] ... is, is agreeing with them, and they're coming up with this terminology, and they're building this thing. And then, you know, I'll look at it, I'll be like, 00:10:07,972 --> 00:10:12,652 [Trey Grainger] "I don't really know what this is." And, and they're trying to explain it to me. 00:10:12,652 --> 00:10:12,732 [John Wessel] Right. 00:10:12,732 --> 00:10:15,112 [Trey Grainger] And, and I try to run it, it doesn't even run. I'll be like, "Oh, no- 00:10:15,112 --> 00:10:15,292 [John Wessel] Wow 00:10:15,292 --> 00:10:17,932 [Trey Grainger] ... it will." And so, and I- I've had that multiple times. 00:10:17,932 --> 00:10:18,082 [John Wessel] Wow. 00:10:18,082 --> 00:10:19,652 [Trey Grainger] Like, I'm not talking about a single instance. 00:10:19,652 --> 00:10:20,222 [John Wessel] Mm-hmm. 00:10:20,222 --> 00:10:24,192 [Trey Grainger] So, um, and I've had... I- I've seen some people just kind of go off the deep end a little bit- 00:10:24,192 --> 00:10:24,332 [John Wessel] Right 00:10:24,332 --> 00:10:27,892 [Trey Grainger] ... and get... it's, it's almost like we used to talk about filter bubbles with, like, recommendations- 00:10:27,892 --> 00:10:28,112 [John Wessel] Mm-hmm 00:10:28,112 --> 00:10:29,761 [Trey Grainger] ... and how you can just get, like, stuck in a bubble. 00:10:29,761 --> 00:10:29,792 [John Wessel] Yeah. 00:10:29,792 --> 00:10:33,252 [Trey Grainger] I think a lot of people get stuck in these, like, AI- 00:10:33,252 --> 00:10:37,481 [John Wessel] Yeah, they're stuck in, like, an algorithm in, like, YouTube or in social media. 00:10:37,481 --> 00:10:43,371 [Trey Grainger] Yeah, yeah, yeah. So, and we used to talk about that all the time, but I feel like people can get stuck in a, like, als- almost like AI conversations or AI- 00:10:43,372 --> 00:10:43,402 [John Wessel] Mm-hmm 00:10:43,402 --> 00:10:46,232 [Trey Grainger] ... or even, like, product development concepts. 00:10:46,232 --> 00:10:46,832 [John Wessel] Mm-hmm. 00:10:46,832 --> 00:10:50,272 [Trey Grainger] And, um, like, uh, hopefully they'll get out at some point, but- 00:10:50,272 --> 00:10:50,542 [John Wessel] Yeah 00:10:50,542 --> 00:10:51,532 [Trey Grainger] ... I, I don't know. I feel like- 00:10:51,532 --> 00:10:52,972 [John Wessel] It's a little scary 00:10:52,972 --> 00:10:55,472 [Trey Grainger] ... I, I... it's almost like they're creating parallel universes that- 00:10:55,472 --> 00:10:55,652 [John Wessel] Mm-hmm 00:10:55,652 --> 00:10:59,552 [Trey Grainger] ... only work in their brains, and the AI is guiding them further down that path. 00:10:59,552 --> 00:11:00,132 [John Wessel] Yeah. Yeah. 00:11:00,132 --> 00:11:03,811 [Trey Grainger] So I feel like, yeah, everybody needs to touch grass and have human interaction. [laughs] 00:11:03,812 --> 00:11:09,332 [John Wessel] Yeah. Yeah. [laughs] Yeah, for sure. Um, so, so kind of future looking, um, 00:11:10,532 --> 00:11:12,652 [John Wessel] what, what are you personally most excited about, 00:11:13,812 --> 00:11:28,532 [John Wessel] um, or professionally as far as potential? So if you, you, you know, plot it out, and I even hesitate to say 5 years, but plot it out a few years based on some trajectories that you currently see, what's most exciting and what's most scary for you? 00:11:28,532 --> 00:11:29,432 [Trey Grainger] Uh, in my specific- 00:11:29,432 --> 00:11:31,912 [John Wessel] Person- yeah, y- yeah, personally, professionally, either way. 00:11:33,032 --> 00:11:37,752 [Trey Grainger] [sighs] So I, I think professionally... So I, I work in, uh, AI search- 00:11:37,752 --> 00:11:37,922 [John Wessel] Mm-hmm 00:11:37,922 --> 00:11:48,532 [Trey Grainger] ... and, um, I've, I've sort of kind of been at the forefront of AI and, you know, particularly over the last couple of years with, um, all, all the AI engineers, um, needing RAG and needing- 00:11:48,532 --> 00:11:48,852 [John Wessel] Yeah 00:11:48,852 --> 00:11:54,972 [Trey Grainger] ... uh, to be able to, like, use search and retrieval to improve AI systems, uh, what I do is, like, more in demand than ever. 00:11:54,972 --> 00:11:55,462 [John Wessel] Yeah, for sure. 00:11:55,462 --> 00:12:03,571 [Trey Grainger] And, uh, it's been awesome seeing the influx of interest. Um, but there's also a lot of, uh... 00:12:05,052 --> 00:12:10,512 [Trey Grainger] there's a lot of hard-learned lessons and concepts that, um... like, I'm, I'm tr- training people, I'm teaching people- 00:12:10,512 --> 00:12:10,632 [John Wessel] Mm-hmm 00:12:10,632 --> 00:12:11,262 [Trey Grainger] ... but 00:12:12,412 --> 00:12:15,621 [Trey Grainger] a lot of things are changing. So, like, a- agentic search in my space- 00:12:15,621 --> 00:12:15,621 [John Wessel] Mm-hmm 00:12:15,621 --> 00:12:16,832 [Trey Grainger] ... is really i- 00:12:16,832 --> 00:12:17,492 [John Wessel] Yeah. Very good. 00:12:17,492 --> 00:12:21,682 [Trey Grainger] Everybody's talking about it right now. Very few people are implementing it well, and most people who are- 00:12:21,682 --> 00:12:21,682 [John Wessel] Yeah 00:12:21,682 --> 00:12:24,152 [Trey Grainger] ... it's very expensive, it takes really long to run- 00:12:24,152 --> 00:12:24,592 [John Wessel] Mm-hmm 00:12:24,592 --> 00:12:26,132 [Trey Grainger] ... and it's marginally better than other- 00:12:26,132 --> 00:12:26,172 [John Wessel] Mm-hmm 00:12:26,172 --> 00:12:30,102 [Trey Grainger] ... approaches that we could use historically. But, you know, we can all see the writing on the wall and- 00:12:30,102 --> 00:12:30,222 [John Wessel] Right 00:12:30,222 --> 00:12:30,912 [Trey Grainger] ... how it will get better. 00:12:30,912 --> 00:12:31,892 [John Wessel] Direction, yeah. 00:12:31,892 --> 00:12:42,495 [Trey Grainger] Um, but it's, it's, it's kind of crazy that... The average engineer can use AI to throw something together in a day- 00:12:42,496 --> 00:12:42,836 [John Wessel] Mm-hmm 00:12:42,836 --> 00:12:48,376 [Trey Grainger] ... that is 80 to 90% as good as what I would build over a couple of months. 00:12:48,376 --> 00:12:48,876 [John Wessel] Right. 00:12:48,876 --> 00:12:53,576 [Trey Grainger] Um, and so it's almost like the slop is good enough in a lot of cases. [laughs] 00:12:53,576 --> 00:12:54,015 [John Wessel] Yeah. 00:12:54,016 --> 00:12:56,695 [Trey Grainger] Uh, but they don't-- The problem is they don't realize when it's not. 00:12:56,696 --> 00:12:57,046 [John Wessel] Yeah, right. 00:12:57,046 --> 00:12:57,836 [Trey Grainger] And so they, they, they- 00:12:57,836 --> 00:12:58,036 [John Wessel] Right. 00:12:58,036 --> 00:13:02,016 [Trey Grainger] But there's all, all these implementations all over the place that just aren't that good- 00:13:02,016 --> 00:13:02,086 [John Wessel] Right 00:13:02,086 --> 00:13:11,606 [Trey Grainger] ... but they're good enough. And, uh, the hard part for me is, you know, one, finding the opportunities where, uh, finding the opportunities where 00:13:12,716 --> 00:13:16,096 [Trey Grainger] the difference between okay and great matters. 00:13:16,096 --> 00:13:16,676 [John Wessel] Yeah, right. 00:13:16,676 --> 00:13:24,536 [Trey Grainger] And then convincing people of the difference and educating them. And so I, I, I see this world where it used to be that I would 00:13:25,616 --> 00:13:37,656 [Trey Grainger] talk... Like, I was at the forefront of my industry. I would talk with people. I would help them get from here to there. And, and now people really just wanna throw it together quickly and build it themselves and almost have you, like, QA it for them. 00:13:37,656 --> 00:13:38,306 [John Wessel] Okay. I don't understand. 00:13:38,306 --> 00:13:40,636 [Trey Grainger] I don't want to be someone's slop QA. 00:13:40,636 --> 00:13:41,056 [John Wessel] Yeah. 00:13:41,056 --> 00:13:52,516 [Trey Grainger] Uh, but it, it... Like, I've, I've got a lot more from the, like, "Hey, can you come, you know, help us understand and build to..." Um, "Can you, like, give us a little guidance? Then we're gonna throw something together, and then we just want you to review it." 00:13:52,516 --> 00:13:52,856 [John Wessel] Right. 00:13:52,856 --> 00:13:55,676 [Trey Grainger] And it's-- And then the thing I'm reviewing is, is awful. 00:13:55,676 --> 00:14:00,376 [John Wessel] Yeah. Yeah, yeah. I've asked this question to a bunch of people today. Would you rather read code or write code? 00:14:02,356 --> 00:14:04,116 [Trey Grainger] Uh, I mean, writing code's more fun- 00:14:04,116 --> 00:14:04,306 [John Wessel] Yeah 00:14:04,306 --> 00:14:04,776 [Trey Grainger] ... for sure. 00:14:04,776 --> 00:14:05,966 [John Wessel] That's what everybody says. 00:14:05,966 --> 00:14:09,176 [Trey Grainger] I think everybody says that. Um, because when... Yeah. If it's someone else's code, 00:14:10,476 --> 00:14:24,236 [Trey Grainger] it's usually either, you know... Not that it's always bad, but you, you have to do a lot of work to wrap your brain around the, the mental model that they're following and why they're doing the things they're doing. Um, and as long as you can discuss it with them, it's okay. 00:14:24,236 --> 00:14:24,286 [John Wessel] Yeah. 00:14:24,286 --> 00:14:29,336 [Trey Grainger] But it's, uh, uh... But I-- And I'm totally fine reading someone else's code and ideating. 00:14:29,336 --> 00:14:29,586 [John Wessel] Sure. 00:14:29,586 --> 00:14:36,756 [Trey Grainger] But I-- What is one of my biggest pet peeves is when someone, um, has the AI build something and then they throw it to me and say, "Will you review it?" 00:14:36,756 --> 00:14:37,356 [John Wessel] [laughs] Yeah. 00:14:37,356 --> 00:14:38,066 [Trey Grainger] And I'm like, "Did, did you-" 00:14:38,066 --> 00:14:39,446 [John Wessel] And they don't, and they don't even know how it works. 00:14:39,446 --> 00:14:40,726 [Trey Grainger] "...did you even put any thought into this?" 00:14:40,726 --> 00:14:41,196 [John Wessel] Yeah, they have no idea how it works. Yeah. 00:14:41,196 --> 00:14:43,646 [Trey Grainger] Like, they, they literally spent 30 seconds with Claude- 00:14:43,646 --> 00:14:43,646 [John Wessel] Yeah. [laughs] 00:14:43,646 --> 00:14:45,396 [Trey Grainger] ... and now I'm spending, like, an hour- 00:14:45,396 --> 00:14:45,996 [John Wessel] Hours 00:14:45,996 --> 00:14:48,696 [Trey Grainger] ... or multiple hours trying to figure out what in the world- 00:14:48,696 --> 00:14:49,296 [John Wessel] Right 00:14:49,296 --> 00:14:53,266 [Trey Grainger] ... Claude did and why it did and what he told Claude. So that's, that's like a big no-no for me. [laughs] 00:14:53,266 --> 00:15:14,216 [John Wessel] [laughs] Sure. Yeah. Yeah. Yeah. So, and this kind of leads to my next question. Um, as far as your specific industry, and you kind of partially already answered that, like, is there, is there any, like, one thing... I mean, you, you may gave it to every- everybody that, um, is watching this. You have a really cool talk on, um, 00:15:15,316 --> 00:15:18,385 [John Wessel] on search. And it's... Let me show you. Worm, wormhole- 00:15:18,385 --> 00:15:18,976 [Trey Grainger] Wormhole vectors. 00:15:18,976 --> 00:15:20,296 [John Wessel] Did you get to name that? 00:15:20,296 --> 00:15:20,906 [Trey Grainger] I did. Yeah. 00:15:20,906 --> 00:15:21,476 [John Wessel] You did. [laughs] 00:15:21,476 --> 00:15:23,156 [Trey Grainger] If you go to mine and search for it, it's like- 00:15:23,156 --> 00:15:23,356 [John Wessel] Yeah 00:15:23,356 --> 00:15:24,716 [Trey Grainger] ... you know, created by Trey Grainger. 00:15:24,716 --> 00:15:25,096 [John Wessel] Yeah. 00:15:25,096 --> 00:15:25,856 [Trey Grainger] It's kind of, kind of- 00:15:25,856 --> 00:15:26,456 [John Wessel] Super cool. 00:15:26,456 --> 00:15:30,206 [Trey Grainger] I've, I've done that a few times in my career. Uh, semantic knowledge graphs, actually. 00:15:30,206 --> 00:15:30,236 [John Wessel] Okay. 00:15:30,236 --> 00:15:33,986 [Trey Grainger] I was the, I, I was the 1st author on the original paper on semantic knowledge graphs. 00:15:33,986 --> 00:15:34,536 [John Wessel] Very cool. Okay. 00:15:34,536 --> 00:15:36,196 [Trey Grainger] And then wormhole vectors as well. 00:15:36,196 --> 00:15:36,436 [John Wessel] Yeah. Yeah. 00:15:36,436 --> 00:15:44,126 [Trey Grainger] Um, and it's... Wormhole vectors is fun because it's, uh, the, the... It takes a lot of building up to, like, get to understand the concept. 00:15:44,126 --> 00:15:44,136 [John Wessel] Yeah. Uh-huh. 00:15:44,136 --> 00:15:46,016 [Trey Grainger] But once you have that base understanding- 00:15:46,016 --> 00:15:46,216 [John Wessel] Yeah 00:15:46,216 --> 00:15:48,036 [Trey Grainger] ... and you explain it, it's actually not that complicated. 00:15:48,036 --> 00:15:59,366 [John Wessel] Yeah. Yeah. So give, yeah, give the listeners just, like, kind of the highest level version of, of kind of what a wormhole vector is. And, and you just, you just spent a whole hour unpacking it, [laughs] so this is super challenging. 00:15:59,366 --> 00:16:00,146 [Trey Grainger] Okay. All right. 00:16:00,146 --> 00:16:01,135 [John Wessel] Putting you on the spot, but... 00:16:01,136 --> 00:16:02,116 [Trey Grainger] Most succinctly- 00:16:02,116 --> 00:16:02,476 [John Wessel] Yeah 00:16:02,476 --> 00:16:02,836 [Trey Grainger] ... uh, 00:16:04,036 --> 00:16:43,696 [Trey Grainger] today when people run search, the sort of, you know, quote unquote "state of the art," meaning, like, what 95% of people do, is hybrid search, which is a lexical search which searches on keywords matching the, like, actual text or strings. Um, and it uses a algorithm called BM25 to score. And then, uh, dense vector semantic search, which is using a transformer-based model to take text or images to encode them into numerical vectors and then to, you know, compare vectors for similarity and find things that match the concepts and the meaning of a query, uh, you know, documents that match the concepts and meaning. So those are the 2 main paradigms that people use. 00:16:43,696 --> 00:16:48,576 [John Wessel] So if I'm Googling something today, like, those rough concepts would be, would be, would be applied. 00:16:48,576 --> 00:16:49,165 [Trey Grainger] Yeah, yeah. 00:16:49,165 --> 00:16:49,176 [John Wessel] Okay. 00:16:49,176 --> 00:16:58,296 [Trey Grainger] Yeah, exactly. And whether it's a specific, you know, text keyword ID match would use more of that lexical approach, or if it's more of a question or, you know, conceptual thing, it would use- 00:16:58,296 --> 00:17:01,996 [John Wessel] And they're merging those 2 things to get the best result. 00:17:01,996 --> 00:17:16,256 [Trey Grainger] Yeah. And so, so what a, what a typical hybrid search would do, it would use a fusion algorithm to essentially run a lexical query and run a dense vector semantic query independently and then take 2 separate sets of results and try to, like, stitch them together- 00:17:16,256 --> 00:17:16,266 [John Wessel] Yeah 00:17:16,266 --> 00:17:18,046 [Trey Grainger] ... you know, semi-intelligently. 00:17:18,046 --> 00:17:18,556 [John Wessel] Mm-hmm. 00:17:18,556 --> 00:17:27,936 [Trey Grainger] Uh, the problem with that, um, one of the things I said in the, the talk is it's almost like you have a crime and, uh, you've got, like, different police departments, maybe it's like the FBI- 00:17:27,936 --> 00:17:28,366 [John Wessel] Yeah 00:17:28,366 --> 00:17:34,275 [Trey Grainger] ... the, um, you know, the local police department, whoever. And, and they're all investigating. Like, the lexical team is investigating. 00:17:34,276 --> 00:17:36,155 [John Wessel] So they start at the same starting point, technically. 00:17:36,156 --> 00:17:55,816 [Trey Grainger] They start at the same starting point, which is what the user typed in. But the lexical team's only looking at some data, and then the, you know, vector team's only looking at, like, the meaning, but not the actual, like, you know, words. And so you end up with 2 different conclusions because they have different evidence. Um, and then they report their conclusions differently, and then you just kind of, like, vote on which ones might be, be the case. 00:17:55,816 --> 00:17:55,916 [John Wessel] Yeah. 00:17:55,916 --> 00:18:21,676 [Trey Grainger] Versus if you could actually pull all the evidence together and relate all the content together, you can do better. So what a wormhole vector does is essentially lets you query in... with one type of query. Um, lexical and dense embeddings are, are, are 2. There's other... You can do graph. You can do... There's the behavioral vector spaces, which, you know, probably won't talk about that today, but there's all sorts of different ways. I teach an online course. We can talk about that too- 00:18:21,676 --> 00:18:21,846 [John Wessel] Sure 00:18:21,846 --> 00:18:48,006 [Trey Grainger] ... where, where I go through all of this for some of the top, uh, AI engineers and search engineers in the industry. But, uh, essentially what a wormhole vector does is allows you to query in one of those, for example, with keywords, to find relevant content- And then based upon that content documents to generate a query and another vector space. So for example, start with a keyword query, find the top relevant documents, use those documents to generate an embedding- 00:18:48,006 --> 00:18:48,186 [John Wessel] Mm-hmm 00:18:48,186 --> 00:19:10,736 [Trey Grainger] ... that I could then go find the place in the dense vector space that maps to the same idea, the same concept that I found in the den- the sparse lexical space. Then fi- find more nearby relevant documents, inspect those, and I can keep jumping around between different vector spaces or different types of queries and just sort of collecting and relating the, the documents, which is, again, like the crime scene example. 00:19:10,736 --> 00:19:10,776 [John Wessel] Yeah. 00:19:10,776 --> 00:19:17,176 [Trey Grainger] It's like pulling all the evidence together. Um, and it's something that, yeah, you could programmatically do now- 00:19:17,176 --> 00:19:17,636 [John Wessel] Mm-hmm 00:19:17,636 --> 00:19:26,576 [Trey Grainger] ... but with the rise of agentic search and agents, which like two-thirds of all, um, LLM calls these days, um, uh, like to ChatGPT or- 00:19:26,576 --> 00:19:26,836 [John Wessel] Mm-hmm 00:19:26,836 --> 00:19:28,376 [Trey Grainger] ... Claude or whatever actually end up running searches. 00:19:28,376 --> 00:19:28,656 [John Wessel] Interesting. Yeah. 00:19:28,656 --> 00:19:29,736 [Trey Grainger] Search, search is critical- 00:19:29,736 --> 00:19:29,966 [John Wessel] It is 00:19:29,966 --> 00:19:36,816 [Trey Grainger] ... to ground LLMs with up-to-date information, factual information, and non-hallucinated information that they can- 00:19:36,816 --> 00:19:36,906 [John Wessel] Yeah 00:19:36,906 --> 00:19:38,506 [Trey Grainger] ... you know, process and reason over. 00:19:38,506 --> 00:19:38,526 [John Wessel] Yeah. Yeah. 00:19:38,526 --> 00:19:46,146 [Trey Grainger] And so, um, with wormhole vectors, this notion of jumping and traversing and sort of figuring out if something's relevant or not and finding more relevant content- 00:19:46,146 --> 00:19:46,146 [John Wessel] Mm-hmm 00:19:46,146 --> 00:19:51,636 [Trey Grainger] ... fits very nicely as a skill or a set of skills that would go into an agentic loop- 00:19:51,636 --> 00:19:51,916 [John Wessel] Mm-hmm 00:19:51,916 --> 00:20:00,696 [Trey Grainger] ... to allow, uh, better search and better context discovery, um, and, and ranking. So I'm, I'm doing a lot of work with it and really excited about it. 00:20:00,696 --> 00:20:00,816 [John Wessel] Yeah. 00:20:00,816 --> 00:20:01,276 [Trey Grainger] But it's, um- 00:20:01,276 --> 00:20:01,796 [John Wessel] It's really cool 00:20:01,796 --> 00:20:13,596 [Trey Grainger] ... it's, uh, a new way to approach, uh, traversing through content and context and ranking that isn't quite as, um, uh, partitioned and, and sort of isolating- 00:20:13,596 --> 00:20:13,776 [John Wessel] Yeah 00:20:13,776 --> 00:20:15,156 [Trey Grainger] ... of, of, of context. 00:20:15,156 --> 00:20:45,095 [John Wessel] So say, so say that was, um, just to go down this rabbit hole just a little bit, say that was fully implemented in whatever AI tool I use every day, ChatGPT or Claude or some other tool. What, what i-- And, and, and maybe we're a couple years out as far as, like, some... Like, it's a little faster than maybe it is now. What, what, what's kind of the practical benefit for people? So if you've got that search embedded in whatever you're using every day, which you said two-thirds of, you know, chats have some kind of search, part of them, it- it's, it's faster, it's more relevant. Like, what, what's the practical benefit for, for people? 00:20:45,096 --> 00:20:46,736 [Trey Grainger] O- of tying a search engine with the LLM? 00:20:46,736 --> 00:20:51,616 [John Wessel] Of, of... Well, no. Of, of, of, um, agentic search versus, like, some of the limitations of what we have now. 00:20:51,616 --> 00:20:52,396 [Trey Grainger] Got it. So 00:20:54,636 --> 00:21:02,936 [Trey Grainger] the best way to think of it is search engines historically have been optimized for, uh, fast queries- 00:21:02,936 --> 00:21:02,945 [John Wessel] Yeah 00:21:02,945 --> 00:21:03,696 [Trey Grainger] ... low latency- 00:21:03,696 --> 00:21:04,156 [John Wessel] Yeah 00:21:04,156 --> 00:21:05,156 [Trey Grainger] ... high scale- 00:21:05,156 --> 00:21:05,456 [John Wessel] Yeah 00:21:05,456 --> 00:21:07,316 [Trey Grainger] ... and one shot. One shot- 00:21:07,316 --> 00:21:07,516 [John Wessel] Yeah 00:21:07,516 --> 00:21:11,546 [Trey Grainger] ... meaning you send the query in, you get a set of ranked results, and then the search engine's done. 00:21:11,546 --> 00:21:11,996 [John Wessel] Get it back. Yeah. 00:21:11,996 --> 00:21:13,536 [Trey Grainger] Uh, they might be good, they might be bad- 00:21:13,536 --> 00:21:13,616 [John Wessel] Mm-hmm 00:21:13,616 --> 00:21:16,896 [Trey Grainger] ... but everything is optimized for getting the best 1st result. 00:21:16,896 --> 00:21:17,436 [John Wessel] Yep. 00:21:17,436 --> 00:21:20,456 [Trey Grainger] Uh, and the reason for that historically is humans were the ones consuming- 00:21:20,456 --> 00:21:20,466 [John Wessel] Yeah 00:21:20,466 --> 00:21:21,126 [Trey Grainger] ... the results- 00:21:21,126 --> 00:21:21,126 [John Wessel] Sure 00:21:21,126 --> 00:21:23,616 [Trey Grainger] ... and they're not gonna... They, they don't wanna run 5- 00:21:23,616 --> 00:21:24,416 [John Wessel] Page after page after page 00:21:24,416 --> 00:21:27,256 [Trey Grainger] ... searches. I mean, they, they will if they have to, but, you know, it's not ideal. 00:21:27,256 --> 00:21:27,716 [John Wessel] Right. 00:21:27,716 --> 00:21:47,366 [Trey Grainger] Uh, also, uh, one thing, just backing up for a second, um, one way I like to describe search engines to people is all a search engine is, is a cache. It's a cache of relevant context and information. Like, if you have an agent, in theory, you could have the agent just crawl through, you know, thousands and thousands or millions of pages- 00:21:47,366 --> 00:21:47,366 [John Wessel] Mm-hmm 00:21:47,366 --> 00:21:48,436 [Trey Grainger] ... even billions, the whole- 00:21:48,436 --> 00:21:48,516 [John Wessel] Yeah 00:21:48,516 --> 00:21:49,216 [Trey Grainger] ... entire internet- 00:21:49,216 --> 00:21:49,276 [John Wessel] Right 00:21:49,276 --> 00:21:50,945 [Trey Grainger] ... and find the answer. And eventually- 00:21:50,945 --> 00:21:50,986 [John Wessel] Eventually [laughing] 00:21:50,986 --> 00:21:51,776 [Trey Grainger] ... if it's big enough- 00:21:51,776 --> 00:21:51,876 [John Wessel] Right 00:21:51,876 --> 00:21:54,296 [Trey Grainger] ... it will, you know, after you've spent all your money on tokens- 00:21:54,296 --> 00:21:55,216 [John Wessel] Yeah. [laughing] 00:21:55,216 --> 00:21:57,316 [Trey Grainger] ... and after, you know, millennia, it will get you the answer. 00:21:57,316 --> 00:21:58,256 [John Wessel] Yeah. [laughing] Right. 00:21:58,256 --> 00:22:00,236 [Trey Grainger] The, the reason that we have the cache- 00:22:00,236 --> 00:22:00,246 [John Wessel] Mm-hmm 00:22:00,246 --> 00:22:01,826 [Trey Grainger] ... is so that you don't have to spend all that- 00:22:01,826 --> 00:22:02,196 [John Wessel] It's the shortcut 00:22:02,196 --> 00:22:02,766 [Trey Grainger] ... it's the shortcut- 00:22:02,766 --> 00:22:02,766 [John Wessel] Yeah 00:22:02,766 --> 00:22:07,496 [Trey Grainger] ... is that I want to pull back relevant context from the entire internet, from the entire world- 00:22:07,496 --> 00:22:07,505 [John Wessel] Mm-hmm 00:22:07,505 --> 00:22:08,856 [Trey Grainger] ... from all of my data- 00:22:08,856 --> 00:22:09,536 [John Wessel] In milliseconds 00:22:09,536 --> 00:22:09,596 [Trey Grainger] ... in milliseconds- 00:22:09,596 --> 00:22:10,076 [John Wessel] Yeah 00:22:10,076 --> 00:22:11,076 [Trey Grainger] ... so that it can be processed. 00:22:11,076 --> 00:22:11,816 [John Wessel] Yeah. Okay. 00:22:11,816 --> 00:22:16,396 [Trey Grainger] And so, um... And, and we get this argument, like I started to hear a lot of, uh, AI engineers- 00:22:16,396 --> 00:22:16,576 [John Wessel] Mm-hmm 00:22:16,576 --> 00:22:19,496 [Trey Grainger] ... arguing like, "Oh, you don't need a search engine. Just use grep. You don't have that much data." 00:22:19,496 --> 00:22:19,996 [John Wessel] Mm-hmm. Yeah. Right. 00:22:19,996 --> 00:22:21,836 [Trey Grainger] And that's true. If you've got a thousand documents- 00:22:21,836 --> 00:22:21,846 [John Wessel] Right 00:22:21,846 --> 00:22:23,505 [Trey Grainger] ... you can grep through that super fast. You don't need- 00:22:23,505 --> 00:22:23,505 [John Wessel] Right 00:22:23,505 --> 00:22:28,826 [Trey Grainger] ... something sophisticated. Uh, but that doesn't mean that grep, like an agent with grep- 00:22:28,826 --> 00:22:28,826 [John Wessel] Mm-hmm 00:22:28,826 --> 00:22:29,836 [Trey Grainger] ... is as good- 00:22:29,836 --> 00:22:29,945 [John Wessel] Right 00:22:29,945 --> 00:22:31,436 [Trey Grainger] ... as an agent with good search. 00:22:31,436 --> 00:22:31,636 [John Wessel] Yeah. 00:22:31,636 --> 00:22:33,496 [Trey Grainger] It, it may be sufficient- 00:22:33,496 --> 00:22:33,666 [John Wessel] Right 00:22:33,666 --> 00:22:43,496 [Trey Grainger] ... but, you know... And, and I also hear this with tools. People say, "Oh, instead of having a complicated search engine that's really good at that one shot, what if we just had some, like, very simple tools and just let the agent iterate- 00:22:43,496 --> 00:22:43,566 [John Wessel] Mm-hmm. Mm-hmm 00:22:43,566 --> 00:22:50,316 [Trey Grainger] ... as many times as you need until it gets the result?" And the reality is, if you have a, an undergrad student- 00:22:50,316 --> 00:22:50,636 [John Wessel] Mm-hmm 00:22:50,636 --> 00:22:56,796 [Trey Grainger] ... who isn't an expert in something, and you give them 10 years and tell them, "You need to solve this problem and- 00:22:56,796 --> 00:22:56,856 [John Wessel] Mm-hmm 00:22:56,856 --> 00:22:59,696 [Trey Grainger] ... use whatever tools you need," and they can iterate as many times as they want- 00:22:59,696 --> 00:22:59,976 [John Wessel] Right. Right 00:22:59,976 --> 00:23:01,456 [Trey Grainger] ... they're probably gonna solve it. 00:23:01,456 --> 00:23:01,616 [John Wessel] Right. 00:23:01,616 --> 00:23:06,716 [Trey Grainger] If you have, you know, someone with a PhD who's been working in the field, they may be able to solve it in a, a week or a month- 00:23:06,716 --> 00:23:06,726 [John Wessel] Mm-hmm 00:23:06,726 --> 00:23:08,676 [Trey Grainger] ... or, or what have you and not take 10 years. 00:23:08,676 --> 00:23:09,176 [John Wessel] Right. 00:23:09,176 --> 00:23:12,376 [Trey Grainger] Um, but that doesn't mean that the intern is better. 00:23:12,376 --> 00:23:12,516 [John Wessel] Yeah. 00:23:12,516 --> 00:23:13,216 [Trey Grainger] Not the intern, the undergrad. 00:23:13,216 --> 00:23:13,726 [John Wessel] Yeah. Yeah, yeah. 00:23:13,726 --> 00:23:14,696 [Trey Grainger] Just because the undergrad is better- 00:23:14,696 --> 00:23:15,056 [John Wessel] Right 00:23:15,056 --> 00:23:16,775 [Trey Grainger] ... um, it's, it's super inefficient- 00:23:16,776 --> 00:23:16,896 [John Wessel] Right 00:23:16,896 --> 00:23:17,876 [Trey Grainger] ... to spend 10 years on something- 00:23:17,876 --> 00:23:18,056 [John Wessel] Yeah 00:23:18,056 --> 00:23:24,276 [Trey Grainger] ... someone else can do in a week. It's just that, like, with enough trial and error, you will eventually get better results. 00:23:24,276 --> 00:23:24,796 [John Wessel] Yeah. 00:23:24,796 --> 00:23:29,456 [Trey Grainger] And so it's the same thing. I think of agents the same way. You can use search to be- 00:23:29,456 --> 00:23:29,486 [John Wessel] Mm-hmm 00:23:29,486 --> 00:23:31,395 [Trey Grainger] ... a hyper-efficient, highly scalable cache- 00:23:31,396 --> 00:23:31,756 [John Wessel] Mm-hmm 00:23:31,756 --> 00:23:34,116 [Trey Grainger] ... so that the agent doesn't have to spend as many tokens- 00:23:34,116 --> 00:23:34,126 [John Wessel] Right. Right 00:23:34,126 --> 00:23:42,116 [Trey Grainger] ... and can be, you know, fast and all that. Um, or you can have the agent just do more work because you don't have those underlying systems. Both will get you to- 00:23:42,116 --> 00:23:42,256 [John Wessel] Right 00:23:42,256 --> 00:23:42,996 [Trey Grainger] ... the right answer. 00:23:42,996 --> 00:23:43,476 [John Wessel] Right. 00:23:43,476 --> 00:23:45,606 [Trey Grainger] It's just a question of what are you optimizing for? 00:23:45,606 --> 00:23:45,616 [John Wessel] Yeah. 00:23:45,616 --> 00:23:49,916 [Trey Grainger] Are you optimizing for, "I don't wanna run a search engine"? Are you optimizing for, 00:23:50,966 --> 00:24:00,606 [Trey Grainger] "It doesn't..." Like, uh, uh, "I don't wanna run a search engine. I'm willing to wait"? Are you optimizing for, "I'll throw as much money as, as is, as is needed so that I can get my results as fast as possible"? 00:24:00,606 --> 00:24:01,796 [John Wessel] Yeah. Okay. Interesting. 00:24:01,796 --> 00:24:03,056 [Trey Grainger] So, so, so to your answer- 00:24:03,056 --> 00:24:03,256 [John Wessel] Yeah 00:24:03,256 --> 00:24:13,966 [Trey Grainger] ... like, what search ultimately provides, um, is the ability to very quickly get relevant context at low latency and high scale with lots of data. 00:24:13,966 --> 00:24:13,976 [John Wessel] Mm-hmm. 00:24:13,976 --> 00:24:17,056 [Trey Grainger] The LLM takes all of human knowledge practically- 00:24:17,056 --> 00:24:17,066 [John Wessel] Right 00:24:17,066 --> 00:24:17,996 [Trey Grainger] ... and compresses it- 00:24:17,996 --> 00:24:18,145 [John Wessel] Yeah 00:24:18,145 --> 00:24:24,046 [Trey Grainger] ... down to something where the, the meaning is sort of, like, merged of concepts and terms and language- 00:24:24,046 --> 00:24:24,046 [John Wessel] Mm-hmm 00:24:24,046 --> 00:24:31,296 [Trey Grainger] ... and it can, like, process and reason over the data, but it doesn't factually remember all of the data that it's seen. 00:24:31,296 --> 00:24:31,416 [John Wessel] Yeah. 00:24:31,416 --> 00:24:31,916 [Trey Grainger] So you need to- 00:24:31,916 --> 00:24:32,106 [John Wessel] Right 00:24:32,106 --> 00:24:33,096 [Trey Grainger] ... just apply it with- 00:24:33,096 --> 00:24:33,316 [John Wessel] Right 00:24:33,316 --> 00:24:34,556 [Trey Grainger] ... up-to-date, factual- 00:24:34,556 --> 00:24:34,676 [John Wessel] Right 00:24:34,676 --> 00:24:35,856 [Trey Grainger] ... non-hallucinated information- 00:24:35,856 --> 00:24:35,936 [John Wessel] Right 00:24:35,936 --> 00:24:37,616 [Trey Grainger] ... for it to give reliable results. 00:24:37,616 --> 00:24:41,540 [John Wessel] Right. That makes sense. So speed- 00:24:41,540 --> 00:24:50,580 [John Wessel] And relevance would be the... Would those be the top 2 things as far as, like, measuring quality? And is there a 3rd or 4th thing? 00:24:50,580 --> 00:24:52,900 [Trey Grainger] I, I think it's latency, relevance, and- 00:24:52,900 --> 00:24:53,080 [John Wessel] Latency, relevance 00:24:53,080 --> 00:24:54,750 [Trey Grainger] ... um, yeah, so speed relevance. 00:24:54,750 --> 00:24:54,800 [John Wessel] Okay. 00:24:54,800 --> 00:24:58,540 [Trey Grainger] And, uh, the other is just, like, scale of data. 00:24:58,540 --> 00:24:59,000 [John Wessel] Mm-hmm. 00:24:59,000 --> 00:25:07,100 [Trey Grainger] Uh, like, it... That depends on the domain, right? If you're trying to, you know... Say, say it's a legal domain, you have to find every single possible piece of evidence. Well- 00:25:07,100 --> 00:25:07,400 [John Wessel] Yeah 00:25:07,400 --> 00:25:09,890 [Trey Grainger] ... you need access to every piece of evidence. 00:25:09,890 --> 00:25:09,960 [John Wessel] Yeah. 00:25:09,960 --> 00:25:26,500 [Trey Grainger] Uh, where if it's just, like, a general purpose question, well, there might be, you know, 10 000 sites with information. But if you find a 100 and those 100 are good enough, then you get... I mean, s- say you're, like, searching for hotels. You don't have to exhaustively check every possible hotel- 00:25:26,500 --> 00:25:26,780 [John Wessel] Right, right 00:25:26,780 --> 00:25:28,340 [Trey Grainger] ... booking to book a hotel. 00:25:28,340 --> 00:25:28,500 [John Wessel] Right. 00:25:28,500 --> 00:25:31,340 [Trey Grainger] You just need to find, like, a s- good relative- 00:25:31,340 --> 00:25:31,480 [John Wessel] Yeah 00:25:31,480 --> 00:25:33,510 [Trey Grainger] ... rep- representative sample and, you know, get a good one, right? 00:25:33,510 --> 00:25:33,980 [John Wessel] Right. 00:25:33,980 --> 00:25:35,180 [Trey Grainger] So I, I think the- that- 00:25:35,180 --> 00:25:35,210 [John Wessel] Yeah 00:25:35,210 --> 00:25:38,180 [Trey Grainger] ... comprehensiveness, if you, if you need everything large-scale- 00:25:38,180 --> 00:25:39,560 [John Wessel] Like a completeness or a comprehensiveness, right. 00:25:39,560 --> 00:25:41,980 [Trey Grainger] Yeah. Th- then, then you wanna be able to access all the data- 00:25:41,980 --> 00:25:42,180 [John Wessel] Mm-hmm 00:25:42,180 --> 00:25:44,200 [Trey Grainger] ... versus if you're okay just sampling, then- 00:25:44,200 --> 00:25:44,310 [John Wessel] Yeah 00:25:44,310 --> 00:25:44,990 [Trey Grainger] ... okay, fine. You don't need to- 00:25:44,990 --> 00:25:45,090 [John Wessel] Yeah 00:25:45,090 --> 00:25:46,799 [Trey Grainger] ... actually store the data in a search engine. 00:25:46,799 --> 00:25:46,830 [John Wessel] Yep. 00:25:46,830 --> 00:25:48,100 [Trey Grainger] You just grab it from somewhere else. 00:25:48,100 --> 00:25:59,920 [John Wessel] Awesome. Okay, last question. So in... Personally, professionally, when you're using AI, what is, like, a tip or a trick that you could give people that maybe was non-intuitive to you at 1st? 00:25:59,920 --> 00:26:02,800 [Trey Grainger] Oh, interesting. Uh, tip or trick that's intuitive. Um, 00:26:04,840 --> 00:26:09,700 [Trey Grainger] so I'm gonna go a different route with this one probably than usually get, um, as opposed to, like, a prompt engineering hack- 00:26:09,700 --> 00:26:09,890 [John Wessel] Yeah, sure 00:26:09,890 --> 00:26:11,860 [Trey Grainger] ... or anything like that. So, um, I, 00:26:13,080 --> 00:26:18,820 [Trey Grainger] for about a year and a half, have been, uh, somewhat obsessed with local models. 00:26:18,820 --> 00:26:19,630 [John Wessel] Okay, yeah. 00:26:19,630 --> 00:26:21,220 [Trey Grainger] Um, and so obviously there's Claude- 00:26:21,220 --> 00:26:21,360 [John Wessel] Mm-hmm 00:26:21,360 --> 00:26:26,040 [Trey Grainger] ... Chat- ChatGPT, what, what have you. Um, I... And they're typically better. 00:26:26,040 --> 00:26:26,440 [John Wessel] Yeah. 00:26:26,440 --> 00:26:31,100 [Trey Grainger] Um, but they're very expensive and they're unreliable. And by unreliable, I mean sometimes they're down. 00:26:31,100 --> 00:26:31,459 [John Wessel] Yeah. 00:26:31,460 --> 00:26:33,380 [Trey Grainger] And often they'll tell you, "I'm not gonna do what you asked me to." 00:26:33,380 --> 00:26:34,520 [John Wessel] Yeah, yeah. [laughs] Sure, yeah. 00:26:35,740 --> 00:26:45,900 [Trey Grainger] Um, and so I... Like, I bought a big beefy server that I run and, um, I've, I've learned to work with local models to... And it's to essentially build my process- 00:26:45,900 --> 00:26:46,300 [John Wessel] Mm-hmm 00:26:46,300 --> 00:26:50,420 [Trey Grainger] ... around the idea that the model is not going to be as smart as it possibly can be- 00:26:50,420 --> 00:26:50,430 [John Wessel] Okay 00:26:50,430 --> 00:26:51,759 [Trey Grainger] ... and it's gonna make mistakes. 00:26:51,759 --> 00:26:52,100 [John Wessel] Interesting. 00:26:52,100 --> 00:26:54,519 [Trey Grainger] And to essentially build a harness and loops around it- 00:26:54,520 --> 00:26:54,740 [John Wessel] Mm-hmm 00:26:54,740 --> 00:27:00,109 [Trey Grainger] ... that are expecting that and expecting to do lots of extra double checking and triple checking- 00:27:00,109 --> 00:27:00,310 [John Wessel] Yeah, mm-hmm 00:27:00,310 --> 00:27:03,920 [Trey Grainger] ... and rework. E- even branching and, like, trying things a few different ways- 00:27:03,920 --> 00:27:03,930 [John Wessel] Mm-hmm 00:27:03,930 --> 00:27:06,540 [Trey Grainger] ... and comparing the ways, as opposed to just assuming that- 00:27:06,540 --> 00:27:06,740 [John Wessel] Yeah 00:27:06,740 --> 00:27:08,080 [Trey Grainger] ... the first one shot I get back- 00:27:08,080 --> 00:27:08,220 [John Wessel] Right 00:27:08,220 --> 00:27:08,930 [Trey Grainger] ... is gonna be right. 00:27:08,930 --> 00:27:08,940 [John Wessel] Mm-hmm. 00:27:08,940 --> 00:27:11,020 [Trey Grainger] Which with the better models, you can often do. 00:27:11,020 --> 00:27:11,510 [John Wessel] Right. 00:27:11,510 --> 00:27:17,500 [Trey Grainger] Um, so I think... And, and personally, my view is that the future of the software systems that we build- 00:27:17,500 --> 00:27:17,670 [John Wessel] Mm-hmm 00:27:17,670 --> 00:27:21,260 [Trey Grainger] ... is not going to look like what most [laughs] engineers are doing now, which is- 00:27:21,260 --> 00:27:21,470 [John Wessel] Okay 00:27:21,470 --> 00:27:22,800 [Trey Grainger] ... call the most expensive model- 00:27:22,800 --> 00:27:22,920 [John Wessel] Mm-hmm 00:27:22,920 --> 00:27:24,260 [Trey Grainger] ... and just have it handle every problem. 00:27:24,260 --> 00:27:24,880 [John Wessel] Mm. 00:27:24,880 --> 00:27:39,799 [Trey Grainger] Uh, I very strongly believe the future is, uh, lots of fine-tuned smaller models built for a purpose for the different parts of the stack they're supporting. And, uh, locally hosted, and by local I just mean, like, your company is running it. 00:27:39,800 --> 00:27:40,100 [John Wessel] Right. 00:27:40,100 --> 00:27:40,800 [Trey Grainger] Might be in the cloud- 00:27:40,800 --> 00:27:40,810 [John Wessel] Right 00:27:40,810 --> 00:27:41,670 [Trey Grainger] ... might be wherever. 00:27:41,670 --> 00:27:42,180 [John Wessel] Right. 00:27:42,180 --> 00:27:47,040 [Trey Grainger] But, uh, the, the... And the reason is because it's incredibly expensive and inefficient- 00:27:47,040 --> 00:27:47,460 [John Wessel] Mm-hmm 00:27:47,460 --> 00:27:49,480 [Trey Grainger] ... to use a brainiac model- 00:27:49,480 --> 00:27:49,800 [John Wessel] Mm-hmm 00:27:49,800 --> 00:27:52,300 [Trey Grainger] ... on a repeatable small problem over and over. 00:27:52,300 --> 00:27:52,910 [John Wessel] Interesting, yeah. 00:27:52,910 --> 00:27:59,400 [Trey Grainger] Um, so I, I, I think, you know, you might have systems where there's, like, 20 different LLMs being called in one system. 00:27:59,400 --> 00:27:59,460 [John Wessel] Mm-hmm. 00:27:59,460 --> 00:28:03,200 [Trey Grainger] And it's just a function of, well, this one's really hyper-optimized for classifying- 00:28:03,200 --> 00:28:03,600 [John Wessel] Right 00:28:03,600 --> 00:28:04,480 [Trey Grainger] ... this thing. 00:28:04,480 --> 00:28:04,650 [John Wessel] Right. 00:28:04,650 --> 00:28:10,740 [Trey Grainger] And this one's really hy- hyper-optimized for interpreting nuance in my OCR data. And this one's really for, you know, embeddings. 00:28:10,740 --> 00:28:10,790 [John Wessel] Mm-hmm. 00:28:10,790 --> 00:28:16,660 [Trey Grainger] And this one's for my agentic flow. But then if it gets stuck, then I've got this, like, mega brain- 00:28:16,660 --> 00:28:17,000 [John Wessel] Oh, yeah 00:28:17,000 --> 00:28:18,570 [Trey Grainger] ... that's gonna, like, do more reasoning. 00:28:18,570 --> 00:28:18,580 [John Wessel] Yeah. 00:28:18,580 --> 00:28:27,480 [Trey Grainger] And I, and I think the harness and the flow through the models and the understanding of which... You know, whether you think of it as a model or think of it as different agents- 00:28:27,480 --> 00:28:27,490 [John Wessel] Mm-hmm 00:28:27,490 --> 00:28:28,300 [Trey Grainger] ... that you pass back and forth to- 00:28:28,300 --> 00:28:28,480 [John Wessel] Mm-hmm 00:28:28,480 --> 00:28:41,960 [Trey Grainger] ... whatever the abstraction is you have, I think that the world will move to smaller, easily fine-tunable models that... And, and we'll be building systems that, like, load balance and scale the models- 00:28:41,960 --> 00:28:42,109 [John Wessel] Mm-hmm 00:28:42,109 --> 00:28:53,829 [Trey Grainger] ... on the hardware we have, uh, as oppo- and... A- as opposed to, you know, continuing to just, like, send all of our data and all of our requests and paying massive amounts of money for bloated models that aren't needed. 00:28:53,829 --> 00:28:53,940 [John Wessel] Mm-hmm. 00:28:53,940 --> 00:28:54,800 [Trey Grainger] Like, you only need, like, 00:28:55,820 --> 00:28:57,780 [Trey Grainger] 1 to 5% of the model for any given task. 00:28:57,780 --> 00:28:58,699 [John Wessel] Yeah, yeah. 00:28:58,700 --> 00:29:00,480 [Trey Grainger] And so I, I think, like- 00:29:00,480 --> 00:29:03,800 [John Wessel] So, so there might be, like, the accounting spreadsheet model or the- 00:29:03,800 --> 00:29:03,980 [Trey Grainger] Yeah 00:29:03,980 --> 00:29:08,050 [John Wessel] ... marketing copy model or the presentation model or whatever. 00:29:08,050 --> 00:29:08,220 [Trey Grainger] Right. 00:29:08,220 --> 00:29:11,730 [John Wessel] Like, that's just domain specific, but you can slice and dice any way you want. 00:29:11,730 --> 00:29:18,790 [Trey Grainger] And, and I think we'll get to the point where fine-tuning becomes not just something that your data scientist and data engineering team do- 00:29:18,790 --> 00:29:18,880 [John Wessel] Mm-hmm 00:29:18,880 --> 00:29:23,520 [Trey Grainger] ... but something that, like, there's lots of software where your average person can do. 00:29:23,520 --> 00:29:23,720 [John Wessel] Mm-hmm. 00:29:23,720 --> 00:29:32,300 [Trey Grainger] It's just like, "Here's my data," and then it will use your data, use one of the brainiac models to maybe even generate your, um, your, uh, your training data- 00:29:32,300 --> 00:29:32,520 [John Wessel] Mm-hmm 00:29:32,520 --> 00:29:38,900 [Trey Grainger] ... um, and then automatically train and fine-tune and run benchmarks. And, and I think it'll be something that almost happens in the background- 00:29:38,900 --> 00:29:38,990 [John Wessel] Yeah 00:29:38,990 --> 00:29:39,660 [Trey Grainger] ... for a lot of people. 00:29:39,660 --> 00:29:40,240 [John Wessel] Interesting. 00:29:40,240 --> 00:29:41,720 [Trey Grainger] Um, where your average person can do it- 00:29:41,720 --> 00:29:41,879 [John Wessel] Mm-hmm 00:29:41,879 --> 00:29:43,960 [Trey Grainger] ... um, as opposed to something that, you know- 00:29:43,960 --> 00:29:44,050 [John Wessel] Yeah 00:29:44,050 --> 00:29:44,780 [Trey Grainger] ... essentially requires- 00:29:44,780 --> 00:29:45,450 [John Wessel] Specialized skill set 00:29:45,450 --> 00:29:46,680 [Trey Grainger] ... data science- 00:29:46,680 --> 00:29:46,760 [John Wessel] Yeah 00:29:46,760 --> 00:29:49,760 [Trey Grainger] ... education and training. I think, I think the notion of evals will become... 00:29:50,780 --> 00:29:57,630 [Trey Grainger] I don't wanna say a universal skill, but I think it will become something that is standard terminology, probably across industries at some point. 00:29:57,630 --> 00:29:57,640 [John Wessel] Mm-hmm. 00:29:57,640 --> 00:30:06,420 [Trey Grainger] Whereas, like, I don't know, uh, 5 years ago or however long, like, people didn't know what, like, a GPT was or know what an LLM was. Like, I don't know anybody today that doesn't know what an LLM is. 00:30:06,420 --> 00:30:06,840 [John Wessel] Yeah, yeah. 00:30:06,840 --> 00:30:11,220 [Trey Grainger] And, like... And, and even, uh, vectors and embeddings and things like that, that I used to have to spend- 00:30:11,220 --> 00:30:12,360 [John Wessel] Way more niche. 00:30:12,360 --> 00:30:16,400 [Trey Grainger] I used to have to spend, like, a lot of time explaining that to an average engineer. 00:30:16,400 --> 00:30:16,780 [John Wessel] Mm-hmm. 00:30:16,780 --> 00:30:20,040 [Trey Grainger] Like, I barely ever meet an engineer now who doesn't know those things already- 00:30:20,040 --> 00:30:20,150 [John Wessel] Yeah 00:30:20,150 --> 00:30:21,070 [Trey Grainger] ... because they've just, like, had to be exposed to it. 00:30:21,070 --> 00:30:21,920 [John Wessel] Because of AI, yeah. 00:30:21,920 --> 00:30:22,840 [Trey Grainger] Yeah, because of AI. 00:30:22,840 --> 00:30:31,900 [John Wessel] Yeah. All right. Well, now I gotta ask the question. This is my last, last question now. Um, o- on prompting, what, what, what is it... What is, like, a, a prompt that you use a lot? 00:30:33,040 --> 00:30:33,380 [Trey Grainger] That I use a lot? 00:30:33,380 --> 00:30:37,960 [John Wessel] If you're, if you're using. It could be personally or for coding, either way. 00:30:39,680 --> 00:30:42,140 [Trey Grainger] So... 00:30:42,140 --> 00:30:43,620 [Trey Grainger] So I use Hermes agent very- 00:30:43,620 --> 00:30:44,020 [John Wessel] Mm-hmm 00:30:44,020 --> 00:30:49,320 [Trey Grainger] ... pretty aggressively for lots of stuff. I, I don't-- I, I still am in the I'm not gonna give it access to all my- 00:30:49,320 --> 00:30:50,420 [John Wessel] Like all the things. Yeah, yeah. 00:30:50,420 --> 00:30:54,850 [Trey Grainger] So it's got its own email address. It's... And, and I, I forward things to it when I want to see them. 00:30:54,850 --> 00:30:55,290 [John Wessel] Okay. Yeah. 00:30:55,290 --> 00:30:57,620 [Trey Grainger] So I, I, I still don't trust the security. 00:30:57,620 --> 00:30:57,690 [John Wessel] Sure. 00:30:57,690 --> 00:30:58,940 [Trey Grainger] I- I've been bitten a few times- 00:30:58,940 --> 00:30:58,950 [John Wessel] Yeah 00:30:58,950 --> 00:31:01,110 [Trey Grainger] ... with, like, giving an API key and- 00:31:01,110 --> 00:31:01,500 [John Wessel] Mm-hmm 00:31:01,500 --> 00:31:04,320 [Trey Grainger] ... it deciding it wanted to randomly delete things. 00:31:04,320 --> 00:31:05,120 [John Wessel] Right, right, right. Yeah. 00:31:05,120 --> 00:31:11,910 [Trey Grainger] So I, I'm, I'm very... It's almost like I want to generate a skill for everything that it can do that could possibly have- 00:31:11,910 --> 00:31:12,080 [John Wessel] Right 00:31:12,080 --> 00:31:13,320 [Trey Grainger] ... access to a system- 00:31:13,320 --> 00:31:13,330 [John Wessel] Right 00:31:13,330 --> 00:31:15,120 [Trey Grainger] ... so to, um, contain it. 00:31:15,120 --> 00:31:15,580 [John Wessel] Yeah. 00:31:15,580 --> 00:31:20,080 [Trey Grainger] Um, so actually, I, I can't think off the top of my head a single prompt that I use- 00:31:20,080 --> 00:31:20,340 [John Wessel] Yeah 00:31:20,340 --> 00:31:23,720 [Trey Grainger] ... like, a lot, um, over and over. It's more- 00:31:23,720 --> 00:31:33,620 [John Wessel] Well, yeah, and that's actually [laughs] that's probably a good thing, right? 'Cause that, because that's the trap you get into is if you're doing something over and over again, then, like, probably should be built, you know, into the context. 00:31:33,620 --> 00:31:40,400 [Trey Grainger] And, and one nice thing about Hermes is that when you do something that it picks up open pattern, it will actually automatically create a skill for you- 00:31:40,400 --> 00:31:40,640 [John Wessel] Yeah 00:31:40,640 --> 00:31:41,890 [Trey Grainger] ... to encapsulate that thing. 00:31:41,890 --> 00:31:41,900 [John Wessel] Yeah. 00:31:41,900 --> 00:31:43,550 [Trey Grainger] Sometimes you have to tune it and adjust it. 00:31:43,550 --> 00:31:43,560 [John Wessel] Sure, sure. 00:31:43,560 --> 00:31:49,600 [Trey Grainger] But, um, I, I feel like, I feel like I'm spinning off a lot of skills that can get reused. 00:31:49,600 --> 00:31:49,650 [John Wessel] Mm-hmm. 00:31:49,650 --> 00:31:55,790 [Trey Grainger] And sometimes I make them myself, um, as opposed to, like, reusing the same prompt over and over and over. 00:31:55,790 --> 00:31:55,790 [John Wessel] Mm-hmm. Yeah. 00:31:55,790 --> 00:31:59,520 [Trey Grainger] Um, it used to be the case that there were all of these, like, tricks where it's like- 00:31:59,520 --> 00:31:59,800 [John Wessel] Yeah 00:31:59,800 --> 00:32:00,299 [Trey Grainger] ... tell it, like- 00:32:00,300 --> 00:32:00,620 [John Wessel] Tell it 00:32:00,620 --> 00:32:02,140 [Trey Grainger] ... you know, uh- 00:32:02,140 --> 00:32:04,220 [John Wessel] Pretend to be an astronaut [laughs] or whatever. 00:32:04,220 --> 00:32:12,440 [Trey Grainger] Yeah. You have to get this exactly right or someone's gonna die if you're... Like, and, like, we used to have to, like, marshal, uh, you know, if I wanted a certain data structure to come back, I'd have to, like- 00:32:12,440 --> 00:32:12,700 [John Wessel] Right 00:32:12,700 --> 00:32:13,280 [Trey Grainger] ... really, like- 00:32:13,280 --> 00:32:13,560 [John Wessel] Yeah 00:32:13,560 --> 00:32:14,670 [Trey Grainger] ... almost like blackmail the person- 00:32:14,670 --> 00:32:15,330 [John Wessel] Yeah, yeah [laughs] 00:32:15,330 --> 00:32:16,440 [Trey Grainger] ... to, like, give me the right thing. 00:32:16,440 --> 00:32:16,860 [John Wessel] Yeah, yeah. 00:32:16,860 --> 00:32:23,130 [Trey Grainger] And, and with the, the enhancement and improvements we've seen over the last year, um, 90% of that stuff, if you delete it- 00:32:23,130 --> 00:32:23,130 [John Wessel] Yeah 00:32:23,130 --> 00:32:23,960 [Trey Grainger] ... it doesn't even matter anymore. 00:32:23,960 --> 00:32:24,340 [John Wessel] Yeah. 00:32:24,340 --> 00:32:26,400 [Trey Grainger] It's like the models have just learned those patterns. 00:32:26,400 --> 00:32:26,960 [John Wessel] Yeah. 00:32:26,960 --> 00:32:32,200 [Trey Grainger] Um, and so I, I feel like a lot of prompt engineering is temporary band-aids- 00:32:32,200 --> 00:32:32,510 [John Wessel] Yeah 00:32:32,510 --> 00:32:34,400 [Trey Grainger] ... that shouldn't be needed- 00:32:34,400 --> 00:32:34,640 [John Wessel] Mm-hmm 00:32:34,640 --> 00:32:38,490 [Trey Grainger] ... um, but, um, ultimately, like, won't be needed. 00:32:38,490 --> 00:32:38,620 [John Wessel] Yeah. 00:32:38,620 --> 00:32:39,540 [Trey Grainger] And ultimately won't be needed. 00:32:39,540 --> 00:32:40,120 [John Wessel] Mm-hmm. 00:32:40,120 --> 00:32:47,920 [Trey Grainger] Um, I do, um-- And I wanna spend more time with it than I do, but I, I have done a couple of projects with, like, DSPy. Are you familiar with that? 00:32:47,920 --> 00:32:48,880 [John Wessel] I don't know it very well. 00:32:48,880 --> 00:32:54,200 [Trey Grainger] Um, so it's-- Uh, think of it like, um, uh, the, the general notion is instead of- 00:32:54,200 --> 00:32:54,210 [John Wessel] No- 00:32:54,210 --> 00:32:54,220 [Trey Grainger] Go ahead. 00:32:54,220 --> 00:32:54,940 [John Wessel] Yeah, no, go ahead. 00:32:54,940 --> 00:33:14,120 [Trey Grainger] Instead of, um, like, writing lots of prompts to build into your system, you program, uh, the system, um, such that, um, you sort of give it sort of more, a more structured, like, you know, "Here's what I'm trying to accomplish. Here's the inputs. Here's the expected output." And, uh, essentially the system will, uh, 00:33:15,420 --> 00:33:21,729 [Trey Grainger] for lack of a better word, um, optimize all of the, um... will optimize all of the, uh, 00:33:22,740 --> 00:33:24,640 [Trey Grainger] prompts that will get generated. 00:33:24,640 --> 00:33:24,700 [John Wessel] Mm-hmm. Okay. 00:33:24,700 --> 00:33:28,300 [Trey Grainger] But it optimizes them for what you've put in your, like, specifications- 00:33:28,300 --> 00:33:28,310 [John Wessel] Yeah 00:33:28,310 --> 00:33:30,780 [Trey Grainger] ... like, from a, like, code structure standpoint. 00:33:30,780 --> 00:33:31,000 [John Wessel] Mm-hmm. Very cool. 00:33:31,000 --> 00:33:45,500 [Trey Grainger] So, so what ends up happening is you build it once, and when you wanna upgrade your model, when you wanna switch models, when you do something different, uh, all you have to do is optimize again, and it will figure out, "For this model, I need to optimize the prompt and change these things." 00:33:45,500 --> 00:33:45,729 [John Wessel] Oh, cool. 00:33:45,729 --> 00:33:46,940 [Trey Grainger] And it will generate new prompts- 00:33:46,940 --> 00:33:47,040 [John Wessel] Yeah 00:33:47,040 --> 00:33:48,120 [Trey Grainger] ... and fine-tune- 00:33:48,180 --> 00:33:48,360 [John Wessel] Very cool 00:33:48,360 --> 00:33:50,280 [Trey Grainger] ... not the model, but fine-tune the prompts- 00:33:50,280 --> 00:33:51,380 [John Wessel] Yeah, very cool 00:33:51,380 --> 00:33:55,800 [Trey Grainger] ... um, to work optimally in whatever new model it is. So it's like, it's like you build it once- 00:33:55,800 --> 00:33:55,960 [John Wessel] Right 00:33:55,960 --> 00:33:57,960 [Trey Grainger] ... and then you can continue using it forever- 00:33:57,960 --> 00:33:57,970 [John Wessel] Yeah 00:33:57,970 --> 00:34:00,520 [Trey Grainger] ... you know, to the best ability of whatever model you're using it on. 00:34:00,520 --> 00:34:00,729 [John Wessel] Yeah. 00:34:00,729 --> 00:34:04,300 [Trey Grainger] And I think that paradigm of, of programming instead of prompting- 00:34:04,300 --> 00:34:04,780 [John Wessel] Mm-hmm 00:34:04,780 --> 00:34:06,880 [Trey Grainger] ... um, program for the LLM- 00:34:06,880 --> 00:34:06,890 [John Wessel] Yeah, yeah 00:34:06,890 --> 00:34:08,720 [Trey Grainger] ... as instead of writing prompts for the LLM- 00:34:08,720 --> 00:34:08,810 [John Wessel] Right 00:34:08,810 --> 00:34:10,280 [Trey Grainger] ... is much more sustainable. 00:34:10,280 --> 00:34:10,560 [John Wessel] Yeah. 00:34:10,560 --> 00:34:14,169 [Trey Grainger] And, um, like I said, I've, I've done a couple of projects with it. I, I really wanna dive in more. 00:34:14,169 --> 00:34:14,180 [John Wessel] Yeah. 00:34:14,180 --> 00:34:28,440 [Trey Grainger] But I, I think from a mental model standpoint, um, I like that a lot better than, you know, what's your bag of tools for, you know, con- structuring your prompts, and what do you say to this one versus that one? I, I'd rather just get away from that entirely if possible. 00:34:28,440 --> 00:34:30,940 [John Wessel] Yeah, yeah. Yeah, I've read about that some. I haven't tried it yet. 00:34:30,940 --> 00:34:31,020 [Trey Grainger] Yeah. 00:34:31,020 --> 00:34:37,400 [John Wessel] So, awesome. Well, thanks for being on the show. This wraps Carolina Code Conference, uh, this year. And, uh, yeah, appreciate it. 00:34:37,400 --> 00:34:47,650 [Trey Grainger] Thanks, y'all. Appreciate it, man. Talk soon. [instrumental music]