Skip to content

Update 417-tdd-prompt-engineering.txt #103

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 30, 2023
Merged

Update 417-tdd-prompt-engineering.txt #103

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 15 additions & 16 deletions transcripts/417-tdd-prompt-engineering.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@

00:00:14 And when doing that, the slight unpredictable nature and probabilistic style of LLMs makes writing and testing Python code very tricky.

00:00:23 Interpromptimize, from Maxine Bocheman, and Preset.
00:00:23 Interpromptimize, from Maxime Beauchemin, and Preset.

00:00:28 It's a framework for non-deterministic testing of LLMs inside of our applications.

00:00:33 Let's dive inside the AIs with Max. This is Talk Python To Me, episode 417, recorded May 22nd, 2023.

00:00:54 Welcome to Talk Python.me, a weekly podcast on Python. This is your host, Michael Kennedy.
00:00:54 Welcome to Talk Python to me, a weekly podcast on Python. This is your host, Michael Kennedy.

00:00:59 Follow me on Mastodon, where I'm @mkennedy, and follow the podcast using @talkpython, both on fosstodon.org. Be careful with impersonating accounts on other instances, there are many. Keep up with the show and listen to over seven years of past episodes at talkpython.fm. We've started streaming most of our episodes live on YouTube. Subscribe to our YouTube channel over at talkpython.fm/youtube to get notified about upcoming shows and be part of that episode.

Expand All @@ -20,7 +20,7 @@

00:01:41 And it's brought to you by The Compiler Podcast from Red Hat.

00:01:45 to an episode of their podcast to demystify the tech industry over at talkbython.fm/compiler.
00:01:45 to an episode of their podcast to demystify the tech industry over at talkpython.fm/compiler.

00:01:53 Max welcome to Talk Python to Me.

Expand Down Expand Up @@ -92,7 +92,7 @@

00:04:11 So it's a playground for people trying to understand and visualize and explore data.

00:04:16 So I started working on Apache SuperSend in 2000, I think it was like 15 or 16 at Airbnb too.
00:04:16 So I started working on Apache SuperSet in 2000, I think it was like 15 or 16 at Airbnb too.

00:04:21 And we also brought that to the Apache Software Foundation.

Expand Down Expand Up @@ -294,7 +294,7 @@

00:13:39 Other examples are simply AI just released Lemur, which is a large language model, but really focused on transcribing speech, which I think is kind of cool.

00:13:49 Microsoft reduced Microsoft security, released Microsoft security copilot, which is a large language model to talk about things like Nginx misconfiguration and stuff like that.
00:13:49 Microsoft released Microsoft security, released Microsoft security copilot, which is a large language model to talk about things like Nginx misconfiguration and stuff like that.

00:14:00 There's just a lot of stuff out there that's coming along here, right?

Expand All @@ -306,7 +306,7 @@

00:14:18 We probably shouldn't get into the ethics part of the debate here, 'cause that's a whole series of episodes we probably won't wanna get into.

00:14:26 But what's interesting is Databricks came up a model for what's called Facebook, came up with one called Lama, and they open sourced and/or leaked the weights, so you have the model topology with the pre-trained weights.
00:14:26 But what's interesting is Databricks came up a model for what's called Facebook, came up with one called LLama, and they open sourced and/or leaked the weights, so you have the model topology with the pre-trained weights.

00:14:39 In some cases, there's open source corpus of training that are also coming out and are also open sourced.

Expand Down Expand Up @@ -348,7 +348,7 @@

00:16:48 If you see your editor completing statements just because the word appears elsewhere in the file, but it's not actually relevant to that code block, that should make you really nervous.

00:16:57 I've been a happy paying customer of PyCharm for years. Hardly a workday passes that I'm not deep inside PyCharm working on projects here at Talk Python. What tool is more important to your productivity than your code editor? You deserve one that works the best. So download your free trial of PyCharm professional today at talkpython.fm/donewithpycharm and get work done. That link is in your podcast player show notes. Thank you to PyCharm from JetBrains for sponsoring the show and keeping Talk Python going strong.
00:16:57 I've been a happy paying customer of PyCharm for years. Hardly a workday passes that I'm not deep inside PyCharm working on projects here at Talk Python. What tool is more important to your productivity than your code editor? You deserve one that works the best. So download your free trial of PyCharm professional today at talkpython.fm/done-with-pycharm and get work done. That link is in your podcast player show notes. Thank you to PyCharm from JetBrains for sponsoring the show and keeping Talk Python going strong.

00:17:31 Right. It's interesting to ask questions, right? But it's more interesting from a software developer perspective of can I teach it a little bit more about what my app needs to know or what my app structure is, right?

Expand Down Expand Up @@ -596,7 +596,7 @@

00:31:21 But I think for all these examples that I've seen, I'm really tempted to take the prompt that they had and then give it an instruction or two or more and then figure out how to get it to come up with the right thing.

00:31:31 super important skill.
00:31:31 prompt crafting super important skill.

00:31:33 You know, you could probably get a boost of for most knowledge information workers, you'll get a boost of 50% to 10x for a lot of the tasks you do every day if you use AI well. So it's great personal skill to have go and develop that skill if you don't.

Expand Down Expand Up @@ -750,7 +750,7 @@

00:38:41 So that's not great.

00:38:42 - Yeah, train it on 4chan or let it, you know, go crawl 4chan and read it.
00:38:42 - Yeah, train it on Fortron or let it, you know, go crawl 4chan and read it.

00:38:48 It's not always gonna be nice.

Expand Down Expand Up @@ -1088,7 +1088,7 @@

00:55:07 So that it's the same philosophy behind, you know, Airflow of like, you know, it's almost like a little DSL to write your test case.

00:55:15 So you could, you know, it could read from a YAML file for instance, in the case of what we do with spider SQL, there's a big Jason file of all the prompts and all the databases.
00:55:15 So you could, you know, it could read from a YAML file for instance, in the case of what we do with spider SQL, there's a big Json file of all the prompts and all the databases.

00:55:23 And then we dynamically generate, you know, a thousand tests based on that.

Expand Down Expand Up @@ -1248,9 +1248,9 @@

01:02:22 So I was like, I wish I could just say, just go read the GitHub, just read it all, read the docs, and then I'll ask you questions.

01:02:30 And then Chajupiti's not that great at that currently, at learning things it doesn't know, for reasons we talked about.
01:02:30 And then Chat GPT not that great at that currently, at learning things it doesn't know, for reasons we talked about.

01:02:38 BART is much more up to date, so you can always, for those projects, Chajupiti might be better at Django, 'cause it's old and settled, and it's better at writing code overall, but Bard might be decent and pretty good for--
01:02:38 BART is much more up to date, so you can always, for those projects, Chat GPT might be better at Django, 'cause it's old and settled, and it's better at writing code overall, but Bard might be decent and pretty good for--

01:02:52 - Right, if you ask advice on how to do promptimize stuff, it's like, I don't know what that is.

Expand All @@ -1262,7 +1262,7 @@

01:03:21 But yeah, it might be better too for those familiar with the idea of functional programming, where each function is more deterministic and can be reasoned about and unit test in an isolation.

01:03:32 Chai GPT is gonna be better at that 'cause it doesn't know about all your other packages and modules.
01:03:32 Chat GPT is gonna be better at that 'cause it doesn't know about all your other packages and modules.

01:03:37 So really great for the utils functions are very deterministic, functional, super great at that.

Expand Down Expand Up @@ -1392,7 +1392,7 @@

01:08:32 - Yeah, it's really quite amazing.

01:08:34 It went through and said, okay, here's a function, and it knows talkbython.evm/episodes/all.
01:08:34 It went through and said, okay, here's a function, and it knows talkpython.fm/episodes/all.

01:08:39 Use h, get the title, and let's just finish this out, Max.

Expand Down Expand Up @@ -1522,7 +1522,7 @@

01:12:52 PyCharm Professional understands complex projects across multiple languages and technologies, so you can stay productive while you're writing Python code and other code like HTML or SQL.

01:13:04 Download your free trial at talkpython.fm/donewithpycharm.
01:13:04 Download your free trial at talkpython.fm/done-with-pycharm.

01:13:09 Listen to an episode of Compiler, an original podcast from Red Hat.

Expand Down Expand Up @@ -1565,4 +1565,3 @@
01:14:32 (upbeat music)

01:14:35 [BLANK_AUDIO]