What's New in AI for Web Developers

Treats out there. Today we are going to do, an update show. Maybe 3 or 4 months ago, maybe 3 months ago, we did a couple shows on AI machine learning you as a developer, how you can use all these types of things.

And like everything In web development, stuff is moving super fast. So there's a whole hoard of new things that were announced, rid. New tools, new abilities.

So we're going to go through those as well as explaining what these things mean as well. Because if I say like, oh, yeah, there's new vector databases and embeddings, you might not necessarily know what that is. So we're going to explain, like, how you might use an embedding and what they're for, as well as the new tech that we can use to search and store those values.

Rid With me as always, mister Scott Talinski. How are you doing, Scott? Hey.

I'm doing pretty good. I was painting this weekend, which I didn't give myself enough time. I didn't buy enough paint. The classic issues that you do when you're painting paint. Life. Yep. Yep. And I had my, you know, 6 year old helping me paint.

So that was a lot of fun because that means the job was extra messy and, difficult and took a lot longer. But it was a a ton of fun.

However, I did run into a situation that Sentry couldn't fix, Wes.

I dropped a can of paint at Home Depot and it flew up everywhere.

No. No way.

It Splashed all it splashed all over my legs and my shoes, and I'm, like, walking to my car covered in paint. I obviously got somebody. Rid it was it was just a fumble.

It was just like a, woah, and everything went to slow motion. And Next thing you know, paint hits the ground and blows up.

Oh,

that sucks. It's so embarrassing. It was crowded. It was a crowded ready. Sunday at Home Depot, and I got some some older older gentleman to, who worked at Home Depot, and I told him, and you could just see the look of disgust on his face like this guy.

And then I It's impossible.

I that guy disappeared, and then I told some younger employee who is, like, way cooler about it. And he was just like, man. It happens just about every day, so you're okay.

Really? He said it happens all the time. I was like, I'm so embarrassed. I'm so sorry. I feel really awful about this. Yeah. It was, yeah. My wife my wife was saying, hey. Can you I was I'm like I'm like I had just gotten in my car To, like, leave Home Depot, and my wife text me, hey. Can you go pick up this and this and this from Home Depot? I was like, oh, Court. I am not rid. Going to go back in there. I am, very embarrassed. I want to get out as fast as possible.

Oh, that's my nightmare. Actually, my nightmare is it happening in the car. You know? Like, once I just threw up a can of paint and it fell over, and luckily, it didn't open. But

that's that would be a bad day. Yeah. Well, you know what? Century can fix is Your code issues and your code problems.

So if you have errors and exceptions in your code, Sentry can be there to let you know where they are, and they can help you find them. And they even have some really neat tools like an AI assistant to really help get in there and potentially give you a solution.

And that's really fitting for this episode that we're talking about is these AI tools here today.

So, Wes, I think the the big I don't know if you'd call it an elephant in the room, but the big Thing in the room nowadays for developers in AI tools is this brand new tool that People are really slowly starting to get access to, which is vzero.dev from Vercel.

Rid. Do you have access to vZero?

Not yet.

I've I've signed up for it and I'm patiently waiting, but I am very excited to get access to it. And the idea with vzero is it's from Vercel and it takes a rid. Prompt input, so just text and will generate you UI components.

From that specific prompt. So out the other end, it's going to give you readable copy pasteable, react code that is based on, How do you say this? Shad cn or? Yeah.

Shad cn UI and Tailwind CSS That you can copy paste into your project. So basically, like, okay, imagine like what are some of the UI pieces on the news syntax, right? Like a player, A list of guests. Each guest has a profile photo, a Twitter link, a GitHub link and a tag, whether they are a guest or a host, right? Or a menu bar with a bunch of icons for saving and undoing and redoing values. If you think about all that type of stuff, You can pretty quickly code it yourself. Well, maybe not pretty quickly. Or you could just type what you want in And get I like that they named it v0. You get the 1st version of that and copy paste that, throw that into your your application. You're up and running. So this is pretty, Pretty interesting. I had not seen really, not anything like this, but, like, I think we're seeing a lot of, like, AI websites.

But this is the first thing I saw, which is like AI React code.

Here, give me this React and Tailwind CSS code rid. To do the following, and I don't have access to it yet, but they do show you the outputs So you can see what people have been generating over time. And it's it's impressive as much as we've become numb to how amazing AI has In the last little while.

I'm pretty impressed with it, so I think this is going to be a Awesome tool for quickly building out a custom UI components, but very quickly

for version 0. Yeah. Yeah. And this is maybe the I I you know, it is such a limited thing, and I understand why it's limited in terms of it needs to be tailwinded. It it's rid components in its React. Right? I get it.

But it is the first thing that I think I've seen in this space Where it has made me go, oh, boy. You know? This is this is, rid This is a, you know, a big step towards eliminating the developer from the process of things. You know? Yeah. It makes me a little nervous if I'm being entirely honest. I I think it's fantastic, and granted it frees, you know, people like me up to solve Different and bigger problems than coding the same UIs over and over again, and that aspect of it is lovely and wonderful.

But I gotta say, as somebody who loves to do their craft, you know, it does make me a little nervous.

Yeah. It's And we'll talk maybe about more about this at the end is like, are our jobs still at risk? How do we feel a couple of months later after this type of thing? But It's also really interesting seeing like you can go back through the prompts.

You know, you click on the first one. It This is a toolbar that is pill shaped and dark, right? And then you go to V2, and then you say separate each section with a vertical border.

Rid. And the next one, change the gear icon to a menu icon, and you see over rid. Forty prompts, you see the end output. I'm like, that's what coding is. You know, you build it. Okay. Let me change this icon. Let me Make it smaller. Let me put vertical bars between the each right and, it's almost as if English is the the DSL here, right? You'll hear that word a lot. DSL domain specific language, right? It's the syntax for programming something. CSS is a DSL, so maybe English is the next DSL.

Ready. Yeah, I don't don't pooh pooh this because it's a specific tech stack. Like I guarantee that this could be expanded to many other tech stacks as well.

Yeah. And and and when I when I said there was constraint, I didn't I didn't mean to pooh pooh that because I do think people are Not at all. Yeah. Yeah.

Rid. I I I yeah. It is. Again, it is one of those things that and it does kinda make me a little nervous that, you know, hey. This is a This is a tool doing a lot of work. But, again, you know, it makes us so we don't have to solve the same problems over and over and over again. Right? Like, how many times have you built a data table? How many times have you built tab system? I know you just built 1 yesterday.

So, like Yeah.

Over and over and over again is really how it goes. So I'm excited to get access to this. I hadn't even requested access to it Because, honestly, I'm not going to be using it professionally right now because I'd I'm not using those tools. So it's it's kinda pointless for me to have access rid. To it other than just to go, oh, you know, let me see what it can do. So in that regard, I've just been following along what other people are doing with it and am very, very impressed. Next we have here is Anthropic

and Claude.

So Anthropic is another company that is similar to what OpenAI rid is at least in my mind.

And they have a chat product which is called Claude.

And then they have In API access, which has they have 2 different versions of their API. 1 is fast and Not as good or as accurate. Maybe. I don't know what the highest quality. And then the other one is a little bit slower and even higher quality.

Rid man, man, let me tell you, it is fantastic. I don't know if I'm just getting tired of rid the way OpenAI phrases things. But even for the Syntax website, I found myself battling The tone that it used. Right? And I know you can you can give it tone and and train it and whatnot, but Even something as simple as, like, saying stop using emojis in the generated tweets. Just please. It was it was really tricky to do. Stop using hashtags.

Stop using emojis.

Stop using so many exclamation points. Like, that's not how people talk. Yeah.

Exactly. And And maybe it was at the time, but I feel like the anthropic quality is So much better. And I've been using the Claude chat recently just for my own personal stuff. And so the Claude Chat is now open to everybody. You can get into the beta. The API access, I finally got access to it after about 3 months of of trying to Apply for it in email and whatnot, and I have been playing around with it and it is fantastic. So the biggest, Let alone the quality and the tone and whatnot that it has. The biggest thing that Anthropic has is a 100,000 token limit. So let's let's talk about what tokens are right now.

When you send queries to OpenAI or any of these, machine learning models, you are only allowed to send and get back a certain amount of text. And the way that they measure what that text is is via tokens.

So if I say, Hello, my name is Wes, Every single one of those words is counted as a token. But if I put a period in there, a period is a token, a quote is usually a token.

And ChatJPT and OpenAI APIs have limitations on how many tokens you are allowed to read. Send and how expensive it is per token, right? So the 2 big models in OpenAI, which is the models are basically things that have been trained to work on it. So we had GPT 3.5 and then we got GPT 4 and GPT 4 is a lot more expensive And the token limit is currently 8,000.

And then GPT 3.5, which is still Pretty good in most use cases can take up to 16,000 tokens. So to give you context, the entire Transcript of an hour long podcast between Scott and I is about 16,000 tokens, Meaning that you could talk for an hour, transcribe every word and send that to some AI, and that will be 16,000 tokens. But the response that you get from those APIs is also included in that token limit. So when I was working through it, it kept being like I kept getting really short responses, and I was like, why? I kept telling it, no, give me more words.

And it wouldn't give me more words. And I realized, oh, I was sending it too much data, so I only had like room for 500, a 1000 tokens to send back. So it was condensing it intentionally to stay within that token limit. And that can be a little bit frustrating because if you want to give it more context for how things work. If you want to give it an entire JavaScript library of code and then ask a question, you can hit upon those limits pretty quickly.

Rid. And the token's limits have gone up and up very quickly. It used to be, like, 2,000 when we started talking about of stuff. I do wonder,

because you're mentioning some of this stuff. I I see that Claude exists here As a chat GPT like system, right, that you can chat to it. Yep.

But it can't do things like web scraping just yet. So that's kind of a bummer for me. Have you tried to tried to, like, implement anything where you're you're giving it lots of data And getting then asking it questions about that data. And and if so, are you just, like, copying and pasting? So I took

The entire dump of the syntax, it allows you to do attachments, which is pretty cool. So Claude is the Browser based chat UI, right? And I gave it a CSV of I think at the time we had like 80 responses to the And I asked it to summarize the data as well as tally it all up, and it did a fantastic job because Both Claude and the anthropic API have a 100,000 token limit, meaning you could send it rid. Way more. I could send it if 1 show is 16,000 tokens, right, you could send it 5 or 6, 7 different shows all at once and say, All right, here is 6 shows about CSS.

Summarize them all into 1. So being able to have larger context windows makes the AI even more powerful of this type of thing. So I'm pretty excited about it, to have access to the API as well. Obviously, it doesn't do web scraping and and all of that type of stuff. But I'm really not worried about that because I'm I don't like I use the The chat UI, but I'm more interested in, like, how do we use this programmatically in our applications? And especially for the syntax website, That 16,000 token limit we kept hitting up hitting up on. So I would have to go through every single utterance and shorten it. So you say, alright. You go through every single utterance to say shorten this by 30%.

And then you would get it back. And then you combine all of those that are now 30% smaller You send it all at once and say, all right, well, here's here's a 12,000 token summary of The podcast or the transcript? Now I have a bit more room to play.

Yeah. So when you say that, Claude is is like rid Like, chat GPT, it it's like the this is completely their own model. That's correct? And Claude is the name of the model? Yeah, yeah, yeah. Anthropic is the company

like OpenAI and Claude is the conversational

AI Okay. Like Gotcha. JPGPT would be. Yeah. Cool. Yeah. I've been playing around with it just now as we've been talking. It's it's pretty neat.

I would love for them to add some sort of a scraping mechanism. That's something that I've been getting into quite a bit lately with rid Chat GPT plug ins is basically giving it a URL of some documentation And then asking questions of that documentation because it it that to me is, like, a much easier way to like, if I have questions, can I do x, y, and z? Or one thing I'll do is I'll give it 2 really similar libraries, and I'll say, can you tell me what's different between these 2 libraries and the nice table? And it does that really nicely just based off of the doc. So,

yeah, I think once there's that scraping, I think that'll be rid. Really lovely. What you could do, and obviously this is a bit of a pain in the butt, but you could file save the entire page And then upload the that HTML output. Right? And then,

tell it to parse that. Or use another tool to scrape the information from it. And then yeah. Yeah. Just curl dash 0 and then pipe to pb copy, and then just paste it in. Word. There's also Facebook's rid Stuff which I haven't gotten to use since it's for noncommercial research purposes only, which is the large language model Meta AI, but But I see people talk about it all the time.

Again, this isn't one I have used particularly, but it's Facebook's model. So, rid You know, I we'll see we'll see about this one because you would imagine they have a lot of data. It says it's not being used in any of their products, which, You know, I guess you can take them at their word for that, but, you know, it it at least exists. It's you often see it as llama, l l a m a. It's it's pretty cool because it's open source, so you can download it yourself,

and you could you could host, you could run it yourself.

And that actually brings me to the next one, which is Cloudflare.

Last week at the time of recording, Cloudflare had a bunch of announcements, rid. Some really cool stuff around their workers as well, which we are having somebody from Cloudflare Workers product manager on to talk about rid because I'm I'm a huge Cloudflare fanboy. I don't know if you can tell by by everything that I talk to on this Talk about on this podcast. So a lot of the AI stuff they rolled out last week is pretty exciting. So probably the first one Is the Cloudflare AI serverless or workers. I'm not exactly sure what they're called, Cloudflare AI workers.

And these are a set of prebuilt models that you can just have access to and run-in your workers. So meaning that it's often Often these models are extremely large. Like if you go to the Facebook one. I was going to see how big it is.

But they're often gigs and gigs and gigs of data, right, or maybe even terabytes of data. So it's not something you could just easily spin up on your there's lots of smaller models as well that you can Just spin up yourself and run on your own computer. But a lot of the really big ones that's why we have to use APIs to rid Interface with them because they're running on some beefy gaming PC up in the cloud somewhere. Right?

Speaking of gaming PC, I'm I'm sorry. There was a a a photo from a a a trial that was making the rounds of all the lawyers sitting at their table with their, like, you know, their normal lawyer laptops, And one of the lawyers had a gaming PC with, like, neon underglow and a neon background of it. I was just thinking, that's hilarious. That is,

Yeah, super funny. Oh, that's awesome. So, Cloudflare AI workers has access to a couple different models.

The Lama model that we just talked about is available to you in Cloudflare Workers. So I guess you don't have to request access via Facebook because you could just use it in a Cloudflare Worker directly, which is pretty cool. So they have that one. Then they also have speech to text, translation, sentiment analysis, Image classification and embedding. We'll talk about what embedding is in just a sec. The sentiment analysis one is pretty interesting to me, right now.

I think it's text only. It looks like it is text only and you could take.

Yeah, you could take like a I love Cloudflare or I hate Cloudflare. And then it will tell you what percentage is positive and negative. Like there would be rid. 99% positive and 1% negative. What are the chances that this is a negative tweet or utterance or whatever it is that you're you're working with.

And I want to do that with photos because rid. I want to have, like, thumbnails for the new syntax website at some point and I don't want to go through and find the thumbnails. I would love to Take like maybe 200 screenshots from random points of the video recording and then say, here's 200 images, rid. Run sentiment analysis on every single one, and then give me the ones that are laughing or surprised Or funny face or something like that. You know? Because, like, those would be good images for that and then run those through a background remover

And then make some sort of, like yeah. Some funny laugh eef. Yeah. Yeah. That that's awesome.

As long as it's like, Funny, handsome. Let's get the ones where Scott and Wes are both handsome.

Because sometimes we'll we'll, like, take a photo, and, like, one of us is, like, making a fun face. The other one's making a, like, a grumpy face or or not paying attention. Exactly. Because you can never get it all at once. You're right. So

Wouldn't it be nice to to do it or like even just show me the top 10 and I click I click the ones that we want? I think that I think that's within

doability. I I do too. Maybe not yet or or or relatively soon. I think he I asked. I it would be so the The hardest part would be, like so if you give it a video, that's obviously not going to work. I think you could find The ones that are, sentiment I think you could do sentiment analysis on photos, you know, pretty not easily, but you could do it. So the hard part is is, like, how do you get to process enough photos to have it actually find something good? Over the course of, like, an hour long podcast, you're thinking about, like, frames. How many do you need? Like, how many frames do you need to be able to find one that's decent? I think that's probably just through experimentation, but, you know, could be expensive experimentation.

That's a a good question, I don't know the answer to that. I asked on Twitter a while ago, like, if there's anything that does this. And one Kind of interesting. Matt from Caddy told me that if you open up the SQLite database of Apple Photos, There's all kinds of data in there about, like, smiling and ethnicity and all kinds of interesting stuff. And I was rid. That's really cool, but I was also really interested that you can just open a SQLite file

from some of these applications. Yeah. I mean, if you think about it, like Google Photos rid. In Apple Photos for a long time. They can identify people and dogs. I mean, my my dog with dreadlocks, When when she had her dreadlock shaved at one point, she looked like a little poodle, and Apple Apple Photos was smart enough to be able to determine that that was the same dog somehow. A dog with dreadlocks and a dog that looked nothing like it. But it's like, oh, yeah. That's the same dog. I have no idea how it does that. Yeah. You know, a really creepy website I'm going to put you on to,

facecheck.

Id. So what it does is You can upload a photo of somebody and it will search the Internet for other photos of it. So I literally I just took a screenshot of you right now And I'm waiting for it to process. Sometimes it takes a couple of minutes because you have to wait in a queue.

But I took a photo of myself And just like not a photo that's online, not my Twitter photo. Like, I literally pointed the camera at myself, took a photo, uploaded it, and it found Probably 20 photos of me throughout the Internet. Like it knew exactly who I am. So that's really spooky. And if that's something that you can just use for free rid On a website? Like, imagine, you know, like Facebook's been doing that for many years. Here we go. It's searching faces. Maybe I can I share my screen with you?

80% done. Oh. Yeah. Although some of these are not definitely not me, But they they look like a scruffy or worse version of me.

What's this guy with this guy who looks like a troll on here? What's he doing? Okay. Hold on. Hold on. Don't don't look at those.

The fact that I just took a screenshot of you, and it says 80% Accuracy, I found 1, 2, 3, 4, 5, 6, 7, 8, 9, 10.

10 photos of yourself, and it tells you, rid Oh, 12.

Is this you? Which one? Let's see where that's from. Yeah. It is. It it found, like, a a photo of you from 10 years ago.

And then obviously it finds a lot of photos that are similar, but it gives you a score. Right. So if something is 90% Accurate.

This is wild. That's that is crazy.

And it's kind of funny the ones that it thinks That is you. Because, like, I would never mistake any of these people. But But I think it's the glasses. The glasses or sometimes they're even wearing a similar shirt. This 1 guy

It's, like, bald and not even close to looking like me, so I have no idea where where to pull that one from. But Yeah. That one

but, Like, sometimes like, maybe there's something that LinkedIn page. Yeah. Like s yeah.

LLMs are unreasonably good for technical reasons. Rid No. There's there's not you're not on this page at all. Wow. This is yeah. It is very, very wild. But impressive that that does that. So Wow. Are no longer yeah. That's you can walk around with a mask on. Yeah. Right. I know that is very scary.

And and did you have you on on the topic of AI stuff, you know, before we move on to more Cloudflare things, Have you seen the new trend of AI hardware things? There's been, like, 3 AI hardware devices announced in the past week. Oh, what? Rid. From from who? Well, one of them is from a company called Rewind, and

Rewind kind of like records your screen all day long and Yeah. Well, the Yeah. I've I've been on their email list, but I've never pulled the trigger. Basically, it records your rid all day, and then you can search your life. Yep.

Seems so cool and so sketchy. Invasive.

They they have a AI tool that you wear is like a pendant around your neck, and it records everything you say and do all day. And it's like, No. No. Thank you. I don't I don't need that. But, then there's the the AI one from humane or whatever that is where it projects onto your hand. And, like, that's 1 device. It projects onto your hand, and you can use your hand as, like, a really crappy smartphone. That is 1 device that I do not understand why anybody would want. It looks awful. It looks like it functions bad.

It looks the the the pen itself, it looks like a like a 19 nineties iPad iPod. It it's just I mean, your 2000s iPad, But it it looks bad. So I'm shocked when I'm hearing so many people, like, praising that thing because, to me, it looks pretty pretty rough.