Cloudflare Workers Are Next Level With Rita Kozlov And Brendan Irvine-Broque

Welcome to Syntax. Today, we have a awesome episode. We have 2 folks from Cloudflare Workers on today.

I've been a Cloudflare fanboy for many years. I've used it for quite a few of my projects.

And in fact, I think we've done a couple episodes on Cloudflare is not just a DDoS thing. Cloudflare has tons of really cool products behind them, and one of the really cool Is Cloudflare Workers, which is their serverless edge running basically running server code on Cloudflare.

So we have them on. We've got Rita Kozlov and Brendan Irvine Broek on today to to talk to us. So welcome. Why don't you give yourselves a bit of Instruction. Say who you are. Hey.

Excited to be here. I'm Rita. I'm senior director of product for our developer platform workers.

Been at Cloudflare for 7 years and, particularly focused on building out our developer offering.

So really excited that you guys wanted to talk to us about that. Cool. And I'm Brendan, product manager for the Cloudflare Workers runtime. I focus very deeply on the APIs that we offer in workers, web standards, kind of everything that's developer facing that you work Within in a Cloudflare Worker. Nice.

Cool. Well, before we get into it, I do wanna say, if you're pushing out code anywhere, CloudFlare. It's going on the Internet. A lot of people are using it. You wanna make sure you have some error and exception handling tracking, and Sentry is a great place to do that. So head on over to century.io.

You can always use coupon code tasty treat,

all lowercase, to get 2 months for free from Sentry. Yeah. You can, Sentry's got a Cloudflare Workers integration. So

There you go. They got something for everything.

Let's start with there's kind of 2 things I want to get out of the podcast for the listeners is the first I Everyone listening to understand, like like, what are workers and, like, what are the use cases and why might you want to use 1? And then 2nd, I wanna dip into, like, the alternative JavaScript runtime, which is like like, obviously, we have Node and then we had Dino, and you probably heard of Bun, and there's We had Jake on from Fastly, and the other day on Friday, somebody announced, I forget if some company, Some Wasm company, rolled out winter dotjs, which is another server runtime, right? Like we're starting we have way more Server runtimes than we do actual browser runtimes for JavaScript now, which is wild. So, we'll talk a little bit about the effort to Keep those standards based so we're not having IE all over again.

So let's start. Like, What is what's what is Cloudflare Workers and what are they useful for?

So you kind of mentioned this earlier. Right? A lot of folks know Cloudflare for things like CDN, DNS, DDoS, but that's so much more or we do so much more than that now, and That's where we started. And when we, started onboarding larger and larger customers A few years ago, we started running into this problem that I think every single startup hits. Right? Of you want to onboard larger customers, but they have So many requirements for you, and there are so many customizations that you have to make. Right? So it started out with if I want to add a header, if I want to cache Only if this cookie is present, if I want to do something only from this country.

We had a very manual process for it. There was An engineer that went behind the scenes that wrote an if statement somewhere that's like, if, you know, it's syntax FM, then do this thing.

And obviously we knew that that wasn't gonna be scalable.

And so we wanted to make it possible for anyone to be able to service themselves and be able to do this on their own.

And I think that this story, this origin story is really interesting and really crucial to kind of understanding workers because this Came with some requirements and constraints. Right? So if we are a CDN and one of the main value props that we provide is that we make your site faster, There's no way that by making it customizable, we can make it slower. Right? Yeah. And so we started looking at different approaches. How can we do this? And, you know, the standard way for deploying code somewhere has always been a container. You can spin up a container.

But that introduces this problem of, okay, now you have a 15 second cold start maybe.

And That's just not acceptable when you're trying to make things faster.

So we started thinking about how do we make this possible? And actually, You were just talking about standards. Right? We turned to the browser, said, okay, there's already a way for doing this in the browser. It's called Islets.

And we thought about, well, This is the most battle tested software out there, the v eight engine. What if we deployed it to Cloudflare's network edge And allowed customers to deploy their own code on our network.

And this magical thing happened, which is that Developers, and we've realized that what you could actually do once you were deploying software at this scale in every single data center was so much more full than being able to customize a CDN.

You could actually build applications that felt like they were scalable from 0 to To Infiniti, we're instantly really, really performant and fast because they ran really close to the end user, and enabled all of these different use cases. And so Well, workers in a nutshell are a way to run serverless code at the edge. I the way that we think about it is a completely different paradigm for building applications In this way, that is how we always wanted it to be, but had all these constraints getting in the way of it. Wow. So,

it was, like, really interesting how you said, oh, well, maybe we could just run clients cut at the edge here.

How does something go from an a big idea like that to actual execution?

Does that take a long time? So from the moment that we started thinking about it to it being live, it probably I think, you know, the rule of things is that the 0 to 90% takes Mhmm. Not that long, relatively speaking. Right? And then the last 10% has been the past 5 or 6 years. Right? So Yeah. We had The initial prototype, I think running within a few months and released it, the the original version of it back in 2017, I believe.

I've often talked about Cloudflare workers on like there's kind of like 2 ways you can use it. Like, first, you can use them as Just a server. Someone will hit your thing. You can do some work and and return the data, or you can use them as like a something that sits between Your server or your CDN sits between the the user and the actual end result, And you can sort of jump in the middle, that's how I like to describe it, and say, hold on, let me let me figure something out, you know? Like, let me figure out, What country you're from? And if that's the case, let me redirect you. Or, do you have this cookie set? If so, then let me render out something a little bit different.

Do people use Cloudflare Workers for for both of those things? Like do people can you run an entire Application in Cloudflare Workers, like your entire business on it?

Yeah. I think so so much of what we're seeing right now on our platform is people really doing that and building Totally new greenfield, full stack applications on Cloudflare, and that's really where we're trying to go, right, if you've seen some of the announcements we've been making around allowing you to connect directly to your database, accessing data, working with, the Cloudflare Q's product that's deeply integrated with workers.

Like, it's all kind of to that end of how you build something complete. But then there's also this really interesting angle I like to talk about with, kind of outside in refractors, where you can kind of take the outermost layer of an application And start to kind of peel back the layers and put the outermost layers on cloud flare, on workers, and then start to build maybe the net new Pieces of what you're building on workers, because you can control the outermost layer, you can control the the kind of routing of what, of the entry point of your application. And so I think that's really powerful for teams that, you know, maybe feel a little bit stuck today, that really want to See the benefits of serverless, see the benefits of workers, but don't see necessarily that path to incrementally migrate.

We really wanna make that a a great path for people. I was gonna say, what we often

see is people start their journey With 1 of 2 things. 1 is what's a part of my application that's really slow that I would like to have it be faster, or what's a part of my application that I'm having a really tough time scaling, and those often do sit at kind of that in between layer. So I find Auth To be a really great example of that, where we have so many customers, including NPM was actually one of the first ones where The auth server is always this source of choking point and is this perfect thing that sits between the user and your origin server. And once you offload it, you unlock both a lot of scalability and performance.

Do you also see people rewriting large monoliths Into like a we heard it from a couple of listeners where they say I use Cloudflare Workers to we take our existing PHP app that's 37 years old, and we still need the header and the footer, and we need different parts of it. But Like, I want to be able to fold in my fancy new application, but we simply do not have the infrastructure to do it. So they were simply just like Rebuilding their website 1 hunk at a time, and, eventually

eventually, they'll get rid of the The old monolith, is is that a common use case, or is that just a a couple one offs we heard about? I think that that kind of architecture you're describing where there's kind of this islands approach is Definitely something we've seen a fair amount of, and we've done some of our own, blog posts talking about that externally.

And so, yes, definitely a thing that we've seen happen quite a bit. Okay,

that's awesome. I also wanted to to just give a couple of different use cases that I found Super handy for Cloudflare workers to our listeners because, one, I paid for the SaaS called CloudApp for the longest time and They let you have your own custom domain name, but a couple of years ago they took features out and then they started making The it basically was for screenshots and sharing images and and redirecting URLs. And they took the ability to they took the read URL redirects out entirely because People probably using it for spam or something.

And then they, like, started making it look crappy where they put their logo all over it. And, like, it was just like the product It kept getting worse and worse and worse. And so what I did is I I wrote a Cloudflare Worker that basically just sat in front of it. And I took my domain name back and I said, all right, whenever somebody goes to it, it was west. Ioforward/abc123.

Just go fetch the unbranded version of that asset.

And then if it's a redirect, redirect them. If it's an image, download the image And pass the or simply just return the fetch request from that and it will proxy it. And then I also I added in my own CSS. It made it look cool and like There's some stuff here and there that, like, often whatever, sometimes gets into trouble. But I thought it was so cool that you could just like proxy an entire website, Add in your own, and that's always the thing like, oh, should we build it ourselves or use a SaaS? And this is kind of like, well, I'm using a SaaS, but I can sprinkle on my own little Secret sauce with a worker on top of it.

No. I was gonna say, do do you wanna give the audience maybe a little bit of 1 on 1 or 101 for, like, what Specifically, makes the edge important.

Yes.

What is the edge? Where is it? It's such a funny question.

We have, a bit of a love hate relationship with the word edge because, first of all, I I think it's really hard to wrap your mind around. So to your point, it's not all if you don't know what it is, it's not a very intuitive thing. Right? And so the way that we prefer to describe it is generally, you know, a global network of 300 data centers.

And it it's The idea of the edge, right, is that you can get as close to your users as possible or as close to device as possible without being On the device itself.

But the tricky part of it, especially when it comes to talking about compute and workers, Is that people do have the this association with edge being something that's very lightweight and designed for kind of niche purposes. Right? So It's what you would reach for if you want, something really latency sensitive, really high performance.

And don't be wrong. These things are super important. We see the Impact that a 100 a 100 milliseconds of compute or a 100 milliseconds of latency might have on an entire company's revenue. Right? Especially in ecommerce. So latency is super important. But at the same time, it does make it slightly more niche in terms of, like, How you think about it. So you start to narrow down your use cases by, okay, well, why do I want this to be really, really fast? I think that the thing that gets lost about How powerful it is to be able to run this network is just the scalability and flexibility that we get in how we schedule things And how much of the cognitive burden in terms of what a developer has to think about, it takes away. Right? So if you've built an application before on a traditional cloud provider, The very, very first thing that you actually have to think about is what region are you gonna run this in? Where do you want it? Where do you want it? I Wanted everywhere, please. Yeah.

I've always found that to be such an insane question. I don't know on day 1 where my users are going to be. Maybe I'll start in On the East Coast, but maybe my application is gonna take off on the West Coast or in Australia. Who knows? Right? And that's where the kind of concept of region Earth came from, which is that That's not up to you for the developer to have to think about. I think the onus is kind of more should be on the provider to figure out where is your traffic and Optimize it for that. And so that's our conception or how we think about the edge. I love that. There's so many times I ask myself

as a developer, Why are they making me think about this? I have so many other things to worry about. So I I've I actually have not heard the Regent Earth thing. That that's super cool.

Yeah. When I did my like, when I 12 years ago when I launched my 1st product, I I launched a thing, and then I I woke up the next day to Bunch of emails from Germans, and they all said, your downloads are slow.

And I I had the thing on Amazon s three, and I learned You got to put CloudFront in front of it, and then they charge you for that and they charge you to download it. And, Then they'll distribute it all around the world. I just was thinking, like, I don't care, you know, make it fast, you know. And the same thing with the with these edge functions as well is, like, Hopefully, in a couple years, you shouldn't think like, that's a good use case for an an edge function. Hopefully, in a couple years, it's I wrote the standard JavaScript,

and I employ this sucker to Cloudflare or whatever else, and they're gonna figure out all the sort of hard stuff for Yeah. And on that point of what are the hard things, we're starting to really push the bounds of, as Rita was pointing at, this the kind of scheduling requirements.

So one thing we've built is this feature called smart placement with workers, where if your worker talks to a back end and makes, you know, a couple of requests to say an API that is either yours or a third party API, and that back end is in a particular location, it's actually the most performant thing To run your worker as close as possible to that back end rather than running that worker as close as possible to the user.

But, you know, you shouldn't, as a developer, have to think about, you know, where your that backend resource is. It might not even be something that you control.

And so what smart placement does is we understand where your worker is making requests to, and behind the scenes, we're automatically figuring out, Oh, we really should run this really close to US East

one because your worker's always talking to it, and we know that that's gonna get you the best performance. Oh, that's that's the thing we get all the time, anytime we talk about Edge Functions, we get tweets that say, yeah, it's great until you realize your database is halfway across the world and The initial connection to the worker is great, but

then it's gotta go across the world to get your data and and come all the way back. So, oh, that's really neat. I didn't know that you had done that. Yeah, and we're trying to apply that same kind of thinking across our products, whether it's durable objects or d one or some you know, how do we manage pools of connections To databases, that's not just like one one off feature for us. It's like that's how we think about scheduling on our network, is Not just, oh, we're gonna run this as close to the eyeball as possible. It's your problem. Go figure it out. But, like, we wanna make those decisions for you. And this Might not be the

sexiest thing to think about too, but if you talk to any developer, especially in Europe, one thing that's Constantly on their mind is GDPR and how do I meet compliance regulations.

And so the other thing that we found is that it's actually much easier to take something that's running globally and scope it down.

Right? So if you want your data to always very specifically in Germany, or in the EU, it's a lot easier to go, okay, my application needs to be compliant by these laws. It's much, much harder to do the inverse, which is take something that's running in 1 region and then scale it up to additional regions. And many companies started in a region that's Not even compliant with where they're trying to be. So this flexibility just opens up so many different use cases. Mhmm.

Let's talk about that, the flexibility. So you get speed, you get flexibility.

There's always that triangle of of pick 2.

And there's there's always trade offs that you have to make when when you get something amazing like that.

What for someone who's like a traditional Node. Js developer coming at this, they're used to having a file system, They have a long running server, you know, like they're just thinking in that approach.

The even the switch to, like, the serverless mindset is is a bit of a hump. So, like, What is the trade off that you have to give when you think about building with a worker? I think that we respect how much trust people are putting in us is the way I would frame it. It's like, mhmm, there is a trust thing, right? Like, we are taking on more responsibility on your behalf. We are managing not just the infrastructure, but we are Making it so that you don't have to be responsible for that server. We're thinking about, you know, where problems like I was alluding to before about where you should Run where you should operate.

How do we handle restarts of that instance in that server so that you don't have to? And so that's a that's a kind of a responsibility that we take very seriously, but we know that it takes that trust From somebody who's building on our platform to trust us to do that for them. That's kind of the way that I would frame it. Also, like like hard limitations

of the thing is A Cloudflare Worker only runs for I think on the free plan, it's 10 milliseconds. On a paid plan, you get up to to 50 milliseconds. And, People always look at the 50 milliseconds like that's that's not a lot of time. So are you able to explain the whole wall time versus compute time? The thing is it's not 50 milliseconds of of it running, right?

Yeah. So we actually, we're recording this on Halloween today, and we just are launching our new workers' pricing model, Oh. That, has much higher limits than what you were alluding to. So we actually have 30 seconds of CPU time per request.

And for context for if you're you know, you haven't thought about how much CPU time your worker might be using, you know, most workers are using, you know, somewhere around 10 milliseconds, like far less than 30 seconds, so it's plenty kind of per request.

And so to maybe break down that difference For listeners, around wall time, and or duration, what's often referred to as, like, gigabyte seconds, which is kind of a confusing metric Mhmm. Versus CPU time, CPU time is is the time that's the that's actually spent by the CPU doing actual real work, versus wall time or duration or gigabyte seconds can encompass time where there's no work happening. You're just waiting on IO. So maybe you made a A request to an API, and you're waiting to hear back. But until you hear back from that server, there's no work happening in your worker.

And so what that means is, like, Duration of wall time, those are things that are often out of your your control, especially if you're building an application that talks to other services that aren't yours, versus CPU time.

Like, that's the stuff that that that, you know, you you wrote a a for loop that is a little bit gross and takes a lot of time, or you brought in some big dependency that's doing extra work when it doesn't need to. That's the stuff that you control as a developer, And so, that's kind of the difference between those 2 and why, you know, we really think it's the right thing for for you to to pay based on CPU time, because that's Something that that you can have you can really shape. Mhmm. Yeah. That makes a lot of sense,

especially for me personally because I I would have just you see that the time limits that people talk about rather harshly as being, just kind of blanket time limits. But, really, So many of the the times that we end up having, large amounts of wait time are IO or or that stuff. Yeah. Your request.

Totally.

So I'll give an example of, for people listening of limits that I've hit in the past. And quite honestly, I'm really excited to hear that you've increased it There's been a couple of projects where I'm like, I'm going all Cloudflare for this thing. You know, Cloudflare stack everything.

And then One of my packages had, like, a weird load time and, like, not even network. Like, literally, there was some bug down the Required tree that was causing the actual object instantiation to take, like, way too long, like 80 milliseconds or 100 milliseconds. And that was too much, so I had to go to a different server server to to actually host that thing. And I've hit that a couple of times where I'm just like, ah, like, there's a There's this hard can't do it on Cloudflare and like Puppeteer is another good one that now I'm part of the Puppeteer beta. You can take screenshots with Cloudflare. Like a lot of those Pain points are getting solved.

So another one I have is I have a course that hit a recipe API.

And that recipe API went down.

And I said, shoot. I don't wanna rerecord the video, but there's no other recipe API that has the exact same Data structure. So what I did, I went and fired up a Cloudflare Worker.

I I scraped some other website for recipes, And then I parsed the HTML output, and then at the other end, I gave the exact same structure of data that the it was expecting. And now I know it. Never never give learners their I'll always give them a Cloudflare Worker URL because if anything changes, I can always I can always massage or fill in that that data at some point and That, we're scraping the data. That's a fetch request. That, I'm not paying for that. Right? Coming back, I have HTML, Then I have to parse it. I use the package called LinkDom, which is worker friendly, and then I can loop over the divs and scrape the The name and the image and all that stuff, and that takes anywhere from 30 seconds to 70, 70 sorry, 30 milliseconds to 70 milliseconds to actually do that. And I often go over the 50 milliseconds, but it's not like I don't think it's a hard limit. Right? Like, if you go over sometimes, they're not gonna just Shut her down. Right? That's correct. We we do a little massaging of the limits,

there, but we just generally don't think that you should hit that. Yeah. Yeah. Fifty millisecond limit. We're trying to make sure that every everybody

isn't in that case. It's kind of like a Like a tough love kind of thing, I bet, being like, well, why is your your thing taking 200 milliseconds? You know? Like, Of course, we could make it larger, but it's probably better for your bill at the end of the day, your users who are visiting your website, that You figure out why. And that's what I did in that case is I I some guy from the React team actually helped me figure out how to use the dev tools to figure out What's actually taking a 100 milliseconds? And it was that thing taking, like, 99% of it, and the rest of it was 3 milliseconds.

Yeah. And, you know, one risk that comes up as you introduce higher limits is that people you're kind of subject to denial of wallet attacks or that Somebody finds some infinitely complex HTML tree that they send to your worker, and suddenly, you know, it's spending 15 seconds every request, and you're getting billed for that. And so what we're doing is, we're introducing limits that you can configure Okay. On a per worker basis, so that you can say, Hey, I know that, you know, p 99, this should only ever be two 150 milliseconds of CPU time. I wanna set the CPU time limit a little bit lower just as a safeguard in case there's a scenario that I haven't thought about To make sure that there's no case where, like, my work will never be taking 5, 10 full seconds of CPU time per request.

What are some use cases where you might want to increase the limit to several seconds or or maybe up to 30 use cases? Like, what were you seeing from your

So I think one product I might highlight is Cloudflare Queues.

So it's important to remember that workers aren't just being used out there for incoming HTTP traffic, but also for processing, you know, batches of messages and actually doing important Background processing of of work, and there's real computation there. And oftentimes, people are processing not just 1 input in there, but a batch of different inputs.

And then so that's where you really see, a bit of a higher, CPU time per invitation.

Or similarly, Sometimes when people import large packages or with web assembly, especially if people are trying to Import whole existing programs that they had running somewhere else.

Those generally tend to be a bit more

CPU heavy. Mhmm. Yeah. We so we just rolled out a news and tax website. Yeah. And 2 things that were pretty big is, 1, we, Combine a bunch of audio clips before we send them off for transcription and into a big buffer. Right? And we're doing that with a WASM package, FFmpeg.

And that takes yeah, that takes a little bit of time before we actually send the thing off. And the other one is we use Puppeteer to take a screenshot for our Open Graph images, and that is dog slow on, once we deploy the actual thing. It's got are open like we keep it warm as long as we can. But even just taking it, loading a page, take a screenshot is is a little bit slow, and we're still running that on a Like a cron request, but that is the perfect use case. I I really wanna move it to Cloudflare queues because that's perfect use case for something like that. And we have our browser rendering API, which is designed exactly for this use case also. Oh, yeah. That's the so it Has like a puppeteer API. Right? But it's not actually puppeteer.

Exactly. It's not spinning up puppeteer. I think this is actually a perfect example of your question before about Trade offs. Right? Is that you do sometimes we want to provide as many of the tools as possible that developers are going to need. And so When we hear about something like, oh, I wanna be able to run puppeteer in a worker, realistically, jamming all of puppeteer into a worker is not going to be optimal neither for your performance nor for our memory footprint. And so we'd rather Offer up a service or an API that makes sense within the context of the environment that we're running this in. Yeah. So I guess even

to to go along with some of the things that developers are often needing, I guess, like database products. Right? I've seen the the Cloudflare d one.

Is that a database that is Distributed, or is it just a serverless platform for databases?

That's a really good question.

So the way that we think about, I would say data in general, when it comes to running applications with workers, we Provide a few different options depending on, again, the different use cases and what customers are going to need. And try to make it as Simple as possible for anyone to be able to reason about how their application works without being a distributed systems engineer.

And so for most use cases that you're going to use a database for, you actually don't want it to be live everywhere all the time.

That doesn't really make sense, and you're going to hit very weird consistency guarantees. So d one actually does generally live in a single point of presence today.

But one of the things that we're working on Is, the ability to do read replication so that you still, for things that are frequently read, get The best performance possible. Cool. And I'll mention one other feature that we recently announced that kind of goes hand in hand with that logic, which is called hyperdrive.

And so, yes, data is a big investment for us. We are constantly rolling out. We rolled out d one, which is our own database. We have r two or object store, and a bunch of other tools and features and products. But we also recognize that data specifically is one of those things that's really, really hard to move. And as an organization, you might even spend time re rewriting your But probably you're not going to spend the time moving your data.

That's a really tough challenge. But you can still reap the benefits of deploying your application itself on the edge while accessing that data. And the way that we do that is through one of our new products called Hyperdrive, Which allows you to connect to your database. It'll handle connection pulling and everything like that, but it will also make your data feel distributed.

Oh.

By cashing your queries. So would it be safe to say this is like a planet scale like product? Is that

Accurate? Are you familiar with them? So this really lets you connect to any database, even an existing one. So let's say you had a Postgres database In AWS, or you had a MySQL database living out somewhere, you can use HyperDrive on top of that, and it's handling the Serverless connection pooling and caching

queries where it can. Cool. Okay. That makes sense. Yeah. Okay. Because so we did an episode on Serverless databases, which is kind of explaining.

Alright. Well, you got key value stores and then you have, your existing, like, relational databases. And often Then those relational databases don't work well in serverless because every time you run a function, it needs to set up a connection. And a A lot of times you can't have that many connections to your database. So the answer to that is you have to have pooling. Right. And that's a special sauce that What am I Scott, who are we using? We're using PlanetScale, but there's also several others. Was it Neon? PlanetScale. That's that's a special sauce that PlanetScale does is they They do the pooling on their end, and then they'll they'll connect it to it. So you're saying hyperdrive will allow you to, like, sort of just, like, run A connection pool and then you can connect to it. You can hear DB. I'm assuming

you guys have fat pipes between all of the different Cloudflare locations, which makes Connecting to wherever that pool is fast? Exactly. So things run through our own network and kind of we're figuring out where that pool of connection should actually live, Making sure that's in the optimal place for your worker, and you don't have to think as much about, like and if you think about so many database drivers, You have to manage that connection pool yourself, and even just that layers on some complexity, and we're trying to To create a world where you just don't think about it, kind of in what you're pointing to about PlanetScale, where you really don't have to think about your connection pool. Yeah. Yeah. In my neighborhood, they're laying fiber.

And I guess the jacket that they fished the fiber through is orange.

And I was looking at that being, like, oh, Cloudflare is is doing a pipe to everybody's house, but I had nobody to tell that joke because nobody would like Nobody nobody's gonna appreciate that joke.

Yeah.

We appreciate it. Yeah. I got your job. Okay. Yeah. Thank you.

Let's talk about The actual runtime. So Brendan, that that's that's what you work on, right? Like Cloudflare Workers runs JavaScript, right? But it's not A Node. Js runtime, right? Right. And so, you know,

one way to think about it in your head is that we use V eight under the hood just like Chrome does, and so a worker is, in some sense, akin to a tab in Chrome.

And, a thing that we cared really deeply about at Cloudflare is kind of code portability and using web standards.

And so I'll shout out James Snell, who folks may know from from Twitter, who's contributed a ton to Node. Js over the years, works on the open source Cloudflare Workers runtime called WorkerD.

And he's actually been the one really Pushing and making a big push on our side to, you know, be more standards friendly, to Be kind of at the leading edge of thinking about things, not just selfishly for ourselves as our own runtime, but trying to make sure that there's an ecosystem of different run times that All kind of represent the shared needs of server side developers building applications in JavaScript, so that we're kind of On the same page so that the ecosystem of libraries across npm works across different things, so that you as a developer, you're not constantly facedown with, Oh, I want to use this library. Is it going to work in runtime X or Y or Z? And I think Cloudflare was incorrect me if I'm wrong, but Cloudflare was very early to this,

We're going to make a standardized JavaScript runtime because you basically took the servers service worker API, which a service worker in the browser allows you You control like, if you make a request to a URL, a service worker can literally, like we say, jump in the middle and say, uh-uh, I'm I'm going to serve you back a cache or I'm going to modify the response in some way. And that's all based on request, response, fetch, add event listener.

And Surprise, surprise. That's the Cloudflare Worker API as well. Right? I think I can pretty safely say,

we were the 1st to use this model.

And I'll give credit actually to, Ingvar who was on our team at the time. So when we were trying to figure out in the 1st place, okay, how do we do this? We're not using containers. We're using the V eight engine.

And what are we gonna do? Are we gonna run node? That doesn't seem Quite right for our model.

And, Ingvar shows up and goes, there's a model for this already. It's called service worker.

Why don't we just do that? And since we wanted to adhere to WEB standards, we did not want to make up something proprietary that was Clafler special sauce that just seemed like a really, really perfect fit to us.

That that's awesome. And that really paved the way for, what is the the winter c g. Right? Like, so do you wanna explain what winter c g is and and Cloudflare's part of it? Yeah. So winter c g is,

a community of, developers is open to anybody, who are all working towards, making, developing the set of standards that, server side JavaScript run times all share, so across Deno And the work workers' runtime and others that are, as you talked about earlier, kind of emerging right now, what are the set of APIs that All of those run times share and support, and also voice kind of representing that group of people when we talk about, JavaScript standards with TC 39 or, browser standards, because there's some things that, you know, may make sense in a browser, But may need to be tweaked a little bit or have a different context when you're using them as on them on the server.

And so that group meets regularly. If anybody's listening and wants to get involved, it's Open to anybody.

We really welcome external participation. We I think that group would love to hear more, from people about, you know, what are the Things that maybe you've relied on in Node in the past when you built applications that you'd love to see a broader set of run times all support.

I would call out 1 API we've been working, on over the past year as kind of an example of where the This group is having an impact, which is the, TCP sockets API called Connect that we've been working on.

So we really wanted to to be able to support TCP connections from workers, to be able to connect directly to databases.

But, when we were looking at the ecosystem, you know, There's an API for this in Node. There's an API for this in Deno.

There's been past attempts to make the an API work in browsers that haven't quite succeeded.

And so there wasn't really a a standard for us to lean on.

And so what we we did was we kind of worked within the community of Enter CG to say, okay, we're gonna support this on Cloudflare Workers, But that's not that's not where our kind of work ends.

We're trying to make this the API that is shared across Node and Deno and getting the Cool community on board and hearing people's feedback along the way, because we don't think it's the right outcome for there to be, like, You know, you're Cloudflare Postgres database driver, and then there's a separate database driver that's the Node one, and it's this forked ecosystem, and it's impossible for All the wonderful open source maintainers who, you know, put their work into this for free to have to, like, deal with all these differences.

And so we really care about people building open source libraries as, like, kind of our primary customer on our runtime team, And Winter CG is kind of the the way that we make sure that we're listening and getting feedback and kind of working out in the open. That's nice.

I I'd look at this list of of companies. I I see a bunch of big, big, big, and small companies, and I just, like, wonder, Logistically, how does something like this, 1, come together? Like, this is this is wild that all these companies can work together on something like this. But also 2, like, side question to that. Where does where does ByteDance and TikTok fit in? It seems like a lot of these people are hosting companies.

Let me can can we just preface this by saying, we've asked Ryan Dahl. We asked Jared from Bun. We asked Jake from Fastly, and nobody can tell us why TikTok is part of winter c g. So we are. You're our last hope. Why is TikTok part of winter CG?

That's fascinating. I answered the 2 questions in order. So Okay. I just have to shout out, James Snell on this podcast, because he's really been the driving force behind this effort, and when it comes to how this has been even possible, I think it really speaks to, like, you You know, the importance of being able to build community and build relationships and build that trust over time, and that's something that he's really led the way on, I mean, helping people see the long term, right, that, like, we're not all in it for ourselves, that, you know, our work is gonna outlast any one Product or platform or company, and, like, this is really helping everybody see the bigger picture behind it.

And so I think to That end, I think there have been people, even who aren't working directly on their own run times, or maybe have an internal run time that they've experimented with, who wanted to to get involved over the course of time. So I I don't know if I'm gonna be able to solve the TikTok mystery, but I know that there's a lot of people across a lot of Across a lot of companies, you know, like Bloomberg, right, where like Bloomberg doesn't have a runtime, but there's many people on TC 39 working at Bloomberg who make big contributions, who get involved. Well, they do have a runtime. It's just not something that

you, Wes, might use. But there's a runtime that operates They're systems. Right? And so for them to be able to take advantage of what are the different APIs and how do we make sure that, yeah, as libraries and things batter built. They're reusable.

It it's surprising how many that's one thing that we learned very quickly as we started to embark on this is How surprising it is how many people are trying to do this exact same thing in various ways. Run their own JavaScript runtime. Yeah. We,

we asked the folks From the folks from Bloomberg are coming on. They they're just going through the whole legal thing where they have to get it all approved. But we also asked the folks from Shopify, we said, like, like, what's the deal? Why is Shopify on this thing? And they said, like, we just want Our frameworks, but this was before the remix acquisition, but we basically want it to run wherever, you know, like we don't if you're building a headless Shopify Application, we wanna make sure that runs

absolutely everywhere. So I thought that was pretty cool answer. Well and I think that it touches on to your point of What we've learned is everyone is trying to run a runtime, and sometimes it is the case that you do need your own specialized runtime.

But most of the time, what happens is companies encounter a similar situation to us where they want to be able to Customize something infinitely.

And so to be able to do that, they want their own customers to be able to run on trusted code.

And as it turns out, running an it's one thing to have a group that standardizes APIs across run times and have a bunch of open source run times that doesn't make it easier to reason about. But operating your own run time is still and scaling it and Having multi tenancy and all of that is a really massive challenge.

And so as we started having these conversations with more and more customers, That was what led us to eventually release workers for platforms, which actually Shopify itself has built, Hydrogen on top of. That's one. Yeah. Is To exactly satisfy this use case of just because you want your customers to be able to run untrusted code doesn't mean you have to become overnight an expert in, running a massive multi tenant system.

Interesting. I never thought about the the untrusted COVID. It makes sense because I also, I sat on a board for Wix, and Wix is now rolling out their own.

You can run serverless code on there, but every time you want to use a package, You have to get it approved by them. Every time there's an update, somebody's got to comb through and make sure that it's probably fine, you You know? Mhmm. And it's because, like, you just can't be letting random people running random code because they're gonna mine Bitcoin or do something nefarious with it. So

I never thought about that. That's great. If someone at Wix is listening to this, I promise I can make your life a lot easier.

We'll make that we'll make the connection. But, yeah, yeah, that would be a really interesting use case for it. I never thought about that. So hano. Js is One of my favorite ways to build a server lately. So it's it's it's kind of like the Express, But it's based on web standards. So instead of you can still do middleware and all that, but like you get a proper request and response and you can use all of the Web APIs that have entered JavaScript in the last couple of years.

I was surprised to see Cloudflare just went and hired the Hano JS guy, which is amazing.

But my question behind that is, do you think we'll ever see a a web standard for Were that, like for servers for doing routing? Or is that kind of like outside the the scope of of what JavaScript should cover? That's an interesting question.

1, I think that there are pieces, if you peel back the layers of the onion, that are opportunities to standardize.

So one, for example, is, around JSON schemas and what, Fastify does with them to performance optimization for its routing layer.

And there's a number of different libraries for kind of working with JSON schemas and kind of doing the cogen, of a faster router based on that schema.

And one conversation that, you know, we've been having a little bit ourselves is like, Is there an opportunity to standardize that API for, you know, what is a JSON schema? How does it get transformed? And is there an opportunity for us to kind of work with the winter CG, to to to shape that? And that might create, like, one of many primitives to make it easier for some of these different frameworks to do a little bit less themselves and rely on more standards under the hood. And so I could imagine, you know, a future a number of years from now where there are enough base primitives that Some of the different server frameworks can kind of level up in their thinking and worry less about the underlying, primitives that they're having to each build in isolation themselves That makes sense. So so better

lower level primitives

that we can then build something like a hano on top of. And you'll we'll see that. I think that you know, I I mean, it's there's so many opinions about this, but, like, people have thoughts about routing. I've seen so many of the arguments around rails routing. Right? Like, that's there's so many spicy topics around, like, oh, it should be file structure only based routing or, You know, I have this one weird middleware constraint that I bring to the table. Yeah. So I think that there's an that's a space where Probably all we all benefit from there being some diversity in the ecosystem.

What does the local dev, thing look like. Because you are running in a constrained environment when you run-in Cloudflare Workers, If you want to do you if you want to like I should say if you want to write the code in your Versus Code editor on your computer, what does that look like so that it You ideally get as close to as a worker environment as possible. Yeah. So when you,

run Wrangler locally, which is the CLI for CloudFlare, under the hood that actually uses WorkerD, the open source Cloudflare Workers runtime.

And so when you run Wrangler dev, You're actually not running a a node based development server. You're running a development server that uses our own runtime, And it'll actually pull in, based on, you know, compatibility dates, the, you know, ver the appropriate version of the runtime that you need. And so you get the same, the exact same thing that we're running in production when you're running locally.

And that's been a big effort that We've been pushing towards, and it's one of the many reasons why we open sourced the workers' runtime, because nobody wants that bug where they're like, Oh, yeah. Like, I was building this thing, and it was in Node, and I deployed it to this runtime and, like, why am I seeing different behavior?

And so that's why we've done that. Oh, yeah. We we are very familiar with That style of bug. That's why. The last week

trying to do Vercel only errors, Which was I know Vercel's middleware runs on on Cloudflare workers, but this was just regular service service. And we Here we were committing, pushing, sitting on our hands for 1 minute and 21 seconds.

I've seen red red errors for, you know, all day long, and yeah. I just know that Worker D was a thing. I I always thought it was Mini Flair that ran locally. What what is Mini Flair? Yeah. So, it's a bit of a funny story here. So Mini Flair kind of evolved

out of our need to, you know, have some form of local dev experience, and it was kind of this shim layer that said, Okay. What's running locally is node itself, but we needed to to have some way of exposing, you know, the way that the API that you Used to, grab something from an R2 bucket or to talk to queues or some other service, as well as kind of shim some different some differences along the way.

And as you can imagine, that got Bigger and bigger and more and more challenging to maintain, and so it became more of the direction we're going in now of, like, how do we just give you the actual thing, the actual actual runtime.

And we're actually trying to slim down that layer a little bit because, we just we can put more things directly in the runtime. We can give you Exactly what runs in production, and that's what's gonna give you the the kind of guarantee that you can trust. One thing I would shout out about before around, local dev being consistent is if there's anybody listening who's building a framework, out there, you know, we'd love to work more with The people at frameworks of all sizes and scales to make it easier for your users to Rely on the workers' runtime in local dev. So many frameworks have their own kind of local dev server that they've built into their projects, and When I talk to people, that's often something that's kind of painful to maintain yourself.

And we'd love to be part of that story and making that easier for you And having less bugs reported to your project, where developers are saying, Hey, like, I tried this in local dev, and I'm deploying to Cloudflare, why doesn't x work? We really think that if we We have some APIs that we can give you to make that process just so much easier and reduce reduce the burden that's on you. Oh, yeah. Cool. That's really cool because, like, Our our new Syntax site, we built it thinking

this will probably run on Cloudflare Workers, but You don't know until you deploy the thing and and yet, like, that's that's the point where people go, scrap it. You know, let's let's go put it and throw it on Vercel. So that's really cool that you can run that locally because then you get that immediate feedback. And also when you're building the thing and you hit a use a package that's not supported or A node API that's not supported, you'd know right away until before it's too too late. And we want to know about these errors.

We're not running Node. Right? But we do recognize that people do exactly what you guys described. Right? Which is Use a framework that relies on some random package, and we want to understand what are the APIs that we're missing from supporting that package because, even if you can work around it, etcetera, etcetera, I think there there's nothing better than, You know, hitting deploy and that deploy working on the very first try.

So,

don't be shy about letting us know about that. That's good. Because Because you do polyfill a bunch of of node APIs, right? Because the same thing with Deno is like ideally, yes, everything web standards beautiful, but real world, A lot of these packages like request JS, which is like hasn't been updated in 10 years. That's one of the most popular packages in the world, right?

Those APIs need to be Polyfilled for the time being. Yeah. And we actually go a layer beyond polyfilling where we're providing built in versions of those APIs that are native to the runtime.

And so if you take a peek at the Cloudflare Workers' runtime API's docs, there's a number of different APIs, including, say, Nodecrypto, buffer a couple others, and that's more the direction we're going in is rather than you having to think about your own polyfills or what you're putting in your own Code that's shimming something in.

How do we just provide that for you via kind of a compatibility

layer? Is there anything that, like, Anything right now where you say, oh, yeah, that. If you're if you're doing this in node, it probably won't work? Like, is there any, like, heads up that you should be thinking about?

Something like worker threads, like, it's probably not necessarily the best fit. Like, I would just say generally, like, we just want to hear the feedback, So don't hold it back. Don't think that, like, Oh, like, this will never work on Cloudflare Workers. The chances are that Mhmm. We're thinking of something along the way, And there's probably an idea marinating in somebody's head at Cloudflare, and we'd love to to meet with you and talk to you. Okay. And 1 I know we're running out of time, but I have so many questions.

You did a talk at one of the WASM conferences.

What's the story for, oh, I don't want to run JavaScript. Can I run Python on,

on Cloudflare? Stay tuned. We're working on some exciting things.

But I what I would what I would say is, like, you know, met with a bunch of people who are who have worked with Wasm, but also people who are just developers who are used to writing Python or Ruby or any other language and who are like, What's this WebAssembly thing? This is like, why is this harder than when I deploy to other places? Why do I have to think about, like, compile targets and all this stuff? And so That's really what we wanna go out and solve. It's like, you know, how does it how do you just think about uploading your Python script As if you're just writing pipe Python and you don't have to worry about all this cruft around WebAssembly. That's somebody else's problem. That really is our problem to to help you solve. The other thing I would maybe highlight, though, is, like, I do think there is an interesting opportunity with with web assembly, because we are Very, web standard centric and very browser centric in making some things that are today only possible in a web browser Also work, on workers.

So we've been working on adding support for the WebGPU API, and I believe, We're one of 2 run times to do so. Deno did some work on this maybe in the past to unlock some of the things where, you know, you can do, draw things to a canvas, or do client side inference and do that maybe in a worker, at some point. And so we're really trying to push the bounds there on, like, what APIs we can offer and bringing web standards into our runtime.

I like that a lot. Wow, man. There's so much I want to talk to you about, but we're running out of out of time here. The one thing I'll tell people to go and and research is all the AI stuff you just announced a couple of weeks ago as well. Like, that stuff is super exciting. Being able to just have access To these models with the import statement, like, if you wanna dabble with AI stuff, certainly check out the their workers.

Have you if I may ask, have you tried it yourself yet? Not yet. We, we did a show where we did a couple little updates on it, and I thought it was really nifty that you can simply just like, you have a version of a lot of these models. Right? And then you also did a hugging face thing? Yep. We are constantly expanding the number of models that are offered through us, and that's another thing that we're constantly seeking feedback on and are reacting to it pretty proactively.

And we've partnered with Hugging Face, so our goal is to start to support as many of the models in their Catalog as well as possible. Wow. Yeah. That that's really cool. We we want to do

we our new website is a bunch of AI stuff, but there's A long list of some things that we wanna we wanna especially around, like, images and and art around each of the episodes. So, I'm going to certainly dip into that next time I I build something.

Alright. So we move into the supper club questions real quick, Scott. Yeah. Let's do it. We got a couple of, quick ones here. What are

APIs that you're excited about? Could be browser APIs. Could be Just straight up JavaScript APIs. It could be anything. The WebGPU API, I know I said it before. I think this is super powerful. It's early days. The types of things that people are building in browsers are Crazy and being able to jump between client and server really easily. I just think there's there's tons of opportunity there. And what what other kinds of stuff Could you use that for on the server? So I was actually just on a thread with somebody who, has some some serverless functions Where they're doing kind of image processing and processing of GIS data, but it's very latency sensitive. And just the fact that GPUs are Fast at some types of compute, when you need that to happen in a particular place or where you have some latency sensitive needs, I think that's really powerful.

There's, you know, some lighter weight AI model inference that we've explored there.

There's rendering.

So, you know, I've talked to people who are building applications where they want to be able to, Do things that are kind of like CAD, but, less on the client side. But there's some interaction that Happens in near real time between client and server, and so be just being able to, to do that kind of visual modeling, without having to think about infrastructure,

I think is is a pretty powerful pattern. It might be a bit of an underdog API, but I think the cache API is actually really cool.

It's an API that's available within workers, and a lot of people, like, KV, a key value store can be really, really helpful for cases when you want to persist something. Sometimes you need to just stash something away in the same data center because you might need to come back for it.

And so cache API provides a lot of flexibility. You can do things like cache post requests.

Yeah. I I think it's really neat and solves a lot of problems, and people tend to sleep on it a little bit. So I thought I'd give her a shout out.

My, one nefarious use of the cache API is, cloud Cloudinary, which does image resizing and whatnot.

I put Cloudflare in front of that so you're not paying for pin on that. You know? So you could stay on the free plan. Smart. Yeah. That's just smart, Wes. And then if if it's if it's cash, you can you can cash it in, CloudFlare. Cost savings is a legit use case. Don't knock it.

Mhmm. Alright. So now it's the part of the show where we talk about sick picks and shameless plugs.

A sick pick could be anything, something that you just Enjoy in life. Could be a podcast, a TV show, a product of any particular kind, just something that you are into right now.

Do either of you have a sick pick? I'll pick a kitchen gadget. Yeah.

Okay. I don't like Accruing things, and so I'm very picky about which ones I choose.

But buying a bench scrape has been a massive improvement in my life. Just the efficiency of moving things from my cutting board to another is unmatched.

So

Shout out to my bench scraper. That's that's great. I I was teaching my daughter how to use a a bench scraper the other day because she's getting into baking, and the countertop is disgusting.

And I was like, you don't wipe this up. Use a scraper. And she's like, this is amazing.

It's so good getting flour off of Yes. Anything. Yeah. Yeah. Big fan. Nice. Good. That that was a sick pick. I like that. Alright. We'll move into shameless plugs.

Anything you'd like to plug to our audience audience before you,

part ways? You already touched on it, but the AI announcements that we made recently, we're really, really excited about.

So a few weeks ago, it was Clafler's birthday week. We like to give back a bunch of presents to our community.

And so, this year, we announced 3 things in the AI space. First is workers AI, which enables developers To deploy models in the same serverless way that they deploy code to workers today. So you don't have to think about scaling up GPUs or Even getting access to GPUs or GPU capacity or anything like that. As Wes mentioned, you just kind of pick your model and Type in dot run and there you go.

So, shout out there.

We also announced Vectorize, our database. So you can use that alongside the workers' AI embeddings, for building, you know, all of the Chat apps and rag use cases that we're seeing blowing up right now.

And regardless of what other providers you're using For your AI, you can use our AI gateway to get observability, caching, and rate limiting.

So all of that good Visibility into your AI deployment. So if you're using OpenAI and you're like, what are all of my tokens going towards? Really, really great tool for demystifying that. So definitely go play with our products.

I'm really, really excited about them. Oh, that's awesome. I forgot about the vectorized thing as well. We had built a, like, a question. We took every transcript and fed it into a embedding.

And we have all these vectors now, and I put them in some database right now, Zeta, but I was really excited to see that you launched a vector database because I'm sure that will be nice and easy to query and probably cheap. I don't know. I didn't look at the pricing, but I assume so. That is one of our Staples. Yeah. So yes. Great. Brandon, do you have any other

shameless plugs? I think I think Rita plugged All the right things, I would highlight the AI stuff, and please just give us feedback. Like, we really just want to hear from you. And more than anywhere I've ever Worked in my own career. Like, this is a place that we really read it and take it seriously, and so many of our internal discussions are literally like, Hey, this person is talking about this thing on Twitter sent a great GitHub, repro sharing kind of their feedback about our runtime, and so we really are listening and Value everybody's time they put into to sharing where they want our platform to go. Yeah. I've I've tweeted about several Wrangler things, and they have all been

fixed. Like, I I remember once I made, like, a a worker on the website, and then I wanted it locally or vice versa. Like, I wanted to push it, and they're like, well, You kind of choose 1, and that's where you work on it, right? And now you can sink them or push them. I forget exactly what it was, but Fixed. So I appreciate all the feedback. Where where should people

give you feedback? We actually really love getting feedback on our GitHub repositories.

Find that some of the best feedback comes from there, and people give great, like, reproductions.

So, Cloudflare /workerd for the workers' runtime and Cloudflare/workers DK for Wrangler, the CLI.

Those are great starting points and lets us have kind of an open dialogue with you and other people to chime in and support along the way. Also in our Discord,

we have a great community there both to ask any questions that you have. Someone will answer them pretty promptly and also provide feedback On our products, our PMs pretty actively are looking in there and scour it for

any past Feedback when whenever we do start building our features. So Beautiful. Well, thank you so much for all your time. Really excited about this one. I think people are really gonna enjoy it. Yeah. Lots here. Thank you. This is great. Thank you for having us.

Head on over to syntax.f m for a full archive of all of our shows.

And don't forget to subscribe in your podcast player or drop a review if you like this show.