You Need Stale While Revalidate

Welcome to Syntex. On this Monday, hasty treat.

We're gonna be serving you up some stale, takes and then revalidating them. So we're gonna be talking about stale while revalidate, which is a caching technique that is really pretty simple to add to your projects to really improve overall speed And really make life easier on you when you're trying to get some caching down that, well, still requires information to be up to date. My name is Scott Talinski. I'm a developer from Denver. With me, as always, is Wes Bos.

But before we get into the contents of this episode, If you're, you know, caching anything, you gotta make sure you have some error and exception handling traffic on your your services because, hey, Sometimes that cash can get in the way for some reason.

It can cause some problems. You might need to know what's going on. But more importantly, if you have something like centuries replay where you can see what the user's getting on their dashboard. You can even see if they were accidentally getting some potentially cached or incorrect Old stale data.

So, head on over to century.i0 and use the coupon code tasty treat, all lowercase and all one word. You'll get 2 months for free. All right, so let's talk about the stale while revalidate.

This is a header which you're probably thinking I don't need a podcast about headers. This is such a cool header. You're going to want to listen to this one, and we're going to explain, first of all, like like why you might need it and then and then how it works. So For the purpose of this podcast, we'll talk about 2 examples.

First, one of them simply just being an HTML page that when visited.

That HTML page needs to be cached so that the database queries, the rendering, all that stuff that needs to happen on the server doesn't need to happen all the time.

Second 1 is the open graph images on the Syntax website.

So those take a while to generate and we need to cache them so that Once you take the 15 seconds to generate it, they're going to be loaded instantly after that. However, like we're constantly changing the names of podcasts. And like, you don't want that data to be cached for too long in there because then you have to have a whole whole thing where you invalidate your cache. So those are the 2 examples we'll refer to this, But you, the listener, can think of any time you want a asset, whether it is an HTML page, an image, some JavaScript, some CSS.

If you want that to be cached for some amount of time, then you might Want to dip into using the stale while revalidate header. So why? Scott, you wanna grab the first one? Why might you want to cache something?

Yeah. Well, the thing is with caching is data oftentimes doesn't really change that much. You know? You can think about it. If you have a CMS, You update some data. Right? And you're serving that same data to a lot of people.

And whether it is caching your your server response or your data.

If you think about caching as a whole, just a concept, what you're doing is you're basically saving that information So that way when someone else requests it, it doesn't have to do the expensive operations, and it can just serve up The already saved data typically could be saved in memory, could be saved in your browser's cache.

Caching is a technique used just about in every single facet of computer science. Right? You're often needing to save things for later so that they're easily accessible.

Yeah. That way expensive operations don't get run more than they need to. Exactly. Like, you you might have a

Page that takes 800 milliseconds to do database query.

But what if that was 0 milliseconds for. Yeah, right. For absolutely everybody. Right. If it's the same database query or even if you if you only cache it for 1 minute, you know, that's you can do that.

Another one is you can Sorry. Yeah. Sorry. Let me even just hop in there too to make things a little bit clearer for people who don't have any experience with caching. Think about images. Right? You have a a big old image.

And if you download that image once, that takes some time.

And if you cache it, It stores that image on your computer. So the next time you click refresh on that web page, it's not having to go off and repull it down. It's just there. It already exists.

Another use case for caching is you can put the caches closer to your users. So, Akash can live in 2 spots. It can live on your computer, which you visit a website and then you go back to that website later.

Your like Scott just said, you don't have to redownload that, but it can also live on a CDN so you can put a CDN in front of your assets or you use something like Cloudflare or Vercel.

They will automatically CDN eyes a lot of your assets And they'll say, all right, well, you generated this Open Graph image. Let me go throw a copy in Australia.

Let me throw a copy in Boston, And then that's gonna load fast for people

all around the world. Yeah. Well, they can also be a cache can also be on that just a straight up server too.

Stuff. Just, you had said a cache can be in 2 places on the browser or a CDN. I just wanna make sure that that's clear. They can all stop by yeah. I'm talking about, like, the actual cache header. So you're right. Your server

itself can respect those cache headers.

But generally what you're doing is, yeah, you're sticking a CDN in front of it. Yeah. Sorry. I was also speaking on a cache broadly rather than. Oh, yeah. Yeah. I'm more thinking about like like a cache header here where because stale while revalidate is a header that you set on your responses, then stuff? It's either I don't even wonder. I think probably like your ISP might even might even do caches as well. I would think so. Yeah, it's a question. We should ask the Cloudflare folks that. They would probably know that. Hey. I got a an interesting sidebar question. Yeah. You know when they have, like, Movies on an airplane that are stored onto a hard drive. Do you call that a cache? Oh, yeah. For sure. Yeah. That's a, like, a local cache where it's sitting in. Like, Like lots of ISPs have Netflix caches, right? Yep. So they don't have to go all the way to Netflix to do that.

Another reason for a cache is you might limit how often you're allowed to hit an API. So if you have and maybe, for example, you might only be allowed to hit an API 30 times an hour, So you might have to cache it for 2 minutes. Otherwise you're going to run out of API hits, right? You can't be hitting at every single request. So it's good to You can either cache like the output or like Scott said earlier, you could also just have like an object or a key value database. But that's not really what we're talking about here. Talking about that, the actual outputted header.

Or you might be doing something expensive, such as generating images. So Every time we generate an Open Graph image, it takes about between 8 and 15 seconds, depending on if the Headless browser is hot or not. Basically, I kept the browser around so that if you come If you try to generate another one, like 10 seconds later, the browser's already started up and that that'll cut 5, 6 seconds off of everything.

But if you think about that, like, oh, I'm generating hundreds of thousands of these for every single request.

You're paying the bill to generate all of those. So it often makes sense to just generate it once for however long and then

serve up the cash version. Yeah. Well, okay, let's get into then just straight up normal caching headers, which you typically see in just as max age or s Max Age. And that basically is a is it is it it's total seconds. Right? That's how what a Max Age is said too? Yeah. It's It's it's one of the few things that are seconds based. Yeah. I know. I wanted to say milliseconds, but then I've made a max age calculator for myself because I was so annoyed with having to calculate it out all the time. Yeah. And so I'll post a link to that. I made it in Svelte if you wanna see it.

But, like, Basically, it's how many seconds you want this thing to be cached for before, the browser

Or the CDN goes off and gets a new version of it. And the difference there between the the 2 that Scott just said is max age is going to be cached on your browser, whereas SMAS Max Age, the S stands for shared, and that will tell your CDN. It is okay to share this between multiple people. So one possible I've seen this a couple of times security issue is If you accidentally s Max age somebody is like logged in state, you could possibly be serving up They're logged in state to the next user that comes because you said this is a shareable cache. So you kind of have to be careful with that because you have to think, okay, If I'm saving a copy of this user's rendered out, like, let's say it's your eBay, you're logged in and you see all of your account details.

If you if they cache that on a CDN, then the next person that comes along is going to see the wrong logged in state.

And that's a security issue there. So certainly be careful. Don't just think, oh, s Dash is Like, super or something.

Oh, yeah. S dash is better because it's on the CDN. In in that regard, if you wanted something cached for both of those, Would you use both of those headers, Wes, or just 1 or the other? That's a good question, actually.

I would think that S Max age is it will both be respected by the CDN as well as the client. But let's pause a second, double check that.

I asked Chat GPT, it says yes.

I'll tell you this. I've never set both of them on a single response, and I've done this a lot. So I would assume so that it would be cached both by your browser as well as, And you can tell that pretty quickly by I guess we could go to the the syntax website or go to the syntax. Fm/og forward slash 600.

JPEG and that will give you the actual rendered out open graph image. And then you can look at it In your network tab, see if it is cached.

Oh, but then I have the dev tools disabled cache on There We Go. Yes, it does. So S Max Age will cache it both at a CDN level as well as at a browser level.

So again, if you're.

Which makes sense because you're you're not spending the bandwidth to get to the CDN If you already have it downloaded into your browser cache.

Now enter stale while revalidate, right? The Max, you give an access a Max Age header. That is how long the asset is good for.

Then you give it a stale value.

And the idea with this is it tells the client How long it's okay to reuse or serve up a stale asset.

So in the example of the Syntax Open Graph images, We were hitting 10 seconds to generate the image, and then You could put like a let's say we put a 1 minute max age on there. So every time for the next minute you refresh the page, You're going to get fast reloads, but after 1 minute, that is that asset is considered stale.

So the browser goes back to the server and says, Hey, I need a new one. This is a stale image, and then you got to wait another 15 seconds.

Stale while revalidate will do is you can say, Okay, this is good for a minute, but for up to 10 minutes past that, It's okay to serve the stale one up, right? This is stale bread. You can still eat it for the next day or so, but we should probably make some more, Right? So stale while revalidate will just say, Oh, oh, shoot. Okay, all I got is this stale PNG of the the open graph image. You can have it for now because it's still fine. They said it would be fine for the next 10 minutes. It's okay. But It's okay. I'm going to go out back and make a new one.

So the next time somebody comes, I'm going to have a fresh one for them. And then I've renewed the max age on there. So what it does is it kicks off a background task and it doesn't make the user it gives you the stale version immediately, which is often fine.

And then it'll kick off a background task that nobody is sitting around waiting for, and it will regenerate a new version. And then you come back A minute later and you've got the new version instantly. So the idea is, and this is the same with like a React website that would be rendering, is that If you visit a page, you might be getting stale content, but you should know that in the background you're regenerating a new version And you are never going to be sitting around, sitting on your hands waiting for a database query that is slow to resolve or a weird rendering or a download of hitting an API that is really slow. So all of those things can happen in the background while you're waiting for it. And that's what the stale revalidate says. Keep it for this long and it's okay to serve a stale version within this many minutes.

But if I'm serving up a stale version within this many minutes, I'll go make a new one so the so we can have fresh.

But, you know, you gotta feel like it it has some degree of accuracy there because I think the reason why that metaphor is perfect because it also the use case in which stale while revalidate probably isn't a good idea.

If you were to be serving up stale bread to the queen Yeah. Or just somebody. Right? Somebody of highest importance. The the bread must be at its very best. This bread must be perfect.

Then at that point, You're going to probably not want to serve stale content to those people.

So there are instances where The fresh content always needs to be there. And in that case, yeah, you probably you know, you're you're gonna be Finding alternative caching methods or caching for much less time. Yeah. Maybe even, like, on the seconds. I I've I heard that, you know, you probably don't wanna use this Content that's updating really quickly, like like sports scores that need to be served. Like, if you go to, like, espn.com And you load the site. You want the scores to be exactly up to date all the time, so you're probably not gonna wanna serve stale Scores TO Users? Yes.

I think the way that you said it is it makes sense.

Sometimes the fresh version is worth waiting for.

Correct.

But also, like, with the let's say ESPN, let's say every 3 seconds They are.

You could cache the scores for 3 seconds and then.

Say still while revalidate after another 3 seconds. So what is happening? ESPN gets A 100,000 traffic visitors in one second. Right.

That's not uncommon. Right. Lots of people watching hockey. Lots of people refreshing wanting to know the score that if you get 100,000 people a second, that's 300,000 requests that are being served up as cached. And then once you get over that 3 seconds, you're on the 4th 2nd.

Now people are getting stale content, But it's only 4 seconds old, and there's been a background thing kicked off to regenerate the latest version of it. So It really has to do with like how much traffic you're getting with the syntax website, the old syntax website.

We had a one second stale while we validate on it. Why? Because we needed a way to regenerate the website for the newest shows, right? Because there was no like cron that would automatically populate it. So what was happening? But we didn't want every time you visited the website, we didn't want the person to say, Alright. Well, I got to sit here and regenerate the whole website.

And just in case there's a new episode. So what we did is We put a one second still while we validate on it, meaning that you visit the website, you get the cached version, but you immediately kick off a regen for the next person.

Next person comes along. They get your the one that you you generated to be cashed, and they kick off a generation for the next person and the next person. So you're always kind of off by 1 in that generation, meaning that You're getting one second old content, which might be okay, but you're also getting it instantly because you somebody else kicked off the generation for you That that's been cooked beforehand. Why wouldn't you just toss stale while revalidate

on everything?

Stuff? That's a good question. Like, first of all, you might not be getting that much traffic and you don't want to have to be dealing with with caches?

I mean, they're pretty simple.

There's there's stuff. Yeah. Yeah. It's true. May maybe there is, like, a use case where one second is not long enough or sorry, is too long like Imessage, you know, or you're doing like a Zendesk back and forth. One second could be kind of annoying to get that type of thing. And in that case, The caching doesn't really matter if it's rendering fast enough for you. Do you really need to sit around and wait for cash and have this like off by one error? Stuff.

And in that case, you also need enough traffic to kick off those next renders If you don't have enough traffic in the example of the syntax website that I gave you, if there wasn't enough traffic for somebody to kick off the next regenerate, then Then you're just sitting there waiting for the site to generate. And it only took a couple of milliseconds or 100 milliseconds or whatever to generate the site. It wasn't really that long, but It's still the difference between a couple 100 milliseconds and 0 is quite a bit.

So, yeah, it's not something that I would just I don't know. Like, Should you throw it on anything? Everything

for me? I really feel like it's sriracha at this point or something or whatever people just like to put on everything.

I'm sure that there'd be some some issue. And also, like, the one second example, Like it's got to take longer than that to propagate. If you've got a CDN network, let's say if you have a CDN like Vercel or Cloudflare and they've got 50 locations, you know, How long does it take to take that HTML file and propagate that through the world? So I don't know if the one second would be ideal in all of those cases because you have to literally wait for it, but even even a minute would make a lot of sense.

Yeah. I know. I well, that's my biggest question is is, I guess, why would I just not use this all the time? Because it feels like I want to. Yeah.

If you were if you had, like, a, like, a Twitter or a YouTube where you have, like, the header and it has, like, notifications in it, Like you probably don't want to cache that because now you're spending CDN space to cache that data Interesting. For absolutely every but you could Just cash it on the on the user's client as well. Word. Cool. Yep. That's it. I thought that's I told Scott this would be a 10 minute episode, but here we are 22 minutes in.

I think we had to get our food metaphors in. Exactly. We got it.

But tweet us at Syndax FM if you have the different caching headers, which is episode 464.

So go to syntax. Fm/464.

You can listen to the episode Cash control headers explain.

We go through all of them.

Immutable, no mutable, private, stale if error. That's another one is like, Well, what happens if it breaks? Are you allowed to give them the old version for the next 10 minutes? Max age? Stuff. So listen to that, but I thought we we should go entirely in-depth just for stale while we're validated. Yeah. Absolutely. I love it. Alright. Peace.

Head on over to syntax.fm for a full archive of all of our shows, And don't forget to subscribe in your podcast player or drop a review if you like this show.