How We Built a Netflix Style “Save for Offline” Feature Into Syntax

Hey. I am really excited about this because I did not realize that this was the EPA that you needed to to store stuff locally, in the browser.

I always thought it was, like, file system APIs or IndexedDB, and turns out that there JS a really nice cache API that's not just service worker. I've used it plenty of times as service workers, but not for storing data in the browser. And I've always wondered myself, like, I would love to be able to, for my courses, like, save them in the browser,

It's any any of the storage APIs on your browser. Yeah. Because there's there's lots of ways there's lots of ways to store data in the browser. Right? You have local storage. You have IndexedDB.

You have what are the other ones? Cookie. I just sent you a whole list. Is cookies coming from that? I guess that's not right. Probably not. Local storage, cache API, IndexedDB, service worker, and file system. So you talked about the file system API.

We're talking about the cache API here. IndexedDB is kind of like local storage, but it's a bit more flexible for doing, like, full blown database stuff, and then local storage is just a nice key value store. What I'm what I am curious about this cached API is that it's you said it's based on request and response, meaning that those are web requests and web response. And and that's generally, the way that works with with a service worker is that when the browser requests like a a CSS file, you can write a service worker to jump in the middle there, and then you can say, oh, I've I've I've already stored this and and send it on back because it's in the cache API. But when you're not in a service worker, that's primarily a fetch request. Right? Mhmm. The thing you need to think about this is that it it is essentially

I'm in the dev tools right now for Riverside as we're recording this, and they are sticking it in, IndexedDB.

Yep. And you can see the little chunks. Like so there's a there's a API in the browser called media recorder. Node sure if they're using media recorder or not, but the get user media will give you chunks of the user's video, and you can put those into the browser storage so that you can upload them as you have as you have space to do so. And I was always curious what that looks like. And I guess the reason why they're using IndexedDB here instead of the cache API is because, like you said, cache API is only for Wes ESLint response. Right? It's not for, oh, I have a I made an image in the browser, or I have this canvas element that I wanna be able to persist, offline in in when I refresh. If it's not actually a a request to a server, those the cache API is not for that. Right?

a request. Right? Yes. So it it makes sense that you would request you would save it.

That is when the data is gone because you cannot promise that the data will be there forever.

We'll talk about persistence in in just a second, but, eviction is when, at some point, your hard drive is going to get full. Between all of the origins, which is like a domain name on a browser, you're going to be using too much of the space, and there's only so much that Chrome can be allowed to use on your user's computer.

And when you get close to that limit, the browser will just purge it for you. So I always like to think of these caches JS, unfortunately, you can you can keep them there for fairly long, and it it works fairly well. Like, we use Riverside, and I don't think ever we've lost a recording. Maybe maybe once.

But at a certain point, the browser will purge it, and you cannot guarantee that it will be there forever.

Oh, that's it's it's probably because it was pulling it from your browser's cache initially.

Yeah. It's it's interesting. I'm not quite sure why. The whole response and, like, streaming thing is is really funky. I was doing that with the the Syntax website as well where I had to, like I was trying to modify a response, and Node was not letting me because I was trying to, like, modify a header. So I essentially had to, like, clone the response and then loop over every single header and add it in manually Yep. Because it was it was kinda funky.

That's awesome. I was just looking up your streaming thing that you're asking about, and I remembered that we did a show on on streaming. And when you want to both use a stream for something, like save it to the cache and view the progress, you can't do 2 of those things at once. Right? You're either you're either saving it or you're viewing the the actual progress. If you do want to see both of them, you have to use the dot t method, and that will basically t split it split the stream. In in one of them, you can create a reader that will see watch the progress, and then the other one, you can throw you could probably just pass directly into the cache API.

Although adding that metadata, it might get a little bit angry that you're trying to modify the headers Yeah. Before you put it in. That's probably the issue you had.

Oh, that one. What what's that issue you just got? I think it's related. Failed to fetch MP 3.

Contains an ISO. Non ISO code point.

Yeah. This one's working. There Wes go. It's probably there's probably something in the show notes that is not a string of fine. Like that. Okay. Yeah. You're doing JSON Node string of of the show and there's probably something in there. And if you're trying to set that to a header, you're trying to stick the entire show into a header as storage. Mhmm. It's probably

Yeah. But But even even then, if if somebody's name has, like, a an accented character in it Yeah. Yarn you gonna hit that? So I'm trying to think just trying to look at the the show notes if they're what it could possibly be.

Node it, but then you got all those problems as well.

Yeah. Nice.

essentially for MP 3 files. Pretty neat. Awesome. One last thing we wanna talk about is just persistent storage.

I was always curious about this as to when does it get sort of pushed out. And we talked about the browser.

Once it gets full, it starts to sort of purge.

And I found some interesting stuff on online about this. I'm just going to read through it. This is as each origin gets a higher storage limit by default, the browser will evict data by origin, which is a domain name.

When the total usage of all origins is bigger than a certain value, the all the overall quota is calculated based on disk space.

So, again, it's it says, alright. Well, now Chrome is taking too much of of disk space. What do I do about that? However, at at that point, the browser says, alright.

What do I delete? You know? Do I just start deleting random stuff? Because some of it will be, yeah, you you cached a CSS file.

It's fine. We'll just go download that again. And some of it will be more important stuff like, oh, yeah. I I saved a recording that hasn't uploaded to the cloud yet, and the only place that exists is in the browser cache right now. So that's that's pretty important. Right? So there JS an API called navigator.storage JS the API that will allow you to see how much space is available for you to store stuff, and navigator dot storage dot persist is the API that will it used to, like, pop up a thing and says, hey. Can we store stuff in Chrome? And now it's just based on flags, like how often you visited the website, if you've bookmarked it. There's all these things. Kind of the same idea of when you wanna autoplay video. That sounds good to me. Scott allowed to autoplay video unless the user has interacted with the website a whole bunch. There's just, like, a a whole there's a whole bunch of stuff that goes into calculating the score of a website. So I had, now you can Scott just say navigator.storage.persist, and it's it's kind of like a pretty please, can we have elevated, storage? Like Yeah. Like, don't delete us first, please. It doesn't pop anything open, and it simply was returned true or false. So I ran it on Chrome on Riverside, and it it returned true. Why? Because I'm here often. I spend a lot of time on this website. I interact with the website. I've bookmarked it, And then I tried it on a website I've never been to, and it obviously says no, automatic. So I don't know what those flags are by browser. I think it's it's up to the browser to decide what is important and what is not.

But I thought that was kinda interesting.

Alright. So I'm in Chrome forward slash or colon slash slash media dash engagement.

And this will actually give you the information about how much a user has interacted with it, and it will give you whether it is a high media engagement score or not.

And if you have a high media engagement score, that that is one of the ways that you're allowed to autoplay videos. So look at this. YouTube, 16 sessions.

6 of them, I actually played something.

Last time I played something was January first just because I I don't use Chrome for anything other than recording this podcast. And then it but it is a high media engagement score. Right? And then all of these other websites here are just, like, domains or origins that I have interacted with, and it has a low media engagement score. I'm kinda curious if I look it up. I I switched to Microsoft Edge.

the play method has switched to a promise.

So what you can do is you say await video Scott play or audio dot play.

And if you're not allowed to play, you catch it. So just put a dot catch on the end, and and then you can you can say, oh, it's not done. And, yeah, the the real trick, though, is just just listen for any any click at all on the website and play something at that point, and then you're you're in.

flag. But once you click the button that turns on autoplay, there's your click. Yeah. But what if they refresh

doesn't work.

Then they have well, no. It it that's a good a good point.

I tested it when I was doing my autoplay, and it was very easy to get a high enough media engagement score.

I think I think the user had to to watch, a couple minutes.

No. You can do it if your media engagement Scott is not I Node, but it doesn't like it.

No. You're no. You're wrong about that. As long as your media engagement score is higher, and I'm reading it right Node, consumption of media must be greater than 7 seconds.

So as long as you can play something for 7 seconds that is in unmuted state, you can you could do that. And I bet you could get around that by playing a permission flag. Just give me a yes. Give me a yes or no. Come on. Yeah. It's true. Are you are you going to allow this? Because, honestly, there's most websites I don't want autoplay on unless it is actually a video website like, like YouTube or a core my course platform.

Another problem. Permissions like, you see this on Ios all the time. They pop a permission up, and they try to do, like, a fake permission. Because if you're gonna click no, they don't wanna show you the real one because then you gotta tell the user how to go into your settings and and turn it off. Like, once they hit no, you're it's over. There's no normal people are not turning that setting back on.

no fun stuff. Permissions is a hard thing. Yeah. Alright. I think that's good enough for today. Hopefully, you enjoyed that. We'll catch you later.

Peace. Peace.