Skip to main content
793

July 10th, 2024 × #local-first#offline#speed

The Local First Landscape

Discussion of various tools and libraries for building local first web applications

or
Topic 0 00:00

Transcript

Wes Bos

Welcome to Syntax. Today, we've got an awesome episode for you today on the local first 1st landscape. There are lots of libraries, frameworks, helpers in the Local 1st space, lo fi as as a lot of y'all like to call it.

Welcome to podcast discussing local first landscape and related libraries/frameworks

Wes Bos

But sometimes they do a Scott, and sometimes they do a little. And today, we're gonna come at you with the explanation of, like, what does the local first landscape look like? Real quick, before we get into it, let me tell you about a behind the code discussion with back end experts that's coming up, with Sentry. It's on Tuesday, July 16th, so about a week after you're listening to this. And we'll have Taylor Otwell from, he's this, obviously, Deno founder of Laravel, Paul Cobblestone from Supabase, Soren Schmidt from Prisma, and Gits from Node technicals TSC. We had him on the Node, and Sarah Guthals from Sanity. Check it on out. We'll have the link in the show notes. You can sign on up, and it's gonna be an awesome awesome little panel.

Wes Bos

How's how's it going today, Scott? Hey.

Scott Tolinski

Going good. Yeah. Summertime, kids are home. You know, you're doing all that stuff.

Scott Tolinski

Yeah. Just filling, filling all that kids camps and all the school stuff. So yeah. Yeah. I'm sure you guys are feeling it too. You're out at the cabin getting that summer life in. Oh, yeah. We got our kids Yarn, at camp the next 2 weeks as well, which is like I never went to camp as a kid,

Wes Bos

and I always hated the camp kids. I didn't hate them, but they camp kids always talk about camp.

Wes Bos

And, like, the all their jokes are about someone at camp, and, like, they it's it's like somebody who does CrossFit. You know? They just cannot not talk about camp.

Wes Bos

And now my kids are going to camp, so I'm kinda curious if they're gonna become those kids.

Scott Tolinski

Yeah. And it seems to be a thing. I don't remember this when I was a kid, but it seems like especially, like, with working parents and stuff, it's like, alright. Summer's here. Kids going to camp.

Wes Bos

The camp is nuts. Like, my wife's in January. Like, 1st week of January, she's on group chat with all the other moms All the moms. Being like, alright. Camp is coming up. You know? And, like, some of them will get like, my wife's friend has 4 kids. I guess we have 4 kids as Wes. But so she'll get her grandparents and her sister, and everybody will be up at 7 AM. Everybody's got a computer in order to, like, get them registered in time for camp, and it just it's a wild world.

Scott Tolinski

Yeah. Luckily, we have a we live by a university, and Yeah. The kids' camp stuff that they have at universities, at least at ours, is so good. Like, he can learn ice skating or learn all or learn play soccer all day or or go and do different sports at the college.

Scott Tolinski

And then most of the the teachers are, like, student athletes. So, he could get, like, a nice bit of experience lacrosse and hockey and swimming or whatever. You know? Yeah.

Scott Tolinski

Yeah. So let's get into the local first stuff. You know, I've been I've been diving deep into this for a little while, and I've been exploring this space for quite a bit now. But I really got my hands dirty with some applications, and I'm gonna show you some really primitive and some, less primitive applications. I'm gonna show you, different libraries. I'm gonna show you real code. I'm gonna show you the working applications running in my local host. So I have a lot of stuff to show. I wrote a blog post on this. It's on my my website. I'll link that up. But we'll basically be covering a lot of the same ground here. Basically, local first.

Topic 2 03:32

Local first basics - data runs locally in browser before syncing to network

Scott Tolinski

You know, we're not gonna get too much deep into the the deep description of it because we have a whole episode on that. Check it out. We'll link that up in the show notes as well. Local first is really just this this idea of your application is running the data locally first. It's running your application's data directly from the the browser. It's not having to load it off network.

Scott Tolinski

Just like how your phones use a Sanity database on the phone Wes it uses a native app, it saves directly to that. That's why you're not having to wait for a network request. You're not having to do things like optimistic UI because the UI is super fast already. It's saving it directly to a file on your machine.

Scott Tolinski

And then the network sync all happens in the background. You tend not to think about it. And my my thoughts on this generally is that most of the best applications you're using today, the stuff that you use and you say, dang. This is fast. This is good. Yeah. JS built with this technology,

Wes Bos

especially if it's a web app. And Do you have any any examples of of local first apps that you use daily that are just like,

Scott Tolinski

So I I don't use Linear Daily, but I have used Linear, and that's the 1 that I know a lot of people use that's local first. An app that I have been using locally daily is called the tweak Scott Wes o, tweek.s0.

Scott Tolinski

It's a to do list app that is incredibly awesome.

Scott Tolinski

It's super duper simple, but very fast.

Scott Tolinski

Superhuman is a email client that a lot of people use that's local first.

Scott Tolinski

And a lot of these things are really just taking the ideas from native app development and applying them into the web.

Scott Tolinski

And because of that, you get applications that load instantly. In fact, my habit tracker, Wes, that I've moved over now to being local first on my machine, it loads in 50 milliseconds.

Scott Tolinski

Wow.

Scott Tolinski

Absurd.

Scott Tolinski

So the loading times are in very, very fast. Right? Because you're not having to do a network request before your application loads.

Scott Tolinski

The local is private because it's stored on your application. Unless, of course, you set up syncing, then it becomes a different story. Right? Then you have to worry about that.

Scott Tolinski

If it's just local, then you don't even have to worry about auth because auth is not a thing on your own device. Right? You're already authenticated into your own device, your own browser, whatever. So Yeah. Pass that on to the the device or browser, like you said. Yeah. And many of these applications JS the way they work is they'll let you get started. And then the moment that you sign up is when you wanna start syncing your data off of the device. Right? So auth still becomes a thing, and it's still a thing you have to think about. But, if you're just running locally, it's it's a different kind of story if that data runs locally. Either way, there's a lot of things that make something local first, and I highly recommend checking out local first web Scott dev and reading all this stuff, watching some of these videos, or just listening to the episode we did on it. We cover it pretty well.

Topic 3 06:16

Many local first tools available, some do everything while others are more flexible

Scott Tolinski

So that feels like, cool. I got this. Right? Then you get into the build side of this website, and you see all these different tools. And you start clicking on them, and you start looking at the diagrams and wondering, oh my. I'm in over my head. Right? There's a lot here. There's there's language like CRDTs for conflict resolution. There's syncing engines. There's different types of local databases, and it can really make your head spin. So I I gave a handful of these a try, in a real I use a SvelteKit code base, but you can use anything. Doesn't matter, obviously.

Scott Tolinski

I gave a lot of these a try, and we're gonna start kind of from this side of things where the platform is doing the very most for you. Okay? Because that's the way this works JS that some of these do everything for you. ESLint, they store your data. They sync your data. They handle the data on the client side. All you do is write a query and you write a mutation. That data changes. It's synced. It stored everything.

Scott Tolinski

And then there's other ones that say, you already have data or you want more control over your data, where that data lives or how that data lives. And what we're gonna do is we're gonna provide the the storing it locally and the syncing of the data in the conflict resolution, or they're just going to handle that conflict resolution, or they're just gonna handle the syncing. There's, like, so many different little facets here. So they either go from doing a whole lot for you to very, very little depending on how much control you want over your data and how your app works. Do you think that

Wes Bos

there are so many options right now because people want to integrate them into existing systems? Is there a lot because it's we're early in the sort of lo fi system? Like, do you like, is it eventually gonna be like, oh, I need state management for React? There's probably 3 or 4, like, solid options. You think we'll ever get there, or do you think that it's this is a little bit more peeled back than that?

Scott Tolinski

Yeah. I I think we'll get there. I you know, what I think is is more likely is that a lot of the platforms that we might use might implement these types of features into into their platform, or perhaps they link it in a like a plug in type of way. You're gonna see many of these things have a Supabase, like, add on, and that it's like if if you're using Supabase, you might be in the best place to to utilize any of these things because, Supabase has a lot of connectivity with a lot of these things. So, in in other ways, yeah, people come from having a lot of their own data or their own systems. And as somebody who just converted a website that was not built to be local first to be local first, that's tough because JS we'll get into, you have to deal with, like, row versioning.

Scott Tolinski

You have to deal with, like, how does the content actually delete when you have these types of systems, the patches, all these types of things. The history of the data doesn't exist if you're applying it to data that already exists. So it's a whole world. Right? And and so let's start at the very most Bos. And so this is 1 that I use called Triplett.

Topic 4 09:24

Triplett handles schema, queries, syncing, and back end in one full package

Scott Tolinski

Triplett is really pretty neat.

Scott Tolinski

Looks like they just had a new release. But Triplett is a do everything for me type of platform. It's the full stack database.

Scott Tolinski

So this thing is really pretty neat. It's neat because it is open source. You can host it yourself if you want. I am hosting my own.

Scott Tolinski

It gives you offline support, a real time sync. It's TypeScript and everything like that.

Scott Tolinski

And for the most part, you're pretty much just using their query syntax to do everything.

Scott Tolinski

So you utilize a a query syntax, which, you know, it's their own query language, but it's it's ultimately pretty pretty easy to parse. You're gonna have to yeah.

Wes Bos

JS this a hosted thing that you no. It it wouldn't make sense if it was a hosted. Why is there pricing?

Scott Tolinski

So there with many of these things, you need 2 types of server instances.

Scott Tolinski

So they can be often hosted on the same server or a separate server or whatever. Okay. But you have to think about it this way. You have 1, the syncing server, that that thing is handling the conflict resolution and handling the syncing side of things. Then you have your actual server that's storing the data.

Scott Tolinski

Right? It's you're handling your database fetches and updates.

Scott Tolinski

Okay. So many times, those are the same thing. And in triplet, it's the same thing.

Scott Tolinski

So in triplet, the syncing server and the data server are all 1, and it's kind of like a black box.

Scott Tolinski

Triplet understands that, you know, to build an open source project, maybe you need a product behind it, and that product is a fully managed 1 of these things. So a fully managed data host that includes a nice little back end by you too. Like, the, UI has, like, a full full, like, table back end type of deal. Okay. So this is like

Wes Bos

triplet would be if you were to, like, look at existing database solutions. It would be your your MongoDB or your MySQL, but it's also your, like, drizzle. Right? Like, your Yes. Your schema definition in in query language.

Wes Bos

But then it's also your, like, 10 stack query or whatever, which you use on the client side. Like so it's it's everything, which I guess is why you're you're doing this 1 first.

Scott Tolinski

Exactly. It's everything. You Sanity see this is what, like, creating a schema and something like this does. Okay.

Scott Tolinski

And this is a good option if you, 1, trust that this product's going to be around in Node time. Granted, you can host it yourself, and there is, like, active development on it. I've been in their Discord. I've been, working on it. It's really nice. If you're starting a project from scratch and you want the easiest possible solution, both this 1 and the next 1 I'm gonna show you are it, because they work very well. And a lot of the docs show react, but, like I said, this is a SvelteKit site.

Scott Tolinski

You can see hi. Add. You can see it adds it instantly to my list. Not only does this add this Can you open the network tab and and do 1? I'm curious what that looks like.

Scott Tolinski

So I'm not even clearing this.

Scott Tolinski

So you can see nothing in the network tab. No requests.

Scott Tolinski

We do have this WebSocket transport that's always open.

Scott Tolinski

I don't Node. Did that show you the messages here? Yeah. Let's see the messages in the WebSocket.

Scott Tolinski

Oh, come on. Where are the little grabbies?

Wes Bos

Finding the grabbies is is 1 of the hardest things. Yeah. If you have a Oh, there you go. Thick line.

Scott Tolinski

Okay. Yeah. So here it is. You can see it's being done over the WebSocket here. So that's really nice and easy. Right? And if we take a look at the application, this is the triplet. We have triplet cache, which you can see looks interesting, and the triplet outbox, which again looks interesting. Many of these things look interesting because they're not handling data in the types of ways that we're used to. They're they're thinking about, like, the collective patches and things of the data. Like, what is the history of the data? When I do something, it's this has been added to this, has been added to this. Like, what's the the version of the software? There's a lot more here than just like, here's a database full of all the data, which is why these things are so tough.

Scott Tolinski

So this thing, as you can tell, if I refresh it, you can see it's persistent.

Scott Tolinski

This is being stored locally. It's being backed up to a database off-site that I'm running on my own Coolify as well, and that's all being done.

Scott Tolinski

All I'm doing is instantiate instantiating a triplet client. I'm saying, use IndexedDB for my storage mechanism mechanism. You can actually change that if you want. And then inside of my page, dot Svelte, all I'm doing is running a subscribe. This this code is dead simple. And, again, I'll I'll make this public, so put it in the show notes. Again, it's not super full feature. Dead simple. We just have a form on submit. All we're doing is we're grabbing our triplet client. We're inserting a to dos. As long as it passes the schema, everything's good. To update the data on the UI, all I'm doing is I'm creating a state variable, and then I'm subscribing it. When that subscription is up you know, when that subscription notices a change, it updates the local state, and then that outputs here. Is there a better way to do this? Sure. This was a quick little demo.

Wes Bos

Cool. I I just was curious what else you could use besides IndexedDB, locally. Right? Like, that's what it uses locally, and then it syncs it to the server with whatever database it's using on the server. And you can by default, it uses memory Yep. Which memory is the easiest to get up and running. But if you, like, refresh the page before, before you're you're totally done syncing, then it's gone. Right? So IndexedDB will keep it in the in the browser and allow you to sync it. There's still a chance of losing something, but very unlikely in that case. And the other option is SQLite, which is cool for seems like that's popular in the React Native world because you can have a database on device.

Scott Tolinski

Yeah. The SQLite in the browser world is really interesting. And as we'll see, there's even, like, Postgres in the browser, which is also very interesting using Wes. So these are kind of some worlds that people are exploring. At the end of the day, most of these things end up kind of just like tossing it all into IndexedDB, and that's totally fine as long as it it's fast and it works for you. Next 1 on here is Evolu Evolu. I don't evolution. It's a shorting of evolution. Whatever they wanna call this. This 1 is really sick, and you can tell it's sick just from their their banner. A local first platform designed for privacy, ease of use, and no vendor lock in. Right? Like, how good does that sound? And it uses SQLite in the browser, Electron and React Native. It uses its own CRDT. It handles end to end encryption.

Topic 5 15:59

Evolu built on Effect, uses SQLite browser, end-to-end encryption

Scott Tolinski

You can have a free server. Now that's kind of iffy for me if you're using their free sync server because how long is that going to exist? You know, for me, I'm the type of person who's probably just gonna run my own anyways.

Scott Tolinski

Uses Keyeslee as its as its built in ORM. So instead of having its own query language syntax, it's using an ORM, Keyeslee's syntax, which I actually I liked Keyeslee when I used it here, and it supports all all the stuff. It works really well. So this 1 is very similar to the last option where it does a lot of things for you. The only reason it's different is it does things just ever so slightly differently. Right? Another thing is this is really built on top of effect. We've been talking a little bit about effect on this show that the TypeScript platform effect.

Wes Bos

So a lot of the stuff in this is being done with effect, so you'll see effect all over this code base. Yeah. I like the the documentation has the the the first line of the documentation is a quote. If it doesn't work, if the app developer goes out of business and shuts down their servers, then it's not local first.

Scott Tolinski

Exactly. Exactly.

Scott Tolinski

And it's fantastic that that that the software is kind of 1 making, a bit of a progress here. But it's also like, it's good for the user beyond just being like a a privacy thing. It's good to use these applications.

Scott Tolinski

So you could see this code. This is all React code here. I'll show you some of my code in a second. It's not that much different.

Scott Tolinski

You're essentially, again, creating a schema. You you're creating your tables, and then you're writing your queries. Now this 1 is a little bit more verbose to do the same things because instead of just writing your query syntax, getting some data subscribing to it, you're writing your query, then you're loading your query, then you're subscribing your to your query. Like, it it doesn't all it doesn't do it all just at once. A lot of these things is kind of interesting, but most times I don't know if you work with a lot of subscriptions, but when you subscribe to something, you usually get that data right away.

Scott Tolinski

With this 1, if you subscribe to something, you only get that data if the data changes. So you have to load it 1st and then subscribe to it. And just to get the the data initially, maybe that's something a choice they made. Either way, this thing's pretty neat because it does have, like, the owner of the data built in, and it does that all through, essentially, identification numbers. Like, you can create an owner of this data based on a unique identifier for that user.

Scott Tolinski

You can re then reset that user, yada yada yada. Now a lot of these platforms are great too because they handle this syncing over WebSocket, and they have things all stored in it, like, a room type of deal based on, like, where that data is being stored. And it makes things like multiplayer applications really easy in addition to writing these local first platforms.

Scott Tolinski

So this 1, again, very, very, very similar. There is some churn in this Deno, and I don't not like serious churn or anything, but they mentioned that if you're hosting your own Vercel, that right now it's express Bos, and they're going to move it to be express or effect based instead of express based.

Scott Tolinski

So I I don't know if hosting your own is, you know, great option right Node. But, again Mhmm. Hey. It's just the sync server. Right? So let's take a look at some actual code here.

Scott Tolinski

This is the same thing, the same basic application that I have.

Scott Tolinski

The only difference here is that I'm doing it all inside of a Svelte, Rune here where I'm I have my this is just, like, kind of a a shell state where you can load a query. It saves the data to rows and row based on what you're looking for, and then you subscribe and it updates, and then it outputs that. Nothing crazy there. So, again, I create my to do table. I create my database.

Scott Tolinski

I create some indexes, and then I create my Evolu Evolu.

Scott Tolinski

Yeah. I I don't know. It's a word I can't say. And then from there, you're pretty much just loading it up. You're running again like evolu.create to do, pass it in. Some of this stuff gets a little crazy. This Wes decode sync, pnpm empty string, all this stuff gets a little crazy in here. And to be entirely honest, I don't know why you're doing a lot of these things. They come from, effect.

Scott Tolinski

I'm just kind of following the flow for these same types of things. So I I still need to dive quite a bit into effect given that some of this does look a little intense with the SD code sync Deno empty string 1, 000 type of deal. But at the end of the day, you're you're just calling a create command that's saving it locally and updating it. We can see this in action in this 1 here.

Scott Tolinski

Hi. Again, you could see nice and fast. This time, you'll notice check this out. This 1 actually has a service worker in this case. Yes. This one's using a service worker right here. And so, again, these platforms all difference differ slightly in how they're choosing to accomplish these things. Mhmm. But the end result is that they're trying to do the same thing.

Scott Tolinski

And I don't know. There's no IndexedDB in this 1.

Scott Tolinski

Either way, this 1 is persisting.

Scott Tolinski

If I refresh, it comes back.

Scott Tolinski

So I don't know why that's not showing up. But hey. You don't even know where it is? I don't even know where it is. I thought it was being stored in, IndexedDB, but maybe this is Go to your network tab and refresh and see.

Wes Bos

I'm curious now if Wes we gotta find it.

Wes Bos

Go to click on the XHR tab.

Wes Bos

Yeah.

Wes Bos

Hang on. Request payload.

Scott Tolinski

Yeah. It's hitting their syncing Vercel.

Scott Tolinski

But is that hitting the local?

Wes Bos

Turn I I would say, turn your Wi Fi off, but then I would lose you. So don't do that.

Scott Tolinski

Yeah.

Scott Tolinski

Yeah. And, also, I don't have a service worker set up on this anyways.

Scott Tolinski

That's a good point. I don't know where this 1 is storing it. It's funny. I thought I I had all that buttoned up. But

Wes Bos

Click on session.

Wes Bos

Local.

Wes Bos

It might be in the service worker.

Wes Bos

Yeah. But shouldn't you be able to see it? That's a good question. Oh, well, we can well, you can dive a little bit more into that.

Scott Tolinski

Yeah. That's a good point. I'll also dive into this 1, because even then, it doesn't feel as instantaneous. So maybe this isn't even, storing in my local server at this very second.

Scott Tolinski

Either way, this platform is 1 worth checking out, because I think it does a lot of good things for you, and I think it does give you it gives you enough control without making you do too much. Okay? The next 1 that people often talk about is RX DB. This 1 is really interesting because, again, it takes the place of your, basically, your entire database. Right? Makes real time applications easy, sync with any back end. See, this is another 1 where Supabase really comes into play here because if you're using Supabase, it's really easy to add RXdb to your application.

Scott Tolinski

Much easier than it is to just do on your own.

Scott Tolinski

Online is optional. Again, RXdb here is kind of like, you could think of this 1 as as taking place of the the sync layer, the the the UI layer, the local storing layer. But then if you're getting ESLint, where JS this data? How does this data then store itself on my own database? Then you gotta start worrying about your own stuff. Also, a big maybe not a huge bummer about RxDV, because, again, these platforms, perhaps, like, the ones that stick around or the ones that have better paid products, This 1 does have a lot of, like, piecemeal type of add ons. If you want this or that, you want any of these things, you you have to add them on as tiers to your pricing, which, you know, I don't know how costly this could get, but it definitely is gonna be more expensive than the ones that you're hosting your own syncing server, your own database. Because if you were to add all of these with a employee size of, like, 10 people here Yeah. It would be $200 a month. So just a That's pretty pricey.

Scott Tolinski

Food for thought. Yeah. People seem to like this 1. This is 1 that I haven't actually gotten to use personally, but, again, people seem to like it.

Topic 6 24:08

Electric SQL uses Postgres via WASM for syncing and conflicts

Scott Tolinski

Electric SQL is 1 that's been making a lot of of waves because this uses Postgres. This 1. Yeah. This 1 uses Postgres in the browser via WASM.

Scott Tolinski

It's the standard sync layer for local first apps. Once again, this is a sync layer, for local apps directly on Postgres.

Scott Tolinski

I wanted to add this to my application using Drizzle.

Scott Tolinski

Drizzle now supports their Sanity in the browser, but there's some, like, iffy things with migrations and stuff there.

Scott Tolinski

So this 1 does seem like it's an interesting option to use and certainly 1 that's 1 that has 1 of the the better, like, documentation.

Scott Tolinski

Like, look at this. You can see local first latency, 5 milliseconds, cloud first latency. Right? It does a good job of showing you, like, why this thing is interesting.

Scott Tolinski

Conflict free resolution and, like, what you can use it for. So you can see user 1, user 2.

Scott Tolinski

You can keep apps in sync. Again, I think this 1 is great.

Scott Tolinski

It might be the type of software that you would be looking to add on to something.

Scott Tolinski

It does say drop into your existing stack, drop in compatible. It works with any of these tools.

Scott Tolinski

But a lot of this stuff kind of requires rethinking of how your application works and functions.

Scott Tolinski

But if if you're using any of this stuff right here, right, if your application falls into the this kind of frameworks, this is at least a happy path for you here. So if you're using Prisma with Postgres and React and, you Node, hey.

Scott Tolinski

You're you're in a a good place. Now there is some things about switching from Redux, switching from GraphQL, switching from Wes. But, again, in my experience, a lot of these things are very difficult to migrate an application to unless your technology stack is, like, tuned to be working. Like, are you are you in a completely different database tech? Then you're probably not gonna wanna move to this thing. I'm curious what the what it looks like to take something like this and

Wes Bos

convert, like, 1 feature on the website. So maybe, like, we have, like, a notes feature for, like, a video. You know? Like, keep your own notes right here. Like, you you have an existing tech stack, but, oh, this feature would be great offline because people wanna watch videos when they're on the train and write their notes, and that would be annoying if if they went through a tunnel and didn't have service for a few minutes.

Scott Tolinski

Yes. Well, I I did just that, and, I have I have some thoughts there. I did that with Replicash, so I'll talk a little bit about that when I talk about Replicash. Alright. Yeah. So Electric SQL, really good option. Interesting 1 to to try out. I I think it's a neat 1, and it's it's 1 I think that has some of the most juice behind it in terms of, I mean, you can tell their website looks very nice. Yeah. It looks great. They clearly have, it together. PouchDB is also interesting here. Now Pouch is it basically works with CouchDB as a way to do local first with Pouch and Couch.

Topic 7 27:08

PouchDB works with CouchDB for local first syncing

Scott Tolinski

Now I never used CouchDB.

Scott Tolinski

Honestly, I haven't used CouchDB.

Scott Tolinski

And the thing that kinda turned me off from this was alright. Now I gotta learn about pouch, and I have to learn about couch, and then I have to learn about how they connect. I have to learn about setting up pouch setting up a couch. It's like I gotta learn, multiple things at once here. Yeah. And it's like, alright. I I don't need to learn a new DB tech while I'm also learning a a lot of other new things. I've used

Wes Bos

Couch DB many, many years ago. Yep.

Wes Bos

I've talked Wes I I did, like, a real time drawing application for dentists to draw on top of, teeth X rays.

Wes Bos

So the the idea was that, like, a dentist would get on the phone with a specialist, and they would both be on, and they would be drawing over top of the X-ray and the specialist would be like, oh, yeah. And then they would draw and circle something.

Wes Bos

And that required, like, both real time as well as saving where they had drawn and who had drawn where. So at the time this this was probably, like, 12 years ago. Yeah. And at the time, CouchDB was the hot thing. That's why NPM is built on CouchDB as well. I still have the T shirt, but it looks like it's owned by IBM now.

Scott Tolinski

Yeah. I I actually you know, I I never worked with Couch at all, but, you know, it's been on my my radar. This is apparently inspired by, CouchDB. So I don't think you have to run CouchDB. Like I said, this is not Wes I got to try it, but it's 1 a lot of people seem to really enjoy.

Scott Tolinski

So it could be an option, especially if you worked with Couch in the past, and this is, like, something that you're looking to do. Again, a lot of people seem to really like this 1.

Scott Tolinski

Tiny bass is another 1 here, going along with, like, the the Bos kind of naming side of things. And this one's pretty interesting because it does quite a bit more for you in terms of, like, what it's doing for you. But at the end of the day, again, like many of these things, like, what this is is it's a reactive data store for your local data.

Scott Tolinski

It manages that aspect of things.

Scott Tolinski

What it doesn't manage for you is is it doesn't manage the server side of your own things. I know this is, like, the way the industry is, and this isn't, like, not surprising.

Scott Tolinski

But so many of these will only have examples in React, which is kind of a bummer. Like, I get you know, I don't need, like, Svelte examples, but, like Yeah. You could just show, like, general web platform.

Wes Bos

Wes. It's the same as when you have, like, an example for fetching data, at least give me a JavaScript fetch and a curl. You know? And I can I can infer it from the curl if I'm not using JavaScript, but it's the same thing with, like, show me React, and then show me vanilla

Scott Tolinski

JavaScript because I can infer it to whatever other thing I'm using? Yeah. Because tiny base, you can see it's like getting started, creating a store, writing and learning and transactions and all the stuff. And then the moment you do, alright. Let's see how you do with UIs.

Scott Tolinski

React Direct Hooks, React Dom, using contacts. It's like, okay. It's only a React platform, which is fine. You Node? That's where the industry is. But, like, give me at least give me the web. Like, show me what a a version of this in straight JavaScript

Wes Bos

works. Okay. Yeah. So this is just a client side library for him. It's almost like a state management library, and then at the time it comes to syncing, you're it it says you gotta figure it out Vercel?

Scott Tolinski

Let me confirm that. Easily sync your data to browser storage, IndexedDB, SQLite, CRDTs, and Electric SQL. So you would use this in addition to another tool, which is also another thing that's confusing because a lot of these things, like, will say, use Yjs and use Electric SQL. Use this and that with it. And I'm like, dude, I thought that's what you were doing. Like, why what are you doing that's different? So they a lot of them fit into very small puzzle piece type positions here. So it's kind of, like, important to know what you're looking for when you're looking at any of these platforms in terms of, like, what do I actually need? Do I need the whole syncing layer? Do I need just the conflict? And I'm gonna handle what happens to save the data locally and the data remotely.

Scott Tolinski

I wanted to save it locally. I wanna handle the sync and the server. Like, what do you need to handle, and and what what of each of these things fills that space? It's a it's a tough question, which is why this stuff is so confusing.

Wes Bos

I don't even Node. And maybe you can answer this at the end. What should I use? But it's true. Like, do you wanna go all in and be possibly limited, or do you want to go a little peeled back and have to do a little bit more work? And I kinda just want everything.

Wes Bos

I I So But I also don't wanna be caged in.

Scott Tolinski

I know. I I said I want everything in my blog post, and that's where the Triplett and Evolut, those types of platforms. But then Yeah. Here, there's an inherent sense of, like, lock in there. And and they those platforms feel more like Meteor in a way, which I really like because they do so much for you. And if you're looking for that type of platform, that's great. Now if you have an existing site or you know what? Honestly, this 1, RepliCache here, RepliCache .dev, is 1 of the services and applications that really I think it's like what the pros use. Right? A lot of people want what RepliCache does because they it does it very well. And I I've gone pretty deep into in terms of implementing this on my own, and I really think it's it's a neat platform. That said, this 1 will really teach you about everything that needs to happen inside of a local first platform because RepliCash holds your hand much less. It does the difficult things for you, but you still need to have an understanding about how it works. So, again, this 1, RepliCache gives you, you know, real time collaboration via WebSocket's instant UI loading because it's all being loaded from an IndexedDB.

Topic 8 32:58

Replicache widely used, handles sync but requires custom back end

Scott Tolinski

You can handle your own back end stuff yourself. In fact, you have to. It doesn't have any, like, automated back end things, offline support. It just works.

Scott Tolinski

This 1 has a pricing where you need to pay a license for it, which is interesting. However, that they're not doing anything with that. It's just a license for it.

Scott Tolinski

Now that license is free if your commercial app is making less than $200, 000 a year.

Scott Tolinski

So if your app is making less than $200, 000 a Yarn, even though it's commercial, it's free. If your app is noncommercial, it's free.

Scott Tolinski

So you're really only getting to pay, like, $500 monthly if you're already making $200, 000 a year. That's Yeah. Pretty reasonable to me.

Wes Bos

I it JS. But like many things in this world, I think it gets expensive really quickly. Right? Like, you got a 1000 users using your application.

Wes Bos

I guess if you're you're you're paying $500 a month. So if you're making at the very at the very basic, you're making 200 k, you're spending $6 a year on

Scott Tolinski

Syncing Software.

Wes Bos

Syncing Software. Yeah.

Scott Tolinski

Yeah. And it honestly, it does a really good job. And that said, there are other cheaper options that, you know, are probably a little bit less interesting.

Scott Tolinski

But RepliCash, super interesting. I implemented RepliCash into my habits app.

Scott Tolinski

Let me see if I could let's see if this works, Wes. This is gonna be in fact, let's open the network tab here.

Scott Tolinski

Oh, yeah.

Scott Tolinski

So you can see it handled the push and pull.

Scott Tolinski

It worked. Oh my gosh. This was not working. I mean, it worked last night before I went to bed, but, like, 11 o'clock, it wasn't working.

Scott Tolinski

And so just that simple act of a toggle is doing a lot here. And it might not seem like it's doing a lot, but it's it's doing a whole lot. And in fact, if we look at any of these, like, pusher poles so when I click that, it fired off a push event to my server. That push event gave it some information, like, in terms of, like, what the mutation actually is. But it didn't just give it information about that mutation. It gave it in information about a lot of mutations.

Scott Tolinski

And when we're storing data in this type of thing, we're looking at row versions. We're looking at, like, when was this particular piece of data last updated? What version is the server on right now? And the cool thing that Replicash does is it handles all of this through being returning patch messages, and it then it handles all that data merging and everything. And if you look at your actual IndexedDB for this, this is what it looks like. It's kind of crazy. It's not like what you might expect. There's a lot of information in here.

Scott Tolinski

So how does this 1 work? What's going on here? Let's take a look at some real code because, like I said, I think this is the 1 I dove the most into and has the most going on here.

Scott Tolinski

So with Replicache here, I'll show you. I'm initializing it. Right? I'm giving my my thing a namespace. I'm saying it's in development, it's habits, and my user ID. That way, whenever it goes to sync, it's syncing with that very specific type of data. Again, I have a license key for it. And then I give it a push and pull URL.

Scott Tolinski

The push and pull URL is like these are the 2 routes that handle what happens when I change something client side and it syncs.

Scott Tolinski

Okay? So when I change something client side, what RepliCache is doing is it's running 1 of our mutations.

Scott Tolinski

So, like, if I create a check mark, all it's simply doing is putting a key in the database.

Scott Tolinski

And then it's telling RepliCache to fire off a message, a sync message to the Vercel.

Scott Tolinski

And that sync server has to accept that patch message. And it has all kinds of information in terms of, like, what the late what the current version of this is, what the mutation that's being done is.

Scott Tolinski

And then on that push server, which let me tell you. This stuff gets a little crazy.

Scott Tolinski

That push server, it looks like this. It's crazy. So I have a a request handler.

Scott Tolinski

I have to open a DB transaction, and then I have to iterate over every mutation that's being sent. I then have to check to make sure that that last mutation is the only 1 I'm processing or the only ones that I like, only the mutations that I haven't processed are the ones that I'm going to process. You have to check all that yourself. It it's not gonna do all that for you. And then you process the mutations that you have that you haven't processed, and it does those in order. So, basically, if I go offline, I do, like, 5 different things. It stores those as an array of mutations. When I go back online, it sends those to my server, and it processes them in order of which we

Wes Bos

ESLint interesting. So if if you, like, made a to do and then edited the to do and then deleted the to do Yes. It doesn't just tell you don't do anything. It tells you to to perform all those steps in order?

Scott Tolinski

Yes. And how you handle that is up to you. You could you could Bos, that's neat. Yeah. So it you have full control of handling. Because if you if you look at this, eventually, all I'm doing is getting into inserting things into my database directly.

Scott Tolinski

So at the end of the line JS you are writing your DB calls the same way you're doing it before. You're just handling it in the push server instead of, like, a route. So, typically, in an application, you have an endpoint. You send data to it. That data handles the form. It saves it to your database right there. This is sending all these patch messages to 1 central sync.

Scott Tolinski

That central sync is then divvying it out to where it actually saves in your database. I'm curious about the and and maybe I'll get into this, but the the CRUDTAs,

Wes Bos

Node t's, in order to The CRUDTAs. It's like, if you went offline and you did it on your phone and then you edit it also on your on your computer and then you went back online, that's where you might have a a conflict. Right? Do you are you taking any of those into account?

Scott Tolinski

Yeah. You you are. And and let me tell you that also gets a little bit hairy there. Because, Wes, let me let me introduce you to the back end strategies section of the RepliCache docs Wes it tells you all the different ways you can handle that.

Scott Tolinski

So guess what? You have to do that yourself. It tells you which are like, this is actually nice. Right? The reset strategy. It's easy, and it's difficult. It's fast. It's, you know, pick any 3 of the angles of the triangle.

Scott Tolinski

Yeah.

Scott Tolinski

The 1 that I ended up doing was the row version strategy, which, has the best performance, unfortunately, is the most difficult, because hey. You Node? YOLO.

Scott Tolinski

It has, read off and partial sync. So I thought this was maybe, like if I'm gonna go for the do it all myself method, I'm gonna go for the the the the row version strategy. I'm gonna make sure so each row has a version to it. Whether or not those versions collide, I don't necessarily know how that resolves. It's a little intense in the code.

Scott Tolinski

There's a there's a client version, a server version, and an individual row Vercel, and so we're just making sure that those versions are all all correct. I I would assume that they're yeah. Yep.

Wes Bos

I don't know what happens when it gets out of sync is what I'm trying to say here. Okay. Yeah. There's I think I I don't think that this is any software can solve this because it's a it's a It's a fundamental problem. Yeah. Yeah. It's a fundamental problem. Like, I just took a screenshot of Notion this morning. So I I closed my Notion.

Wes Bos

I had the same Notion thing opened in 2 Notion tabs, and then I closed my computer on Friday, and it came back today, and there was a little pop up. You can no longer make edits. You cannot open IndexedDB. Notion's local storage may be damaged. Yep. And I think in that case, like, it has to, like, tell you, hey. Like, there's nothing I can do here because probably somebody else edited that document and did something different well over the weekend while I had that thing closed. And that was fine for me, but at least it tells you or you can, like, throw them in a unable to save local storage that you can then just copy paste yourself.

Scott Tolinski

Yep. Yep. Yeah. There's there's different solutions here, both UI based and, like, back end based. Ultimately, it's tough stuff when you're getting into this. But, like so we had that push server. Right? Whenever I make a mutation, it handles the the change locally first. It sends that push message off. It also creates a poll. Now it it poll, P0LL it P0L l's, the PUL l's. It polls the polls so that, like, every once in a while, it's just kind of sinking in the background no matter what anyways. And what that sync happens is it checks the server versions, and it checks to see if there's been any changed data from the versions. So if habits has changed or checks has changed, it's going to check to see if any of those have changed. If any of those have changed, it creates a patch, and that patch looks a little something like this. You're saying it's a put operation. This is the key, and this key is for the local data, and this is the value that's being passed in. And then it sends that patch back to my client where the RepliCache client handles that patch.

Scott Tolinski

It merges the data locally, and then it reruns my subscriptions.

Scott Tolinski

So that's the like, I mean, this stuff, it's like RepliCash is doing so much for you here, but the amount that you're still having to learn and pick up on your own is still kind of wild.

Scott Tolinski

And I I really enjoyed working in this platform once I got it working. And you do have to really get a handle on, alright, where are things being stored? Like, if this key doesn't match up with the other key in my patch message, everything's not going to work. Right? If my versions are off by 1 for some reason, it's not gonna work.

Scott Tolinski

I had a I had a crazy bug, Wes, where I had optimistic UI code in my button because when I clicked the button, I wanted to see it flip. I'm I'm used to working on the network. I had optimistic UI that was being done through a toggle.

Scott Tolinski

My my local data was returning so fast that my opt optimistic UI was firing after the the local data.

Scott Tolinski

So it Wes, like, untoggling what I was doing. I was like, holy cow. So that that that should give you an idea about how fast this stuff can be, which is really the benefit here. So, yeah, it it's really pretty amazing.

Scott Tolinski

I'm gonna make some of this stuff public because I wrote this really neat Svelte 5 rune that allows you to you create a cache. Right? It creates your ESLint cache, so you initialize your cache, and then you can call, like, cache Scott sub. So in any of my layouts or any things like this, you could see I'm subbing to habits and checks. And simply by doing that, it's going to populate cache.data.habit, cache.data.check.

Scott Tolinski

And those are always staying in sync no matter what push or pull is happening. If I was on my phone editing something, it would fire a poll on my computer, and that data would update automatically, giving you that whole real time business that we love and, you know, all that stuff.

Scott Tolinski

So that's Replicash. Replicash is really pretty neat. Again, if you wanna check that out, I'll make some code public. I'm thinking about doing a whole tutorial series on how to do RepliCache in an app like system. Another 1 that's really neat is PowerSync. PowerSync is very much in the same space as RepliCache.

Scott Tolinski

This 1, again, what's nice is that they include all the frameworks here. It's not just like, hey. Deal with it. Here's React docs. You Node? Again, they're thinking about more of the web as a platform. A lot of, really awesome people are using this thing. But what PowerSync does JS you can tell, again, this 1 also has pricing. You pay for it. It's a sync server. This 1, again, PowerSync SDK saves the data to a local database. It handles the syncing side of things, and then you write your back end rights to any Postgres database. So this 1, again, requires Postgres as far as I know. But, again, it sets up in this space of handling the harder stuff connects to Postgres in a noninvasive way.

Topic 9 44:37

PowerSync similar to Replicache but works specifically with Postgres

Wes Bos

In the last 1, you were showing me how you had to literally loop over and handle every single 1.

Wes Bos

PowerSync will take care of that and then just give you the like, what to update?

Scott Tolinski

It takes load more of it. Okay. With the with the caveat that you have to be using Postgres, and you're probably gonna be wanting to have the setup from the jump. This is not something I think you can just layer on top of the existing Node. Because 1 thing that you'll notice with any of these Wes you start getting into what's stored in the database, what's stored in the database is no longer just my data. It's my data. It's the version. It's the local cache, ID. It's a lot more information.

Scott Tolinski

So that way these syncing systems can have an idea about how to actually sync them. So it's it's no longer just your data, and you have to account for that. But, PowerSync, if you wanna talk about EZ, has an add on to Supabase like so many of these do. If you wanna see the true power of Supabase, just look through how many of these different options utilize Supabase in terms of handling everything. So in this case, if you are using Supabase, it pretty much handles all the stuff for you, and it syncs. It sets up your buckets. It has a it works with the client side Supabase library, and it it does quite a bit for you. So, again, this 1 is a neat option.

Scott Tolinski

There is pricing. Let's see.

Scott Tolinski

Free Scott limits. You can self host it. Hey. You could self host it. That's great. And if you can self host it, like, that's almost the route I'm gonna go to. In this option, if you're self hosting it, then you'd be having your own sync server, which is its own self hosted sync server. Then you'd have your database and back end server hosted somewhere else as Wes. So you might end up having a server a server and an app. Or if these 2 are in the same app, then that's a whole thing as well. So gosh. This is why I wrote this blog post. If you wanna, get this distilled in a a more written way, you can check out the blog post as well. Check out all of these options. And, again, check out any of these other options because, by all means, like, auto merge is a really cool project. Hocus Pocus is a really cool project. I mean, a lot of these are just really amazing. YGS is really great. This is a lower level 1 that just handles the conflict resolution stuff.

Scott Tolinski

So a lot of these things are definitely worth checking out and worth your time. But these are the ones that I've tried.

Scott Tolinski

Some do a lot for you. Some do very little for you. As you saw all the stuff I had to do in cache to get it to to work, yes. But at the end result the end result is an application that loads incredibly quickly.

Scott Tolinski

This this data mutation is happening.

Scott Tolinski

It's saving to my database.

Scott Tolinski

It's saving locally.

Scott Tolinski

It's yeah. Really, really pretty pretty amazing. So yeah. Check it out. Let me know if there's any of these tools that you've used, if you're interested in local first software, if there's anything that we didn't touch on in this episode that you'd like to learn more about within the confines of Local First. Because, again, I I'm doing a lot here. And if you're interested in maybe picking up some of these libraries or or tools, let me know, and we'll maybe throw a tutorial course together.

Wes Bos

Sweet. Sweet. Sweet. Alright. Let's get into some sick picks. I have a pretty good sick pick for you today, something that I'm really happy with, and that is a portable tire inflator that has a battery in it. So I was going through our our van, and we have, like, this big box of, like, ratchet straps and, stuff for towing and and all that stuff. Right? And in in that, you also have a a 12 volt pump. If the tires go low, you can use that to pump the tires up. And I always found myself hating that because you have to, like, plug ESLint to the cigarette adapter and, like, hope that it reaches it. And I find myself not using it for the tires all that often. I find myself using it for kids' bikes and Basketballs.

Wes Bos

Fastballs and all of that stuff. Wes. So I found this 1 that was very, very small.

Wes Bos

It has an internal battery in it.

Wes Bos

It can run wired over the 12 volt or it can just run off the the internal battery. You can use it as a, like, a external battery if you wanna charge your phone as Wes. And it's awesome. So the other day, our I I bought it and I threw it in there. And, I was like, alright. I'll I'll test this out. And we had the tire light go on. We had a couple low tires, so I got out. And it's great. You set it to what your PSI should be. You clip it on Mhmm. And you you turn it on, and you can just, like, let it hang off the tire and Scott of just walk away. You're not sitting there waiting for for the thing to pump up. I don't know that if you had a totally flat tire, if it would, like, it would probably eventually get there. It's obviously a very small pump internally, but it it got me up. I was, like, 5 PSI down, and it took maybe 2 or 3 minutes to get back up. So I was very impressed with it. And being able to just, like, not be chained down by the huge cord was was such a nice improvement. And it comes with, like, all the adapters to blow up like AAA pool floatie or something like that.

Scott Tolinski

Yeah. You know what? So III do the same thing with my you know, what's so nice about it is not having to hunt for the the air air things at 711 or something. Oh, yeah. Yeah. Yarn Twain. Yeah. Yeah. I don't care if it takes me an hour because I have to charge it in between.

Scott Tolinski

Mine that you have to charge it. Like, I can do 2 full tires before I need to recharge mine. So Okay. 2 full tires recharge, 2 full tires recharge.

Scott Tolinski

Yeah. Not a big deal. I'm gonna sick pick, actually, something very similar because I I recently had my car battery die.

Scott Tolinski

Kids love turning on those lights and stuff.

Scott Tolinski

And, a buddy of mine let me borrow a battery powered car jumper kit. Oh, I've got 2 of these. They're the best. They're the best. He keeps he keeps it in his car. My car was like, he's like, here. Just take this. And it's and, again, just like what you it's a little battery. It's connected to jumper cables. III was jumping my car because I I had to take it. I I jumped it with my wife's car. Yeah. That was a giant pain. Her battery's in the trunk. It's hard to get to. And then he gave me this thing. It's like you just said on the hood, turn it on, you jump the car.

Scott Tolinski

Holy cow. So, yeah, if you're the type of person that, you know, you worry about having a battery die or something like this, keep 1 of these charged up, just throw it in your trunk, and you never have to worry about it. So But Yeah. IIA huge fan of Node.

Wes Bos

It's, like, you don't I still carry jumper cables, but I was thinking the other day, I was like, you probably don't need jumper cables when you have this because it will give you like, I've got so many crappy old things at the cottage, old lawn thing, a couple ATVs.

Wes Bos

And, like, the batteries on these things always die.

Wes Bos

And I I just have that ready to go, always charged. You can probably get, like, I don't know, 7 or 8 jumps off of it. Depends on how big your your engine is, obviously. But the other day, some poor guy flagged me down. Hey. Can you give me a jump? And I was like, now is my time to shine. No way. You had to. I was like I was like, brother, I'll do you 1 better. You know? And I just brought it out, clipped it on. He's like, where did you get that? And I was like, I don't know. You can get them at literally everywhere, And the price on them has really come down as well. Yeah. They're great. You can get them in all all all kinds of different price range. Shables plugs. Let's check out our Syntax merch store. I'm not wearing 1 right now, but go to Syntax Scott f m. And in the URL bar, you're not the not the URL bar. In the navigation, you'll see swag. It'll bring you to century.shop.

Wes Bos

Got a whole bunch of really cool new t shirts. We got can coolers or koozies. We got the syntax deck. We got the syntax basketball.

Wes Bos

Really cool stuff coming up as well. Check it on out.

Wes Bos

That's it for today. Thanks, everybody, for tuning in, and we will catch you later. Peace.

Share