Zero Sync is the Future of Data Loading

Welcome to Syntax on this Monday hasty treat. We're gonna be talking about a new library, new, system, caching query language, way to build websites that is really extremely impressive and allows for not only the fastest things that you've used, but great experiences and a really nice developer experience in a lot of ways. I'm talking about what some something called Deno sync. So we're going to be diving into what this thing is, what what kinds of, you know, projects this enables you to actually work on and and why it's so cool. I've been personally working in it quite a bit. My name is Scott Tolinski. I'm a developer from Denver. With me, as always, is,

New West, New Boss. What's up, bud? Hey. I'm excited to be back. It's our 1st episode of the year recording. Had a nice little, Christmas break, and now I'm like yesterday was, like, my 1st day back, and I was like, I feel a little Yeah. Same. You know, I had this huge list of little fussy things to deal with. And now today JS just like, I made a video, and I'm researching and coding. I was like, yeah. This feels good. Yeah. Good video too, by the way. I I saw that pop up, and it's, something on Versus Node, feature that feature that I use all the time personally. So Yeah. Yeah. I've I had some of those videos Yarn just I'm sick of people asking me, what is this? Yeah. So I was like, alright, I'm a get video. I could just point to the video. Yeah. Point to the video. Here it is.

Sick man. Yeah. I was spending a lot of time yesterday just getting through my to do list. And then, you know, we have our 1 on ones and all of our meetings. And it was really nice to to get back in the swing of things. But at the same time, you're like, all right, I am refocusing now on everything I have to do, starting for the new year. And it feels good. But at the same time, I don't feel like I'm at, like, peak performance just yet. You know? I think that that'll that'll come next Thursday, Friday. Yeah. I gotta rev up. Definitely.

And if you want to see all of the errors in your application, you'll want to check out Sentry at sentry.i0forward/syntax.

You don't want a production application out there that, well, you have no visibility into in case something is blowing up and you might not even know it. So head on to reduce entry dot I o forward slash syntax. Again, we've been using this tool for a long time and it totally rules. Well, let's get into it, man. Deno sync. I'm gonna be sharing my screen. If you're listening in audio, you know, I'll do my very best. I'm not going to, you know, sacrifice the audio podcast for visuals here, but I do want to at least show you at least how very fast this system is. And if nothing else, the screen share that we'll be doing in the video version of the podcast will be there to show you a little bit about maybe some of the syntax from the docs, but also specifically just how fast this syncing library and this this platform is. So, 0 sync, what is this? What what is this all about? It kinda comes out of the Local First space. I know they were a little apprehensive about calling it a Local First system, but since Local First is the the term that is sort of stuck even though there's, like, what, like, 6 or 7 different things that, make up local first, like encrypted data and, you know, the ability to download your data. Like this isn't this isn't those things. This is mostly a sync engine, a caching library, a like a a front end query library, like a sync system, WebSockets, those types of things rather than, the the full on quintessential or or, like, coined term local first version of local first.

Yeah. So as as far as I understand it JS, Deno will allow you to build apps both the it'll take care of the database part. It'll take care of the client side querying real time part, and it'll take care of the any back end logic that needs to happen before it actually hits a database. Is that a fair way to put it? Like, this is for building apps that have data, and data needs to be put into the ESLint.

Yeah. So what this allows for is apps that work with local data and you modify that data locally as if you're working in like a local cache. And behind the scenes, 0 handles 1 storing that local data. It handles sending the patch messages to a sync server. That sync server is going to then take those patch messages and then distribute those changes to the database.

And because it's working with this this patch message system of how it's sending small incremental amounts of data, you're always sending very small amounts of data unless, of course, the user doesn't have the, like, the the the initial pull down, of data, which even that can be very, very fast. So what it does behind the scenes is it's pulling down data, storing it to your, local data on your computer, on your client, and then always interacting with that. And then it's handling that that the data meshing. It's handling the real time stuff like you said. But the coolest thing about XeroSync compared to some of these other ones is that it doesn't take over your entire, system to the point where you're having to buy in completely to their database. Right? You're not like there there's another library called ESLint DB that I really like that is kind of like a Firebase clone, but also does local first very well. But for them, you're storing everything on their server, on their platform.

You know, with this, it's bring your own Postgres database.

You bring your own Postgres database.

It doesn't have to be entirely devoted to 0.

0 can sync and cache whatever you Node it to be. And maybe perhaps you have some other things that aren't that are just stored in that database. Either way, it doesn't take over your whole database if you don't want it to. Okay. So, like, if you have an existing app that's Postgres or MySQL or something like that, it's not like you

you can you need to build from scratch. Right? Correct. Often, I'm sure that drives people crazy JS, like, we have 15 years of of legacy code. We can't just

just drop this whole thing and use the hipster stack to to build from scratch. Totally. And I'm actually gonna be converting a a preexisting Postgres app that I have to 0 to give that a try because so far I've only built things from scratch with it. But if you look at the database tables for the stuff in Xero, there's nothing Deno specific within your specific table. So like right now I'm building a card game with 0 and I have 2 tables, 1 for the current hand and one for the game itself. And if you were to look at both of those tables, there's nothing 0 specific in either of those tables. It all kind of happens elsewhere, which is great. Like, you know, you'll be able to bring your own own data, your own database and it all just it It's so good. So, I want to show you really quick folks who are watching, you'll get this little bit of extra in terms of seeing how fast this is, but I'll also describe it. So I have a, a, an application that they built called 0 bugs. They're actually using this as their real bug tracker, which I thought was interesting. They were like, well, we want to build a project to showcase how good, 0 is. So we might as well build our own bug tracker, which is exquisite. So first and foremost dogfooding in the biz. It's a dogfooding for sure. This JS, and what's the best dog food out there? I'm not sure what. We use Avoderm. This is the Avoderm of dog fooding. This is the best dog food. Or what is it? Those dog food commercials that were, like, you store dog food in your fridge? That's what this is.

So, I'll actually I'm gonna pop open the network tab so we can see this, like, initial load speed. Alright. What's new in Chrome? Blah blah blah. So, initial load times are extremely fast. Dom content loaded in a 137 milliseconds.

Very fast. Not even a noticeable change. And this is not server side rendered. This is all client side rendered. So server side rendered people who think, like, the fastest way to get it JS something going is a cache HTML page. This is certainly not that, and it is incredibly fast, consistent. Can you can you click on the disable cache and do it? Because I know everyone's screaming at you right now. Sure. A 182 milliseconds.

Wow.

That's that's really good. And the reason that is so fast is because it only needs to download what? Like, some HTML and some JavaScript. And the 1st time, it needs to download data, but most of your heavy lifting, which is your database, is already in your

your browser. Right? Yes. There you go. You see it. So yeah. All of the data is stored in your browser. And unlike other, you know, ways of doing local data, it's not storing necessarily stuff that like, it's not storing your database as an exact, exact copy of your database in the IndexedDB.

Like, there's a lot of interesting ways that this data is being stored in here. And I'm not going to get into the details because, honestly, I don't know of how this data is stored in load. They they do some really intense database research in terms of how they've constructed, the way that this is interacting here. And there's they actually link in their docs to a, oh, man, a scientific paper on incremental view maintenance.

I I think, like JS, like, a web developer, you just need to understand it's storing the stuff that it Node. Right. Knows how to get the stuff it needs fast.

Exactly. That's exactly it. I I see a scientific view paper called automatic incremental view maintenance for rich query languages with diagrams and math, and I say, alright. Somebody else figured that out. That's cool. They told me actually from the 0 folks told me that my last time I showed how fast this was, I was actually limiting how fast it actually was because I had too much delay on my keyboard when I held down a key. So I'm gonna go in my keyboard settings really quick and remove that delay.

Key repeat rate is going to be fast, and the delay until repeat is going to be sharp. Alright. So they have it set up keyboard shortcuts. I'm gonna open an issue. This is a full issue, super fast, and now I'm gonna hit the j key. So you can see it, it it instantly loaded the next one. Again, the data's local. It's not having to do a network call here. Everything you need to run this application is currently within the client. So, I'm holding the keypad. If you're watching this, I just scrolled through, I don't Node

50 pages loaded in a matter of, like, Node second. Like, it's it's instant loading of from page to page. And and often, you would you would sort of, like, try to preload things. Right? Yeah. You would but that's not necessary. And this is preloaded to an extent and that that data is in your database. Yeah. That's fine. When you ask for it, it's it's right there. What's so cool about this

is that it also includes real time.

So if somebody is making changes to this or leaving comments on this or giving this a reaction or a thumbs up, you'll see it happen. It'll pop in and it'll just work. And that happens out of the box for free. So like I said, I I'm building a card game with it. And the way the way that the card game works is I just update the database in like a minor, hey, this card was played, for this specific hand, and everybody just sees that card being played because it happens through WebSocket in real time without any effort. And all those things are great because I could do a lot of this stuff with the precursor to this library, which was Replicash. Right? But in Replicash, I was having to write like a 12, 13, 14 step process to receive patch messages on my server and handle them accordingly.

And there were instances there where I goofed things up in terms of how I'm versioning rows or what, you know, is that process actually as fast as it could be and what happens in fail states? And like, I didn't have to do any of that with Xero. So I gotta say, beyond just being that fast, the developer experience for this thing is great. So to get into exactly what you do, you're basically giving it a Postgres database address in an env Yarn.

You're creating a schema.

That schema has to be its own thing. It's a Deno schema. Now Nate from 1 actually, who's been on the show, Tomaguie, Node, he actually created a Drizzle two Deno schema translator. So you could write your schema once in Drizzle and translate it to 0 if you'd like.

I personally have been using Drizzle to do my migration still to handle my my database style stuff. And then I have my Deno schema itself, which is its own thing to identify the schema. You also get easy, super easy relations.

It all just functions super well, to be honest. The way you query data is from a client side query language called ZQL and it is o r m ish. So you create like your 0 instance and you do a z Scott query and then you give it the collection and then you put in your your parameters like a Wes or an order by. Kinda kind of feels Prisma e. Prisma e. Any sort of o r m. Right? So Yeah. Like if I'm if I'm in that that, Z bugs example, z.query.issue.where.

And then you have your your, you know, qualifiers to how you want to have that. And if you want 1 of them, you do a Scott one on the end or a limit or whatever. And then if you want related, all you have to do is inside of there do like a related where you get a function and then you get access to

another collection that could like. Nested data. Like, get me episodes.

And for each Node, get me the guests. And then for each guest also me also get me a list of episodes they've been on. Right. Totally.

And again, since this is all local, it is all you don't have to worry about super nested relations and maybe like n plus Node issues or anything like that. Because again, the data the data query aspect is so fast that like I have never had an issue with this, Robin. Like having to think, oh, is this query causing an n plus one issue? Is this and then not to say that I I don't know if you can hit those issues. Honestly, I'd have to talk to the the experts on that one. But, it's it's all just so fast that you haven't really have to think about that that much. So either way you have this query language, it works really nicely. It's its own thing. But with that query language, you get this real time cache where everything is a subscription and anytime data changes, that data automatically updates, your UI automatically updates. It is all just real time.

It's real time. It's instant. It's local first. It's all these things, and you get to bring your own database. So if if we're we're talking my my dream, this is my dream here. You know? So what you're saying is

there is a database. I'm I'm saying that in quotes because the data is also in the client.

And if you write a query for, like, a list of posts, if something else were to mutate those posts, update, delete, add a new one, then automatically that query is is reactive, and you don't need to do any re fetching

or loading. Is there is there even a loading state in this? I you know, I've I have not written a loading state, since I've been using this. Okay. Things things do initially come in at least in the the Svelte bindings that I wrote. I I'm pretty sure this is the same way in the React version because I I I based mine off of that.

Things do initially come in as undefined, but it's so fast that it's only there to, like, you know, do a quick if statement. So I I've never seen a loading spinner with this. Maybe, perhaps, if you have a lot of data coming in on the very 1st initial load, that could be something. Yeah. Because at some point, there there might be like, you load the app for the 1st time, and there is no data, and you need to get that data.

There may be a loading state.

Yes. But it is it's not maybe not something worth, throwing a loader up. I'm I'm sure we'll we'll see some specific instances.

Yep. So querying data, we talked about that. Writing data is the exact same you do, you know, z.mutate.collection.insert or upsert or update. I actually it's funny. I didn't even know there was an upsert, and, I got a comment on my YouTube video saying, hey. There's an up cert. Oh, cool. I love an up cert. Can you can you explain what an up cert is for for the audience? Yeah. So an up cert is like an insert and an update combined into 1 Wes if the record of the ID that you're inserting in does does exist, it's going to update it. And if it doesn't exist, it will create it. It's super handy for not having to do if thing exists, do insert or update. If thing doesn't exist, do insert. Right? It just kinda saves you that ESLint, especially if you're always going to be adding the same data.

I do love things that have an upsurge. I don't know if there's downsides to it. I'm sure there's some people who have strong opinions there. Yeah. Like, for example, on the Syntax website, if we were storing data about,

an episode that you have listened to, you might wanna just throw an up cert in there because it's like, well, if they haven't listened to this episode yet, if this is the 1st time that we are are logging progress of this audio, then create it. Otherwise, update it. Right? And it's nice to just say upsert.

It is. Whatever. There's also batch updates.

There's authentication baked in, which is JWT auth. You basically are just giving it your either your auth secret or this JWK stuff, which I've never used. But you you basically give 0 your auth secret. You pass it in your auth token j w k. I know. I I gotta learn what this stuff is. It's, yeah, it's a thing that people requested.

And then from there, you get access to the permissions API where you're essentially inside of the schema getting that JWT decoded in here, and you can do a quick comparison if creator ID is equal to essentially the user ID of whatever the decoded JWT JS, then that is like your auth functions. So you can get granular with your auth permissions for inserting, deleting, updating, all that stuff, and you have full control over that, which is great because in the earlier versions of this, this stuff wasn't in here. And that Wes, like, what even though this library JS an alpha, we'll talk about that in a second. I I still was like, oh, I can't use this on anything because I can't control permissions. But now I can control permissions. It's like, it's still alpha, but I'm gonna at least try some stuff on here. That that's 1 question I have had for a long time, which we had Johannes on, is to talk about this all this local first stuff. And we're like, how do you get a little bit more fine grain control over? You should be able to view Wes. Specific things or even at a field level.

And his answer was, like, it's more of like Local First is more of like a document level. You know? You either have access to it or not. But I don't I don't know that that's that's always true for the type of apps that I work on, especially if it's like, well, it's I would like for it to be local first, but also it just needs to be a regular web app. And I I still want all of those features.

So, like, you can also preload. So you can run a query and then just run preload on it if you want if you want to make sure that data is there, you know, on initial load for all these things. There's a lot of great demos that show some of the stuff in action too on their Deno. Migrations, it's a little bit interesting right now. You got to kind of do some hand stuff. Again, this library ESLint alpha, so I'm sure these things will change. Deploying, they have some docs on deploying. You really have like if you want to push this thing to production, you have you have 3 things to consider, right? You have your database, which is just a Postgres database.

You have the sync server, which is its own thing. It's a server that you can. They have a Docker container for it if you want, or a Docker image for it. If you do want to just throw it up with Docker and then you have your app. So you have your database, your sync server and then your app.

And that's pretty much it.

Other than that, you're just running things on config. There's some recipes in here JS far as working with other libraries. They have a 1st class citizen react library here that is like the way that most of their demos are. There's also a 1st class Solid JS library.

And then as far as community, there's the one which is the full stack react native framework from Nate and then, 0 Svelte created by yours truly. I actually wrote this and then, view or 0 view as well. And then there's like some of those other things like Drizzle 0 or Prisma 0 to get you up and running. These felt bindings for this are pretty Node. I if I do say so myself, they work within, like, kind of the way that Svelte data stores have been going with this class syntax. So you just create a new query with your query inside of it, and then that gives you a reactive, state variable.

And, likewise, everything just kind of worked as it does in there. How how hard was it to write this adapter

Oh. For Svelte? I'm curious. Like,

it was harder until Svelte released a create subscriber that they added during the, advent of Svelte this holiday. And that create scriber made this subscriber made this easy. Now, in the very bare bones of this, it was super easy. But what I ended up doing was following what their react version was doing. And I created a Vue store, which was a map of each individual like cached query. So it's like the query itself and the user who's querying it in that storing it in a map. So each of those things can become its own reactive,

its own reactive value. Great. Use case for a map because the key of the map can be not just a string. Right? Sanity Correct. And in this particular case, we're creating a hash of the query itself

and then adding on the client ID. So it's the client and the the the hash, and that is,

what it would be. Apollo did something similar to this as well Node every every single query you've ever made is just hashed out. Yep. Wow. So it's, what, couple 100 lines of Node? 147.

Able to write an adapter? That's that's impressive. That just goes to show the flexibility of the library

of you can move it into an entirely different framework in 150 lines of code. That's impressive. Yeah. It was tough trying to translate some of the React stuff because React had all this, like, I don't know, synchronized React state stuff that I was like, I don't even know, like, what this is doing. I have to research now. You need a lot of that.

Yeah. These obscure react library or features to figure out what to do. The solid version was a good thing to to look after, too. But hey, it all works. It's super good. As far as can you use this today? That's like a big question JS like, can you use this today? And you probably shouldn't be shipping anything to production on this today unless you are super adventurous.

They do have a roadmap.

They're looking to be in beta by, Q2 of 2025.

So Q2 and they want to add like vector based search, call or cell level read permissions, which is interesting.

Cache size management.

So there's like a lot of little things.

But I think the the surface for the API, I don't know if it will change that much, but I can't imagine it will. I've been on it since the 1st release and there there has been a little bit of change, but Vercel, not that much crazy stuff. So I can't imagine they're gonna go nuts on this.

I think it's well worth your time if you're interested in building applications that function like superhuman or linear.

These apps that are as fast as the one that I showed off here Wes they're loading, you know, hundreds of records in a second. If you're interested in building stuff like that, I don't know a better choice, than this library to me personally, specifically because you still get full control over that that database.

And if for some reason you wanna move off of it at any given point, your data is not like your your data is not directly tied to 0 in a way that makes it unmovable. So Yeah. I like it. Yeah. I'll say that. I like it a lot. I'm trying to I'm trying to think, like yeah. So desktop

applications, like you said, linear issue tracking, probably something like Notion would be a really good, good use case for it. Games, real time multiplayer.

Games.

I saw it like my card game. Right? Yeah. That made let me tell you. It made building a 4 person card game where I just made a a room, which is an ID.

Yeah. And then if you join that room, you can sit at a chair. You get an ID, whatever, and then you automatically get real time all through WebSocket. Somebody plays a Yarn, everybody gets that update, and you click refresh, and it's instant, and you still have those updates. You know? Man.

It's pretty dang cool. So That's sweet. For games specifically, I got a demo of it was like mouse positioning. Get this. It was mouse positioning in real time persisting into a database.

So, he was moving his mouse on the screen. I was getting the value of where his mouse was on my screen in real time, and then if you clicked refresh, I was still getting that person, like, wherever that last one. Oh, yeah. It's insane. Yeah. Insane. So you could you could literally

store absolutely as much as you want in absolutely everything and not have to worry about, like, like, rate limiting because I assume it it does that for you. Like like, one thing I do is with my progress event on videos is I'll have to, like, debounce or throttle Yeah. That because I don't wanna send every time that you you have an update, it's every every couple seconds. But I really only wanna I I save it locally every second, but I really only wanna sync it every, like, 10 seconds or so. So that would be a a good use case for this as well. Right?

Yes. Yeah. Definitely. They, they directly say, like, scaling, all Scott of Cool. Yeah. I I don't, you know, I wish, I knew a little bit more, and I probably have been told at some point how it handles the synchronization, behind the scenes where in terms of, like, how it knows when to push what and what how much to push so that it's not pushing every tiny little thing. But the date the amount of data that it's sending to your server is not, the entire data every single time. It's just patches of data. And because of that, it can small, tiny bits of data. Interesting.

It's cool, man. It it's super cool. And this is like, I was looking forward to it a lot, so I I bug them, like, constantly. Like, can I get access? Can I get access? And then Dax got access, and I was like, okay. Dax is in. I gotta get in. So I got to be the 2nd person in their little, in their little group, and I thought that was pretty cool to get my hands on it. So I'm definitely, like, a tinkerer in that regard. But I built a few things with it. I was building a survey app with it, which was really super nice and fun to do, my card game with it. And so I've got a little bit of different experience building different things and, impressed. I walked away impressed. I'll say that. Cool.

So you also have a video on the Syntax YouTube channel, detailing us as well? Yes. It's called,

0 will change web development forever. And that sounds dramatic. It's click Sanity. Yeah. But I do actually firmly believe that because this is like, this to me is like everything I've been searching for post meteor is really what it is. And, I may I'm even building like a library on top of this that includes off and it's like meteor like. So that way you log in and you automatically have templates that you can modify and stuff. So, I'm I'm doing some of that stuff just for fun, and I'll talk I talk a little about that. I'll I'll I'll update more once it's further along. But I used it to build my, survey app, and it went really well. So, stoked about it.

Awesome. Thanks so much. Well, I definitely gonna have to give this a go, put it on my list of things to check out. But I am stoked that you're finally able to talk about it because Scott has been sending me

Tolinski screenshots and demos of how amazing it is for months now, and it's it's really exciting. Oh, yeah. I just I want to talk about it. And and for people asking, people who were asking before, I will be doing tutorials on this for both reacts, felt whatever. Whatever you want to see, I'll do it because I like it. So yeah.

Awesome. Alright. Thanks everybody so much for tuning in. We will check you later.