Standard Schema: The Universal JavaScript Data Interface

Welcome to Syntax. Today, we're talking about standard schema. Schema, that's a word I'm gonna say wrong the whole episode, so buckle on up. This is a a new effort from the devs, Fabian Hiller. He's behind ValleyBot. That's a JavaScript validation library.

David Blass, he's behind a a relatively new validation library called Arctype. And Colin McDonnell, we've had him on the the podcast as well. He is the creator of Zod, tRPC.

Basically, all of these validation library authors sort of got together and put together this thing called a standard schema. And today, we're going to explain to you why and what that is, and why this is good for you as the end developer.

My name is Wes, developer from Canada. With me JS always is Scott. How are you doing today, Scott?

Doing good. Just got back from mountain. I went skimming.

I went I went to the mountain to go skiing. I actually did. Skimming?

Yeah.

Yeah. No. I'm doing good, man. Just hanging out, doing doing, some creative coding and having fun. I I'm excited to learn about this stuff a little bit because I guess what I don't understand about this whole thing is why. I mean, I get that having a a common interface amongst these libraries is is great if that's what this means. But in the same regard, then, like, why not

just collaborate on those the library itself? Yeah. Yeah. Yeah. Why not make that a a standard library?

Yeah. But a large part about what these libraries are doing is oftentimes with data validation.

Data validation, one of those big things that, like, hey. If you're not, validating or parsing your data correctly, you might get some errors in your code. Those errors, you might not know about them. And unless they pop up in your your Century dashboard because you're seeing some something of property is undefined, and then all of a sudden your users are having a bad time because maybe your data wasn't being validated correctly, and you didn't know it was in there. So tell you. Let me let me interrupt your your ad. Tell me. I got a Yes. Perfect. Example for this. So somebody asked

the other day, how do you maintain that? Like, you're if you're scraping data from a website, like, what do you do when that that changes? You know? Or if you're pulling data back from an AI, it's returning to you JSON, you have to still validate that that data looks like what you're expecting. And and what happens when it doesn't look that way? What happens when the website you're scraping changes? Well, catch that error and then send that data off to Sentry, and you get alerts right away when that data is not looking like what you're expecting.

Yeah. That's important, because, hey, if the data isn't looking like what we expected, our code's gonna break. So, yes, check it out. You're in trouble. Yeah. Century.ioforward/syntax.

Sign up with the coupon code at tasty treat. Get 2 months for free. Alright. Let's get into it. I'm I'm stoked to learn a little bit about this stuff here because I primarily use Zod, and Zod is the only one. But this seems like a world where every other week, somebody's like, no. Zod's out. You gotta use, ValleyBot. Bos, ValleyBot's out. You gotta use Arctite. Okay. What what are we on today? Well, now we're on standard schemaling.

Wes. So Zod, ValleyBot, and archetype are not being replaced. The idea with standard schema I will let's peel it back in a second. So a a schema, if you haven't done done this before, is you describe what your data looks like. And when you have a piece of data, you can validate it against the schema to make sure that it looks like what you're expecting. So a very simple example would be you have a schema for a person, and, yeah, their name is a string, and their age is a number. Right? It can be a lot more complicated than that. You can check that things are capitals. You can check for things that are formatted as a ZIP or postal code. You can check that they don't go longer than a specific length. Right? Whenever you're dealing with data in an application, you're going to need to validate that data, before you go ahead and throw it in a database or before you display it to your actual user. Mhmm.

And, of course, we have Zod, ValidBot, Arctype.

Yep is another popular one. There's there's lots of them out there. So those are the schema definition and the validation libraries. Right? And then on on the other side of things, you have tools that need to validate your data before going forward. So if you think about Hano JS or Express JS, you have a a middleware that's taking in a post request from somebody that's filled out a form, and you wanna validate that they have completed all of the fields correctly.

You wanna validate that they're not sending data that is not needed. You wanna make sure that the length of the body that they sent is not the Bee Movie.

Yes. Because it has to be short. Right? So you you'll have to to to validate those. Right? So 10stack router, 10stack form, Hano Express, all of these libraries often provide a sort of a a way to validate your data. We have a, like, a little bit of a a problem there because these libraries that need to validate your data, they either choose a validation library and and consider it blessed. They say, like, we use XOD for validating your data. And if you use something else for describing your schemas pnpm doing validation, then you're either out of luck, or you have to write, like, a like an adapter or a resolver Mhmm. Function that will sort of translate it from from one to another. So the idea with standard schema is it's not a way to define what your data looks like. So the Node API, the ValleyBot API, the archetype API, those are all different APIs, and they've all approached it differently.

They've not, likes like, got together and said, you know what? Let's just make Node validation API to to end it all. Those are still their own packages.

So what they have agreed is a standard way to define several things. First of all, how to actually validate the data. So what's standard schema does is anyone who implements standard schema will simply tack on this tilde standard onto each of the schemas.

And that way, when you pass in a schema into something like Deno or TANSTACK Form or React Router or anything that will support standard schema. So all these library authors will now say, okay. I don't have to just pass me a standard schema. I don't care who made the schema. All I care about is that I can access a couple things. First of all, I wanna be able to call the validate method on your schema, and I don't care who has made it. I'm just gonna call a validate method on that schema, and then that validate method will be called and will be able to return data in a structured way. So the the Scott of the second standardization they've had here is that the return value from validate is always going to be in the same shape. And if you take a look at the actual standard schema specification, it's really just 58 lines of TypeScript that has very rigidly defined what the inputs and the outputs of this look like. So it's not even something library authors have to necessarily NPM install. It's just 60 lines of TypeScript that they just have to adhere to that will allow library authors to know that the inputs and the outputs are always going to look in a specific way. If you do take a look at the TypeScript code, you can see that the result is either going to be a successful result of whatever type you passed in or it's going to be a failure result. Right? And then the failure result is going to be an object with a property of issues, and each of the issues is going to be an array of individual issues. Each issue itself has a message property on it. You see what you see what's happening here JS that you normally have to write, like, a translation layer between these libraries. And if you think you have 5 or 6 different large validation libraries and 40, 50 different libraries that consume Oh, man. Schemas, then you're talking about, like, 5 or 6 times 50. Right? Then you're talking about 250 possible adapters between the 2, and it just gets out of control very quickly.

Yeah. That sounds like a nightmare to have to support all of that stuff.

Yeah. So I could suck at this now. Okay. Yes. The other benefits

that it has is it's a standard ways for inferring both the input and output types, from it. So you can see here here I have, there's an infer input, infer output, and a TypeScript, and that will, give you if you pass it Wes schema, doesn't, again, doesn't matter what type of schema. Could be a Node schema. Could be ValleyBot. Could be archetype.

It will always give you The right, Node, type. The right returned type, which is, again, a lot of like, you think about somebody like Drizzle that needs to, like, maintain the Drizzle to Zod adapter.

Now they don't have to necessarily do that for every single validation library out there. You can simply just use the standard schema inside of Drexel.

And then they they haven't shipped support for this yet, but I imagine that this would be something that they will ship support for relatively soon.

Interesting. Cool. So last thing we have here is, like, a a library that actually consumes a standard schema will simply need to look a little something like this Wes they can run the validate method on it. Again, they don't have to care what library is being used.

And then they will either get the resolved value, which is great, or they will get an array of those issues that I talked about. And and at that point, it's not up to the validation library. It's up to the framework. What do you wanna do now that you have validation errors? Right? Mhmm. Do you Sanity throw them into state to display an error message? Do you wanna redirect the page? Like, is it a post request where you need to redirect the user to a different page? Do you need to return the error as JSON? Do you need to translate those errors into some human readable thing or something that is safe going from the server to the client? That that's up to you. Right? That that's Yeah.

Validation is just a a primitive, and then the actual what happens at that point is up to whatever library framework, flow control you're actually using.

Wow. So, yeah, that is that is the whole idea behind standard schema. It's it's really interesting because, like, again, it's 60, 58 lines of TypeScript, but it it was it seems like a pretty big effort and a lot of thinking, from all of these people to come together and agree on what this these shapes will look like, and then this should make life, again, much easier for library authors and, that need to actually implement validation.

Yeah. I you know what I'm interested in this is, like, do you primarily see end users using this or just library authors? I don't foresee

end users using it a whole lot unless it is something like the inference is really nice. So, like like, one thing with Drizzle is, like, Drizzle has a property on it that's called, like, infer. You can you can take the Drizzle schema, and you can infer your return type from the actual schema. Right? And I could see this being a really nice way to just sort of standardize how inferring those values work, but I see most of the actual heavy lifting here simply just being done both by the validation libraries that implement standard schema and the, libraries like routers and,

Node and Express that actually consume these types of things. Interesting. Cool. Well, thank you for the rundown. I saw this pop up. It's definitely something that's in my wheelhouse, but to be honest,

did not really understand what the deal was. So I really Yeah. I'm too sure that first I thought, like, oh, are they coming out with a new validation library? But that that is not the case.

Not the case. Well, glad to hear. And, interesting stuff. I think this is good. Having any sort of standard connection or whatever

is only good for being able to support all these different libraries that we're constantly inflicting ourselves with. So Totally. Yeah. Yeah. And it it only opens the door to, like, more libraries to to be created.

So if somebody has an idea for better, validation libraries, like, Arqtype came out. They they do it entirely with TypeScript types. It's pretty nifty. They could automatically just start to use it in in whatever, library you're using. You don't have to, like, wait for support to be implemented.

Yeah. That's great. Fantastic. Cool, man. Well, thank you so much. This has been really in interesting.

You're welcome. Alright.