Node.js CJS → ESM

Welcome to Syntax, the podcast with the tastiest web development treats out there. We've got a good one for you today. We're going to talk about moving from CJS CommonJS to ESM rid. ECMAScript modules. So this is a big transition that's been happening in the node ecosystem for a while.

And I just rid. I think it's at a point now where things are starting to break if you're not in ESM.

And rid. I just moved over my entire platform as, I'm I'm sort of I'll talk about it in a second, but, I think it's a good good time to talk about this and sort of go through it because it's Unfortunately, not very easy to do this type of stuff. We're sponsored today by Sentry.

Sentry is the error and exception tracking platform. Actually, this is the perfect I got the perfect ad read for this one, Scott.

So after I converted my entire application to ESM, I tested it and everything. But you never know when you do the first deployed, especially when you are changing the way that you run your application. So basically I deployed that sucker, and then I just sat there ready. Waiting for the Sentry emails to come in or like refreshing the dashboard for things to pop up. And luckily only 1 like 1 minor thing had popped up. And I was very happy, very happy about that. But it was it was rid Very easy for me to quickly go in.

I immediately made that into a GitHub issue, and then, it dumped all the information into the GitHub issue. I could go in, I fix And then you in century, you mark it as resolved, which basically is saying, like, get out of here and don't come back. I hope you don't come back. I hope you don't come back is really what I'm saying. Yeah. Yeah.

And and what Century will do is it will say, okay. Like, you fix rid this. It will it will tell you in which commit you fix it to or you can pin it to a specific version.

And then if that same issue comes back, rid. It will tell you, hey. Like, you said you fixed it. You said you resolved this, but it's now coming back and here's the information about it. So, fantastic. I I love it. I have 0 issues on my node back end right now, and it gives you a, a bunch of robots dancing around. So a big fan of that. Yeah. Check it out. Century. I owe. Use the coupon code PastyTreat. Gives you 2 months for free. Thank you, Sentry, for sponsoring. All right.

Let's ready. Get on into it. Maybe we should just talk real quick about our own experiences.

I know you have been on ESM for a lot longer because because of the,

you are on what's it? Why? Media. You are a meteor. So you you've been on ESM for a while. Right? Well, so I do. Yeah. I guess it's it's not it's not correct to say I was on ESM with Meteor, but, Meteor had the Import style syntax even though it was an ESM as its default means of importing from the start.

So rid What's a very wild thing to me is that I completely missed CJS syntax to the point where When I see c j s common JS Like a real wire? I almost always have to I don't have to look it up, but I'm always like, Wait. What is the how do I do that again? Because I just flat out never use that style as Meteor by default import Syntax style for everything, and then it compiled it to c j s behind the scenes. And then so when I moved the site over to, off of Meteor, I moved it first to just a straight up Vite server with React, and that was ESM based. So I went from imports compiled to, ESM or imports compiled to CJS to straight up ESM, and now with Svelte and whatever.

When I rewrote my API, using ES build, I wrote it with ESM from the start. So, yeah, I I get to control my environment to the point where I can control my node version. I can control everything.

I will choose to use ESM because You either do it now at the point when you're making all the big changes or you do it later at some other point. And, I'm I'm the type of guy. If I'm ripping stuff out, I'm gonna rip it all out at once and then you should put it back together. Yeah. So,

are you compiling and running ESM on the Server or are you compiling and running common JS on the server? ESM on the server for my API. That's what I did as well is that I said, like, I don't because It's kind of confusing as you can write ESM. I'm using air quotes here and compile to CommonJS and you can run-in CommonJS, you can ship as CommonJS.

And then there's the second one, which is like going full go, ECMAScript modules, ESM, and actually running it on the server in.

And that that's what I wanted to do. I wanted to just be rid of all the common JS. So, it's kind of complicated. You hit. There's a whole bunch of gotchas in the differences between the two.

There's lots of different file extensions and whatnot. And then on top of that, we're rid. We're not going to talk too much about TypeScript, but adding TypeScript into the mix makes things somewhat complicated. But also TypeScript makes things Somewhat easier to to actually make this transition.

I would say it makes it easier overall.

Yeah, you ignore the fact that you have to be writing Types in TypeScript. Just the fact of, you know, you you get some niceties for free essentially from TypeScript handling the or they, I mean, I had ES Bill doing the compiling, but your TS config handling a lot of, settings that you might have to help and assist in that area. Yeah. Yeah, I actually,

what did I do? No, I didn't do that. Rid I sorry. Sorry. I had a little bit of TypeScript in the repo already, and then the move over was I still I just turned on allow JS. We'll talk about this in the show where we convert entirely to TypeScript as well. But basically I did use TypeScript as the The bundler for the entire thing, which was kinda nice. Even though 90% of the app wasn't TypeScript, you could just say allow JS, rid.

Turn your head to all the errors, and it's still goodbye. You could still compile it. Just no problem. Yeah. Do you ever see that Simpsons meme that's like,

do not do not enter. And then in, like, small letters, it says, or do.

I'm just a sign There's something. It's like like, I can't enforce it. I'm just a sign. I'm just gonna say it. Yeah.

Yeah. That's the that's the TypeScript with, With Java. Allow JavaScript. Yeah.

Alright. Let's talk about why ESM. Like, what's the benefit? And And quite honestly, it's a lot of work with very little benefit to it. So rid there are a couple benefits to it. So let's talk about what they are, but it's not gonna be there's nothing amazing. Rid To to the point where a lot of people think, like, maybe ESM was a mistake, because there was nothing wrong with CommonJS. The reason We're annoyed with CommonJS right now is because we have all of this interop that we have to do between the module formats.

But back when it was just CommonJS, rid. Things were good. But then also people were not using things in the browser. So let's talk about the benefits of of moving to ESM. First of all, It's a spec. Common JS is it is a spec, but it's not like a blessed one by the people who create JavaScript, TC39.

So By having a spec, they can implement it in Node. Js, in Deno, in the Cloudflare Workers, in the browser in like literally anywhere that wants to support JavaScript. They have to implement that spec and make it work according to the spec so that ideally you'd be able to share code between your front end and your back end with no issues. I didn't realize that common JS was created by Mozilla.

Was it really? Up some history on this. Yeah. I didn't either.

Comment JS was started by mo by Mozilla engineer Kevin Dangore in January of 2009 to address the lack of common accepted modality practices in programming.

Rid. Yeah. No. I didn't I didn't necessarily realize that. Maybe yeah. Maybe if it wasn't a Mozilla I I don't see here if it was, like, a Mozilla thing or if it was just Kevin doing it for himself. When was Node. Js created?

Node. Js was initially released May 27, 2009, rid. And this was created January 2009. 2000. Wow.

So props to Ryan for taking this 3 month old project. Did I don't know. Didn't did Node have

Modules? I don't ever remember. And and I was very early in Node. I don't ever remember not having a module system. Yeah. I don't either. I remember going to, like, a meetup that was, like, rid. Very early on, and, like, it took an hour just to get an express server up and running for everybody and then deploying to Heroku.

And just because it was so foreign to everybody at the time, you know, like, what are we doing? And then by the end of it, I remember being like, wait. Why are we doing this? Rid My god. I'm just serving an HTML file. What what why is this so interesting? Yeah.

Oh, that's good.

Rid. What else do we have here? It's apparently more performant.

And some of the reasons behind that is, the different one there's a bunch of differences between CommonJS and ESM, but, rid. One of the ones is that ESM is what's called statically analyzed, meaning that, you can rip through the entire rid. Require or import tree, in a single pass and know exactly what dependencies are needed by your dependencies are needed by your dependencies. With Node. Js, Like, this is one thing I had. You can import something inside of a function. You could you could make a import, or you could, I had, like, a string that was like of my config file was config dash development and config dash production. And I would just take the note M rid Mhmm. And, interpolate it into the string. You can't do that with with the SM because that's not static.

So rid. The the benefit to that apparently is, like, better tree shaking, meaning that you'll be able to figure out which parts of the code you do and do not need. Rid. I know, like, Vite and Webpack and all these things have gotten pretty good at figuring that out, for all module systems. But,

rid. I guess being statically analyzed makes that even easier. Yep. And, I mean, you've you've stated this a couple of times, but the fact that it's a browser rid. Back that works in the browser is a big deal.

You know, we do this in one of our our JavaScript courses where We say, hey. We're gonna be just writing a script tag here. And in the script tag, we're importing things. And there's just something really magical about that Yeah. To be able to just straight up use import syntax in JavaScript without any compilations In HTML. Yeah. I I remember being shocked the 1st time I did that. Wait. Like, you're allowed to do this? Holy cow. Yeah. It it's awesome. Even like, I do a lot of demos And often I'll do

a script tag. And if you put type a module on it, you can you can import stuff from from URLs, but you can also use top level await inside of a script. Top level of weight. Yeah. Yeah. That's that's the next world. Yeah. ESM allows you to have a top level away. And what that means is you can await things inside of a module without having to have an asynchronous function. So Previously, what people would do is they would have a function that is a sync. I would I would have a function called go and then it's called start.

Yeah. And at the very bottom of the page, you run it or you have an iffy that's a sync and it runs immediately. That's annoying because if you want to, programmatically export something from that module that is based on the asynchronous data, like you need to load a file in ready. 1st and then export some data from that file. It's not really possible. So having top level await inside of modules is super, super handy. Even like, Some of my modules I will.

Import I import like a list of stuff from GitHub when I hit it and the 1st time it's run rid.

And it's nice not to have to,

like, have a have a iffy or something like that. I'm a big fan of that. Yeah. And sometimes you just import a file, And you want that file to run and do something. You don't need to even wait for that promise to resolve or do anything like that. Like, you know, I just want, You know, this thing to start up and start doing its thing or or this to initialize, you know,

Sentry. I just want Sentry to initialize, so I import the file. When that file Opens up. It will start to initialize, you know? Yeah. You you fire it up and you don't care about like like, obviously, you can catch any errors and log them if that happens, but you're not waiting for rid The thing to come back, before you can do do the next thing. That's actually a good question I had.

Maybe this is an entire different show, but in so in my app platform I have when somebody buys something, there's a number of, like, things that I do. I send in send them an email with the product.

Rid. I send them a notification to my phone that someone bought something. I add them, to my database.

There's some newsletter markings that happen if they're in there that say they had bought something.

And all of those things. I don't wait for any of those things to come back because any one of those things rid. Could that could fail or take a longer time? That's kind of annoying. And I thought, like, okay, like, I'm logging any errors, but, like, maybe that's a better

rid. Better use case for a queue. Right now, I just kinda fired off into queue. Yeah. Yeah. And I was like, maybe I need a queue. It is probably a better case for a queue. Yeah. Toss them into a queue, And then that queue can if it fails, then it can retry.

It can Mhmm.

Yeah. Yeah. But then you just that that's the world where you get, like, here's the The good, better, best kind of solution, right? Yeah. I was sitting there thinking like,

I'm just assuming all of this stuff will work. The only the only one I'm really waiting for is like the credit card charge.

And then, like, it's never been an issue before, ever rid. In the like 8 years I've been running it, but I'm thinking, oh, maybe it should be in a queue. Like you said, the one day that the database is not able to connect, you know, then there's a charge without a database.

Mhmm. And that's a bit of an issue. I have emails sent to me, like, Custom alerts outside of Century that are like, hey. This thing that's not supposed to be happening is happening just in case, you know?

Yeah. Just so I can yeah. Just some extra alarms there. That that's a good case. I actually have a a card for a logging show, which is it's different than Error tracking and whatnot, but just straight up logs. There's a lot of systems for that. And I was thinking about like maybe we should do a show saying, like, what do you log? How do you log? Where do you log in? Yeah. How long do you keep

development verse production logging. I have in I wrote this, like, really nice little handy, middleware in SvelteKit that analyzes all of my requests.

Yeah. As they go, just just timing them. And based on how long the request takes, it gives me different emoji.

So if the request is is is really fast, It's a rocket ship. If it's kind of fast, it's a bunny. If it's slow, it's a turtle. And ever it's just like in my console while I'm just browsing the site, I could just see. And it has what type of request, what path it was to. And it's just, like, at a glance, I could see, oh, I'm in turtle time right now. These requests to this page are all slow for some reason. Let me take a look.

Rid That's great. Okay. We'll we'll put that on the books. I think that's a good show, is logging.

What else? A lot why else your nodemodules folder or from a local file.

Eventually you'll be able to import from a URL that is what Deno and BUND support.

Rid. And then also, Dino and BUN also support just npm colon package name, Which is really cool. You don't have to npm install something that's not part of Node as well. But Node does have a flag experimental network imports, which will allow you to rid. Import from a URL, but I wouldn't touch that until it's ready. I've I've gotten bitten way too many times on that kind of stuff. Even I haven't been touching that, and I

love Love to get into stuff like that before it's ready. And, yes, some for me too. Like, it was I I started at ESM way too early, and, like, most of the things I hit were packages Not publishing ESM versions yet or versions that, were angry with ESM, like, specifically would break if you tried to do it. Even, like, DoorDash had some issues for a while, so the the most of my issues were resolved once the community caught up a little bit. Another neat thing is that in stage 3 is import assertions, which would allow you to import different types of files rid with a specification at the end of the import. So you do import thing from thing and then the keyword assert and then an object, and that allows you to Assert what type of file this is. Is this a JSON file? And that would allow you to straight up import, like, a JSON module With the eventual event eventuality of being able to do, like, things like assert web assembly or any different types of, files that you could import into JavaScript and even have them be, like, a transformed version or something. So, that's that's interesting. Yeah. Yeah. Even just, like, A couple of times I've just wanted to import text from a file.

And then you have to you have to, like, use the the node read file API.

Well, let me tell you, if you're on Veet, which gotta love Veet. Veet has the best little helper for this, and it's the raw imports.

Yes. It's seriously one of my favorite things about Vee, because sometimes you do just want to import a file as text and not as JavaScript or anything. And I did this when I I wrote my, like, storybook thing. I was importing a file as text to parse some things out of it. And I found this to be really super easy, and the way you do that in Vite is just by import whatever you want, question mark, raw.

For your prem? Rid. Yeah. I did this for our m j m l templates even. Oh, that yeah. That that's the perfect use case for it. And then you don't have to do the read file type of thing. Like, I want what does that compile to? Probably node APIs?

Oh, that's a great question. Haven't looked.

Or it just puts it in a string and makes a JavaScript file that exports that string. I got to imagine that's it. Yeah. I mean, probably I could look at my chunks.

It's I think I'm not node APIs because that would not work if it was client side. Yeah. You you talk and I'll I'll I'll check out my chunks while that's going on. Okay. I also use that import assertion the other day in another project. I needed to log the package. Json name and version. I was working on like a MPX script and I needed to figure out what the current version was.

And I The way that you require your in CommonJS, you can require JSON files. No problem.

In ESM, you may not require JSON files. Rid. So the the fix for that is these import assertions. It's not fully supported everywhere yet, and I ended up having to rid. Fall back to just, node read file APIs, unfortunately.

But eventually, that will be there.

Actually, the reason why I fell back is because I needed an ESLint plug in to support linting those types of assertions because it's a new syntax.

But I was just like, I'm not doing this. I don't feel like adding this another plug in for that. I'm just going to read file and move along with my life. Yeah, there there's an element of that to all of this stuff. It's like,

What can I do to just get moving at the end of this? Yeah. Did you find it? Or No. I'm still looking.

I I built my stuff.

I have my chunks.

There's a lot of them.

Scrolling through them.

Rid I don't see anything named

the thing that I would be expecting it to name. The behavior is similar to Webpack's file loader. The difference is that Import can be used using absolute or public paths.

That makes sense because Webpack's file loader, you like tell that how to handle different types of mime types.

It must be. It must be a string. Yeah. I'll find it eventually here. Let's talk about actually making your project ESM.

So if you have an existing project, there are kind of a a couple different ways to go about it. By default, Node thinks your project is CommonJS, and therefore, dot JS files are Assumed to be CommonJS.

You can flip that by putting type of module in your package. JSON. And I do that so much. I kind of wish it was part of the npm init thing where it would ask you.

But the sort of Shortcut to that is you can run npm pkg set type equals module, and that will update your package JSON. That feels like more work than just Going into the package. Js and change again.

I only do it because I have it in my like, warp history and it pops up. It's like, are you trying to set the module? But yeah, it's kind of annoying that you have to do that every single time. So when you flip it, now it thinks that your .js files are ESM Using import export and your your other files that are having a .mjs extension are going to be common JS. So There's a lot of a lot of people get up in arms in the ugliness of the new file extensions. So there's there's 2 new file extensions. Dotmjs for modules and .cjs for common JS.

And then the .js could be either. It could be depending on if you've set that type of module.

And I've come to think like, okay, maybe this isn't such a bad idea because of how the interop needs to work.

And if you have a common JS project and you need to rid. Create it or you're like, All right, well, all new modules going forward are going to be ESM.

You don't need to do anything other than just name it .mjs rid. And everything will will start to work.

But I should say that, ESM rid. Can import CommonJS packages? No problem.

CommonJS cannot import ESM packages Synchronously, it must be in a sync function. There's no top level away. Then you start getting in a world to hurt.

And we're at a spot now where A lot of the big module authors, all of Sindre's stuff, all of the the the whole MDX ecosystem, it's built on Remark. All the Remark of the only. I use Facebook. Yeah. A lot of big authors are saying, you know what? ESM only. And that is forcing a lot of, rid people, myself included, say, alright, it's time to to flip this sucker over. For me, for rather than having to do the dance Where I was like some ESM and some common JS at all time. I said, you know what? I'm just gonna go YOLO, go for the whole thing.

I flipped Type A Module on, and then I ran what's called a code mod on my entire codebase. Oh geez, yeah. And it Transformed every single require and export and module that exports to the closest equivalent in rid.

In ESM, and they're not totally the same. We'll talk about that, but that gets you a lot of the way there. And that was nice. I didn't have to fuss with any fancy names. There's no, like, renames in my history.

It's all just simply JS files.

Yeah. That's that. Honestly, that's Probably the single biggest benefit that I had to avoiding CJIS syntax in my code base from the get go is that I didn't have to change a single import statement, and they were already import statements, and it just worked. So it was more or less making sure the modules are all supported as is. By the way, I did find the MJML stuff, and I'm seeing it as a string inside of one of the chunks. Just a big, long string inside of chunks.

And I don't know if you've ever looked at these chunks files before, but A big chunk fan.

The the I'll I'll show you this screenshot here. I'm very confused at what's going on in here, and I would actually love to figure out what What the heck this is, actually doing here? This is within one of my chunk files, and I haven't really pawed through these things before. But it it's it's maybe, like, about a 1,000, semicolons.

What?

Rid. Yeah. I'm very confused.

This is one of the the Vite build chunk files. And before that, it's like a string of What almost looks like DNA sequences, where it's a c d d c a e which I would imagine are just, you know, generated variables or something, but I've never told me that as well.

Yes. I've never taken the time to look at these. Not your source map? I dude, dude, this is oh, this is the source map.

Rid it's a dot net. No. You're right. Okay. Okay. So this isn't the chunk file. This is the source map, which actually makes way more sense. Sorry. I did a find here, and I usually have maps excluded from that. So

yes. Okay. Imagine you're shipping rid. 30,000 semicolons?

It's it's way smarter. Okay. Way's way smarter here. The chunk is still just a string. So this the the import, the raw import is still just a it's now just a constant variable string of the template.

Okay. Okay. Yeah. That that makes way more sense. I was wondering how they did that. That makes a lot of sense. I like that. So but can you explain to me what's up with the, semi colons in the import map regardless? I have no idea if maps are there's something about pointers and references to lines of code.

Rid. But I don't even I've never actually looked into that. Like why? I'm gonna tweet this out because I think this is fascinating.

Yeah, there's gotta be Some interesting like thing about like how many semicolons is how many lines or something. And like it doesn't necessarily matter that it's 10,000 semicolons because That's what Gzip does, right? Sending 1 semicolon or 10,000 semicolons should be the same size. Or like a broad lay compression as well. Yeah. Yeah. Plus, like, oh, maybe that's what yeah.

But plus, like, no one's downloading your source maps unless you pop up a dev tools or century is trying to figure out where you're like the actual error happened in a compiled version, but it wants to show you your source code.

Rid. Yeah.

Alright. Let's talk about converting to ESM. So I talked about the first thing I did was the code mod syntax.