Supper Club × Shipping ESM with Mark Erikson

Welcome to Syntax, the podcast with the tastiest web development treats out there. Tara, today we've got a good one for you. We have, Mark Erickson on who is maintainer of A massive amount of, Redux itself, as well as a bunch of, react Redux, rugs, redux, dunk, monk, Monk, you know all those packages that go along with it.

And The the read the Redux family of pack the Redux family. Yes. Love it. The Redux.

I follow Mark on, Twitter, which is a good follow. You certainly should do that as well. About the same time, I was working on the show for Syntax on converting to ESM, and I had moved my entire platform over to ESM. I was hitting all kinds of issues left and right and and whatnot. And

A lot of it stemmed to

the package authors have to ship them in a way that works absolutely everywhere, and It's very close to impossible to do so. It's very complicated. I saw Mark, very smart guy, maintains Redux, maintains all these packages, Having trouble with it as well, and I'm just like, there's some something going on here, but he figured it out. Right, Mark? You figured it out?

I I have figured some things out. I I okay. Like, preamble here. Number 1, I think people assume That library authors, possibly like myself, actually know what we're doing.

And that I think that may be an overoptimistic assumption A lot of the times, you might be surprised to know how many dirty, ugly hacks I have shipped. It's just like, well, I tried it a couple times Yeah. And it looks like it worked.

Ship it, see what happens.

I mean, like, I I put actual care and effort into things. Yeah. But a lot of times, it's just sort of a best faith. Well, I hope this works.

And, yeah, I've I've learned a lot over the course of this year trying to mess with this stuff, but I still genuinely don't feel like I actually know what I'm doing To a to a very real extent, and I've been begging for years for someone who actually knows what they're doing To write some kind of a comprehensive authoritative guide to, like, all the things you need to know about publishing a package in today's ecosystem.

And so far, no one has done that, or at least not not to the, like, the full extent that I'm Picturing in my head the resource I wish I had.

I've I've found some resources, and they've been helpful.

But a lot of this is very much Trial and error and trying to assemble bits and pieces the other people mention.

Yeah. It's it's a super complicated world, and, Like, we'll get into all the permutations of it. But, like, I was just like, you need to write this source because literally there's not. There's some, like, stuff sprinkled in the docs, which is helpful, but then you get into, like, supporting older module types and and touch script, and You just wake up in the corner sort of rocking. Yeah. I I put up a like, in the middle of

the journey that this year has been, I I put up a tweet back in April where I said, like, here is a list of all the things that I have to think of as a package maintainer In, you know, trying to publish a package in 2023, there was, like, 10 or 12 different bullet points, And that like, I probably could have come up with another half dozen easily.

And, you know, that that tweet definitely got some attention that, know, ricocheted around a bunch of retweets, bunch of agreements, you know, bun winked in a bunch of places, and, yeah, it Struck a bit of a nerve. Yeah. It truly feels like it's

impossible to know exactly what every setup everyone's rocking with, what Bundlers, how many different ways of consuming what? And as a con as a user of these packages, it does feel Like, there there are so many instances in which you install something, and you're hit with an issue where the way that the documentation says to import it, is not correct. You need to import, a non default, or you need to import asterisk as default or you know, you you're just Kind of guessing, and you and you know generally how to work with these things, and you get it working, or sometimes you don't, I suppose.

But I can't imagine being in a position where you have to maintain things that so many people are using in so many different setups, across the board. Has it has it gotten Easier over the past 5 years, or is that has it gotten harder now that ESM has become,

you know, everywhere here? Oh, it's it's a lot harder, but, I mean, to a certain extent, that's because things had worked for a while, and I didn't mess with any of this stuff for a long time. So to to recap just a little bit, you know, I maintain Redux core, reselect, Redux thunk, React Redux, and Redux toolkit.

I'd shipped some major versions for a few of those over the last couple years, especially React Redux, but the packaging contents had not changed in a long time. And so the the driver for a lot of this is that we like, at the start of the year, we decided it's time to ship Redux toolkit 2 point o, the first new major version of that we've done came out in 2019.

And that kicked off the realization that, well, actually, we need to do major versions for all of our packages.

And the big thing that I intended to do was get, quote, unquote, proper ES module compatibility, whatever the heck that means. Yeah.

And modernize the bundle output, you know, stop transpiling things to ES 5 for IE 11 compatibility.

You know, just try and get things modernized across the board. And so, like, our packaging had not changed for any of these libraries in a long time. And we were still going off of build setups that were years old.

And so it's been through trying to work on all these that I've Suffered and Yeah. Learned a lot of different things over the course of the year. So let's let's talk a little bit about,

Like the different types of modules that people need and because people might be listening and be like, well, why don't you just write the whole thing in ESM, Ship the whole thing in in ESM. It's a standard, and it works, and and it's all good and and whatnot.

But the reality of it is is that, like, people consume them In, I think, 3 different module types, and then there's all these different tools. So, like, can you tell us about all the module landscape out there? Okay. K. History lesson.

I'm I'm I I am a really big fan of understanding tools and ecosystems in historical context. Like, We're here, but how did we get here? Like, what what brought us, you know, to this place? Whether it's talking about, you know, state management tools or modules or whatever. So the first issue is that the JavaScript language never had a built in way to define packages or modules or export things In the same way that, like, you know, Java or c sharp has, like, a package keyword and Mhmm. You know, things are associated by file paths and whatnot.

So, originally, it was just you have little in line script tags in your HTML or little bits of JavaScript directly in a string event handler, and then people began writing larger programs, and they're like, we need to organize this code kind of like other languages let us do.

So one of the one of the 1st module formats was actually using the, you know, immediately invoked function, whatever it is, construction.

We just we just finished a

show on closures where we explained that, so that's Mhmm. Timely.

Yeah. So, like, you you wrap your JavaScript code inside a function and immediately call it. It provides a form of encapsulation.

You can control what it returns. And this is actually kind of the 1st pseudo module basis.

On the browser side, there was a module format called AMD or asynchronous module definition.

And this allowed kind of like a waterfall approach where libraries like Require. Js Could load the 1st module, see that it depends on modules named b and c, go download those, recurse, recurse, recurse.

Eventually, all the leaf modules are loaded, and then it unwinds and loads the rest of the application.

On the other hand, the node ecosystem Came up with the common JS module format, which was designed for synchronous execution, directly reading files off disk Immediately.

Then you had UMD, universal module definition, which is a bizarre wrapper That could simultaneously 1 file be loaded as either an AMD module, a common JS module, Or a script tag assigning globals to the window.

AMD is basically dead at this point.

UMD is mostly getting shipped for use as, like, CDNs.

So maybe I I wanna make a library available For use on, like, a code pen as a script tag link Oh, yeah. Or something like that. Like, an unpackage, something like that. People are just whipping up a quick demo. We put the script tag in. And so with the ES 2015 spec, the language committee actually formally defined a module syntax With the import and export keywords and some aspects of how it's supposed to behave, but they left Some of the lower definition about how the modules get loaded and how do they interrupt with each other up to the runtime environments to decide. In other words, browsers, Node, Deno.

And that's where a lot of the trouble started. Browsers kinda went ahead and figured things out, But Node had common JS interop to worry about.

And so the process of Node defining, Here's the technical aspects of how common JS and ESM will interrupt, and here's how we will even figure out whether a given JavaScript File is supposed to be common JS format versus ES module format has been a very, very long complicated journey that I will not even attempt to recap here.

I think we about a year, year and a half ago, we we attempted to go down that route because it all starts with the package JSON where you have type of module and then Your j s files are e s m. But if you don't have that, then your j s files are c j s, and you have Michael Jackson script. And and it's A lot of people are frustrated being like, why did they botch this so hard? But there was, like, a there's a lot of, like, There was a lot of problems that they had to sort of think through. And the fact that especially because common JS is dynamic, you could just say, like, require back text config dash process. Like, I did that in my thing. I put The process dotenv variables into my requires. You can't do that because ESM is statically analyzed. Right? It has to be has to be able to crawl the tree and and figure it out. So there's some some differences there as well. So where this really starts to become a problem is that, Traditionally, publishing a package to the npm registry has involved

shipping the lowest possible common denominator as the build output Pure library. So you might be writing your code in ESM syntax and, you know, TypeScript 4 dot whatever, But you have to assume that the consumers I I don't know how they're gonna use it. It might be Node. It might be Webpack. It It could be, I don't know, in a browser somehow.

So, traditionally, it's been converting the modules Mhmm. To common JS format because more tools understand that, And transpiling your fancy latest and greatest JavaScript syntax back to ES 5 because, who knows, one of my consumers might be targeting IE 11 then further application.

And generally, like, trying to imagine here's all the possible ways that someone might Want to consume my library.

Am I shipping something that will work in that particular possible environment? It's true because we

We talked about the e s m troubles, and Scott said, you know what? I've never had any troubles with e s m. And I was like, you know why? It's because you're writing e s m. Yeah. Yeah. But it's being converted to common JS. And all the packages that you're using are probably still common JS. Like, if you go into NPM and look at the exported code, it's all Common.

JS, a majority of it is still Common. JS. And this us using ESM is It's just at an author time, and then we use some sort of tool to to go through it, which is why the transition has been so, so painful.

What is the benefit to going all in on ESM? Is there is this worth it? Because I that's honestly what I went through when I switched it. I was like, wow. That was a lot of work. Yeah. Yeah. Right. Is it worth Why did I do why did I even just do this? I'll I'll be honest. I don't have a good concrete list of benefits that I can point to off

The top of my head. I I know that there again, there's there's been lots of discussion. I've read lots of articles.

I don't Have a specific list of, well, here's what you get as a result.

And in some ways, there's actually even a couple potential cons.

For example, I saw an article from Jared Sumner, the author of the bun runtime. Mhmm. There there there was a back and forth. Someone from Deno wrote an article Saying, like, common JS is killing the JavaScript ecosystem, and Jared Sumner came back and said, well, actually, common JS is still, like, the majority of the registry. It's getting used more. And in fact, because of the way the ES spec is defined, importing, parsing, and loading an ES module file Can actually be like, I don't know, 30 or 40% slower than if the same file was written in common JS format.

There I'm not saying there aren't technical benefits. I'm just saying they don't have a good concrete list to point to right this 2nd. Yeah. So why not standardize on

CJS,

you know? I I think some of it had to do with wanting, you know, that that static analysis capability. You know, the fact that a common JS file Could just at runtime say module dot exports dot some field equals whatever, and it it could be very, very dynamic, And that's not as friendly for tools? I mean, I I guess the the biggest single benefit that I would point to for ES modules as far as, like, the browser side ecosystem Would be tree shaking. Mhmm. You know, the fact that a a tool can statically analyze a file and say, here's all the variables that got exported, And I know that the rest of the code base only imported a and b, and therefore, like, We know c didn't get referenced, and we could just leave it out of the resulting bundle. I know that's easier to do with ESM modules.

I've seen a couple examples of, like, plug ins for parcel that can do some of that for common GS, but they might not be as efficient. Yeah. I was just gonna say the same thing is

That was that's that's the thing everyone touted is static statically analyzed code, meaning that you can peer into 1 file and see The web of trees.

You can see every little, it require another file, import something. That file, import something. You're able to see that, whereas with common JS, you're you're not able to see that because it's possible at runtime.

You would literally just Export something or dynamically require something which could be a bit of a pain. So you, the author, needs to then Ship a permutation that works in ESM, CommonJS, UMD, and you also said development production, And then throw TypeScript into the mix there. You start finding yourself like, I remember I was shipping a tiny little library, And I think I had, like, 9 different permutations of things that needed to be tested.

And and, Like, that's just a a silly little library that yours is is obviously much bigger. Right? So is how many do you even know how many different permutations there are? I I'd have to go double check,

but I think I'm shipping on average between 5 10 different unique Build artifact output files per package entry point.

And then and then Redux toolkit Specifically has 3 different entry points because you've got the core library, and then you have the RTK Query data fetching add on That is available in either non react or react specific flavors.

So there's 3 separate entry points Times the 6 or 8 different artifact files.

So, like, at the moment, I'd I'd I'd have to go double check, but I think for RTK, with the betas right now, I'm still shipping Common JS development and production builds, a fully modernized ESM file that has, like, all the latest and greatest JavaScript syntax, none of that being transpiled away.

A Webpack 4 legacy compat e s m file that is still e s m syntax, but it's a dot j s extension instead of dot mjs.

And it targets ES 2017 syntax because Webpack 4 does not know how to parse The syntax for object spreads or optional chaining.

I'm ship I am I've stopped shipping UMD bundles for the upcoming releases, but I am shipping a browser Targeted ESM file that has been precompiled with, you know, process n of node n of equals production Because the the other ESM file still has the note end checks in there, and therefore, it would not run straight away in a browser.

So, like, that browser ready ESM file is kinda like the replacement for a UMD file if you wanted to load it off of unpackage In the script tag Mhmm. I think that's it off the top of my head. But, like, you you like, you see, like, even with all that, like, there there's at least 5 or 6 different files in there just for 1 entry point. Yeah.

How do you define like, you so you're shipping this. You npm install something, and you get Ten different versions of this thing.

And your user just simply imports the thing, and it works beautifully.

How are you defining that? It works. Yeah. But ideally, it works beautifully.

How do you define

Where that is, is that all done in the package JSON? Yeah. Yeah. So there are a series of different top level keys that different tools look for.

So node traditionally looks for a top level field called main, and it expects that that file is going to be common JS.

Browsers sorta or sorry. Bundler sort of invented a field called module, and they would normally expect to find an ESM Formatted file there.

Other tools, like like, unpackage literally looks for an unpackage top level key As like, this is servable if someone is trying to use unpackaged as a CDN kind of a thing. I think CDM JS maybe maybe did the same kind of deal. Mhmm. TypeScript looks for a top level types key.

And then the and those are all kind of the outdated ways to do it. So the the more modern way to do it is that you define a top level key called exports, and it's Actually, like its own little weird nested language in the form of JavaScript object keys.

So top level key is exports, And that should typically be an object.

And then inside of there, the first level keys would be, like, the names of your entry points. So, like, you know, you could have, like, just a dot would be, like, the primary entry point.

Or in the case of Redux toolkit, you know, like, ./query, .query/react.

And then those can be nested objects.

And in the nested objects, you can say, Here's where to find the types file for this entry point. Here's where to find, the import key Says here's where to find an ESM file.

Require says here's a common JS file. Default is a fallback that might also be common JS file.

But then you can have even more nesting by adding additional conditions like development or production, again, as like Arbitrary object keys. And so you can just sort of, like, recursively nest these in multiple levels of detail To try to define the the sequence of here's how this tool is going to find this file, and then as you said, hopefully, it just works In the end?

Yeah.

Yeah. You mentioned, you know, one of the permeations being like a Webpack specific thing. Do bundlers Get in the way here more often than not, or do they end up smoothing out some of this pain? Well, bundlers are great,

but, I mean, just like any other piece of software, a a it exists with certain capabilities at a certain time. And so, you know, Webpack 4 was Widely used for years. It is still widely used. It's it's what's in create react app version 4, like storybook 5 or 6, I think.

I know Expo's WebTarget use used Webpack 4. It might still not a 100% sure there.

So, like, Webpack 4 still gets, you know, millions of downloads. And so I, as a package author who wants my users to just have this stuff work out of the box Yeah. Need to take that into account, but the Webpack maintainers, if you look at the issues, very specifically said, oh, we're we're working on Webpack 5. We're not gonna, like, build Support for the exports keyword into Webpack 4.

And so I still have to ship a bundle using the main The the, like, the module keyword with slightly older syntax just so that Webpack 4 is happy. Man.

Because it it you can also run into problems as if you, the package author, have, like, dependencies upstream that don't necessarily work Well, with you as well, because it's not just how you work. Did you run into any of that where you had to go upstream and tell somebody else, hey, can you can you fix this thing? Like, Even on the syntax website, as I author in TypeScript, the editor is happy about my, like, named import, But in reality, it's a module exports with as an object, you know, and their typescript does some dancing to to allow you to make it a fake named export, but then when I bundled it, it broke. You know? Did you run into any of that? Yeah. That that was actually the where my 1st attempt to update Redux Toolkit's packaging

failed.

So Redux Toolkit has 4 dependencies, Redux Core, ReSelect, Redux Thunk, and Emer. And the good news is I control 3 out of those 4, but my my very first PR that Tried to modernize Redux Toolkit's packaging. I I added type module to try to declare that everything in here is ES modules, I swear.

And, added the package exports field.

And what I immediately found was that Jest, our test runner, Was choking on both Redux Thunk and Emer because they Redux Thunk really only has a default export, And then it, like, actually attached a a nested field to that default export, so you could sorta kinda pretend it was a named export, but not really.

Immer had both a default export and named exports.

And when I made the ES module switch, Suddenly, just was getting confused.

And if I did, like, import thunk from, quote, Redux thunk, Instead of handing me back the default thing, it was handing back an object with a key of default inside That had the default thing, which of course broke.

And I I had a similar problem with Emer.

So I was able to publish a new Redux Funk alpha that tried to make some changes. It sorta helped.

Emer, I think my first Fix was just switching to using its named imports instead.

And then Michelle Westray, the author of IMR, had already been planning to work on Immersion 10 this spring.

And so he was actually in the process of getting ready to, like, drop the default export, probably because I said it was causing some issues.

But, yeah, like, if if there were more dependencies involved, that could have gotten even messier. I guess on that same same,

line about default export. Do you think default exports were a mistake or default imports

seen people arguing pretty vehemently against them.

I don't I'm not like I'm not on a crusade against them, but I've certainly leaned more towards named imports for the most part, both as a library author and day to day development.

I did see a very good explanation.

Tejas Kumar did a a talk at, React Rally last week where he built, like, a tiny, tiny version of a framework like next in, like, 15 minutes.

And one of the things he showed was, like, why why do these frameworks ask you to do a default export support of your component, and it's because they're doing, you know, either a common does require or, like, an await import.

And we don't know what named field your component might be at, so it's easiest just to grab the default thing And then, like, render that component at the framework level. So that part makes sense.

But, I mean, just general principle, I'm leaning more and more towards named import exports

as a whole. I just found the the tweet I had publishing the packages, and it was to test it in all the different environments was 8 different environments where it's 4 people who have type of module set in their package JSON needing to import Into a JS file, a CJS file, a CJSTS file, and a TS file.

Because the like I said earlier, the meaning of what a .js file is changes depending on what's in your package. JSON. Right. And then on the flip side, you have people who are running common JS still but want to import it into JS and, like, Hokey.

And when I was doing that, I ran into problems where, like, you can't use some features like top level await or, What's the other one? Import dot Import dot meta or something like that? Yeah. Import dot meta, something like that. Did you run into any of those issues as well, or does the bundler take care of transpiling Those things to older version?

Oh, that that was I mean, that was actually the train of thought where I where I ended up at after this first attempt Exploded in my face. It's like, okay. Like, I I went out and I did some research, talked to some folks like Andrew Branch from the TypeScript team, Matush Brzezinski who's you know, knows way too much about this stuff. Yeah.

And it was, like, it was it was very clear that me just attempting to reason about This in my head was not going to be sufficient. I needed sources of truth.

So I ended up I I wrote a tiny little example app in Redux Toolkit Whose only job was to import from all 3 entry points.

And I then proceeded to paste the example app code Into, like, 4 different projects. Create React app 4 with webpack 4. Yep. Create React app 5 with webpack 5, Vite, And next and just and I set up c a CI job that would just build each of those, so it doesn't compile, And then actually run a tiny little playwright test that just verifies that everything interacts okay on the screen.

And then I also tossed in a pair of, like, similar, like, node related exercises where, like, one is, like, node In standard mode, another is note in ESM mode, importing from either an ESM or a CJS file, and with some assertions about here's what I Got imported in from what path? And like, it feels like the ecosystem would really benefit From having some kind of this as a service.

Like, let me provide my library Yeah. And some snippets of example code, and then you go off and build and run this against a dozen different Tool combinations and tell me where it explodes.

I like that.

What about this, are the types wrong? This is something I run into every now and then where it thinks it can't find the types, but they're clearly there. And if you or you right click go to definition and it Doesn't show up.

Did you have any pain around that?

Not well, not pain. This was actually a hugely wonderful discovery.

So Andrew Branch is a member of the TypeScript team, and he has been he has both implemented a number of module resolution features. Like, I believe he implemented the new module resolution node 16 as well as the module resolution bundler options.

He has been working on writing documentation for the TypeScript website that explains a lot about how TypeScript views modules and the state of the world and tries to find things.

And so because of that, he started writing what was initially just a website, Are the types wrong? And the idea is you go there, you type in a package name, version, or maybe you upload a local package tarball, And it scans all the package definitions and it says, here's what I think is there and here is how TypeScript is going to interpret All the stuff you've defined, and it actually flags like a half dozen different common problems that he's seen in various work packages in the wild.

And when I first ran across it this spring, it was just a website, But I looked at the repo he had, and he had it split into folders for the website and the core logic.

And I'm like, oh, I I need to make use of this somehow.

So I actually wrote a very, very ugly hack where I would, like, clone his repo in CI, and then I had my own script that would import from that And, like, try to make command line calls and scan my own package and then print the information out in a table at the command line.

It was a nice little proof of concept, and it worked enough for me to make progress.

There is now an official CLI wrapper for that capability, And I actually still need to switch our own CI jobs over to use that CLI tool, but it's it's great. Like, being able to see, like, especially from the command line in local development, like, given the current packaging definitions, here are the problems that are showing up is Huge.

There is also another sort of similar tool called Publint, that tries to look for some potential Packaging issues.

I've only run that one a few times. It hasn't actually found anything wrong. So either it's either I'm Close enough to write that it's not complaining or it's just not sufficiently detailed. I don't know. Yeah. I ran mine through that, and It told me that the types need to be defined before

something else.

I think it caught a bunch of stuff in mind, so It's a really great tool. The type of people that go deep on this stuff, like, bless them because this is

this is hard. It's not the Stuff I would wanna be going deep on, but I I appreciate that they, chose to do so. The it's TypeScript, and, like, the community's general large adoption of TypeScript, do you feel like that has made this process a little less Painful?

Yes, no, and yes.

So TypeScript is wonderful. Like, I love TypeScript. I am a I am a very much a die hard TypeScript fan. I basically refuse to write any code that's not TypeScript going forward in my career.

But it's also true that as a library maintainer, it adds at least another 2 or 3 levels of complexity that you wouldn't be dealing with if you were only writing and publishing plain JavaScript.

I I actually did a talk for TS Congress Last year called lessons learned maintaining TypeScript libraries, and I detailed a lot of the efforts we have to go to to, you know, write and publish our types, Test our TypeScript types, you know, like, all that kind of stuff. And so in the same way, like, as a As an application author, I want my libraries that I use to have good TypeScript types.

And, therefore, as a library author, I need to make sure that my TypeScript types work and that they're published correctly, but it's more tools and more levels of complexity. And so even something like, you know, should you pre bundle your TypeScript types into a single file versus leave them as individual .d.ts files. Or and one of the one of the actually remaining outstanding things that I still Probably ought to try to fix with our betas.

Right now, our betas have, like, one set of TypeScript types Per package. Mhmm. And it's in a .d.ts file extension, but our JavaScript code gets shipped as either an ESM file or CommonJS file. And TypeScript is complaining that, well, your package exports pointed me to an ESM file, But your types point to a dot d.ts file, which is implicitly saying it's representing a common JS file. So So now we got this potential mismatch between an ESM JavaScript file and, sorta kinda common JS type definitions file.

And Andrew Branch argues that if you're gonna ship both kinds of JavaScript formats, you really should ship duplicated copies of your TypeScript types be with matching file extensions because there could Technically be both runtime and type level slight differences between the 2 cases.

Now now now in practice, it might not actually be a problem, but, like, if you really, really want to be absolutely correct, You should really have a dot d.mts file to match the e s m j s and a dot d dot c t s file to match the common JS.

And, yeah, you should really generate those with separate TypeScript settings for it to all work right. Oh, man.

And tools generally aren't set up to do this right now. Do you think that's why the the Svelte team has decided just to go with the the JS doc style typing and try to avoid

t s and I I I'm pretty sure they're they're not I I think they're doing everything in in JS doc style. Yeah. And and that's that's a large part of it. I mean, I, myself, I like TypeScript syntax. I would not want to try to write TypeScript as JS dot comments, but I can see the point that they, As the library authors find it easier to just keep it as a dot JS file with the embedded JS stock syntax and not have to worry about generating this stuff or at least, like, I I I know one of the rationales was, like, sometimes they're developing on the library, And they've got it, like, installed into an app in node modules, and they just want to be able to hand edit the library in, to debug it and figure out what's going on. So I I differ on preferences, but I can totally understand the train of thought that led them to that conclusion.

And what's your tool stack for actually building all of these different versions? What what are you using Vee? No. Actually so,

as I talked about in my blog post, like, all of the like, 4 out of the 5 libraries had been built with roll up and babble Through the current major versions.

Redux toolkit, starting in version 1.6, we switched to a custom ES build wrapper script.

And so As I was messing around with this stuff earlier this year, at one point like, after my 1st attempt failed and I was coming back around for the 2nd attempt, I I decided to work on the Redux Slunk Library because it's the smallest, shortest, and simplest one that we have.

And, you know, Rollup and Babble still work. There's nothing inherently wrong with them.

But, you know, ES build's a lot faster and some poor to deal with.

So I actually thought about copy pasting our ES build wrapper over from the Redux toolkit repo, but I knew that other wrappers existed.

So I found one called TS Up, and I decided to give it a shot. And it was really simple to configure, And it worked great.

And one of the things that it does is it will actually pre bundle your TypeScript typedef output into a single file.

So I got that set up for Redux Thunk really fast, and I've sort of copy pasted and tweaked that Configuration across to all the other libraries at this point. So the next major versions that go out should be built with TS up Across the board. Oh,

and what about tests? I know that just currently does not port e s m or or at least it's in beta. Did you have to switch to anything else?

Yeah. And, you know, like, it's possible that I could have gotten just to work Eventually, but at you know, when I when I was trying this stuff for the 1st time, I'm like, the code seems to run, but the tests are exploiting for some other problem. What is what is the fastest thing I can do to try to get past this blocker? And I had seen a lot of folks mention that VITAS Supposedly has much better support for ESM along with potentially, you know, just running faster in general.

And my 1st reaction was, I don't wanna have to switch my test runner just to get past this file format problem.

But like after a couple of days later, I'm like, You know what? I'll give it a shot and see what happens.

And it was actually really easy to migrate the tests.

You know, it required a new config file, But it was a relatively simple config file.

And most of the changes to the actual test code was literal search and replace of, you know, just dot, you know, f n to v v dot f n for the most part.

And so, like, it was actually, like, each of those basically took, like, a single night to migrate over one of the repos.

And VITAS ESM support was good enough that it didn't seem to suffer the same importing problems that Just was. The tests Seemed to mostly run faster.

So, actually, right now, I think we've still got GEST on our On our main branches and v test on our, like, pre, like, prerelease

beta branches, which, You know, keep going back and forth between those. Yeah. I personally found it was a pretty easy drop and replacement I was very surprised. For when we made that transition, and it's a really, Really great experience to have that happen, especially when you're when you're, so used to everything breaking all the time. I I have a question,

and This is more of like a philosophical if you think we'll ever get there is like, will we ever get to a spot where we just stop Bundling all of this stuff. Like, wouldn't it be nice if the code that you wrote is the code that you shipped To to NPM, where you're just like, I'm going to ship TypeScript to NPM or I'm going to I'm I'm going to ship ESM to NPM.

And if you need to to support an older browser, you bundle it.

It's probably not really possible to but do you ever foresee a a future or have thoughts on that? I I think

it's hypothetically, technically possible, but it would require massive shifts in how the community normally builds employee's applications. So realistically Mhmm. I I don't think we'll ever get there. Eve even if you assume that your standard application developers build tools are configured to fully process source code from node modules out of the box and handle compiling TypeScript out of the box.

Even just as a library author, there are so many assumptions that go into how I am writing my code versus how it needs to be shipped for people to use it.

And, you know, in terms of TypeScript versions and syntax settings, and Users aren't going to be wanting to think about any of that stuff. And True. Yeah. The settings and the processing are finicky enough. I don't think that Application level build tools can afford to make those assumptions right away. So,

no, Realistically speaking, I don't think we're ever gonna get there. What do you think about, like, people who are just, you know, just saying, already, I'm only shipping

e s m right now, tough deal with it. Yeah. The the biggest one there is, Sundari Sorgos, who is, you know, prolific author of node related libraries.

He switched to going e s m only, like, a a couple years ago.

He he has a guest that he links to anytime anybody complains about this that Details his rationale for why he did it.

It it is certainly a simplifying assumption.

And if You think that your consumers either are along for the ride enough that you can make that assumption or you just Don't care about messing with the details, and you're just not gonna put up with, you know, trying to support multiple build tools in multiple environments.

Cool.

If it works for you, great.

On the other hand, I've certainly seen plenty of people complaining that, you know, like, The latest version of Chalk or NodeFetch or something like I I upgraded and, like, all my apps Stopped working because it's DSM only, but I'm not ready for that, has definitely caused its own pain.

So, I mean, I'm I'm not I'm not, like, yelling at anyone for making that choice, but

it it has consequences too. Yeah. I think we need a couple people on both sides. You know? Somebody because he certainly has kicked a lot of butts into people.

Alright, fine. We'll upgrade our build tool because when you hit one of those packages, You have to literally either change how your code is written or you have to upgrade your infrastructure, both of which are are pretty disruptive.

And Sometimes it's not even that it's a package that you're explicitly using. It's that you are using a package which uses another package, and it just ripples through. And he has so many of these tiny little packages that it sent a bit of a ripple through the ecosystem, which is is good. It kind of kicked a lot of people in the butt. But On the other side, we just need people like you that could just make it work. Like, I got enough on my plate. I'm trying to build this application to book a hotel.

I don't want to have to think about all this other stuff, and it's nice when it just obviously, you've probably lost the sleep for Thousands of other developers.

Yeah. I I am I am very much a call it a compatibility maximalist.

Like, I have taken on the responsibility of maintaining these libraries, and I feel the responsibility to do the best job that I can to make it usable by as many people in as many environments as possible.

And, you know, it it is sort of self inflicted pain in that respect, But it it is something that I take very seriously. And so, yeah, I am sort of well, okay. I am groaning and, you know, Griping and complaining about, you know, all the stuff that I've gone through this year, but and I I wish the situation had been easier, But I am also intentionally taking on the effort of making this stuff work so that my users can just install Redux and Redux toolkit. And, hopefully, fingers crossed, it literally does just work for them. Exactly. And,

Hopefully, you're doing a talk, this podcast, your blog post.

You're also helping other package authors not have to do that. So

Admired bull. Like, I I I was I tried to be very clear. Like, none of this material that I've posted is, like, the definitive guide that I wish existed, But, hopefully, by sharing what I've had to go through, at least other people will get a sense of here's a few things that don't work. Here's some things that I might run into. Here's some things that I might need to try, and hopefully, it's a a useful resource that way. And I've I've been very pleasantly surprised. Like some blog posts I write, I know that, you know, a lot of folks are gonna find this useful. You know, the the 2 biggest examples were My my post on React rendering behavior and my Redux versus context post. That one's that one's always fun.

This one, like, I I wrote it trying to get the information out there, but I I honestly didn't have a lot of expectations in terms of, you know, how many views It might get or whatever.

And I've I've seen it getting passed around a lot.

You know, some folks have said it's been, you know, making the circles inside, you know You know, their their company's Slack channels or whatever, a number of other library authors have come up and said either, like, I've experienced the exact same thing, Or I'm about to start working on this stuff. Thank you for putting the information out there.

So it's it's actually been Very gratifying. I've gotten a lot more positive feedback on it than I would have expected. That's great.

Last question I have is, What about source maps? Did you run into anything? I source maps are notoriously

finicky to get working properly. I I haven't had too too many issues there.

The the Redux libraries have shipped, like, our, like, our current major versions actually ship A mixture of of stuff.

Like, React Redux version 8 still ships individual source files.

So you at least get that aspect even though they've been transpired a little bit, but there's no source maps in the package.

Redux toolkit does ship source maps, especially because we pre bundle those files.

Redux Core does not ship Source maps at the moment, but I believe I've got everything configured so that the next major versions do Ship source maps.

Self plug for a moment. Yeah.

So I'm I'm very involved with React. I'm I'm not on the team, but I use React in all the apps that I've worked on. I'm very aware of what's going on with it, like the React team, the React ecosystem.

React as a library has never shipped source maps for their production artifacts.

And, you know, they they they're they on top of that, React still does not ship ES modules. There's, like, a 7 year old issue asking them to ship ES modules, and it still hasn't happened.

But what what they ship is common JS, and it's all of React bundled into 1 common JS file either for development or for prod.

And if you compare what that file looks like versus the original source, even that even the development build has been radically transformed compared to the original source files.

And it's hard enough to understand the original original React source code. Like, I know a few spots that are relevant, like the file that where all the hooks are defined.

Yeah. But, like, the guts of React is still pretty much a black box to almost everybody, myself included.

And my day job is working for Replay, and we build a time traveling debugger for JavaScript.

We love source maps. We want every library and every application to have source maps. And there are literally so many times when we've been debugging even like a problem in our own application code and we step back upwards, and we're inside of React. And it's not just React's source code, which is impossible to understand in its original form. It's the minified production build of React, and it's a whole bunch of single character letters and nothing makes sense.

So a few months ago, I actually sat down and on the day job, modified React's build pipeline to generate source maps For the production, build artifacts.

And that PR has been open for, like, the last 4 months.

It's there And it works and it's waiting to be merged.

And I kept having a back and forth between Dan Abramov and I because, like, one little test kept failing And I had an exclusion to not generate a source map for that because like something something, the stack trace gets modified and blah blah blah.

And it's still waiting to be merged. And I actually, at this point, have no idea when it will get merged. I hope it will get merged at some point.

But even once it's merged, like, it's been a year and a half since the last stable release of React 18 came out, 18.2.

Who knows when 18.3 will happen at this point. But, like, literally, if we could just get that merged and out there, it would at least, You know, make error messages better for the average user. It won't help them understand it, but at least it'll be readable.

Yeah. We'll at least post a link to that That PR, and and I don't know if people can post a little thumbs up on it. Don't plus don't don't post a plus 1 in there, but maybe.

You know? Maybe.

Maybe a little thumbs up will help. I don't know. Probably not. Because React doesn't ship types either. Right? They're so the the React types have already always been maintained by the community. So the at types slash react and react on packages.

Now having said that, the primary person working on them is considered part of the React team at this point, a guy named, Sebastian Silberman.

And the team sort of actually now views them as sort of meaningfully owning the types, but it is not actually included in the React packages, and it does not automatically get added. You still have to, you know, install those separately.

Let's move into our separate club questions. It's our set of questions we ask everybody on the thing, who comes on the show. We're Curious what computer, mouse, and keyboard you're using.

So at the moment, I'm using, my my day job laptop. It's a Windows Dell, Vastro, I think. So in kinda standard corporate y laptop. I am the only Windows user at replay, Which has occasionally made for some awkward moments, but it's also there's also been a couple times I found, like, window specific issues, so I guess that's a good thing.

And is that is that that's by choice? You just, didn't get enough pain

doing the all this ESM stuff? I I've always been a Windows user. It's what I'm comfortable with. Yeah. It actually took me I I only even switched over to using Git Bash As my standard gel, like, 2 years ago, I was still using the old school, like, Windows command prompt prior to that. But, like, Windows is what I've used my whole life. It's what I'm comfortable with. There's the occasional pain points where people, like, expect that you're running on a UNIX alike,

But for the most part, it's been pretty good. And do you use Windows subsystem Linux for your shell, or you just go on PowerShell?

I use both. So I I still do most of my, you know, most of my day to day work on the actual Windows Windows side, but with, like, Git Bash for shell commands.

I do use WSL for a number of things.

Like, right now, our our back end repo really doesn't even, I think, build at all on on the plain window side. So that one, I've I've got solely cloned in WSL,

and I do Stuff over there. It's good. People are going to be happy because we ask these questions, and, like, 90% of the time, it's, I don't know, latest MacBook.

I, term.

Yeah. I guess in that regard, like, what is the the text editor theme and font that you use? Using Versus Code like most other people.

The theme is, drat. I I haven't even looked at that at that in a while. Eagle night Something somewhere.

Yeah. Eagle oceanic next. I can't remember where I ran across that one.

Font is normally, some form of droid sands mono. Been using that one for a while.

The like, There's, like, the original droid sans mono.

Someone else went in and made, like, dotted versus slash zero variations.

I think, technically, I'm currently using one that's like droid sans mono n f that has, like, some of the, like, the extra special, like,

Shell graphics characters embedded in there. Quick pause from the supper club questions because I just saw something really weird. I'm curious if you know would know why.

React React types has more downloads than React itself per week.

20 versus 22,000,000.

Why would that be? Do you have any any any wild guesses? No. No. I I don't immediately have an explanation for that off the top of my head.

Are you looking at, like, the literal core React package as a comparison? Yeah. I know that Next does some Stuff about, like, vendoring React in some cases.

Maybe it's possible that, Like, there might be some build setups that, like, in in in essence, vendor react and don't expose

I don't know. I I don't have a good answer for that one. Yeah. Because I was like, oh, maybe next like, includes React now, but they they they don't.

It's still a pure dependency.

Maybe maybe React Native

comes into play somehow here. Oh, yeah. Yeah. That could be it. Wow. If everyone listening, if you have a A thought as to why there would be more downloads on the TypeScript types for React over just React itself. Let us know. Anyway, sorry. I just was very, very thrown by that, so I thought I'd I'd ask you while you're on there. Back to the supper club questions.

If you were to start coding from absolute Ratch, what would you learn? Like language wise? Yeah. Language wise or maybe even just, like, literally anything. Like, someone comes up to you and say, hey, I wanna Learn programming.

What would you tell them to learn? So my first language back in college was c plus plus. It it gave me some good fundamental knowledge. It's not where I would recommend starting both because of the the literal amount of syntax Yeah. Characters, you know, the asterisks and semicolons and everything.

Traditionally, I've recommended Python just because it is syntactically, it's very simple, And you can write anything from a basic hello world script up to, you know, large object oriented type things.

But JavaScript is a great place to go and to go with as well just because doing stuff in the browser is probably, like, the The lowest amount of friction to seeing something useful on screen, you know, it's kinda that difference between, you know, writing your type in your name, Name, hello name, command line program versus, like, an old school, like, turtle graphics kind of a thing where you actually get to see the results.

And so, you know, starting off with JavaScript and getting to do things in the browser and just, you know, refresh the page or draw something on a canvas is probably the lowest amount of friction to actually seeing something interesting happen. Where where do you go personally to stay up to date On new things. I am unfortunately terminally online for for better or for worse.

So, traditionally, it's been the the React and JavaScript Reddits.

The programming Reddit to some extent, although that's been less interesting lately.

Hacker News, Twitter, Few other collected social media sites. I'll be honest and say that over the last year or 2, and especially within, like, the even, like, the last month after getting back from vacation, I've found a lot of the online chatter to just simply be less inherently interesting to the point of being annoying.

Whereas, like, my eye like like Yeah. Like like, whereas that previously, my my eyes might have skipped over seeing a bunch of, like, You know, article headlines from from beginners and okay. I've seen that. I've seen that. You know, if the eyes just sort of ignore it, Now I actually have a little bit of a reaction to where it's like that's actually, like, actively uninteresting.

And and and I think that's that's mostly a me thing. Like, I don't know. I've been feeling the same the very much the same where,

especially on on Twitter where I I don't know if it's because people are getting paid for engagement now or what, but it it does seem like the discourse has has looped around so many times now. You're just seeing The same hot bad takes over and over again, and it's just it's kind of really

drowned itself out. Social media is great In a sense that I've been able to talk to people who I never would have interacted with otherwise, I've gotten to get to know folks, and then, you know, met a lot of them in person. Mhmm. And it's allowed me to share information and help people. So I do I have gotten very real value out of all my time on social media, But it's also horribly addictive. It's so easy to get sucked into stupid angry arguments, and it is also such a waste of time. I agree. Yeah. All of the above.

Yeah. Alright. So the last section we have here are sick picks and shameless plugs.

Hit us with your sick pick first. Alright. So my sick pick is actually a talk from the React rally conference this last Friday.

There's a lady named Shirley Wu, and I I'd seen a couple of her her talks before. She Oh, yeah. She does amazing data visualization work.

And I I'd seen a couple of her talks either online or in person previously, but she did a talk this last week that Absolutely blew my mind. It was a personal journey, and she talked about her experience growing up as a child of immigrants, growing up in America, getting into the software industry, and how all of those sort of, like, made her feel like she had to put herself in a box To conform.

And she had, like, dreams and aspirations, but she had to learn how to actually kinda, like, stand up for herself.

And she talked about how she's, you know, gone through this journey of trying to express herself, wanting to be an artist, dealing with, You know, being shut in the in the pandemic and, you know, cultural fears and all these other things.

And let's be honest, most conference talks are forgettable. Now it's either you've you've seen the information before or it's just not relevant to you or, it'll be up on YouTube later.

This was Utterly, utterly incredible.

At the moment, like, you can you can watch it via the livestream from React Rally on Friday, which is still up, but the the individual talks should be online in a couple weeks. Okay. And this was an amazing talk worth watching. Can you repeat the name of the talk and

The author again, because I know people are trying to search right now. The speaker was Shirley Wu.

The the author the the title of the talk was, Take up space.

And it's about 25 minutes into the React rally day 2 live stream from Friday. You wanna know something wild here?

React rally is apparently a very hard thing to search for in YouTube because there's a ton of People making reaction videos to rally car racing, which oh, no. I would have never predicted that. But There it is. She did this talk at another conference,

about a month ago. So I I I think she said she had done it she had done it at one one of the conference and was maybe gonna do it at 1 more, but, Yeah. It was it was utterly fantastic.

Awesome. Cool. What about, Shameless Plugs? Shameless Plugs. Okay. So Day I am day job. Like I said earlier, I work for Replay, and we build a time traveling debugger for JavaScript.

And, like, basic sales pitch is you download our specially modified versions of either Firefox or Chrome, open up your website, hit the record button, use it for use the your website for a couple minutes, You know, press the button that, oops, accidentally makes everything explode.

Hit save, it uploads the recording to the cloud.

And now you can debug Any line of code at any point in the recording exactly as it ran on your machine.

So, you Like, one of the hardest parts of debugging is, like, you you first, you figure out the steps to reproduce the issue, and you start doing it. You have some break points. Maybe you added some console logs. And oops, I went a little bit too far, and now you gotta gotta start the whole process over again.

And with replay, you make the recording once, And now you can debug it as many times as you want.

Devs can make the recording as you're working on a bug.

QA folks can make a recording and, you know, when they file an issue, send it over to the developers and we're even working on a test suites product That's in early beta right now where we can instrument your Playwright or Cypress configs so that every time you push a PR and, you know, 3 out of 100 tests fail, it makes the recordings and you can debug the failed test exactly as it ran in CI.

It is an amazing tool. I love working on this product with this team, and I'm biased, but it genuinely Revolutionarily changes how you approach debugging.

And we're just getting started. Like, over the court the last 6 months, I've been able to build some nuke, some new features that use, like, an API for our time travel back end To dig into what happened and find things like when you click, open up the the line of code for the react On click handler and jump to the point in time where it ran.

Or integrate the read the React dev tools by finding every time that React rendered and then post processing and extracting the data.

And we've got proof of concepts for other features like, tell me every time you called set state in your app or performance profiling. Like, how long did it take to Dispatch a Redux action, run the reducer, notify subscribers, start the rerender.

Like, we've proven that we can write new features with the analysis APIs, we just need time to actually build the rest of the feature.

So, like, a year from now, we're gonna have some really amazing cool new stuff. Is that why they you got a job there? Because, like, redox was kinda first on the scene with time traveling because everything was immutable, so you could Go back in time and turn something off. Is that why you you got a job there? It it it very much relates. I mean, item 1 was that I I when I started looking for a job a year and a half ago, I had established a pretty good public reputation.

And so it companies were asking me, like, would you come work for us? And Replay was the only place that really, really looked interesting for me, and it fit all the criteria I had. I wanted to build a consumer facing product That is going to be valuable and help people, technically challenging, small team, you know, like, bunch of really good senior folks on staff.

And I I've been there for a year and a half, and I am still loving it. Like, I am genuinely excited to be working on this stuff. Awesome.

Beautiful. Thank you so much for coming on. It's it's been great to hear your journey of of ESM and all the little ins and outs of it. And hopefully Either if you use a package and it works, thank your package author. And if you have to

move a package over yourself or Another code based e s m. Hopefully, you learned a thing or two, so appreciate it. Yep. And one other one other on that note, one other quick shameless plug. Oh, yeah. So Redux toolkit 2 point o is in beta right now. Put that out a month and a half ish ago.

I don't have a timeline For when it will actually be final because I'm still trying to figure out, like, how many other things need to be done across all the Redux packages so that we can ship major versions on everything simultaneously.

Realistically speaking, it's gonna be months of additional effort, especially because I am doing this in my free time, not as part of my day job. But the biggest thing that would help right now is for people to actually try installing the betas in your own applications. You don't have to ship it to production. You probably shouldn't ship it to production yet, But try it out, see if things build, see what kind of breaking changes you run into, see if there's anything besides the stuff we expect to break.

And please give us feedback on, you know, e e even knowing if it worked is good, but also, like, what broke, What new APIs like, try the new feature APIs, and, you know, if you don't like them, tell us now so we can change before it goes final.

And, like, giving us feedback would be extremely valuable.

Cool. Is that on the, next branch, or is there,

version branches for those Just for people. The the reader if you wanna see the actual source code, Redux toolkits 2 point o, we have a v 2 dot o dash integration branch in our repo.

And then the, you know, the releases all have tags so you could, you know, pull up the tag in GitHub And dig through the source code there. Well, awesome. Thank you so much for coming out. This has been unreal. Thank you. I I appreciate it. It's been fun. Well, the discussion has been fun. The last year, maybe not so much. Oh, it was all worth it. Cool. Alright. See you later. Later. Peace.

Head on over to syntax.fm for a full archive of all of our shows, And don't forget to subscribe in your podcast player or drop a review if you like this show.