Supper Club × Remotion React Video with Jonny Burger

Welcome to Syntax, the podcast with the tastiest web development treats out There. Today, we've got one for you. This is pretty cool.

Programmatically making videos using React.

What? Like, if you would have said that was a sentence, a couple of years ago, I would have said, what are you talking about? But I stumbled upon this project a couple Couple days ago only, and, I was like, this is really cool. We gotta get Johnny on to talk about it. So, we are sponsored by 2 awesome companies today. Gatsby,

the fastest way to build modern websites. And Tupelo, an app built specifically for remote Hair programming, giving you high resolution screen sharing and low latency remote control.

Wicked. Well, Johnny, thank you so much for coming on. I appreciate it. Yeah. Very happy to. I have the podcast in my player, and I was very excited when I saw the DM From you.

Oh, that's cool. We always like when people have heard of us. Like, I always have to, like, explain who we are to everybody when I bring them on, or when I propose that they come on the podcast. And then a lot of people are like, Yeah. I've been listening for a couple years. Or, yeah, I've part a couple episodes here and there, so that's really cool.

So do you wanna give us a quick rundown of who you are and what you do and what Remotion is? You can take as long as short as you want. Yeah. Sure. My name is Tony Burger. In America, they call me Tony Burger. Both is fine. Is that your is that your real last name? That is my my real last name.

Yeah. I guess it's a Swiss thing. I'm from Zurich, Switzerland.

I've been a app developer previously until, like, 2 years ago, and also did some open source project in the React native space. And it's been about 18 months since I started Remotion, which is a web project

about making videos programmatically in React. Yeah. Nice. That's like, this is really cool because so I'll tell you how I stumbled across it. So I've been making short form content, and someone said, hey. Yeah. Like, you need to check out Remotion as a way to programmatically export your videos because I I was having to deal with this, like, weird Twitter issue where Twitter was cropping the video, and I just had to make it, like, Twenty pixels wider, and, now I have to, like, export 2 versions of it, and it's a pain. And I was like, well, like, wouldn't it be nice if I could just, like, have a script or something that could, At the very least, just make the size a little bit bigger and export it. Right? And someone was like, you should check out Remotion because, like, obviously, it does a lot more than that, but, it it that's something you could do as well. So I was like, okay. I'll I'll check this out. And so I kinda went into it, and I was like it's like make videos programmatically.

Clean. Yeah.

I fired it up, and there's, like, this whole, like, previewer that's built in React.

And then putting your videos together, at least the far as far as I got into it is, you actually write code to actually just describe where they go, and you can create different scenes and put them together. And I was like, this is this is really nifty.

So, like, why why did you make this type of thing? Is this a problem that you're trying to solve or what?

Yeah. I had a Really similar motivation for building Remotion.

I was already making videos before with the Adobe Suite After Effects, but I was a much stronger developer, and I I knew that's the web was very powerful for making graphics.

We have CSS, SVG, Canvas. We can do three d stuff.

Recently, I experimented with rough JS For making, like, some hands drawn type of stuff, the the web is really powerful.

And my idea was that If I could just make a bunch of screenshots of web pages and then put that together and code that into a video, I could probably make something that that looks better than what I would be able to do with my abilities in with Adobe programs.

Yeah. Yeah. I know. After Effects specifically is one that gives you kind of, like, just enough scripting things that it makes you feel Powerful, but at the same time, anytime you're used to working in WebTech, you know just how much of a pain in the butt the the scripting in in general and And, after effects can be. But not only that, like, working in, like, a vectorized context is no fun in any of those situations. Right? I mean, you You can't work with the vector tools that we know and love typically. So, in comes the web, which you can now Adjust your things to various camera sizes and and whatnot. Did you have a background specifically in motion graphics before this, or were you just a hobbyist?

I was definitely just a hobbyist who was, I would say, pretty good at making mobile apps at that time.

And my plan was to make a new app and launch it. It was like a WhatsApp sticker app.

Nothing big came out of it, but I shortly before lunch, I decided, okay, I need to make a a launch video.

And then I I started my experimentations with, trying to coach that in React and, animates it using code.

And so I put off the actual app that I wanted to do for, much longer and worked on the remotion instead.

And when I launched it, it got such a Feedback that I was in in a constant feedback loop and never really had time to work on the original app. So now Remotion is my main project. Isn't that Funny how that works where

you're the the the thing that you have that is the need in your workflow ends up becoming The project.

It feels like that's happened so many times where where people work on a project, they hit a need, And then they build an entirely new project around that that then becomes the product. I mean, it's exactly what you're you're describing. It feels like that is

A very common theme we see with successful products. I guess it makes a lot of, sense that, if you just, try to build Products, especially for developers, you'll end up hitting a lot of problems that you find start up ideas while building like, normal regular projects all the time, where if you think you could just invest a lot of time in it, it would be so great.

I think The the stuff that developers face, so much of it, is still unsolved.

Mhmm.

Totally. Awesome. So, like, what type of videos are people making with this type of thing? Because I was kinda looking at it. I was like, this is really cool. Some of the examples are unreal, but I was like, okay. Like, me as a video creator, like, I'm probably not gonna use this to make my Screencast, I don't think.

Is there a specific type of video, marketing video, or whatever TikTok video that people are are making with this? Yeah. So we try to

make the framework as generic as possible to, allow a variety of use cases. But I can mention a few videos that that I like. For example, a marathon in Switzerland, used Remotion to to batch make, finish our videos. So they would film at the event and then, send a personalized video to everyone who finished the marathon and they Over late time, like, greetings from a famous, runner.

That's one thing that I was Kind of, proud of that they made it like that.

And there are, like, various other use cases where It was necessary for them to create like, I would say, like, data driven videos is an is a category that where Remotion is Very strong ad where you fetch data from API, be it weather or, like, Stock prices, and then people make bots out of, that where the bot post a video every day. Yeah. Yeah. I I see this a lot with,

like right now in TikTok, the popular videos are people take a, a popular Reddit Story, and and they take, like, a video from YouTube, and then they do some AI voice on top of that. And, I was like, this this is unreal For being able to take data, like you said, that changes at any given time, and can pipe it in. And you don't have to specifically like Tell your editor, all right, here's the file of data, and then they have to copy paste it in, and you have to make sure they get it right, is you could just get that hot out of an API Right into the video. And if there's an issue or you need to update it or anything like that, you can constantly just pull in the fresh data, which is so cool.

It does seem like, you know, that is a major strength of of programming. Right? You're able to do a lot of operations With, you know, over the course of data, to be able to do things rapidly that you would take forever to do by hand. I mean, I have, what, like, 400 courses on the whole of tutorials.

I could generate an introduction video for every single one of my courses simply based out of the data we have in our database And a template and some parameters, you're passing that into a programmatic interface for creating videos. The next thing you know, you have All of this content that would have taken so long to do by hand. And then you you get into more tools on the web, and you could probably stitch that together in other videos or make a whole work all around it. I I see a huge amount of potential for for a tool like this.

I have some questions eventually around the like, Converting in, you know, what would be like a project into a product. We can talk about that. But do we wanna get into the the tech stack a little bit? I've dove into your your repo, and I've pulled out some dependencies. And I wanna, like, maybe talk a little bit about how you made this thing. So I I do see that you're using Turborepo and Vite and PNPM, so all the the cool stuff for the build tools. But in terms of, like, actually creating Dynamic video content.

I see 3 j s. I see Lottie.

I see FFmpeg, Puppeteer.

I'm interested in, like, kind of what's doing what. In this in this type of project, what are you using 3. Js for specifically?

Okay. Maybe we should start with, the the core of which, the the idea of how Remotion works is Very simple. So we we give you one hook. It's called use current frame and and a way to specify the width and height of the canvas. And then you can render anything that you want. And then as the the value As the return value of the hook changes, as the time changes, we just tell you, hey. Render something different over time.

And then we use Puppeteer to make screenshots of that and FFmpeg to encode the video. I would say these are the most Necessary tools that are in our stack, that we need to have.

So the idea is Very very simple. And with just that, you're not able to do

so much yet. Okay. I have a I have a time and and a blank canvas. Can I can I, sorry? Can I have you elaborate just a little bit on that puppeteer, and Yeah? FFmpeg.

So so what What are you doing? Are you are you rendering the current react tree into a puppeteer, instance, so to say. I don't know what they refer to them as. And then recording doing a screen recording of that. Is that what's happening?

Sort of. So, I I guess we don't use Puppeteer anymore. We have kind of inlined that all in, in our codes to make it a bit more efficient.

But what we do is we open a Chrome browser, we load the codes that you have written, and we Iterate over the duration of the videos. If your video has 900 frames is 30 seconds long, Then we set the time, take a screenshot, set the time again, and screenshot it again. So we That's we make a lot of images.

It's, it's not a screen recording because that way we we also we ensure that there are no frame drops. So You can make, like, really complex stuff that maybe takes, like, 2 seconds to render in a in a browser, but we wait for all that to be to be done.

And there's also a way to await, network requests, for example, or any asynchronous operations.

So that way the the render will maybe not be in real time if it is a complex project, But we can ensure that there are no flakers, and everything is, very smooth. I I don't know why I'm so surprised

that You're just taking screenshots because, like, video is screenshots. It's just lots of small images played at a frame per second, and then it's smooth to us. But, For some reason, I thought that there would be a lot more, like like, a whole rendering engine and whatnot. But, like, The rendering engine is Chrome. And because of that, you have all of CSS available and all of HTML available and all of React because that was my next question is, like, Can you use all of CSS, or is it like this, like, React Native thing where, like, you gotta use your own limit subset of something? But it's It's full blown whatever works in the browser and you could take a screenshot of. Right? Yeah. Exactly. And,

as you can imagine, if we just use this very simple technique, You can do pretty much everything. There are no no constraints.

I guess what's the drawback of it is that, it is Pretty hard to create complex animations with just, React and CSS.

And sometimes CSS also gets in in in the way. So for example, CSS animations, they just Play on their own. Right? Oh, yeah. Whereas Mhmm. We have a model where everything is totally Declarative. You you get to frame from a React hook and render something, and you're also Able to await something, that makes it so that the What we render is not necessarily what is being displayed in the real time in the browser. Mhmm. And we can also, for example, play Something backwards or play them half the speed. You kind of have control over the speed, whereas with a CSS animation, it just Place in one direction all the time. Uh-huh. So there are some rules to remotion, and one of the most difficult ones for users to get right is that Each frame that you render needs to be idempotent.

So for for the same time, you Always need to render the same thing. So, you cannot use CSS animations or, like, random random values.

But I think it's very important that we introduce these rules because then we can also parallelize the render Instead of just opening 1 chrome tab, we can actually open 4 or 8 chrome tab and paralyze the rendering because, if all frames are idempotent, then the final image will still be smooth.

And as an extension for that, we have also made something like, that we call Remotion Lambda where you can distribute your Video render across like 100 Lambda functions where each Lambda function renders one chunk of the video, and, then we concatenate it back together, and then it's a super fast render. Oh, super fast render. You could literally spin up, like,

2 2,000 Lambda servers, and and all of them render 1 frame and then shut down. I mean, it's like having, like, a a Pixar studio, you know, where they have Rendering tiny little bits of animation across several computers. I remember doing that in college, and, like, this is that same thing, but to Both a larger and a much smaller degree. I'm impressed with just how much of this is, like, a true True video editor style of things where, like you said, because each frame is always going to be the same, You can step through your animations and always know that this thing is always going to exactly how you'd expect Final Cut Pro to work, right, or any of that stuff.

Yeah. That's way more intense than I think I was getting from the the surface of things. Can we ex I'll explain to our audience really quickly,

because I know there's a lot of new developers listening, and they probably are hearing the word idempotent and not not understanding what that is. We talk about this On the podcast every now and then is, pure function.

Meaning that no matter you could you have a function, and given that you pass it in some data, like you have a A animation, you say give me frame 7.

It will always, always, always return the exact same thing. That's beautiful because people always ask like, Oh, why, like, why do you need pure Pure functions and and idempotent JavaScript. This is a really good example of why you would need that because, regardless of how many times you request Frame 32 of this thing, it will give you the same thing every single time. And because of that, you get all the benefits that,

Johnny, just explain to us. Yeah. You can you can kind of think of a video as a function that takes in a current time and Returns a a different image, based on the time. And it is kind of beautiful because Now we actually use a real React functional component to do that.

I realized it's it has some beauty in it. Yeah.

Does. Yeah. Functions all the way down.

That it's so cool. So okay. So the other dependencies that you're in in your repo are are more or less tools to assist with video creation. So, like, people might not have heard of Lottie before. Do you wanna talk a little bit about how you use Lotte?

Yeah. So Lotte so it is an optional package that you can install And, include Lottie animations in your video.

I guess, Lottie, if you would just import it, by yourself with without our package, you would have this problem that it just plays Independently of Remotions' time.

So we need a way to synchronize it. Otherwise, we cannot achieve this idempotency.

So we have a bunch of helper packages that take things that are usually, not idempotent or let's say, they would just change their appearance Regardless of whether you update a react state or not, this can be Lottie animations, GIFs, and, yeah, we also have few other integrations and we pack them up so that you can use them in remotion.

And we also want to create a streamlined way for users to include these types of things.

If if you just use the Remotion core, I think, oftentimes, you are pretty, on your own on how to implement certain stuff. So we try to provide a bunch of helper Packages that do all kind of things, effects, presets allow you to use third party libraries.

Because As I said, if you just have a time if you just have a time stamp and a blank canvas, it's pretty hard to animate something complicated.

Yeah. I do see, like, a motion blur package as well inside of here, and that feels to me like it would be a particularly challenging issue.

Is the the motion blur and trail effects you're doing, are those like, How how does that all fit into the the puzzle here? Yeah. So motion blur, it's something that

you can do in most Video editing programs. So we also need to have a way to do it. And we we have Two types of blur.

The first one is just that it you pass some it's a React high order component, and by the way, a really simple implementation like 12 lines of codes, where it just takes any React markup And duplicate it.

And on each iteration, it changes the time stamp a little bit so that Each layer lags a bit behind the previous one.

And that's how you create a motion blur effect. You you duplicate it and, make each layer a bit less transparent.

And that was the naive Implementation, and then there's, like, the sophisticated implementation called camera motion blur, which uses the CSS mix blend mode property, which, yeah, is pretty advanced. And you can even set like a shutter angle so you can say, okay, the blur depends on which way your Camera is set up.