We Need Your Help With The Secret Sauce

key. Welcome to Syntax. In this 686th episode of Syntax, We're gonna be talking about the new syntax website secret sauce.

And this might be a new segment or a new type of of episode that we'd like to do, and and we're actually gonna Be needing your help on this one for future episodes because what the secret sauce is is we want to know What is the secret sauce behind your product? The stack. What types of tools are you using? What types of libraries. Where does it host? What what are the things that are practical about real world applications? And we're gonna be kicking it off with the new syntax website. We're gonna be showing you our secret sauce.

We should let's tell how we got the name for it.

When we met up in Toronto last week, we had I forgot who wrote it down. Somebody wrote secret sauce on the whiteboard and we're like, why don't have something called secret sauce. Like, that sounds great.

So that was a name that didn't have any sort of Content behind it. The other day I was thinking I was like, you know, it's one thing I love to do is to talk to people who work at these companies Generally, like larger products like Netflix or how KAYAK or like any any one of these companies.

I want to know what does it run on? What's their stack? What's? Yeah, what database do they use? You know, how does it all go together? What's what is specifically does Netflix used to deploy apps to their TV. You know, there's so much secret sauce there, and I want a single show Where we talked to somebody from we won't go in-depth on all the different tech, but I just want to know, like, what are the pieces that, put it together. You know? What's the secret sauce?

What are you working with? Yeah. And I think one of the best episodes we did n. For our separate club interviews was when we were talking about, the rendering of Google Docs, like, how wild that was.

I wanna hear more stuff like that, and I I'm excited to dive more into this concept of secret sauce. So if you're out there And you work on something that could be interesting for this. Hit us up. Yeah. Or if you have a suggestion, like, Like, what do you what do you think about? Suggestion.

Yeah. Well well You can email me directly at [email protected], or You could hit us up on any of the socials, hit our our supper club question form or our or what's it called? What's the, not the supper club question. Our pinxt. Fm forward slash potluck. Yeah. The potluck submission form. Either way, just get get in touch with us, or we will get in touch a. With you, our people will talk to your people, and we'll set something up. So let's dive into the syntax secret sauce. But before we do so, let's Talk about one key component to this syntax secret sauce, and that's how we took, track our errors and exceptions because syntax is presented by Sentry. And Sentry is a A tool that we use that actually we didn't have in this outline here, but Sentry is a key tool for us to be able to track errors and exceptions. Just Just yesterday, man, I was having a weird issue where the database wasn't quite Working where I our database to the markdown files.

It should have reimported all of them. The import was successful.

M. You'd go to a show page, and it would give me an oopsie daisy page, which is our 500 level error.

And I was getting really frustrated with this, and luckily, we have Sentry. So I popped open Sentry. And guess what? I saw that Part of the database sync wasn't running, and Prisma didn't run a DB push.

So Prisma hadn't gotten the, schema changes, and that's what was causing all the trouble.

And that error was difficult to find without century. So if you, have bugs, if you're shipping software, You wanna track those bugs? Sentry. Io. Alright. So let's start off.

This is not exhaustive tech of the The new site. We did a we did a show on that, and we'll do another one when we actually launch it. But, again, this is the secret sauce. This is the things that we do. So the thing is built in JavaScript, full stack JavaScript. Well, it's all TypeScript, right? Soup to nuts TypeScript, Node. Js. On the Server side, we did consider doing an alternative runtime, maybe Deno, BUN, Cloudflare Workers, Fastify Fastly. I always forget to mix those 2 up.

Fastly Fastly is the platform. Fastify is The server.

Yeah.

Serverless functions for all of the back end work.

We are And that is for things like the API, all of our logic, database saves, queries, importing all of the data, the cron jobs that does all the transcription And show note summarization.

That's all in serverless functions.

We do not have any edge functions, but I specifically have A use case for them is we're starting to use these like little short URLs syntax. Fmmeetup was one of them, and I threw that in Cloudflare Worker rule, but like you got to pay $5 a month once you have more than 3 rules on Cloudflare. And I was like, we could we could throw that in an Edge function So quick, just have a like a JSON file of redirects and we could we could do that right in and edge function instead of having to pay and and use some UI to do redirects.

Yeah. And it's an interesting bit of things. I mean, you're able to get those thrown up in 2 seconds, and it just works. Yeah. For for the framework, we're using the SvelteKit with Svelte as the, the front end framework. SvelteKit has the full stack framework, so it's handling, the data loading, pulling from a database, Connecting, to any sort of server side, stop dealing with actions or how we send things to and from, the front end to the back end. It's all SvelteKit.

We're using MediaChrome, which is from Mux for the audio player on the website, and MediaChrome is great. It's a web component that you can use for video players or audio players or anything you want, and it is extremely full featured.

And I'm I'm somebody there who's used every video player on the block. And I gotta say MediaChrome or in general, Mux video, the whole Mux packages there for The, any type of playback is they're just they're just lovely, and you don't have to use mux to use these. These work with any sort of video stream whether you're streaming or using audio or or anything. They're they're fantastic.

Website. In fact, we have, like, a fuzzy finder for every episode on the entire site.

It's really, really slick. It helps you find things very quickly, a And we're doing so using something called flex search.

Unlike, like a paid service or anything like that, there's no paid component. It's free. We're essentially loading up all that data, loading it in via a worker, and then it's available directly when you need it. In fact, most of the code of our implementation of FlexSearch, you know, came from the SvelteKit and Svelte.

N. I think they call it site kit is what they they call it, but it it's what they use to build their own sites. It came directly from their code base, And there's a lot of nuggets in there. Obviously, we adapted it to our needs, but, I'm really, really impressed with FlexSearch. It's super fast. It works very well. And, again, you're not having to pay for an additional service there to get really nice and easy search going. Yeah. I'm I'm actually curious about that because I I haven't I don't think I've actually asked you questions about that.

So when you use it, it searches through Show notes, show titles.

Those those are the 2 things, I guess.

Anything else? Yeah. Basic Basically, right now, it is just show notes or show titles, and it could be anything that we tell it to. So it it's really like The the way it works is you you're kind of like building a tree of data. Yeah. And that data is is coming directly just from a database call.

And anything that I want to put into that tree can be used as a parameter on the search.

So that that's really quite it. Do you have to build like an index of all of the shows to go into the search? Like, I'm just curious, how does FlexSearch know about All of the shows. And and is this done server side or client side? It's done server side,

and the index is built by FlexSearch.

And so, basically, what I'm doing is like a database call, and I'm creating essentially an array of all the shows. And then I'm I'm passing that into FlexSearch, and and I'm creating an index there.

And FlexSearch creates essentially f The tree, and then Mhmm.

It creates the links and the breadcrumbs and everything and spits it out on the UI. So that way when I Start typing. It automatically and instantly

finds everything from that giant list. Oh, okay. So it it just it just sticks everything in memory, and then it's able to have access to that pretty quickly.

Yes. But it's doing it in a service worker. So Yeah. Or is it a service or just a worker, not a service worker? Sorry. Those 2 always confuse me. Yeah. Oh, okay. Yeah. I'll I'll we'll have to go deep into this Because it's actually, it's not that many files, and so you could really go through it pretty quickly in terms of, like, what exactly It is happening here, but, if you're out there and you wanna take a look in our implementation for this thing, m. You know, the the relevant stuff is there's a content dot JSON route, and that's essentially running just a normal Prisma client f. Find mini and then, creating it and using a a push to push it into the structure that we want it to have, And then that's sending it as a content dot JSON.

Then we just have a worker that hits that content dot JSON with a fetch n. And then

throws it into FlexSearch, basically, and then we massage a little bit. Isn't that, like, loading every single show note on the client side then?

Is that huge? On the client side, it's been it's doing so in a worker, though, so it's not doing it on the main like, you're not nothing's being interrupted there. Okay. But

I don't see it. I don't see a big request, but, like, this JSON file.

Yeah. Does it have Every single show note ever? Yes.

And that's being loaded client side? Yeah. But isn't that, like like, 8 megs of

of data? I'll tell you exactly how many megs of data it is. Content dot JSON. It is, 4 megs. Okay.

So it's this is similar to Our API route currently that has when we hit that API endpoint, we load every single Every single syntax ever on that API endpoint because that was not a big deal. Obviously, we need to paginate that, if we wanna Go any further. Actually, that's we get a message from Vercel every now and then that our we're going over our bandwidth because That specific file is is pretty large.

Mhmm. But maybe that's a concern. Yeah. That's okay. Whatever.

We styling. Let's talk about that next, which is scope styles.

One thing about Svelte is you simply just Put the styles in style tags in the same file that you are creating your components, and all of the styles are then scoped for that. There is a number of global styles as well for things like, text layout, you know, the fonts, the colors, the Sizes of everything. Global variables.

Sometimes you just need a global right utilities. No, you need a good baseline before you get in.

To worry about.

Also, we've used a class of highlighted somewhere else. You know, that's not a thing because they're all scoped to what you need. Some utility classes specifically around there's 1 for adding grit to something. There's a couple around font sizes that are just like little handy utility classes if you want to change the font size.

And then there's lots are on layouts. Yeah, there's a whole, like, zones and layouts thing, which I think will save that For the light and dark mode

podcast. Yeah. There's a lot there.

And it it it's interesting because, like, Pretty much anytime you need to place anything on the site because we did some really, really slick stuff with, like, a grid based layout system. So using, like, grid specifically for vertical and horizontal layouts.

And then you can now basically place anything anywhere Anywhere you want at any given point, and you can do so with a class nice and easily, or you can do so with straight up, named grid columns in CSS. So a lot of lot of fun stuff there. The Svelte animation API

was very cool. I'm getting a little in the weeds here, but I had a lot of fun learning the Svelte animation API. It's not something I had used. I took an example that Scott did with the pagination. So I built out the pagination for the show's page, and it's so fast That when you click the next button, it just goes to the next page. It's already preloaded, so it's instant and it's a little bit it was a little bit disorienting Because all of the numbers shifted to the left by 1.

So Scott threw like what, 6 lines of JavaScript at it and like Added a little swoopy animation

and it like made it. It gave so much more context as to what was happening and it's so good. Very happy with that. Yeah. You you essentially just drop in a, like, a flip, just like a single directive on a DIV, and then it handled The flip animation, since everything was a keyed each block, it kind of knows where everything is and is able to do those flip style animations super easy, n. Took no time, and then I just added an in and out transition as well. We also use the page transitions API, which, h. Has gotten really easy inside of SvelteKit. It's not quite the same as what Astro is doing. I think people get confused there when they hear that Astro added the The page transitions API because they're actually polyfilling it back to older browsers where the way that SvelteKit is doing it is basically just, there there's just a a Simply a function in or a, sorry. Not a yeah. There's a a, an on navigate a. Kind of life cycle method that just runs when the site is navigated.

And then you can simply tap into that and then, therefore, take advantage of the new view transitions API using just straight CSS through the rest of the site. So we did that. It's progressively enhanced fully. Works really well. Love it. Some other cool stuff, tooling Inside of this, we use Prettier for our formatting, which, you know, did we didn't really change that much in the Prettier. You know, I haven't really touched it.

Very, very specific stuff, prettier plug in Svelte.

Trailing comma none. Did you add that?

Added by Scott Talinsky. That's surprising. I love trailing comma. No. I didn't add that. I I thought we did have Yeah. There must have been a default. Trailing comma. But we did we did go semi colons.

Yes.

We need something to enforce Much to my dismay. We need an ESLint rule to enforce, snake case. Because I had written a bunch of the transcript stuff outside of the show, and I did it all in camel case.

And then that, coupled with my default to just go camel case, is there's still quite a bit of camel case in there that needs to be, like, we need a rule, and then we need to be able to list out where all the camel case because refactoring with TypeScript is easy. You can rename a function pretty pretty quickly,

But I would like all the time. Yeah. Snake case.

Yeah. I like Snake case. It's easy for me to read. It helps. I know people. Yeah. S. So that's not the JavaScript way. I get it. It's just easier for me to read.

Post CSS, we're using post CSS with a preset ENV to basically s Give us access to newer CSS functionality where we can with largely not having to worry about it. We progressively enhanced some things, some things we're just waiting on. We're using PNPM as our, as our, I guess, you would say your your NPM replacement for fetching and storing your node modules For making sure we're all on the same version of Node. So that way we can always make sure that if you're firing up the site. You have the right version of node, running scripts, all that stuff with PNPM.

I really like it. It works fast. It works well.

And then because we're using SvelteKit, the whole thing is built on Vite. And one thing I really like about the SvelteKit version one changes that they made, they basically have have made Vite an actual front and center thing where you still have access to your Vite config. Right? They didn't obfuscate that. So If you wanna add Vite specific things or you wanna get into the Vite side of things, you could still do that with with no issues. I'm just, I'm on the website right now, And I realized

one of the images is being base 64 encoded into the JavaScript, and it's like I think it's too big.

Okay. We'll figure it out. Yeah. We got a fit we got fit and finish to worry about. And then Oh, yeah. For sure. We're just gonna launch it,

And, you, the the listener, will help us find all these little bugs. Although maybe it's not. I think I think Vee inlines stuff when it knows that it's okay to inline it. So maybe it's not. We'll look into it.

Let's talk about database.

So we are using MySQL, With a Prisma adapter, so Prisma allows us to generate our schemasor we sorry, we define the schemas And we defined all the relationships. We defined, all right, this is the show. A show has a show number. It has a title, has a description, It has and then it has a relationship to guests and then also has a relationship to a transcript. All of that stuff, right? And then Prisma will then translate that over to a MySQL database for us, as well as give us the typed API That allows us to query all of the data, which is really cool because, like, we don't have a show type. We just have A type of whatever is returned from a specific query that we've wanted, right? Like I might want a show with just the number and the name, But you also might want to show with the number, the name, all of the guests, all of the transcripts.

So the way that the types Flow from the schema to Prisma to you have a query on the server and it's returned from the server And populates the types on the client side like it's unreal how the types just ripple all the way through the entire thing.

And there's just a couple spots in the application where we've had to explicitly define data types.

Yeah. That's awesome. Everything else is just Autumn, it's a it's a bit of a different, like, way to to get used to it, and sometimes it's annoying when you hover over top of a type and it's just just vomit. I would I would prefer that it was just a show type, but then you're like, well, the show might or might not have show notes, and you have to Do an if statement, and this way it knows

if the data is there or not. Yeah. You know what? And and it's one of those things that when I was working a lot in Graph q l. I felt like I was defining my types in 4 different places, and now it's like, alright.

You really just have 1 place, and it and It cascades the entire app using Yeah.

The Svelte kit, the way that they handle types, but also how Prisma generates our types and stuff. It's all just very, a. Very nice to work in. Services.

These are some things that I think you know a little bit more about considering you were doing a lot of this AI work. But we're using several things For our transcripts as well as, AI tools to identify who's talking when, a I OpenAI, Claude, and Deepgram. Do you want to dive into those? Yeah.

So for the transcripts, we're using Deepgram to transcribe the shows. We've we've talked about that in the past as well, but They do a very good job at diatarization, which is detecting who the speaker is.

It doesn't tell you their names, and we can't train it on n I, which would be nice. But we fix that by tacking on a little hunk of audio that is known to the end of every episode When we transcribe it, so it just says Scott says, like, I'm a I'm not gonna say it because I don't wanna screw up the actual Yeah. Our whole plan here, but I'm a cool guy who loves cheese. And then Scott says specifically that weird sentence.

And then I find out What speaker number that's associated with and that's how we know if it's Scott or I speaking.

Then we also Have some AI show notes, which are I'm very happy with. It will do really good summarization as well as give you like markers For the different topics throughout the episode, so if you're like, oh, I want to I want to know what Wes was talking about, it'll give you a really good like subheadings.

You can quickly scroll down to One of the subheadings is see what we were talking about. I think that's gonna be extremely helpful where you say, oh, what what did they just say? You know, what was that library they mentioned? I didn't catch it. Sometimes I'm on a podcast and I have to rewind 6 or 7 times, and I cannot understand what that One word that they're saying because I want to look it up. I think that the show notes will be very sorry. The AI notes will be really good for that.

We are feeding those into OpenAI, and that has been pretty consistent. But then I also have been playing with Another one called Anthropic Claude, and they have been fantastic.

So, I'm going to it's a little bit More wild and a little bit more hit and miss getting JSON from them, but I think it's worth it for the quality that we get. Yeah. Yeah. A lot of interesting stuff there.

Testing. We use, I you could consider TypeScript as being part of your testing infrastructure. I might Because it can help save you from potentially finding bugs.

But we're using for, you know, more more classical testing. We're using Playwright for end to end testing, and that's the only testing that we're doing. We're not using any mocking. We're not using any of that kind of stuff. Basically, just fire up the site with the developer DB. And And the things that we're not testing, I'm not testing the, I'm not testing a lot of database actions, really. I'm I'm using Playwright to jump through the site, Click the player. Make sure then the episode starts playing when you click the player.

Every time you you visit the page, the The specific header that you're supposed to expect to be there is there. So you're making sure that all of your pages are loading as expected.

The things you're expecting to be there are there, And the functionality that's important is there and works fine. And for the most part, I've been liking Playwright just fine. I really like how When you run it without the UI, you can run a lot of tests in parallel, and it's just really fast. Hosting.

We are we've been like kind of devin' on it on Vercel, and been pretty happy with that. I'm Fighting a WASM issue right now because with WASM libraries they need to you don't explicitly, include the Wasm file, especially when I'm using like a library. So I'm trying to figure out how to like get that Wasm file included in the build For Vercel, we did consider going to fly as well because some of the transcript stuff Takes in some of the importing stuff. Oh, no. The importing stuff that you did is pretty fast. Right? Like, how long does it take you to import all 600 shows? Like, a couple seconds? Yeah. A couple seconds. If you're on if you're on local, it takes

milliseconds. It's so fast on local.

Obviously, in a production environment, it's a it's a couple seconds. That's impressive.

So some of the stuff I'm doing, which is hitting APIs, takes 30, 40, 50 seconds.

So I was just I had to make sure that that sort of longer running processes, would fit within the timeouts of a serverless Because as soon as you start getting into over, I think it's 300 seconds is the max you can extend a serverless function on Vercel to.

Then you start having to look at, running a longer running process. And Fly is kind of Kind of neat because you can run like you could basically just spin up a long running node process For 2 minutes and then spin it down and you only pay for those 2 minutes. So it's kind of nice because you you get the benefit of Long running node processes.

But then you also have the benefit of you're not paying for it from the time you start it to the time Like until you shut the thing down, you're only paying for as long as you need it for. So I don't know. We're We're not sold on Verstylog just yet, but it's been been working really well for us, especially now that they have Cron jobs as well. It's in beta. That's one thing we've needed for CSM. A lot of the transcript stuff. Word. Hosting planet scale, right? So it's MySQL, but we host on planet scale and that allows us To interact with it via serverless functions, it does connection pooling for us, so we just have to connect to PlanetScale and it handles all the Hard stuff for us. And then, we are also using Cloudflare for some of the redirects, although I talked Earlier, I would like to move those redirects into an Edge function so that we don't necessarily have to pay for that, as well as It's part of the the code. Right? Like Yeah. I would hate for someone to come along and be like, hey, where's this, like, your redirect happening? Oh, you have to log into the service and and whatnot. I would love for the short URLs to be in the code, not even in the database, like a file

With all the redirects, it'd be kinda neat. Word. Well, that is our secret sauce. This new site is gonna be launching, hopefully, by the time, you hear this episode.

F. Oh, yeah. Yeah. So

syntax.f m, check it out. Check out the repo. There's a ton of tasty stuff in the repo.

And, you know, maybe there's Some Easter eggs throughout the site as well. So take take some time to check it out. I think, I'm really proud of the code base, proud of the the work that we've done here, and it's it's pretty

That's it for today. Thank you so much for tuning in. We'll catch you later. Peace. Peace.

Head on over to syntax.fm for a Full archive of all of our shows.

And don't forget to subscribe in your podcast player or drop a review if you like this show.