HTTPS + Tunnel Your Localhost - Cloudflare Tunnels, Ngrok, and more!

Welcome to Syntax.

On this Monday, hasty treat, we're gonna be talking about Tunneling your local host in h t t p s, the type of thing that you will eventually need to get into Whenever you're working on a website that needs to access some sort of secure context locally Or perhaps you need to have your local host available to the outside world for any number of reasons. We're gonna be talking all about that and more. My name is Scott Talenski. I'm a developer. And with me today, as always, is Wes Bos. What's up, Wes? How's it going?

Oh, not too much. Actually, I want to, add something to a previous show that we did because I was driving this morning listening to our show on web streams.

And I totally forgot the best use case of streams I've ever, I've ever heard of, and that is Not sandwiches.

It's unfortunately not sandwiches. It's the, GitHub tarballer.

So you know when you go to GitHub and you download a repo? Mhmm. That zip Downloads immediately.

And I always like for like I knew I've known this for for a couple years. But before that, I was always just like, How do they zip it so quickly? But when the files in that zip could change at any time, they're not Generating zip files of every repo ever. Every time there's a commit, they do it on demand, but it's it's fast as heck, especially for, like, Your repo could be 50, a 100 megs, and it still it still starts to download immediately. And that is web streams. They're using I remember hearing that they wrote the tarpauler in Node.

And what they do is as they are zipping your repo up, You're downloading it as they're working on zipping the rest. So it's it's a perfect example of streams. I thought, oh, should I mention that?

I didn't know you could do that in zipping. I didn't know you could send some bit of data along and, like, incrementally

zip. I didn't know that was a it. And the possibility even. Look. I guess because, like, the way zipping works is that, like, if you watch a zip file on your computer, It grows incrementally.

So instead of putting those chunks to your file system, you're just literally sending the chunks Over the wire, and then their your user is putting them on their file system. I thought, oh, man. That's a genius use case for that. I remember when it Somebody said, oh, they they wrote the tarballer in Node. I always thought, like, oh, like, why was why was Node good at that? It's because Node was awesome at streams When it wasn't released.

So Yeah. Curious.

I had no idea that, that you could do that or that they did that. So thank you for bringing that, to the start of this one. Cool. Yeah. I I always thought about,

one more thing. I always thought about, like, if I I offer downloads for a lot of my courses, and people share them and put them online. It's kind of frustrating to have to deal with it. And I always thought, wouldn't it be cool if I could, like, watermark The video with their name as you download it, you know? And that would probably be it. That's a lot more. You need, like, literally, A video processor, but you could theoretically do it with that as well. Yeah. It I mean, the

the DRM in that sort of way is always very interesting because, you know, so so my uncle was the editor in chief of a magazine, A guitar magazine for, like, most of his career. And so because of that, they were always getting CDs before they were released, promotional CDs.

And record companies, this was like peak Napster era. We're having a lot of trouble with people who would get these early release copies.

Somebody would rip it, and then that CD would end up on the Internet right before it was released. Oh, yeah. So I distinctly remember when my looks my uncle, every year for Christmas, would just send us a box of CDs because They would just stack up in his office, and he would just shove them off his desk into a box and then let us fight over them. Right? So it was always awesome to get them, but there was a distinct Year where we open the box, you put on the CD, and it's like, this is a promotional copy for Brad Talinsky. Like it. Like, the the audio was watermarked with his name on it so that you could always trace back exactly where it would have come from. Yeah. Very funny. That's funny.

Well, we're not talking about that today. We're talking about, HTTPS Cuddling and you're getting basically getting your local host online for other people to see as well as getting your local host via https.

We'll talk about both of those things, but we'll do real quick. Our sponsor we're sponsored today by Sentry at Sentry. Io.

Their tagline right now is, I really like they're always changing up their home pages. Working code, happy customers, take actions on broken lines of code, crashes, Busted API calls, with the only developer first application monitoring built to give you answer, not answers, not clues. That's it's totally true. They We'll help you figure out what went wrong, when it went wrong, for who it went wrong, why it went wrong, where it went wrong, all of those good things. Who, what, why, it. I wear when all of those good things. It is fantastic. You gotta throw it in your application before it's too late. Check it out. Sentry. Io use coupon code tasty treat for 2 months free.

Tick. Cool. Well, let's start with h g, HTTPS or an SSL certificate locally first, And then we can move on to tunneling. So, you know, 1st and foremost, you might be thinking, alright. I have my local host set up. I typically Run everything I need to run off of localhost, and my stuff just works fine. In what universe would I need to have an SSL certificate for a a local host, based website. And more importantly, is is that even, like, something you can do straight up? And why isn't local host necessarily good enough? And and we we should say that at at 1st and foremost, local host is a secure context. Right? So by by default, you don't have to do anything for local host to not be a secure context, And some browsers kind of handle this a different way, but, it's not necessarily an SSL,

But it's a secure context nonetheless. We don't have to worry too much about that. Right? Yeah. Exactly. That's we're gonna talk about a lot of the JavaScript APIs that don't work unless you're on a secure context.

And the answer to that is local host is considered a secure context. Even though it's HTTP, Yeah. Browser treats it as so because it knows that it's running on your local machine locally.

Yeah. There's no requests

Necessarily going out. Yeah. You need to be encrypted there. I I had a TikTok about using Caddy server to get HTTPS locally.

And I had so many people saying, why what's the point? Why would you ever want that? Why do you so, like, The answer to that is, first, a lot of APIs must run-in a secure origin in order to work.

It's so much so that, like, it's almost not worth not running one of these things.

And the answer to that is, yeah, you could use local hosts. But then if you have multiple applications, You have to run them on multiple ports. And that gets kind of annoying because, like, I ran some some project on the default port, like 51/54 or something like that. And then that thing installed like, what's it called? A web worker where Where is the cache? And then I ran something else on that later, and and they would still give me the cache version of the old one. And you pretty much need a new port for every single one. And then, So like your browser history gets messed up. And so what I like to do is just put I have like local domain names for every single project, which is like I use dot local host.

So you can say, like, I have, like, boss monster dot local host. And then I have, like, JavaScript 30 dot boss monster dot local host. And I have a catchall route for all of that, and that is really nice to be able to do. However, you lose the secure origin when you do that. So you have to use something that gives you

HTTPS so you can use all of those APIs. Yeah. And I I typically do dot dev. I know that's like now a thing, but I've always done dot dev, whatever my domain that I'm I'm working on just that. Oh, yeah. Instead. But it's it's only because I don't Have any .dev domains and I largely don't think about that. Don't care. Yeah. Yeah. I was I was using .dev.

Sorry. I was using dot wes for the longest time. Oh, that's fine. Until I moved to caddy server. And caddy server will only work if you use dot Local host or dot local, which is something that your computer your computer recognizes those 2 things as Like on your local network and it will sometimes you type in a domain name and it thinks you Googled it. That never happens with .localhost and .local. But,

Well, if you update your host file, I've I've had Caddy server work just fine with that dev. And Really? Yeah. Yeah. Because I I used to do that with level up dot dev, especially when we were working on our authentication piece.

There there's some weirdness with Safari and local host storing cookies, HTTP only secure cookies.

So I needed to run a domain, and I did dot devlevelup.dev.

Oh. I had my host file updated, and I was able to use Caddy. Could probably dig up my config file somewhere for that. But, yeah. So, basically, that gives you access to both. If you use something like Kati, it gives you access to both an automatically generated SSL certificate on the domain that you're using locally and It allows you to develop locally, and the one of the big things that comes along with this is it it feels to the browser More like what you're running in production even if the code isn't bundled, minified, whatever.

You're still running a context that's more similar to what is being run when you're running off of a normal server for production. Anytime you're dealing with, things like HTTP only cookies, that can be, a situation that ends up helping helping you make sure that your setup is actually working it. Because there are a lot of times when maybe perhaps your authentication system, if you're writing your own, will work fine on local host. You'll push it to production.

And then all of a sudden because of how you set up your cookies, when you go to change those to be an actual domain, they might not work correctly.

Yeah. And whether it's a subdomain or whatever, they might not work correctly. So you need to, be in the same type of context to be

Sure that that authentication piece is going to work. You must replicate your local environment as close as possible to the Your hosted environment and if you are any way different, you could hit a lot of little bumps here and there, little issues. There's there's even like, there's a list of Top level domains on GitHub that the browsers ingest.

And it has .vercel.com.

It has s dot heroku app. It has dot Netlify.

And basically, if you have a website like scott.versal.com, CS. The browser knows to treat that as as if it were scott.com because it's possible that I put Something on some west. Vercel.com and you put something on Scott. Vercel.com.

But if the way the cookies work is If you have a cookie that's domain wide, then I could access your cookies from my website. And if you're on a public host Like that. Same with, like, Wix and all that. I thought that was really interesting. I ran into a major bug years ago, and it ended up being oh, us. Because I had back end whatever.com and frontend.whatever.com.

Worked great locally, worked great in production, did not work in the staging where I was using the the Vercel or now URLs. And it was because the browser said, uh-uh, Those are in my mind, those are the like 1.com and 2.com.

Mhmm.

Yeah. That that is one of the trickier bits there. That's Certainly one of the things that are are no fun when you're dealing with that. Absolutely. So, you have a list here of things that are, in need of secure context. Now like you said, these don't necessarily need an HTTPS on local host to work, but they do need local hosts. So if you you, You know, they just need to secure context like WebRTC for doing video, whether you're doing a webcam Streaming video, geolocation, the clipboard API, and not I think it's not even all of the clipboard API. I think it's just the The read? Yeah. I think it's just reading off of your clipboard. I I'm not a 100% positive, but I've I've used, the clipboard API copy rather than paste, quite frequently.

It's been fine. Worry about. I guess everything is secure context nowadays. It's it's, like, almost hard to get things not in a secure time. Pretty much every new API

His over secure contacts, WebRTC, geolocation, clipboard, like you said, eyedropper API.

And it's because A lot of these APIs can be used to fingerprint you.

You may sometimes hear, oh, we would love to have that, but it can be used for fingerprinting.

What that means is that They could take things like interesting things about your, your browser, like what browser version is it? Does it support web share API? Does it support Bluetooth API? Is there something in their clipboard? How is the user holding their their phone at any given time? And then you go to another website, and you are you have all the same features as well as you're still holding your phone in the same direction. It says, I bet this is the same person. Even though you went from scott.com to buy a tv.com, They can fingerprint you and advertisers ruin absolutely everything. So this is like 1 step to trying to fingerprint you.

Isn't it? Isn't that the truth? Yeah.

Right. Isn't that the truth? Let's talk about tunneling now.

So sometimes it's not enough just to have SSL locally.

Sometimes you have to open up your application to the outside world so that other people or other things can actually access it.

So many services require your application to be open to the outside world so that they can hit it. Here's a couple examples I've run into.

I'm looking into integrating Apple Pay into my checkout right now because, Apple Pay now works on any browser On Ios. It previously only worked in Safari on Ios. And as of 3 or 4 months ago, now it works in like, I bought something the other day and it prompted me to use Apple Pay from Firefox On iOS. And I was like, that's nice. I should add that to my to my checkout. So I was, like, going into it, and then Apple Pay requires you to add a domain name Before you can use it. And you cannot you cannot run it locally. There's Just no way to absolutely test this locally. So you must be running it on a public facing domain name That has been verified. You have to upload a special file to your well known folder. Known. Yeah. Yeah. Yeah. And so the the the solution to that is you use what's called a tunnel, Meaning that you put a publicly facing URL on your local host, and that means anybody can visit that website And and go to it. So I I used one for Apple Pay, and it it validated it. And I could test it in the browser via that special URL.

Another thing I ran into is web scrapers. So Snipkart, the way that Snipkart validates the price of something Is because you set the priceyou literally set the price in HTML, and you think, oh, anybody can change the HTML. So the way that Snipkart validates the price of something is that it pings your URL on its own servers. It scrapes the page, and it checks for the HTML being there. And if That is not publicly facing, you're kinda out of luck because they can't go into your local host unless you have Surfaced it some way. Same with webhooks as well. If you wanna test webhooks, Stripe sends you a webhook for something that needs to be publicly facing so they can literally at hit the URL.

Yeah. The webhook one is the one I've hit the most, specifically, like, dealing with reoccurring billing. Right? If you're trying to test reoccurring billing, You know, you wanna make sure that when you cancel a subscription, that that subscription cancels.

You need to have a webhook. That webhook needs it. Be accessible Vaught via the actual Stripe interface or Braintree's interface to be able to then send a message it. To your local service and actually trigger the event from happening. And that that's one of those things that you definitely hit, when you when you first hit it. You're like, alright.

What do I do here? How do I get this? And tunneling is is really the answer. And I do think, you know, the Best visual way you can picture this is to just say, you know, we have a a tunnel going under a fence. That fence is the bridge between The public Internet and the private local Internet. So you have your private local, server, your local computer, and it's like a physical tunnel from the outside Through underground into your local machine, basically giving, another computer access to Your local machine.

So Yeah.

You need to consider that that it it is giving outside access to something on your local machine, and that's something you kind of have to be cognizant of to make sure that you're perhaps not exposing things that you shouldn't expose it. To a public URL, you're not giving file system access or something to a public URL. There's not, like, a route on your website that's Delete system 32 that's available for a local host to come in. Yeah. Or, like, you have a simple file server running and someone can

Try to look for dotenv, you know, localhost forward slash dotenv.

Thankfully, most things like Vite will

Detect that and be like, no, you shouldn't never be able to serve this file via the browser and it will shut it down, but you gotta be careful. Yeah. Via the happy path or, like, the suggested path, there there's not a ton of ways you can get into trouble. But when you start getting off that happy path and doing whatever you want,

Like, you kinda have to at least be cognizant of the fact that you're opening this up. One that I just switched to is called Cloudflare Tunnels, and I was really happy to see this. This has been It's relatively new service. And what it is is you brew install Cloudflare or you download the binary from the Cloudflare docs, and then you have this just CLI that you can run. It's called Cloudflare d with a d on the end. And it it does a whole bunch of stuff, but the one thing is a subcommand. It's called tunnel. You run cloud I'm just gonna say Cloudflare. It's Cloudflare d, but I'm gonna trip over that. So Cloudflare tunnel dash dash URL, and you give it your local host 6969 or whatever. And what that will do is it will then spit out a publicly facing Cloudflare domain name that you can then visit as if that were your application. And then you can send that URL to literally anybody. And that's the way all of these tunnels work is that it gives you a URL For your local host, and you can then as long as that process is running on your computer, people will be able to specifically visit that URL and run your application As well, you do need to be careful with like hard coding URLs. I know like WordPress specifically does this where Like the images in WordPress will be saved with, like, local host 6969.

And if that's the case, then you have to, like, Run like a find and replace in between there, but it's not not the case for most, most applications these days. Word.

The The one that I've used most, and I I would assume is very popular, is the [email protected].

This one was always the easy option.

They've since kind of expanded. There's a lot of, paid features in this one in terms of, like, having a consistent domain and giving you a bunch of, like, extra stuff, but I've never done anything beyond the free plan because it's never been a deal breaker for me to just, like, Maybe I'm testing a webhook one day, and what I do is I just spin up ngrok.

And I use the free version or whatever. I Spin it up. It gives me a domain. I copy and paste that into the Stripe webhook interface that it has on their system. I test my endpoint, and I'm typically done. But that's not necessarily a great workflow for, like, a permanent setup. So if you're getting into a permanent setup with ngrok, you're you're gonna have to pay some money. Yeah. Yeah. That's Why? So when I when I was doing my,

t shirt selling online using Snipkart, I paid for a year worth of ngrok so I could get A consistent domain name because you can't every time you restart the ngrok process, it gives you a new domain name. Same thing same thing with the Cloudflare tunnel. However, the Cloudflare tunnels also allows you to Set up a custom domain with it as well. I'll talk about more of that in just a second. So ngrok has been great. It's just like rock solid. Whoever, like, It just, like, kept getting more expensive every single time, every year that I want. That's really the downside. Yeah. It's $120 a year right now. And it's like, that's That's more than I pay for hosting some of my applications, and it's just it's not worth it to me. So I canceled it and looked into using Cloudflare tunnels, and I was pretty happy to see that It is free. And I know every time I mention either Cloudflare or anything that is free, yes, if you don't pay for the product, you are the product.

And People are very wary of Cloudflare as a company because they control so much of the Internet, which is both of those are good things to think about, I guess.

Yeah. But they do do a good job. They do? I'm a big Cloudflare fan. And probably at some point, Cloudflare will do something stupid, And everyone's gonna go, where's where's your blood flow now? You know? Did you see that? It's the same thing with any time. Yeah. There's, like, a wise thing.

A 100 messages. Did you see the wise issue? I'm like, oh, gosh. Like, I'm not a spokesperson. I just like I I like the stock, you know? Yeah. I hear you. Other one is Local Tunnel.

This is one that I have used for many years at a time. So Local Tunnel is totally free and N gives you a consistent domain name, which is great.

However, the consistent domain name was flaky for me. Like, it didn't it wasn't a 100% of the time. And I don't know if it was like if you kill the process and you start it again, if it, like, hadn't freed up that domain name or somebody else took it.

So that's Kind of annoying having to do that because if you can't get the domain name, then you have to literally go to your Apple Pay or to your a shopping cart and, like, change all of those domains.

And they also throw up, an interstitial, which means I think that's The word for it. Basically, they click they throw up a click to proceed because the problem with giving people Consistent free domain names is they will use it to spam people or to phish people.

So they always throw a little thing up like, Hey, this is a temporary domain name. Like, don't put your credit card or anything into any of these URLs.

And that was getting in the way of The crawlers and the Apple Pay and all of that. And there's docs on how to get around it, but you have to rely on, like, in my case, I was like, Hey, Snipkart.

Can you, like, put the special header on your crawler? And there's, oh, yeah, maybe. But, like, I can't. I need to Fix this right now. So I moved to ngrok. So local tunnel is a very good option as well, but I found it just to be a bit flaky lately. So I've I moved away from it, in the last well, you have also tunnel

2 dot dev, which I have not Used myself. I don't I've never even heard of this until this very moment. So, what are your thoughts on tunnel 2? Yeah. This is one

A lot of people replied in the tweet thread that they had used, and this one looks great. It's $2 a month, which is more than enough, especially If you want the consistent domain name and don't want to have to go through all the Cloudflare steps, this is an awesome, Awesome approach to it. So I would probably check this one out as well.

Apparently it's open Is this the one that's open source? I know Localpanel is open source.

Yes. OpenCore is As well. I always thought about that is, like, maybe I should just run like a Docker image on my NAS and have full control over this type of thing. You know?

Yeah. That is certainly a great option as well as you get into SSL certificates for that kind of stuff. And

yeah.

No fun.

So let's let me talk real quick about the Cloudflare custom domain names. So Via the Cloudflare API, you can set up a tunnel. You can call it like local or you can call it Scott or whatever you want. And then you CS can set up a domain name that will point to that. And anytime you run that specific tunnel, your domain name will attach to it. So what it does is it goes into Cloudflare sets a c name record for a subdomain, which is like test.scott.com or local.scott.com.

And then every time you run that, it will proxy all of that traffic through your own custom domain name, which is awesome, especially if you're, like, sending this stuff to clients While you're working on something, it's nice to have your own domain name and you can see it being edited in real time, especially if you're using like a Vite or something. You're on the phone, like, I'm not sure about that red.

Save immediately updates right in front of them. Right? So Skyfire is pretty cool that you can do that type of thing where you can put your own custom domain on it. I've thought about just make like registering My own domain name for this type of stuff because oh, that's the other thing is you can also use this to put in front of your Synology NAS.

Apparently, it's against the terms and conditions, but a lot of people, what they do is instead of fussing with all the Synology URLs, all the like, I I have tried so long to get my Synology HTTPS working, Any idea. Doesn't freaking work. Like, it gives me the domain name, it registers it, it self signs it or it signs the certificate with Let's Encrypt. It does all of that stuff. And, like, I know that. I know how all this stuff works. And then you visit the URL and it just says, can HTTPS error blah blah blah. So what you can do is, you can install this on your NAS as a Docker image, Cloudflare, and you can tunnel all the traffic To a, like, a bossfamily.net or something like that. You can log into it, which is pretty cool. And then also Cloudflare has a product called access control, which is has a very generous free, and you could put access control on The Cloudflare level, meaning that, like, okay, I have this secret super website that I wanna open up to the public Internet, but I don't want to let just a random person stumble upon this. You know, like, somebody could look up my DNS records on my domain name and and say, oh, That's where Wes is testing his localhost. They could run a script, ping it every 5 seconds, and then, oh, something's running. Let's go check out what Wes is working on and whatnot. So You can put a username, password, or access control at the CloudFort level without having to actually code that into your application.

Mhmm.

Yeah. You know what? I always just use, I know back to the Synology thing, the quick URLs or whatever.

Yeah. The quick connect. Yeah. But does that work with, like like I specifically want it so that I can run my home assistant and app stuff. It does not. Yeah. So that's the problem that I run into here.

Yeah. All the all the apps. Like, I can't get access to my home bridge when I'm outside the house.

And I want to open the garage door when I'm not within WiFi.

Totally. Yeah, I know you can buy Apple home pod. I'm not doing that. I'm going to go the hard way And figure out how to how to use this. So I think I'm going to apparently it's against TOS somebody said, but

I'm not not running, not not piping movies through it. You know? Cool. So that's about it for tunnels and h t t p s locally. See it. Let us know if there's anything that we missed that you're using this for, stuff that you think is important to know, and we will hit you up with one of those, retweets and Maybe start a conversation about it. So as always, thanks so much, and, we will catch you on Wednesday.

Head on over to syntax.fm for a full archive of all of our shows, And don't forget to subscribe in your podcast player or drop a review if you like this show.