Supper Club × GraphQL as an Aggregation Layer with Filipe Ferreira of Sky TV

Welcome to Syntax.

This is a podcast with the tastiest web development treats out there Today, we've got a really good one for you. We got Philippe Fiera on, and he works for Sky, which is big like, I'm sure most people know what that is, but, a big TV provider out in the UK as well as, some stuff in the states now.

And we are going to talk about, some GraphQL stuff as well as like, I've got a 1000000 questions about Working on a streaming platform and and whatever else pops up, so buckle up for that.

We are sponsored by 3 awesome companies today. First one is Gatsby, The fastest headless website, Retool, the fastest way to build internal tools on top of a database and story block The headless CMS with limitless technology support to help you create the perfect project. We'll talk about all of them partway through the episode.

But with me as always is mister Scott Dalinski and Felipe. Thank you so much for coming on. How you doing? Hey, guys. Thanks for,

inviting me. Yeah. Pretty good. Yeah. You're welcome. So,

we were chatting after our last GraphQL, show, and you said, like, hey. Like, one thing you didn't Talk about with GraphQL is that, a lot of companies use GraphQL sort of like a gateway to normalize, different APS here. What did you say? Hey, Wes. Long time viewer of the podcast. In bigger enterprises, it's more used as a filtering and aggregation layer for existing APIs, which I thought was really interesting. So, Yeah. What do you wanna give us a quick rundown of who you are, what you do, where you're from, all that good stuff? Yep. So,

I'm from Portugal, but I work for Sky in the UK.

I work on the GraphQL team. We support a lot of, products the Sky builds throughout the world and other companies such as NBCU. So we got, the new streaming service, Sky Showtime in Europe. We also support, Peacock in the US and other Sky streaming services throughout the rest of Europe as well, such as NOW TV.

And, yeah, after the last show you guys were talking about, and a lot of people talk about as, GraphQL as a replacement for your REST layer on top of the database.

But I think, how we are kinda using it and how a lot of people are using it is more of a filtering migration layer. So the analogy I like to use is if you go to a big, Like hardware store, you don't just go in and buy what you want. Right? You probably have a list of things. You go to the cashier, and you say, I would like these things, and they'll bring it to you. And I think that's what GraphQL kinda does for us is we can like, the front end can call us and say, this is the data we want. We know where it is. We can filter and act for them, and then, they'll have it ready. Yeah. That's a good metaphor because you're thinking, like, The store doesn't care who makes the products necessarily. Right? Or you don't care,

going into the store. You don't have to go to each of those products stores individually Who makes what? You're just going to the store where all of the information is, and you're saying, hey. This is all the stuff I want. I don't, You don't necessarily know the original origins of all this stuff, or maybe I do know the origins. But I'm not traveling to each of those origins specifically, and just give me that the data from there. That's a a great metaphor.

Yeah. And, also, it allows your front end teams to not have to worry about because What you have with a lot of GraphQL projects is you have to orchestrate quite a lot of calls because you might call this endpoint and this other endpoint, and then you upgrade this data together.

And basically GraphQL just abstracts all of that and gives you one response.

Maybe paint the picture for me of, like, what do the systems look like inside of Sky, where I'm assuming you have multiple systems, some probably many years old, some probably new.

And then you have Your team, which has the GraphQL layer, and you sort of are the gateway to all those, how to maybe visualize that for us. Okay. So

You might have a team that's, like, in charge of doing your like, the user profile authentication kind of thing. Let's call, let's call it the ID team. Right? So they'll build they'll build that for the whole let's say, the whole of the company, and there might be 100 or so teams that will rely on that.

Right? And they build the REST API, for example.

You can't just ask them to change to GraphQL. You're gonna just go, You know what? Let's just ask you to change GraphQL for us. And all these other 99, teams just have to change for it. That that's not feasible.

So what you can do is kinda look at, You know, what is actually a database? A database is a database, as we know, but maybe with GraphQL, a database is also an API. Right? I like just the rest of the guys. So so you might have something like on the profile that you need for the user.

For example, let's say the language that the user wants their website to be in.

And you might may want to have, like, which products the user has available.

Let's say you have, like, the ID team, which has your profile with your language that you've set. And let's say you have another You have another team, let's say subscriptions team, that has your product. You might want to have that in one call so that when the user logs in, that's available to your front end.

So you can call GraphQL and say, this is the you know, I want the ID. I want, like, the name. I want the, subscription the products the user has, and then just have 1 call for all of that instead of having to, you know, make multiple calls and

Wait for everything to come back and coordinate all of that together. T I guess it doesn't really matter, but, like, is it a separate team that implements the GraphQL API Specifically, where you're working or does every single because another option is, All the teams can implement it yourself. And then what's the what's the word where you stitch all the GraphQLs together? Federation Forget stitching? Federation. Yeah. How how does it work at Sky?

At the moment, we're just kinda starting. So, we just our team just builds our own graph.

We we can work with back end teams.

And if they wanna build their own graph, then we can federate to them. But I think Going forward, we're gonna be like a gateway. Even if they fed rate, we're still gonna be the the one that our front ends will call. Yeah. Yeah. Everything goes through you. So it's all being collected essentially into,

for the end end user consuming this thing like one API. Right? Does the user who's on the front end team, do they have to be aware at that case Where the information is coming from, if it is a federation.

So, like, if I or federated, a federated API here. So if, like, I'm pulling in information, does the UI team at all care about where this information is coming in, or they're just asking for the data they need and getting it back. They don't have to care. They can just ask,

for the information that they need on the GraphQL layer, And then the GraphQL can go and, you know, may maybe he makes 3 or 4 calls to different back ends or different APIs.

It doesn't it doesn't like, the front end doesn't need to know where the data is from. Is there any, like, major concern around performance in that regard? It seems like kind of caching

some things in that way might be tough if something relies on something. You you might hit some waterfalls or some And plus 1 issues is, like, caching a big big component of that federation?

Yeah. So, we actually have quite a lot of caching that we do. And we are actually able to mitigate system, like, system failures because of our caching. So we have the 1st kind of layer of caching is in memory caching.

So we run everything serverless, by the way. I don't know if I said that. But we run everything serverless.

So, yeah, we run Serverless. You can do this thing called provision concurrency in AWS that you can run serverless, but you always have a function warm and ready. So you're kind of running a server by serverless,

which is a bit good. So every single function that you have is Do you keep at least, you just keep 1 of them warm all the time?

Yeah. You can do that in AWS in, like, your landline. What's the benefit of that? You don't have start up the function, you can also use in memory caching

with Oh.

Memory caching. Oh, yeah. Because in in memory here, I'll I'll let you keep going. But And memory caching is exactly what I'm using to do the thumbnails on my website, because I generate the thumbnail, and then I have this JPEG data, and I just stick it in memory. And it stays there, like, long enough, like, 20 minutes or something like that. But you're saying I could also just make that what's that what's the word called again where you keep it hot all the time? Provision con provision concurrency.

Provision concurrency. Okay. You can say, like, I want 4 functions, radio all time or whatever, how many you want.

So we have that layer.

Okay.

We also have Redis.

So we use I don't know if you guys ever seen, a A pattern in a GraphQL called data loaders?

Yeah.

So with data loaders, what what we kinda build is We can cache back end responses, be it from a database or an API, up to 15 minutes if they don't have any, like, user information.

So we can also cache that into Redis.

And the biggest cache that we have is actually just your normal CDN caching that you can put on front of GraphQL.

You can actually like, we at one point when we started, we had, like, maybe, like, 98 sent of all the calls cashed because it was just like normal get calls that you can cash for 15, 20 minutes. Uh-huh. And that's like like, give Give me some examples of stuff that might be cached, like a a list of movies or a list of TV shows that are available. Yeah. Like a collection of movies, your, like, website labels, you know, like, parts of the UI that don't change as much.

But then, like user information, you can't cache that due to data protection.

Oh, yeah. And that that always goes so that goes straight to the database. So if somebody goes to there and they say, I wanna see What shows I watched last, like history or something like that, that would always go straight to the database. But that's not necessarily a Huge strain because it's just 1 person doing that. Is that true? Yeah. Also,

because it's a back end call, and our servers are collocated basically in the same data centers as the back ends.

So it's not like, it doesn't really matter what kind of data you push there As long as the the the data out of GraphQL into the actual, user is quite small.

Oh, so What maybe this is a better question first is, like, what's the stack for the front end then? What does that look like? So the front end is just a simple React app. You just literally have, like, React And Redux, you're kinda standard.

That's that's pretty much it. And then

are you doing any, like, server rendering? Because you said No. No.

No. No server rendering. But because it's it's like, so, like, where's the benefit in doing the front end and the back end on the same server then?

So when I say back end, I mean, the APIs that GraphQL is calling are in the same data center. The so, like, the GraphQL server

And the database and all your other servers that are running are on the same machine, so there's no overhead. There's no round trip across the world to get some data. It literally is in the same server rack. Right? Yeah.

That's cool. So GraphQL what what we actually can do in GraphQL is ask for more data than we need, and then we cache that. So, for example, if you're asking for the movies, instead of asking for the movies you want, you just ask For all the movies, and then you just pick the ones you want and serve that to the front end.

So that's when I said we can mitigate System failures.

If your movie catalog service is down, you you have all the movies, so you just pick the ones you want and set up the front end. Oh.

Oh, I see that because it's because it's on the same server. It's cheap to query literally everything. Stick it in redis, and then Filter the stuff to like, what's expensive is sending it to down the wire to the actual user. Right? Like, that's what's slow. That's what it could be megabytes over. You're not gonna send a list of a 1000 movies to user. You're gonna you're gonna send 10 or 4 or whatever they search for. Yeah. And

When I say that GraphQL is, like, one of the best for this is because you specify to GraphQL what you want. So we know we know what You want in the front end, but in the back end, we can just say, give us everything, and then we just pick the ones we want. Yeah. That's neat.

That's cool.

What about caching on the client side as well? Do you do anything like that? Does it make sense to cache it when it's so quick from, from your GraphQL API? I know you said you use Redux. We with GraphQL, we're kind of moving with the Apollo

client integration in the front in the front end. And they do some caching, locally.

We don't do any extra caching besides that Because it doesn't really make sense at that point. But it does have some caching. And there's some something to do with, like, how you structure your GraphQL IDs And pull and using it between queries.

I'm not I'm not too much on that side, but there is some cash in there. Yeah. We use I've used Apollo in, my courses a couple times. And

if you give everything, like, a unique ID like, you you you generally, you you say, like, movie Colon, the idea of the actual movie. And you can generate those pretty easily.

And does a great job because if you're not If you try to display that movie again, you know, oh, it it just grabs it straight from the cache. It is a huge pain in the butt with pagination.

I've forever gone down many rabbit holes of of doing pagination,

but everything else has been really, really nice. I'm a big fan of that. I think when you say ID, it's just using the ID type in GraphQL, isn't it? Yeah. Yeah. Just using the ID type. What's great too is that, you know, it makes it I mean, using something like Apollo like that, it makes it really easy to throw in, you know, local storage side caching as well Or that normalized cache if you want it, or if you don't, it's just storing it in memory as the cache itself Where the you you can specify for for people who haven't used Apollo before.

You can specify Where that data is coming from in each of your queries, you can specify if if it's coming from the cache, the network, the network, and cache depending on how you want it. So you can tailor each query to your liking based on what you need at any given point. Were you around at Sky when they decided to go serverless? Do you know, like, how long they've been

On totally serverless? Yeah. So we are actually the 1st team. And now everything at Sky is serverless or just Your team. Just my team for now. There is other teams that are, like, picking up the work, because we built like a a kind of library that helps, Going like, the the whole infrastructure side of things. Oh, really? Can can you tell us about that? Yeah. So, we built, like, a library that, handles the whole serverless part So that you just in your in your app, all you have to do is your app. And you then just with a very small layer of configuration, you say, this is My function that I wanna start, and then the the library handles all of the deployment, and how many provisional concurrency you want, how many Lambdas and all that stuff. That sounds a lot like,

when we had what was her name? Christie. Christie. Yeah. When we had Christie on, she worked For the For, the, Liberty Mutual. Liberty Mutual. Yeah. So it seems like Yeah. That was literally her job was Making, it easy for the rest of the the developers to actually get up and running on serverless and making that really easy to do. We have a good track record of making libraries and making things easy, for the rest of the department. Is any of that open source, or is that an internal tooling only? It's It's internal telling me it's it's quite hard to open source. Let's take a quick break and talk about one of our sponsors, and that is Gatsby. They are the Fastest front end for the headless web.

Gatsby makes the hard parts of building a performant website much simpler with prefetching, lazy loading, an image optimization baked right on in. Hey. Guess what guess what Gatsby has? A GraphQL data layer. Oh my goodness. This this is can be not more perfect for the show. Gatsby's GraphQL data layer allows you to source content from anywhere, or you can use 1 of the 1,000 plug ins to, to source data. And that's really cool because Gatsby has a single GraphQL data layer.

However, it's likely that your data for your website lives in several different areas. Right? And what you can do is you can write plug ins or you can get Plug ins that will source your data from your your headless CMS or maybe from GitHub or maybe from some markdown files like I'm doing. And you can federate that all into 1 GraphQL API, and you can use it both at build time. They've done that Forever. But you they also you can host it, and it's a real time GraphQL API now as well. So pretty cool. Check it on out atgatsby.devforward/syntax to get your 1st site up and running in minutes. Thank you, Gatsby, for sponsoring.

I'm curious about the streaming aspect of of how Sky works.

I'm not sure if that's your department or or if you know anything about that. But as 2 guys who run video streaming platforms. I'm always I'm always interested. Pick the the brains of it. Do you know much about the streaming platform inside of Sky, how that all works? I actually used to work on the client team before. Oh, so there you go. So, the client team would be the person building the video player. So, yeah, we,

I worked on the team that built the Apple Apple TV player, but also built another product that Sky has called SkyGoat, both for Mac and Windows.

Oh, and what so you built it. I've never actually talked to anyone that's built an Apple TV app.

What is that built in?

We were using a product that actually got bought by Turner Media called UITV, which is a Which is a c plus plus engine that took, like, react native code, and you could use that to build your apps. That's cool. So

you were writing JavaScript

ish? Yes. So you're just writing React Native as you would write React Native, And then they would take that.

We're using, like, your SDK for, for example, your Apple TV.

You put that together, and their engine would Then spit out your, your application. I think it's used by, Twitch TV, for example. They use it as one of their big customers.

There was That's cool. Those are the ones. Yeah. But, it got bought by Turner Media. So, they now

Own it, so we don't use anymore. What about that? What was the other one that you said you were working on? I go for PCM PCM Mac.

That actually is using just Electron. Electron.

And is it, so it's just JavaScript, HTML, CSS? Yeah. React, Electron. That's it. Well, what kind of stuff did you use inside of that? Like So we use a,

we, we have a DRM library, that actually hooks on onto the player. The player was just Using, was also a bit proprietary from Cisco.

Mhmm.

But we do have, we do also have, like, The, the Google player, which is called Shaka Player? Sorry? Yeah. Shaka Player. Yeah. Well, I've used that we've used that in the past, or we have a version of that.

I think Peacock was using that, but, in the US.

And also, So, basically, it's just a it's just a, m three U8 streaming, into a player. In this case, Shaka player or Or Cisco DRM player for the UK, or we we have a bunch of different players because you can put it on the phone, you can put it on, like, iOS and Android, you Put it on the different, different, like, platforms, but it's at the end of the day, it's just 1 team building kind of that integration library. Can can I ask something you can reveal?

Where did did y'all decide to host, The video actual video files do I mean, I'm sure it's just in some storage, but did you go with the AWS route, or did you go Google? Or, is that is that proprietary information, or or can you discuss, where you decide to host that? Not sure it's proprietary,

but, we have multiple CDNs.

It's just, I think I think Not a 100%, because that's like a different department completely that handles the whole video ingest and play out services.

Yeah. No. Yeah. Totally. But I think it's just s 3 with, like, some CDNs on top of it.

So things like Akamai or Like a Cloudflare and all these CDNs.

So I think it would it would it would be something like that. Okay. Cool. Yeah. Yeah. Like, you've got, like, a a CDN that has you just, like, raw, high quality video.

And then you got something sitting in front of that that will Stream it, parse it, encode it, make smaller versions of it, 2 80 p for someone trying to stream on some crappy Wi Fi.

That stuff's always really interesting to me to see what they look like because I've I've gone through, like, a a couple rabbit holes of of that myself as well as,

I like to poke I like to poke around the different, like, websites.

And, like, when I see a video Totally. When I see a video, I, like, inspect source, see where it's coming from. Well, I think, I think, like, AWS released they have, like, this, like, I think it's, architect that they use for, like, building different service together. And they have, like, a Yeah. A recipe called,

like, OTT video platform and something like that. So a lot of companies are There's a VOD platform from Amazon because I actually spun this up. And what it does is it gives you A database to, it gives you an ingest process. It gives you storage. It gives you a database to keep track of Versions and all of the different, processes. It also gives you functions that encode the various videos. So on ingest, It's encoding all of your versions at that point, storing them in a folder, and then storing them into a database. And the cool thing about having that recipe there for you Is that the that's a lot of things that are connected to each other. Right? And it makes it so you don't have to write any of that connective tissue between all of those services Within AWS, I still found it to be pretty tough. So if you if you hear that and you say, oh, that that's great. I'm gonna spin up an instant one of these. I found it as a, a newer person to that kind of AWS environment, it I found it to be pretty rough.

Still, but it's great that they have that. I think that's a a big step forward. Yeah. No. I know that's not what we use, but I've seen companies that with a pretty pretty similar stack, like different websites. So I think

If people are looking at that, that's a good kind of start to kinda see what these bigger companies are doing. Oh, yeah. Totally. And if, if you if you fail on that, you can always check out Mux, One of our sponsors in the past sent us because,

that's what we ended up doing. I was like, I I'm not smart enough for this. Let me, let me pay a service to do all of this ingest process for me. Speaking of, like, ingest and all that, you were we were talking right before Scott jumped on is that, in the past, You've had to do, like, support for live events, and you're in the UK. So when the Super Bowl is on, you're up in the middle of the night. Can you see that? This has nothing to do with GraphQL, but I just thought that was the most hilarious story, and I thought it'd be cool to share that. Yeah. Like,

Because, as I said, we support the product in the US called Peacock TV. The knife Scott has seen it.

Yeah. When when there is live events, we do have to support it.

Different members of the team will will have to be up at 4 or 5 in the morning if it happens to be in California.

But yeah. Oh, man. That's that's wild that that you have to do that. So are when you have to support it, Like, what does that mean? Are you just, like, sitting there with your JavaScript files open, ready to fix any bugs that pop up? Or, like, what does that look like? So there's different types. So there's, like, release support, where you're kind of just deploying things and seeing how it goes.

And live events is more of, like, For example, the big ones is more of just monitoring systems and reporting back to people to say, this is okay or this is failing, kind of thing.

But it's just the only ones that we have to sit on a call is just US. I think it's a US thing. I don't know.

So you're just, like, Sitting and monitoring to make sure, like, certain thresholds aren't reached before either dynamically scaling up or preparing for those types of things. Or Is that, like, always a concern? It doesn't feel like there's that many outages nowadays in major services.

So is is it all just scaling up As needed. We don't actually need to do that because we have such a great tools and monitoring.

The Our average response time when there's an issue is, like, 35 seconds. Wow.

That's wild. And so you have you just have stuff that will, like, auto scale up, like, So many people are watching the It's just a function. Right? The latest Downton Abbey. And, oh, my goodness.

40,000 people are watching the That or whatever whatever you watch in the UK. What's that? There's a show that always played on Canadian TV, and I thought this is what British people Must watch all the time.

But I always I always call it Downtown Abbey because I thought that's what it was forever. And I called the Downtown Abbey to my wife the other day, and she was like, What? Do you actually think it's Downtown Abbey? I was like, wait, it's not? And I just found this out. So the fact that, I I I just got schooled on that, man,

I need to get some education there. Yeah. I mean, for us in GraphQL, it's quite easy because it's just functions. And if we need to scale, We will do, like, before we know something is up. We we normally just, Like, maybe, like, a couple of hours before, we'll add a few more provision concurrencies.

But besides that, we don't need to do much because we can just Auto scale our functions

to to whatever limits. Coordination Street was the show I was thinking about that all Coordination Street.

Oh, yeah. That's been around forever. Yeah.

Yeah.

Oh, that's cool. Like, we I I was just saying, like, with monitoring, we our, Like response time, which is the time it takes for something to fail, then you need to pick up your phone and answer it. It's like 36 seconds. Wow. Oh, that's really cool. Yeah. So let's take a quick break to talk about one of our sponsors today, which is Storyblocks.

Storyblocks is the headless component based CMS with a real time visual editor.

Their editor is really a power Feature of this Storyblock platform because it's a it's a great editor, and it it's one of the things that I think Storyblock was really based on When they initially started working on it because I remember using Storyblock, a couple years ago, and and I was super impressed with their editor, but it's only gotten better since then. There's a lot of really great tools that empower content creators to make their own changes independently and see them in action. So, like, as you're making changes, you can get A, you know, a real time update that allows you to see these changes before you deploy them, which we all know with headless environments is one of the big challenges you're working with. Content you're working with data, you hit save. You wait for this thing to rebuild itself, perhaps, and you have to go and see as you might need to make some further tweaks. Well, you don't have to do that anymore. You get these, nice visual UIs to be able to see your changes Before you deploy them with Storyblock, it's also ecommerce ready.

There's, their own robust SDKs and APIs, Internationalization options, you can do so much with Storyblock, and it's all about the content.

And it lets you integrate with any modern framework with, without any effort whatsoever.

What's also cool is that they have a component based approach, which means that you can create reusable content blocks, Which is great. That's like a a thing from I remember back in the days to do a lot of blocks in Drupal, and that's a a concept that has always really stuck with me, this idea of having a block of content that you can keep with you and and move from place to place. So Storyblock gives you the component based data approaches so you can have Componentized data as well as components in your headless site. So check it out today at storyblock.com.

You can use the,

The link in our show notes to let them know that you heard about Storyblock from Syntex. So thank you so much to Storyblock for sponsoring. Alright. I have 1 more question. It's, like I forgot to ask you. Like, what language are you using to write this whole GraphQL layer, and and are there any libraries that you use To make that a little bit easier on you. Can you guys guess what language we're using? JavaScript.

Yeah. It's actually TypeScript. Or TypeScript. Is the same. Yeah. So, we're using TypeScript, just using, in terms of language. But, libraries,