Supper Club × Voice Coding with Pokey Rule

Welcome to supper club on Syntax. Today, we've got another Amazing guests. We got Pokey Rule on today to talk about voice coding. We did an episode, like, I don't know, maybe about a year ago on voice coding.

And I forget who it was. Somebody was like, you gotta check out, CursorList. It's this new kinda Versus code extension that works with Talon. And I think, like, the the big the big decipherer here is that it works based on, like, ASTs. Tease. I get smart about where your code is rather than just, like, moving the cursor around and copy pasting, selecting, and I think that's pretty cool. So, we're excited to get into that. My name is Wes Bos. I'm a developer from Canada. With me, as always, is mister Scott Talinski.

How How you doing today, Scott? Doing good. Ready to go. Ready to get into some voice coding. We all know as, 2 podcasters here, We like to talk, so so it's,

kind of, kind of our thing is talking. So, yeah, I'm interested to see what you can do with your voice here. We're sponsored today by 2 awesome companies. 1st one is the Stack Overflow podcast, your weekly source for conversation with working devs and tech leaders. And Directus, the first Open data platform that turns any SQL database into an API and no code app. We'll talk about all of them partway through the episode.

So, Pokey, thank you so much for coming on. Really appreciate it. How are you doing? Yeah. Thanks for having me. I'm doing well. We have sun in London, so I can't complain.

Nice.

That's good. You want to give us a quick little introduction to, who you are and what you do? And then and then maybe we'll get into, To like, how did you even end up in the spot where you were trying to code with your voice?

Yeah.

So my name is Pokey Rule.

I, I'm originally from California. I've been writing code, almost as long as I can remember.

And, Yeah. Eventually, it caught up with me. I wrote 1 too many line of code and gave myself repetitive strain injury, which is like neck pain sort of, etcetera.

Had to stop coding for a while in graduate school, and so then decided I would try to get into has a way to stay intact without writing code, and, all turns out I hate management.

Some people absolutely love it. I found it wasn't for me.

And so I quit my job in 20 December 2020, to start a company, once I'd been managing for a while, my RSI was doing pretty well, and then I just coded like a madman for about a month, gave myself an RSI flare up.

My yeah.

So my wife had seen this, this video of, Emily Shea, voice coding.

Amazing video. I recommend anyone check it out.

And so I was like, oh, just give it a shot.

So I started using it, I would say start of 2021, and got up to about 70, 75% of my old productivity, but really struggled to get beyond that.

And so then, I had an idea for a way to make things better, and I guess that's, that's the story of today.

Nice.

So, yeah, it is it is funny how many people that you hear about Getting into management from coding who just that doesn't vibe with them, or they they do it for a little bit, and then they're just like, man, I gotta get back into what I was doing before just because I I you you hear it all. They're very different skills and very different kind of interests. You know? It it's it's a different thing wanting to be a a manager of programmers than it is Being a programmer They're completely different skills. I mean, in some sense, being a manager of programmers is closer to being a manager of

Any other profession that it is a manager of than writing code. Right? So it's like it's just a totally different profession.

Yeah. That that's interesting. We've wanted to have a a show on that as well because I certainly work with a lot of people who I very much enjoy, who are managers, and then I've worked with other people that are totally out to lunch and generally don't understand the whole coding thing as well. But it is a it is a whole Other skill set, especially sometimes I see these Totally. People. They're like they have, like, 8 hours of meetings in a day. And just like, how do you do that? Life. That was my life. Yeah. Yeah.

Yeah. And it was pandemic, so they resumed meetings. So I was just like, alright. Enough of this. I'm gonna talk to computers instead of people.

Yeah.

Oh, man. Oh, man. That's that's crazy. So I I I watched a quick little video you did at this conference. I'll put the link in the show now. I think it's called Bang Bang Con, where you were showing example of how you code with your voice. And I've seen a couple people do it over the time, but I've never seen anyone do it, like, so smoothly, and mostly using, like, real world language. So, let's get into that, like, first part about, actually just, like, coding with your voice. Like, what do you use to do it, and and how does it even work? Yeah.

So I guess, well, to answer your question, literally, I use a microphone, which is actually an important component.

So, you generally wanna get at least a decent microphone.

And, I'm using an engine called Talendvoice, which is basically a, a platform or library.

It's like a UX library for building Non keyboard and mouse user experiences.

So it is completely flexible.

Talent voice out of the box ships with no commands.

It is a way to define a grammar, which allows us, you know, what you can say, and then it has an engine which will listen for things that you've put in your grammar. And then it's a cross platform control library that you can use so that your voice commands actually do things in a cross platform fashion.

That's talent. What is that? So what is that built in? Like like, how do what does that look like? So okay. You you download and you install Talon or you you you set it up. Like, what do you do from there? Do you say, okay.

Here are some commands I wanna do, like move left, move right. Is that what you do? Well so, in Theory, you could, but there is a community grammar.

And so the community has kind of centralized around a, a set of commands which are universally fairly useful. Okay.

And so, usually, what you'll do is you'll download and saw Talon, and then you will clone, the community grammar.

And Oh, okay. Then you could do things like it'll it'll map the keys. Right? So you can say because, like, literally, you can't even type a letter with just talent, by default. Right? It has no grammar. So the community the community one will have things like, if you say air, it'll type the letter a. That will type the letter b, etcetera. You can say focus Chrome to focus Chrome, the application, and a whole bunch of other things. So That's in the, community grammar.

Okay. Okay. That's really neat. Yeah. So so the grammar is basically as a definition, it's basically just mapping Kind of commands to actions. That's that's the point of a grammar? Yep. Exactly.

And it's kinda like a giant reg x. Right? So Not every so the commands aren't just literal. Right? So several commands are literal. Right? You know, you can have a command, like so, you can you can define a little command. You can say, like, you know, Email Susan, and she could just on the left would be email Susan, and on the right, it would be an action that sends an email to this specific person. Right? But many commands are gonna be, like have captures in them like a like a regular expression. Right? So for example, when I said focus chrome, Really, that's part of a generic capture based command, which is focus and then application. And so then I could say focus Chrome, focus Slack, focus Versus code, etcetera. Right? Have you have you ever gotten into writing your own grammars? Kinda seems like they're they're written in Python. Is is that correct? Do you do you have any knowledge there? So, yes, Writing my own grammars is, pretty much what I do for a living these days. Oh, awesome. Oh, yeah.

So, they are half Python half what's called Talend script.

And so Talend script are these files that end in .talon.

They are fairly simple, and you basically it's like It's of the form, command, colon, and then action, basically.

Right? So it would be like, you know, email Susan, colon, and then on the right hand side, you define what happens, in that case. Right? Fundamentally, what defines your grammar. But then if I have something like focus and then I can put an angle brackets, application, right, That can match any application. I guess you focus Chrome, focus, whatever.

The definition of this generic application capture, that'll happen in Python.

Interesting. Do you ever get into situations where, like, you have a a grammar that is defined whether one that you've defined or someone else has defined And you're triggering it by accident? I just think about, like, you know, using my phone. I'm I'm talking to my wife, and I'm saying a word that sounds vaguely like, Gargle? Gargle or sorry or Yeah. Paluxa or any of those, and then they got gets accidentally triggered. Is that something you have to deal with?

Yeah.

Absolutely.

So I guess alright. So so, yeah, I have a couple things to say there. 1 is, the community grammar. So I'm one of the maintainers on the community grammar, And we are there's about 6 or 7 of us who are who are active maintainers, and we meet every couple weeks. We do backlog triaging, etcetera.

And, basically, one of the things that we're we're we try to do is make sure that the commands that go into the community grammar tend to be a little bit longer, and so that makes them less likely to fire. There's there's always this constant tension between commands that are short so that you can say them rapidly, and commands that are long so they don't get misrecognized, and just happen by accident. And so if they go into the community grammar, we try to skew a bit longer because we don't know necessarily whether someone's gonna want them And then trigger them accidentally.

The Oh, yeah. Second thing I'll say in that direction is that, if you are like, let's say you know? So there's a way to turn off, your voice coding engine.

And, if you forget to do that and you're just like you know, someone comes in and you start talking to them, I mean, it's like a like a a family of cats with lasers dancing on your keyboard. I mean, you could surely destroy your whole hard drive.

So That's hilarious.

Remove system 32.

Yeah. My my my daughter was in my office the other day, and I was doing some coding, and she came in and grabbed I have a little QR code scanner on a barcode scanner, and she came in and grabbed it. And she loves playing with it. So beep beep beep, Finding anything with a barcode in the room, and I'm just coding, and just random stuff keeps popping into my code. I was like, what is going on? Like, a 49b24, and it took me a couple of minutes to realize, oh, she's scanning stuff, and it's another input. It's just inputting that that text into it. So I can imagine it's kinda similar to that. Yeah. Yeah. It's it's a lot like that.

Let's talk about one of our sponsors. It's another podcast, the Stack Overflow podcast. For over a dozen years, the Stack Overflow podcast has been exploring What it means to be a developer and how the art of practicing software programming is changing the world. From COBOL to containers, Rails, React, Stack Overflow podcast is the best place to learn what's happening in development.

Each week that you're gonna hear conversations with developers and leaders from the top Technology companies, and you will understand how technology is made and where it's headed, is hosted by Ben Popper, Cassidy Williams, and Saidora Ford. The Stack Overflow podcast is your home for all things code. 2 episodes a week. Check it out wherever you get your podcast atstackoverflow.blogforward/podcast.

Thank you Stack Overhaul podcast for sponsoring.

Let's get you to do an example because, like, I've got a lot of questions about how the actual code works and and how you've implemented it all. I think, like, if people hear What it's like, before we go into, I think it it'll sort of frame that discussion better. So you said you prepared some sorta like, something that you're coding this morning? Yeah. So, basically, like, I keep a log of all the commands that I run.

So, I yeah. It was just Just right before, up to the wire here, I was just hacking on something.

So I just pulled up the log. I'm happy to read it off and Pretend like I'm voice coding.

Yeah. Yeah.

Cool.

Float state blue find disk.

Copy block Foxplex.

Drink x spawn, paste that.

Bring line spun before oats.

Bring blue ink to bat and kick. Chuck line spot.

Disc.

Drowse. Is that Like, drows? Yeah.

Honestly, it sounds like you're speaking not a different language, but, like, if you were to, like, aliens were to land on Earth, and you were trying to communicate with them.

Blue. Purple one. Like You should see the looks I get when I'm in the cafe.

I I I think about when I'm,