21: Geoffrey Litt: Software You Can Shape

Episode summary: Geoffrey Litt (Website, X) is a designer, engineer, writer, and researcher at Ink & Switch, where he champions malleable software: the idea that ordinary people should be able to mold the digital tools they rely on every day. Ink & Switch is an independent research lab focused on how computers can help us think and work. While researching and writing, Geoffrey and team also build products and prototypes to explore how their ideas can exist in practice. Geoffrey got his PhD at MIT CSAIL, where he built on his inspiration around computational media like spreadsheets, hoping to push more software toward the ethos of end-user programming, but without the technical complexity.

In a sense, why should using software and changing it be any different? Previously, he built software for teachers at Panorama Education, which he joined out of school as one of the first Geoffrey and collaborators recently published a definitive piece on malleable software and we discussed it in detail. We dig into why most modern apps feel like sealed boxes rather than flexible tools and environments, and what changes when your app, document, or workspace, feels more like Lego than machinery. Geoffrey makes his case that we want software tooling to feel like a chef knife, not an avocado slicer, and we talk about how the best designed tools help users up a smooth slope of learning and ability.

He argues in favor of deeper understanding, illustrated by one of my favorite ideas: The Nightmare Bicycle. Welcome to Dialectic Episode 21 with Jeffrey Litt. Jeffrey is a designer, engineer, writer, and researcher at Ink & Switch. Ink & Switch is an independent research lab focused on tools for thought. Essentially, how can computers give us more leverage as we think and work? His focus there is malleable software. The idea that regular, non-technical people should be able to change, edit, and tinker with the software tools they use every day. Jeffrey's obsessed with spreadsheets because they're what you could call computational media.

The idea that using software and changing or editing the software should be one in the same. We talk all about how more software could feel like spreadsheets and what is required both from a design and an ecosystem and infrastructure perspective to make this true. As you might imagine. AI and LLMs are a significant lever to help make more software malleable, as in theory, anyone can edit code at the speed of thought. While I think this conversation will be particularly interesting for designers and those who make software, I also think that anyone who uses software will be challenged by and hopefully inspired by Jeffrey's ideas.

At the root of all of this is a notion that we should care about our tools, our processes, and our environments, not as an ends, but as a means to working on and creating the things we care most about. And while that applies to software, I think it applies to our lives too. Steve Jobs comes up at the end of the conversation, and I couldn't help but think of that incredible quote of his from years ago. where he says that the minute you can understand that you can poke life, and if you push in, then something will pop out the other side, that you can change it, that you can mold it.

That's maybe the most important thing. I think Jeffrey embodies that deeply, and I hope you enjoy the conversation with him. Here he is. Good to be here with you. Good to be here. Jeffrey Lit. Me too. In what seems to be summer in C. It's usually warm. Yeah, summer is here for sure. We have lots to chat about today. It's clear we could do this for a long time on a long range of topics based on our lunch before this. I think we're going to stay relatively narrow for a little while at least.

And we're going to talk about an idea that you've spent at least, I think, five years working on, which is at root about software and programming, but really about this idea you have of malleable software. I'll read from you first. You say the best environments come from adaptation. When the people living or working in a space pay attention to their own needs and gradually evolve to meet those needs, the result is beauty and quality. Everyone deserves the right to evolve their digital environments. It's the only way to fulfill our creative potential and maintain a sense of agency in a world that increasingly resides in software.

It seems like you've been chasing after this idea and this dream for a while, as I mentioned. Maybe starting zoomed out, why is this idea so interesting to you? And why is code and software broadly the creative medium of choice that you've been so attracted to? You know, I think it all started from, I find it almost painful when people can't change their surroundings, their environments, their tools. Whenever I see an example of someone fighting against the grain and unable to do the creative work they want to do or unable to focus the way they want, and it's because there's some tool in their life that it's either pushing a weird incentive on them or just isn't configured the way they want, it's this visceral feeling of that person is lacking agency right now and they've been robbed of something really important.

Like one specific story maybe that was one of my... waking up to this moment, I guess, was I used to work at the startup building education technology software for schools. And we would make these data reports where we would think really hard about, you know, what's the best report for K through 12 schools to see. And then, of course, you get on feedback calls and people are there's a lot of stuff that is missing or that's wrong. Right. And sometimes it's tiny stuff like, hey, could you change this one word in the UI?

Because at our school, that word is really contentious. And. Something I learned working at a startup is usually you're supposed to say no. If you're a good product manager, you're screwed if you say yes. And so a lot of your life is telling people, we can't do that just for you. And I think in Silicon Valley thinking, this is kind of valorized. It's like, it's great. You end up with a simpler thing. Don't just give people a faster horse. But sometimes I think actually people know what they need. And the only reason we don't give it to them is because we assume this kind of mass produced product.

And it's true. If you're building it in the factory, you obviously can't build it for just you or just me. But I started getting really intrigued by, could there be another way to think about software that grants people a little more control and agency over their lives? And to answer the second part, you know, I think the reason software matters to me is it's the medium that I chose to start building in when I was younger, I guess because I could and I didn't have to have permission. And it's taking over our lives, right?

How much time do we spend? I'm in my phone or in my computer, I think. It's a portal. It's a portal. And I almost feel like if we can't figure out how to give people agency in their digital worlds, that's so much of our lives now that we might even start losing the feeling that we own or control anything. If we spend all our times in platforms that dictate every detail for us, I think there's a way of thinking that might be lost. That's really scary to me, you know? And on the flip side, it's really exciting to imagine a world where more people feel, you know, there's this famous Steve Jobs quote, everything around you was invented by someone sort of just like you, I think is the paraphrase.

And I think that's a really exciting way to think about things. It's a great way to open up the conversation. I also just think, I think if you asked, if you surveyed young people today, particularly 20 year olds, we are starting to see what you just described, which is that people, feel as if they have no control over their own life. And that's a terrifying thing. Part of it, I think, is everything in the technology world has become more complicated. Like, it used to be that you could open up the car and understand a lot of what was going on.

Now there's like 100 computers in there or something, right? You can't open up your iPhone and meaningfully repair it or anything. And I think... In my own life, I think about, you know, I currently rent an apartment and there's a really big difference in mentality. I think when you start thinking a little bit more like an owner and not a renter, it could be something as simple as a toilet is an object that I can understand. Right. And like do something about as opposed to, oh, no, like call for help immediately.

And I just it's something I try to cultivate in myself over time. That part of myself is like, can I actually understand what's. what my environment is built of and take more agency over shaping it yes yeah it's i found that overwhelmingly and i don't know if you would even explicitly think of yourself as a designer but that feels to me to be the most it's the way you started your answer the most design oriented philosophy of the world which is like when i encounter friction i want to shape it and i want to fix it or tweak it or make it a little smoother And I, yeah, I think it's a really powerful gift to imagine giving to other people.

I want to talk more about it before we go deeper. And you started to talk about this maybe a little bit, but why at a super high level too, why should people, why do you think more people should program? Why should more people, and granted what that means might change or be in the middle of changing, but why should they be interested in using code as a creative medium? Yeah, I mean. to be clear you know i don't have a strong opinion on whether people should be programming more i think as you said that's a what that even means is changing fast right now and frankly i think the fact that until now you've needed to learn to program to shape your digital environment is actually one of the biggest problems in our software world and so it's not that i'm against people learning to program it's just that i think programming is not more people programming is not my goal but the reason i want more people to get involved in crafting their software is that i think Especially in creative work, there are certain types of work that basically can only emerge out of the correct creative environment.

I think of it as like it's almost shaped like a leather shoe around the person rather than the person trying to contort themselves to fit into some tool. And you see this with them. I love looking at photos of people's creative spaces, like their studios. And there's amazing ones like Stuart Brand when he... wrote this great book, How Buildings Learn, had this like, I think it was like a shipping container or something, and he put up like a bajillion photos on the walls and was sort of surrounded by the space he created, this container literally to do his best work.

And I really feel like for myself, the choice of tools matters. The tools that I use to write with people, the tools that I use to draw and think with, meaningfully affect the outcome of... my thought process it's not just kind of like zillion environments matter when you're in a room what does a room feel like what yeah it matters a lot and this goes back to um the original promise of computers was to let people create the ideal environment for themselves to do their best work i think there's some great thinking from pioneers like doug engelbart who is known in a shallow way for inventing the mouse but in a deeper way he actually kind of invented the idea of computers as a way to augment the human intellect, in a way.

And he has this great example he gives of, imagine you had to write an essay with a brick tied to your pencil. And think about how much that quote-unquote de-augmentation would affect your ability to think. And now imagine the inverse of that. What's a tool that would accelerate, or not necessarily accelerate, that would deepen your ability to do your best work? And I just find that potential. so intoxicating. And I think computers are, computers are probably the, you know, the only tool we've ever invented that are a universal simulator that can, it's a meta medium that can kind of do anything.

And so the ceiling on how, how cool that could be for people in terms of our ability to think and create is just so, so, so high and we're so far from hitting it. And so I think that's the potential that excites me, you know? Yeah. It's easy to forget that this is a very new medium too. And if you think about it in sort of super abstract of physical world and the digital world, we are only in the very early innings of getting accustomed to the dynamics of this new.

The other thing I think it's really powerful to think about is just that sewer brand idea you talked about, like the environment, like those environments are never, the environments that feel like that are never top down. Like they are never constructed and pre-designed. They always, it's like more like a garden. It's like a very Jane Jacobs kind of idea. 100%. Which is cool. They have to evolve, I think, slowly and as the needs arise. And, you know, they sort of gradually form around a person and a way of working rather than being planned up front.

I think of, you know, I think Christopher Alexander has been an influence on both of us. And he has this really lovely idea that you shouldn't really focus too much on the blueprints for the house. When you build a house, actually, you should... Roughly make a plan, adjust the windows as you build it because you'll see where the sun's coming in. And then crucially, you know, you shouldn't actually build the whole house. You should kind of build part of it and then start living in it and wait and see. And as things come up, you'll finish it off or evolve it over time.

And I find that way of thinking to be, I really think that's how a lot of the best things come to be in the world. Yeah. They're iterative. The irony, too, is that that's really hard to do with bricks or with wood or whatever. It should be easier with the kind of digital plastic idea of code. Let's dive in a little bit more. I have a smattering of quotes of yours that kind of in some ways encapsulate some of what we were just talking about and also kind of set up how I want to talk about what malleable software kind of is as you think about it.

First, you say, in the physical world, the act of crafting our environments comes naturally because physical reality is a malleable place. Some of what we were just chatting about. You go on to say, many small tweaks, taping a post-it note to the wall, rearranging some drawers, moving a piece of furniture can be done instantly without asking anyone's permission. We can also take on larger changes that require more effort and skill, like building a workshop, renovating a kitchen. We work and live in a physical space that we can control. This is both the Stuart brand and the Christopher Alexander of the architecture.

You then say computerizing work led to a loss of agency. Previously, new ideas took minutes to try with a roll of tape. Now they could take hours of complex configuration if they were possible at all. Why even bothering imagining new processes? The team developed a sense of learned helplessness. Software ought to be the ultimate medium for free expression. We are not bound by the laws of physics. In practice, the structures we have today for creating software too often get in the way. For skilled developers, they introduce mountains of incidental companies.

complexity, making it far harder than it should be to build great user experiences. And for end users without much programming expertise, using a computer usually boils down to prefabricated experiences created by developers without much hope for modification. And then finally, this is the dream of malleable software, editing software at the speed of thought. Obviously a bunch of the themes we were just talking about, but what you're implying here and clearly what you've spent so much of your work on is that As much as that vision you open the conversation with is amazing, like that's not really how software is.

It's that great irony. I guess my first question is like, why is it so critical that these mediums or formats or tools or whatever they end up being be extremely reactive and editable in that like speed of thought type of way? And then maybe more personally, how have you... edited your tools and your environments, whether it be digitally or physically? How has that kind of informed some of this? Yeah. So first of all, I want to say, yeah, you're quoting from an essay that I wrote with some collaborators, Josh Horwitz and Peter Van Hardenburg at the research lab, Inconswitch, where I work on this research vision.

And I just wanted to say this is all, a lot of it is our collective thinking. And we have a lot of collaborators at the lab, too, who've contributed to the work that's gone into this. So it's by no means, you know, a solo effort. That should be a good blanket statement on, I'm going to quote you a lot. Yeah, and that applies to, you know, the whole way I think about this is heavily influenced by a lot of other people. To answer your question, maybe I'll start with a story of modifying tools in the moment that actually comes from writing the essay that you're quoting from.

So I was writing this piece and we have an in-house writing tool, collaborative writing tool that we use. And it's designed to be malleable, ideally. And the piece felt really long. And we got some feedback. Hey, guys, this is way, way too long. It's not a short piece, or at least when I read it. And, you know, we want to go in depth and be rigorous, but also we want to respect people's time and really be concise. And so I was trying to figure out why is it too long? Like, where's the fat?

And I couldn't really tell from scrolling. It was just too long. So what I wanted was I realized I just wanted like an outline with a word count on each. section. So I could kind of feel it out. And I guess, you know, maybe I could have gone to the app store and tried to find some way of doing that. Or maybe there's some website where you can do that, but it's not something that's immediately obvious how to, to find, you know, but it was a need that I had. So in our system called patchwork, where I was writing this thing, what I was able to do is tab over to my AI assisted IDE and say, Hey, I want a view of the essay that I'm writing with an outline with workouts.

And You know, a couple minutes later, it's done. I run one command. And in our environment now, that is a new tool I have in my arsenal that's running on top of the existing essay that we were already working on. And I can share that tool with all of my collaborators and we can have conversations around it. And as the essay updates, the tool shows the latest word counts, right? And so I think to get to the first part of your question, the reason that it has to be in the moment is so often...

By the way, how long did that process take that you just described? I think it literally... It took about five minutes, is my guess. And importantly, there's two halves of that. One is AI code generation has unlocked tremendous velocity that wasn't possible before. But two, there's something that our environment has that I think a lot of AI coding tools out there don't have today, which is it's a story for composing smaller tools together into a larger workflow. So I could go into Cloud Artifacts and make a tool that can do an outline word count of a markdown document.

But then... How does that access my existing document? Do I keep copy-pasting it in every day? And what we want really is an environment where those new tools that we create so quickly actually have a place to live in an environment, almost an operating system, you can think of it as, where... it's sort of expected that you're going to be making new stuff constantly and integrating it together with stuff you already have. Which is how the real world works. That's totally how the real world works. You don't need an API to use your hammer in your work.

You can buy a new hammer or a new knife and bring it into your kitchen or your wood shop and it instantly starts working together with all the other tools you already have. You don't go build a new kitchen every time. And I think a lot of what we talk about in the essay is that the way that software is built today. sort of assumes that each company runs a kitchen where they make all the knives and they make all the spatulas. And there's no ability to go buy a different knife at the store and bring it in.

There's no ability to compose tools. And so that's sort of a goal that we're reaching for in this environment. And the reason it matters, so not every tool is possible to build in the moment. And it's by no means like a universal thing that has to be true. However, there's a lot of things that are good ideas that if it was going to take me a day, I don't have a day. to build that thing. I'm focusing on writing this essay. And so the, if I can- Less than even, between five minutes and a day is just like some amount of focus and like train of thought and friction and like all of this nuance that like you might be willing to go take five minutes, but an hour would break.

There's a critical threshold there where if you can stay in flow and change something, then it becomes worth doing. And that's, I think, a huge- Like a person in a workshop who might modify a tool. while they're building something absolutely and you know woodworkers i'm not a woodworker but i've talked to woodworkers about they call them jigs you can build a custom thing to help you with one project and yeah yeah i interviewed these this couple cw and t and they're like they're these industrial designers who just their their workshop is literally all basically they build tools that they sell but they mainly build tools for themselves that's so cool yeah and i mean Those custom personal tools, some of them might be easier or harder to build, but obviously there's a ceiling on like how much effort you're going to put into something that's not even, it's not a product that I'm shipping.

It's just a scrappy tool. Or by the way, you run the risk of becoming the optimizer. One of the things I was thinking about reading a lot of your stuff is the like classic Apple Notes bell curve meme. That's the downside. The downside case of being a tweaker type person is like you get over focused on your tool and you stop focusing on the work. Yes, 100%. And that's absolutely a trap for anyone who enjoys thinking about tools for thinking. You know, I guess in some sense, I'm now on a multi-year sidetrack of building not just tools for thought, but a platform for building tools.

It's bigger than just you, which I think makes it a little more tolerant. But yeah, I think you're... I like evaluating people's tooling based on their end output, not on how cool their tooling is. And so I'll take positive lessons from anyone who's shipping good things into the world, not from someone who has like the coolest Obsidian setup or whatever. Yes. we love obsidian yeah i mean we're i think we're both obsidian fans but uh there's there's a correct amount of investment i've spent a lot of time doing what you just described thinking i was going to make a quick tweak with even with chat gpt and then getting stuck and yeah yeah okay we're gonna we're gonna come back to a bunch of the things you just said but i want to stay high level for just a little bit longer one metaphor that you referenced a bunch of times like kind of continue to see coming up is alan k's like notion of popping open the hood which i think is really a beautiful metaphor it's also a little ironic given that i'm not sure most if they drive most people probably aren't popping open the hoods of their cars or they're driving a tesla and it's a front but the essence of it is beautiful there's another metaphor here which is like just exposing the seams of something conversely i suppose we live in a world that increasingly tells you you don't need to how understand how a system works you don't need to even see the internals understand the internal dynamics arguably that's what's happening to software right now why uh stefango who i've interviewed who i think you've you're loosely familiar with has a great essay on like don't delegate understanding that's playing with this a little bit but yeah what's your case maybe in the specific software case but more broadly like why why should we actually want to pop open the hood sometimes and understand how systems work this question is plaguing me right now so i'm glad you asked it you know um as you alluded to i think we're AI is kind of a tricky moment because I think it's making more things possible without developing deeper understanding.

And so this question feels urgent. I think the reason that resonates with me goes back to, so you talked about Alan Kay popping the hood. He wrote this great essay in the 80s about this idea of what would it feel like to pop up in the hood of software and what would you see? And the reason he wanted people to do that was that He says he wants people to be able to edit their tools the way they edit their documents. It's very directly this malleable software vision that we talk about.

And he says, the way you're going to do it, here's how you do it. You have people use the thing, but then surreptitiously, as they use it, they're accidentally learning how it works. And that's great because then the moment they think of something to do... that they want to work differently or if they just need to repair it or something, they've already developed that knowledge. And if you don't learn how it works as you use it, then the moment something is wrong, it's broken or doesn't work the way you want, you're starting from zero.

And I think that's a really lovely way to think about the value of understanding. I wrote a blog post about this metaphor called the nightmare bicycle, which touches on this. Yeah, it's this lovely idea from this book called Changing Minds by Andy DeSessa, which I think is one of the best books ever written about computational thinking and the value of understanding stuff. It's also, when I initially, you mentioned a few times, I haven't read it, but I did a little more digging. My initial impression was that it was a... software book it doesn't seem to be at all it seems to be a book largely about education and like a even in like in kids and sort of like first principles thinking and this kind of idea absolutely um i would say that it's a book that falls into some of the tradition that alan k started where programming isn't just about building apps programming is about developing your own mind um and i think decess's work falls in that tradition seymour papert and logo are another you know reference there of um the point of using the computer isn't to yeah it's you you being changed by it you're you're thinking being changed by it is the point you know but anyway to get back to this this nightmare bicycle idea he has this story where he says you know he had this nightmare about this this bicycle this hypothetical bicycle where the gear shifters are not numbers they are buttons that have modes on them it's like Gravel mode, uphill mode, downhill mode, like puddle mode, whatever.

And it comes with this manual that says, if you're going uphill, hit uphill or whatever. It's intuitive. It should be. It's intuitive. And he says, you know, you can imagine the product manager that came up with this because they would say something like, people can't understand numbers. We need to show them how to use it. And the thing he points out is this might be fine if you're only sticking within. the four buttons that are there. And then the moment you hit downhill on gravel, what do you do? Because those four buttons aren't teaching you anything about how it works under the hood.

And so you're not developing a more general framework, a kind of physics to think about what can this thing do and how do I make it do what I want? And the thing is, bike gears are actually not that hard to understand. Most people can figure it out just fine. And you don't even need math understanding, really. It's just... You get some experience with it and you start to feel how it works and you're good. It would be very hard to understand. My theory is, having learned it a long time ago, it would be very actually hard to understand looking at a piece of paper, but incredibly quick while riding a bike to tie it back to your...

Totally, yeah. And I think it's kind of a really damning critique of a dominant design philosophy in Silicon Valley products. Maybe it's actually better to expose more of this mechanism so that people can, again, surreptitiously develop that understanding of the, of the innards. And then that has all these secondary benefits. Yes. You give the example of the microwave that has the popcorn button as another example of this. Yes. Another nemesis of mine is the popcorn button. A little less bad, but quite frustrating. Are there, are there software examples that come to mind either on either end, either in the bad case or in the actually how bicycles are that are like intuitive in a really wonderful way that we maybe take for granted because we're used to them.

Great question. I think there are some lovely positive examples of software doing it well. Spreadsheets is one that I always go back to as one of the greatest inventions in software history. And there's a lot of different things that are great about spreadsheets. But one of them is that they, I guess, are made for accounting, kind of. And maybe that was their initial nexus. But we've all made spreadsheets for all sorts of stuff, right? You can plan. basically anything on a spreadsheet just by throwing together some tables. You can do financial stuff like budgeting, but you can do a roadmap.

There's all sorts of things you can do. And what's great about a spreadsheet is it doesn't try to force any particular use case. It just gives you kind of a bag of ingredients and you learn how cells work. You may learn how formulas work. You learn how to color things in. And with those few primitives, there's a lot of different... use cases that you can serve once you start building up that knowledge. And the other thing that's great about spreadsheets that's related is you don't need to be a super expert to get started.

Just like, you know, you can actually ride a bike without knowing how to change gears. And it might be kind of difficult at times, but you'll power through it. Spreadsheets, you don't even need to know much about formulas to get started. And so I think that's an example of a pattern that it's not just spreadsheets. There are other examples of software that has this kind of um combinable legos quality i think of like notion is a more recent example that i like um trello there's there's various products out there that have this this quality you have referenced this i think the the term originally comes from web straights but this idea of computational media which is at least adjacent or similar to malleable software if not very related i think to me at least the spreadsheet feels like the quintessential example i wanted to read that definition from the web straights page about it that i thought was pretty cool they say Computational media is software that blurs the distinction between documents and applications and between developer and user.

We defined four principles that modern computational media should adhere to. It should be one, malleable so the user can change and shape software to their personal needs. Two, shareable with others and enable collaboration. Three, distributable across a variety of heterogeneous devices and operating systems. And four, computable by supporting custom computation from within a document. I love that you brought up spreadsheets. You clearly obsessed with them. I think I found your PhD thesis and you literally open it by saying like, all of this is basically based on the idea that like, why can't making apps be as easy as making a spreadsheet or using a spreadsheet, which is pretty amazing.

The other thing I think that's cool in there is that it has that element of the bicycle, which is the learning curve is explicitly tied to using it. And it can be as usable by the most basic person as the most complex person. Are there other elements that you find yourself coming back to about spreadsheets that make it for the, like the template for computational media? And maybe as an additional question, like, should we think about malleable software and computational media as effectively pointing at the same thing or are there slight differences you would point out?

That's a great question. I'll start with the first one. So, oh man, there are so many things that make spreadsheets great. I love spreadsheets. So here's another one from the list that is really important that I think is also a quality of most good computational media is that they allow for informality. This is something that we're obsessed with that it can switch. We think a lot about how physical media like pen and paper are incredibly good at allowing informality. You can sketch anything. Yes. Even if you don't know quite where it fits or even if you...

Don't even know how to write it down in words. You can express it. And then you can figure out later how to clean it up and categorize it. Back to the other thing, by the way, no APIs. Yes. Which is ridiculous, but like real. You own your information. And digital media tend to be really, really good at formality. So a database of a million sales records is probably better to manage in software than in a filing cabinet. But they often are really bad at informality, at letting you... express things before they're ready to be categorized or captured in a formal way.

And I think this is really true of a lot of software, which forces you into premature formalism of how you think about something. So a concrete example is we've tried building a couple of new computational mediums for different use cases. Recipe management and travel planning are two of the big ones. And let's take a travel planning. situation right there are apps out there that will help you plan a trip and what that means is you can ingest all of your hotels and your flights and all that stuff and it shows you this pretty picture of what your trip is going to be and then it's like okay it's exactly that it's a pretty picture it's a pretty picture and it's like very nicely formatted and everything and the mock-up looked great right and then it's like okay now i want to jot down a few notes about what's going to happen this day oh and then i might be

doing this one of these two days. Oh, and here's a list of restaurants and ideas for things to do. And where do you put all that stuff? Often, you know, I'm thinking of apps like Kayak Trips or TripIt. There's others. They have a database schema. And if your thoughts don't fit into that schema for what they think a trip is, you're screwed. And so we developed, and this is me with collaborators, Paul Sonotelig and Alexander Obenauer, developed this computational medium called Embark, where we said, okay, let's start from something that looks more like an Apple node or an Obsidian node.

And you're just... writing stuff down about what you want to do and it's completely unstructured and then can you gradually start giving it a little bit of structure just as much as you need so maybe you'll organize it by day and maybe you'll separate out some things and maybe you'll you'll tell the computer you know oh this this hotel it's actually this google maps link and so then your computer can do stuff with that and it sort of goes from a very sketchy thing to gradually having the uh the computer be able to help you out more but it's very smooth and you never have to like hop into a specific rigid way of thinking you know and i think that's something spreadsheets are are really really good at it also causes the biggest problems with spreadsheets frankly you know you you messed up the formula or like the table didn't the range didn't get extended so there are there are problems with it but i think people often completely forget about those problems and try to or about the good parts and try to they try to solve the problems they say you know we can't run our business on spreadsheets we need 10 sas products to do it for us and they forget that actually the spreadsheet had these amazing benefits that you're losing you know yes yeah i think of um and i'll link to embark which which isn't there's there's no like that's not usable I dug around a little bit.

We've not publicly released it. There are reasons for that. And it's not intended as a commercial product, I should say. This was a research exploration for us where the goal isn't you can use it. The goal is we and you see the ideas and get inspired to sort of build a better computing fabric. In any case, I'll link to it. And there's a vibe in it of you're comparing it to like four different windows of the travel planning thing. We're sitting next to one of these boards where you can stick pins in.

And you can imagine layering on more and more contacts as you go in this very bottom-up way. And it really makes me think of the magical promise of computing is this magical substrate that you can imagine if you had a magic bulletin board that as you stuck to it, it updated. It would be really cool. And there's little threads of that, which I found pretty awesome. We're going to talk a lot about the very tactical. side of designing malleable software but before we get there one last kind of like high level thing or philosophical thing i should say which is a couple of metaphors you use that i find really helpful the first is around legos you say it's very valuable to have someone think through an entire unified product experience and make it all fit together in a coherent way rather than having to put together pieces so kind of a case for like the toy out of the box, but also, and this is from an essay where you're talking about foam, which was an early kind of obsidian like editor.

You say, I think this is a neat way of creating value in software. The essential value of foam isn't code. It's the opinionated curation of existing building blocks. And so it's like, if one end of the spectrum is the prebuilt toy that comes out of the box, the other end of the spectrum is just like, you have to build it entirely yourself. Lego is like a cool metaphor. You've talked about it in the context of like, you can build exactly the Lego set that They tell you to build with instructions, or you can go wild mode, play mode.

Is the construction element of Lego still too much friction, maybe? Great question. I think this is a really important point where I don't want to be misunderstood, which is that the vision of malleable software is not that everyone should build everything from scratch. That would be an enormous waste of time, I think. Other people are making great stuff. Often I have lots of apps that I use that are 95% perfect for me. I love them. I agree with the taste of the creator. However, I think the point of the Lego analogy is that when someone gives me a Lego set and I love 95% of it and I hate that that turret is in that position, it's no problem.

I just take the turret off and move it. And it's built out of pieces I already know how to use and I understand. And I can do that last mile tweaking. And eventually, you know, maybe if I keep... playing with this lego set for a year it might look pretty different at the end but it's not like i started from zero right i think that's that's kind of a philosophy that i find um really could be applied more to software granted i think it's worth noting and we'll probably talk more about this later lego is one company and one very explicit design language and there's a whole bunch for sure yeah there's a big tension there of uh they have an API standard that they adhere to, which is critical, right?

Yeah, if only it was the dictator who told us exactly. Yeah, and I think we think a lot about that problem ourselves of how do you create the, the Lego standard is very prescriptive and very precise, but it's also open-ended enough to enable a lot of stuff. And so how do you, what's the equivalent of that in software? I think also Lego, you can mix and match if there's a part of that set and part of that set that are both good. And you want to make, you know, a spaceship that has those wings and that cockpit, you can probably figure it out.

And the problem, one of the problems with apps is that, so there are certain app categories that I really care about that, let's take email, for example. I don't use email very much at work, but it is one of the big pains in my life. So I try to go to anywhere as possible to manage it reasonably. And I think a lot of people maybe feel similarly. And I have found so many email clients that are 80% of what I want. but they're each a different 80%. And I keep thinking to myself, you know, if I could just have the categorizing algorithm from that one and the inbox view from that one, that would be the perfect email client.

And I would even probably pay for both products, you know, just to have the ability to mishmash them like that. But the way we build software today, that is not a thing. Not even close. We are so far from having software being built in a way where that is possible, like Lego mix and match. What it means is that basically I need to go all in on one company doing everything right and just go with their strengths and their warts. And I have no ability to mix and match. Yeah, it's telling that the kind of quintessential strategy for many people in software today is literally just, oh, I only use Apple or I only use Google.

That's the extreme, extreme. Totally. Yeah. And again, back to the physical analogy, I think there are a lot of non-software places where we have more modularity than that. We talked about the kitchen. You can swap in a different knife and a different cutting board and different pots and pans. And it's a pretty modular thing. And there are levels to it. Like maybe it would be a lot harder to change out the handle of the knife separately from the blade. So there are kind of nesting layers of this stuff. But I think in software, we tend towards like these big monolithic things instead of starting from more modular pieces.

Yes. The second metaphor of the two I mentioned, also from that foam essay, you say, but foam is more like a recipe than a final dish. A recipe tells you a good combination of ingredients, and you get to make the food yourself, substituting ingredients to your liking along the way. And then one other quote, you say, yes, the restaurant food is truly better in many ways, and there's a role for restaurants and society, but I wouldn't want to live in a world where no one cooks. And food is something we can only choose off a menu.

Software is increasingly heading to that place. A few other kind of related ideas. I love this little essay you linked to about like an app can be a home cooked meal. You have an essay yourself. You say your pie doesn't need to be original. You're talking about like the notion of like every time you bake a pie, it doesn't have to be compared to every pie that's ever been baked, which is really beautiful and empowering and cool. And it ties a little to the Lego idea. If I were to be critical, I would say both Legos and home cooking are much closer.

much more on the play or leisure end of the spectrum than the utility end. They're not as serious, at least in most cases. And so I'd love for you to riff a little bit on why you think malleable software also applies on this more serious end of the spectrum for people doing real work, people coordinating, people working on teams. I think it's like, it's a little bit like the Apple Notes thing again, which is like, oh, serious people use Apple Notes instead of tinkering with their Obsidian or whatever. Yeah. You know, I think you're getting at an interesting aspect of malleability, which is that maybe there are two different reasons to do it.

One is because you get better, more productive tools or somehow you do better work sort of as a means to an end of an output. Which goes back to the importance of the lightweight flow, lightweight edits. Yeah, totally. Yeah. Like there I was. not trying to enjoy making tools. I was trying to make a good essay. And then there's a second thing, which is more about the inherent enjoyment or meaning we find from engaging with the world in a certain way. I take your point on cooking home-cooked meals. I might push back a little and say, actually, home cooking is pretty serious.

It's the main way that most people feed themselves. And what you feed yourself matters. And when you think about how are you going to eat in a healthy way? How are you going to... have your kids have memories of a certain culture um this is it's true it's personal um and it's totally true that there's also an inherent i enjoy cooking and i find meaning in the maybe you're even meal prepping at the beginning of the week on sunday yeah it's a great pushback you know but i think it's it's um it's also it is true for me that i would not be able to eat the food that i want if i couldn't cook at home i mean maybe if i like hired a personal private chef or something i could but No amount of restaurant and meal kit is going to suit my needs.

And I'm pretty particular about the food that I want to eat. I like food a lot. And so I do think even in these smaller scale, more local and personal use cases, sometimes it is the case that the ability to tailor it to me is one of the most important things. And no amount of like professional skill at a mass production factory far away. is ever going to make up for that kind of ability to make it you know i cook for me and my wife and i know what we both like and i can kind of adjust things and make it so that's something we're both going to love and again we've we've tried meal kits and it just fails to deliver on that totally it's a great pushback there are a couple threads there that i think we'll come back to both the notion that you know better for yourself than others and the notion that Maybe we actually, in the software metaphor, might soon have a professional private chef that can cook for you, which I think is interesting.

I want to go a little more grounded first. Feel free to push back as much as you want on this. But as I was reading through and kind of thinking about this, I'm sort of seeing there is like two core buckets for malleable software today. One is designed malleable software. So you could think about this as like a settings menu to a robust plugin architecture like Obsidian to... something like spreadsheets, like native computational media. The other is like hacked valuable software, which would be like, you write a ton about browser extensions, but I think another great example would be games modding, which is really cool.

It seems that you are particularly inclined to the hacking category of things. I mean, I think you give a lot of love to both. Maybe I find the hacking thing particularly interesting. And so my question is, one, would you... edit that push back on that framing at all and two perhaps almost philosophically do you think there's an opportunity to reclaim hacking in the broad sense particularly in a way that makes it more accessible because right now i think it's sort of like a very small subset of people can hack and then for the rest of the people that it goes back to the root of the philosophy of all of this which is that like should software actually be something that anyone can tinker with these are totally you're absolutely right they're different philosophies and I think they're both good, but they're really different from one another.

And we think a lot about the trade-offs. The best thing about the hacking approach, the modding, the browser extensions, is that you don't need to wait for the person who made the thing to give permission or to set up the interface to hook into. You just do it. And you don't need their permission. And often they can't even stop you. They might be able to make your life harder. Which you make an amazing point about. Adblock is a super simple version. Sure. Although I'll point out, I think Adblock gets a lot of attention among browser extensions and it is the most popular extension.

And there it's very adversarial. The websites like don't want you to do it and you're doing it. So that is a thing. And I think often it's not that the original creators of the app want to stop you. It's just that they're busy. So one analogy I think about in physical space is like imagine imagine if like every time you wanted to stick a post-it on the wall you had to call up the wall maker company and say like can you install like a hook for me to add a post note to the wall please you get you have another example of just like painting your wall yeah does the homeowners association approve of green if so we'll apply it to everyone if not no yeah exactly and it would be so ridiculous to block

tiny tweaks on the original creator, but we do that all the time in software. You can't change a single word. You can't change a single color. Totally. And I think what's great about hacks is that there are a way around that. And especially for tiny tweaks, you can often get away with just kind of like messing around from the outside without official support, as long as the platform is built in such a way that enables that. This is something we think a lot about is, you know, browser extensions work because the Chrome team did a good job.

designing a certain interface. It's actually both of those things at the same time. It's the upfront design decision that enables the hacking. Exactly. But it's an upfront design decision at a lower level. It's a foundational platform that thought about this so that they did that once. And then all the app developers don't have to think about it. And this is, I think, a really consistent pattern in systems that I like for malleability is... Assume app developers don't have time to think about this. They're not anti, they're not pro. They just, it's whatever.

Can you make it so that when they ship software, it naturally inherently is going to be moddable? And I think the web platform is probably the best platform out there in terms of this right now. It's much better than iOS, for example. And so I think that's really cool. There are problems too. Browser extensions have a lot of issues. They break security and privacy are like a complete nightmare. And that's a big challenge for. Yeah, it's telling there are very few browser extensions. Almost all the best ones are incredibly lightweight.

You give this amazing example of so much of what you're talking about in the like the Bank of America, like checkbox extension. But it's like stuff like that. It's not robust apps. And I think that's kind of illustrating the point you're making. Yeah, there's a depth that there's a there are limits to what you can reasonably do hacking around from the outside. But again, like. Back to the earlier point, often it's 95% good. And what I need changed sometimes isn't that big. It's just tweaking around the edges. And for that, I think hackable software platforms are great.

Something we think about a lot at Ink and Switch is how can we do better than browser extensions? So what would it look like to create a platform that's even more malleable than the browser? And I think a couple aspects we think about are one, AI exists now. So everyone can code. to some approximation, and it's going to get more and more true. How would you design an operating system that people are using from the ground up, assuming that everyone's going to be constantly writing code, which is not the assumption that we started from in today's operating systems.

And second, at Ink and Switch, we've done a lot of work around this thing we call local first, which is a philosophy around owning your data. It relates to file, not app, which Steph was talking about on your podcast. And that's another piece that for us fits in because One of the big problems with browser extensions is in cloud software, you don't own the data or the software. It's like running off on some server. And actually, most things I want to change in websites can't be done in browser extensions because of that fundamental architectural choice where all of Google Docs is running on a server.

I don't own it in any meaningful way. And so we think about in our work, can you give people not only ownership over... their data but also ownership over the software in a deep way so that this hacking approach could be applied to any part of your software and you can change you know anything yes yeah many ways a lot of this in practice is about making more stuff hackable and making hacking more lower friction more possible exactly hmm now a quick break from the episode to talk about something for dialectic listeners from hampton Hampton is a membership for founders and entrepreneurs that provides both in-person and online community for support, advice, and accountability in the founder journey.

One critical part of being an entrepreneur is managing your own financial life. I think unfortunately for most of us, talking about finances, even with close friends, is a little bit taboo, let alone getting a more holistic view from a wider set of peers or people. So fortunately, Hampton surveyed over 100 of their members with net worths from a million to over $100 million and asked them a wide range of questions about their money, how they spend it, how they're doing with regard to their financial goals, what their quote-unquote number is, how close they are to it, the timeline to get there.

Tactical stuff like risk tolerance, a breakdown across their investment portfolios, how much they're in cash relative to the year before, how much they pay themselves, how much they're burning monthly at each of those wealth ranges, even stuff like estate planning and philanthropy. It's a fascinating data set that Hampton's added editorial and commentary over the top on too. And it takes a topic that is typically opaque or at the very least, you'd only have a small handful of data points on and gives a much more transparent look at it. I found it really valuable to look through.

And if you're interested, I've linked Hampton's 2024 wealth report in the description. You can also just go to com, click reports in the menu and fill out some basic information to get the report. Thanks again to Hampton for supporting Dialectic. Now back to the episode. I want to talk more about, to me, you are clearly a design oriented person. And so much of this is about design, whether it be at the high level system type stuff we were just talking about or in the Berry Micro. We already talked about the Nightmare Bicycle, which was one of my favorites.

At root, so much of this thinking, the ideological and philosophical underpinning of this is basically the fundamental belief that people understand their own needs well and are competent, can learn, and should be trusted. That's the kind of like simple explanation of the nightmare bicycle, right? It's like, in some ways, it's sort of like... being maternalistic. It's saying like, I don't trust people to be able to do stuff. Another idea you explore here that I think ties into this is this idea of a smooth slope or a gentle slope. And in many ways, like the funny thing with the bicycle analogy is like, this is the great benefit of software is that it is adaptive.

It isn't like you make one decision once on the handlebars and like, that's how it is. The gears are the way they are. To me, there are sort of, there's almost like two slopes. So in this kind of design space, one is a slope of agency or learning, which is like the bicycle or really any high skill floor problem. And then the other is a slope of malleability, which is like how malleable should it be? Obviously, those two things kind of like run into each other, I suppose. But I'd be curious for you to think about like, is there any world where you might actually like you want these thoughtful, gentle slopes?

But you also might like over design the slope of malleability such that it develops some of the elements of the nightmare bicycle. Does that make any sense? Yeah, I totally see what you're getting at. So let me start by telling people a bit about this idea of the gentle slope and what it means. So we talk about this in our essay on malleable software, but there's this lovely paper by Alan McLean and some co-authors where they introduced this idea of the gentle slope. And the idea is... You want to go deeper and deeper into customizing some system and often in software At some point you kind of hit a wall where it's basically like past this point.

Good luck You're gonna have to learn a program or something and they call that a cliff because you kind of Have to instantly learn a ton more stuff to make one more step for make this point with the browser extension It's either like you can use a browser extension or you can learn a crazy amount to be able to make one exactly Making browser extensions is not easy even for programmers. It's like leave the browser, open some API docs, like write a bunch of code, like distribute it through an extension store.

There's all these barriers. And the idea of the gentle slope is if you want to do something that's a little more complex or more involved, you should only need to learn a little bit more stuff or go a little bit deeper. And we want to minimize the amount of upfront cliff learning that you have to do to get further. I think about this in the analogy of physical space. So obviously, There's a big difference between moving a post-it note, moving a desk, and moving a wall, right? Those are not the same thing.

But I do think physical space often has a reasonably gentle slope in the sense that to move a desk, I don't need to hire a general contractor. I can ask my friend to pick it up with me and move it. And as I get deeper into crafting this space, there are going to be skills I have to learn and people I have to cooperate with. There's kind of this like linear relationship and software. I think the way that this analogy applies is like it's as if you had to hire a construction crew to move a post-it note or something.

Right. It's this very incommensurate like amount of effort invested versus like the thing you were trying to do. Yes. Yeah. Yeah. The only solution like we only have one way of moving things and it's a it's a moving team. Exactly. You can't move with your hand. It's a it's like a it's a backhoe or something. And so. In the browser extension example, we think a lot about, okay, what are other things you could do to smooth that slope? So if I can make a theme and change a color in the browser, then I don't need to learn how to make a browser extension if that's all I wanted to do.

Yeah, the browser company explored some productizing some more. I think that's a perfect example of take some part of what people want to do and make it easier to do and establish that linear curve. And it doesn't mean things aren't ever hard. And it doesn't mean you can magically do everything easily or that you can do everything yourself and that you never have to work with anyone. All it means is that easy, simple stuff shouldn't be ridiculously complicated. Well, and critically, too, there's a broad idea that I'm a huge believer in that I think I originally sourced from this guy,

T. Nguyen, who writes about games and agency. And it's like what a good game designer does is they make the ability and goal attention like consistently in the right delta. Like it's never too out of whack. Totally. And I love thinking about game design as a inspiration for this stuff i'm not a game designer and i sort of almost wish i had that that skill set because i think those people are often doing some of the coolest work in terms of like what i guess we call onboarding yes um but it's all agency that's it yes it's um they think a lot about how do you organically develop people's skills by staging the learning process into small bites appropriately and disclosing it naturally in the course of using the thing that's where you know I might push back a bit on your idea of those two slopes of learning and malleability.

I think they're very related and they might be the same slope. While you use the thing, you should learn more about how to use it. And part of using it is changing it. That's just a natural progression. It's a little paradoxical and it's not really how a lot of things are in the real world. It's this sort of unique part of software. And it's an ideal frame of software that isn't always the case, which is, I think, what maybe is causing it to feel. It's definitely not always the case. But I think it's more like a person, like as as we like a person is the classic trouble, like we think about ourselves as always in stasis and as other people as static.

It's like, no, we're both reflexively changing all the time. And software is, as you earlier said in the conversation, like it's it's a relationally dynamic medium, which is really important. Yeah. And. You know, earlier you talked about computational media and blurring the line between document and application. And I think that's relevant here because in some sense, the problem is that we think of using and changing as two things that are separate. But with a document, like, are you using it or changing it? I don't know. It's a material and you're doing stuff to it.

It's just one process. Great. And I think software ultimately... would be better in a lot of cases if it felt more like that. So you don't enter a separate mode to change the software. It's just a natural part of using it. When you think far future, imagine hypothetically we have an AI that can, in a snap of your fingers, tweak any part of your software instantaneously, literally at the speed of thought. At that point, what does it even mean to use a piece of software? might ask if a feature exists, and if it doesn't exist, it will materialize instantly on the fly, you know?

And so I think we could see a lot more interesting blurrings happening as we figure out the cogen stuff and we figure out what does better foundations look like for this. You know, what's funny is like, this is how TikTok works. Am I training the algorithm or am I watching? Totally, yeah. You wouldn't think about it that way, but it's very... I don't know how I feel about that comparison. It makes me nervous, but yeah. I think those guys are masters at... that notion though of like incrementally moving you up a curve or taking you somewhere it's it's not even fully conscious and again maybe in a dark pattern yeah and and you could say the same thing about match three games in some sense i think games are a a healthier example to learn from hopefully i guess what i'm pointing out is there's a notion that like this guy gabe laden who was a game guy and he he talks about this notion that like The only game in the future might be like a game where you open it, and depending on how you play level one, you end up in Match 3 or League of Legends.

Yeah, yeah, yeah. And that's inside of, I think, what you're starting to point out. Totally. Okay. You referenced it briefly earlier, but one of the core parts of this most recent essay is this notion of tools over apps. You say decomposing applications into composable tools. And there's a broad, it's not exactly the same idea, but there's a broad notion, I think, today that software is, extremely like container first or container oriented, which you might even describe as sort of like the combination of the tool and the environment like bundled together.

You use a lot of these metaphors that think about allowing tools to be open and dynamic and free and move across different environments. The classic metaphor in the piece being this idea of a chef's knife that you can carry around the kitchen. There's one. excerpt where you say a fluid workflow where all your tools materials are always at hand is impossible in the world of big box applications. It's as though you have to carry ingredients from one end of the kitchen to another every time you want to use a tool.

The other funny metaphor you use is the like the avocado avocado slicer, which you say like apps are avocado slicers, which is this ridiculous tool that I didn't even know existed that has like three parts that are all good at kind of slicing an avocado. And your critique on the piece is like You just learn to use a chef's knife. Like you will be so much better for it rather than buying this stupid piece of plastic. There's some of this too. And you brought up in Bark earlier, a few of these ideas you built, which is like creating sort of like little spaces for a set of tools to exist inside and versus the nightmare of Google Maps and kayak and flight.

And I was doing this this week. I'm trying to book some travel from Europe. It's like, does the flight exist? that allow you to edit the medium while editing content, which I think is really powerful. Just an excerpt on those two that I thought was cool. You say, both Potluck and Embark provide a gentle slope. They start out as user-edible content and layer on interactive behavior optionally and gradually. They also support tools, not apps, using documents as a substrate for composing tools. You can gather all the relevant information for a project in one document, and then all the tools in that document can easily access the context, and then you can visually compose tools within the document.

And so I guess... So my sort of interpretation of this is almost as though you're really talking, you're certainly oriented around tools, but it's actually about two ideas. It's about creating substrates or environments and flexible tools. And like those two things are both pretty needed if the earlier thing is like the app is the integration of those two in a way that's bundled. Is that the right way to think about it? Like, can they be thought about independently or do they, I have a bunch more questions, but do they need to be thought about kind of in that?

combined way it's both the kitchen and the chef's knife yeah i think that's a reasonable way to split it up and in some ways designing the substrate or the environment might be the harder part and you know the reason i think the reason we don't have this isn't just because someone was evil and didn't want it. It's because it's really hard to do well in software. And it has a lot, it comes back a lot to that stuff about formality we were talking about. A lot of the difficulties is when you represent stuff in a really formal way that's really particular to your way of thinking about it, then it's no longer compatible with the way someone else is thinking about it.

And that's very different from physical reality. Like an apple is just a thing. Yeah, there is already a defined substrate. Totally, yeah. That somebody's decided on. It's a substrate someone who knows decided on. And we all really understand it well because we were, our brains are wired to know what's possible in a substrate. And that's a, I think, a really high bar that we might not quite be able to hit in software, but it's worth treating that as a north star. But we take it for granted that we're like the extreme opposite end of the spectrum.

Totally. It's just there's no shared substrates at all. You're living on little islands. There's no, yeah. And so, you know, some of the sub problems that I think. There's been lots of really interesting ideas in computer science about how to solve these challenges, but I don't think we've fully figured out the answers yet. One, how do you represent information in a way that different tools can work with it, even if they have different opinions about how to think about that information? We did a project that I can switch called Cambria, where the idea was sort of letting different views of information coexist at the same time so that different apps could kind of see it in a different way.

lens while having some shared underlying thing that they can both use. That's one problem. This is basically just effectively another way to do what Stefan and the Obsidian guys or the local first folks are doing with files. Absolutely. Yes. How do you, what's the file format? Yeah. And Obsidian is a great example where they've picked Markdown mostly as their, as their bet. And there's a lot of really good things about that, especially the fact that it's already in use, but that also prevents them from. doing certain things right yeah by the way json has a bunch of benefits totally yes every one of these formats the the big dimension that i think is is the hardest to get right is how much structure and specificity in domain specificity do you want to encode in the information so json is great it's very general can represent a lot of things but it doesn't necessarily get you that much just saying it's json is like okay what what are the fields in it you know i could have a microsoft word document that's another file format It's now open.

It's very, very, very complicated because it's accumulated decades of the Microsoft Word feature set, you know. And so I think getting the level of representation right is really tricky. I'm optimistic that AI will help here because one of the things that AI is really good at is translating information across formats. Yeah, you add in MCPs, whatever. Exactly. And also it's really good at interpreting less structured information. And so I think it's possible that maybe the shared substrate will be less structured. It could be a folder of PDFs that all of our apps use because that's how the world worked before apps is like it was all pieces of paper that we have OCR.

We have OCR in our heads and it's, you know, like that's how it worked. And so maybe maybe that works for for for tool composition. I don't know. Anyway. Data representation is one challenge. Also, just like, how do you make user interfaces that feel good when they're made up of different parts? So if someone good makes an app that's nicely designed, then it's very cohesive, right? It all fits together. Yeah, Notion by default is way snappier than Obsidian. Totally. Yeah, they're very opinionated about... their blocks and you know i'll note like they they let you embed third-party stuff but when you do it feels really weird like if i embed a tl draw whiteboard in an ocean it's really obvious that it came from a different planet yes and i think some amount of that is okay and inevitable a sort of mishmash scrappy aesthetic i think is fine and maybe something we should try to get more used to it reminds me of like

a house where someone actually collected stuff over 20 years versus a house that like some developer like designed and you can well you know i was thinking of is like um i went to florence for the first time last summer florence is notion like it was there are basically two design periods it has the exact same design language and like it's really cool but the city is like trapped in the 1500s or at least the 1900s and new york has problems but it's pretty cool too Another influence that reminds me of is, what about old European villages?

Often those, so this connects to Christopher Alexander's work. I think there's a way that things can be beautiful that's not top-down. And Alexander talks a lot about this idea of patterns that can compose so that there's coherence. It's not random, but also it's not all uniform and factory produced. I think that's a really powerful idea, not just at the aesthetic layer. It also goes deeper than that. You can get a ton of coherence out of everyone build your own house, but you know, here's the local vernacular. This is what a roof should be made out of.

And these are the colors that we like to use because that's what we use. And there can be a lot of creative freedom inside those. Exactly. Exactly. And it's it's not quite what people call a design system, I think, which tend to be much more prescriptive and top down. It's closer to an open ended set of patterns that people intuit how to apply. We think a lot about, a crude way of saying it is, can you make the right 50 built-in UI components so everything has the same look and feel?

That's actually sort of what macOS and iOS do. They give you 50 components and like a 300-page book telling you what feels good and how things work in this world. In some sense, there's a lot to critique about Apple, I would argue, especially in the platform level, but this is one of the things they're best at, is doing this in a way that both seems to be empowering to developers Somewhat cohesive. Absolutely. I'm a huge fan of Apple products, which I agree with you. There's a lot of critique in the way that I don't think that they value malleability very much.

I don't think they value developers very much. That is also true. That's a whole separate conversation. They definitely value a cohesive user experience. And I think I don't like bad user experiences. I don't love Linux on the desktop because it doesn't feel good to use. I think there is an inherent tension between skilled professional designers crafting everything really well and people doing stuff on their own. And the synthesis might be this kind of pattern language. Yeah. On the note of the environment and substrates, any thoughts on how this is going to evolve in the sense of like, I think you spent a little time at and written about dynamic land as one kind of extreme example, Brett Victor.

How substrates is the future? Maybe an additional question to tack onto that would be that one thought I continue to have as you make all these wonderful metaphors is that software or bits broadly basically lack dimensionality, at least until we get to VR. And so some of these like metaphors around the ergonomics and the malleability and spatial feel and even like the post-it note, like they're good, but like they aren't totally exactly apt. So, yeah, I mean, it's a very open-ended question and maybe just totally postulating, but any, either stuff like dynamic land that you've seen or other examples or just kind of speculation on how maybe we actually might get more freedom in the environment substrate space as HCI changes?

So dynamic land, for those who don't know, it's basically, it's sort of a room scale computing experience. And the way it feels is that you sit at a table with people and you think about stuff. And you have magical materials on the table that come alive and do more than paper can do. That's sort of my bulletin board from earlier. Exactly. It's the magic bulletin board. You know, things start animating. You can run simulations. That's their vision. It literally just, I think it's like projectors on the ceiling. It is, although Brett.

So first of all, you know, Brett's a huge influence on my work. And actually one of my co-authors on the Malleable Software essay, Josh Horowitz, who I've also collaborated with, worked at Dynamic Land and built a lot of their stuff. It is projector based, but that's not the point, I think they would say. No, I'm just trying to fill in like people can pick it up a video, but like what you just described sounds a little fantastic. They would be. I think what that team wants people to focus on is it's a prototype of how the thing should feel.

So it doesn't matter how they built it. What matters is. Yeah, you have things coming alive on the table. Maybe things are even moving. Yes. Automatically. Yeah, I guess what I'm trying to clarify is you're not wearing glasses, there's no VR. No VR, yes. It's in the real world. And I find that really... With pieces of paper. So I've visited a few times and I was not involved at all in it, but I got to feel what it was like to be there. And the thing I found most compelling and awesome about it was you can sit around a table and make eye contact and I can see what you're looking at and I can point at stuff.

So cool. When we want to... change the color of something, we just can like scribble on it with a pen. They have some cool examples of leaning into physics. Like if you want to make a spinner, just like make a spinner, you know, use a thumbtack and a piece of paper and that's your spinner, right? So it's this lovely kind of starting from physicality. It's combining the two, the tension we've been talking about this whole time, like this. So much of your work is trying to make aspects of the digital world more like the physical world.

And they're just like, let's literally combine them. Exactly. And I find that I think it's a really bold and inspiring approach. It's not what I'm doing, partially because I work remotely, which started doesn't work with their, you know, around the literal table vision. Yeah. But I think I do think a lot about. I think it's fairly like I think I read somewhere that Brett said he hasn't programmed on a real computer in a long time. Like he only programs in dynamic land. I mean, that's pretty cool, right? Another thing that I find really inspiring about their research approach is they really have tried to live in the thing that they were making.

And that's a legacy that I think they drew inspiration from computing pioneers like Doug Engelbart, who I think they thought that the way you build a good thing is you try it out yourself for years and you really go deep on living in that space. And that's something we try to do at our lab, too. Very Apple-esque. I mean, that's where Brett came from. For sure. It's prototype-based and experience-based. It's not theoretical. And we try to do that at our lab. We call it living in the future. Sometimes it feels futuristic.

Sometimes it feels like we took a step backwards by accident. You know, but that's sort of all part of it is being accountable to make something that really works for you is, I think, a really important part of a design process. So I find that inspiring. Cool. We talked a lot about the sort of, to extend the metaphor of the kitchen. On the note of the knife and tools, maybe a simple or too basic question, but like, what is the difference in this context between a feature and a tool? Like in Embark, a lot of those things you're describing, like you talk about them as all, there's all these tools that live in this spreadsheet.

And granted, I would argue. probably that ai is only going to collapse feature and tool in theory but like is there is it worth distinguishing between those two or or maybe more broadly like how do you think about what the object of what a tool is in this idea assuming that you have different substrates and environments one way i think about that is maybe there's not a hard boundary but there's a spectrum from something more like a feature more like a tool and i would say an avocado slicer is more on the feature end and a knife is more on the tool end and what i mean by that is You know, the press release for the avocado slicer that they wrote before they designed it said, we have this person in mind.

They're like a 32 year old product manager at Google and they need they're hosting a dinner party and they need to slice 10 avocados. We're going to make a perfect thing that does avocado slicing. And what are the stages of avocado slicing? And, you know, how can make it so easy and so intuitive? They don't need to learn anything to use it. And it's very driven by like a. a problem in a specific problem a specific scenario with zero attempt to generalize further um and then a knife on the other hand is very different i think it's more like a it's a it's a mechanism it's like a it's an object for cutting stuff you know what the you know what the uh the avocado thing is is more like an appliance yes appliance i think is a is also a slippery word but i agree that appliances are often closer to the like, do one thing.

It's like, do the job for me so that I don't have, it's like a toaster. Yes. I mean, in this case, not literally, you still have to do, but you could imagine an avocado slicer that is like a toaster, which is like, put it in the box and it slices. Yeah, exactly. And I think when I, I think one of the qualities of a good tool is that you can do different things with it and you can keep getting better at it and learn to use it. And there are good examples in software of this.

You know, we've talked about some of them. I think parts of spreadsheets and parts of Notion have this quality of you learn this primitive and can apply it in a lot of different ways. Yeah, programming language, even if you wanted to. Absolutely. Once you know a good programming language, you can do, programming languages are, you know, an advanced Lego kit, I think. So totally. Yeah. There's one little bit where you mentioned one kind of non-obvious benefit of substrate or environmental interoperability, or at least the notion of being able to bring your own client to something is being able to act, it's not actually like the notion of I'm going to try every notes app.

It's that I've used the same thing forever. Like I've used the same coded editor forever, which I think is very cool. I'm curious if you have any personal examples of that and maybe more broadly, like what, what do you, we were talking about this a little bit, what do you makes a great tool? What makes it, what's like a chef knife of software or even just one that like you have gotten to go deep on and gotten to enjoy. This is an aspect that I've come to appreciate more over time as I rack up years.

I think I used to be really interested in what's the coolest new best tool that I could pick up right now and try them all. The mid-curve. Yeah, exactly. Yep, I've been through the mid-curve. Maybe I've made it to the other side. I think something I really value now is thinking on decades scale in terms of longevity, like what are tools that I plan to use my whole life. what's going to be worth investing in that's actually going to last that long. And we don't think about, sorry to interrupt you, we generally, this is inside of all this, we don't think about digital stuff in this way.

And only now that we've, people around our age have been using it for a little longer, but when you started, it was all new. The internet was new. Exactly. And now I'm thinking about... Where do my notes live so that I can use them when I'm 50? Yes. Where do my photos? All the photos I've ever taken in my entire life are in this photo reel. And I think it's backed up somewhere. And if Apple keeps being good, it should all be fine. But wait, is Apple going to be good in 20 years?

I don't know. And a lot of the local first file over app philosophy, I think, is applying this to your data. But I also think it applies to software. There's a paradox in malleable software that I find really funny. There's a study that Wendy McKay, who's an HCI researcher, did at MIT back in the 90s where she asked people, why do you change your software? And one of the top reasons was because they shipped an update and I didn't like it and I just wanted to keep it the way that it was before, which is hilarious.

And we've all been through this, right? And I think if you come from a product mindset, you tend to think of those users as bad. They're the ones who want to stay in the past and they don't want to learn the cool new stuff that we're shipping. Why don't you want to use our cool new stuff, man? But there's another way of looking at it, which is if I own my environment and I like it the way it is, you don't get to come in and change it. Part of owning a house is if you like the way the house is, you get to live there forever, right?

And there are affordances, by the way, that come from having used this knife for five years or lived in this house for five years. Exactly. And sometimes software feels like I've been using this knife to cut onions for five years. And then one day I show up to cook dinner and it's like, hey, we've made a new knife. It's way better. Try it out now. And by the way, we threw away the old knife. It's not there anymore. And it's been deprecated. And I'm like, oh, no, this is terrible timing.

You could have been more polite, but also like I didn't need the new one anyway. And so, yeah, I think that's. And we use these tools all day. You notice a gram of weight in a knife. Exactly. You cannot, it is not worth investing in learning the thing if it's not going to stay usable and stay predictable. And so I think it's sort of an ironic thing that part of ownership is keeping things the same as well. Yes. We texted about this a little bit beforehand, but one idea that ties into this tool and environment framework briefly alluded to earlier earlier is is the notion of a container around code at all and and that sort of being what software or apps are today i'm hesitant to make this only about ai but it feels like the dominant kind of frame around it which is we now have a world where you can ask a really smart choose whatever metaphor you want butler assistant appliance

to just do the thing for you capture intent and like i don't need to know how it happens i don't need tools i don't need apps i don't need software you have this and this gets to maybe the tension here but you have this line in one of your essays where you say chat will never feel like driving a car no matter how good the bot is which is great and we were texting you're like yeah but also waymo like it's like we're almost moving from like we used to drive and then we had cruise control now you just have waymo actually still have tools.

And like, how do you think about where we're going as like the containers around this stuff start to start to dissolve? I think for what it's worth, one of the core things you're driving at in the tool framework at all tools over apps is you're sort of pointing at this, but like, in a world where you can edit code at the speed of thought, you don't actually need to write programming, like you don't need to program yourself, you just ask the butler to do things for you. I have some of my own ideas, I think you're getting at some of them with the the chat and driving a car idea.

You have another example where you say like, what if ChatGPT produced a spreadsheet for you? But yeah, I'd love, I'd love your thoughts on like where containers and tools should show up beyond just intent. This is something I'm grappling with right now. And it's, I want to try to unpack it with you. I think there is a bit of a trap that I'm trying not to fall into, which is that I have a lot of creative friends who love direct contact with the work. I think I'm sort of that way too.

A simple way of putting it is, I like writing words. It feels good to do that activity, right? Independent of the output. Independent of the output, exactly. And that is a separate question from, do I actually need to be involved in the details to get the best output in this context? And I think unbundling those two things is like a vital first step to trying to figure out what's going on here, because from the perspective of providing economic value to the world, No one cares what I feel about doing the work.

It's about, is the output good? Yes. So to a first order, there might be something you can say, which is like, look, if the bot gets good enough, you can keep doing the thing as a hobby, but you don't need to do it anymore because I can just do it for you. So I think that's one possible stance. However, I do think that when you're trying to do certain kinds of creative work, being involved in the details is the only way to get to the unique output that would have come from you.

And there's just no shortcut. I would certainly claim that with writing today. But again, I could still be underwriting. So I think it's obvious with writing. Writing is the clearest case because writing is thinking. Coding is harder for me. A lot of, to take myself as an example, a lot of the work I do is prototyping user interfaces. much of the code that I write when prototyping user interface, not only am I going to throw away the code probably, the code doesn't really matter. What matters is how it feels to use.

And so in that case, I'm finding a lot of use for AI in helping me do that task. I'm moving faster and I feel that I'm able to prototype and get to the interesting parts faster. However, I do sometimes wonder, oh man, am I accidentally losing opportunities to notice things? In the process of AI is making a lot of micro decisions that I might have been more involved with before. And maybe if I had been involved in that decision, I would have had a spark of an idea that I'm not having now.

And so I try to monitor that and think about, you know, what's the quality of the time I'm spending? And one way I try to think about it is if I'm doing serious work, I don't think there's a way to reduce the amount of time that I'm spending on it. That's not my goal. It's more like... Can I ramp up the quality of the time I'm spending so that every minute is like focused on the important bits? And, you know, maybe that's a way to get better results. Yeah. Yeah. There's an essay I read a while ago called Reality Has a Surprising Amount of Detail.

I love that essay. It's great. And it's inside. I don't know. Me editing this. I could have AI transcribe it. I do do that. You know, the AI do everything. And yet. I'm sitting with the same tension though as you, which is like, I don't know if it's just fear. I've actually, I used to work with lots of Twitch streamers and YouTubers and stuff, and they would always have this like, oh, I can't let somebody else edit the videos. Only I can do it. And what they all eventually found is that like, no, if you actually take the time and train a person, you have more time.

But yeah, I think the question is, maybe we'll come back to, but the question is like, where do you want to? You don't get to spend a lot of your time on much, but where do you actually want to spend a lot of your time? Yeah, and I think one question that I've been trying to overlay on this is when is it worth it to get in the details and when is it possible to delegate? Editing a YouTube video is a great example of a tricky case. I think the more variability there is in the output and the more subjective and taste-driven it is, the more important it is to be involved in the details, I think.

And at the far extreme of this is something like art, maybe, which I'm not an artist, but... I basically think humans being involved in the details of art is inherently part of the artistic process. And maybe on the other far extreme, let's go back to that Waymo example that I was, you know, lobbing at you, right? Like often I just want to get somewhere and there's not any variability in the output. I know where I want to go. Driving the car, I'm not going to change my mind about where I'm going.

So if I have this clear objective, it's verifiable and it's optimizable along like However you can get me there cheaply and fast, I will take and safely. I'll do like literally, it doesn't matter to me. It's a black box. And I think in those cases, it's appropriate to delegate because it's a closed-ended problem. I think about, you know, this I think is sort of a roadmap for where I think AI coding is going to go, which is basically any subtask that has a very clear criteria for what success looks like is going to be automated.

They're going to... trained models. Yeah, like no knowns, basically. Exactly. And this surface of specifiability, I think, doesn't always match what we intuitively think of as difficult necessarily. So I might have a goal that's like, make this thing 100 times faster and make it 100 times cheaper to manufacture and make it 100 times more robust. Or I could have a more insidious goal, like make the conversion rate 10x. These are clear goals that can be measured. And I think AIs will eventually figure out how to just like optimize any metric.

It's a math problem. It's a math problem. And it might be really difficult by human standards to do some of the tasks, but we'll figure it out. And so the tasks that are left for me to work on are the ones where we don't know what the answer is. Yeah. One last note on the container idea, which is like one thing that I think you rightly got at, which is like, what am I? where do i want to go through the work and the complexity and use tools for my own purpose but on the other end like i'm curious what you think of is just broadly going to happen to like software people use like presumably people will talk about this world like you're just going to have a master agent that like like why do you even need user interfaces why are you still interested in designing user interfaces are you just designing for like hobbyists who are doing this for fun like i don't think so and so i'm curious Obviously it's like a really broad idea, but like to what extent do you, what types of use cases and applications, like maybe a simpler way to ask this might be like, where do ergonomics still matter?

Where are the cases where like chat isn't as good as like, it's not a steering wheel. It's never going to feel the same. Yeah. I sort of regret the steering wheel analogy as a choice a bit, because I think, again, maybe in most cases it doesn't matter that much where you steer. Maybe a better example is, Chat will never feel like holding a paintbrush. I think there's, we can sort of map this distinction we're making about things where the output is known versus unknown to like input output modalities in an HCI sense of like what modalities are going to be best for the kinds of interaction we want to have.

I think if you have a fuzzy intent and you want to say it in natural language and have something go do it, go do something for you, maybe natural language is a fine. is a perfectly fine way of, you know, voice or chat is a good way of specifying that. I do think often on the computer to human side, text is not sufficient and we really want more visual stuff like showing on-demand interface. I was just travel planning on ChatGPT and it's like, it's actually not that. A wall of text is not a good UI.

People know that though and I think that will... We'll have generative view. One other thought here that you get at with the like, what if ChatGPT produced a spreadsheet is like, chat isn't really a computational media, but... presumably we're going to start getting chat gbt is going to start generating me uis that i can play with on the fly or whatever totally yeah and there's there's layers there of um maybe we'll maybe we'll be making new computational media within chats and stuff but yeah i think direct manipulation by which i mean like clicking on stuff and moving things this metaphor of like the screen has objects on it that you can think about it engages a different part of our brain precision is way more involved you can point to stuff and there are big advantages there

I'm really excited about hybrid modalities. Like an idea that I'm obsessed with is hybrid voice and pointing. So I want a Minority Report style, you know, use my fingers and my voice simultaneously and say, you know, there's a famous HCI paper from decades ago that's called Put That There, where it's this really old demo, but this guy's just pointing at stuff and saying, put that there. And it moves it there. And it's a perfect demonstration of like the power of this hybrid modality. And I think that's going to be really cool.

There's all these different... mashups, but I think you can basically break the problem down into like what are the pros and cons of each input and output method and what tasks are appropriate for them and go from there. I do think, I will say, I hope we get a new operating system at some point. I mean, using the eye tracker on the Vision Pro was pretty magical. It felt a little bit like what you were just describing or like the dynamic land type of stuff. And I don't know. Yeah, maybe it will.

Maybe it will be VR, AR. I'm not sure. I think AI might disrupt. our way into a new operating system emerging. I really believe that the big deal isn't going to be a new IDE for developers. The big deal is going to be a new environment for users to live in. And so we'll see. Yeah, on that last note too of Precision, you have a line where you say, any creative endeavor where the person is reaching for greatness benefits from Percephes' control. And then as thesis or Linus, someone we both know says great creative tools allow for virtuosity, which I think is the, maybe that's in the creative sense to your point, but like, there's still a lot of room for that.

One other bit about design that I found incredibly just interesting and surprising. And the more time I spent on it, like resonant that you pointed out briefly is about empathy. And I suppose this is, probably pretty different than most designers view on design and most people's view on design. And as you said earlier, like the average sort of Silicon Valley orientation is a set of quotes that I'll read in full. You say, in some domains, the problem is obvious and the solution is really hard. g. the prompt might be make cheap, clean energy.

The rest is engineering. But in my experience, most design is the opposite. The hard part is understanding the needs and the solutions fall naturally, maybe with a bit of editing from a pro. In these domains, I believe empowered end users will beat external experts every time. I think it's possible to have great respect for the craft of design while also respecting that people know their own circumstances and can learn to design quite well themselves. And then finally, perhaps most strongly, I'm pretty pessimistic that it's possible to design truly great software for someone other than yourself, especially if that person is operating in a complex environment.

You talked about it at the top of the conversation around designing for teachers earlier on in your career. And the root of this that I think probably comes up verbatim or close a few times is like, maybe we just need more people to design software. If you're right, if the hard part is actually understanding needs, why aren't more people capable designers? Yeah. You know, to fill in that backdrop a little bit. The story that led me to this conclusion actually was designing for teachers. I was a designer developer. We had a lot of former teachers on the team.

We were doing the correct thing and talking to teachers and principals all the time. But I had never been in the classroom. And I always felt like, man, I'm just operating off of this kind of hazy view of what a teacher's day is like. And I hated how I would... constantly be blindsided by new things i hadn't understood about being a teacher that i think i would have understood if i had been one you're like trying to understand the bicycle gears on a piece of paper something like that yeah exactly i'm like i'm i'm reading like a book about it and i haven't experienced it enough and you know a lot of it is like stuff that isn't part of the formal job specification either it's like When do teachers do stuff?

Like, when are they on their computers? How much time do they have? Like, I know they're busy, but like, what exactly does that feel like? And it's just, I, there's all this design thinking stuff about maxing out your empathy. And, you know, to credit design thinking, I think one of the strategies people use is like, not only deep ethnographic embedding, but like, get as close as possible to actually doing it yourself. So maybe, you know, some... People might say I should have gone and become a teacher for a year and then I would have been in better shape.

And that's, I'm sure that's true. But I also think to push it further, maybe 100% of teachers are not going to be good designers of software. But I do believe, and I actually know from talking to certain ones that some of them are. Some of them combine the lived experience and the deep understanding with a certain way of thinking and a design mentality about improving stuff. And maybe let's say it's 10%. of teachers, how many of those people are currently empowered to do anything in their software worlds? Yes. Not many.

And I sort of see sometimes the mission is like, we have a 1% of people can build software today. We don't need to get it to 100. We just need to get it to 10%. And that 10% is going to be, you know, distributed across every profession and every, you know, corner of the world. And we just need those local experts to learn enough to do useful stuff. I love this idea. It also, I think, starts to get us to like, we've been talking a lot of theory, how this is like really going to start to improve in practice.

Like, how are we going to get more malleable software? I think we haven't talked extensively about like the multiplayer side of this. You frame it as sort of like communal creation. This idea of the local developer, the best. version of this that I found was when you're talking about the one guy in the office who is great at Excel. It's like, actually, most people aren't great at Excel, but you know a guy who can do it for you, which I think is great. There are two sets of quotes that I wanted to read that I think nail this.

The first is just broadly on the single player versus creating what I think you call malleable infrastructures. And there's even more fundamental reason to think of malleable. malleability through a communal lens we use computers together a product team needs a single system for tracking projects a department at a hospital needs a single system for patient intake forms a community organization needs a single video for system conferencing these communities are certainly not well served by one size fits all applications but the solution can't be every user for themselves malleability instead malleability malleable infrastructures need to help communities build and maintain shared solutions to their problems and then on the local side

Studying spreadsheet use at companies, Bonnie Nardi found local developers, amateur enthusiasts who could write bits of code for their coworkers and guide their team up a gentle slope of spreadsheets. Malleable software can look to examples like this. Clay Shirky termed this pattern situated software, describing how his students were able to rapidly build software for their communities by taking advantage of social infrastructure or context-sensitive information. On an even more intimate level, Robin Sloan memorably described how an app built for his family could be a home-cooked meal, as we talked about. whether it's 1% or 10%, like these local people, what do we need to do for them?

How do we, you also talk about in this context of like increasing software literacy, which is really cool, like compared reading rates or literacy rates in the past. But like, yeah, I guess what are the characteristics of these people? Are they basically your market? Like, what do these people need? Is it everything we've been talking about? Are there specific frames you think about in terms of the person in the? Yeah, I love this term. local developer another one that i like uh from alan mcclain's work that i said earlier is software handyman i think is a nice way to think about it this like local person who's like knows yeah knows maybe a little maybe a lot more than you but isn't like some they're not like a construction crew they're kind of a mid-tier person and probably you could learn to do most of what they do if you owned a home and cared and learned it right um you don't need to be a super professional there's kind of this this middle zone and i think that's where local developers fit in You know, I think to your question of what do we need to do to empower them?

So Bonnie Nardi, who's an HCI researcher who's done a lot of foundational work in this field, including this book called A Small Matter of Programming. She, I believe, coined the term local developer in this work that was studying spreadsheet use. And they noted a couple interesting things about spreadsheets. So we're going to add to the list of 10 reasons why spreadsheets are great here. So they kind of pinpointed what is it that makes spreadsheets good for... working with local developers, right? So there was a couple of things. One is that the novice person can still do a lot of it themselves.

Like they can enter all the numbers and they only need to ask for help for the parts that they can't do. And when the expert comes in, they kind of work in the same shared space as where the other person already was working. So they're just entering formulas that you could have typed yourself. This is not like, you know. There's the zone you do and then the other person's operating behind some wall that you can't see. You kind of share the space. And what that lets you do is there is education that can happen.

It's not magic. It's not magic. It's all there. And you can pick up some of the formulas. You can read what they did and maybe you can tweak it and then you can copy paste it next time. And there's a lot of sort of like tacit transfer that can happen from coexisting in that space. And I think that is to put a point on it. In software, often what you have is there's the lobby where you work. And then if you want anything changed, there's some hidden like boiler room somewhere, which is the code that you never see.

And the developer might say, sure, we'll do that feature for you. And then they do it and it magically appears. But you don't know, you can't see it. You can't edit it. So I think what we need to enable the software handyman, the local developer, is actually not necessarily just tools for those people. We need everyone to be working in these spaces where you can more easily bring in help. And I'll note, so increasingly, you mentioned this earlier, that help could be AI. So I'm really enamored by this idea that the local developer could be your AI.

And that has all these nice consequences where, again, the goal isn't that you delegate to them in the end. Actually, the goal might be that you end up not using AI anymore. If your local AI teaches you all the spreadsheet stuff you need to know, you don't need them anymore. That's actually, I think, a pretty cool endpoint. Yeah, it might be a little optimistic, but it's certainly possible. Speaking of AI. You say you said, well, I want to say one thing about that. Like, yes, I think often people respond to this vision by saying, man, this is like you're being very optimistic about human nature.

People don't want this. Right. And I think if you're talking short term, you totally might be right. I'm not sure. But if you're trying to make a startup that's going to make a billion dollars by next year, maybe this whole thing is a bad idea. However. I really believe that people respond to their culture and their environment in terms of how much agency we feel. The way we're brought up really shapes the way that we act. And so one of the reasons I care about this is that I think if every teenager lives in their iPhone and you can't change anything in your iPhone, then there'll be a self-fulfilling prophecy of this pessimistic view of people as passive consumers.

And I think there's a lot of forces in our society that are pushing us that way. And so anything that can... not just the other way i think even if it doesn't apply to most people today it could contribute to a more positive feedback loop mediums shape us very deeply i read um i read neil postman amusing ourselves to death earlier this year and he makes this argument about like obviously culture is being shaped by television but his his stronger argument is that we ourselves are being shaped in terms of how we think Yeah, you embody that in a line.

You say customizing software in this way is an uphill battle because the tools and infrastructure that we use to develop and deploy software treat users as passive recipients rather than active co-creators. Since you brought it up, let's talk about it a little bit more. As I mentioned earlier, this is an agency story. I think at the end of the day, you sort of have to believe an agency to be doing what you're doing. I am definitely... sympathetic to like the environment argument. The mediums affect people. I think at the end of your piece is where this came from.

You're talking about culture and you're like, how do we cultivate a movement towards personal agency where people want to modify their environments broadly and digitally or physically? It seems like the world is like broadly moving to like, people don't want home cooking. They want DoorDash, like let alone software. Like, is there something at root that Is there a seed of agency that you are pulling on or you think that we're all plastic and you can push the pendulum either way? Like maybe that's a more philosophical question at a very broad level.

And if so, like at the very least, what types of environments or constructs do you think are going to make people? We want our, let alone software, we want our entertainment to be automated. That's the way things are going. Bleak, huh? I can only speculate. I don't know the answer. Or feel free to speak personally, too, if that's easier. Sure, yeah. I mean, I guess I have a sort of an axiomatic belief that most people are pretty plastic on this dimension. And I think every experience that you have where you do something and it matters enforces the idea that...

what you do matters. And every experience that you have where you try to do something and you're not allowed to, or it doesn't matter, reinforces the idea that you're, you, what you do doesn't matter. And I think that loop can be really, really strong. And there's, there's all sorts of, I've seen, you know, sort of cute ideas to take the DoorDash example, right? I've seen someone posted at one point a design that was like, It was like a DoorDash menu that had home cooking as like one of the options in addition to all the restaurants.

I think a tricky thing is that in the DoorDash example, what you're fighting against is a company who's incentivized to make you forget that delivery food is not the only option. They want you to think, oh, I have a thousand choices. How great. Yes. What I'm supposed to do is pick one of the thousand choices and I'm good. Same thing with Netflix, same thing with TikTok. Yeah. And there's like very strong and heavy forces pushing that. And so I don't know the solution exactly, but I think it's true of a lot of people that it's probably true of everyone to some extent that what you see on the menu and how people around you behave really, I think it's true of me.

I'll say, I'll speak for myself. It has an effect on, I'm, I. I don't think I was always a particularly high agency person. And I think of it also as a trait that I'm constantly trying to get better at. And I have role models in my life where I see them doing stuff. I'm like, wow, I want to just do stuff more like that person. And I think a lot of it is the way the education system works. Get good grades. Get into a good college. Don't think too much about why you're here is a philosophy that doesn't inspire a ton of agency.

Whereas an alternate approach of like... what's a project you want to do and why and what you need to learn to do that project is a different way of thinking and it took me I would say sadly it sort of took me like until late college or after formal education to realize I don't think that's I think that's probably how the world worked for most people and it felt a little bit you know better late than never but I wish someone had had started this loop for me when I was 15 and not 25 and I think our school system is actually probably inversely correlated to this you're probably better off

if you aren't in a traditional school system to click this early. 100%. The more I reflect on my schooling experience, the more radicalized I get against grades and a lot of the way education works, because I think it is a very agency-robbing experience in many cases. It's tricky, though. I'm not saying there's a role for skill development and formal practice and stuff. I just think that, you know, I'll actually tell a story of them. One of the things that got me to where I am right now was I did this little one-week retreat at this place called Recurse Center, which is, they call it, I think, a programmer's retreat or something.

It's basically, you're a programmer and you want a place to just go get better at your craft or think about stuff. And it's a room with a bunch of other curious, hungry people. And you go there and you do what you want to do. And there's no curriculum. It's intentionally extremely open-ended. Part of that process, most people go for like three months, but part of the process is figuring out what you want to do and having some false starts and having some existential angst. And there's a lot of like, it's nothing's being spoon fed to you.

And it's very inefficient in that way. And that you probably spend half the time in weird spirals. But at the other end, what you get is this like renewed conviction and sense of like, oh, I figured out what I actually want. Where I want to go and now I'm super super motivated to pick up these 10 skills that I need to get there. Yeah Yeah, and that's that's why I ended up going to grad school I was not like planning to do that and I realized okay now I know big gap to like five or six years after school Yeah, I wasn't gonna do like PhD is like a terrible deal.

It's like you don't get paid much It's like you got to work really hard and write all this stuff, but I realized okay to get to execute this vision that I've come to now I'm like devouring these classes and books and stuff. And I think that's, for me, that sequence of mess around, do it wrong, figure out where you want to go first, and then get a coach to help you learn works way better. And I think school just completely screws that up most of the time. Yes, because you're not doing it for you.

Exactly. Yeah, you're doing it for someone else's made-up sequence that doesn't apply to where you are right now. Yes. You... It's a little more tactical and less philosophical, but you've written about how can we make programming feel more like a video game? You have this little blog post about this Claude debugging GUI you built. Maybe that's one very simple version of this. And I don't think it's surprising at all that lots of kids who are addicted to video games or Minecraft end up becoming programmers. But do you think, is that one version of the type of context or environmental way that you can push people more this way?

Totally. And I think an even better example of this in this context we're talking about is Seymour Papert, who's an early pioneer of constructionism and education philosophy around exactly what we're talking about. Like, do projects, don't be fed facts. He had this whole web of ideas around, he talks about, you know, how do you learn French as a kid? You live in France. Boom, it happens automatically. How could you learn math like that? You just need math land. And if you live in math land, you'll pick up math. That's what video games can be.

I don't know if he would call it a video game exactly, but he thought of designing these environments like Logo, this programming language, not really to teach programming. Actually, his goal was more to teach math. And he thought that if kids lived in that place, let's sort of... immersed in these concepts, they would naturally pick up what they needed in order to do cool stuff. Like in that in that in logo, it was like, I want to make a turtle draw a circle. That's really awesome. You know? Yeah. And so I think that's a huge inspiration for me.

You mentioned some debugger projects I've done. I think that's that could be one ingredient. I believe that like being able to intuitively visualize what the heck is going on is really powerful as like one ingredient and feeling like you're you're seeing it and getting it and you want to participate in it. It's literally being able to pop up in the hood and see something, you know? And a big problem with software is that you can't do that. When I was in fifth grade, my teacher let me take apart a vacuum cleaner and it was epic.

I didn't really get what was going on, but it was just like, oh my God, I can unscrew this thing and see what's there. Lots of engineers with stories like that. And I think kids have a ton of agency from birth, I think. And kids are really naturally curious. And I think a lot of this is just creating prepared environments where you can let that take its natural course and things will work out fine. On that note, especially given maybe the lack of connecting to the school part, at least initially, you seem to be someone who has like a very strong default towards like hacking and tinkering and experimenting.

Where do you think that comes from? Maybe it was the vacuum cleaner, but... I don't know. I think I'm the kind of person where once I get an idea for how cool something might be, I just have to try it and see. And I'll ride that wave for like a day. Yeah. And I get kind of obsessed. And so I think I just enjoy that like first initial seed of something and playing out all the implications. But where does that come from? Because by the way, what you're tapping into, how do you induce agency?

You get somebody excited about something. Anyone will do anything if they're excited about it or they want something. But there's like a curiosity there that is like connecting in a way that like, this is what video games are good at is like they help the kid connect in a way that they'll actually be motivated to do something. But I think that that's the question that like, I mean, obviously we don't get to choose what we care about, but I'm curious. Is there even, have you found a pattern in the things that are attracted to you in that way?

You know, maybe one thing I'll say is I love, so. I love it when work that I do changes the way that other people see the world or think about stuff. And to me, that part is really important. It's not just me doing something. It's like other people reacting to it. And I guess as a kid, you know, I would like make weird animated videos and send them to my friends and they would like laugh at them or whatever. And that was cool. I think now in the Internet age, one thing I find powerful about living on the Internet to some extent is you can just put stuff out there and so many people can see it and react to it.

You know, I think Twitter is a medium that we both participate in a lot. For better or for worse. Seriously. But the for better part is like, oh man, I can assemble the hundred people in the world who are wanting to nerd out about this super niche thing and we can talk about it. And I can say something and people will listen if it's valuable to them. Global show and tell. Yeah. And that possibility of changing how people see stuff. It's just, I think I find that really... exciting the idea that ideas matter and sure making things real matters too and execution matters but like ideas can be really really powerful and so it's just fun to get stuff out there like that yeah there's obviously plenty of risk of over rotating on this front but like ideas maybe have never mattered more which is a cool cool thought maybe a couple of things that i just wanted to hit quickly before we wrap up on the malleable software stuff I brought it up briefly.

The AI thing hanging over so much of this, you had a tweet in March, 2023. You said, if it seems likely to me that all computer users will soon have the ability to develop small software tools from scratch and to describe changes they'd like to make to existing software that they're already using. And then by the way, one funny thing is that a lot of software is actually built like this under the hood. Engineers don't make things from scratch these days. They find libraries and assemble them into products where the seams are hidden from the user.

You were talking about foam. In the foam model, the modules are exposed, giving me the power to open the hood. One view here would be that like all software is just going to be malleable because like all software is going to be amalgamated and not handmade. Maybe that's optimistic or maybe that's likely. I disagree. I disagree. And or at least I think that's not the whole story and I'll explain why. So in our essay on malleable software, we have this analogy to try to make this point, which is. Imagine that there's no more home cooking like we've been imagining and you can only eat at food courts.

So there's like maybe 200 things you could order, but that's the list. AI is a sous chef. AI is this like maybe intern, maybe brilliant chef who you've now hired for 20 bucks a month to help you out. You bring the sous chef to the food court. What are they going to do? They like... You're just still stuck ordering for the menu. They don't they don't have access to the kitchen like the point here is that you don't control the production of the thing at all and what your hand it is like an artifact this Sorry to interrupt you that just to clarify that this shift doesn't necessitate more software being open source and that's why it's still the food court exactly so You're you're jumping ahead a little bit.

Yeah, my point which is that like There are other things I need to change in the software world to actually take advantage of AI. And software being open source is one of them. And it's true. I think if all software was open source and you combine that with the fact that AIs can code, that's a pretty good start. Because now if I want to change something, I just like theoretically pop it open, edit it and I'm done. Yes. I still think there is more to the picture, though. Like the more that you can do without coding, the better.

There's a lot to figure out around like. Now, if I want to share my version of the app with you, like, how do I send it to you? And then how do we decide, like, are we going to keep using it together? Or are you going to go make your own remix now? How does that work? Even open source is like, it's necessary, but not sufficient, I would say. Like, it's an awesome thing for the world. But if you've ever tried editing an app that you're using that was open source, it's not easy for a lot of different reasons.

Like even getting it running on your machine might be like hours of work. And so there's just all these barriers that I think. I like to sort of flip it as like, how do you design for everyone having a local developer or a sous chef? Like assume. Or maybe even like a world star Michelin chef, but still operating. Exactly. And the answer is like everyone has a kitchen. So like if all of my software was built in a way that I have little tools composing and, you know, back to the earlier story, like when I built that little tool while making this essay.

we have a malleable environment where the whole point is that you do your AI coding and you take the thing it built and bring it into your kitchen and you combine it with the stuff you already have. I think that's more of the ethos that we need to, we need to figure out how to do that well to actually take advantage of. It relates a tiny bit to the notion that we're all going to have like an MCB server. It's like, it is more of this sort of like, how do I create like my corner that can interop with the world?

But like, yeah, yeah. I think you hit the nail on the head. Like, how do you manage your own context? How do you manage all of your stuff in a way that you can, because AIs love context. They love knowing about you. But it's creepy that Sam Altman knows so much about me. I want to have my own little thing that knows everything about me. I've made a personal assistant. My wife and I have an AI assistant that we use to help run our lives now that I made a couple months ago.

And the more it knows, the better it works. I want to tell it how our weeks run, what food we like to eat, when people are visiting. That's a lot of stuff. This pattern of like you have your little world of stuff about you and you bring tools to it is a really powerful thing you're pointing to. You're also that. I mean, we do the thing you're referencing I'll link to it. You guys made you made this little like telegram butler who like speaks to you formally. And like that probably is pretty dang close to the prototype of like what.

our whatever agent like we are going to have intelligences that do stuff for us and that we're we have intimate conversation with all these types of things and it's going to need to be able to control and internalize your world it works and this is an instance of malleability my own life like i love that i control how that thing works i know how it works it doesn't do random stuff i don't want to do doesn't change what i don't want to do i can constantly add new stuff to it and it's like i have a butler that i can train over time and make it do what i want and There's sort of no limits to what I'm allowed to do with it.

And this is so different from like Siri or, and the crazy thing is I did this, you know, I had a baby last month and like, this was like vibe coded at 2am with one hand, this whole project. Right. And the fact that I'm a soft, that I have software engineering training helped. It's not that anyone could do this right now, but the things you can do for yourself that are possible. on the side are absolutely different than they were. And I think this cost-benefit calculus is like shifting really rapidly.

It's something I'm feeling in my own work. Right. Yes. Two very empowering and interesting ideas there to me. One is perhaps what area will malleability be more important than like our personal super intelligences or whatever? Like literally. And number two, Going back to the conversation we were having around agency and whether people are going to care is like, yes, we've come back to this theme that increasingly more and more things can be automated for you. But also we are moving more and more into a world where like more is the floor.

in this the floor is getting lower and the ceiling is getting higher and like the possibility space is getting wider which like if you just lean forward even like the video game meme of like leaning forward in your chair if you even lean forward just a tiny bit like if if the non-technical person couldn't vibe code that assistant two months ago or now they definitely will be able to in six months and like that ramp is just going to keep going absolutely and man i love what you said about these personal assistants are gonna have huge influence on our lives.

This is already happening. Like the Twitter algorithm affects my thoughts a lot. And that's one of the reasons why I think, for example, like Blue Sky, pick your own feed algorithm is a great example of a certain kind of malleability. Yes, yes, yes. It's a lever you have over how you want to live. And the lever you have over the person who's literally hardwired to your attention and has a switch on it. Exactly. Right now it's Elon. Exactly. And I think. I think a lot about my susceptibility to like the feedback loops in my life and how I can, I want to figure out my values, pick my feedback loops, and then have my loops support me in enacting my values, not like be constantly fighting.

Yeah, that's agency right there. Absolutely, yes. And I think it's like Odysseus tying himself to the mast. It's like, how do you set up the system in advance so you can get the outcome you want? Brian Johnson talks about this in an old essay. Now he's... He's gotten more extreme, but he literally was like, I revoked evening Brian's eating privileges or decision making privileges. Yeah. You know, like, that's not a choice I'm going to make, but I like that he can make it. You know, this is the possibility. Yes, yes, yes, yes.

My last question on this that we haven't really talked about that is a huge boogeyman around everything you've explored around malleable software and whether it can really happen is incentives and commercial incentives specifically. As we've talked about, you need tool makers, you need infrastructure makers, you need kitchen makers. Explore all these metaphors. You need them to actually want to and be incentivized to enable this type of stuff. You can give the example of Chrome early on. Like one decision made a long time ago enabled so much in the way of browser extensions.

Before we explicitly talk about the commercial piece, which is, I think, super important, from a super baseline incentive standpoint, I found myself coming across this a lot in technology where like, there's a temptation to believe that like people will do something for an ideological reason. You run into this in crypto a lot. My favorite example is just like, the great genius of Tesla was like, don't buy the electric car because you're like a tree hugger. Buy it because the car is sick. And like, that's how you, same thing with fake meat, whatever.

You're not just relying on sort of ideology. What do you think the most compelling reasons? We're talking abstractly and it's probably hard to be too specific here. What do you think the most compelling reasons for toolmakers, app developers, infrastructure builders to actually think in this way or at least read the piece and like want to buy into this? What are the ways that they should be selfishly incentivized to do this? This part's really important. And it's, to be honest, I think it's a part that I have much less confidence on.

It's less a narrow technical question and it's a very tricky. Almost economics question. Yeah, coordination problem. Yeah. I think, though, there are some simple reasons to point out. So you're a software developer. You have 10,000 bugs in your backlog or 10,000 feature requests. And you're in this meeting and your customer is saying, can you add this feature that competitor B has? And if you don't, I'm going to go with them. And your answer is, well, we have restrictive resources. We can't do it. Wouldn't it be nicer if they could just add that one thing they needed and stick with you?

And this is not like a new idea. Like plug-in systems exist for lots of apps for precisely this reason that you can serve the long tail by basically outsourcing a bunch of stuff to external developers or even to users themselves. So I think that's a win-win for users and devs if you let people do more on their own so you don't need to do as much work. Or even in some cases you have a product like Obsidian where like... The product in many ways, one of its main selling points is accessibility.

Those are maybe less rare today, but they do exist. So Obsidian, I think what's interesting is that Obsidian takes it a step further, which is that it's a more open system because the data is portable. Most plug-in ecosystems have this catch, which is that it's a proprietary plug-in ecosystem that's actually trying to increase the gravity around the lock-in around a single... That's why it's good for the developer. Right. But I think at the same time, that's a lot of what damages, like the reason apps are siloed is because each app developer is trying to hold on to their moat.

Right. I mean, like this is getting the commercial part a little bit, but like if I'm Notion or Figma, why would I possibly want people to use other clients? Like my entire business is built on the fact that I have lock in. It's not exactly the plug-in idea but like that like it's hard not to stare at this stuff like Google Docs Yeah, maybe a little like that's not really where Google makes money Maybe a little easier Google has some of those that ethos but like most software to the extent it's gonna have a mode It's not making the software anymore.

It's the network effects I want it to be less mad or maybe not madly but I certainly wanted to be less than interoperable in theory Totally I think if you look at what are the greatest success stories here. I think of like the web. Yeah, I think of email so Email is not owned by any company. The web is not owned by any company. There's players in this ecosystem that, you know, their email clients, their email hosts, there's different websites. And a through line there is that no one value captured the whole ecosystem.

It was created in a context where like Tim Berners-Lee decided this, I'm not going to become a billionaire off this. I'm just going to let it happen and try to not capture the value. And I think ultimately someone has to take that move of provide tremendous utility to the world. in an open way where they're not capturing the value. And then other people can live on top of that. What about, is there a more realistic template? I mean, yes, hopefully you have amazing things like that happen. Is there a more realistic template in just like commercial open source?

Is that the closest thing we have here to commercial models that make sense here, which is like you're incentivized to build a relatively open, malleable thing in part so that it will actually succeed? And by the way, you can, like a lot of open source companies, monetize laziness, monetize people hosting, like all these types of, monetize being the best and first client for it. That was my best attempt in trying to think about it. I think it's a great comparison. I'm not sure if I'm reassured or terrified by that comparison.

Like commercial open source is really hard to do well. And there's a lot of cases of people not getting it quite right and making awesome things and failing to make great business out of it. So I don't know. But yeah, I think there's some comparisons there that make sense. Like owning the code is not the moat. Yeah. We haven't talked about it a ton. You have been a part of a number of at least a couple notable, interesting research-oriented groups. First MIT and your PhD, now Ink and Switch. This is a strange, especially in maybe, obviously not broadly strange, although Ink and Switch is pretty unique.

but a relative to the world that you are often peripheral to and adjacent to, like most people doing really practical stuff in software aren't usually doing it at PhD programs, I suppose, with my intuition. I'd love for you to just talk about those two worlds and like why they've been empowering for you, how you think about... Is research the right long-term place for you? Is it a temporary thing? Like, how do you think about that mode having done it now for, I would guess, what, five, six years? Yeah, so I did a PhD for, I guess, yeah, four years.

And then I've been at Ink and Switch sort of throughout that and since finishing. And also just like what actually is Ink and Switch? Yeah, so great question. It's a bit hard to explain. Ink and Switch is an independent research lab. It's privately funded and... We sort of think of ourselves as bridging between academia and industry. So we're not a startup. We're not shipping a product today. We're not trying to make a huge return on a VC timescale. On the other hand, we're also not an academic lab. So we're not just interested in like figuring out the truth about the world.

We want to change the way the industry works on, let's say, a medium term timescale, like five to 10 years away, not one year away. Cool. Actually, the genesis of the lab. The lab was started by the founders of Heroku, which was a successful startup. And they felt that, well, not to put words in their mouth, but my understanding is that they felt that a startup wasn't an appropriate vehicle incentive-wise for producing the outcomes they wanted to see in the world. And so that kind of gets to the way that I think about what institutions do I want to be part of or affiliated with.

I think it's all about what incentive loops are the closest to the ones I want to be guiding my work. Most jobs have strings attached and you just got to pick the best strings. Where's the money coming from? What does the money want to happen? On the one hand, I think startups are amazing because they have such a good feedback loop with reality. If the thing is not useful, it will die. And that's it. And I love that reality check. I think is amazing. The problem is you can't dream beyond.

It's really hard to sell something that people don't want yet. but that you think they should want or that you imagine this world where they could want it. You just don't have time to play that out. And so on the other hand, academia, I think often, you know, publishing papers is the main goal. The number of good published papers that you have is how your career is going to be judged. Right. And I think there is a lot of problems with that, especially in HCI, which is the field that I was going to say.

Also, as part of your thesis, I think you built at least three. I don't know if you would call them apps, but like very real software things. Totally, yes. You wrote a nice thesis, presumably, too. It takes a lot of work to do. I mean, you know, some of the problems I see with HCI academia are like there's often sort of a separate conversation happening over in the academic world that doesn't influence what's happening in tech enough, I think. Right, right. There's great work happening, but often the transfer function is poor, I think.

And especially when things are moving so fast, it can be hard. And there's also just... There are certain ways I think of the bar that a paper has to meet to be published is both too high and too low. Like they care about rigor that I don't care about, but they also don't care if it's interesting, for example. So what we try to do at Ink and Switch is create incentive loops for ourselves that will lead to change in the world of the kind that we want to see. And so we try to think about, okay, you know, how can we change the minds of designers and engineers who are building software today?

What artifacts can we put out into the world like open source or spin out products that people will actually use and build on top of? For example, the lab has this data layer called AutoMerge that is maintained as an open source thing that people can build software with and build on top of. It's part of the local first movement is providing this as a public good to the community. So we think of sort of a portfolio of ways of influencing the world that hopefully... nudge things the way you know we want them to and our unique advantage is not that we're smart or talented or that we you know although i hope that we are those things it's sort of that we have a different structure like and the fact that we aren't vce backed and being a startup allows us to go places that other people can't go yeah allows us to think about what would be the substrate not like what is the app and i think that's that's a cool opportunity i think to have a unique and like differentiated contribution to the space of ideas people are talking about we briefly talked about ideas earlier and the importance of sharing them to what i said do you think about the near medium and long term around like i think your research and writing is deeply inspirational uh and presumably is having real impact and that could continue to compound there's also a world where you could return to building production software however that's going to look in the future you spent five years at a startup at the beginning of your career.

How do you think about that set of, I'm not asking for your imminent career plans, but like, yeah, how do you think about that trade-off? Yeah, I love both sides of it. I probably want to ship to users again eventually. I don't know when. There are real trade-offs. And I think one of the biggest dangers of being an idea space is that I like to say, like, how often are you being slapped in the face by reality? At a startup, it's every day. You go try to sell stuff, people don't buy it.

And you're like, well, I guess that's not how reality works. And your mental model is being like really aggressively updated. Hey, you get that from tweeting too, maybe. That's, yeah. Well, it's a different loop. Tweeting, tweeting, you need to have cool. interesting novel insightful ideas for sure there's like a thing you're optimizing um it's not sometimes not even novel i think you need to say things that seem contrarian but everyone actually yeah that's that's a perfect algorithm for you've cracked the code but so like in a way i think that actually encapsulates like um one way i think about my own work sometimes is okay i have put in the work and i know how to get twitter likes does that matter like is it aligned with the change i want sometimes like yes there is value in knowing how to sell an idea and knowing how to get people to understand it.

But I don't want that to be the only loop guiding what I do. You know, I think of kind of you want a good mix of incentives driving you. And so something we're trying to do more at our lab is like have more loops with real people. We tried a project last year to have astrophysicists write science papers and do science in our tools. It actually didn't go that well. We learned a lot. They weren't able to do it yet. So that was a slap and face from reality, right? We have a project ongoing right now where we're trying to help kids collaborate better on game development.

And we're running tests with teachers and soon students in classrooms. And that's, again, you know, we're saying we want to make collaboration more accessible. Is it? Let's find out, you know. And so I think that's, even in a more researchy sense, that's totally essential. Totally different topic, but you have a post on ChatGBT as a muse and not an oracle. I think it was pretty prescient. It was back in 2023. You said, I've been wondering, though, if this framing is missing a different opportunity, the notion of ChatGBT as an oracle or search.

What if we were to think of LLMs not as tools for answering questions, but as tools for asking us questions and inspiring our creativity? Could they serve as on-demand conversation partners for helping us to develop our best thoughts as a creative muse? Any reflection on that piece? How has your use in that way evolved? I'm glad you brought it up. I've been thinking about it recently. I think it's probably from my personal perspective become increasingly true. I wrote that because I was frustrated by people thinking that an LLM was supposed to be used as like a fact database.

when it's actually really bad at being a fact database. And I mean, they're getting better, but it's kind of a boring way of using them. Andre wrote that piece a while ago about like hallucinations being the point. Exactly, yeah. Some of the most interesting moments I've had with LLMs are treating them as a thought partner just to kind of, sometimes if I want to talk to someone and it's like late and there's no one around and I want someone to shake up my thoughts and throw some new seeds of stuff into my thought space, an LLM can be a great, way to get more entropy into my thoughts.

And then maybe it's as simple as ask me questions. There's this idea that I love from Gordon Brander, who has a great blog and has thought a lot about creativity. He points out that there's this deck of cards that Brian Eno made called Oblique Strategies. I don't know if you've heard of it. Basically, when you want to be creative, it's a deck of cards and you draw one and it has a very vague koan-like saying on it. My friend Sean has this deck, actually. I don't remember them exactly. It's like, flip it upside down or something and The point isn't that that tells you what to do.

Yeah, it's just again It's like a seed that intersects with the current contents of your mind to totally shift your ground Yeah, and I think of what's funny about that is a deck of cards can do that and then LLM is like a Super super powered version of that thing. The bar is low for creativity support tools to shake up our thinking. And an LLM can probably do better than a deck of cards. Yes. But one of the things I noticed in that piece that you're probably, my intuition would say that you are incrementally better than the average person at is prompting this type of behavior.

Obviously, so much of LLMs, as we talked about, is like adding in context. Maybe a simpler question would be, have you found yourself prompting more aggressively in this way? Or have the models just gotten better at this in a way that you don't need to guide them? I'm not sure. I think I probably still do the prompting reflexively and I should test if I can remove it. A very common thing I do is just say, ask me questions about what I'm thinking before telling me your thoughts. A simple technique I like is give me 20 options, not one or three.

Because LLMs, I think of them as like the speed and the breadth of generation is a huge asset. And they're really good at generating options. there's like zero cost to generating 20 options instead of three. It's instant. Yes, with a person, it would be super high. It'd be super, yeah. We're not used to that. Exactly. And so I love generating these long lists and then usually most of the options are bad. Maybe there's one that's good or more often there's like parts that are good and a few of them. And I see this like, it's like we had a brainstorming session and there's like 50 post-its on a wall and I apply my taste to kind of like extract the best ideas from that corpus, you know?

So I think that's another example of that. I like that. You can definitely get pretty creative with playing to the model's strengths. Well, yeah, we could talk about that all day. LinkedIn Dive, you worked at Mubi from 2011 to 2013. Whoa, throwback. On designing the streaming platform, it sounds like. Yeah. They just raised $100 million from Sequoia. Really? I didn't know that. At a billion-dollar valuation. Yes, and Marine at Sequoia. That's news to me. News to you. Wow. Yeah. congratulations movie yeah i'm curious i like movies i actually am not a movie user although i've i've played around with a little bit yeah curious how did that happen any thoughts i mean that was a long time ago yeah oh man no one's ever asked me about this in an interview i have fun stories from that time i was in college i didn't know how to do much that was useful to be honest i was looking for an internship i think i applied to like the big tech companies and bombed my interviews and they were like, yeah, you can't really program.

And I was like, okay. So I found Mubi, which was this tiny startup at the time. I think it was like under 10 people. They were based in London. What was really fun about my time there was everyone was too busy to help me or teach me anything. Well, that's not true. I got great support when I asked people for help and they were really lovely people, but it wasn't like an intern program. It was just, I showed up and there was a desk. And I kind of sat there and was like, what can I do?

And accidentally ended up working on really important stuff because there weren't enough people. I was going to say, did they have technology people? They did. They did. Yes. They had great tech people. The part that I ended up helping out a little with, I really don't want to overstate my contribution at all. It might have even been that negative, honestly, if you've ever worked with an intern, was they had this, the CEO had this really smart idea, which was. It's hard to compete with Netflix because Netflix has a big catalog.

Yes. And it's really expensive to license all those movies. So what if you competed on having a small catalog? And the way you do that is you focus on curation and taste, which maybe already was doing. And the key insight was everyone's tired of Netflix because you can't pick what to watch. There's too many options. So he had this brilliant idea that was, we're going to show you one new movie a day. It'll be there for a month. And if you don't watch it in that month, it's gone. It's off the platform.

And so there's urgency created. There's people with taste recommending to you. And it was this brilliant win-win where users get a better experience and it's way more affordable for the company to license great content that they couldn't afford otherwise. And it was beautifully positioned against the existing player. And so I got to have a front row seat to seeing that happen and see this company go through some pretty... what felt to me like bleak times i don't know and to see that creativity kind of spark a reinvention of the thing was just really cool to me you know well yeah they i think they have 20 million users now and the ceo just got a crazy profile in vanity fair and yeah you'll have to go check it out they're getting great yeah it's cool you grew up a little bit in japan and presumably go back there it's one of my favorite places and you have a japanese mom like curious how Japanese culture or living there has inspired you, how you think about things, any influences there?

It's a big part of my identity. I spent middle and high school there. I love having been exposed to both S. and Japanese culture as a kid because they're really different. The level of care you experience in Japan is ridiculous. And I think a lot of people see this when they visit. It shows up in the cleanliness of the benches in the train station. In my experience, it's pretty hard to get a bad meal in Tokyo because even at the sort of like average place, they, they care, they give a shit and they like, they lock in the details and they take pride in the work.

And it's really common, I think, in Japanese society for people to take pride in labor that might be considered low status in the S. The janitor has an important job to do and they're part of a society. I had some really cool experiences. In elementary school, I was living in the, but my mom would take me and my brother to Japan every summer. And we would spend a whole month at a Japanese elementary school, which was so cool. And it's different. I mean, there's actually a recently documentary that came out about this, but you're serving lunch to your fellow students.

You clean the school. The students are like running around the floor with rags. There's much more of, I mean, to the agency point and what we were talking about education. In some ways, Japanese education is worse in agency development, I would say. But in these interesting ways, it also has these glimmers of amazing developing a sense of responsibility and membership in the community that is vital to, I think, holding the way society works together. By the way, you take the most extreme examples that seem just totally insane in Japan of the little kids who are able to ride the subway alone.

or whatever, like the little kids in the little hats. Like, it's like American society to never support. And it seems to be rooted in a great deal of every person in that society feeling like they have some kind of responsibility to care for it. Totally. On the theme of agency, there's, I think, a stereotype of Japan that's like, it's a bunch of salaryman business guys in, like, boring meetings listening to their elders, like, tell them what to do, which is definitely... partially true probably but there's a really vibrant culture of indie businesses and entrepreneurship and like the restaurant scene in Tokyo is amazing because there's all these restaurants and bars that have like six seats and many of them you know many of them aren't famous and probably aren't the kinds of places tourists would go but there's these tiny businesses and they're popping up everywhere and I think Some of it's probably like zoning law or whatever.

Well, one of the best things about it is the three-dimensionality of it. Yes. You go up to the eighth floor of the office building in the back of the hallway and there's like a hidden, there's so many cool like no sign. And there's a lot of reasons why that is probably around the urban design. But I also think just there is a ethos, it seems, of like taking pride in your work and doing something not to make the most money. And also like the goal isn't to delegate it. Or scale it.

It's just like the person wants to run one bar with six seats and that's what they want to do. And they're fine with that. The goal is to keep doing it. Yeah. And the goal is to do it well and do it excellently. I think that's, I think American culture, especially in tech, is all about growth and scale. And there's something beautiful about that, kind of like keeping it small. Yeah. All about ends too, rather than means. We were having a conversation at lunch before this about probably sounds like you, like I, are a person who is dispositionally someone who wants to do lots of different things.

I'm curious how you have related to like. various levels of depth in your life. I was going to ask about music because you seem to have a lot of instruments in the background of Zoom calls. It sounds like you play the cello. That's like one area, for example, obviously software has been another area, but you're also someone who's like not staying in one place totally. Like I'm curious how you relate to maybe, especially in the context of like the Japanese philosophy of like mastery. How are you holding that? Have you found the right balance?

Do you hope to have more mastery in the future? Are you trying to go wider or deeper? It's something I really struggle with. Yeah, I love doing different stuff. I hate job descriptions or like job titles. I think it's always so... We hate legibility. Yeah, I mean, people try to like... what's the difference between a UX designer and a product designer? It's like, who cares? Just make great stuff and learn the skill set you need to do it. Yeah, making stuff, by the way, I should add, is clearly a really strong consistency for you.

Yeah, I do think, like, I respect masters of crafts a lot, and I think there are degrees to it of specificity, but... I definitely aspire personally to master in depth a cluster of skills around designing, developing, writing, and communicating that I think is sort of the bundle that I feel is necessary to do the work I want to do and see the change I want to see in the world. And a lot of, I think, a PhD program in grad school in computer science is actually, it's not about being a better programmer usually.

That's actually often taken as a prereq. A lot of research is about a bundle of skills that's like thinking, writing, reading, like reading papers is a skill. And yeah, making stuff and prototyping. But I love that it's this like coherent basket that you need to have. You kind of need to be a Mario in Mario Kart or whatever. Like if you're well-rounded in all those places, you can do a great job at this craft. Yeah. This holistic craft. Yeah, I like that. had a child for the first time very recently.

What do you know now that you didn't a few months ago? Oh, man. It's been really fun. Almost like more fun than I could have expected, I think. The highs have been so high, I was telling you. It creates a really interesting perspective to know that there's this really important thing that matters most to me now. And a lot of stuff that mattered before to me, it still really matters, but in a slightly different way. I can care about it, but ultimately there's something more important. I think can sometimes be maybe a freeing way to think about something.

You take a little bit of the pressure off and, you know, I want to do great work. I want to change how the world thinks, but also like if I see my daughter smile and like play with her, that's a good day, you know? And so that's been different for me. Yeah. We were also chatting at lunch about... you you had observed like there's something kind of pure in a pro athlete like a basketball player in terms of how does what they do actually matter it's just just a ball ball game it's just a sport but like they're they're perfectly honest about like how much they care about this thing regardless of how like quote unquote important it is for the world then we're also just chatting about like You said your brother's a doctor, like an athlete, a doctor, like they know right away.

They know when they're 20 years old, like what they're going to invest all their time in, what they're going to care so much about. And then you also, you tweeted, I think today, working with people who have high standards is tiring in the best way. So I was like marinating on that, like combination of ideas. We also talked about my conversation with Nabil and this notion of like, what are you going to care about? What are you going to choose to care about? You only get so many things to care about.

So, yeah, I suppose my final question is, what else is tiring in a good way? Why? It's a little bit like what I asked you earlier, but why do you labor and tinker and experiment and iterate and push beyond the default? Yeah. I'll tell you the story of where that tweet came from today, maybe, to get at this. So the secret is we're recording this before this essay about malleable software comes out. You've seen a draft, but it's not done yet. And it's been a long process. We've been working on this thing for months.

There have been a huge number of drafts and we're close, but we're not done. And one of the things that we believe at Ink and Switch, and I think that I've learned from working with the people that I work with now is there is tremendous value in just sweating the details and caring way more than everyone else. And it's hard and it sucks. And it's like, you can feel the finish line, get closer and then further away and closer and further away. And you just wish it was over. 90% done, so I only have 50% left.

Yeah, you wish it was over. And I think... Sometimes I personally fall prey to like, whatever, just get it out the door, which is what the startup does. The startup, you don't have time. You just have to cut it and go and iterate. But one of the nice things about writing essays at a lab is that you can actually do it differently and say, no, we're just going to wait until this thing is perfect. And then we're going to put it out there and people will notice. So that's something I've learned from some of my collaborators too.

Like every detail from the photography to the margins to every single word. The individual choices won't be visible, but you can feel when it's all there. And it's a very special feeling. I think you see that in Japan at a restaurant often, you know, to go back to that. It's incredibly rewarding to try to do that and aspire to it yourself. And it's tough, but having people around you who enforce that for you. Some of this is picking the right people around. I've sent this essay to a number of friends who I really respect their opinions.

And I've gotten some pretty negative or, you know, this part is great, but here are 10 deep flaws types of feedback. And it makes my day because it's like, this is so cool that I found people who I trust. And when I read what they say, I nod and I respect their taste enough to say, okay, we got to try again. And that's just really, I think it's rewarding. And it's something that, I think I want to keep getting better at. Yeah. If you're going to make anything, why not make it wonderful?

Go for it. Exactly. I mean, Steve Jobs has a lot, has a lot of good stuff to say about this, you know, showing, forget for Steve or Johnny, like showing people you care or expressing your care through things you make, I think is a lovely way to live. That's all I got. Thanks. This was really fun. This was wonderful.