← CoRecursive: Coding Stories
CoRecursive: Coding Stories
Story: Beautiful Code - Inside Greg Wilson's Vision for Software Design
00:57:20
Greg Wilson has been on a decades-long quest to transform how we teach and talk about software design. From getting rejections for using the term “beautiful code,” to empowering scientists through workshops on Python and Unix, Greg has pushed to...
Loading summary
Transcript103 lines
- [00:00]
Adam Gordon Bell
Hi, this is co Recursive and I'm Adam Gordon Bell. Today we're taking a look into the world of software design and architecture. Do you remember the swine flu? H1N1 Back during that time around, like 2009ish, I was working at a software company. I was sitting at my desk enclosed by this blue pastel cubicle walls in a large gymnasium like Open Office when one of the more senior devs, Cedric, comes over to the area and he has kind of a mischievous smile, which isn't good. And he says, I need to show you something. He asked me to pull up a branch in TFS and find a specific VB script file. VB script script being, you know, Microsoft's answer to PHP. Go to line 307, he says. And you know, on line 307 it says something like to do figure out the logic for this part. It's not even a code comment, it's just plain text right on the page. And it broke the page. The page wouldn't render. And yeah, that was my handiwork. And it was in a branch that we were going to release. Cedric thought it was kind of funny. You see mistakes like that happened more than they should have back then and that was such a minor one that it was humorous. This was the chaotic world we lived in. We had no code reviews. We were all moving super fast. We had a shared branch that we would use per release, everybody pushing code to it and then testing it and releasing it. It was a digital wild west where we moved fast, but it was a wonderful that anything worked at all. Later I joined a company with a code review process and man, that was a wake up call. I hated it at first. I know what I'm doing, just let me do it. But after a little bit I saw the value. The worst code issues are never these small comments or little bugs, right? They're the features that are slightly misdesigned that you end up stuck with for years. And if you can catch some of those in code review, if there's a forcing function for people to discuss how it's working, tricky conversations can happen. We can discuss the trade offs and the software improves and the team does as well. Hopefully that process is kind of what today's about.
- [02:36]
Greg Wilson
All right, one last test. Voice is coming through.
- [02:40]
Adam Gordon Bell
Yeah, sounds good.
- [02:41]
Greg Wilson
Welcome to wkfi. Going to spin some old vinyl for you this evening.
- [02:45]
Adam Gordon Bell
Ladies and gentlemen, that's Greg Wilson and he's not a dj. He's an accomplished software developer and teacher. He's Taught scientist, software engineering. He's authored many influential books, one of them being Beautiful, a pioneering work born from his frustration with the field. It's sort of what we're talking about today because Greg's been on a crusade to elevate software design from the mire of mindless feature building that I was doing back in swine flu days to a forum as respected and revered as architecture itself. You see, once you start reviewing code, talking about potential solutions, and as soon as it's beyond the level of like linting and code formatting, you need a language for discussing approaches. And giving us a language and a repertoire of software design has been one of Greg's goals. But before we dive into Greg's approach to software design and software architecture, let's rewind. Let's go back to the 80s, when Greg was an undergrad at Queen's University and he didn't want to be a programmer at all.
- [03:54]
Greg Wilson
I was doing electrical engineering as an undergrad for reasons we don't need to go into. And to be honest, I'm actually quite clumsy. I have a scar on my right hand from picking up a soldering iron by the hot end twice in one hour. And the second time I had my hand under the cold water tap. The lab tech came over and put his arm around my shoulders and asked me if I'd considered switching into software. And at the time I thought he was doing me a favor. And looking back, I realized the next semester was the course on power transmission lines with 50,000 volt transformers and things like that in the lab. And I think he just didn't want to have to clean up a mess that I would probably have made.
- [04:31]
Adam Gordon Bell
So Greg didn't become a human light bulb. He switched to programming. And he fell for it pretty hard.
- [04:37]
Greg Wilson
I was in the basement of Jeffrey hall at Queen's University, a cold snowy day in Kingston, Ontario in 1982. I was sitting in front of a green on black text terminal using a dumbed down version of vi and I was writing Pascal to play three dimensional Tic Tac toe.
- [04:56]
Adam Gordon Bell
This was a programming languages course assignment. Greg was still in engineering, but he was starting to transition to software.
- [05:03]
Greg Wilson
And it was my first encounter with recursion and with searching trees and backtracking and all of that. And there's that moment when you can feel the pieces clicking together. You've been looking at all this stuff scattered on the table and you realize that this goes here and this fits here and this other thing's going to tuck in over here and all Of a sudden, the jigsaw is starting to take shape and you can see what the picture is going to be. And every next piece, you can see where it belongs. Because I'd gotten so deep into solving the problem that I'd completely lost track of the fact that it's now 3:30 or 4:00 in the afternoon. I haven't eaten for eight hours. I'm getting a headache. But it was the first time I'd lost myself in solving a problem. And it was a rush. I won't say bewildered, but I realized that, like, this was one of the few moments in my life when I realized I'd had an experience that was going to change me, that this was something I'd never experienced before. And I didn't decide there and then that I was going to be a programmer. But the moment has stuck with me.
- [06:12]
Adam Gordon Bell
What was that feeling like?
- [06:15]
Greg Wilson
Honestly, years later, the first time I went scuba diving in warm water down in the Caribbean, I had that same feeling. And I've had it. Sometimes you're out on a bike, like you get on a hill, and as long as you can keep your balance, you can go as fast as you want because gravity is doing all of the work. But you gotta do the steering, you gotta keep your balance. You gotta think about whether you can make this corner or whether there's gonna be sand on the road and you're gonna skid out. But it's such a rush. I've had it occasionally playing music. I used to play tenor sax. And there will be times when you're on the beat and you're with the band and everything is just clicking together and I. You know, people talk about flow, but there really is something special that happens there.
- [07:01]
Adam Gordon Bell
Sometimes I know this feeling. Probably you do too. Sometimes I feel like my life's just been about chasing it. It was the same for Greg. He was hooked. He finished his undergrad and then he went off to Scotland to do a Master's and a PhD in computer science, where he ended up working at a high performance computer lab at the University of Edinburgh.
- [07:20]
Greg Wilson
Lots of physicists building lots of code that I never really understood, doing very complicated quantum things. But at the beginning and end, somebody's got to like, read all the files in and write all the files out. Somebody's got to checkpoint the computation in case the machine falls over. These days that kind of work is called research software engineering. Rse. You know, the analogy is the astronomer who builds telescopes but doesn't look through them. You need to understand the problem but you're not the one who's actually doing the science. You're doing the support for the science. So I spent six years doing that in what is now the Edinburgh Parallel Computing Center.
- [07:54]
Adam Gordon Bell
Greg got a PhD, and presumably he learned a lot of computer science. But really the thing Greg loved was falling into a programming project. That was the thing, right? But it wasn't clear how to get better at building software.
- [08:08]
Greg Wilson
There were Usenet newsgroups, but there wasn't anything like Stack overflow. There wasn't anything like today's online community. Most people who were learning programming, it was either through direct contact or you'd go and you'd buy a book and hope that it would teach you more than just the syntax of the programming language. I picked up bits and pieces, but by the time I left in 1992, I knew enough to know that there had to be a better way. I felt like I was banging my head against a wall or banging a wall against my head. Some days, everything felt like it was taking a lot longer than it should and it was a lot harder than it should be, and that I was doing the same thing over and over again and not getting better.
- [08:52]
Adam Gordon Bell
Greg knew this feeling because he had been here before as a saxophone player.
- [08:57]
Greg Wilson
I'd go months where I could tell that I wasn't any better than I had been three months ago, six months ago, that for whatever reason I was stuck. I'd plateaued. And years later, I learned some of the science behind learning and teaching. It's hard for people to break out of that on their own because they don't know what they don't know. There's this story from, I don't know, decades and decades ago. You know, man gets shipwrecked on an island and plays chess against himself for 10 years, and at the end of it, he's the world's best chess player because he's had nothing else to do. And it's false. It doesn't work that way. You need other people to knock you out of your ruts, to show you things that you yourself would never have invented or discovered. And there's a handful of people who can bootstrap themselves in isolation. But it is such an extraordinarily rare thing that almost all the time you'll advance some way on your own, and then you'll. You'll hit this point where you need somebody to say, look over there. Try this. Here's another perspective on it. And I didn't really have anybody in Edinburgh who was doing that for me.
- [10:02]
Adam Gordon Bell
But one day something did happen, and it propelled him to new levels of skill.
- [10:06]
Greg Wilson
I guess it would have been 1986 or 1987. Somebody handed me a copy of the book, the UNIX Programming Environment, right? One of that trilogy that Brian Kernighan co authored with other people at Bell Labs. And it's not just about how to use Unix. In fact, that's almost incidental. What it's about is how to think about solving programming problems by putting together pipelines of simple tools, each of which does one job. And I'd been using Unix for six or seven years at this point, but I'd never understood that there was a philosophy behind it, that there was a coherent design. Everything is a file. Even devices are files, even processes are files. They can all be connected like this. There were half a dozen ideas that just fit together, and then you could do almost anything with them. There aren't nearly as many examples of that kind of conscious design thinking in computing as we would like to believe. I actually believe very strongly that the major reason for the success of Unix was those books. Because after you'd read the C programming language, the UNIX programming environment, elements of programming style, you couldn't help but see the world the way they saw it, right? It's like calculus. Once you've seen it, every problem in physics is a calculus problem. Like, you can't unsee that way of looking at the world.
- [11:32]
Adam Gordon Bell
So Greg wanted more breakthroughs like that. He wanted to get better at his craft, not just be a good research engineer or a good computer scientist, but to be a great software developer. And so he headed back to Canada and he got a job at IBM. And IBM was a big deal in the world of software.
- [11:50]
Greg Wilson
Was lucky enough to spend a year working under a guy named Bill O'Farrell. And so this would be 95, 96. C is coming on strong. And he'd been doing object oriented programming since before most people knew the term. He understood not just how to write C, but how to think about breaking your problem up into pieces that could be represented by a class hierarchy where the instances of that hierarchy are interacting in predictable ways.
- [12:24]
Adam Gordon Bell
Do you have an example?
- [12:26]
Greg Wilson
Oh, sure. So we were working on a library for handling different kinds of concurrency at the threading level, at the multiprocess level. And he came up with a really elegant way to use multiple inheritance, so that any particular actor in this system was inheriting some behavior from one parent and some from another. And when you compose them, they would fit together and give you something that would actually run. And thinking about the two or in fact three class hierarchies that you build so that any actual object is inheriting from three different parents and one of them is giving it methods A, B and C. One of it's giving it methods PQ and R. But the PQR methods are using the ABC methods and vice versa and they fit together. So there's a contract between these. It's not a composition relationship. It's actually all married together in one class.
- [13:20]
Adam Gordon Bell
It was elegant, but IBM was struggling, making the job a bit tough. So Greg joined a startup working on an early single sign on tool.
- [13:30]
Greg Wilson
It was a group that had worked together in a couple of previous companies. That's where I actually learned most of what I know about software engineering, about building product, because I was finally with people who knew how to play as a team, play as an orchestra, whatever analogy you want to use. There was a backlog that was regularly grouped. It was the first time I'd ever met a product manager and this guy was going through and talking to users and figuring out how to do the 90 degree pivot from what are the users complaining about to what features do we need to build? And that was the first time I saw somebody who had a systematic process for doing that, for identifying gaps in the market, what is worth building, what might sell, coordinating a team that when I joined it was 10 people grew to the high 30s by the time the dot com crash came. And as with those months that I spent working with Bill O'Farrell, it was like, oh, somebody else has thought this through. Somebody knows how to do this and I can learn from them. And I think a lot of this can't be taught through books because every learner's questions are going to be different. I could talk at you for hours about how to ride a bicycle. It wouldn't make any difference to your ability to ride a bicycle. There's some things you have to do in order to learn because there's so many special cases, there's so many times where the right answer is it depends. But that doesn't scale and it's gatekeeping. If you're growing up in a small town in western Canada, when are you going to bump into people like this?
- [15:05]
Adam Gordon Bell
Greg is a teacher at heart, and for a certain type of person, when there's important knowledge that's hard to teach, that looks a lot like an opportunity.
- [15:14]
Greg Wilson
I got interested in whether it would be possible to teach the things that I'd learned systematically and at scale. Which meant in a classroom. Early 2000s, the University of Toronto asked me if I'd be willing to teach a course on software architecture.
- [15:31]
Adam Gordon Bell
Definitionally, software architecture describes a piece of software's structure and its behavior, its design, its trade offs and how those things might affect maintenance or scalability or whatever. These are the type of things he had learned slowly at IBM, at the startup. These are the type of things that Greg wanted students to learn. He didn't want students to have to get just the perfect job where these skills would be presented via osmosis. He wanted to give people a path to learn these things.
- [16:03]
Greg Wilson
What I hoped I would accomplish was giving 20 year olds the lessons that I hadn't had as a 20 year old. I was trying to figure out some of the stuff on my own. And what's worse, when I was 20, I wasn't even trying to figure it out because I didn't know there was a thing to figure out. I didn't know software design was, was even a thing. But the idea that there's some sort of higher level thinking that can go into what are the pieces, how do they fit together? I'd never seen that. It wasn't something that was taught in classes, at least not the classes I was doing and wind forward 20 years to the early 2000s. Now I understood that this is a thing and I'd done some of it and seen some of it. Okay, can we pass that on? Can we teach that explicitly so that newcomers aren't in the same situation that I had been in at their age?
- [16:57]
Adam Gordon Bell
So Greg started with a literature search and then he collected textbooks that other universities were using for software architecture courses.
- [17:05]
Greg Wilson
I had exactly 12 books with the word software architecture in their title sitting on my shelf. Let's say that each of those books is 300 pages long. So that's on the order of three or 4,000 pages. The thing I did count was how many of them actually described the designs of actual software systems. And there was less than 30 pages, less than 1% of those pages describing real systems.
- [17:30]
Adam Gordon Bell
That's wild. What were they, what were they doing?
- [17:33]
Greg Wilson
Well, there was a lot of UML and design patterns.
- [17:35]
Adam Gordon Bell
Yeah, makes sense.
- [17:36]
Greg Wilson
Yeah, but it makes sense if the books are being written by non practitioners, by people who have jumped straight to the theory but don't yet have a repertoire to base that theory on. And there's a lot of that. Right. One of the reasons the original work on design patterns was so impactful was Eric Gamma and others had spent years building real software systems. So they're drawing on things that they've done out in the real world for their patterns. One of the few good books after that on design patterns was called Design Patterns in Ruby. But every single line of code in that book was taken from the Ruby standard library. He only described patterns by saying, not only is this real code, but it's real code that I didn't write that's in the standard library. And if I can't find an example of a standard library, this isn't the pattern you need to learn.
- [18:29]
Adam Gordon Bell
This is what Greg was looking to teach, right? Look at the great works of those who've built real software. See the decisions they've made, the trade offs they embraced. But he wasn't finding that and so he broadened his search. Meanwhile, he's teaching programming at the university and he's noticing some gaps there too. The students are missing some key practical skills.
- [18:51]
Greg Wilson
I have seen students who were getting A's in senior undergrad courses spent hours well into the wee hours of the morning trying to debug something when other people in the same class with lower grades were solving the same problem in 15 minutes because they knew how to use the debugger in VS code. Very little correlation between grade and tool proficiency. I think there are people who have gotten through undergrad by just working harder with primitive tools. There are other people, because they're sitting beside the right person, whatever it is, have stumbled over a better tool and as a result they're able to get more done in less time.
- [19:33]
Adam Gordon Bell
This is another gap, right? Many students, successful students, just don't have the skills they need to be successful software developers. Greg wants to bridge this gap. But even though Greg was ostensibly teaching software engineering. Right, practical skills, the university wasn't interested that much in practical skills. This was supposed to be the science of computing. Yet in some fields the approach is different. In some fields, practical skills are key.
- [20:01]
Greg Wilson
If you do geology or mechanical engineering, of course there are lab exams. You have to prove that you know how to use the lab equipment. You have to be able to prepare cell cultures to get a degree in biology. You're just expected to be proficient with the tools so that you can do the science. Can we screen record students solving particular problems? And they submit the screen recording and the TA watches it at 10x and gives them a grade on how proficient they are with the tool. Okay, so here's another thing that's missing from undergrad curriculum. I don't know any university anywhere that has an undergrad course on debugging. And yet professional programmers spend 40 to 60% of their time debugging and novices spend 90 or more percent of their time debugging. I write five lines and now I have to figure out why it's not working. And we never teach that, which is nuts. And again, I don't know how to teach debugging in a book because it's all about the how, not the what. It's riding a bicycle. It's a process rather than a thing, like the syntax of Python.
- [21:07]
Adam Gordon Bell
Greg did try to address this. He found someone who could teach debugging.
- [21:12]
Greg Wilson
Mike Conley, who works at Mozilla on Firefox, does a webcast every Wednesday called the Joy of Code, in which he picks a bug or an open issue in Firefox that he hasn't seen before and just dives in, right? And just live casts as he tries to figure out, is this actually a problem? What is the actual problem? Why did this ever work in the first place? How do I track it down to its roots? Like, he's showing people how. You do what I spend most of my programming life doing, which is not writing code, it's spelunking. It's cave diving, right? It's climbing without the safety harness, and then going, oh, why did this ever work in the first place? We don't teach this.
- [21:54]
Adam Gordon Bell
But Greg could teach this. And so he brought and Mike for live code reviews of the Mozilla code base in class. I assume many students wouldn't catch every detail. I certainly wouldn't. But they'd see how to think through a code change. What is the thought process? But still, Greg needed a great curriculum for this software architecture course. And so he searched far and wide. He wanted students to see beyond the code, to understand the design, the big picture. He needed to teach them design.
- [22:23]
Greg Wilson
I was reading a lot at the time, trying to figure this out. And there are disciplines that have a long history of teaching design. And this is one of the things that frustrated me. In your first year in an architecture program at any university I've ever heard of, you will look at literally hundreds of buildings, right? You'll look at photographs, you will look at floor plans, you will look at sketches, you will read notes about the design. You will learn about the history of how those designs have evolved. I don't know of any English language university that offers a course in which you read the great programs of history and take them apart and figure out, why were they built this way and what is this way in the first place?
- [23:03]
Adam Gordon Bell
What do you learn from, like, if I show you Frank Lloyd Wright house, what do you Learn, get out of it.
- [23:10]
Greg Wilson
The first thing you learn is what a musician would call a bag. A bag of tricks, right? Most musicians have a bunch of licks or riffs, patterns that they will play, and hundreds or even thousands of these little things. So they're not thinking in terms of notes or even in terms of chords. They're thinking in terms of passages. They've got larger pieces that they can assemble in the same way that you as a programmer would look at a data structure and say, oh, that's a doubly nested loop. So you're thinking at a higher level than somebody who's encountering loops for the first time and having to step through atom by atom. You're thinking in terms of molecules or organs or entire organisms. So you can search the space of possible designs much faster and much more efficiently because you're searching in bigger steps, you're seeing a bigger picture. And the same is true in architecture. People who have not just looked at enough buildings, but analyzed enough buildings, had their attention drawn to particular features. When the time comes for them to design a building, they'll say, okay, well, we're going to want an atrium. They're not going to start by saying, well, we need to have a door with some glass in it and an open space where we can put a reception desk. And, like, they're not thinking it through from first principles. They've got a much larger set of pieces that they can pick from and assemble, so they can do their design far faster and far more efficiently.
- [24:38]
Adam Gordon Bell
A repertoire.
- [24:39]
Greg Wilson
Yeah, yeah, a repertoire. There you go. Right. The other benefit of it, there's a thing called a studio class in architecture. There's a dozen of us, right? And the assignment is design a bus shelter. Here's a ground plan. Here's where the roads are. Here's a couple of constraints, like how much you're allowed to spend or the weather conditions or things like that. Design a bus shelter. So we all design bus shelters, and the Prof. Goes eeny, meeny, miny, mo and picks you, and you go up to the front and you spend a few minutes presenting your design for the bus shelter. The other students in the class then critique your design. That's not the same as criticize, but they give you feedback. Why did you decide it this way? Have you thought about this alternative? What about this? What about that? Right. The Prof. Then gives all of those students feedback on their critiques. He doesn't talk about your bus shelter. He's trying to teach the rest of the class how to do the equivalent of a code review. How do you give people feedback? What do you look for? I don't know. Any English language university that explicitly teaches students how to do code reviews.
- [25:47]
Adam Gordon Bell
It turns out there is research into teaching students how to critique code, though it's not phrased as MFA style Group Catrice. Instead it's framed as a way to outsource grading.
- [25:58]
Greg Wilson
If you've got an undergrad intro to programming class with 2,000 students in it, spread across 10 sections, you're going to have a squad of grad students who are marking those assignments. If you give any particular assignment to two different grad students, they're not always going to agree on the grade, but you hope they're going to be pretty close and you can actually measure how close they are statistically. It's called integrator agreement. How close do you and I agree on whether this is an 8 or an 8.5 or a 9? So the question is, can you get the first year students to grade each other's assignments as well as the grad level TAs are grading those assignments, it.
- [26:37]
Adam Gordon Bell
Turns out you can, right? You give them a grading scheme, points off for poor variable names, points off for confusing structure and so on.
- [26:45]
Greg Wilson
The grade the undergrad gets is how closely their review matches the one that the Prof. Did of that code. So it turns out that after three iterations of this, each of which takes about half an hour, the undergrads can grade one another's intro Python assignments as well as the TAs were grading. Right. So you can teach people how to do review.
- [27:06]
Adam Gordon Bell
To Greg, it became somewhat obvious that this type of reading comprehension of programs was a great place to start. That's how we develop a taste for good software. Reading code matters.
- [27:18]
Greg Wilson
You can teach musicians how to listen to music, right? That's a large part of an undergrad music degree is what should you be hearing in this song? What are the interesting parts? How is this put together so that when the time comes for you to do it, you'll have a larger repertoire of ideas that you can draw on? Graphic designers do this all the time. Anybody who's designing a UI is drawing on this enormous vocabulary of specialized terms and references and, and ideas. They're not just picking colors and fonts.
- [27:49]
Adam Gordon Bell
The world of graphic design, it's a fascinating comparison. Like architecture, it's all about feedback and criticism and refinement. And they've got this language. They have a way to talk about what looks good. They can point to examples, they can see what works and what doesn't. What's sleek, what's awkward. That's what Greg wanted for software design. He was after a library of examples to pull from, and he was frustrated that it didn't exist.
- [28:15]
Greg Wilson
And it annoyed me so much that in the spring of 2006, I hunted around on the Internet. I got email addresses for 300 people, and I sent mail to them one at a time. I spent a whole weekend, a long weekend, actually firing off emails. Hi, my name is Greg Wilson, and I'm putting together a book for O'Reilly called Beautiful Coat. Now, I hadn't told O'Reilly this yet, but I was pretty sure that we could connect the dots, right? What I want from you is a chapter. Pick the most beautiful piece of software you've ever seen and just write me a few pages about what makes it beautiful.
- [28:54]
Adam Gordon Bell
What Greg needed was examples to put in front of students to build up their taste. And to him, emailing practitioners was a great way to avoid a very specific trap because he needed diverse and real world examples.
- [29:08]
Greg Wilson
I didn't want to go out and pick them myself. I wanted people who had built real code to come back and show me things that I would never have stumbled over by myself.
- [29:18]
Adam Gordon Bell
The list of authors in Beautiful Code reads like an early 2000s software design hall of Fame. Brian Cantrell on the Solaris kernel, Carl Fogel on subversion, Michael Feathers on building a testing framework, and so on and on for 33 chapters.
- [29:35]
Greg Wilson
Another contributor wrote about a set of FORTRAN libraries called blast, the Basic Linear algebra subroutines. And his argument was, this code is now almost 30 years old and is in daily use and is the workhorse underneath almost every piece of scientific software on the planet. As far as he was concerned, something is beautiful if it's been so useful for so long. The beauty comes from the utility. And this has proven its beauty by being useful.
- [30:04]
Adam Gordon Bell
Yeah, the farmer who loves his old tractor that has just been going and going.
- [30:09]
Greg Wilson
Classic jeeps from the 1940s and 1950s that will not die. The machines back then, and they were ugly and they were loud and they polluted, but actually watched people put chains and haul a jeep out of a river where it'd been sitting for a couple of years, take it into the shop, and a week later it's back on the road. We had people writing about just a whole variety of things, and it was a great book. I mean, it was a really enjoyable book. We raised almost $100,000 for Amnesty International because rather than divvying up author royalties, 36 Ways We Just Said, send the check to Amnesty. But it wasn't any use for my intended audience. Undergrad students couldn't make sense of it because there were too many different programming languages, too many different levels, too many different perspectives. You had to have already been working in the field and have seen these issues to appreciate it.
- [31:10]
Adam Gordon Bell
That's where software architecture and talking about beauty seems the most challenging, right? A beautiful building. You can go over all the details of why it was built that way, what the constraints were, but also you can see it all at once. A software design needs thought. It's like a dense poem. You need to mull it over. You need context to see that beauty. So anyhow, by then Greg was in his third semester of teaching this architecture course and he sort of thought this gap was just too large.
- [31:45]
Greg Wilson
And I told the department to cancel the course. This would have been about 2003, 2004. It was pretty clear after three and a half years that academia wasn't for me.
- [31:56]
Adam Gordon Bell
But all was not lost. This book became a hit. And even me, the no code review me from back in 2009, even though there wasn't much design talk at work, I did have a copy of this book and I did read most of it and so did many others.
- [32:13]
Greg Wilson
I do remember being completely convinced by this exercise that there was a hunger for this kind of thing that I think Scott Hanselman talks about dark matter developers. Have you ever heard this phrase?
- [32:30]
Adam Gordon Bell
I think so. Just they're. They're not visible, I guess, or something.
- [32:35]
Greg Wilson
Most of the matter in the universe is invisible to us. It doesn't radiate, as far as we can tell. All it does is interact with gravity. We can tell it's there, but we can't see it. And Hanselman's point 20 years ago is that 90 plus percent of programmers never post a question on Stack Overflow, much less answer. When they don't have public GitHub repos, they don't have personal websites. The few percent that are radiating information are atypical, but they're the only ones we can see. So beautiful code convinced me that there was hunger for learning about software design among dark matter developers. It wasn't just the few who show up to conferences and have blogs.
- [33:16]
Adam Gordon Bell
So although his approach to teaching via real world examples didn't take off in academia, it seems to have been soaked up by real world software engineers. And maybe that's been Greg's biggest impact on the world. And then he created more books, books that followed it, because he kept going with this idea and we'll get to that shortly. But actually, there's another side to Greg's journey, another adventure in teaching. And in some ways, it's where his impact on coding has been the largest. It started back when Greg was a research engineer, back when he was in Scotland. He was reading Brian Kernahan's books, and he was thinking about that magical place in New Jersey. He's thinking about Bell Labs.
- [33:55]
Greg Wilson
I think we sometimes mythologize a few places and moments. Bletchley Park, Bell Labs, Xerox Park. But on the other hand, for the space of a few years, a few people working together can change the course of an entire industry. Right? They really did push us in a whole new direction. I don't think they could possibly have realized at the time just how big an impact they were having, but, man, most of the programming I've done over 43 years has basically been footnotes to what they did at Bell labs.
- [34:27]
Adam Gordon Bell
In the 1970s, Greg had had this Unix programming epiphany, and it frustrated him seeing smart scientists miss out on all these tools. A lot of research involves crunching data, and researchers were being held back by a skills issue. So he spread the word, crawled a.
- [34:45]
Greg Wilson
Bunch of people, and we wrote a couple of journal articles basically saying, look, scientists are taking weeks or months to do things that they could do in hours or days because nobody's shown them that there's a better way. There's a tool to do what you're doing by hand. You can write macros for editors, right? Learn how to do that. Regular expressions are useful when you're dealing with messy scientific data files. Here's how they work. None of this is rocket science to me, but it's unfair to expect somebody who has spent years learning about quantum chemistry to somehow magically know about command line pattern matching tools.
- [35:23]
Adam Gordon Bell
Greg actually went to a chemistry professor, and he told her, you know, chemistry department needs to start teaching data processing. They need to start teaching computer programming. And not just chemistry undergrads, but geology students and others as well.
- [35:37]
Greg Wilson
And the professor I was talking to finally lost her temper and said, okay, so what do we take out? Do we teach them less organic chemistry or less thermodynamics? And she was right. Every undergrad science and engineering program is bursting at the seams, and they're still only scratching the surface. You've got four years. You can't teach people more than 1% of modern geology in four years. And now I'm telling you that we're going to take away 10 or 15% of that to teach them programming. I understand the pushback, and I think they should learn the programming, but I understand why people are saying, great. It's not enough to say we should add X, you've got to tell me what to take out. It's like budgeting discussions in government. Everybody's got something they want to spend money on. The hard part is, what are we not going to do so that you can have your road or your school or your hospital or your tax cut? That's where the rubber really hits the road.
- [36:29]
Adam Gordon Bell
So Greg refined his thoughts. Maybe not undergrad classes, but what about programming boot camps for grad students? He jotted down his ideas, and the IEEE published them.
- [36:42]
Greg Wilson
And John Reinders, who was running the Advanced Computing lab at Los Alamos, reached out to me and said, put up or shut up. If you think that teaching this stuff will make a difference, I'll give you a couple of chances to do it. You put together a workshop, you come here, you teach my physicists these things, and we'll see if it makes a difference. I gotta tell you, you kind of feel like the elevator dropping underneath you a little too fast when somebody says, all right, all this griping you've been doing, show me. So, yeah, you know, you think you can make a better chili? There's the kitchen.
- [37:16]
Adam Gordon Bell
Los Alamos is no joke, right? This is the location of the Manhattan Project. And Greg's basically told them, you know, they'd be better at their job of particle physics or weapons research or weather simulation if they knew some Python, if they knew some command line tools. But, I mean, he sort of had a point. And so he and Brett Gorda went down there to teach a workshop.
- [37:38]
Greg Wilson
You got a classroom with a bunch of people who are physics, geology, chemistry, pure mathematic, some of them my dad's age, some of them quite skeptical, because if you've worked in a large institution for any number of years, you've seen a lot of bandwagons roll through. And anytime your boss says, here's this training course, your first reaction is going to be, if I just sit here quietly, eventually this will just pass me by and I can get back to work. But there were a lot of people then who really did want to improve their skills so that. So they could get more done in less time and with less pain. A lot of them, they would have had somebody rooming with an undergrad who did computer science. So you can see that they're somehow able to do this stuff with a couple of lines on the keyboard that's taking you forever to do in whatever WYSIWYG tool you've got or using Fortran and compiled code. So you're open to the idea that better is possible. And that's often the hardest thing with learning, is to convince people that improvement is actually possible.
- [38:41]
Adam Gordon Bell
The first thing Greg would do is teach the basics of Python. These researchers were used to working in Fortran. This was a big change. One memorable group of three had been coding together since the 1960s.
- [38:53]
Greg Wilson
They had a code base that was over a million lines of Fortran. And they weren't allowed to tell me what it simulated, but they could tell me that what it simulated was really, really loud. Right?
- [39:04]
Adam Gordon Bell
So Greg did his workshop at Los Alamos and he did it a couple of times and he refined it.
- [39:10]
Greg Wilson
This was and is a struggle. If you've only got a little bit of time, what do you teach? What's going to have the most impact on this audience? I eventually realized that what I was doing was the equivalent of teaching first aid. My sister in law is a paramedic here in Ontario and their approach to care is based on the same science as what you get from your GP if you walked into their office. But they're coming at it from a very different angle. My sister in law will say that she does everything a doctor does, except she does it in a ditch in the rain and she's okay if need be with not being in sanitary conditions. Her job is to make sure you don't die and keep you stable in the ride back to the hospital when you become somebody else's problem. And what you teach somebody who's going to be working under those conditions is very different from what you teach somebody who's working in a well funded hospital where they've got a supply closet right there with everything that's ever been invented and they've got a reliable power supply and they've got three people who can step in if they need a spare pair of hands. Figuring out which bits of what you and I do on a daily basis are worth teaching to a data scientist or to a physicist or to an economist is a computer science problem departments don't think about because that's not their audience.
- [40:29]
Adam Gordon Bell
A lot of this, like first aid was basics, but very crucial. How to use source control, how to use Python for quick data scripts instead of Fortran. Turns out if you're a meteorologist doing weather simulations, some software engineering, some shell scripting, a little bit of Unix philosophy can be quite empowering. So the workshops were a hit and soon, other research institutions wanted in.
- [40:53]
Greg Wilson
Not long after that, I taught one of these workshops on my own at NCAR national center for Atmospheric Research in Boulder, Colorado. And partway through the first afternoon, I just, as an aside, mentioned profilers. And one of the people in the course came up to me at the break and said, what is this? And I explained that you can actually, with a couple of compiler flags, you can get the program to record where it's spending its time so that you can see what you need to optimize. They'd been banging their head against this problem, manually inserting timing calls and logging stuff for weeks, trying to get the data they needed to improve the performance of this really big piece of weather modeling code. And I had just told them the compiler has that as an option. And so they just left my class and they came back about an hour later and they're like, you've just saved.
- [41:44]
Adam Gordon Bell
Me a month by introducing these researchers to the power of profilers. He didn't just save them a month of work, right? He gave them new skills. And skills compound across all of your future work. Greg's powerful insight was that software engineering skills could expedite research, potentially accelerating scientific discovery itself. The ripple effect of this knowledge transfer is immeasurable, and it probably helped increase our understanding of the world. And it also helped, in a small way, I guess, to make Python a fixture of scientific computing. But yeah, Greg eventually left this behind, right? He did the IBM thing, he did the startup thing, but this idea never left him. This idea of computing as first aid. He sometimes thought of it like teaching people to build sawhorses.
- [42:34]
Greg Wilson
I don't know if you've ever done any serious carpentry, but you see a house builder show up, the first thing he does is he makes himself a couple of sawhorses, right? Five pieces of wood, nailed them together. Five pieces of wood, nail them together, you got a couple of sawhorses, you're probably going to leave them behind when you're done the job, right? There's a lot of sawhorses in computing.
- [42:54]
Adam Gordon Bell
And then after his startup time and after his somewhat failed attempts to teach software architecture, he thought maybe he should revisit that because it had had a big impact and he did like working with scientists.
- [43:07]
Greg Wilson
It's more fun to work with scientists than it is to work with many or even most of the people you will bump into in Silicon Valley, because scientists are motivated by the problem they're trying to solve. You don't become a chemist or physicist unless you love chemistry or physics.
- [43:26]
Adam Gordon Bell
And so Greg took his workshop curriculum, he put it out there under an open source license, and he started a nonprofit software carpentry, and just started out on the road teaching grad students how to build sawhorses.
- [43:38]
Greg Wilson
So many grad students had seen a bit of programming in first year, typically Intro to Java course that had nothing to do with the kind of problems that they were going to tackle. Maybe had pushed around a bit of Sass or SPSS in a stats course. And now here they are in grad school and there's a terabyte of data sitting between them and their thesis. So teach them a bit about the UNIX shell so that they can start doing things with scripts instead of with GUI tools, so that they can automate those things. Teach them a bit of Python and later a bit of R so that they understand that they should break their code up into functions that they can reuse in the test. Teach them version control, teach them how to write, test it. Fairly simple things, but life changing for a lot of the participants compared to.
- [44:28]
Adam Gordon Bell
His original goal of raising the level of discussions around software design and architecture, pushing the field forward compared to all that. This knowledge he taught grad students was quite basic. But a lot of times, like first aid, the basics can be life changing.
- [44:44]
Greg Wilson
I was teaching a workshop once and teaching people the basics of the UNIX shell. And one of the things that we show is how to write a three line shell script that's basically a for loop for each file in this directory, process it the following way. And one of the women in the class started to cry. She was a grad student. I found out at the coffee break when I went and spoke to her about this, that every evening for about a month she'd been coming into the lab and she'd go to the first machine, type in the command to process file number one, which would take about 20 minutes. Then go to the next machine, type in the command to process file two, right? Six machines, get them all running, sit there and knit. When they were finished, she'd do a second batch, when they were finished, she'd do a third and then she'd go home. She had over a thousand files that she had to process and she was treating it the way you would treat cell cultures or cell assays is you just got to put in the hours. She wasn't crying because she had just realized that she could have written a three line shell script and leave it run over the weekend and it would be done. She was crying because she had just realized from the comments that the Two guys sitting beside her were making to each other that they had known for a month what she was doing, that they knew how to solve it, and that they hadn't told her because they thought it was funny.
- [46:01]
Adam Gordon Bell
This was another part of the whole teaching thing, right? When you teach knowledge openly, you knock down barriers, you knock down gates. No one can hoard knowledge out of spites or misogyny or whatever. And Greg's teaching was hands on. No slides, just him and his computer showing people how to do things. It's really the kind of stuff you would have seen if you'd been sitting next to him back when he worked at the High Performance Computing Place. But now it's in a classroom and he's breaking it down step by step.
- [46:32]
Greg Wilson
For most workshop participants, the single biggest thing they got out of the workshop was the belief that this was something they could do. For so many people, their earlier experiences of programming had been so off putting, so confusing and belittling, that they had just come away believing that whatever the gene is, I don't have it. And that's why I teach. It's so empowering for people to give them the tools so that they can go and build their own sawhorses so they can put up their own drywall, so that they don't have to wait for a contractor to become available to fix a leaky faucet. We called it software carpentry because it isn't software engineering. Most of us aren't designing a hydroelectric dam, but we should all know how to do a few home repairs because it puts the power back in our hands. And teaching people how to design software means that they can finish that and then go back to whatever the original problem was. And I don't know who first said it, but, like, the only reliable way to be a 10x developer is to go and find 10 people and make them each twice as productive.
- [47:44]
Adam Gordon Bell
So Greg helped advance the careers of many scientists, and probably their research too, by teaching them Python and Unix and tool usage. And on the software design and architecture front, Greg is still trying to push the ball forward. He put together the architecture of open source applications in 2011. He did volume two of it. In 2012, he wrote Making Software what really Works. In 2010, he did a book, Software Design by example in JavaScript, targeting students. That was in 2022. And then he did Software by Example in Python, which just came out in 2023. So he's trying to push this forward. But Greg still thinks we have a long ways to go before we can talk about software design effectively as a field, but he's trying, right? He doesn't want others to waste 10 years figuring it out alone. And he's ready to share what he learned from Bill at IBM, from his startup time. He wants to pass it on, he wants to make it simple. But most of all, I think he wants us to be able to talk about the approaches we use in various real world software, right? Why is LLVM more modular than gcc? What's the trade off there? What are the design trade offs of matplotlib or of rstudio? Can we actually talk about the trade offs between, you know, having services and having monoliths, instead of just dunking on one being better than the other?
- [49:13]
Greg Wilson
If we put the right examples in front of people and draw their attention to them, they don't have to spend years wandering around in the forest wondering if there's a road somewhere, right, that feels worthwhile.
- [49:27]
Adam Gordon Bell
These books, the Architecture of Open Source Application series, now four volumes strong, they're catching on. They start discussions, chapter by chapter of what makes for a great design, what a real world code base looks like, what the constraints are, how you trade them off. They give us the vocabulary we need to talk about these things.
- [49:49]
Greg Wilson
If you listen to two undergrad architecture students having a conversation about a bus shelter, they're talking at a level so much more refined than the level that even experienced software designers can talk about software, because they've got centuries of insight and critique and vocabulary to draw on. They've got the terms and the ideas. Industrial design is an interesting field here. It was consciously and deliberately created in the 1910s, 1920s, 1930s. People like Loewy were trying to convince society as a whole that mass produced artifacts can be considered aesthetically. There was this feeling then that if it's mass produced, it might be useful and cheap, but it's not art. And people like Raymond Loewy were saying, no, no, there's another category here. There are things that we mass produce but are nevertheless beautiful. And they won. People today will take iPhones seriously. They will take the design of a set of speakers seriously from an aesthetic point of view, not just an engineering point of view. But it took several decades of essays and arguments and public talks and university courses and creating a profession to get society to believe that handmade and beautiful weren't necessary correlates of each other. And if beautiful code and the architecture, software, open source applications and other things that I've worked on are one small step towards starting a conversation about us Being able to talk about the aesthetics of software in the way that the UX designers we work with can talk about the aesthetics of an interface, then that was worth doing. But that discussion hasn't even started yet. You know, design patterns from the early 1990s are still as far as we've gone in terms of having a vocabulary for talking about why NGINX is more elegant internally than Apache.
- [51:54]
Adam Gordon Bell
Part of the problem, Greg suspects, is that there's sort of an implied hierarchy about what's valued in our field.
- [52:00]
Greg Wilson
Programmers look down on UX design, they look down on graphic design. And yet those are the people that we work with most closely who could start to tell us how to take design seriously and how to look at it, talk about it and think about it. There have been essays recently about why programmers don't take CSS seriously. And the answer is because design is seen as a feminized profession. You take a look at a startup, the programmers are guys, the designers are women, and the social status that comes with male, female distinctions, pink collar jobs, things like that. Designers are over there writing sonnets to each other, and we programmers, bang rocks. Bang, bang, bang, rocks. Good, right?
- [52:50]
Adam Gordon Bell
All is not lost, though. Beautiful Code might have been too advanced for undergrads, had too much implied context. But you know, I think it's really the discussions that Greg sparked among his book's contributors that really pushed the field forward, because that was the first step to a world where we can discuss software design at a higher level. And that's Greg's legacy. By championing the use of real world examples in software education, he's bridging the gap between theory and practice, giving a glimpse of a world where, you know, in PRs we can have real discussions drawing on common examples of real world software and the trade offs that they have. That would be progress because there was a time when even talking about design, when talking about code being beautiful might lead to eye rolls. Here's an example. Back when Greg sent out those 300 emails for beautiful Code, the first response he got back was from Rob Pike. And if you don't know Rob, right, He worked on UTF, he worked on Unix, he worked on Go and Plan 9. He's exactly the type of person that Greg wants us to learn from. Whatever Rob knows, can he teach us? But here's the gap in our language or in culture. Because what Rob said to Greg was he'd never seen any beautiful code. Beautiful code wasn't a thing. Greg was, I assume, saddened by this, but he shared his response with Brian Kernahan. And Brian wrote the first chapter of Beautiful Code about some code written by Rob Pike. It was this simple regex matcher written in C. Brian spent a chapter talking about it. You see, Rob spent time polishing that code until, according to Brian, it's some of the most beautiful code he's ever seen. And if you read the chapter, you get to see how Brian thinks about design and what makes that code beautiful.
- [54:43]
Greg Wilson
You look at it and you go, well, of course it has to be like that, right? Like, that's elegant, that's really cleanly put together. There's no wasted motion, there's no wasted parts. It does exactly what it's supposed to. I couldn't add anything. I couldn't take anything away.
- [55:01]
Adam Gordon Bell
And also by writing that chapter, Brian was in effect saying, you're wrong, Rob. Code can be beautiful and you're actually one of the greats. And here's the thing, here's the thing that kind of blows me away. About to get a little excited in his analysis. He sounds a lot like what had blown Greg away back at IBM all those years ago, right? Once you consider the solution, you see the beauty in it. Because Beautiful Code doesn't hit you at a glance like a beautiful sunset. It requires thought. You work to understand the code. You dig in, you consider the various alternatives, and you realize, if I had solved this problem a hundred times, if I had solved it in 10 different ways, this solution would be the clearest. This solution is the cleanest. That was the show. Big thanks to Greg. Find him@thirdbit.com, find him on LinkedIn, find him v Wilson on Mastodon, or check out his books and the 30 other things that he's constantly up to. He's still working with scientists, he's still trying to push forward the field. Amazing person. And if you like the podcast, you might like my newsletter where I cover similar topics. Or you might like to follow me on Twitter AM Gordon Bell where I try to share behind the scenes details. And for true fans, the best thing you can do to support the podcast is go to co recursive Comm slash supporters. There's a link in the show notes and join as a podcast supporter, you'll receive access to bonus episodes. Lately, what I've been doing is kind of reflecting on the themes in the show and one of the supporters, Patrick CL Loyal supporter. Thank you. Patrick said, I really like the format you've been doing with the bonus episodes where you expand on things from the main episode and reflect them upon your own experiences. So if that sounds interesting or if you just want to show your support, please check out the supporters community. And until next time, thank you so much for listening.