On today’s podcast we have Eric Holscher joining us again – this time to talk about documentation. Eric is the co-founder of Read the Docs and Write the Docs and his mission is to elevate the status of documentation in the software industry, so there is no one better to address this topic! He talks about getting started with writing documentation and developing a culture of sound practices around the process and then he shares why updating your docs are so important, despite it often being neglected. We also have a conversation about why it is necessary even for self-documented code, what the various functions are, and later Eric advises on some of the best role model products when it comes to documentation. Be sure to listen in!
Key Points From This Episode:
Transcript for Episode 116. Documentation Hell and Documentation Heaven with Eric Holscher
[0:00:01.9] MN: Hello and welcome to The Rabbit Hole, the definitive developer’s podcast in fantabulous Chelsea Manhattan. I’m your host, Michael Nunez. Our co-host today.
[0:00:09.8] DA: Dave Anderson.
[0:00:10.8] MN: Today, we’re talking about documentation, heaven and horror stories.
[0:00:16.3] DA: Man, I know the horror of documentation for sure.
[0:00:21.4] MN: Of having them or not having them?
[0:00:23.4] DA: Who can tell us what heaven looks like with documentation?
[0:00:26.5] MN: Yeah. We have a special guest, a returning guest, we have Eric Holscher. How’s it going Eric?
[0:00:33.1] EH: Good. I don’t know if I can quite bring you to salvation, but I’ll do my best.
[0:00:39.2] DA: We spend so many years in the dessert, you know? Got to be a Promised Land somewhere.
[0:00:45.9] MN: Somewhere.
[0:00:46.8] DA: Yeah, why don’t you tell the people about why you’re qualified to help us out here?
[0:00:53.3] EH: Sure, yeah. I think the best hat for me to wear today is actually the cofounder of the Write the Docs conferences which is a kind of global conferences community which is a global community built around software documentation, inclusive of developers, tech writers, port folks, kind of everyone who cares about documentation.
[0:01:10.2] DA: Yeah, I’ve met Mikey Ariel, we had her on the show actually back in – a friend of the show, Mikey Ariel.
[0:01:16.1] MN: Friend of the show?
[0:01:17.4] DA: Representing in Prague.
[0:01:19.4] EH: Yeah, she’s the chair of our Prague conference, I’d like to say she’s the me of Europe.
[0:01:24.9] MN: Nice.
[0:01:25.8] EH: Maybe I’m the “her” of America, another way of putting it.
[0:01:29.6] DA: Columbia.
[0:01:31.5] MN: Talking about documentation, depending on whatever tech shop you may work at, could be hit or miss on whether on the belief of documentation. We work at agileBase companies and I believe one of the tenants of agile, right?
[0:01:52.0] DA: The manifesto, as it’s called.
[0:01:53.6] MN: Right, one of them is working software over comprehensive documentation. That means we don’t write any documents, right? Well, we don’t write any of them.
[0:02:03.7] EH: That’s what I heard.
[0:02:04.9] MN: We don’t write any of them, if we use agile.
[0:02:11.1] DA: It is a terrific statement. Because my story with documentation, personally is like, I worked in waterfall environments that were regulated that required lots of documentation and forethought. Upfront about requirements and design and diagrams and plans.
[0:02:28.4] MN: That documentation is different than like tech documentation that Eric was referring to in his conferences. Is that correct to say?
[0:02:35.2] EH: I would say so. I mean, I think there’s a small overlap when you think about like design documentation and kind of justification for systems but I certainly think contextually, a lot of people take the job manifesto out of context and use it to say no documentation and I have an excuse now where as yeah, I think it’s much more referring to the classic kind of waterfall model of we’re going to write 50 pages before we write a line of code.
[0:03:02.2] MN: Yeah.
[0:03:04.2] DA: Right, we’ll think about the code really hard.
[0:03:07.6] MN: As we write it in regular, in a document.
[0:03:09.7] EH: Yeah, it’s interesting because I feel like this is maybe come back a little bit around to – there’s some stuff called Readme driven development or documentation driven development. I feel like that’s the kind of modern take on it but it’s really writing documentation about the code, much more so than about the design of the system or anything like that. It’s an interesting parallel but I think it leads to much better results.
[0:03:34.6] DA: Right, yeah. I guess the tough thing with things like requirement or design is that the reality will often hit and then the half-life of that documentation’s usefulness might soon be out the window.
[0:03:50.4] EH: Yeah, with Readme driven development, a lot of what I’m doing is writing the kind of EPI interface or the user experience that someone who is using my class is going to have, it’s a much more kind of code focused and thinking about the design of the system. I really don’t think that’s what kind of the classic agile manifesto is referring to when it’s talking about documentation.
[0:04:10.5] DA: Yeah, that’s true. I recently, just as a little fun exercise, I wanted to see if it was possible to write rspec style mock setup in Python and so I kind of followed that a little bit because I was just reading the Readme of rspec. Just trying to write the software from that user interface and it was kind of interesting like kind of just following from the API and letting like the code underneath it be less important than the assertions that it meets the spec of the document.
[0:04:43.9] EH: Yeah, totally, it’s an interesting approach and I think it leads to better kind of user experience for the software to think about how it’s being used at a high level. I find often I’ll just start implementing and my implementation will dictate what the public API looks like.
If I do that design upfront, it will generally be better design of the code and the public API.
[0:05:05.8] DA: Yeah, it’s kind of like the same philosophy as like test driven design.
[0:05:11.3] MN: I have a question though, suppose I am a shop, I am a development shop and I now at willing or wanting to write documentation, how do I start and what are best practices that I can introduce going forward?
[0:05:25.1] EH: Totally, yeah, I can answer that a couple of ways so that I’ll focus on kind of one, which is called kind of the doc code methodology. What that really talks about is treating documentation like code and so there’s a couple of kind of avenues to that.
One is the documentation should live as close to the code as possible. Think code comments, think restructured text to mark down files in their repository and really what that gives you at the end of the day and the way to kind of build documentation culture within a company is to then start requiring documentation updates along with the code.
You can have the literal pour request where the feature is being built, will also contain the documentation for that feature. At least he developer documentation, maybe not your product docs or whatever. They’re really kind of building engineering or documentation culture and engineering teams is that the closer you can look like to code development, the more integrated it can be with the development process. You know, the more likely it will be to get updated, the more likely it will be to correct and get done at all, really.
[0:06:30.6] DA: I really like the idea of thinking from the user’s perspective and documenting that interface, especially with like API’s, it’s really nice to have something like Swagger where it will cover what your rest endpoints do and tell you how to interact with them.
That really does fall down if your team doesn’t have the discipline to not accept the work if you change the interface of the API and have someone go back and make sure that it’s up there properly.
[0:07:04.5] EH: Yeah, I think there’s a good analogy that I hear that I like to talk about which is a lot of these were the same complaints that automated testing had 10 years ago. I actually kind of started my career advocating for testing and then that kind of became a thing that everyone advocated for, so I did move on the documentation.
[0:07:24.3] MN: Just became too cool.
[0:07:26.7] EH: Yeah, exactly. I couldn’t get easy talks at conferences anymore just by proposing, you know, testing talk so I had to move on to docs talk. No, but I really think the advent of continuous integration is what made automated testing go from a net negative to a net positive, right? It’s that kind of reinforced value and kind of moving the value of writing the test closer to the act of writing the code. That’s really what I advocate for with documentation, that’s what Read the Docs does.
It’s just a product that I work on. It’s basically CI for docs, right? Whenever you verge the internet, it really like, pushes the value of writing docs closer to the time you're writing the code. Whereas if you’re doing doc updates and they only go live every like two weeks or whatever when a new version goes out, it tends to go stale, it tends to not to have as much value and so I do think there’s an analogy there between kind of CI for testing, adding value and then something similar for documentation.
Making sure that that’s getting to the users as quickly as possible.
[0:08:32.3] DA: Really having that feedback loop where you’re just continually seeing a nice thing. There’s a treat every single time that you write your document.
[0:08:43.7] MN: Yeah.
[0:08:44.0] EH: A lot of times, yeah, you’ll see a lot of people who document stuff in Confluence or something and they’re like, we’ll update the docs later and you know, tomorrow never comes kind of scenario where the docs never get updated and so keeping it close to the code and then keeping it kind of live updated. I think really increases the value you get out of the docs because yeah, the value degrades over time so if you write something and it doesn’t get shipped to the users for months, maybe the life that it’s valuable is only two months.
I think the making sure you’re getting the most value out of it is really how you make it worth writing.
[0:09:20.3] DA: I tried to write or contribute documentation for a long time but I found that the feedback loop would be challenging if I don’t own the dock server, you know, I’m not accepting the PR and I’m just working in a vacuum, it really helped to look at like an auto building tool where it’s doing live reloads of documentation as I’m going.
You can really tighten that feedback loop even to like the writing aspects of it.
[0:09:52.2] EH: I do think the other big thing there is you know, similarly with code comments, making sure that the comments in your code are getting rendered in your documentation and a good way is the other place where the common in the code, just the most likely to get outdated when the code changes.
Then an external file in the same repo and then something that’s external and completely to the code, right? You’re losing the chance of something getting updated with the code the further it lives. Making sure that that’s right next to the code is being shown to users in a valuable way is also another good place to make sure stuff gets updated.
[0:10:28.7] DA: Right, those random docs and Google Drive or Confluence are just like in the seventh circle of hell. Never getting updated.
[0:10:37.4] MN: Gentlemen. I want to say something. My code is self-documented. Why do I need to write documentation on my self-documented code?
[0:10:48.4] DA: It’s beautiful, it’s like a high Ku.
[0:10:49.8] MN: You can read it and you totally understand it, there’s no little – it’s just right there and it’s always documented for you in my code, why do I have to take the necessary step to write documentation when it reads so beautifully.
[0:11:05.4] EH: My favorite answer to this is actually a rhetorical question and that is, where does the tutorial go in your code?
[0:11:12.0] DA: The tutorial for my code?
[0:11:17.0] EH: Right, how do I learn to use your code? Or you know, similarly, kind of where do the install instructions go in your beautifully crafted piece of software? Kind of the point there that the more serious point is, you know, documentation has many possible uses and the kind of my code is self-documenting thing. Really is just saying that you can read the code and understand how the system works.
[0:11:45.6] MN: Right.
[0:11:47.6] EH: The how is answered in the code but the other two big things, the biggest in my mind is the why, right? A canonical example of that is I made the system one way and I had to change it because the old way didn’t work. A comment in the code is a great place to store that information for the next person so they don’t try to update it, right?
I do think the how can be quite valuably communicated in the code but they why and also just kind of the myriad other things that shipping software needs, you know? A tutorial to get used to it, installations instructions, that is also very important to documentation, if you want your software to be used by other humans. Yeah, if your software’s life is just to be written in red by you and executed only by you then that’s great, right? But once you actually start having other people who are using your code, the value of documentation becomes quite obvious I think in my experience.
[0:12:41.7] DA: We did an episode, number 56, comments considered harmful and comments considered helpful and I really love some of the examples of like comments that were pulled out of the Microsoft Windows or Word codebase where it’s like every time you try to refactor this method and you think that you are smarter than I am, please add the number of hours you spend on this here because there is a reason why something may be insane looking to you that may not be contrary.
[0:13:16.0] EH: A lot security stuff as well, right? It’s like this log in function is incredibly inefficient for a reason.
[0:13:25.3] DA: Right, yeah and some of that documentation may exist inside of your social control too. The why and the existence side of the commit message and the history associated with that but there are some things that need to be a little bit more front and center for sure.
[0:13:42.9] EH: Yeah, definitely and really I mean this isn’t the best way to explain it to developers but the documentation truly is marketing for your software. If you are writing an open source code or maybe another hypothetical is good here like, how many people go to a GitHub repo and they see the projects has no Readme and they’re like “I am going to use that code.”
[0:14:05.1] DA: Right, zero stars, zero Readme, yep that is what I am going for.
[0:14:10.0] MN: That’s the one fellows, that’s it.
[0:14:13.0] DA: Although I do like it when the code actually can be self-documenting like I really enjoyed working on GraphQL APIs over the past year or so or more in like when you write code in GraphQL for your API then you get this nice introspectable schema that through tooling can be a document and can be interactive and can be the marketing page for you wonderful API to all the users be they internal or the external. So yeah, trends towards that like get me pretty excited.
[0:14:51.5] EH: Yeah, I know and I mean I think it is a false equivalency or false dichotomy right? Code can be “self-documenting” but still need written words but that people who are arguing for documentation are arguing for less readable code.
[0:15:10.3] DA: Right.
[0:15:10.9] EH: It’s like we are arguing that you need to solve more problems basically.
[0:15:17.3] DA: I never thought about that but yeah, I do feel like it’s like, “Yes! Why would I write less readable code? Why are you telling me to write less readable code? I am going to write the best possible code.”
[0:15:27.9] EH: And then you have to tell people how to install it.
[0:15:32.0] MN: Yeah because that is important because you want people to use it. The best way to do that like as you mentioned is not someone who is going to sit down and read the lines of code necessary to run it. You want to be able to have like an introduction to the code kind of like the table of content of your code is the Readme that you need to read and you’re like, “Oh okay, I have an idea on how to use this.” If I want to learn more about it, I can read the code.
Which is great but those synopses of it could be in a documentation that exists alongside your code. So just to clarify, this isn’t like commenting code. Excuse me, having comments in code that’s like super-duper long worth’s of paragraph where it makes my file, it makes the file of the code like 300 lines because it is riddled with comments like this will not work you know?
[0:16:23.0] EH: Considerate programming I believe is a new idea there.
[0:16:27.1] MN: Yeah, so I don’t think that it is that and when you actually mentioned documentation alongside your code but more like an introduction to it –
[0:16:33.4] DA: You know that comment that says, “Calculate the velocity and then velocity equals…” you know whatever.
[0:16:39.9] EH: That is documenting the how. It’s not what you want to do or yeah, like I know what the functions does. Tell me why the function exists, like give me a little existential crisis for yourself.
[0:16:49.9] DA: Right like why is this pizza delivery app calculating velocity? What is going on here? Are you shooting pizzas out of a trebuchet or something?
[0:16:59.1] MN: Hey, whatever it takes to get it to your house as hot as possible, right?
[0:17:04.0] DA: What are some good role models for documentation that I should look to? Who is doing it right? What projects have some braggable documentation that I can get inspired from?
[0:17:17.4] EH: Sure, I will be a little cliché and name a couple of the obvious ones. Stripe is one that people always mention. I think their innovation was kind of the living documentation or updated thing that the magical saying that happens when you go to Stripe and you can copy and paste the code examples and they have pre-sold out your API key. So when you run the code examples from their docs, it shows your data. That is a pretty cool tip of the hat.
[0:17:46.6] DA: And making your life as a developer a lot easier like thinking about the end user.
[0:17:53.8] EH: You know it is a great user experience thing. If you treat documentation as a product that is a UX win for that product.
[0:18:01.4] DA: You know I haven’t looked at the Stripe docs and I am looking at it right now and it is pretty slick. There is animation going on and things are moving and it’s pretty cool.
[0:18:12.3] MN: That’s awesome.
[0:18:12.8] EH: I mean they have a team of people who treat documentation like a product and work and they care and they know that that is a core competency to their business and so they resource it appropriately and one of my other favorites is the Jango documentation. Jango is really what got me to open source and into Python and all that stuff. So just reading the Jango docs they are comprehensive and they have tons of good examples and they do a good job of a lot of this kind of stuff. So I think they are another classic.
[0:18:40.6] DA: Yeah that is so true. I’ve had to teach a number of colleagues Jango on projects that we were ramping up on and they’d be like, “Oh what books should I read or what blog post should I look at” and it’s like why? Don’t even, just look at the documentation like go to the tutorial. The references are really good like that is true. I never really felt like I needed another reference or another blog post like it is all in there.
[0:19:10.2] EH: It is terrible for all the authors in the Jango community.
[0:19:13.4] DA: Yeah although there are some great books out there, but they have to be like this is a very specific use of it. It is not a general thing because you already covered on doing 99% of what you need to do.
[0:19:28.3] EH: Yeah, and so those are some of my two favorites. I think there is a lot of other great examples out there. Sadly, it is much easier to find bad examples than it is to find great examples, but you know.
[0:19:40.3] DA: Yeah let’s throw some people under the bus, let’s do that.
[0:19:42.2] MN: Oh I mean I think I have been a client that have done really well documentation and know like I have been in two extremes and when I am in one extreme where it’s like documentation is required you have to keep updating the documentation, the team is responsible for updating the documentation. That is in your estimates when you are building out a new feature and even then, I always feel like, “Oh man wait, this is way too much.”
I am definitely in the camp of, “My code does, it is so expressive you can read it and it will just make sense. If it doesn’t make sense, read the test. The test will tell you what is going on” but then I’ve been in the other side where there’s zero documentation and that was like, “Whoa, wait a second. Where am I? What am I doing? What is going on?”
[0:20:33.7] DA: I feel I’ve been in that case too where I was pushed to write documentation like doc strings for these classes that we are doing like calculations, calculating retail prices and things like that and I’m like, “Oh this is very self-evident” and then I had to teach someone else how to use it to extend it and it was like, “Oh wait, this code is shaped like my brain. No one else knows how this works.”
[0:20:59.9] MN: No one.
[0:21:01.5] DA: Although it is beautiful and there is many tests.
[0:21:05.6] EH: So you just gave me a perfect segue to one of my favorite analogies, which I will shamelessly now share, which is I really do think that for developers a lot of the value of documentation is making it easier to get a mental model of the software. So one of my favorite analogies is basically documentation is JSON for your brain. It’s a way to serialize your mental state so it can be loaded into someone else’s brain without having to execute the entire program in their own head.
It is JSON for software designing concepts and I think that is a really powerful way to think about it is it is a serialization format that is more expressive than software for translating a sub set of ideas around software to other humans and I think however you apply that methodology, use it in the right place and it will be great. Use it in the wrong place and it will be terrible just like any tool, right?
[0:22:04.8] DA: Right, that is so interesting is like writing from the very dawn of civilization. It is like I am taking a thought out of my head and it can go into your head now and you may think the wrong thing and that is maybe a bug.
[0:22:22.2] EH: Right, arguing against documentation at some levels are going against communication, which is like a core human technology and obviously that’s a little strong and a little hyperbolic but I really do think that it is a tool and it’s technology and it can be used equally as well as any other tool or technology and people’s experiences tend to be shaped by how it’s been done in the past but the fact that we all had good experiences around documentation proves its value.
And now the goal is how do we build the industries knowledge of how to use the tool to make it more valuable for everyone and that is really what write the docs is about is sharing those best practices and building a community of thought around this stuff to advocate for the things that work and try to get more people doing and having good experiences with documentation.
[0:23:12.7] DA: Yeah, there you go.
[0:23:13.3] MN: There you go.
[0:23:14.1] DA: Documentations not just for clay tablets.
[0:23:19.2] MN: So transferring knowledge and code at the same time. Eric, how can people contact you?
[0:23:25.1] EH: Yeah, so the easiest place to find me on the internet is just to ericholscher.com. That is just my name and that has links to everything else, to Twitter, to GitHub and all of that good stuff.
[0:23:34.4] DA: Cool, you got anything to plug?
[0:23:36.1] EH: Yeah, I’ve mentioned the docs a couple of times and so I’d be remised to not mention that the Prague conference is happening here in the middle of September in Prague and then we also have one in Sydney, Australia happening in November and then also we have lots of other content. If you want to go just go to writethedocs.org. There’s newsletter and there is a Slack and lots of other places, meet ups in four or five different continents and lots of places you can engage in that community.
So if software documentation is what you are into that is somewhere you should go and kind of talk with folks and get better at it.
[0:24:07.4] DA: Awesome, love it. Write those docs.
[0:24:09.3] MN: Write the Docs.
[END OF INTERVIEW]
[0:24:10.6] MN: Follow us now on Twitter @radiofreerabbit so we can keep the conversation going. Like what you hear? Give us a five-star review and help developers like you find their way into The Rabbit Hole and never miss an episode. Subscribe now however you listen to your favorite podcast. On behalf of our producer extraordinaire, William Jeffries and my amazing co-host, Dave Anderson and me, your host, Michael Nunez, thanks for listening to The Rabbit Hole.
Links and Resources:
