How do you really choose the right documentation tool? In this podcast episode, Sarah O’Keefe (Scriptorium) talks with Paweł Kowaluk and Michał Skowron (Guidewire Software) about building a successful tool selection process, the realities of docs as code, and what happens when the technology becomes the unpredictable variable.

Paweł Kowaluk: It’s funny how programming used to be deterministic, and it was the people who were messy. We always knew that people are going to be whimsical and maybe harder to rein in, but the technology is going to be predictable. Whereas now, technology is not predictable anymore, and you give it a prompt and you hope it’s going to do what you want. You adjust the system prompts and change the weight of things which are retrieved versus metadata, et cetera, and it doesn’t always work the way you expect it to.

Sarah O’Keefe: And now the people are being asked to be the deterministic layer, right? To be the QA on top of the AI.

Paweł Kowaluk: That’s actually very insightful. I like that. That is true. The human in the loop or whatever you call it, that’s supposed to be the voice of reason.

Related links:

Scriptorium: AI in the content lifecycle
Tech Writer Koduje podcast
Tech Writer Koduje: DITA as code – a modern approach to the classic standard
Tech Writer Koduje: Are people abandoning docs as code?
Tech Writer Koduje: A tech writing CCMS can also be a broken promise

LinkedIn:

Transcript:

Introduction with ambient background music

Christine Cuellar: From Scriptorium, this is Content Operations, a show that delivers industry-leading insights for global organizations.

Bill Swallow: In the end, you have a unified experience so that people aren’t relearning how to engage with your content in every context you produce it.

Sarah O’Keefe: Change is perceived as being risky; you have to convince me that making the change is less risky than not making the change.

Alan Pringle: And at some point, you are going to have tools, technology, and processes that no longer support your needs, so if you think about that ahead of time, you’re going to be much better off.

End of introduction

Sarah O’Keefe: Hey, everyone. I’m Sarah O’Keefe, and welcome to the podcast. In this episode, we are going to talk about tool selection with a couple of special guests. With me today are Paweł Kowaluk, who is a software architect at Guidewire Software, and Michał Skowron, who is a documentation tools developer, also at Guidewire. Both of them are based in Poland. Welcome.

Paweł Kowaluk: Hi.

Michał Skowron: Hello.

SO: I am glad to have you. For those of you on this podcast that speak Polish, you’re probably already aware that they have the one and only techcomm podcast in Polish that is available out there, and Michał and Paweł are also experts on doc process and tool selection, so that’s what we wanted to focus on today. So I will start and throw it to Michał and ask you the big picture question, which is what does a good tool selection process actually look like?

MS: For me, good selection tool process would be divided in three stages. The first one would be gathering requirements, looking what’s out there, defining what you want to basically achieve with this new tool. Then I would go to a pilot project where you can actually test the selected tool in the real world. Manufacturers and producers of software will tell you that it can do anything and it will promise that, “Okay, you can meet all your requirements easily and we can fix that, we can improve that, we can adjust that,” so everything can be done is usually what we hear, but then you want to test it in real world on a real project, so that will be a pilot project for you and your team.

And the third phase that depends on the outcome of the second phase, which is you either productize the selected solution or you just say, “Okay, that was a bad choice and we don’t need that.” Then we need to go back to the first stage and then say, “Okay, we need to select another tool,” and again, requirements, et cetera, et cetera. So for me, that’s the whole process, and the first stage would be probably the longest one because you need to make sure that you are meeting all your goals.

SO: So what’s the most common reason that a pilot doesn’t succeed, that you have to go back and say, “That didn’t work. We have to try something different”?

MS: It’s usually because you didn’t see everything when you were planning. For example, you have some projects that are very specific or you didn’t see all the problems or things that are coming your way. It’s hard to say exactly what the reason is, but it can be multiple reasons.

For example, using of, I don’t know, branching, let’s say, in a specific tool. When you have multiple versions of your product and you want to keep them separate when it comes to documentation, it can turn out that the feature says, “Okay, you can use branching and then you can do it easily,” and then you start using it and it turns out that it doesn’t work the way you expect it. This is actually a real life example because we had a system that… I’m not going to mention any names or anything like that, but there was a system and they promised us… That was years ago and it was a vendor that promised us that they’re going to introduce a feature called branching, and it turned out after they did that that it wasn’t what we expected. So it can turn out in many cases, in many ways, it can be the problem, but branching is just an example, but it can be many other things that can go wrong.

PK: Hey, if I can jump in here, I got a couple of examples. One is I could call it releasing strategy or versioning strategy overall, which is very hard to test in a pilot project. It’s very hard to scope for requirements because the little problems come out after a while, after a year of publishing, after two years of publishing. And another example which is related is reuse, and this one is down to formulating the requirements correctly. Because I think just saying, “We want to reuse something,” is not enough, because you have to say exactly what you want to reuse and how you want to reuse it and what you want the result to be.

So for example, if you say, “I want to reuse notes and warnings and things like that.” We sometimes call them admonition. So, “I want to reuse these notes in my docs, and if I update a note, I want it to update in every published version of the doc.” Then only if you have these details, like I want to update it once and then want it to automatically update and publish docs, then you will see it’s not working the way you expect it. Because if you just say, “I want to reuse notes,” every system can reuse notes. Even in docs as code, there’s scripts and macros that allow you to reuse notes.

MS: It can be also another thing that, for example, you compare the benefits with the actual cost of implementation, and it can turn out it’s not worth it because people are, for example, reluctant to use your new tool. The training, the cost of licensing, the cost of support is too big, and then you realize, okay, we want to achieve a goal like, I don’t know, reduce the time to market, and then it turns out it doesn’t work because people are struggling with using the product on a real project. And on paper, everything looks cool and you have all these features, you can use them, like Paweł mentioned, for example, reuse conditional formatting and things like that. And then it turns out it’s very hard to use, it doesn’t serve its purpose, and then you have to pay for every additional stuff and people don’t want to use it, so what’s the point?

SO: Yeah. And I think we find that the technical problems in general, if you’ve done your requirements work, then usually, the technical problems are solvable if the people engaged in the project want to sol...