This post was put together by Tom, his AI minions, and me. We have a really neat workflow, but the end result is lovingly hand-crafted by yours truly based on a bunch of background work. I hope you enjoy – this is the last of these posts until we get more praxis under our belts related to the theories we’re proselytizing here. 

We have now covered the concept of “comprehension debt,” or where and what sorts of debt you take on when using AI in your product lifecycle. We’ve covered why it’s important to surface decision points and where to store them. Now it’s time to talk about why all of this is important, what it means about when and how to use an LLM. 

Let’s get philosophical

I think one of the points of human existence and especially of science is to explore and describe a shared reality. Once a shared reality is arrived at, it can be codified and built on top of via further exploration. In Primed to Perform, they refer to this as what is “above the waterline” – EG if you blow a hole in the side of a boat, it really matters whether the hole is above or below the waterline. I believe reality is below the waterline. This is why hallucinations bother me so much, especially when based on texts that do establish a shared reality.

Shared reality is something we can build upon, which assumes some level of repetition, reference, or both. To me, this is where determinism versus non-deterministic (by you or your robot) production comes in – which kind of work are you doing? Noticing one state versus the other (surfacing judgement) is something we touched on previously. So in this one we’re focused on deciding when to do which method.

We cannot trust an LLM to be deterministic in that it hallucinates (again, “new hire on day one, forever” from this post). We will never be able to trust an LLM to be deterministic, because of the nature of the tool. You have to monitor it for alignment with shared reality, and nudge it back in that direction (AKA verification). We all know and accept this as the cost of using this tool. This post is instead about the other aspect of building on a shared reality, repetition. This is about how to use a tool with this kind of flaw to still create outputs that are a part of a stable foundation (as indicated in part by debt).

Detecting repetition

If repetition is the point to decide what to codify, how do we detect we are at a point of shared reality to encode, versus stepping through creative / LLM space to explore still? This is also about using tokens wisely – compute on deterministic properties is less intensive than inference. Some ways to find out where repetition is happening:

1. /insights about prompts and commands getting used a lot. 
2. Tom has written a skill called /wrap about how to wind things up, such as surfacing anything that should become a function. 
3. Asking for a Directed Acyclic Graph can help you see what work is being done when, and what could be pulled out / systematized.

After you detect your repetition, you can run experiments (remember my diagram!) about which way to build a deterministic process to handle the repetition.

But what does this mean about debt?

Tech debt is about deciding to use poor decisions now for expected repetitions which weigh on future abilities, in order to realize value sooner. Comprehension debt is about technical decisions being frozen into determinism without you noticing [ref]. This places an assumption as below the waterline when it might be above the waterline, restricting your space for play and exploration. It also diminishes your ability to predictably tweak the rest of the system you’re building, because you don’t understand what’s going on in the different parts and what the deterministic dependencies are. Doing rework is costly to velocity.

You must commit in order to make things, but you want to commit the right things in the right moments. This relates back to our diagram and when decisions need to be made, along with our post about surfacing decisions at the right points. EG, when a product owner is grooming and ordering, they’re paying down their comprehension debt. Then it is up to engineering to find a way to limit build-up and/or pay down their technical comprehension debt, potentially with the patterns we’ve described in these posts. 

Build a shared reality

Please use AI to help build and maintain our shared reality. Be intentional about what is being built into repetition, and what references you build and maintain. Make it clear what is an assumed agreement, and what is actually desired. Create a shared direction and go that way, verifying and encoding as you go. As the Zappatistas declared, “by asking questions, we walk,” to say that we must move forward while interrogating what we are doing. Doing one without the other leads to negative outcomes.