Tag Archives: complex systems theory

[un]prompted review

Posted on by
Reply

I’m excited to be going to conferences again, after 5 years of not really doing any. I like the thrum of so many people in one place, conversations with random folks in the lunch line, and seeing old friends. The one I went to this week was [un]prompted, about the overlap of AI and security. I saw some tried and true exploits brought to new scale with AI, and I heard about a lot of potential routes to securing existing code bases with AI. I also saw a fair amount of what I’d call “put a bird on it” approaches to AI.

I’m walking away with two big questions (beyond the preexisting “where is all this energy coming from?” and “how does wealth redistribution work with these new models?”), one about complexity and the other about trustworthiness.

What complexity is worth taking on?

Mudge, I think somewhat famously, long ago pointed out that exploits were happening nonlinearly, becoming more likely the larger and more complex a codebase became. In contrast, the exploits themselves were remaining steadily small. So one of my sniff tests now for how load bearing a system can be has to do with how complex and tested it is.

The technical talks I saw at [un]prompted had to do with increasing complexity, not decreasing it. It piles MORE layers on, it doesn’t remove the unknown or unnecessary. The closest I saw to removing complexity were analysis of proliferated documentation to come up with a summary and a (new) single source of truth. I’d like to see more adventures in “cheap” refactors that simplify and streamline code bases.

I’m the vendor now

The conference organizers did a fabulous job on many fronts, but they did not do a good job of stopping sales pitches from happening on stage. So many of these amounted to “your vendor for $thing is slow and doesn’t meet your needs, but ✨our AI can solve this for you✨” which is just so boring. 

Beyond being boring, however, I truly wonder how we can trust any of these providers to not inject backdoors (intentionally or otherwise) when their values so clearly scream that they’re open for business on every front. So saying “hey just ask for what you want and trust the outputs!” seems shady AF. And if we do what some suggested, of making agents fully autonomous, we wouldn’t ever have cause to pause and reflect (let alone catch) this happening.

What I am interested in using these things for

I’m interested in reviewing code humans don’t have time for. Several of the better talks shared the goal of complete code coverage. I’m also interested in putting in guidance and nudges towards doing better work (either from humans or from robots), rather than adding layers on other layers. I’m interested in help for what we know needs doing, and investigations in formats that humans are bad at and machines are good at.

From this conference, I’m now prepared to spend even more time on evaluation than I expected to (50% after baseline systems are in place). And I have new ways of talking about where to interject to inspect the system instead of just trusting it’s working.

I now have more supporting evidence for continuing to think that a workflow or premise needs to be figured out before automation, which happens before AI tooling. And that organizational structures need to allow for this happening at a deep layer, not as something that gets tacked on later as an afterthought.

It also seems like we’re moving away from “zero click attacks” towards “zero user intervention attack” – what can we get agents to do without you noticing?

You belong to society

Posted on by
24

I’ve been unable to continue ignoring a notion that most people I see in online debates about gender1 carry, which is that those in these debates do not think they impact society, and subsequently have no individual responsibility towards it. It is simply a soup of which they are a part, where they are a stone — immutable to the broth around them, of no consequence to the overall flavor.

Let’s talk about emergence, here from the Complex Systems perspective, as the interaction between the parts and the whole. “Can’t see the forest for the trees,” as not being able to see the big picture because one is so focused on the next-scale-down of units (trees), despite these composing the next-larger-up scale (forest). Each has different behaviors, which slightly or drastically effects the other. Or, “the devil is in the details,” in which the opposite happens, the smaller-scale being skipped over while the next-larger-scale is focused on. You’ll note that these things matter to each other. They influence each other. In many circumstances, these two scales are caught up in creating each other in at least some small way2. To claim that one is more important than the other glazes over this connection. Plus, the math doesn’t work out right.

Let’s talk about values. I would like a just and equal world. I bet most of the people I talk to would also like some version of this. Some folk hold other amazing core values such as inclusion or empowerment. Here’s the thing to understand: anyone you interact with3 will be holding something like this inside of them. Maybe not so explicitly, maybe not as an active part of their interactions, but it is there.

Let’s talk about fault. The people that got us to where we are now were doing the best they could under the circumstances. Maybe some were malicious, but generally. they were just surviving. People in power tend to want to continue doing well. People who are out of power generally make do, though they’re likelier to have a generally more shitty time. Inequality makes both sets unhappy. It’s not the fault of the people in power that the structures which allow them to be in power exist; it’s not the fault of those out of power that they were born into a setting that keeps them out of power.

Let’s talk about responsibility. While no one currently alive is to blame for history, we are currently building the next generation’s history. Hell, we’re building our own. And we have a responsibility to act in a way which upholds our values, rather than shirks responsibility as bizarrely tied to fault. I don’t want to take the responsibility to respond kindly to this person because their upset is not my individual fault. I don’t want to help clean up after dinner because not all the plates are my fault. I don’t want to take responsibility for mending the rifts in society because they’re not my fault4.

In each of these, it is not just what you are asking for yourself, but what you are changing in the people around you. When a child is being surly, and a parent reacts badly because a nerve got struck5, the wrong lesson is being imparted. It’s not about the parent’s feelings in that moment. It’s about how the child learns how to react to someone expressing their feelings in a not-yet-eloquent way6.

Sometimes taking on this responsibility to society means shutting up, even when you’re right. Sometimes taking on this responsibility means speaking up, even when your voice trembles. Sometimes this means cleaning the common area, even though you haven’t even been around for the past week. It means having differences and resolving them in a way that makes sense for future generations to also resolve them, even if you’re not happy with the results.

When anyone says “my individual experience matters more in this moment than how we as a society deal with moments like these” I see them as throwing a tiny tantrum rather than building a better world. It’s not their fault. Why should they have to do anything to fix it? This is why I continue to think Laurie’s piece is so great and I get filled with rage and bile at StarSlateCodex. This is why I find GamerGaters outright laughable7. This is why I find some of my geek feminism friends so aggravating at points7. In all of this, I see why they’re saying what they’re saying. Of course those feelings are valid. But that’s not the whole point, is it?

Get our shit together. Focus on where we want to be, and manifest that in each interaction we have. This is what I assume most people are doing, and why I’m now so comfortable saying “I don’t like how we’re doing this, can we try another way?”

I don’t like how we’re doing this. Let’s find another way.

 

1. And race, now, too!
2. Exceptions of pragmatic lock-and-key example, and the theoretical molecule representation of same self model.
3. With incredibly rare exception, not based on if you get along with them or not.
4. Are you fucking kidding me, this is how we get ants.
5. children can be astute little fuckers
6. I am in no way claiming to be amazing at this, merely that I am aware of, and subsequently actively working on, it.
7. “You need to listen to me!” they say, while not listening.