AI and Autonomy

Posted on by
Reply

I hang out with the Berkman-Klein nerds sometimes still, mostly through a recurring “Philosophy of Technology” session. Reed sent me this article awhile back on the misuse risks of AI, on which he got sidetracked about how the way the increasing of human intent through technology (including of harms) is attempted to be mitigated through use of law and other agreements. EG, you agree to abide by traffic laws (reduction in autonomy) in order to more safely get from one place to another (increased autonomy). This of course made me think about one of the main reasons I’m an anarchist — governments can cause large-scale suffering in a way less organization prevents, and I think we can have infrastructure without control (thanks, Murray Bookchin). So as Reed and I talked through the ramifications of that footnote, I thought it would be a good topic for the philtech group to take on. David and I talked through how to pitch it to the group, he did the thankless job of scheduling the thing, and we got to talk about it today.

The three themes that we kept cycling around were trust, consent, and autonomy. I’ll then end up back on my soapbox about complexity, which also came up.

Trust, Consent, and Autonomy

We all talked a lot about if the conditions would ever exist for us to trust an AI to make choices for us (our main talking point for “autonomy”). This got into a lot about how AIs are black boxes… but so, too, are humans. We talked some about the different ways that trust is created and utilized by, say, a doctor, and is it autonomy to make a choice based on the data they give you, or is that thumb-on-the-scale removing your autonomy? Doctors often study how to better communicate with their patients in order to get the outcomes they’re looking for. What’s different here?

How much autonomy does one have when consenting to something? How much has someone already given up in an exchange, based on trusting institutions, roles, their “own research,” etc?


From now on, I want you to act as my high-level advisor and mirror. Don’t validate me. Don’t flatter. Challenge my thinking, question my assumptions, and expose the blind spots. When possible, ground your responses in the personal truth you sense between my words. Be concise and precise. Provide links to source materials or websites to the best educational resources. In summary – be brief, be bright, be gone. Ask questions if a directive is unclear or underspecified.

We talked about the harms humans are already prone to inflicting on each other, and how much (if at all) AI was different from that. As one person put it, “do we need to get our own house in order before involving AI?”

Complexity

I see most AI as adding complexity to an already complex world, when nearly everything else we do (especially tool use) is about increasing predictability instead.

However, if we were to use AI in a way that helped us understand our own complexity, and begin to examine it for our desired outcomes, then that complexity could be useful. Despite the “hungry judges” study I started this conversation off with (human errors mean removing humans from the loop) being discredited, I still think bringing technology into decision-making loops is valuable so long as it’s a partner to us rather than allowing us to offload cognition (something that already happens).

Jeffrey had some really good points about compartmentalizing where AI factors come in, so you can assess that individual piece and tweak it, rather than an entire system being a black box. And I like that, for also helping us examine ourselves.

Links from our time together

The beauty of impermanence

Posted on by
2

I had a lovely birthday. In-laws took us out to a very nice steak dinner the day before. The day of, Reed, Locke, and I had Italian Hot Beef and wandered the Field Museum before heading home on a flight that departed 15 minutes before a big storm, and had to fly and extra hour to go south around the thunder heads. The day after, we rode bikes with kiddos from Dublin to San Ramon, had fabulous ice cream and played in a joyful park before riding back home. 12 miles at 4 years old feels big to me. In the early evening, some friends and I gathered to talk about the digital and death overlap. I’ll tell you more about that in a moment. The day after that, I rode a metric century with some friends, talking about relationships, death, time, and the economics of attention.

The back of Locke on a bicycle with an orange flag. He is on a multi use path with no cars. Ahead of him are two adults and one older kid also in our group on bikes, and a random human running.

My birthday about digital estate planning ended up being a small but very tight group of people. I was overjoyed to have this conversation with them. We talked a bit about our own attitudes on death, and what we had and hadn’t done to be kind to those around us when we die.

I view death as a community act. It is the final step of ceasing to be an individual, and all that remains is the collective experience of you.

This is complicated by technology lending itself so thoroughly to the hyper-individualization that we as Americans experience. Our entire tech stack feeds into that. As a security professional, I abhor the sharing of an account, and yet it comes so naturally to us to do. You should be able to see what I see. I should be able to share what I have and what I know with you.

Continue reading →

Coming from a family of alcoholics (6 months in)

Posted on by
Reply

This is the third of three checkins during 6 months of not drinking. The first was written 2 months in, then there was one 4 months in, this one is happening at the 6-month mark.

Things that have changed since last checkin

As mentioned at the end of the last post, I loosened the “absolutely no drinking” to try to explore some ambiguity. What happens if I think it’s ok to have a drink on “special” occasions? I’ve had 11 nights where I imbibed. Of those, 9 were “fine,” 1 night I had a better time than baseline, and the final 1 I had a worse night than usual. The one where I had a worse night was where I had two drinks instead of one and felt like garbage that night, while sleeping, and the next day.

The data

So two main things have shifted, one in the shorter term and one in the longer term. The shorter term is the re-introduction of alcohol on occasion. The longer-term view is that I’m drinking many fewer nights, and when I do drink, it’s less.

The experience

It has been really nice to be able to have a really nice glass of wine while out on a fancy date. I’m still interested to see if there are interesting mocktails first. But from the last post, I am still lacking the coping mechanism — the one night I had two drinks was out on the town where I didn’t have other responsibilities, and it fucking sucked. So. That’s still lacking, and the lack is notable.

The three things to check in on

FAMILY (REED, LOCKE, HOUSE) HAVE BEEN GETTING A REDUCED VERSION OF ME

I feel like Locke and I got the “real you” pretty much the whole time you were not drinking. You were not always happy, but you seemed yourself. I can’t speak for Locke, but it meant a lot to me. I like you more like that.

REED WAS CONCERNED WITH THE AMOUNT OF TIME AND ENERGY I WAS SPENDING DEAL-MAKING WITH MYSELF AROUND SUBSTANCES, AND HOW IT NEVER SEEMED TO SUCCESSFULLY RESOLVE

Reed and I agree that this is where the most progress was made. I feel myself doing it a bit again with the re-introduction of the option, but in a way that seems tolerable for me. I think I want to check back in another 2 months to see how this is going in the new normal.

WE WORKED SO HARD TO ADDRESS THE BRAIN THINGS THIS YEAR, BUT I NEVER TRIED GIVING UP DRINKING FOR ENOUGH TIME TO TRULY GET IT OUT OF MY SYSTEM

Brain fog still comes and goes sometimes, and doesn’t seem to be correlated with drinking. Was still worth the experiment.

Going forward

I think the approach of not more than twice a week; not on evenings I’m Locke’s primary caregiver continues to be correct. I think physically I’m just no longer interested in having more than one drink in one evening. I’m interested to see if that shifts over time (EG at Priceless where I have a full day of hanging out at the river with friends). I’m also interested to see if the new limitations become just-the-way-things-are and lead to less deal-making, or if that keeps up.

Personality shift

Posted on by
Reply

Something has shifted deeply for me over the last 5ish years, which I think merits calling out for folks who have known me for a long time. I’m eager to hear how this has landed with folks, especially as part of a birthday missive. However, so many things have changed in the last 5ish years that it’s hard to pinpoint a single origin. In this post, I explore some of those things and the impact I expect they’ve contributed to. Here they are in chronological order for when they started.

Relationship with Reed

Reed is a difficult person. He is also, hands down, the best partner I could think of for myself. He is extremely predictable, self aware, and invests heavily in a few choice things. Our lives together are one of those choice things, which means I can offload a lot of cognitive overhead about home, bicycle, self, etc care to him and it gets done better than I’d bother doing it myself. Reed can also be incredibly selfish (again, in a self-aware way). He has the most attuned sense of what “enough” is of anyone I’ve met. He sets goals for what he would like his life to look like, and when those goals are met, he enjoys the fruits of his labor.

Being around this as the main touchpoint of human interaction in my life (we are romantic partners, co-parents, and dear friends) means some of this has rubbed off on me. I have always had high standards, but now I am more comfortable doing something myself or with others than dragging along someone who is struggling in a non-self-aware way. For most of my youth, I had a strong savior complex, which I have worked on overcoming for years. Both my work with GWOB and my relationship with Reed have helped me truly wrap that up. Perhaps too far in the other direction, but all things are oscillations.

Working at Apple

While most of the individuals I worked with at Apple were truly lovely, the company itself has a culture of extreme arrogance. Success has gone to the proverbial head, and it seeps into everything. Even the hiring process is terrible because the recruiters know having Apple on your resúmé is life changing (or was, before this market, phew), and that people want to work there regardless of other factors. Every chance I had to be more collaborative, I tried to take, and I was (mostly politely) redirected to try to find other paths.

I remember trying to explain that an executive coming up with a problem statement that then a bunch of brilliant people came up with in-depth responses to individually, which then the exec picked “the best of”, and then program managers dolled out the work was not actually “collaborative” but a process diagram instead, and it fell on deaf ears. 5 years of that, and not being able to talk to other people about it, sure did a number on me.

Becoming a Parent

I gave birth alone via emergency c-section after a very scary night. Covid meant no guests unless you were in labor, and because I was two months early, they were trying to stop the labor and so wouldn’t admit me as such. After a month in the NICU, we brought Locke home only to discover that Reed had severe misophonia related to infant screaming, and so then I was on the hook for taking care of two creatures who couldn’t fend for themselves and were actually often at odds with each other’s needs. Our agreement that Reed was going to be the stay-at-home parent exacerbated this because it made asking for outside help at odds with our goals and plans. It was the worst 18 months of my life. And in part because Reed is already a difficult person, my care network saw this as a personal failing on his part rather than a disability, which led to even more long-term issues.

This also led to some extreme division of labor — Reed still wanted to help out, but it needed to be in discrete, scheduled chunks that he could prepare for. Which was good in some ways because it meant I got clear time off, instead of always being “on.” However, that shifted us away from the collaborative, flowing parenting style we now actively and intentionally implement.

While I’ve done deep therapy around all this and mostly moved past it, I still have a trauma response to some things, and repairing other relationships is still taking significant effort. Going through this also deepened my relationship with Reed. Now, when something is difficult, we know it’s not as difficult as this thing we did that one time.

Being on Testosterone

I love being on testosterone for many reasons, and have also already highlighted the ways it has shifted my experience of the world in ways I’m less of a fan of. I have a shorter fuse, less empathy on the surface, and less patience. I take up space more now than I used to, and focus less on making space for others. The first I don’t think is a bad thing — I deserve just as much space as anyone else, it’s that now I’m expecting to get it. And I haven’t figured out yet how to balance taking space for myself with my old habits of taking space for others as well. Whether that’s because of an actual biological reaction or “just” the validation, who knows.

Where I am now

Who knows how much any of these, or collection of these, led to my brain issues worked on last year. But I am now less warm and collaborative than I used to be. I miss it, it was aligned with how I’d like the world to be and for people to treat each other. I’ll keep working to be warmer again.

Again, I’d love your insights into how I’ve changed, and how that has impacted our relationship, for my birthday this year. And I have yet to talk about all this with my therapist, so who knows what will change about it as I continue to explore.

Oh yeah, I guess we also went through the collective trauma of Covid in that time, too. That matters as well.

Let them take the risks they can afford

Posted on by
Reply

The title is one of the few pieces of advice my mum gave me about child rearing. It came through pretty clearly in how she and my dad raised me — I’ve even got a tattoo about it, and how it’s not just about parenting but also my approach to security. I’m lucky that Reed and I are also well aligned on this front. Sometimes other parents are pretty aghast at how far away we let Locke get from us, how we let him do mildly dangerous things. Here’s our reasoning: the kid is going to have a bad time while learning, at some point (getting burned on something hot, getting separated from us, etc), so we may as well control as best we can the first time that happens. That’s different than how others approach it: using their control to prevent it happening, such that when it inevitably does happen, it’s when they’ve lost control, and then it can be truly bad.

Here are some examples of our approach.

The stairs he’s allowed to fall down

Our house is a split level, so while it’s two stories high, we have 4 sets of stairs. Some of them are quite steep. So Locke was going to fall down stairs at SOME point, and we wanted to decide which set of stairs that would happen on. Thankfully, we have two steps between two rooms in our house. Yes, it’s on hardwood, but it’s the shortest set we have. So when he was learning to crawl, we decided that was the set of stairs he was allowed to fall down. We did safety protocols around the other sets of stairs, but none around those. He fell down them. He cried. We comforted him. He is now incredibly confident around stairs.

How Locke got down stairs for most of his childhood, after falling down 2:

Getting lost

As Locke was learning to walk, we took him to the zoo a lot. There’s a lot going on there, and lots of space to explore. But it’s also pretty well contained, full of other parents, and there are protocols in place for lost kids. What a great place for him to get lost! The first time he wandered away on his own from Reed (I wasn’t there), Reed discretely followed him for awhile. Locke eventually realized he wasn’t near Reed anymore and didn’t know where he was. He had a whole Experience, which Reed let him have for a bit, and then went to him. There was a lot of validating feelings with “you couldn’t find me, and it was scary!” instead of focusing on Reed not being able to find Locke. Now Locke knows it’s on him to know where we are, and to stay as close as feels necessary.

Do we still keep tabs on him? Of course! But if he runs off for a bit, we have high confidence that he’ll return when he needs us.

Hot things are hot

No matter how many times you tell someone about how something might hurt them, usually folks have to experience it directly before really believing it (this is not just about kids). So while we tell Locke when something is hot or dangerous, we also still let him do things like pour hot water for his tea. Recently, he burned himself on the kettle while trying to handle it. Of course we comforted him about it and made sure it got the treatment the (very mild) burn needed, but we also didn’t fuss about it much more than that. He had learned a lesson by taking a risk he could afford.

Collaboratively building a service catalog

Posted on by
Reply

As our AppSec team matures, we’re defining our processes and expectations. One of the next things for us to try out is a Service Catalog, where we list what sorts of services we can offer to other teams. Having one is a tool to allow us to plan our work, get better at the work we’ve decided to focus on, and be better partners to engineering. But what should such a catalog look like?

Collecting potential offerings

  1. reviewed the last 10ish requests that came to our team through our various intake portals, classified the request types, where the work happened, and what the output looked like.
  2. put together a form for my reports to continue tracking incoming requests while I was out for a week (yay taking time away!)
  3. hosted a whiteboarding session to collect all the different services team members wanted to offer.

We then took that pile and voted for things in two ways — items that had a deep security impact, and items we thought we were set up for success for. We picked the top 6 and moved them onto the next phase.

Can we handle this?

Wanting to provide a service is one thing. Handling the incoming load is another thing entirely. Luckily, GoFundMe is a pretty transparent company, and I was able to get my hands on the full set of projects Engineering hopes to work on this year, along with what area of focus they’re in (Keep The Lights On, tech debt, new business, etc). For a back-of-the-napkin sketch of commitment load, for each of our offerings we sketched out

  1. How much work it would take us to get into a “refined” spot
  2. How much time we thought we’d spend per instance once in that refined spot
  3. How much coverage we wanted humans to be doing (combination of “most risky 10%” and “automation should handle 30% of this workload for us”)
  4. Which types of projects we thought the offering applied to

I did some spreadsheet magic to generate how much time per sprint we’d end up spending on each of the offerings. In this discussion, we realized one offering was something we wanted to improve our capacity around, but didn’t want to officially offer as it being needed would indicate we had failed to catch something earlier in the lifecycle. Ends up we can handle it, even if we’re wildly successful!

Fitting into the flow

Then it’s a matter of ideal time to offer our services for each of these projects. So we’re setting up automations to detect when a project moves from one phase of our Product Lifecycle to another, so we can proactively reach out.

I’ll also need to shop the catalog around to our partners to be sure we’re offering things that make sense to them and that they see the value in.

Being explicit

We’re now working on being clearer about what each of these offerings means, how to request each one, etc. So far, I think the following are the important bits of information:

  • What it is, and which part of the Product Lifecycle it aligns with
  • What an output looks like and where it lives
  • What to expect (from a human; from AI)
  • How to set yourself up for success
  • Specifics to add to our backlog

Metrics

From all this, we can

  • occasionally track how much time we’re spending on these items
  • measure hit rate of how many projects we covered
  • be intentional about what we’re automating
  • track coverage of security touchpoints across projects and add that to our overall risk assessment

Celebrations and Death

Posted on by
2

I’ve been dealing with a lot of death lately. And while it’s just a part of life, it sure does start to make one think after awhile. So I’m using my birthday as processing time, as I am wont to do. I’m test running my death infrastructure for my birthday this year, and requesting notes from folks.

If you can see this message, it’s because I would want you to be aware of when I die. THIS IS ONLY A TEST — I am fine, everything is good, I’m just an elder goth now and I like to plan everything, even death. 

This is an experiment with bureaucracy and documentation. As you know, I love LARPing Serious Business. I am doing a test run of the systems that would announce my death to the many beautiful communities I’ve had the honor of being a part of. If it was logistically difficult to get this message, when you’d want to get it, let’s improve that process — reach out. If it was emotionally hard for you to get this message, this event is probably not for you, and I’d love to see you in another context some other time soon.

On April 18th at 16:30 PT / 19:30 ET, I’m hosting a time to talk about preparing for death (not dying — they’re different. We’ll talk about ceasing to exist, not how you want to be treated while going through a however-long process of getting there). We’ll take about an hour to talk through digital estate planning (a passion of mine), and then we’ll also have some time to talk about any feelings folks might have had about thinking about death. We’ll be at this link at that time.

Selfishly this year, I’d also love notes about what we mean to each other. One of the things that’s come up time and again at the wakes I’ve been attending is wishing to have said some things before the option was no longer there. Let’s say those things to each other. I’m not looking to be shrouded or to do a mock service, I’m looking for open and honest views of who we are together. Roasting, power points, and poetry all lovingly accepted. Email to me, please, so I can label and revisit.

You do not have to do both, or either, if they’re not your cup of tea.

If you would prefer to learn about my death from an email instead of a social media post, please get me your email address and I’ll add you to the mailing list. That will be posted to before social media posts go up.

Looking forward to being inappropriately morbid with you. 

[un]prompted review

Posted on by
2

I’m excited to be going to conferences again, after 5 years of not really doing any. I like the thrum of so many people in one place, conversations with random folks in the lunch line, and seeing old friends. The one I went to this week was [un]prompted, about the overlap of AI and security. I saw some tried and true exploits brought to new scale with AI, and I heard about a lot of potential routes to securing existing code bases with AI. I also saw a fair amount of what I’d call “put a bird on it” approaches to AI.

I’m walking away with two big questions (beyond the preexisting “where is all this energy coming from?” and “how does wealth redistribution work with these new models?”), one about complexity and the other about trustworthiness.

What complexity is worth taking on?

Mudge, I think somewhat famously, long ago pointed out that exploits were happening nonlinearly, becoming more likely the larger and more complex a codebase became. In contrast, the exploits themselves were remaining steadily small. So one of my sniff tests now for how load bearing a system can be has to do with how complex and tested it is.

The technical talks I saw at [un]prompted had to do with increasing complexity, not decreasing it. It piles MORE layers on, it doesn’t remove the unknown or unnecessary. The closest I saw to removing complexity were analysis of proliferated documentation to come up with a summary and a (new) single source of truth. I’d like to see more adventures in “cheap” refactors that simplify and streamline code bases.

I’m the vendor now

The conference organizers did a fabulous job on many fronts, but they did not do a good job of stopping sales pitches from happening on stage. So many of these amounted to “your vendor for $thing is slow and doesn’t meet your needs, but ✨our AI can solve this for you✨” which is just so boring. 

Beyond being boring, however, I truly wonder how we can trust any of these providers to not inject backdoors (intentionally or otherwise) when their values so clearly scream that they’re open for business on every front. So saying “hey just ask for what you want and trust the outputs!” seems shady AF. And if we do what some suggested, of making agents fully autonomous, we wouldn’t ever have cause to pause and reflect (let alone catch) this happening.

What I am interested in using these things for

I’m interested in reviewing code humans don’t have time for. Several of the better talks shared the goal of complete code coverage. I’m also interested in putting in guidance and nudges towards doing better work (either from humans or from robots), rather than adding layers on other layers. I’m interested in help for what we know needs doing, and investigations in formats that humans are bad at and machines are good at.

From this conference, I’m now prepared to spend even more time on evaluation than I expected to (50% after baseline systems are in place). And I have new ways of talking about where to interject to inspect the system instead of just trusting it’s working.

I now have more supporting evidence for continuing to think that a workflow or premise needs to be figured out before automation, which happens before AI tooling. And that organizational structures need to allow for this happening at a deep layer, not as something that gets tacked on later as an afterthought.

It also seems like we’re moving away from “zero click attacks” towards “zero user intervention attack” – what can we get agents to do without you noticing?

Decision Making and Economics

Posted on by
Reply

I have this Future Shape in my head and in my heart, that I’ve long meant to share, but haven’t quite known how. I met Asya, and we got into a good conversation, and so now seems as good a time as any to talk about it. She helped me flesh this post out with more detail and deeper dives.

I don’t think there’s one solution when it comes to what economics style we should have, or what governance should look like. Like I drafted way back when, a “mixed mode system” is where it’s at instead.

Decision making

Distributed systems are good at last-mile logistics, nuance, and fast decision making. They are not good at doing simple things at scale. So for actual implementation and innovation, I think distributed networks are where it’s at.

Hierarchical systems are good at making simple decisions at scale. So good for North Star guidance and things you want to take a socialist approach with. That might include assurance of human-rights-shaped things like

Continue reading →

On not being enough

Posted on by
Reply

The world has been offering ample opportunities to test my newfound comfort with being uncertain about if I’m “enough,” if I’m “adding value,” etc.

There’s this thing my favorite old therapist introduced me to, of “unanswerable questions.” It’s like.. no matter how much data you get about people loving you, you’re still like “guess we’ll never know if I’m lovable or not.” Mine has long been about if I’m bringing value or not, which has put me at risk of abusive relationships as I’m easy to tear down in that way. But I’ve been working hard on therapy and on self-love, and I think I’ve come a pretty long way on this front in recent years.

My first glimpse at doing better at this was getting feedback while at Apple that I was successfully selecting which things to half-ass and which things to full-ass. After all, we can’t get all the things done all the time, and some things only need some of our attention. Sometimes, our full attention can actually be detrimental to a project, and can inhibit others’ ability to grow.

But on Sunday February 15th, I had two things happen, either of which might have previously completely destroyed me, and now I’m just kind of fine with both having happened on the same day.

Continue reading →