Category Archives: GoFundMe

Collaboratively building a service catalog

Posted on by
Reply

As our AppSec team matures, we’re defining our processes and expectations. One of the next things for us to try out is a Service Catalog, where we list what sorts of services we can offer to other teams. Having one is a tool to allow us to plan our work, get better at the work we’ve decided to focus on, and be better partners to engineering. But what should such a catalog look like?

Collecting potential offerings

  1. reviewed the last 10ish requests that came to our team through our various intake portals, classified the request types, where the work happened, and what the output looked like.
  2. put together a form for my reports to continue tracking incoming requests while I was out for a week (yay taking time away!)
  3. hosted a whiteboarding session to collect all the different services team members wanted to offer.

We then took that pile and voted for things in two ways — items that had a deep security impact, and items we thought we were set up for success for. We picked the top 6 and moved them onto the next phase.

Can we handle this?

Wanting to provide a service is one thing. Handling the incoming load is another thing entirely. Luckily, GoFundMe is a pretty transparent company, and I was able to get my hands on the full set of projects Engineering hopes to work on this year, along with what area of focus they’re in (Keep The Lights On, tech debt, new business, etc). For a back-of-the-napkin sketch of commitment load, for each of our offerings we sketched out

  1. How much work it would take us to get into a “refined” spot
  2. How much time we thought we’d spend per instance once in that refined spot
  3. How much coverage we wanted humans to be doing (combination of “most risky 10%” and “automation should handle 30% of this workload for us”)
  4. Which types of projects we thought the offering applied to

I did some spreadsheet magic to generate how much time per sprint we’d end up spending on each of the offerings. In this discussion, we realized one offering was something we wanted to improve our capacity around, but didn’t want to officially offer as it being needed would indicate we had failed to catch something earlier in the lifecycle. Ends up we can handle it, even if we’re wildly successful!

Fitting into the flow

Then it’s a matter of ideal time to offer our services for each of these projects. So we’re setting up automations to detect when a project moves from one phase of our Product Lifecycle to another, so we can proactively reach out.

I’ll also need to shop the catalog around to our partners to be sure we’re offering things that make sense to them and that they see the value in.

Being explicit

We’re now working on being clearer about what each of these offerings means, how to request each one, etc. So far, I think the following are the important bits of information:

  • What it is, and which part of the Product Lifecycle it aligns with
  • What an output looks like and where it lives
  • What to expect (from a human; from AI)
  • How to set yourself up for success
  • Specifics to add to our backlog

Metrics

From all this, we can

  • occasionally track how much time we’re spending on these items
  • measure hit rate of how many projects we covered
  • be intentional about what we’re automating
  • track coverage of security touchpoints across projects and add that to our overall risk assessment

[un]prompted review

Posted on by
2

I’m excited to be going to conferences again, after 5 years of not really doing any. I like the thrum of so many people in one place, conversations with random folks in the lunch line, and seeing old friends. The one I went to this week was [un]prompted, about the overlap of AI and security. I saw some tried and true exploits brought to new scale with AI, and I heard about a lot of potential routes to securing existing code bases with AI. I also saw a fair amount of what I’d call “put a bird on it” approaches to AI.

I’m walking away with two big questions (beyond the preexisting “where is all this energy coming from?” and “how does wealth redistribution work with these new models?”), one about complexity and the other about trustworthiness.

What complexity is worth taking on?

Mudge, I think somewhat famously, long ago pointed out that exploits were happening nonlinearly, becoming more likely the larger and more complex a codebase became. In contrast, the exploits themselves were remaining steadily small. So one of my sniff tests now for how load bearing a system can be has to do with how complex and tested it is.

The technical talks I saw at [un]prompted had to do with increasing complexity, not decreasing it. It piles MORE layers on, it doesn’t remove the unknown or unnecessary. The closest I saw to removing complexity were analysis of proliferated documentation to come up with a summary and a (new) single source of truth. I’d like to see more adventures in “cheap” refactors that simplify and streamline code bases.

I’m the vendor now

The conference organizers did a fabulous job on many fronts, but they did not do a good job of stopping sales pitches from happening on stage. So many of these amounted to “your vendor for $thing is slow and doesn’t meet your needs, but ✨our AI can solve this for you✨” which is just so boring. 

Beyond being boring, however, I truly wonder how we can trust any of these providers to not inject backdoors (intentionally or otherwise) when their values so clearly scream that they’re open for business on every front. So saying “hey just ask for what you want and trust the outputs!” seems shady AF. And if we do what some suggested, of making agents fully autonomous, we wouldn’t ever have cause to pause and reflect (let alone catch) this happening.

What I am interested in using these things for

I’m interested in reviewing code humans don’t have time for. Several of the better talks shared the goal of complete code coverage. I’m also interested in putting in guidance and nudges towards doing better work (either from humans or from robots), rather than adding layers on other layers. I’m interested in help for what we know needs doing, and investigations in formats that humans are bad at and machines are good at.

From this conference, I’m now prepared to spend even more time on evaluation than I expected to (50% after baseline systems are in place). And I have new ways of talking about where to interject to inspect the system instead of just trusting it’s working.

I now have more supporting evidence for continuing to think that a workflow or premise needs to be figured out before automation, which happens before AI tooling. And that organizational structures need to allow for this happening at a deep layer, not as something that gets tacked on later as an afterthought.

It also seems like we’re moving away from “zero click attacks” towards “zero user intervention attack” – what can we get agents to do without you noticing?

The first 30 days

Posted on by
1

I’ve now been at GoFundMe for 30 days! Hooray! In an act of reestablishing how I like to learn and share, I asked if I could write blog posts about my experiences in learning to manage, and got a thumbs-up. Here’s what I’ve gotten up to!

Setting up support

There are a bunch of lovely folk in my life who I respect who also manage. I have put them into a Signal thread where I can ask questions. Yes, they’re all pretty spicy and I’m worried about the fights that might happen. Yes, also I have slept with all of them. Yes, it’s already proven pretty invaluable.

I also asked explicitly for a mentor at GFM who’s established there and could counter balance what I have blind spots in. I’ve been set up with someone and we’re off to a good start.

Read a fucking book

There’s a lot to read out there (and I’m making my way through a fair amount of it, mostly recommended by an infosec Slack I’m in), but I also wanted a focused book for this time. My brother recommended The First 90 Days and it’s been REALLY good so far. The parts that are useful are really useful, the parts that are less useful are easy to skip over.

So far, I’ve learned to separate out focus areas into political, cultural, and technical things, and to check in with those around me about which lane I should be focused in most. I’ve also learned to think about if folks are better at sustaining or being a hero, and at what stage of a business. This is all helping as I get to know new folks, I can plug them into my little database of people.

Get to know the people

I’m not just paying attention to the folks that report to me, or the people further up in the chain. I’m also getting to know peers near and far. As we talk, I ask each person if there’s anyone else I should be talking to. Sometimes they have someone not already on my list, but often not. I take notes and structure the data so we can start our next conversation in a more advanced place. Plus, it’s way better to say hello to someone BEFORE something is on fire.

Hack my own tendencies

The first while at a new job should be about learning the terrain and people. Learning what not to step in. Learning root causes to what might seem like disparate problems.

Problem is, if I don’t have a thing to do, I will FIND something to do, and that’s not great for this period of onboarding. So I have found a low-stakes project that touches a lot of what I’m getting up to, plus some interactions with nearby teams, in order to give me a thing to focus on while I move more slowly elsewhere. You probably guessed this, but I’m organizing our documentation and starting to come up with a stronger onboarding story.

Start to make a plan

I’ve started laying out where I think we are, and where I think we can get to next. I’m waiting to bring this to my reports until I have it pretty well dialed in and have broad categories in place. I want a strong story to tell. But I am testing bits and pieces for readiness and accuracy in most conversations I’m having, regardless of with who.

I’m excited! This seems to be really promising.