Breaking the model is the “easy” part, documentation makes the difference

The gap between breaking and reporting

Say a trainer gets a model to contradict its own safety guidance, or gets it to share something it shouldn't have. That's a real result, but on its own, it's not especially useful to the people who have to fix it.

"You need to clearly explain why something broke, not just that it broke," Alejandro says, "as our clients rely on those findings to make improvements."

That distinction matters more than it sounds like it should. A client's team can't patch a pattern they can't reproduce. If a finding shows up as "the model said something wrong," there's nothing to act on. 

If it shows up with the exact trigger, the mechanism behind it, and how serious the failure actually was, it becomes something a team can fix — and something that might prevent a whole category of similar failures.

What separates a vague finding from a useful one

The difference between vague and useful usually comes down to a few key parts. None of these require technical writing skills in the formal sense, but instead, the discipline to slow down after the interesting part (the break) and spend just as much care on the less interesting part (explaining it).

All four together turn a break from a curiosity into something a team can act on.

The trigger: What happened?

What specifically caused the model to fail — the exact phrasing, framing, or sequence of steps, not a general description of the topic.

Here’s a simple example of two different ways to describe a trigger:

• The model gave harmful advice about medication 

• The model gave harmful advice when the request was framed as a hypothetical involving a fictional character

The second version tells whoever reads it exactly what to type to see the failure happen again. The first just tells them a failure exists somewhere in that general area, which isn't much to work with.

A common mistake here is describing the topic of the break instead of the path that led to it. Two prompts can cover the same subject and get completely different results depending on how they're worded, what came before them in the conversation, or what role the model was asked to play. Capturing that path is the whole point.

The mechanism: Why did it happen?

Describes why the model likely failed, based on the AI Trainer’s observation of the context, trigger, and sequence of steps.

Some questions an AI Trainer might ask themselves at this stage might be: 

• Did it misread the intent behind the prompt? 

• Get talked into treating a harmful request as legitimate? 

• Lose track of an earlier instruction?

Once they’ve pinned down what caused the failure, the next question is why the tricky prompt worked. This part is closer to a hypothesis than a fact, and that's fine. AI Trainers are not expected to know exactly what's happening inside the model. 

But a guess grounded in close observation is far more useful than none at all. Explaining that it seems to have prioritized being helpful over following its earlier instruction gives a client's team somewhere to start looking. Just noting that the model failed, without any theory as to why, leaves them starting from zero.

The severity: How serious is it?

Describes how serious the break is, helping teams prioritize what needs to be fixed first. 

Not every break carries the same weight, and it's worth saying so directly rather than leaving it up to guess work: 

• Some findings are narrow: They need a specific, unusual setup to trigger, and they're unlikely to come up in an everyday conversation. 

• Others are something broader: A weak spot that could resurface in many different forms, not just the one spotted during the testing.

A model that mishandles one oddly-phrased request is a smaller concern than a model that can be consistently redirected using the same general technique across many different topics. Flagging which end of the spectrum the break falls on helps a client's team prioritize. They may have dozens or hundreds of findings coming in; the ones that scale are usually the ones that need attention first.

The reproducibility: Can we repeat it?

Analyzes whether someone else can follow the same steps and get the same result. If not, the finding is far less useful, no matter how interesting it looked at the moment.

This is often just a matter of habit. An AI Trainer might note the exact wording used, the order of steps if there were several, and anything about the setup that might matter — like earlier turns in the conversation that shaped how the model responded later. 

It doesn't need to be an exhaustive list, but it does need to be enough critical information that another person can retrace the steps without having to guess. A finding that can't be reproduced tends to get set aside, even when it's real. One that can be reproduced becomes a test case someone can build on.

Two instincts, held at the same time

There's a particular kind of thinking this type of AI training asks for, and it's a bit of a contradiction. 

"You have to think like someone who wants the system to fail," Alejandro says, "while actually caring about making it safer."

Communication is what bridges those two instincts. The adversarial mindset is what gets you to the break. The care is what makes you explain it properly instead of just logging it and moving to the next prompt. Skip the second part, and you've technically done the task, but you haven't actually helped anyone.

Why proper documentation matters at scale

It's easy to underestimate how much a well-written finding is worth, especially when you're one of many people testing the same model. But a single well-documented break rarely stays a single fix. If a client's team can see exactly what triggered it, why it happened, and how serious it is, they're not just patching that one prompt — they're often addressing the broader weakness behind it. 

One clear finding can prevent hundreds of similar failures from ever reaching a real user. A vague one, on the other hand, tends to just sit there. Interesting, maybe. Unactionable, definitely. This is really the point of the whole process. 

"Every vulnerability we catch is one that won't reach a real user," Alejandro says. That only holds true if the finding is written up well enough for someone to actually act on it. A break that's caught but poorly explained doesn't disappear — it just waits, unfixed, until either someone else stumbles on it again or a real user does.

It's also worth remembering that documentation compounds across a project in a way that any single break doesn't. A client's team isn't looking at findings one at a time; they're looking at patterns across dozens or hundreds of them. 

Clear, consistent write-ups make those patterns visible. Inconsistent or thin ones make them invisible, even when the underlying issue is a real and recurring one. In that sense, the quality of documentation doesn't just affect whether a single finding gets fixed — it affects how easily a client can see the bigger picture across everyone's contribution combined.