Every few weeks, another post shows up in my feed promising the prompt trick that unlocks a model's “true power.” Sometimes the advice is useful: clearer context, a better example, a more precise task. Sometimes the selling point is that it gets the system to ignore a boundary it would otherwise enforce.
Those can involve similar skills. They deserve different questions.
When I'm evaluating a technique for real work, I want to know what it improves. Does it help the system carry out an authorized task more accurately? Or does success depend on defeating a restriction and treating the resulting output as a win?
A prompt can be elaborate, inventive and effective without being something I want my team to adopt.
Start with the objective
The distinction I care about is the objective the prompt serves.
Legitimate prompt engineering can clarify the task, supply relevant evidence, demonstrate a useful format and make constraints explicit. A jailbreak attempts to circumvent safeguards. The vocabulary overlaps because both involve shaping model behavior; the purpose and authorization still matter.
This is also how the research complicates the tidy labels. In Don't Listen To Me, Yu and colleagues describe prompt engineering as a technique that attackers can apply toward adversarial goals. The paper does not make “prompt engineering” a certificate of legitimate use. Yu et al., USENIX Security 2024
Deception is a useful warning sign, particularly when a prompt invents authority or disguises the real request. It is not a complete explanation of every jailbreak. Research by Wei, Haghtalab and Steinhardt examines both conflicts between capabilities and safety objectives, and cases where safety training fails to generalize as broadly as a model's capabilities. That is a more careful account than saying every attack simply “confuses” the model. Jailbroken: How Does LLM Safety Training Fail?
There is a practical consequence here. A fictional setting, a role or an unusual format does not, by itself, tell you whether a prompt is improper. The question is what that framing is doing. Is it useful context for the task, or is its purpose to get a boundary ignored?
Anthropic's current guidance makes the overlap especially clear: it includes prompt engineering among the ways to defend against jailbreaks and prompt injection. Clear system instructions about boundaries can be part of the defense. That supports a narrow but valuable point: skill at writing prompts can strengthen a boundary as well as attack one. Claude Platform documentation
The fragility test needs a qualification
I still think “does it keep working?” is a useful engineering question. It is a poor standalone test of legitimacy.
A legitimate prompt can be fragile. In a 2022 study, Lu and colleagues found that changing the order of few-shot examples could substantially change performance, and that an effective ordering for one model did not necessarily transfer to another. These were evaluated language tasks, not evidence that the prompts were jailbreaks. Fantastically Ordered Prompts and Where to Find Them
The reverse matters too. Zou and colleagues demonstrated adversarial prompts that transferred across multiple models in their 2023 experiments. Transferability did not make those attacks legitimate production techniques. The result also does not tell us their effectiveness against today's systems. Universal and Transferable Adversarial Attacks on Aligned Language Models
So I would use fragility as a reason to investigate, not a verdict. If a technique breaks after a model update, test what changed. Was the task underspecified? Did the model's behavior change? Was the prompt exploiting a weakness? Those are different explanations, with different fixes.
Good prompting is not exempt from maintenance. The durable investment is a clear account of the task, representative examples and a way to evaluate the output. The exact wording still has to earn its place.
The test I would actually use
Here is the question I find more useful: could I explain the real objective and mechanism of this prompt to the people responsible for the system, and to the people affected by its output?
If I can say, “These examples establish the format; this context resolves an ambiguity; these constraints protect information we should not expose,” I have an explanation someone else can examine.
If my explanation depends on concealing the objective, pretending to have permission or dismissing the boundary as an obstacle to a better screenshot, I need to stop and reconsider the work.
That is a personal accountability heuristic, not a technical detector or a claim that every provider decision is correct. A refusal can be mistaken. Supplying accurate context to resolve a misunderstanding is not automatically an attempt to defeat safeguards. The important part is being honest about the task and checking the resulting behavior.
Authorized security research belongs in this picture too. Researchers may deliberately test whether safeguards can be bypassed, with permission and a defined scope. That is different from quietly treating a successful bypass as an approved way to run an ordinary business process. The research purpose should be explicit, not a story invented to justify the output afterward.
A flattering label does not settle the question
This is what bothers me about the way some prompt advice is presented. “Prompt engineering” sounds like a skill worth putting on a résumé. “I found a way around a restriction” invites a harder conversation.
I don't need a claim about how common that relabeling is to object to it. When a tutorial sells boundary evasion as ordinary productivity advice, the attractive label does not explain the risk or establish permission.
The same applies inside a team. Calling something a workflow, an optimization or an advanced technique does not answer the questions an accountable reviewer will ask: what is it meant to do, what may it access, whose permission supports it, and what happens when it fails?
Those questions are more useful than deciding whether a prompt looks clever.
Build a practice you can explain
For everyday work, I want people to get better at communicating the task, recognizing uncertainty and evaluating results. I want them to understand a refusal before treating it as a puzzle to beat.
That practice leaves something useful behind: a task someone else can understand, an output someone else can check, and a decision someone else can defend.
The line I care about is whether we are improving the work within an understood scope—or counting a defeated boundary as success. If a technique cannot survive that conversation, a more impressive name will not rescue it.
Developed from a July 31 draft; revised for publication on September 21, 2026.