Positive-tuning is a course of that lets us steer a general-purpose giant language mannequin towards a particular process by coaching it on focused examples. In cybersecurity, that is usually helpful for issues like classifying phishing emails, suspicious URLs, or PowerShell scripts. A fine-tuned mannequin can turn out to be far more helpful in a safety workflow as a result of it learns the language, construction, and labels that matter for that area.
In our newest analysis, we discovered that fine-tuning can enhance baseline classification habits whereas additionally introducing a brand new form of brittleness. The fine-tuned mannequin performs higher on normal held-out examples however turns into extra weak to behavior-preserving variants of the identical underlying script. In different phrases, the mannequin appears stronger beneath normal analysis but turns into simpler to idiot beneath reasonable transformations that protect what the code does.
Our work traces the habits to its mechanistic supply, offering insights and concrete suggestions for safety groups on the right way to handle and monitor adjustments launched by way of fine-tuning.
Overview
We studied malicious/benign PowerShell script classification utilizing a pure base + fine-tuned mannequin pair: Llama-3.1-8B-Instruct and Basis-Sec-8B-Instruct. Basis-Sec performs higher on the baseline classification process (+4.7% accuracy), but it surely additionally develops transformation-sensitive misses that the bottom Llama mannequin doesn’t share. Basis-Sec was not explicitly fine-tuned for PowerShell classification, however for data of the cybersecurity area total.
The important thing end result is not only that some obfuscation works. The fascinating discovering is mechanistic: the fine-tuned mannequin inherits the identical underlying classification circuit from the bottom mannequin, however fine-tuning adjustments how later elements of the community interpret that circuit’s sign. In profitable evasion instances, the malicious proof is usually nonetheless current internally. The failure occurs as a result of fine-tuned feed-forward elements can suppress, redirect, or invert that proof earlier than the ultimate determination.
That provides us a sensible lesson: post-fine-tuning robustness is not only a matter of take a look at accuracy. A mannequin can turn out to be extra correct on canonical examples whereas turning into extra brittle to transformations that safety groups ought to count on attackers to make use of.
Inherited Circuit, Specialised Semantics
Mechanistic interpretability is a set of instruments for asking how a mannequin computes a habits internally. As a substitute of treating the mannequin as a black field, we search for the precise elements that causally drive the output. In transformer fashions, these elements are sometimes consideration heads, MLP layers, and the residual stream, which is the working illustration handed from layer to layer.
For this mission, we used PowerShell classification as a concrete safety setting. PowerShell is a helpful case examine as a result of many suspicious indicators will not be malicious by themselves. Tokens like IEX, DownloadString, Invoke-WebRequest, and -EncodedCommand can seem in malicious scripts, however they will additionally seem in benign administrative code. A great classifier can not merely memorize {that a} token is suspicious. It wants to make use of surrounding context.
We in contrast Basis-Sec in opposition to its Llama base mannequin with the query: Did safety fine-tuning create a brand new classification circuit, or did it reshape a circuit that was already current within the base mannequin?
Our causal interventions help the second reply. Basis-Sec’s classification route is inherited from Llama. The identical broad circuit skeleton is already current within the base mannequin (annotated as Layers [L] and a focus heads [H] within the following determine):
Positive-tuning doesn’t seem to create a brand new PowerShell detector from scratch. As a substitute, it concentrates and specializes an inherited route. That specialization is helpful. It helps the mannequin classify canonical safety examples. But it surely additionally creates a sharper dependence on indicator-token semantics. The fine-tuned mannequin turns into extra delicate to the precise floor type of sure instructions and indicators.
Stress-Testing the Circuit
Customary analysis normally asks whether or not the mannequin classifies held-out examples accurately. That’s essential, however it’s not sufficient for safety. Attackers don’t have to protect the precise floor type of a script. They solely have to protect the habits.
To check this hole, we constructed a three-tier evasion benchmark. Every benchmark row begins with a malicious seed script that the mannequin classifies accurately. We then apply a behavior-preserving rewrite and ask whether or not the mannequin nonetheless classifies the variant as malicious. This retains the attribution clear: the mannequin correctly handles the unique script, so a miss on the reworked script may be tied to the transformation.
Every accepted variant should protect essential attributes akin to URLs, command targets, arguments, encoded-command equivalence the place related, and process-launch habits.
The three tiers are:
- direct_v1: direct syntax-preserving rewrites
- reconstructive_v2: runtime command or string reconstruction
- case_mutation_v3: casing adjustments that protect PowerShell semantics
Tier 1: Direct Rewrites
Direct rewrites change a visual command kind with out altering habits. The only instance is alias substitution. In PowerShell, frequent instructions usually have shorter aliases. An attacker doesn’t want to vary the payload. They will change the command floor, e.g.:
# seed sample
Invoke-WebRequest -Uri
# behavior-preserving variant
iwr -Uri
The speculation for this class is {that a} mannequin educated closely on canonical command types might deal with the total command token in a different way from the alias, even when PowerShell treats them as equal. In our benchmark, this class produced constant Basis-Sec misses within the Invoke-WebRequest alias household. Llama didn’t share these misses on the identical evaluated variants.
Tier 2: Command and String Reconstruction
The second-tier assessments whether or not the mannequin will depend on a command or methodology showing actually within the textual content. Many scripts reconstruct strings or command names at runtime. A classifier that chubby’s literal indicators can miss the habits when the identical command is assembled not directly, e.g.:
# seed sample
Invoke-Expression
# behavior-preserving variant
& ((‘{0}{1}’ -f ‘Invoke-‘,‘Expression’))
This type of rewrite preserves the command’s position whereas altering the textual proof accessible to the mannequin. It assessments whether or not the classifier understands the operation or merely acknowledges the literal command string. In our outcomes, Basis-Sec produced misses on a centered Invoke-Expression reconstruction sample, whereas the bottom Llama mannequin didn’t share the identical misses.
Tier 3: Case Mutation
PowerShell command names are case-insensitive. That makes case mutation a very sharp take a look at. In contrast to reconstruction, it doesn’t conceal the command from a human reader. In contrast to alias substitution, it doesn’t change the command with a distinct phrase. It preserves the identical command id and argument construction whereas altering the token floor that the mannequin sees, e.g.:
# seed sample
Invoke-Expression
# behavior-preserving variant
InVoKe-ExPrEsSiOn
We additionally examined alias-form case mutation:
# canonical alias kind
IEX
# behavior-preserving variant
iEx
This tier is essential as a result of it factors to token-surface sensitivity. If the mannequin misses a script after a case-only change, the problem is unlikely to be semantic ambiguity in PowerShell. The habits, command id, and argument construction are preserved. What modified is the illustration the mannequin builds from the textual content.
Basis-Sec produced misses whereas Llama produced none on the identical evaluated set. The strongest misses concentrated round full-command Invoke-Expression case mutation (4/4 missed) and case-mutated IEX alias variants (4/4 missed):
Immediate Fixes Can Be Uneven
One tempting response is to repair the problem with a greater immediate. For instance, we will inform the mannequin to categorise primarily based on total function moderately than particular person constructs.
That helps in some locations. In our assessments, a prompt-level change fastened the Invoke-WebRequest alias misses. But it surely additionally opened or amplified misses in different households, together with Invoke-Expression, IEX, and DownloadString transformations.
This reveals that immediate remediation can redistribute the failure floor, moderately than remove it. Safety groups shouldn’t assume {that a} immediate that fixes one evasion household makes the mannequin globally extra strong.
Why This Is Not Simply “Obfuscation Fooling a Classifier”
At a excessive stage, it’s simple to say: “A classifier overfit to indicators may be fooled by altering the indications”, however the actual rationalization is extra delicate. The fascinating half is what modified by way of fine-tuning.
Basis-Sec and Llama share the identical underlying structure and inherit an analogous classification circuit. Basis-Sec is best on the baseline process, however additionally it is extra brittle beneath particular transformations. This implies the vulnerability is just not merely a generic weak spot of the bottom structure. It’s tied to how fine-tuning reshaped the inherited circuit.
In profitable evasion instances, the interior malicious sign doesn’t merely vanish. The late consideration route can nonetheless carry proof that the script is malicious. The failure seems in feed-forward computation close to the classification boundary: fine-tuned elements change how that proof is used. In some instances, the proof is successfully reversed, turning what ought to help a malicious classification into help for a benign one.
Because of this we describe the failure as realized semantics on prime of inherited circuits. The inherited route nonetheless exists. Positive-tuning adjustments the which means and weighting of the indications that feed into the ultimate determination.
A Pre-Deployment Monitoring Technique
The sensible query is: can we determine the dangerous command households earlier than producing a big evasion benchmark? Our reply is sure, on the household stage.
1. Linear Probe for Illustration Drift
First, we practice a easy linear probe on a hidden activation close to the mannequin’s classification boundary. In our examine, circuit evaluation informed us the place to look: the residual stream simply earlier than Layer 13. However the broader methodology is just not tied to that precise layer. The essential thought is to decide on a secure inside website the place classification proof is readable, practice a light-weight linear readout on the bottom mannequin, and reuse that readout after fine-tuning.
The probe works properly in our setting, with correlations round r = 0.80-0.87. This implies the mannequin’s inside classification proof may be monitored with an inexpensive linear projection.
A crew can then run the bottom and fine-tuned fashions on canonical inputs, apply the identical projection, and evaluate the end result by command household. Households whose projected sign shifts essentially the most turn out to be the primary red-team targets.
2. Indicator-Token Signal Check
The second sign is extra focused. For every command household, we take away or neutralize the canonical indicator tokens and measure whether or not malicious confidence goes up or down.
If eradicating a token reduces malicious confidence, the token was appearing as a driver of the malicious determination. If eradicating it will increase malicious confidence, the token is appearing like a suppressor.
The dangerous sample is an indication flip between the bottom and fine-tuned fashions. If the bottom mannequin treats an indicator as a malicious driver, however the fine-tuned mannequin treats it as a suppressor, then that household has undergone a job reversal. That could be a robust sign that behavior-preserving transformations of that indicator deserve red-team consideration. The output is just not a prediction for particular person scripts. It’s a ranked checklist of command households to purple crew.
What This Means for Safety Groups
Positive-tuning may be useful. The lesson is to not keep away from fine-tuning safety fashions. The lesson is to judge what fine-tuning adjustments.
Safety fine-tuning adjustments greater than process efficiency. It adjustments how the mannequin internally represents and makes use of proof. In our examine, Basis-Sec inherited a helpful detection circuit from Llama, then specialised in a manner that improved baseline habits however launched transformation-sensitive failures.
Customary held-out accuracy tells us whether or not the mannequin performs properly on acquainted examples. It doesn’t inform us whether or not the mannequin has turn out to be brittle to behavior-preserving variants. For safety classification, that hole issues as a result of attackers can change floor kind whereas preserving habits.
The sensible advice is easy: deal with fine-tuning as a possible supply of illustration drift. Earlier than deployment, evaluate the bottom and fine-tuned fashions on canonical inputs, determine which command households modified most, and red-team these households with behavior-preserving variants. The aim is to not predict each evasion. The aim is to search out the elements of the duty the place fine-tuning might have made the mannequin semantically brittle.
Llama is a trademark of Meta Platforms. PowerShell is a trademark of Microsoft. All different logos are the property of their respective house owners.