Security
OT network anomaly detection with fuzzy hashing (TLSH)
How a TLSH similarity hash over Modbus/TCP packet clusters flags drift, the alternatives that failed, and what the method misses.
Computing America Cyber. Security assessments, penetration testing and staff training.
Send a capture
In short
- Industrial control traffic is largely deterministic, so a deviation from its baseline is a credible signal.
- Cryptographic hashes are built to destroy similarity. Similarity-preserving hashes do the opposite. Small input changes produce small digest changes, and a baseline needs exactly that.
- A single Modbus request can fall below TLSH's 50-byte minimum, so there is nothing to hash. The unit that works is a cluster of packets from one device conversation.
- A sliding window of cluster digests follows normal operational drift. A patient attacker can still walk it.
- The technique is a primary indicator. It tells you something is off without saying what, and it needs a second check behind it, against another source such as a change record or the device's own logs.
We flag anomalies in Modbus/TCP traffic by hashing clusters of packets from one device conversation with TLSH, then scoring each new digest against a sliding window of recent ones. We published the method as "Anomaly Detection in ICS Networks with Fuzzy Hashing" at the 2024 IEEE Cyber Awareness and Research Symposium (opens in a new tab), where clustering worked best of the four approaches the paper tried. The paper evaluated it on two public capture datasets. Our own tuning is labeled as ours below.
Why regular traffic helps¶
Enterprise networks are noisy: people browse, applications update, and normal is a wide distribution. Industrial networks send the same function codes between the same controllers at the same intervals, day after day. That predictability helps an attacker, but it also makes anomaly detection tractable. A departure from the pattern is rare here.
Why not an ordinary hash¶
A cryptographic hash changes completely when one byte changes. Two frames that differ only in a transaction ID get unrelated digests, so it can't measure similarity. Fuzzy (locality-sensitive) hashes invert that: small input changes produce small digest changes, and comparing two digests is cheap. The technique comes from malware clustering, where it's mature. The paper compared three candidates and chose TLSH (opens in a new tab).
| Candidate | Result | Verdict |
|---|---|---|
| SSDEEP | On packet captures the paper found it gave "exclusively false positives" in most approaches, and it mostly returned "0 or near 0." The likely cause is ours, not the paper's: ssdeep derives its block size from input length, so short inputs share few trigger points. | Dropped. |
| NILSIMSA | Slower than TLSH (about a minute per ten megabytes, single-threaded) and prone to false positives outside email. | Dropped. |
| TLSH | Fast, accurate, scores inputs as small as fifty bytes, and per the paper more resistant to manipulated input than the others: its sliding window and k-skip n-grams offer "a defense against algorithmic file alterations intended to preserve high similarity scores." | Chosen. |
Packets are too small to hash¶
Why not hash each packet? A Modbus/TCP message carries a 7-byte header, a function code and a small payload, and the protocol caps it at 260 bytes (the serial form caps at 256). The Modbus specification (opens in a new tab) sets those limits. A read request is 12 bytes: the header, a function code, a starting address and a quantity. The TLSH README says "In default mode the data must contain at least 50 bytes to generate a hash value" (opens in a new tab), so a request that short gets no digest at all. The paper needed a bigger unit of input and tried four.
- Signature analysis (hash known-bad packets, compare to a blacklist). Failed: the frames are too short and too similar for a blacklist to discriminate.
- Single-packet inspection. Failed for the size reason above, and drowned in false positives.
- Whole-capture hashing against a reference set. It shows a capture came from the same network but can't localize an attack inside it, so it's a forensics tool and no detector.
- Cluster hashing over a sliding window: group the packets of one device conversation, hash each cluster, compare each new digest to the recent window. This worked.
How the bytes are represented matters as much as the unit. The paper tested four encodings (string, binary, hex and sorted bytes) and measured average similarity separately on known-similar and known-different pairs. String separated the two best.
% similarity
- String, similar pairs91.33%
- String, different pairs4.80%
- Binary, similar pairs74.97%
- Binary, different pairs9.83%
- Hex, similar pairs89.13%
- Hex, different pairs31.03%
- Sorted bytes, similar pairs99.17%
- Sorted bytes, different pairs95.43%
A high score on similar pairs means nothing if different pairs score high too, and sorted bytes failed that test.
SourceAverage similarity between packet hashes over multiple runs, from section IV of the CARS 2024 paper.
The window keeps the baseline honest¶
One digest per cluster is a measurement. The detector scores each new digest against the recent window of digests for the same device pair and surfaces clusters whose best similarity to it is unusually low. Because the window advances, the baseline follows normal drift.
Span port
Passive capture. No agent, no scanning.
Cluster packets by conversation
Sized to about one full repeating cycle.
One TLSH digest per cluster
Compare against the recent window
Same device pair.
Flag low similarity
Second check
Against a change record or device logs.
Human review
def score(cluster: list[Packet], window: Deque[str]) -> float | None: """Score a conversation against the rolling baseline, not a packet against a rule. Plant floors are periodic; the rule never fires.""" h = tlsh.hash(b"".join(p.payload for p in cluster)) if h is None: return None # under 50 bytes of entropy: no opinion offered return min(tlsh.diff(h, prior) for prior in window) / DIFF_CEILINGCluster size is the main tuning knob; the paper uses a variable size N. Too small and you're back to the single-packet failure. Too large and a short attack is diluted by a long stretch of ordinary traffic. In our own use we tuned it per device pair, to roughly one full cycle of that conversation.
The paper evaluated the method on the CIC Modbus 2023 dataset (opens in a new tab) and the ICS PCAPS dataset on GitHub, both with labeled attack windows. The paper says the cluster method "worked the best" of the four approaches. It states no detection or false-positive rate, so read the ranking as qualitative.
What it does not do¶
- It isn't a verdict. A similarity score says something is off, not what, and it belongs in front of a second check against a change record or the device's logs, and a person.
- It can be walked. A sliding baseline follows an attacker who shifts traffic slowly enough that each cluster resembles the last few, and every step scores as normal. The paper doesn't test this. Scoring against a fixed, reviewed reference set alongside the window would catch slow drift, at the price of flagging legitimate change; that is a design option, not something we measured.
- It covers one protocol, Modbus. Real plants also speak DNP3, BACnet, EtherNet/IP and a long tail of vendor protocols. The approach should generalize, but each needs its own parser and cluster sizing.
- Canonicalizing cluster bytes before hashing, as intrusion detection rules normalize content, should cut false positives. It's future work, as is layering learned attribution on the similarity signal.
Ask how any plant detector was tested¶
Before you buy any plant detector, ask what its unit of input is, which attacks it was tested against, and on whose equipment. Ask what detection and false-positive rates it reports, and whether anyone tested slow drift.
Sources
- 1.Anomaly Detection in ICS Networks with Fuzzy Hashing (opens in a new tab), IEEE Cyber Awareness and Research Symposium (CARS) 2024,
- 2.TLSH: Trend Micro Locality Sensitive Hash (opens in a new tab), Trend Micro
- 3.CIC Modbus Dataset 2023 (opens in a new tab), Canadian Institute for Cybersecurity, University of New Brunswick
- 4.MODBUS Application Protocol Specification V1.1b3 (opens in a new tab), Modbus Organization,
Next step
Send us a capture.
Twenty minutes of traffic off one segment, and we will tell you whether it is regular enough to baseline and where this technique goes blind, which sets what an assessment of that segment should test.
- Phone
- (214) 723-2510
- Reply
- A person replies, not a sequence: within one business day, from someone who would be on the engagement.
Packages
Start with a package
Four packages, each with its price on the page. Pick one and buy it online.
$199/month
Billed monthly
$182.42/month
Billed yearly
$499/month
Billed monthly
$457.42/month
Billed yearly
- Scammers stopped from sending email as you
- Your website and systems scanned every week
- Your email settings checked every week
- Everything in the 1 to 10 plan
- Covers up to 50 people
$199/month
after a one-time setup fee
- A site planned, designed and built for you
- Hosting and updates every month
$499/month
Billed monthly
$457.42/month
Billed yearly
$999/month
Billed monthly
$915.75/month
Billed yearly
- Your Google profile, fully set up
- A Google post every week
- 2 social posts a week
- Everything in Get found
- 4 social posts a week
- Your top 10 Google searches tracked
from$500/month
after a one-time setup fee
- Your data gathered from the tools you use
- A first analysis that answers your questions
- New findings every month
Related reading
Security5 min readOT network segmentation: when the drawing and the switch disagree
The plant network diagram shows a segmented plant. The switch can show a permissive VLAN rule, a two-homed workstation, and a vendor's remote path that isn't on the drawing.
Data5 min readMachine learning intrusion detection: what the data stages taught us
What we learned building an ML intrusion detector from raw packet capture to live alerts, and which stages were data work.
Security3 min readPenetration test vs vulnerability scan: what you are actually buying
Both are legitimate purchases. A sample report and a look for chaining show which one a quote is selling, and four questions test the vendor behind it.
