Redact PII in bank logs under GDPR: hash it, don't mask it

Originally published at pii-shield.com/guide-banking-logs-gdpr.

TL;DR: An account number in a log line is personal data, and the pipeline carrying it is processing. Redact it before the line leaves the pod, and put a keyed hash in its place rather than a constant, so the log can still answer "one customer or four hundred?" when the 72-hour clock is running. Below: where banks redact today and where each option breaks, a payments log before and after, how to check it worked, and what an entropy scanner will not find.

An account number in a log line is personal data

Nobody argues about the customer table. The log line that says payment.authorised account_number=GB29NWBK60161331926819 is the same account number, attached to the same person, and Article 4(1) lists "an identification number" among the things that make someone identifiable. So the log is personal data, and everything that touches it on the way to the dashboard is processing: the file on the node, the shipper's queue, the vendor's index, the backup of that index.

In a bank this usually comes up during a DPIA, and more often than not about a system that has been shipping logs to a SaaS for three years. The awkward part is that the lines have already left, and you cannot redact what is stored under somebody else's retention policy.

The rest of this post is about the next line.

How banks redact today, and where each option breaks

Where you redact decides how far the raw value has already travelled and who owns the rule. The four places, from earliest to latest:

This is a comparison of boundaries, not of detection quality. A well-kept Logstash filter catches what its regexes describe. The problem is the word "kept": the list was written for the fields that existed when it was written, and the line has been on disk for a while before the filter sees it.

PII-Shield sits in the second row. It runs as a sidecar in the pod, reads the application's log file from a shared volume, and prints the cleaned line to its own stdout, where the existing collector picks it up with no configuration change. The raw line stays on the pod's volume and goes nowhere. The Fluentd, Datadog and Grafana Alloy guides show the wiring for each collector. This post is about what the redaction should do once it runs.

What goes in place of the value decides whether the log is still worth reading

A constant like REDACTED throws away the one thing that makes a redacted log useful. Forty lines carrying the same leaked account number turn into forty unrelated events. When the 72-hour notification clock under Article 33 is running, the question everyone asks first is whether it is one customer or four hundred, and a log full of REDACTED cannot answer it.

A keyed hash can. Same value, same salt, same tag:

login ok account_number=[HIDDEN:88797d]
export started account_number=[HIDDEN:88797d]
export started account_number=[HIDDEN:be171d]

Two of the three lines are about the same customer, and you can see that without knowing who the customer is. Distinct-count queries work. So do joins across services. Nothing readable is stored anywhere. The tag is an HMAC of the value under a salt you hold, so it is one-way for anyone without the salt and confirmable for anyone with it.

Change the salt per environment and the tags stop lining up between production and the test cluster somebody handed a contractor. The same declined payment, under PII_SALT=0123456789abcdef0123456789abcdef and then under PII_SALT=fedcba9876543210fedcba9876543210:

2026-09-20T10:14:09Z WARN payment.declined account_number=[HIDDEN:88797d] reason=limit_exceeded request_id=7c1d9f
2026-09-20T10:14:09Z WARN payment.declined account_number=[HIDDEN:ba8164] reason=limit_exceeded request_id=7c1d9f

A payments log, before and after

Three lines a payments service might write. Nothing here was hand-edited: the "after" block is the output of the released CLI with a fixed test salt and default settings, so you can reproduce it.

Before:

2026-09-20T10:14:03Z INFO payment.authorised account_number=GB29NWBK60161331926819 card=4539148803436467 customer=j.smith@example.co.uk amount=120.00 session=AbC9xY2kQ8pLmN0rZq7 request_id=7c1d9e
2026-09-20T10:14:09Z WARN payment.declined account_number=GB29NWBK60161331926819 reason=limit_exceeded request_id=7c1d9f
2026-09-20T10:15:41Z WARN transfer.rejected from=31926819 to=44017265 sort_code=60-16-13 channel=mobile msg="beneficiary account closed"

After, PII_SALT=0123456789abcdef0123456789abcdef pii-shield < payments.log:

2026-09-20T10:14:03Z INFO payment.authorised account_number=[HIDDEN:88797d] card=[HIDDEN:b9025c] customer=[HIDDEN:fc65b3] amount=120.00 session=[HIDDEN:7e1d32] request_id=7c1d9e
2026-09-20T10:14:09Z WARN payment.declined account_number=[HIDDEN:88797d] reason=limit_exceeded request_id=7c1d9f
2026-09-20T10:15:41Z WARN transfer.rejected from=31926819 to=44017265 sort_code=60-16-13 channel=mobile msg="beneficiary account closed"

What happened, field by field:

• The IBAN was hidden on both lines with the same tag, 88797d. That is the counting property from the previous section, and it is what lets you see the declined payment and the authorised one belong to one customer.
• The card number passed a Luhn check. The email and the session token were hidden for a duller reason: they look random enough to the entropy scorer.
• The amount, the reason and the request id were left alone. 120.00 is protected by a decimal rule, limit_exceeded and 7c1d9e score below the threshold.
• The third line was not touched at all. Two 8-digit account numbers and a sort code went through in the clear. That is the limit of anything generic, and the next section is about it.

Your own formats are yours to write

A 22-character IBAN clears the entropy bar because it looks random. An 8-digit domestic account number does not, and neither does 60-16-13, and no generic detector can tell either of them from an order id. There is no IBAN detector and no account-number detector in PII-Shield, and I would be suspicious of one: your reference formats are your bank's, so the rule for them has to be yours too. It is one line of configuration:

PII_CUSTOM_REGEX_LIST='[{"name":"acct","pattern":"^\\d{8}$"},{"name":"sort-code","pattern":"^\\d{2}-\\d{2}-\\d{2}$"}]'
PII_ENTITY_TYPE_LABELS=true

The third line again:

2026-09-20T10:15:41Z WARN transfer.rejected from=[HIDDEN:acct:cca553] to=[HIDDEN:acct:6b4e22] sort_code=[HIDDEN:sort-code:e026ca] channel=mobile msg="beneficiary account closed"

The marker now names the rule that fired. With the second variable on, the built-in detectors name themselves as well, so the first line becomes account_number=[HIDDEN:entropy:88797d] card=[HIDDEN:card:b9025c] and a reviewer can tell a card from a session token without seeing either.

That label is the part I got wrong for a while. For the first six months every match came out as the same bare marker with no type in it, so a redacted line told you something had been hidden and nothing about what kind of thing it was. An account number and a session id were indistinguishable. Type labels came later, and they are still off by default, because changing the shape of every marker breaks whatever people had already built on the old one. Turn them on in staging first and look at what they say.

Verifying it actually worked

A redaction you have not checked is a belief. The first check is the one I would not skip: put a synthetic account number, a test card and a made-up email into a staging log line you control, then grep -c for each of them on the collector side. The answer has to be 0. Reading the config tells you what you intended; this tells you what happened.

Then take a few thousand lines of production-shaped logs with test data, run the CLI on them with a fixed salt, and read the diff. You are looking for two things at once, values that survived and values that should have been left alone.

Run the same sample with PII_ENTITY_TYPE_LABELS=true. A card that comes back as entropy rather than card is still hidden, but the Luhn path did not see it, and the next format change may not be so lucky.

Two smaller checks. Log the same test identifier from two services under the same salt and confirm the tag is identical; if it is not, the salts differ, and your incident counts will be wrong on the day they matter. And with metrics enabled, watch piishield_redaction_events_total, a Prometheus counter with a type label: a drop in card after a deploy is a format change somewhere upstream, a rise in entropy is usually a new field logging something random.

Last, every rule that protects a value from redaction can also hide a leak. After you add one to the safe list, run the planted-value check again.

Honest limits

Values without a shape are not found. Card numbers, keys, emails and tokens have one. A customer's name in a free-text complaint field does not, and no threshold will find it. This line comes out of the scanner unchanged:

complaint="Mr John Smith says the card was blocked twice"

Finding names needs a model, which costs latency and money, and that is a different tool at a different point in the pipeline. The practical answer for logs is usually to decide that free text does not belong in them, rather than to try to scrub it after the fact.

Some ordinary values look random to the scorer. An 18-character snake_case enum and a short hex-looking id can both cross the threshold. In the sample I ran for this post, the first false positive was a request id and the second was a reason code. Neither needed hiding. Here they are, then the safe list that fixes them, then the line again:

2026-09-20T10:16:02Z WARN payment.declined reason=[HIDDEN:fd4d4d] request_id=[HIDDEN:c4c401]

PII_SAFE_REGEX_LIST='[{"name":"reason-enum","pattern":"^[a-z]+(_[a-z]+)+$"},{"name":"req-id","pattern":"^req_[0-9a-f]{4}$"}]'

2026-09-20T10:16:02Z WARN payment.declined reason=insufficient_funds request_id=req_8f3a

You will find a handful of these in the first diff. Add them to the safe list by shape, not by value, and then re-run the planted-value check, because a safe-list rule that is too wide is a leak with a name.

Hashing is pseudonymisation, not anonymisation. That is Article 4(5). The data stays personal data; you still owe retention, subject rights and a record of processing. What you get is a smaller blast radius when the wrong dashboard is open in front of the wrong person, and Article 32 does count pseudonymisation as a security measure. Your DPO gets the final word on the classification, not this post.

And the old lines are gone. Nothing here reaches back into what a vendor already stored; that conversation is about the vendor's retention and deletion process.

Two operational notes. Rotate the salt and correlation resets: tags before and after a rotation do not match, so plan rotations like key rotations, with a date and a note in the runbook. And the sidecar reads a file: an application that only writes to stdout needs to write to a shared volume instead, or use the in-process library, because transparent stdout capture is still at the design stage. The full list is in KNOWN_LIMITATIONS.md.

Next steps

1. Run the CLI on a sample. Grab the binary or the container from the repository, set a fixed test salt, pipe a few thousand lines through it and read the diff. Note what was missed and what was hidden by mistake.
2. Write the rules for your formats. Account numbers, sort codes, internal customer ids, reference numbers. One PII_CUSTOM_REGEX_LIST entry each, named, so the marker says which one fired.
3. Pick the collector guide and wire the sidecar. Fluentd and Fluent Bit, ELK, Datadog, Loki, OpenTelemetry or Grafana Alloy. Then run the planted-value check on the collector side, not on the pod.

The next log line is the one you can still do something about. PII-Shield is Apache-2.0 on GitHub. Run it on a sample before you run it on a cluster.