I'm a mechatronics engineering student in Hungary. Over the last months I wrote two small parsers/validators: one that reads bank statement files (MT940, camt.053, BAI2, CODA) and checks that the numbers add up, and one that checks and converts SEPA payment files (pain.001 / pain.008). Both are deterministic: no AI, no fuzzy matching, the same file always gives the same answer.
This post is about the boring part that took most of the time: the ways these "standard" files differ from the standard. If you have ever written split(":61:") and felt good about it, this is for you.
Part 1: MT940
MT940 is a SWIFT text format from the 1980s. A statement looks roughly like this:
:20:ROLL-2025-12
:25:DE89370400440532013000
:28C:1/1
:60F:C251230EUR1000,00
:61:2512310102C500,00NTRFNONREF
:86:Year end payment
:61:2501150102D200,00NRTINONREF
:86:SEPA return of January 2025 collection
:62F:C260102EUR1300,00
:60F: is the opening balance, :61: is one transaction, :86: is free text about that transaction, :62F: is the closing balance. Simple. Here is where it goes wrong.
1. The :61: entry date has no year
The first six digits of :61: are the value date as YYMMDD. The next four are the booking (entry) date, but only as MMDD. No year.
The obvious fix is "take the year from the value date". Look at the first transaction above: value date 251231, booking date 0102. Taking the year from the value date gives 2 January 2025, which is a year off. This exact bug is reported in the popular Python mt940 library (issue #17).
The second transaction is worse. It is a SEPA direct debit return: the value date is 15 January 2025, but the bank booked it on 2 January 2026. The value date is more than a year before the booking. Any rule based on the value date picks the wrong year.
What worked for me: anchor the booking date to the statement window, not to the value date. The statement says it covers 30 Dec 2025 to 2 Jan 2026 (:60F: and :62F: dates), so for MMDD = 0102 I try the previous, current and next year around that window and take the candidate that falls inside it (with a 7-day margin):
def _resolve_entry_date(mmdd, value_date, window_start, window_end):
if not mmdd or not mmdd.strip():
return value_date # some banks leave it blank
month, day = int(mmdd[:2]), int(mmdd[2:])
anchor = window_end or window_start or value_date
candidates = []
for year in range(anchor.year - 1, anchor.year + 2):
try:
candidates.append(_safe_date(year, month, day))
except ValueError:
continue
lo = (window_start or window_end) - timedelta(days=7)
hi = (window_end or window_start) + timedelta(days=7)
inside = [d for d in candidates if lo <= d <= hi]
return min(inside or candidates, key=lambda d: abs((d - anchor).days))
(Shortened; the real version also handles statements without balance dates.) Both transactions above now get 2026-01-02, and there is a test for exactly this file.
Two smaller date surprises from real files:
- Some banks (ASN, Citi) leave the four entry-date characters as spaces. The regex has to allow
\d{4}| {4}. - German banks book interest on "30 February".
date(2026, 2, 30)raises, so days 29 to 31 are clamped to the last day of the month.
2. :86: is not one format, it is at least four
:86: is "information to account owner". SWIFT barely defines it, so every banking community filled it differently. The ones I had to support:
-
GVC subfields (German banks, Sparkassen, Volksbanken, PostFinance, Raiffeisen):
166?00GUTSCHR. UEBERWEISUNG?20EREF+...?30COBADEFFXXX?31DE89...?32Müller Maschinenbau GmbHA three-digit transaction code, then?NNsubfields: 20 to 29 are purpose lines, 30 is the bank code or BIC, 31 the account, 32/33 the name. -
Slash tags (ING, Rabobank, ABN AMRO, Triodos, KBC, Wise):
/EREF/.../CNTP/NL91ABNA0417164300/ABNANL2A/Name/City//REMI/USTD//text/ -
SEPA keywords in plain text:
EREF+... MREF+... SVWZ+... -
Free text: everything else, like
Year end paymentabove.
And the separators are not fixed either. Polish banks use the ZBP standard with ~ instead of ?: 034~00VE02~20Faktura 123 za usługi~21księgowe~32Zażółć Gęślą Jaźń. I have also seen @, >, | and ^ in exports. My detection regex for the GVC style is now ^(\d{3})([?@>|~^])\d{2}.
The nastiest detail: SVWZ (the purpose text) is often split across subfields in the middle of a word, e.g. ?23Vodafone Rechnung Jan + ?24uar 2026. If you join subfields with a space, you get "Jan uar". The parser has to know which subfields are continuation lines.
Another one: in German files ?30 is often a German bank code (Bankleitzahl, 8 digits), not a BIC. If you put it in a "BIC" column, the downstream payment run fails. There is a test that checks 37040044 is not reported as a BIC.
3. Encodings
MT940 is supposed to be in the SWIFT character set, which has no accented letters. Real files have them anyway, and they don't tell you the encoding.
- Valid UTF-8? Use it.
- Otherwise, Western European banks usually export Windows-1252.
- Hungarian, Polish and Czech banks use Windows-1250, ISO-8859-2 or the old DOS code page 852.
Guessing wrong turns the Hungarian "tűzőgép" (stapler) into "tûzõgép" (Windows-1250 read as 1252); the Polish "Zażółć" becomes "Za¿ó³æ". Being Hungarian, this one was personal. My approach: decode as Windows-1252 first. If the account (:25:) or a balance currency is Central/Eastern European (HU, PL, CZ, ... or HUF, PLN, CZK, ...), try all three CEE encodings and score each result: +1 for every real accented letter of those languages, -3 for every typical mojibake character (±, ¶, ˇ, box-drawing characters, C1 control codes). Highest score wins.
It is a heuristic, but a deterministic one, and it is tested against the same Hungarian sentence encoded in Windows-1250 and ISO-8859-2.
4. The actual check: do the numbers add up?
After parsing, the useful question is simple: does opening + credits - debits equal the bank's closing balance? For the file above:
{
"opening_balance": "1000.00",
"total_credits": "500.00",
"total_debits": "200.00",
"calculated_closing_balance": "1300.00",
"closing_balance": "1300.00",
"discrepancy_amount": "0.00",
"audit_verdict": "VERIFIED"
}
Amounts are Decimal, never float. There are three verdicts: VERIFIED (matches), DISCREPANCY (doesn't match, or the next statement's opening balance doesn't continue the previous closing balance), and UNVERIFIABLE (the file has no closing balance to compare with). A parser that silently drops one weird :61: line still "works", which is why the balance check matters more to me than any single parsing rule: a dropped transaction shows up as a discrepancy instead of a wrong ledger.
Part 2: SEPA payment files (pain.001)
The other direction: files you send to the bank. Here the problems are less about parsing and more about rules that the XML schema does not check.
1. pain.001.001.03 to .09
Many ERPs still produce the 2009 version, pain.001.001.03. Banks are moving to the 2019 version, pain.001.001.09; the German banks' specification limited the old version to November 2025.
When I compared the two schemas' element paths, the differences that actually occur in SEPA credit transfers were small:
-
BICbecomesBICFIfor banks (FinInstnId). -
BICOrBEIbecomesAnyBICfor organisation IDs. -
ReqdExctnDt(execution date) gets a child:<ReqdExctnDt><Dt>2026-10-23</Dt></ReqdExctnDt>. - The namespace changes.
So conversion is mostly renaming. The part I refused to automate is postal addresses. The 2019 version supports structured addresses (StrtNm, PstCd, TwnNm, Ctry), and the EPC is phasing out unstructured ones (address lines only); the end date has been postponed and not set yet. It is tempting to split <AdrLine>10115 Berlin</AdrLine> into postcode and town with a regex. I don't. The converter keeps the lines as they are and reports an UNSTRUCTURED_ADDRESS warning, because a guessed address on a payment is worse than an honest warning. In the new version, if you do use structured fields, town and country are both required; that is an error, not a warning.
The converted file is validated against the 2019 schema and the SEPA rules before it is returned. If it would not be valid, you get the problems instead of a file.
2. Character set
The EPC implementation guidelines define the Latin character set every bank must accept:
a-z A-Z 0-9 / - ? : ( ) . , ' + space
That's all. No &, no ü, no ő. German banks additionally accept Ä Ö Ü ä ö ü ß & *. So the same name is valid or invalid depending on where you send it:
<Cdtr><Nm>Müller & Söhne</Nm></Cdtr>
profile=epc -> CHARACTER_SET: <Nm> contains characters outside the SEPA Latin character set: '&' 'ö' 'ü'.
profile=de -> no errors
That's why my validator has a profile parameter instead of one global rule. Banks differ in whether they reject the file, replace the characters, or (worst) accept it and mangle the name. Also: IDs like EndToEndId must not start or end with / or contain //, and names are limited to 70 characters.
3. Control sums
The group header and each payment block carry NbOfTxs (number of transactions) and CtrlSum (sum of amounts). The XSD only checks that they are well-formed numbers, not that they are correct. When someone edits a file by hand, or an export adds a payment after the header was written, you get this:
CONTROL_SUM at GrpHdr/CtrlSum:
The message control sum is 999.00; the amounts add up to 1284.06.
Banks reject such files, sometimes hours later in a pain.002 status report. Checking it before upload is a few lines of Decimal arithmetic. Same for NbOfTxs, amounts with more than two decimals, non-EUR currencies, and ChrgBr other than SLEV.
Trying it
Both checks run as HTTP APIs. Uploading a statement file looks like this:
curl -X POST "https://bank-statement-reconciler1.p.rapidapi.com/v1/reconcile" \
-H "X-RapidAPI-Key: $RAPIDAPI_KEY" \
-H "X-RapidAPI-Host: bank-statement-reconciler1.p.rapidapi.com" \
-F "file=@statement.sta"
And a payment file goes in as the raw XML body:
curl -X POST "https://sepa-payment-file.p.rapidapi.com/v1/validate?profile=de" \
-H "X-RapidAPI-Key: $RAPIDAPI_KEY" \
-H "X-RapidAPI-Host: sepa-payment-file.p.rapidapi.com" \
-H "Content-Type: application/xml" \
--data-binary @payments.xml
If you just want to see whether your own file parses, there is a free browser demo, no sign-up: bank statement balance check. The file is processed in memory and discarded, and names and IBANs are masked in the result by default.
What I learned
- Write tests from real files, not from the spec. Almost every rule above came from a file that broke.
- Prefer a loud warning to a clever guess (addresses, unknown encodings, missing balances).
- Make the output checkable. A balance check catches parser bugs you don't know you have.
If you have an MT940 or pain.001 file that breaks in a way I didn't describe, I would really like to hear about it in the comments. Weird bank exports are the only way these parsers get better.
