Every night we export the databases behind our main systems to cloud storage. The exports are kept for thirty five days, which covers a month end and a few days either side, and then a lifecycle rule deletes them. For years they sat on the standard storage tier, and the bill for them was about two thousand pounds a month.
Last year one of my engineers proposed moving the exports to the provider's archive tier, which costs a small fraction of standard storage per gigabyte. The arithmetic looked excellent, and nobody argued. A rule was added to move each export to archive the day after it was written.
The following quarter the storage bill had not fallen. It had risen by about a fifth.
Two lines in the price list explained it. The archive tier is meant for data you will keep for a long time, and it charges for a minimum of one hundred and eighty days for everything placed in it. Anything deleted earlier is billed for the remaining days as an early deletion charge. Every export we archived was deleted after thirty four days and charged for the other hundred and forty six. On top of that, moving data into the tier is priced per operation, and some of our systems export in thousands of small files rather than one large one. We were paying to move each file and then paying for months of storage we had already given up.
Nobody had misread the per gigabyte price. We had simply read only that line. The cheapest tier is cheap on one condition, that data stays there for a long time and is rarely touched, and our exports did neither.
The nightly exports went back to the standard tier and then to a cheaper infrequent access tier after a week, whose minimum period is shorter than the life of the files. The archive tier now holds only what we keep for years: the month end exports kept for seven years for the auditors, bundled into one file per system per month before they move. Every lifecycle rule we write is now checked against the minimum storage period of the tier it targets.
A storage price is a price for a pattern of use. Change the pattern and the price you pay is a different one.
– Serguey Shinder
