I build pay-per-event Actors for the Apify Store. This week I tested what happens to charging when an Actor reboots in the middle of a run, because Actors can be rebooted or migrated during long runs. I tested a forced reboot; I have not tested a platform migration. The result surprised me, so here it is, with the fix I use.
What I tested
I wrote a small private Actor that charges one event per dataset row, then forces a reboot part-way through with Actor.reboot(). I ran it with a low "maximum cost per run" so the limit would matter, and checked the run metadata afterwards. I used Apify SDK for Python 4.0.2.
What I found
After the reboot the SDK reported zero events charged, even though a few had already been charged before the reboot. As far as I can tell, the SDK takes its charged counts from environment variables set when the container starts, so a restarted container starts from the old numbers instead of the current ones.
In one test, the run metadata ended up showing more charged events than the maximum I had set. Apify's documentation says the platform enforces the limit and that charges above the charged count are not billed, so I can't say a customer would have been billed for the extra events. What I can say is that the run's own record disagreed with the limit I had set, and my Actor had no way to notice. I'd rather not rely on the platform to quietly absorb that.
I haven't seen an existing SDK issue describing this exact behaviour, so it may be worth a report.
The fix I use
Three small pieces, all inside the Actor:
-
Keep your own record of charging. In my Actors, a
CHARGE_STATErecord in the run's key-value store is written as "pending" before each charge and updated to "charged" afterwards. After a restart you can read it and know what really happened, even if the SDK's counts say zero. - Dedupe against what you've already stored. Before pushing a row, check whether it is already in the dataset. A restart then can't charge for the same row twice.
-
Keep your own spend limit. Work out how many events you may still charge as: the user's maximum, minus the larger of your own record of earlier spend and the SDK's total, minus what you've charged since this start. Use
Decimalfor the money and round down. Never use floats for prices:(0.011 - 0.005) / 0.002gives 2.9999999999999996 in floats, so rounding down would lose an event.
With those in place my test Actor stopped at exactly the limit I set ($0.011, three rows) in a run that rebooted, and a control run with a higher limit produced no duplicate rows.
How to check your own Actor
Use a throwaway copy, a low per-run cost limit, and a forced Actor.reboot() part-way through. Then read the run metadata and compare the charged counts with the maximum you set, and with the number of distinct rows you expect to have been charged. If they disagree, you have the same gap.
I'm happy to answer questions in the comments.
