Our trade customers order through a portal on our website. Each account has named users, a credit limit and a set of agreed prices, and our terms say that an order placed by a signed in user binds the customer. For fifteen years that was simple, because the person signed in was the person who had decided to buy.
In July one of our larger customers, a facilities company, told us their purchasing team had started using an AI assistant to keep their sites stocked. It watched their stock levels, signed in to our portal with a buyer's username and password, and placed the reorders itself. For the first week our bot protection kept blocking it, because it moved through the pages faster than a person could, and the buyer asked us to whitelist it.
In the third week it ordered four hundred cases of hand towels for a site that needed forty. The assistant had read a pack size from their own system as single rolls rather than cases. The order was picked, invoiced and delivered before anybody at either company looked at it. The customer disputed the invoice, our terms said the order was theirs, and the account manager spent a month resolving it with a credit note and a collection.
Nobody had done anything dishonest. Our portal assumed that the person clicking was a person, and that whoever held the login had the authority to spend. Both assumptions had been true for so long that we had stopped seeing them.
I expect this to become normal. Several of our customers are trialling buying assistants, and the suppliers of their purchasing systems are building them in. They will not all announce themselves.
So we are preparing for orders from software as a separate kind of customer. Software can now order through a programming interface with its own credentials, issued to the customer and never shared with a person. Each set carries limits the customer chooses: which products, how many, how much per order and per week. Orders above those limits wait for a named buyer to approve them. Every order records whether a person or software placed it, and our terms now say so.
For years our systems asked only who had signed in. The question that is coming is who, or what, is acting, and on whose authority.
– Serguey Shinder
