Gizmodo (“Amazon Ditches ‘Just Walk Out’ Checkouts at Its Grocery Stores“):
Amazon is phasing out its checkout-less grocery stores with “Just Walk Out” technology, first reported by The Information Tuesday. The company’s senior vice president of grocery stores says they’re moving away from Just Walk Out, which relied on cameras and sensors to track what people were leaving the store with.
Just over half of Amazon Fresh stores are equipped with Just Walk Out. The technology allows customers to skip checkout altogether by scanning a QR code when they enter the store. Though it seemed completely automated, Just Walk Out relied on more than 1,000 people in India watching and labeling videos to ensure accurate checkouts. The cashiers were simply moved off-site, and they watched you as you shopped.
Instead, Amazon is moving towards Dash Carts, a scanner and screen that’s embedded in your shopping cart, allowing you to checkout as you shop. These offer a more reliable solution than Just Walk Out. Amazon Fresh stores will also feature self check out counters from now on, for people who aren’t Amazon members.
“We’re rolling out Amazon Dash Cart, our smart-shopping carts,” said an Amazon spokesperson to Gizmodo. Amazon confirmed this feature is replacing its Just Walk Out technology in existing stores.
Just Walk Out was first introduced in 2016, presenting Amazon’s biggest and boldest innovation in grocery shopping. The technology seemed incredible, but there were some stumbles. It often took hours for customers to receive receipts after leaving the store, largely because offshore cashiers were rewatching videos and assigning items to different customers. The system of scanners and video cameras in each store is also incredibly expensive.
According to The Information, 700 out of 1,000 Just Walk Out sales required human reviewers as of 2022. This widely missed Amazon’s internal goals of reaching less than 50 reviews per 1,000 sales. Amazon called this characterization inaccurate, and disputes how many purchases require reviews.
“The primary role of our Machine Learning data associates is to annotate video images, which is necessary for continuously improving the underlying machine learning model powering,” said an Amazon spokesperson to Gizmodo. However, the spokesperson acknowledged these associates validate “a small minority” of shopping visits when AI can’t determine a purchase.
Amazon Fresh, the e-commerce giant’s grocery store first launched in 2007, has just over 40 locations around the United States. The company also owns Whole Foods, and many of Amazon Fresh’s experiments are seen as precursors for the large chain.
There is an Amazon Fresh location not far from me and I’ve shopped there two or three times. While the “walk out” technology is interesting and the stores generally hassle-free, I found the selection wanting and the prices high compared to the Safeway and Giant stores that abound.
But, I must confess, I did not know until this morning that there was a trove of workers in India spying on customers. Indeed, before the local store opened, I remember reading stories like this May 2022 entry from Stuff‘s Connor Jewiss (“Why Amazon Fresh’s ‘just walk out’ tech is the shopping experience we all need“):
The last major innovation to in-person shopping was contactless payments, but that’s old news now. These Amazon Fresh stores are entirely checkout-less, allowing you to, quite literally, just walk out. Packed with all sorts of clever tech, we’re going inside one of the stores to find out how they work, and if they’re any good.
How do Amazon Fresh Stores work?
When you walk into an Amazon Fresh store, things couldn’t be simpler. Just fire up the Amazon app on your phone, head to the basket, and hit Fresh Code just under the search bar at the top. You’ll then see a QR code pop up on your display. Scan this at the turnstiles, just like you’d scan a boarding pass at the airport.
[…]
In order to provide the “just walk out” experience, Amazon needs to know what you’re purchasing and when you leave. It sounds simple, but it’s fairly complicated in reality.
To track where you are, Amazon has installed an array of hundreds of cameras into each Fresh store. They follow you around the shop, but there’s no facial recognition involved. Everything gets processed in store or on the app, and all data gets deleted within 30 days. The smart shop just needs to know where you are.
These same cameras are watching what you pick up and put back, too, so Amazon knows exactly what you buy. There aren’t any giant barcodes on products for the cameras to spot. Rather, they use AI to identify items and where they are in the store to confirm if that’s correct. You wouldn’t be picking up toothpaste in the sandwich isle, would you?
In addition to the cameras, each shelf includes a weight sensor. The sensor detects when you pick an item up, and can work out what it is based on the weight. The AI combines this data with the rest to really make sure it’s got the right products.
And when you’re ready to leave? Head back to the gates, which swing open ready for your departure, like your own personal doorman. There’s a station for you to pack your bags, and for cutlery if you purchased any food items in store. Amazon will automatically charge your preferred card as you exit, and you can carry about your day as usual. Head into your inbox or the app for a receipt.
[…]
Can they be tricked?
Amazon’s so sure that the tech can’t be beaten, there’s not even an option to add products it missed in the app. You can remove items you didn’t purchase, but not add ones you did. That sounds like a challenge to us! Summoning my inner Indiana Jones, I went around the Amazon Fresh store to see what I could nick.
In true Raiders of the Lost Ark fashion, I tried the old switcheroo trick with two similar bottles of water to see if the weight sensors could be fooled. Alas, Amazon Fresh correctly worked out which bottle I took in the end.
Similar tests proved just about as effective – as in not at all. Throwing an item into the Amazon-branded bag as fast as possible didn’t fool the cameras. Picking something up and putting it back didn’t either. Nor did putting it back in the wrong place (which I corrected at the end because, those people).
I vaguely recall a similar article about an intrepid reporter trying the old switcheroo trick with organic and non-organic produce with the same result.
While some folks on the Internet suspected that Amazon was using human spies, most just assumed that Amazon’s AI technology was just that good. Apparently, so did Amazon. Not so much, it turned out.
