Amazon Expands Just Walk Out technology to 200+ stores, including college campuses, with advanced AI integration

Amazon’s Just Walk Out technology is now available in 200+ stores globally. The checkout-free shopping solution can be found in locations on the likes of college campuses, and in stadiums, airports, and hospitals.

In a LinkedIn post, Ritu Subash, Head of Professional Services - Just Walk Out Technology, said: "Incredibly proud to lead the team that deployed Just Walk Out at 200+ stores! If you haven't already, experience the magic of JWO at a store near you."

Amazon previously announced last month, “This school year, universities like Emory University, University of Pittsburgh, University of Maine, and University of Virginia are opening new stores, bringing the number of Just Walk Out stores on college campuses to more than 30 worldwide.”

AI

Amazon recently announced an advancement in the technology behind Just Walk Out.

A new multi-modal foundation model increases its accuracy by using the same transformer-based machine learning models underlying many generative AI applications, and applies them to physical stores.

In an online post, Jon Jenkins, Vice President, Just Walk Out technology, AWS Applications, said: “We accomplish this by analysing data from cameras and sensors throughout the store simultaneously, instead of looking at which items shoppers pick up and put back in a linear sequence.”

“For retailers, the new AI system makes Just Walk Out faster, easier to deploy, and more efficient. For shoppers, this means worry free shopping at even more third-party checkout-free stores worldwide.”

Just Walk Out uses cameras, weight sensors, and a combination of AI technologies to enable shoppers in physical stores to buy food, beverages, merchandise, and more, without having to wait in a checkout line, or stop at a cashier.

The technology, which launched in 2018, was built using genie and machine learning available at the time to figure out “who took what.”

Previously, the AI system analysed shopper behaviour sequentially - their movement and location in the store, what they picked up, and the quantity of each item - each action processed one after another.

However, in unusual or novel shopping scenarios (such as if a camera view was obscured due to bad lighting or a nearby shopper), the sequential approach could take time to determine purchases with confidence, and sometimes required manual retraining of the model.

Jenkins said: “The new Just Walk Out multi-modal foundation model for physical stores is a significant advancement in the evolution of checkout-free shopping.”

“It increases the accuracy of Just Walk Out technology even in complex shopping scenarios with variables such as camera obstructions, lighting conditions, and the behaviour of other shoppers, while allowing us to simplify the system.”

“The new Just Walk Out AI system is able to achieve higher levels of accuracy by analysing all sensor data simultaneously, rather than sequentially. It looks at multiple inputs - cameras, weight sensors, and other data - and prioritises what’s most important to accurately determine the variety and quantity of items selected.”

“It also uses continuous self-learning and transformer technology, a type of neural network architecture that transforms inputs (sensor data, in the case of Just Walk Out) into outputs (receipts for checkout-free shopping).”

For example, a shopper might pick up and put down multiple varieties of yogurt, in different combinations, and as they are doing so, another customer might reach for the same item, or the freezer door could fog up, obscuring the cameras’ view.

In complex situations like these, the new model can quickly and accurately determine the actual items taken by each shopper.

This is thanks to its ability to simultaneously process inputs from various sources (including, in this scenario, weight sensors on the fridge shelves), continuously learn from these inputs, and decide which are most important in order to accurately sort out who took what.

This minimises receipt delays and increases the ease of deployment of the technology for retailers.

The new Just Walk Out AI model reduces the rare need for model retraining in unfamiliar shopping scenarios, as the self-learning system continues to teach itself.

To support Just Walk Out’s continuous learning, it is trained on a 3D map of the store to understand placement of fixtures (such as shelves, fridges, freezers, coffee machines, and more) and products, as well as an image catalog of the store’s merchandise so it can visually recognise every item accurately.

This enables the new AI system to recognise shopper behaviour, even if the store is remerchandised and items are rearranged or misplaced.

Watkins said: “The improvements to our AI system are so seamless that you will continue to enjoy the same contactless checkout-free shopping experience you’ve come to expect at Just Walk Out stores, all while protecting your privacy.”

“Upon entering a Just Walk Out store you present a form of payment at the entry gate. The system immediately associates you with that payment method and begins mapping your journey around the store, adding or subtracting items from your virtual cart as you pick them up or put them down.”

“Just Walk Out does not collect any biometric information - the system only tracks how you interact with the products and fixtures (such as shelves or fridges), correctly identifying the products and quantities you leave the store with.”

Just Walk Out is currently available in over 170 third-party locations at the likes of airports, stadiums, universities, hospitals, in the US, UK, Australia, and Canada.

Amazon says it will launch more Just Walk Out stores in 2024 than any year prior, more than doubling the number of third-party stores with the technology this year.

Watkins concluded: “As we scale, the system will continue to learn from everyday shopping scenarios and raise the bar for accuracy and convenience, delivering the benefits of AI to retailers and customers around the world.”