Oktoberfest’17: Computer Vision Automated the Unthinkable
In the summer of 2017 the MoonVision team received a request from Ammer Wiesn, a well-known German caterer. They were in search of a technology that could detect, count, and track an array of dishes at the oldest, largest, and longest party in the world: the Oktoberfest in Munich.
Over the course of 16 days, more than six million visitors from around the globe would enjoy over seven million litres of beer and enormous amounts of traditional Bavarian food. Tracking these orders was the ultimate stress test for a computer vision system.
Quality assurance plays a vital role in the food and beverage industry. The issue of food safety is an important one and we had to combine this priority with process-oriented quality management.
During the 16-day period, over 40 waiters worked long hours and demanding shifts to meet their customers’ and employer’s expectations. As a result, the food service process was structured to:
- Provide the best service for costumers
- Ensure the best quality of food and beverages
- Offer a good working environment for employees
- Guarantee that the cash register system accurately mirrored all transactions
The task of working long hours in an extremely loud and draining environment without making mistakes is a very difficult one. Waiters have to remain focused at all times to minimize errors when handling financial transactions. Additionally, the temptation to bypass the employer’s cash register system for personal financial benefit should be kept at a minimum. In order to reduce the chances of transactional errors, the caterer used “Küchenpass”, a quality assurance system that controls food quality and correct food and beverage orders. Should the system display incorrect values at the end of the day, the waiters would have to make corrections.
Controlling the food service process is tedious and energy-consuming: the head chef has to synchronize the customers’ orders with the information displayed on the receipts, as well as manage the entire kitchen. This is precisely the part of the food service process that was automated by the MoonVision platform.
MoonVision overcomes challenges of food service processes by:
- Providing an accurate status update at any time
- Enriching current processes with the supplied information
- Streamlining the workflows of employees
- Objectively analyzing a situation and turning real-life actions into numbers
- Maximizing the number of orders by avoiding waiting periods
- Supplying a reliable system that can be trusted to operate from day one
- Ensuring almost 100% accuracy
Solutions including the integration of RFID chips and more complex processes were not in line with the owners’ needs. Our computer vision technology followed a pragmatic approach: we filmed the exit of the kitchen, detected dishes and staff, and directly added the information to the cash system.
Making object tracking possible at this level of sophistication meant that MoonVision had to overcome a series of technological challenges.
None of the approximately 20 dishes were served before the official start of the Oktoberfest, meaning that a customized solution for tracking the output of dishes needed to be established as quickly as possible. After only one day, the MoonVision platform extracted relevant scenes and a representative sample of carried objects from the collected footage. With a collaborative labelling tool the owner and other users were able to quickly name relevant dishes in a way that both the cash system and users understood. Additionally, very rapid movements required a tracking solution that worked at high framerates, limited computing power, motion blur, and could identify partially hidden dishes. The final system required only 7% capacity of a modern GPU to deliver information in real-time.
The MoonVision platform tackled different challenges associated with food service processes by adapting the approaches mentioned above, all in an incredibly short setup time.
The platform adheres to three principles to enable the use of computer vision by everyone:
- Computer vision means comprehending reality in real-time and should employ processes understandable to humans.
- We work on learning and adopting something completely new, without making it more significant than it is.
- Collaborating and understanding critical aspects of learning leads to MoonVision naturally integrating processes.
“It is a crazy time I live in - on the first day I was able to set up something I only knew from science fiction, and you could see the system think. For me as a technological layperson it is impressive how quickly I can control such a powerful yet pragmatic system. It makes me humble and optimistic about what AI will be able to do in the future. I am happy about the vision of the guys and how respectful they treat the technology and its consequences.“
This project was the reason for being named “Startup of the Year 2017” by the Austrian Chamber of Commerce and diepresse.com in October 2017.