Future of Cooking: Chefing
Reimagine the future of cooking with machine learning-based mixed reality technology.
Project Overview
Cooking can be daunting for beginners with its many ingredients, intricate techniques, and perfect timing. I, along with several computer science students at UW Reality Lab, created Chefing, a mixed reality cooking app that takes the fear out of cooking and replaces it with interactive learning and fun. I led the design to create an intuitive virtual interface users can see and touch when wearing a MR headset. It not only displays step-by-step instructions, but it also recognize users’ kitchen space and equipments, projecting the instruction right at where it should happen. By using a cloud-based real-time object detection, users are able to “show” the headset what ingredients they have, and related recipes will be populated on the interface. Chefing transforms the kitchen into a space of inspiration, creativity, and culinary delight.
The Solution
Research Highlight
Process Highlight
Started as a research capstone, we explored using emerging XR technology to enhance learning by creating a 3D interface, step-to-step video instruction, and a cooking simulator to gamify complex techniques. We moved from prototyping to a fleshed-out 3D GUI in three months. The app was well-received, but we needed to scale it cost-effectively. After being accepted into the incubator at UW Reality Lab, we explored how to quickly scale the breadth of recipes and used a cloud-based object detection pipeline to recognize kitchen equipment to provide spatial guidance to users. We conducted remote user testing by sending participants a headset and groceries to use Chefing in their kitchen.
Outcome
Learning
Mixed reality is powerful, but also full of constraints
Working with mixed reality is fun! Spatial computing blends virtual objects naturally into the real world, seeing your living environment through the lenses helps eliminate motion sickness, and it has unlimited potentials in training and professional practices. However, the limited field of view, the bulkiness of the headset, and even the tint of the lenses can easily take away the impressiveness of the experience. I learned to work with limited FOV on Magic Leap by relentlessly prototyping the interface until the size is appropriate.
