Back to Projects

Red Handed: A Slap To Keep You Awake

Machine vision model on a Raspberry Pi, controlling a slapper arm. Winner of Flow and 2nd for Qualcomm at MakeUofT 2024

Date: 2024-02-26

Tags:
MLHardware

Inspiration

Decentralizing productivity with flow is the key to your success! Have you ever found yourself in the paradoxical situation where you're too tired to continue working but also too stressed about deadlines to sleep? We've discovered the solution you never knew you needed (and probably still don't): Red Handed. Inspired by the countless souls who've fallen victim to their own procrastination and the relentless pursuit of productivity, we decided to tackle the problem head-on, with a slap. After all, why rely on caffeine or traditional means of staying awake when you can have a machine physically prompt you to stay alert? In the grand tradition of inventing solutions to problems you didn't know existed, Red Handed stands proud, ready to slap the sleepiness right out of you. It's the wake-up call you never asked for but might just appreciate in your most desperate, caffeine-deprived moments.

What it does

Red Handed uses a standard webcam to monitor your facial expressions in real-time. With the power of MediaPipe, an open-source machine learning framework for building multimodal (audio, video, etc.) applied machine learning pipelines, it detects signs of sleepiness or inattentiveness, such as yawning or eye closure.

Here's the step-by-step process:

Facial Detection: The webcam captures live video input, focusing on the user's face. Yawn and Eye Shape Analysis: Using MediaPipe, Red Handed analyzes the shape of the eyes and the mouth. We also process the position of the face for the optimal slap. A significant change in these shapes, such as a yawn or the eyes closing, triggers the next step.

Processing: The analysis is sent to a Qualcomm HDK via a Socket.io socket (Hardware Development Kit), where the decision logic resides. This powerful Android kit processes the input in milliseconds and sends the appropriate Bluetooth signal to the robotic slapper.

Action - The Slap: The real-time feedback loop ensures that the robotic slapper is in the perfect position to deliver the optimal slap. Upon detecting sleepiness, the Qualcomm HDK sends a Bluetooth signal to the robotic slapper, delivering a harsh, safe slap to the user. This act is designed to be more of a punishment than a nudge, a physical reminder to stay awake and focused.

Reward: If you stay awake for the entirety of the Pomodoro timer, you get free Flow tokens!

How we built it

The Pomodoro app on the Qualcomm HDK communicates to the webcam via a Socket.io socket to start analyzing. MediaPipe builds a facial skeleton on the user's face and sends relevant points back to the Qualcomm HDK via the socket for analysis. The Pomodoro app analyzes the data and sends the y-position of the user's face to the ESP module via Bluetooth which gives data to the Arduino slapper. The Pomodoro app also processes the user's face to detect sleepiness and sends the ESP module a "slap" signal to release the red hand.

Challenges we ran into

It was very difficult to build the data pipelines. The Bluetooth took forever to set up while the eduroam firewall would not allow ngrok to process certain requests.

What's next for Red-Handed

In our next move, Red Handed is leveling up from a cheeky wake-up call to the ultimate sidekick in the battle against the snooze. Picture this: Red Handed evolves into your personal hype machine, ready to transform every yawn into a "heck no, let's go!" moment. With its sights set on global domination, we're not just stopping at keeping you awake; we're on a mission to make grogginess fear its name. Imagine a world where Red Handed sets off a ripple of energy, not just in your home office but across continents, making the mid-afternoon slump a thing of the past. We're talking about a future where staying awake isn't just necessary; it's epic, and Red Handed is leading the charge, one exhilarating slap at a time.