Transferable Skills Developed

😁 Collaboration with Other Hackers

🦜 Public Speaking Abilities

📅 Time Management Skills

📝 Critical Thinking and Problem Solving

Technical Skills Developed

👨‍💻 Python Coding Experience

💿 Experience with OpenCV and Twilio Libraries

🐱 Experience with Git and GitHub

💻 Integration with Hardware Components

Full Project Demonstration!

https://www.youtube.com/watch?v=k1GlQNUzvNI

Overview

Our team for DeltaHacks 2024

Blink Twice if You Need Help is a wearable device which facilitates blink detection. Upon blinking twice in a specific time frame, emergency services (which was simulated by my phone) can be called, with information about the user’s location being given. This project was split into four parts:

• Blink detection, which was completed by Gavin
• Calling emergency services and transmitting user location, which was completed by myself.
• Integrating both parts, which was completed by Gavin
• Assembling the hardware, which was completed by both of us

This was my first ever hackathon, and it was a pleasure to complete it with one of my good friends, Gavin Tranquilino!

Project Details

The project began with Gavin using an open source face detection model created in OpenCV to determine if a user had their eyes open or if they were blinking. If the user blinked twice within a hardcoded amount of time, my phone would be called to simulate contacting an emergency number.

A demo of the face tracking model!

Gavin wearing our model and testing blinking

I then used the Twilio service and libraries to send a phone call from my Python script to my phone, which would play a predetermined message. I edited this message to send the users IP address, city, country, and coordinates of longitude and latitude to the person on the other line. Gavin was eventually able to integrate these two functions, where my code was implemented as a function within his.

Implementing this did not come without difficulty, as I had to ensure the coordinates of latitude and longitude, as well as the user’s IP address, were communicated slowly enough for an emergency operator to make note of. I also had to ensure the coordinates were read properly, which involved type checking, string stripping, and string concatenation. Sensitive phone numbers and authorization keys also needed to be kept hidden, which Gavin helped me with.

We then progressed to creating our hardware design, which involved connecting cardboard to my baseball hat while supporting the underside with bubble tea straws and barbecue skewers. We attached a webcam to the end of the cardboard, and adjusted its position so it was in line with Gavin’s face, and close enough for the model to detect it. We also attached a ring light to the design, as we found that increased lighting conditions led to better results.

Gavin and I programming our project!

Our wearable hat design!

Our presentations to the DeltaHacks judges were a huge success! We successfully garnered a large amount of interest in our idea due to its unique, head-mounted design, and the future implications of this technology. This included attracting the interest of one employee from the Communications Security Establishment, as well as various DeltaHacks executives and McMaster TAs.

While we ultimately were not selected as one of the ten finalists, Gavin and I were extremely proud of our design, and were intrigued by the potential of this technology in the future!

Limitations

While the technology of blink and object tracking has extremely high potential, we observed various limitations while programming.

• The model detects faces based on grayscale, which made it difficult for the model to detect my bright blue eyes in comparison to Gavin’s dark brown eyes.
• The model heavily relied on eyebrows to determine results, causing smiling to often be registered as blinking.
• The time between blinks was hardcoded, which didn’t account for different blinking speeds. This could be fixed using Machine Learning, allowing the AI model to develop an understanding of the user’s blinking patterns.
• The camera needed to be far away from the user’s face for proper detection.
• Results varied from user to user due to the faces the model was originally trained with, as Gavin tended to have more success detecting blinks than I did.

A trip to McMaster’s iconic Edwards Arch!

Gavin and I attending a GitHub Copilot Workshop!

• Materials to create the hat were limited to what I already had in my residence, as well as anything provided by DeltaHacks.
• Using TwiML and ngrok to facilitate voice communication between the call recipient and a hypothetical AI model was unsuccessful due to missing permissions on McMaster’s Internet network.
• IP addresses can only return approximate coordinates, and are generally inaccurate. To offset this, the full IP address was also provided.
• While GPS location was tested, it was ultimately unsuccessful and produced highly inaccurate locations.
• The Twilio account used was under a free trial, which caused the following restrictions:
  • A $15 budget to make phone calls, with each call charging a small amount of credit
  • Only emergency numbers and a list of verified numbers, which was only able to contain my phone number, could be contacted.
  • A trial message played at the beginning of each call.
• The 2024 McMaster Sumobots Competition, which I also participated in, was the same day. With only two team members, this drastically reduced the scope of our project to a manageable idea in a smaller time frame.

Based on these limitations, as well as our desire to make the most of our experience at DeltaHacks by networking and attending other workshops, we chose to structure our idea as a proof of concept rather than a fully-fledged application