WADDUMEAN?
Real-time gesture recognition that translates Neapolitan hand gestures for foreigners.
- Year
- 2024
- Role
- ML & Team Lead
- Stack
- CoreMLVisionSwiftUIAVFoundation
Concepts I've been exposed to
01
Machine Learning
02
Multi-cultural experience
03
Advanced Swift frameworks
What is WADDUMEAN??
Waddumean is an app for foreigners that come to Italy and want to fully dive into the diversity of the Italian and more specifically Neapolitan world. The app recognizes in real time the gestures done by people (while speaking) and tells the user the meaning of that particular gesture. For example: a person has its hand closed with all the fingers touching the thumb and moves it up and down. The app recognizes the gesture through the camera of the iPhone and translates the gesture with the sentence "What do you mean?" or "What do you want?".
What did I learn from this experience?
I managed the workflow and priorities of the team and dealt with programming. In particular:
- trained an ML Model to recognize the gestures with CoreML
- learned how to access to the camera feed of the iPhone
- divided the work of the team in small achievable objectives week after week
- motivated the team
- produced a mini short film to advertise the app
Attitude in the project
I measured myself with my project managing skills and learned to handle critical situations and to communicate in an effective way. Handled new members that arrived when the team was already formed. Coding-side I had some difficulties in passing the data from the camera feed to the AI model. Solved this finding a workaround and talking to mentors asking for help.