Air Piano
Try the playable prototype
Allow camera access to play with your index finger, or use the mouse and touch controls without a camera.
The idea
What if a piano did not need a physical keyboard?
Air Piano is a browser-based experiment that turns hand movement into music. The camera tracks a fingertip, a virtual keyboard appears over the live video, and touching a key area plays the matching note.
The goal is not to imitate a concert piano. It is to make the first moment of creating music feel immediate, playful, and accessible: open the page, raise one hand, and make a sound.
The first playable version
The earliest version will stay deliberately small:
- use the device camera for real-time hand tracking
- track one index finger rather than both hands
- show eight white keys: C–D–E–F–G–A–B–C
- trigger a note when the finger enters a key area with a downward motion
- briefly light up the key that has been played
- display the notes for Twinkle, Twinkle, Little Star
- show the current target note so a beginner can follow along
No extra hardware is required—only a browser, a camera, and a little curiosity.
How the interaction works
The screen combines three layers:
- Camera view — the player can see their hand and position it naturally.
- Virtual keyboard — keys are aligned with the movement area on screen.
- Sound and visual feedback — a successful touch produces a note and an illuminated key.
Instead of trying to detect physical pressure, the first prototype uses the fingertip’s position and downward movement as the signal to play. This keeps the interaction stable enough to test before adding more complex musical expression.
What may come later
Once the core interaction feels reliable, the experiment could grow to include:
- black keys and a wider keyboard
- two-hand tracking
- note volume based on movement speed
- recording and playback
- more beginner melodies
- optional rhythm guidance and practice feedback
These are later possibilities, not requirements for the first version.
Why it belongs in my Lab
Air Piano brings together computer vision, spatial interaction, sound, and beginner-friendly product design. It is also a small test of a larger question:
Can an ordinary camera turn the space in front of a screen into a playful musical interface?
The first working prototype is now live. The next step is improving hand-tracking stability through real-world testing.