Artificial Intelligence
Google AI experiment lets you conduct an orchestra by waving at your webcam

Google AI experiment lets you conduct an orchestra by waving at your webcam

Artificial intelligence identifies your limbs, then adjusts instruments, tempo and volume as you move

Like GearBrain on Facebook

Google' latest artificial intelligence experiment uses your webcam and a browser window to let you conduct a virtual orchestra.

Part of the 'Experiments With Google' series, the system uses AI to identify your head and limbs then turns your movements into instructions for adjusting the music. You can wave your arms around to pick sections of instruments to play, then adjust the volume and tempo.

Read More:

Buy Google Home Hub on Best Buyicon

Move your hands, and the virtual orchestra plays through the music using hundreds of tiny audio files gathered from live instrument recordings.

GearBrain had a play with the system before writing this article and, well, we're not classically-trained conductors, so the results were mixed to say the least.


While the music we created wasn't particularly good, we found the AI worked accurately — raising our arms increased the volume, pointing to the left or right changed which section of instruments was playing, and being more animated upped the tempo.

Google says: "It uses PoseNet, a machine learning library that works in the browser, to map out your movements through your webcam. An algorithm plays along to the score as you conduct, using hundreds of tiny audio files from live recorded instruments."

Buy Google Home Hub on Best Buy

Called semi-conductor (we see what Google did there), the system works entirely within your web browser, so you needn't worry about your embarrassing arm-flailing being uploaded to Google. That said, you'll need to give both the website and your web browser permission to use your webcam before the system can be launched.

You also need to stand a good distance away from the camera, so the AI can work out where your arms are in relation to your head and torso.

There's only one piece of music available for now, the first movement of Mozart's Serenade Number 13 'Eine kleine Nachtmusik,' which should be recognizable to most readers after successfully stumbling your way through the first couple of bars.

You can head to

this Google website to try the conductor AI for yourself.Check out The GearBrain, our smart home compatibility checker to see the other compatible products that work with Google Home and Home Mini

Buy Google Home Hub on Best Buy

deleteUnboxing in 10 Seconds - Lenovo Smart Display with Google

Like GearBrain on Facebook
The Conversation (0)

GearBrain Compatibility Find Engine

A pioneering recommendation platform where you can research, discover, buy, and learn how to connect and optimize smart devices.

Join our community! Ask and answer questions about smart devices and save yours in My Gear.

Top Stories

Weekly Deals