Virtual Piano with Kinect and processing

Product video link 

this was an first hands-on experience with kinect V2 sensor and image processing. after having software in place. This was a first example i tried to put online.

Idea was to make any surface as a midi controller or trigger point (Input). So before we start in the details and coding, lets give you some idea regarding the kinect sensor. 

Kinect V2 sensor is an depth sensor made for xbox. They have launched a first version with Xbox 360 and later stage it got up-gradation for Xbox one (second version). This device is made in the combination of  RGB camera, depth sensor and Array of microphone integrated in single device. These combination enables the device to do full-body 3D motion capture, facial recognition and voice recognition capabilities.

Please check video for more details and sensor capabilities with Xbox.

This device can do more then what you seen above. It enable the developers to make new way of interaction with virtual/Physical world. Microsoft community provides an support to the developer to make new interaction with possible sensor capabilities. You can find capabilities of the new sensor here in this video covered by WIRED team.

It can track up to six people, including two active players for motion analysis with a supporting data of 20 joints per player. However, Manufacturer has stated that the number of people the device can track is only limited by how many will fit in the field-of-view of the camera.

There are many successful commercial made by people around the globe, i like personally a below one. It was made by V-energy team with full HD projection in the parking area.(Kudos to development team)

Before you jump in to the software and coding parts, You need to make sure your system is compatible with kinect V2 sensor requirements. It also required window adapter to convert kinect connector to the normal USB 3.0 connection. As new Kinect v2 sensor comes with different(never seen before) connector which i haven't seen before.

Quick start to kinect v2 sensor for the beginners,

kinect v2 sensor works with SDK 2.0, it does not work with older SDK 1.X. For the new comers, SDK is provided by the Microsoft team to the developer community to enable with basic kinect coding capabilities.

To start with you need,

1) Kinect v2 with external power source.

A kinect V2 sensor does not come with power adapter, source it separately.

2) Window 8/10 computer with kinect SDK v2.0. 

3) Dedicated computer with USB 3.0 and window 8 or 10, It does not work with USB 2.0.

4) Programming environment visual studio or Processing 3.0 (in this case)

To start with

Install processing 3.0+ version and also download SDK 2.0. You need to install processing library to make this work with processing environment. So it would enable processing with same SDK capabilities. You can download ThomasLengeling library, trust me it works fine.

download this library in to the processing library folder located in the "My document" folder.

check you computer with kinect first before you star with processing,

You can check your computer compatibility after installing SDK2.0

It should be green in all section.

In my case it shows error with USB controller, 

but works fine with processing, so you can ignore this error.

You can start with examples provided by the Microsoft,

Now you are good to go with processing.

More details from the Microsoft 

https://msdn.microsoft.com/en-us/library/dn188670.aspx