DANCE VOICE – Musical Instrument for Generating Tones Based on the Scanned Position of Bodies in Space

Technology Overview

In the Automotive, Engineering, Plastics and Food industries, camera systems are currently used primarily for control and measurement purposes; like code and character readings, print control, fault-finding, colour identification, position and product recognition, as well as counting and completeness checking. It is used especially where 100% quality is required. It is one of the most important ways to achieve high-quality products. For example, in the Entertainment industry, there is a known solution where the camera vision of a camera system is used to express the modern way of digital drawing by means of body movements. In terms of generating sounds, for example, a method using tapes located on the floor is used. Their subsequent weighting by means of the human body - (a dancer), generates a tone. The method of creating tones based on sensing the position of bodies in space, based on the principle of the machine-vision of a camera system is, however, not yet known.

Technology Features & Specifications

The nature of the Tone Generation method - based on the scanned position of bodies in space according to the invention, resides in the fact that the objects of interest are scanned in space and the height of the generated tones, their colour and loudness, are assigned according to the spatial coordinates of the objects. The coordinates of the object are determined by the machine-vision principle of a camera system when at least one camera senses a scene in which there are objects of interest that are different from the surrounding environment, and other objects with their characteristic colour, shape and/or dimensions. Every single snapshot is pre-processed by segmenting the image based on the occurrence of a given colour, which is given by the user's setting of the acceptable range of individual colour coordinates. On the resulting binary image, a Gaussian filter and morphological operations are applied to suppress noise - and to find contours that clearly bound the objects of interest. Each object of interest is determined by its moment characteristics, which determine the area of the object in the image, and two spatial coordinates

of the centre of gravity. The third spatial coordinate of the centre of gravity of an object is determined, in the case of one-camera capture, after initial calibration according to the size of the area of the interest object in the image. In the case of capture by at least two cameras, the coordinate is determined by the superposition of the images from different cameras, taken at the same time.

The height of the tone is determined by the assigned spatial coordinate such that the image is, in the vertical direction, imaginatively divided into a set of, each corresponding to a certain tone height, and particularly - 1 to 88 sections of the corresponding piano keyboard.

The tone colour is determined by the additional assigned spatial coordinate so that the image is divided horizontally into 1 to 128 sections that agree with the General MIDI standard. Each section corresponds to the tone colour of a particular instrument from a group consisting of pianos, tuned drums, an organ, guitars, basses, strings, the sounds of the ensemble, a jigsaw, slips, whistles, electronic solo sounds, electronic accompanying sounds, electronic sound effects, ethnic, percussion and other sound effects.

The loudness of the tone is determined by the additional assigned spatial coordinate such that the closer the camera's object, the louder the sound is.

The device for carrying out the method - to the invention, includes a camera system consisting of one or more cameras for sensing the objects of interest in the scene; the output of the camera system is connected to the input of the tone generator, where its output is then connected - in the final stage, to a loudspeaker.

Thus, this Optoelectronic device - according to the invention, generates sounds or tones based on the spatial coordinates of bodies, objects or marks, or interest objects - that are then assigned pitch, tone and tone colour, or respectively, the type of musical instrument. The beginning and the end of the tone is controlled by uncovering and covering the interest object. These interest objects can be characterised by their colour, shape or dimensions. Based on the selected differences, a search for these interest objects is initiated, then their coordinates in space are determined, and the corresponding sound is generated.

In the case of the detection of multiple interest objects in an image, several tones differing in height, loudness, colour, and instrument type are periodically played back. Assignment of a given height, loudness, and tone colour to spatial coordinates of the interest objects can be altered to meet the user's needs or requirements.

Potential Applications

Entertainment industry (chorus line, theatre, dramatic art, school)

Market Trends and Opportunities

This type of musical instrument is not on the market.

Customer Benefits

Energy: This is an optoelectronic musical instrument, which leads to energy savings in the production and processing of metal parts of musical instruments.

Environmental: This is an optoelectronic musical instrument, which leads to the investigation of rare woods necessary for the production of classical musical instruments.

Economic: Increase in attendance at dance and theater establishments.

Application in mobile technology.

Other: Diversification of artistic and cultural life, especially of the young generation.

Make an Enquiry