acoustic / WFS / description

Wave Field Synthesis and Holophony

Summary

The animation shows the propagation of the (black drawn) direct acoustic wave and its first reflections in the(large outer) recording room. They mainly determine our spatial perception of the sound event. All conventional loudspeaker reproduction procedures cannot reproduce this complex spatial and temporal structure, the reduction onto a few transmission channels inevitably cause a significant loss of spatial information. This page describe a different procedure. Just like the recording room creates all reflections from the pure audio signal of the source, a (magenta painted) loudspeaker screen in front of the (yellow) listener synthesizes the wave fronts from the pure audio signal and the recording room properties. In difference to the common known Wave Field Synthesis approach the described “Holophony” procedure does not suppress the (inner small) playback room acoustics, but includes it purposefully into the synthesis. Depending from geometry and reflective behavior of the recording and the playback room, the wave fronts radiate directed in such way, that they arrive in the same temporal and spatial progressive rate at the listeners ears like they would do that at the ears of a virtual listener in the recording room. Their dedicated virtual position in those recording room even can be changed during the playback process. That accordingly changes all positions of the virtual sound sources in the model based approach. So the listener can acoustically walk in the near field of the WFS- Loudspeaker screen across the recording room by means of his remote.

  1. The phantom sound sources of the conventionally procedures

The constitution of phantom sound sources is the main problem of the conventionally audio reproduction procedures. Its perception rest upon psychoacoustic principles, the sound source ever will disappear, if we try to bear up one of our ears on it.

The phantom source isn't really a sound source, a phantom source are two sound sources. Thus, the sound from the right speaker ever will also hit our left ear and contrariwise. That exalts the Interaural Cross Correlation Coefficient IACC, the most important factor for our spatial perception. We perceive a sound event as spatial if the signal difference between the ears as much as possible.

Besides, we cannot radiate all wave fronts of a sound event simply from the direction at accidently resides a Loudspeaker. The Head Related Transfer Function mercilessly discovers the fraud, if the angles of inclination of the wave fronts differ considerably. For true to original rendition we need sound sources which we can locate, at least virtually, firmly at all of the starting points of the genuine wave fronts and its reflections I the recording room. That's impossible by the conventionally procedures.

Over and above playback by discrete loudspeakers in any case becomes impress the acoustic behaviour of the Playback room. In normal living rooms overrates the reflected sound energy the direct radiated portion less than 50 inches away from the loudspeakers.

  2. Virtual sound sources

In centre of a loudspeaker sphere would appear a virtual sound source, as visible in that little animation:

As easily imaginable, the sound source in any case would perceived in centre of the loudspeaker sphere, independent from the listeners position. In this respect such sound sources fundamentally differ regarding the phantom source perception. If would be able constituting such firmly locatable sound sources on all starting point positions of the direct wave fronts of the sound sources and all of its reflections in the recording room, the spatial sound field would be indistinguishable from the genuine.

3. Wave Field Synthesis

Around 20 years ago since on the Delft University of Technology was developed a procedure for constitute such virtual sound sources by a line- array of discrete steerable loudspeakers [1]. Those “Wave Field Synthesis” called principle relies on Huygens Principle, what discover the cognition, each point of a wave front may regard as starting point of an elementary wave. That becomes descriptive, if we see holes in a baffle as such starting points. As long as the dimensions of the holes smaller as the wavelength the sound pressure cannot differ on both sides of the hole. The superposition of sufficient amount of such elementary waves delivers the genuine wave front.

More as 300 years ago the Dutch Mathematician Christiaan Huygens explains the diffraction effect by that principle. If covered some of the holes in the baffle the sound pressure may calculated on each point behind by superposition of the remaining elementary waves. Huygens Principle is one of the most important cognition in range of physics; also the diffraction effects for light and radio waves are similar.

In range of acoustics today the knowledge delivers the possibility, restore genuine like sound waves from such elementary waves. That shell becomes shown in the following animation:

As easily visible, we can restore the primary wave front completely from delay times and levels, which derived from the distance of each single loudspeaker regarding the virtual starting point of the wave front. In range of the loudspeaker alignments the virtual sound source doesn't differ from the genuine wave front in its properties.

Unfortunately, the sound field in the recording only particularly established from the direct wave front. The major fraction of the sound energy contain in the reflections. The directions and levels of the first reflections deliver important cues regarding source distance and distances regarding the walls, what imply the room dimensions. The huge amount of later reflections which arrange the reverberation tail provide information's regarding fine structure and properties from recording room surfaces.

The advantage of the loudspeaker row regarding the shown loudspeaker sphere is the possibility for establishing more as one single virtual sound source. Its signal content may originate by different sources, but the same signal content from different source positions can fake reflections of the main source signal. The genuine sound field in the recording room is established from huge amount of such starting points with same signal content. If we reconstruct all of those starting points, the spatial sound field of the source would be recreated from its simple mono signal. After all, the recording room does the same from the real sound source. The only difficulty for restore the genuine sound field is appointing all of the starting points of all of the reflections in the recording room.

In principle there are two different ways in that matter. The more simply method is the model based approach. According the mirror source model become calculated the starting points simply from the room geometry. According the distance of each virtual sound source position regarding respective Loudspeaker is calculable the runtime and level. The reflection factors of the walls are including in that calculation. For Playback the dry recorded audio signal of the assigned sound source becomes accordingly delayed. Such procedure is practicable for restore direct wave and first reflections in the recording room, but the huge amount of discrete reflections in the reverberation tail makes impossible the correct reconstruction of the complete sound field in practise by the model based approach.

By that reason is common practise in the scientific institutes, which refining Berkhouts idea, the application of the impulse response based approach. In prearrangement of the transmitting process becomes recorded the spatial impulse response of the recording room. In that purpose a line array of microphones arranged in the recording room comparably as the loudspeakers arranged in the playback room. That horizontal speaker rows mostly align completely around the listener for produce virtual sound sources at arbitrary positions inside the horizontal plane of the listener in the playback room.

For recording the spatial impulse response a short impulse create on the later position of the primary sound source, per example the soloist's position. That signal will hit the nearest microphone at first; the time differences to the neighbouring mikes determine the wave's direction, if the signal becomes radiate by the same time differences from the related loudspeakers during playback process.

The source in recording process of the spatial impulse response isn't the sound source itself. All starting points of reflections in the recording room are additionally impulse sources. By that reason the recorded impulse responses include the information regarding source position and all mirror source positions in the recording room, say the complete room acoustic information regarding the dedicated source position include in the spatial impulse response. The reflection factors of the walls reflect in the record levels of the impulse response.

Changes the position of the primary source in the recording room, all recorded impulse responses are changed.

By the playback side becomes convolve the dry recorded mono signals of the primary sound sources in the impulse response from the loudspeaker related microphone. But that impulse response is valid only for one single position of the primary sound source. In addition the impulse responses differ slightly from mike to mike, each speaker needs a separate convolution. Huge amount of tasks, but fortunately such convolution is an easy task for a digital signal processor; the signal simply becomes readout from shift register according the impulse response level for the assigned time.

Because the spatial impulse response cannot recorded for all possible positions of the primary source in the recording room and all possible loudspeaker positions in the playback room, the measuring results must become extrapolated during playback for all different positions. The calculation must include the mirror source positions. For the complete impulse response that task is hardly solvable for the current available computing power. Especially for moved primary sound sources the impulse response based approach cannot work in real-time until today.

As advantage of both solutions the virtual sound source cannot align alone behind the loudspeakers. In the animation wouldn't appear any difference for delay times and levels, if the virtual source behind or in front of the microphone row. Thus we would perceive the starting point in any case behind the speakers. But becomes the delay times inverted by the “Time Mirror Approach”, the loudspeakers would produce concave wave fronts. The virtual source appears in the focus point inside the playback area. In certain degree we can walk around yet.

In principle, the wave field synthesis today has the ability for produce a virtual copy of the genuine sound field. All sound sources and all of its reflections in the recording room may produce virtually at any points inside the horizontal plane of the listener. That's different regarding all known procedures; the spatial distribution of the reflections in the recording room is too complex for transmit in some discrete channels. The synthesis from the sound source signal, in the same manner as the recording room establish the spatial sound field, seems the only possible way for a loudspeaker volume solution. Only in such volume solution we doesn't captured in narrow sweet spot, the wave field synthesis restores the field, changes of the listener position in the playback room cause the same changes in perception as listener movements in the recording room. That's a huge step in the direction to true spatial audio.

Though, until today the practically realised installations remain clearly apart from the goal of the indistinguishable reproduction. Most notably the reduction onto the horizontal plane is clearly different regarding the genuine sound field. But the remaining problems seem solvable in foreseeable time. An idea for restore the sound field in all three room dimensions as “Holophony” – Approach is described in the next chapter.

4. The copy of spatial sound field

Since stereophony was introduced we are trying to improve the spatial reproduction of the genuine sound event by increasingly amount of transmitting channels. Yet the sound source itself doesn't performance any spatial sound field; each arbitrary source may regard, at least from certain distance, as a Mono source. P ossibly that source isn't radiating equivalent in all directions, but in no way a spatial sound field would be to transfer. The spatial sound field arises from the reflections in the recording room alone.

That complex reflection pattern we can restore in all three room dimensions if we start from such thought:

Would perforate an enclosed cabinet around the best place in the concert hall with holes, the concert experience would be perfect. Now could each of those holes blocked with a loudspeaker, as described above, steered by a microphone on the other side of the wall, acoustically nothing would be changed. Built into the living room walls at home these speakers would provide a perfect copy of original of sound event!

The problem would arise, too many channels are hardly transferable, and every speaker has finally a slightly different signal. Examine more closely, the waveform is the same in all holes, only the arrival time differs. Therefore all speaker signals deducible from the Mono signal of the sound source, if you know only its position in the recording room. The reflections singing no other song as the tenor, they can synthesize from the signal of source too, when their starting points are known. As described, the wave field synthesis principle would provide a way for restore the loudspeaker signals.

However, the spouse acceptance factor would avoid the marked penetration for a principle, what wants to populate the living room wall all around by loudspeakers. Over and above the limits in computer power would make impossible the convolution into the impulse response for all three room dimensions until today.

The following chapter describe a possible solution for both problems, patented by the author in 2005. The speakers all around become substitute by the reflections of the playback room, only a speaker screen hidden behind the projection wall remains. The computing power problems are solvable by combination of the model based and impulse response based approach.

5. WFS- Holophony approach

Holophony means the acoustic wave fronts arrive at the listener at home (the small inner room in the animation) in the same temporal and spatial manner as at a dedicated listener point in the recording room:

Transformation large field into small playback room

That goal would be achievable by the known Wave Field Synthesis approach, as far as all walls around the listener completely populated by loudspeakers. Besides the effort the acceptance factor would avoid the success of such attempt for the home marked. Moreover, the currently available computing power isn't sufficient for such approach. In the following chapter describe a different idea, the loudspeakers all around becomes substitute by the reflections of the playback room. The remaining frontal loudspeaker field may hide behind the picture screen. The problem of the insufficient computing power is solvable by a combination of the model based and the impulse response based approach.

The problems of the horizontal WFS- loudspeaker rows

Wave field synthesis today is commonly achieved by a horizontal loudspeaker row around the listener. Such solutions are possible today with some hundreds of loudspeakers without unbearable problems. But a broad based breakthrough seems not to be within reach. Besides the acceptance factor, some additional problems haven't been solved yet. The most audible effect is the horizontal limitation. Other procedures, like Ambisonics or Vector Base Amplitude Panning (VBAP), have shown that a real three dimensional reproduction of the sound event has been essential. They are also missing an advantage that WFS offers - a large listening area inside the “volume solution”. The Ambisonic and VBAP techniques can only be experienced in a more limited subset of the playback rooms compared to WFS.

Though additional problems appear using all other solutions until now, the playback room is the biggest single problem. It imposes its own acoustic behaviour for reproduction. Loudspeaker rows around the listener cannot solve this problem. In order to produce the acoustics of another room, the playback room acoustics must become completely suppressed. That's hardly possible in a normal living room. Such a system can be delivered in experimental setups, but without broad market acceptance. Horizontal rows of speakers don't really produce parallel wave fronts which can be steered in all levels - they radiate cylindrical waves, which lose 3 dB of volume every time the distance is doubled. Since the listener is relatively near the speakers, the volume change becomes disturbing. But, what is worst, the sound energy doesn't immediately dissipate. The ceilings, floors, and walls of the playback room impose a new acoustic behaviour on the sound that was originally recorded in another room with different properties.

Near field solution

The only possible solution for avoiding the unwanted impact of the living room acoustics besides wearing headphones is near field reproduction. Two possibilities exist for that, either the speakers are placed near the listener or the radiating speaker surface must be very large. In normal living room environments, the distance separating speakers and listener is around three to four meters. We would need a diaphragm of an adequate diameter to create near field reproduction. This is hardly possible for a single speaker, but by using the wave field synthesis principle, all loudspeakers work together as a single unit. If we use a sufficient number of speakers aligned in a plane in front of the listener (for example, behind a picture screen) the different WFS signals from each speaker could reproduce a simulated curved wall with near field conditions:

If two inch distance between the single speakers is used, such a near field would has a size of 2.43m (~8') x 1.38 m (~4.5'); therefore, the near field condition in a normal home could be reached. The spatial aliasing effects would occur, dependent from the ingoing and outgoing angles regarding the radiating surface, above this frequency:

As an example, a 30 degree angle difference would result in a 13.5 kHz. That's an acceptable value; it is known that our perception isn't very keen regarding spatial aliasing in this frequency range. Technically, such a field would be able to be produced today; some alignments already in existence have placed a comparable numbers of speakers around the listener in a row. Actually, the development goes in the direction of the “speaking silver screen”. Because of the directive effect of such frontal loudspeaker field, the playback room's inherent mirror sources cannot supply sound energy because there is no energy input This generally eliminates its disturbing influence. We will show a way to include playback room mirror source reflections on purpose in the next chapter. This is a key difference with practically all other wave field synthesis approaches to date.

Including the playback room properties for the Holophony approach

The conventional Wave Field Synthesis approach doesn't have a dedicated listener position. Without this, we cannot compare between the sound impression of the playback room and the recording room. As mentioned above, the main goal of the described Holophony approach doesn't aim to fake the acoustic environment in the recording room, the main goal is to restore coincident signals on the ears of the listener in home cinema as would be heard by a virtual listener in the recording room. That would create the same perception. By using this new system design approach, we don't want to depress the playback room acoustics. The reflections of the playback room become an additional element in the transmission chain. We want to substitute loudspeakers completely surrounding the listener, which would be needed for a conventional Holophony system. The way this will be accomplished is by means of using the sound reflections from the playback room walls, ceiling, and floor. In order to re-create the spatial impression, we can more easily use the model based approach. We can simplify the source positions to an easy to calculate model of the most important first reflections. Using this model provides the possibility of calculating the runtimes and levels for each speaker very easily, because only the distance regarding the appointed speaker determines the delay time for the corresponding audio signal. Moreover we can modify such a model for re-assembling the recording and playback room acoustics into a common approach, as is shown by the following small example sketch of the ceiling reflection of the recording room. The outer room 3a is the recording room, and inside is the playback room 3b. In the center is the common listener position in recording and playback room 3i. The source 3c is restored by Wave Field Synthesis in the Speaker field nearly perfectly by the “acoustic curtain” of the frontal speakers. But the problem appears that we must produce the ceiling mirror source in the recording room far outside of the speaker field. In order to simulate the first reflection source using the playback room reflection, we must shift the recording room's mirror position onto the ceiling mirror source of the playback room speaker field. This is accomplished by a simple rotation in a circular path inside the common coordinate system. Mirroring the position and taking advantage of the playback room's ceiling geometry delivers the final starting point (4b) of the ceiling reflection virtual source, whose wave fronts now arrive at the correct time and adapted in direction to the listener.

This procedure simulates in the playback room the altitude of the recording room. By using the same principle and procedure for all reflecting walls is possible accurately simulate the room dimensions that were originally perceived in the recording room.

Combination of model based and impulse response based approach

By means of the currently available computing power the attempt would be hopeless, restoring correctly all initial points of all wave fronts, which contain the whole spatial sound field, in all room dimensions. But there is no need for that. The reverberation tail contains important information's regarding the recording room properties. All detail structure of the surfaces, which determine the timbre of the room, are covered in the reverberation tail, though the direction of incidence of its wave fronts hardly does matter for our perception. As well in the recording room the reverberation arrives from all directions around us, we cannot align the second or later reflections in the recording room any starting point.

On the other hand, the direct wave front and its first reflections in the recording room hardly determine the timbre, but contain all localisation cues for the source position. By means of optically assignments, we have got the skill for determine the room position of those acoustic sources very accurately in all room dimensions by humble distinctions in arrival time or amplitude of those wave fronts. Known starting points of the first reflections provide the main information regarding room dimensions and source distance.For this reason doesn't meaningful handle the direct wave and its first reflections in the same manner during the wave field synthesis, as common practise today. The commonly implemented completely impulse response based approach delivers perfect results, but overcharge the currently available computing power for moved sources in three dimensionally environments. The perception of the reverberation tail hardly changes, if the source or the listener changes its position in the recording room. Only volume and direction changes of the direct wave and its first reflections alter the perception substantially. The according calculations feasible much more easily in the model based approach. The following sketch shows a way for combine the model based approach for the first reflections and impulse based method for the reverberation tail:

Combination of Imp/ Model based WFS; Please klick the picture for enlarge

The model based part on the left side of the sketch is connected via MADI or LAN by the Engine part on the right side. Such detachment of computing and engine was proposed by Wolcott in [3]. The interface transmits all audio source signals and all data regarding delay time and Level for each single loudspeaker position. But for those high amount of data values – each speaker needs for each primary sours and for each of its first reflections in the recording room a separate delay and level value – according [3] are eight updates per second sufficient for an smooth movement of the source. The next screenshot shows those calculation results for one single loudspeaker in a 32 channel system:

according delay and level calculations for two of the speakers, click the picture for change value animation

The number of loudspeakers is user defined extendable in those calculation. The both grew fields in the under picture range are the values for the neighbouring speaker, mounted X = 240mm left from the speaker field centre in the common (Y) plane, Z = 200 mm above the centre. It would be also speaker positions outside the common plane possible, per example additionally rear speakers with according negative Y value. All calculations based on the common system from recording and playback room. That's visible if we compare the delay times for the ceiling mirror source. Its signal arrive on the more above mounted speaker some frames later, what connected with a source position below those speakers.

Of course, the exact coordinates of the mirror source positions needn't manually calculation; the described left part of the upper principle sketch carry the task according the described principles from the room geometry's and source positions and the dedicated listener position in the recording room. In the calculation screenshot example the position of the soloist 1 is dedicated very near (6 inch apart) in front of the virtual listener position. That's causing a very high direct wave level in the calculation, during all reflection levels remain on average levels. That fact wasn't changed by the implementation of the virtual source shifting principles according the DE 102006054961A1 application. By that way is solved the problem of the faulty ITD cues in the range between virtual source and supplying speaker. The solution provides the surpassing direct sound level for that source in the whole listening area.

In the near field of the loudspeaker screen the very dry reproduction doesn't become superpose by playback room caused reflections. The perception of its starting point direction is very firm because no psychoacoustic phantom source detection is needed. And over and above, the sound field now really have three dimensions. The combination with the impulse response drawn method for the reverberation tail is very practicable in the described procedure. In the example the channels 01 and 02 are reserved for constitute the reverberation. The signal can be create by convolution of the summary signal from all input sources into the impulse response of the recording room. For that solution the program deliver a time value, during the first frames inside those impulse response must become suppressed in order to avoid a double production of first reflections.

Another way would be simply recording the reverberation in a discrete channel with an omni- directionally microphone apart all direct sound sources. During the synthesis the reverberation level is only depended from overall volume and he comes from all directions, what's matching its spatial distribution in the recording room. Huge advantage of the described proceeding is the replacement of the spatial impulse response, recorded by a team of highly qualified technicians in preparation of a wave field synthesis recording. Besides, those results no longer must extrapolate according each loudspeaker position during playback, what the main reason for the computing power problems during synthesizing moved sources. For the described procedure normally recorded impulse responses, as available for all possible environments today, utterly sufficient. Together with a crude geometric model of the recording room each dry recorded signal can become rendering as would it be recorded in the most attractive environments around the globe.

Compatibility

The model based approach convenient for reproducing conventionally records. The MODE- switch selects an appropriated configuration. The steerable wave fronts deliver some important advantages in comparison to solitary speakers:

last update 2009-07-04