Improving overview part on latency and processing limitations

parent 731cd8ce
......@@ -726,7 +726,7 @@
<h4>CMake</h4>
<p>
VA relies on <a href="http://cmake.org" target="_blank">CMake</a> to generate project files of your choice that will let you build the project in your preferred development environment. At least in theory. So far, we have only build VA using diverse Microsoft <b>Visual Studio</b> versions and MS compiler versions on <b>Windows platforms</b>, and on <b>Linux platforms</b> using <b>Makefiles</b> and <b>GCC</b>.<br />
CMake is fine, but the <a href="https://sourceforge.net/projects/vistavrtoolkit">Vista VR Toolkit</a> practically improves it by its VistaCMakeCommon scripts. VA uses a lot of ViSTA functionality, and we have also adopted the build methods. It takes a little effort to set some paths to the environment, but it pays off especially of you want to add tests, benchmarks and your own applications that require the VA libraries (and its dependencies).
CMake is fine, but the <a href="http://www.itc.rwth-aachen.de/cms/IT-Center/Forschung-Projekte/Virtuelle-Realitaet/Infrastruktur/%7Efgmo/ViSTA-Virtual-Reality-Toolkit/" target="_blank">Vista VR Toolkit</a> practically improves it by its VistaCMakeCommon scripts. VA uses a lot of ViSTA functionality, and we have also adopted the build methods. It takes a little effort to set some paths to the environment, but it pays off especially of you want to add tests, benchmarks and your own applications that require the VA libraries (and its dependencies).
</p>
<h4>Where to start</h4>
......
......@@ -95,11 +95,11 @@
<p>VA creates audible sound from a purely virtual situation. To do so, it uses digital input data that is pre-recorded, measured, modelled or simulated. However, VA creates dynamic auditory worlds that can be interactively endevoured, because it accounts for modifications of the virtual situation. In the simplest case, this means that sound sources and listeners can move freely, and sound is changed accordingly. This real-time auralization approach can only be achieved, if certain parts of the audio processing are updated continuously and fast. We call this audio rendering, and the output is an audio stream that represents the virtual situation. For more complex situations like rooms or outdoor worlds, the sound propagation becomes highly relevant and very complex. VA uses real-time simulation backends or simplified models to create a physics-based auditory impression.<br />
If update rates are undergoing certain perceptive thresholds, this method can be readily used in Virtual Reality applications.
<h4>Low latency, efficient real-time processing and flexible resource management</h4>
<p>Input-output latency is crucial for any interactive application. VA tries to achieve minimal latency wherever possible, because latency of subsequent components add up. As long as latency is kept low, a human listener will not notice small delays during scene updates, resulting in a convincing live system, where interaction directly leads to the expected effect (without waiting for the system to process).<br />
VA supports real-time capability by establishing flexible data management and processing modules that are lightweight and handle updates efficiently. For example, the FIR filtering modules use a partitioned block convolution resulting in update latencies (at least for the early part of filters) within one single audio block - which usually means a couple of milliseconds. Remotely updating long room impulse responses using Matlab can easily hit 1000 Hz update rates, which under normal circumstances is about three times more a block-based streaming sound card provides. And by far more a dedicated graphics rendering processor achieves, which is often the driving part of scene modifications.<br />
However, this comes at a price: VA is not trading computational resources over update rates. And VA will plainly result in audio dropouts or complete silence, if the computational power is not sufficient for rendering and reproducing the given scene. Simply put, if you request too much, VA will stop auralizing. The number of paths between a sound source and a sound receiver that can effectively be processed is limited. For example, a single binaural free field rendering can calculate up to 20 paths in real-time, but for room acoustics with long reverberation times, a maximum of 6 sources and one listener is realistic (plus requiring the sound propagation simulation to be processed remotely). If reproduction of the rendered audio stream also requires processing power, the numbers go further down.
</p>
<h4>Low latency, efficient real-time processing and flexible resource management</h4>
<p>Input-output latency is crucial for any interactive application. VA tries to achieve minimal latency wherever possible, because latency of subsequent components add up. As long as latency is kept low, a human listener will not notice small delays during scene updates, resulting in a convincing live system, where interaction directly leads to the expected effect (without waiting for the system to process).<br />
VA supports real-time capability by establishing flexible data management and processing modules that are lightweight and handle updates efficiently. For example, the FIR filtering modules use a partitioned block convolution resulting in update latencies (at least for the early part of filters) of a single audio block - which usually means a couple of milliseconds. Remotely updating long room impulse responses using Matlab can easily hit 1000 Hz update rates, which under normal circumstances is about three times more a block-based streaming sound card provides - and by far more a dedicated graphics rendering processor achieves, which is often the driving part of scene modifications.<br />
However, this comes at a price: VA is not trading computational resources over update rates. Advantage is taken by improvement in the general purpose processing power available at present as well as more efficient software libraries. Limitations are solely imposed by the provided processing capacity, and not by the framework. Therefore, VA will plainly result in audio dropouts or complete silence, if the computational power is not sufficient for rendering and reproducing the given scene with the configuration used. Simply put, if you request too much, VA will stop auralizing correctly. Usually, the number of paths between a sound source and a sound receiver that can effectively be processed can be reduced to an amount where the system can operate in real-time. For example, a single binaural free field rendering can roughly calculate up to 20 paths in real-time on a modern PC, but for room acoustics with long reverberation times, a maximum of 6 sources and one listener is realistic (plus the necessity to simulate the sound propagation filters remotely). If reproduction of the rendered audio stream also requires intensive processing power, the numbers go further down.
</p>
<h4>Why is VA a framework?</h4>
<p>You can <a href="download.html">download a ready-to-use VA application</a> and <a href="start.html">individually configure it</a> to reach your target. The combinations of available rendering modules are diverse and therefore VA is suitable for various purposes. The more simple modules provide free-field spatial processing (e.g. using Binaural Technology) for precise localization. More sophisticated modules create certain moods by applying directional artificial reverberation. And others try to be as precise as possible applying physics-based sound propagation simulation for indoor and outdoor scenarios. And there are also possibilities to simply mix ambient sounds that guide or entertain. <br />
......
......@@ -286,12 +286,22 @@
<p>&nbsp;</p>
<h3>Applied VA publications</h3>
<!--
<p>
<i>J. Mecking and M. Vorländer</i><br />
<b></b><br />
<b>Einfluss von Wettermodellen auf die Auralisierung von Flugzeugen</b><br />
DAGA <strong>2018</strong>, München (Germany)
</p> -->
</p>
<p>
<i>J. Oberem and J. Fels</i><br />
<b>Experiments on localization accuracy with non-individual and individual HRTFs comparing static and dynamic reproduction methods</b><br />
DAGA <strong>2018</strong>, München (Germany)
</p>
<p>
<i>R. A. Viveros Munoz and J. Fels</i><br />
<b>Assessment of individual head-related transfer function and undirected head movements of normal listeners in a moving speech-in-noise task using virtual acoustics</b><br />
DAGA <strong>2018</strong>, München (Germany)
</p>
<p>
<i>R. A. Viveros Munoz, Z. E. Peng and J. Fels</i><br />
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment