Adding experimental renderer example jupyter script

parent 6e344b47
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# VA experimental renderer example\n",
"VA has prototype renderers, that can be used for quick auralization of any given situation. A renderer of the **PrototypeGenericPath** class uses a uniform block convolution of required number of channels and filter length. For each source-listener-pair, a new convolution instance is provided. It can be updated using the parameter setter, i.e. the FIR filter can be updated in real-time with an impulse resonse (IR) in time domain directly out of Python (or Matlab). On server side, no files (except for the signal sources, if an audio file is used) have to be provided.\n",
"\n",
"The filter length, the number of channels and the number of source-listener-pairs are only limited by the computational power you can provide.\n",
"\n",
"This combination of a VA server with an experimental rendering module using the **PrototypeGenericPath** renderer can effectively be used for teaching purposes, i.e. to implement a real-time binaural auralization in Python or Matlab."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Prerequisites"
]
},
{
"cell_type": "code",
"execution_count": 42,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Connected.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"c:\\users\\jonas\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\ipykernel_launcher.py:5: RuntimeWarning: Was still connected, forced disconnect.\n",
" \"\"\"\n"
]
}
],
"source": [
"import sys\n",
"sys.path.append( '../../Lib/site-packages' )\n",
"import ipywidgets as widgets\n",
"import va\n",
"if not va.connect() :\n",
" raise 'Could not connect to local VA server'\n",
"else :\n",
" print( 'Connected.' )\n",
"va.reset()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Auralize room acoustics with a measured binaural room impulse response (BRIR)\n",
"### Configuration\n",
"You have to start a VA server, that instatiates a **PrototypeGenericPath** renderer with **two channels** and a filter length that is long enough to fit the reverberation time of the room you want to make audible, for example **2 seconds** (or **88200 samples** at a sampling frequency of 44.1 kHz). If you start the experimental server provided with a VA binary package, this configuration is already in place. See also `conf/VACore.experimental.ini` for further details.\n",
"### Rendering module name\n",
"Check out the rendering module name to know which rendering module to call. Use `get_rendering_modules` like this:"
]
},
{
"cell_type": "code",
"execution_count": 43,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"PrototypeGenericPath:Experimental\n"
]
}
],
"source": [
"for rmod in va.get_rendering_modules() :\n",
" if rmod['class'] == 'PrototypeGenericPath' :\n",
" print( rmod['class'] + ':' + rmod['id'] )\n",
"rmod_name = 'PrototypeGenericPath:Experimental' # alter this if you are using a different name"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Get renderer information\n",
"To receive useful information, renderer usually return available configurations if `get_renderer_parameters` is called without arguments. Try this:"
]
},
{
"cell_type": "code",
"execution_count": 44,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'help': \"\\n --- GenericPath renderer instance 'PrototypeGenericPath:Experimental' ---\\n\\n[help]\\nIf the call module struct contains a key with the name 'help', this help text will be shown and the return struct will be returned with the key name 'help'.\\n\\n[info]\\nIf the call module struct contains a key with the name 'info', information on the static configuration of the renderer will be returned.\\n\\n[update]\\nFor every successful path update, the VA source and listener ID has to be passed like this:\\n listener: <int>, the number of the listener identifier\\n source: <int>, the number of the source identifier\\n\\nUpdating the path filter (impulse response in time domain) for a listener and a source can be performed in two ways:\\n a) using a path to a multi-channel WAV file:\\n Provide a key with the name 'filepath' and the path to the WAV file (absolute or containing the macro '$(VADataDir)' or relative to the executable) [priority given to 'filepath' if b) also applies]\\n b) sending floating-point data for each channel\\n Provide a key for each channel with the generic name 'ch#', where the hash is substituted by the actual number of channel (starting at 1), and the value to this key will contain floating point data (or a sample buffer). The call parameter struct does not necessarily have to contain all channels, also single channels will be updated if key is given.\\n\\nNote: the existence of the key 'verbose' will print update information at server console and will provide the update info as an 'info' key in the returned struct.\\n\"}"
]
},
"execution_count": 44,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"va.call_module( rmod_name, {} )"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Or, now that we know we should use the `info` key, type"
]
},
{
"cell_type": "code",
"execution_count": 45,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'filterdelaysamples': 0,\n",
" 'irfilterlengthsamples': 88200,\n",
" 'numchannels': 2,\n",
" 'numpaths': 0}"
]
},
"execution_count": 45,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"va.call_module( rmod_name, { 'info' : True } )"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Input data preparation\n",
"Let us quickly set up a virtual scene using input data from the Internet.\n",
"Download for example anechoic recordings directly from [here](http://www.openairlib.net/sites/default/files/anechoic/data/judebrereton/modern-clarinet-bb/mono/cl-mod-bb-piece-32.wav) and a binaural impulse response from [here](http://www.openairlib.net/sites/default/files/auralization/data/audiolab/lady-chapel-st-albans-cathedral/stereo/stalbans_a_binaural.wav). Either add the download folder as search path, or put the files where VA can find it."
]
},
{
"cell_type": "code",
"execution_count": 46,
"metadata": {},
"outputs": [],
"source": [
"va.add_search_path( 'C:/Users/jonas/Downloads' )\n",
"signal_source_id = va.create_audio_file_signal_source( 'cl-mod-bb-piece-32.wav' )\n",
"va.set_audio_file_signal_source_playback_action_str( signal_source_id, 'play' )\n",
"va.set_audio_file_signal_source_playback_is_looping( signal_source_id, True )"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Creating the scene\n",
"To update a source-listener-pair, a scene should be set up."
]
},
{
"cell_type": "code",
"execution_count": 47,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Experimental sound source id: 1\n",
"Experimental listener id: 1\n"
]
}
],
"source": [
"sound_source_id = va.create_sound_source( 'PyExperimentalSoundSource' )\n",
"print( 'Experimental sound source id: ' + str( sound_source_id ) )\n",
"va.set_sound_source_signal_source( sound_source_id, signal_source_id )\n",
"listener_id = va.create_listener( 'PyExperimentalListener' )\n",
"print( 'Experimental listener id: ' + str( sound_source_id ) )"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Updating paths\n",
"Now that we have a source-listener-pair established, we can update the impulse response of that path. To do so, we have to assembly a `dict` variable that provides the required information. This `dict` will be transmitted to the renderer and the update will be performed.\n",
"\n",
"### Setting a simple unequal two-channel dirac"
]
},
{
"cell_type": "code",
"execution_count": 51,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'listener': 1, 'source': 1, 'ch1': [0.9, 0.0, 0.0, 0.0], 'ch2': [0.0, 0.0, -0.4, 0.0], 'verbose': True}\n"
]
}
],
"source": [
"update_dirac = dict()\n",
"update_dirac[ 'listener' ] = listener_id\n",
"update_dirac[ 'source' ] = sound_source_id\n",
"update_dirac[ 'ch1' ] = [ 0.9, 0.0, 0.0, 0.0 ] # Length of samples is arbitrary, here\n",
"update_dirac[ 'ch2' ] = [ 0.0, 0.0, -0.4, 0.0 ] # Length of samples is arbitrary, here\n",
"update_dirac[ 'verbose' ] = True; # Get information about update as a result\n",
"print( update_dirac )"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now all we have to do is transmit the update task to the renderer"
]
},
{
"cell_type": "code",
"execution_count": 61,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Updated sound path <L1, S1, C1> with 4 new samples\n",
"Updated sound path <L1, S1, C2> with 4 new samples\n"
]
}
],
"source": [
"msg = va.call_module( rmod_name, update_dirac )\n",
"print( msg[ 'info_ch1' ] )\n",
"print( msg[ 'info_ch2' ] )"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Update by loading from a file path\n",
"\n",
"It is not necessary to transmit an entire impulse response for each channel to the path you want to update. You can also use a file path for a single or all channels."
]
},
{
"cell_type": "code",
"execution_count": 59,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'listener': 1, 'source': 1, 'filepath': 'stalbans_a_binaural.wav', 'verbose': True}\n"
]
}
],
"source": [
"update_filepath = dict()\n",
"update_filepath[ 'listener' ] = listener_id\n",
"update_filepath[ 'source' ] = sound_source_id\n",
"update_filepath[ 'filepath' ] = 'stalbans_a_binaural.wav'\n",
"#update_filepath[ 'channel' ] = 2 # ... in case you explicitly want to update a single channel with a mono IR file\n",
"update_filepath[ 'verbose' ] = True;\n",
"print( update_filepath )"
]
},
{
"cell_type": "code",
"execution_count": 57,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Updated sound path <L1, S1> using (unrolled) file path 'C:\\Users\\jonas\\Downloads\\stalbans_a_binaural.wav'\n"
]
}
],
"source": [
"msg = va.call_module( rmod_name, update_filepath )\n",
"print( msg[ 'info' ] )"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.2"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment