What's new in JACK2 - Linux Audio...
Transcript of What's new in JACK2 - Linux Audio...
What’s new in JACK2
Stephane Letz, Nedko Arnaudov, Romain Moret
Linux Audio Conference : april 16, 2009
Post LAC 2008 state
Desirable goals from LAC 2008 ”The Future of JACK meeting”
Startup & configuration : a bit of workNetJack : yes in progressDesktop integration : yes in progressInternal design : yes in progressClient programming and API : yes in progress
Not anticipedExternal contacts and developmentsSolaris version with new ”profiling” tools
Post LAC 2008 state
Desirable goals from LAC 2008 ”The Future of JACK meeting”
Startup & configuration : a bit of workNetJack : yes in progressDesktop integration : yes in progressInternal design : yes in progressClient programming and API : yes in progress
Not anticipedExternal contacts and developmentsSolaris version with new ”profiling” tools
What has been done?
2008 DevelopmentsInternal design : code restructuration (Grame)Server control API : (Nedko Arnaudov, Grame)D-Bus based server control (Nedko Arnaudov, JuusoAlasuutari, Grame)NetJack2 (Romain Moret, Grame)”Profiling” tools (Grame)Solaris version (Grame)
OtherExternal contacts : RTL, CopperLan, Native InstrumentsOSX and Windows specific toolsFuture developments (MIDI, ”pipelining”, control...)?
What has been done?
2008 DevelopmentsInternal design : code restructuration (Grame)Server control API : (Nedko Arnaudov, Grame)D-Bus based server control (Nedko Arnaudov, JuusoAlasuutari, Grame)NetJack2 (Romain Moret, Grame)”Profiling” tools (Grame)Solaris version (Grame)
OtherExternal contacts : RTL, CopperLan, Native InstrumentsOSX and Windows specific toolsFuture developments (MIDI, ”pipelining”, control...)?
Internal design
GoalsCleanly separate server and client side servicesDefine a server control APIImprove server internal clients (example of use inNetJack2)
Server and client side libraries
Server sidelibjackserver : JACK API (opens client as ”internal” in theserver...) + control APIused by backends and internal clientsused by ”jackd” and new ”jackdbus” control applicationsallows an application to embed the server in it’s process
Client sidelibjack : JACK API (opens client in separated processesusing IPC...)used by client applications
Server and client side libraries
Server sidelibjackserver : JACK API (opens client as ”internal” in theserver...) + control APIused by backends and internal clientsused by ”jackd” and new ”jackdbus” control applicationsallows an application to embed the server in it’s process
Client sidelibjack : JACK API (opens client in separated processesusing IPC...)used by client applications
Control API
Objectivesdynamically retrieve server control parameters, backendand internal clients parameters, get/set their valuescreate/destroy the serverstart/stop the server with a given backendload/unload internal clientsnow used by ”jackd”
Internal clients
How are they controled and loaded?
using the ”old” way (with jack load -i ”parameters” andjack unload)
using the new control API way : jackctl server load internaland jackctl server unload internal
D-Bus description
What is D-Bus?a simple inter-process communication (IPC) systemprograms register for offering services to othersclients look up which services are available
D-Bus server control access
D-Bus server control accessjackdbus executable to start D-Bus servicebehaves as an interface between D-Bus system and theJACK server, to use JACK control API with D-Busserver autostart done by libjack using control D-Businterface
D-Bus server control access
Exported interfacesjackdbus controller object export several interfacesconfigure the server (parameters access)control the server (start/stop)”patchbay” : improved graph state access (connections,notification of changes...)jack control python control toolbetter presented in Juuso presentation on LASH
NetJack2
Designsimplify the usability modelredesign for easier multi-platform supportcurrently developed for LAN only
Componentskeep the master/slaves modelnetmanager (master) and jack net bakend (slave)”adapters” : audioadapter and netadapter
NetJack2
Designsimplify the usability modelredesign for easier multi-platform supportcurrently developed for LAN only
Componentskeep the master/slaves modelnetmanager (master) and jack net bakend (slave)”adapters” : audioadapter and netadapter
Netmanager
Life cycleloaded as an internal clientwaiting for slaves on a multi-cast addresscreate ”proxy” internal clients for each slave, withrequested number of in/out audio and MIDI ports”proxy” ”automatically” appear and disappear when slavescome and go
Net backend
Life cycleappears as ”available”, server is running in ”dummy” mode(clients can connect...)connecting to master, describes it’s parameters : in/outports...retrieve sample rate and buffer size from the masterstarts processingif master disappear, go back in ”dummy” mode
Data streams
Data streamaudio and MIDIlatency control (using ”fast”, ”normal”, ”slow” transmissionmodes...)number of in/out audio/MIDI ports, address, UDP port,MTU parameters
Adapters
Adapting network to audio interfaceaims at ”adapting” (sample rate, buffer size, clock drift...) anetwork stream on an audio interfaceaudioadapter : adapts a slave synched on Net backend onit’s audio card (generalisation of alsa in/out)a version on each plafform (Linux/ALSA, OSX/CoreAudio,Windows/PortAudio, Solaris/OSS)netadapter : adapts the network stream on a slave runningan audio backend
Adapters
Net backend
Master net « proxy »
Audio adapter Audio card
SLAVE
MASTER
Netadapter
Master net « proxy »
Audio backend Audio card
SLAVE
MASTER
Audioadapter Netadapter
Adapters
How does they work?”produces/consumer” model using an intermediate ringbufferresampling if needed (using libsamplerate)resampling ratio dynamically adjusted using Torben HohnPI controler (JACK1)
”Profiling” tools
Objectivesrecord timing informations when graph is runningtiming of backend, signal, wake-up, end date of each clientcompute scheduling latency and client durationgenerate log files and visualization scripts for GnuPlot
Audio driver interrupt
Duration between successives audio interrupts64 frames, 48 kHz, regular interrupt
1290
1300
1310
1320
1330
1340
1350
1360
1370
0 5000 10000 15000 20000 25000 30000 35000 40000 45000 50000
usec
audio cycles
Audio driver timing
Audio period
Audio driver interrupt
Duration between successives audio interrupts64 frames, 44.1 kHz, non regular interrupt
1200
1400
1600
1800
2000
2200
2400
2600
2800
0 200 400 600 800 1000 1200 1400 1600
usec
audio cycles
Audio driver timing
Audio period
Driver end date
Duration between start and end of cycle at driver level64 frames, 48 kHz, asynchronous mode
0
20
40
60
80
100
120
140
160
180
0 10000 20000 30000 40000 50000 60000 70000
usec
audio cycles
Driver end date
Driver end date
Driver end date
Duration between start and end of cycle at driver level64 frames, 48 kHz, synchronous mode
0
200
400
600
800
1000
1200
1400
0 5000 10000 15000 20000 25000 30000 35000 40000 45000 50000
usec
audio cycles
Driver end date
Driver end date
Clients end date
End date of all clientshere at 64 frames, 48 kHz
0
200
400
600
800
1000
1200
1400
0 5000 10000 15000 20000 25000 30000 35000 40000 45000 50000
usec
audio cycles
Clients end date
Audio periodjack_simple_client
ethersonikethersonik-01ethersonik-02ethersonik-03ethersonik-04ethersonik-05ethersonik-06ethersonik-07ethersonik-08ethersonik-09
Clients scheduling latency
Duration between signal date and actual wake-up datehere at 64 frames, 48 kHz
0
50
100
150
200
250
300
350
400
450
500
0 5000 10000 15000 20000 25000 30000 35000 40000 45000 50000
usec
audio cycles
Clients scheduling
jack_simple_clientethersonik
ethersonik-01ethersonik-02ethersonik-03ethersonik-04ethersonik-05ethersonik-06ethersonik-07ethersonik-08ethersonik-09
Clients duration
Duration between wake-up date and end datehere at 64 frames, 48 kHz
0
50
100
150
200
250
300
350
0 5000 10000 15000 20000 25000 30000 35000 40000 45000 50000
usec
audio cycles
Clients duration
jack_simple_clientethersonik
ethersonik-01ethersonik-02ethersonik-03ethersonik-04ethersonik-05ethersonik-06ethersonik-07ethersonik-08ethersonik-09
Adapters profiling : resample ratio
Server running with the ”dummy” driver and audio adapter
0.9
0.92
0.94
0.96
0.98
1
1.02
1.04
1.06
1.08
1.1
0 500 1000 1500 2000 2500 3000 3500 4000
resa
mpl
ing
ratio
audio cycles
Audio adapter timing: host [rate = 48.0 kHz buffer = 1024 frames] adapter [rate = 44.1 kHz buffer = 256 frames]
Ratio 1Ratio 2
Adapters profiling : position in ringbuffer
Server running with the ”dummy” driver and audio adapter
1400
1600
1800
2000
2200
2400
2600
2800
3000
3200
3400
3600
0 500 1000 1500 2000 2500 3000 3500 4000
fram
es
audio cycles
Audio adapter timing: host [rate = 48.0 kHz buffer = 1024 frames] adapter [rate = 44.1 kHz buffer = 256 frames]
Frames position in consumer ringbufferFrames position in producer ringbuffer
Solaris version
Reviewer comment : why on earth would anybody seriouslyconsider Solaris for pro-audio work?
well, this version was funded by french radio RTL...OSS RME MADI 64 in/out driver developed by HannuSavolainenactually not so bad (could be caracterized using theprofiling tools) : below 80 usec max scheduling latencyusing CPU sets on a highly loaded 4 cores 2 Ghz Dellmachineso good enough for RTL needs
External contacts
RTL french radioDeveloping their entirely ”digital” radio using JACK2 onSolaris with a RME MADI 64 in/out cardProfiling tools to help caracterize real-time behaviour of thesystem
CopperLanA ”complete solution for networking all equipment in thedomains of pro-audio and music” recently presented atFrankfurt MusikMesseKlavis Technologies implemented a JACK / CopperLanbridge prototype on OSX
Native InstrumentsFuture line of NI product using JACK API natively on OSXand Windows
External contacts
RTL french radioDeveloping their entirely ”digital” radio using JACK2 onSolaris with a RME MADI 64 in/out cardProfiling tools to help caracterize real-time behaviour of thesystem
CopperLanA ”complete solution for networking all equipment in thedomains of pro-audio and music” recently presented atFrankfurt MusikMesseKlavis Technologies implemented a JACK / CopperLanbridge prototype on OSX
Native InstrumentsFuture line of NI product using JACK API natively on OSXand Windows
External contacts
RTL french radioDeveloping their entirely ”digital” radio using JACK2 onSolaris with a RME MADI 64 in/out cardProfiling tools to help caracterize real-time behaviour of thesystem
CopperLanA ”complete solution for networking all equipment in thedomains of pro-audio and music” recently presented atFrankfurt MusikMesseKlavis Technologies implemented a JACK / CopperLanbridge prototype on OSX
Native InstrumentsFuture line of NI product using JACK API natively on OSXand Windows
Specific developments on OSX and Windows
JackOSX package
integrates JACK with CoreAudio (JackRouter CoreAudiodevice, JackAU audio unit, JackVST VST plugin...)based on Jackdmp/JACK2 starting early 2008 (version0.75)5 versions released in 2008200 download/day, 1100 users on Yahoo Group
Windowsintegrates JACK with Windows ASIO (JackRouter ASIOdevice)some improvements done in QjackCtlclean installer (Romain Moret in 2008)
Specific developments on OSX and Windows
JackOSX package
integrates JACK with CoreAudio (JackRouter CoreAudiodevice, JackAU audio unit, JackVST VST plugin...)based on Jackdmp/JACK2 starting early 2008 (version0.75)5 versions released in 2008200 download/day, 1100 users on Yahoo Group
Windowsintegrates JACK with Windows ASIO (JackRouter ASIOdevice)some improvements done in QjackCtlclean installer (Romain Moret in 2008)
Contributions (1)
A lot in 2008Recent code from JACK1 : Paul Davis, Florian Faber,Torben Hohn...Nedko Arnaudov, Juuso Alasuutari for D-Bus and wafscriptsRomain Moret for NetJack2Tim Blechmann : code cleanup/optimization, SSE code...Marc-Olivier Barre for D-Bus and ”now dead” sconsscripts...
Contributions (2)
Michael Voigt : JACK2 on L4/DROPS research project
L4 : micro-kernel design and DROPS: Dresden Real-TimeOperating System Projectcode source restructuration for easier later porttiming benchmark : less than 10 usec scheduling latency(+/- 1 usec) (RT Linux is 40 usec +/- 20 usec on samemachine)future : has to be used with L4/Linux to run JACKapplications for Linux on DROPS
The future : what is still needed for 2.0 version?
MIDI bridge on all supported platforms (in progress)
JACK MIDI bridge with native API on each plaform(CoreMIDI on OSX, WinMME on Windows, ALSA MIDI(seq/raw))proposal to use the already existing Master / slave modelin backendallows to activate the MIDI backend independently from theaudio backend
NetJackboth versions should be merged
Missing port latency API?Paul and Fons proposal?
The future : what is still needed for 2.0 version?
MIDI bridge on all supported platforms (in progress)
JACK MIDI bridge with native API on each plaform(CoreMIDI on OSX, WinMME on Windows, ALSA MIDI(seq/raw))proposal to use the already existing Master / slave modelin backendallows to activate the MIDI backend independently from theaudio backend
NetJackboth versions should be merged
Missing port latency API?Paul and Fons proposal?
The future : what is still needed for 2.0 version?
MIDI bridge on all supported platforms (in progress)
JACK MIDI bridge with native API on each plaform(CoreMIDI on OSX, WinMME on Windows, ALSA MIDI(seq/raw))proposal to use the already existing Master / slave modelin backendallows to activate the MIDI backend independently from theaudio backend
NetJackboth versions should be merged
Missing port latency API?Paul and Fons proposal?
The future : what is still needed for 2.0 version? (2)
Audio device reservation schemesmoother collaboration with PulseAudiouse Lennart Poettering proposed D-Bus based reservationAPIallows JACK ALSA backend to take precedence overPulseAudio (unconditionnal reservation)available and to be tested since JACK 1.9.2 version
The future : various ideas
Multi-backend modelMaster / slave model for audio backend...
More control frontendOSC (network control: server start/stop, connectionstate...)
”libjacknet” ideapublish the NetJack API (master, slave, adapter) in aseparated ”libjacknet” libraryallows audio components to access NetJack without theneed of a JACK serversuccessfully tested on iPhone simulator (master or slave)
The future : various ideas
Multi-backend modelMaster / slave model for audio backend...
More control frontendOSC (network control: server start/stop, connectionstate...)
”libjacknet” ideapublish the NetJack API (master, slave, adapter) in aseparated ”libjacknet” libraryallows audio components to access NetJack without theneed of a JACK serversuccessfully tested on iPhone simulator (master or slave)
The future : various ideas
Multi-backend modelMaster / slave model for audio backend...
More control frontendOSC (network control: server start/stop, connectionstate...)
”libjacknet” ideapublish the NetJack API (master, slave, adapter) in aseparated ”libjacknet” libraryallows audio components to access NetJack without theneed of a JACK serversuccessfully tested on iPhone simulator (master or slave)
The future : pipelining version (1)
What is it for?allows to better use multi-cores machines with ”sequential”graphsgeneral principle : executes the graph with a buffer size ofD/N (D : driver buffer size, N : divisor)
The future : pipelining version (2)
One example
A B
A(2)A(1) A(3)
B(3)B(1)
A(4)
B(2) B(4)
Dividing the buffer by 4
Two cores running A and B
The future : pipelining version (3)
divisor can be dynamically changed, and causes a”buffer-size change” notificationclient can chose *not* to be pipelined : so hybrid graphscan be runavailable on a separated ”pipelining” branch on SVNtesting in real word situations welcome! (Ardour 2.8 isready for that...)