LabStreamingLayer Overivew

86
LabStreamingLayer Overivew David Medine [email protected] Swartz Center for Computational Neuroscience EEGLab Workshop November 20, 2016 Medine LabStreamingLayer Overivew

Transcript of LabStreamingLayer Overivew

Page 1: LabStreamingLayer Overivew

LabStreamingLayer Overivew

David [email protected]

Swartz Center for Computational Neuroscience

EEGLab WorkshopNovember 20, 2016

Medine LabStreamingLayer Overivew

Page 2: LabStreamingLayer Overivew

0. Outline

1 Introduction

2 Why is LSL

3 A Brief Description of LSL

Medine LabStreamingLayer Overivew

Page 3: LabStreamingLayer Overivew

1. Introduction What is LabStreamingLayer?

open source software libraryhighly extensible and supports wrappers for numerouslanguagesoverlay networkrealtime data streamingfacilitates multimodal data stream synchronization withouthardware triggers

Medine LabStreamingLayer Overivew

Page 4: LabStreamingLayer Overivew

1. Introduction What is LabStreamingLayer?

open source software library

highly extensible and supports wrappers for numerouslanguages

overlay network

realtime data streaming

facilitates multimodal data stream synchronization withouthardware triggers

Medine LabStreamingLayer Overivew

Page 5: LabStreamingLayer Overivew

1. Introduction What is LabStreamingLayer?

open source software library

highly extensible and supports wrappers for numerouslanguages

overlay network

realtime data streaming

facilitates multimodal data stream synchronization withouthardware triggers

Medine LabStreamingLayer Overivew

Page 6: LabStreamingLayer Overivew

1. Introduction What is LabStreamingLayer?

open source software library

highly extensible and supports wrappers for numerouslanguages

overlay network

realtime data streaming

facilitates multimodal data stream synchronization withouthardware triggers

Medine LabStreamingLayer Overivew

Page 7: LabStreamingLayer Overivew

1. Introduction What is LabStreamingLayer?

open source software library

highly extensible and supports wrappers for numerouslanguages

overlay network

realtime data streaming

facilitates multimodal data stream synchronization withouthardware triggers

Medine LabStreamingLayer Overivew

Page 8: LabStreamingLayer Overivew

1. Introduction What is LabStreamingLayer?

open source software library

highly extensible and supports wrappers for numerouslanguages

overlay network

realtime data streaming

facilitates multimodal data stream synchronization withouthardware triggers

Medine LabStreamingLayer Overivew

Page 9: LabStreamingLayer Overivew

2. Why is LSL Mobile Brain/Body Imaging (MoBI)

Medine LabStreamingLayer Overivew

Page 10: LabStreamingLayer Overivew

2. Why is LSL Mobile Brain/Body Imaging (MoBI)

Medine LabStreamingLayer Overivew

Page 11: LabStreamingLayer Overivew

2. Why is LSL MoBI in a Box

Medine LabStreamingLayer Overivew

Page 12: LabStreamingLayer Overivew

2. Why is LSL Large Scale Experiments

Medine LabStreamingLayer Overivew

Page 13: LabStreamingLayer Overivew

2. Why is LSL Synchronizing

Medine LabStreamingLayer Overivew

Page 14: LabStreamingLayer Overivew

2. Why is LSL Synchronizing

The Challenge: To Synchronize Multiple (Asynchronous) Streams

EEG/EMG/EOG – i.e. with BioSemi – 512 Hz

Stimulus Presentation Markers – i.e. with Presentation –random times

Motion Capture – i.e. with PhaseSpace – 400 Hz

Eyetracking – i.e. with EyeLink – 60 Hz

HCI, GSR, Heartrate, Forceplate, etc. ?? Hz

Medine LabStreamingLayer Overivew

Page 15: LabStreamingLayer Overivew

2. Why is LSL Synchronizing

The Challenge: To Synchronize Multiple (Asynchronous) Streams

EEG/EMG/EOG – i.e. with BioSemi – 512 Hz

Stimulus Presentation Markers – i.e. with Presentation –random times

Motion Capture – i.e. with PhaseSpace – 400 Hz

Eyetracking – i.e. with EyeLink – 60 Hz

HCI, GSR, Heartrate, Forceplate, etc. ?? Hz

Medine LabStreamingLayer Overivew

Page 16: LabStreamingLayer Overivew

2. Why is LSL Synchronizing

The Challenge: To Synchronize Multiple (Asynchronous) Streams

EEG/EMG/EOG – i.e. with BioSemi – 512 Hz

Stimulus Presentation Markers – i.e. with Presentation –random times

Motion Capture – i.e. with PhaseSpace – 400 Hz

Eyetracking – i.e. with EyeLink – 60 Hz

HCI, GSR, Heartrate, Forceplate, etc. ?? Hz

Medine LabStreamingLayer Overivew

Page 17: LabStreamingLayer Overivew

2. Why is LSL Synchronizing

The Challenge: To Synchronize Multiple (Asynchronous) Streams

EEG/EMG/EOG – i.e. with BioSemi – 512 Hz

Stimulus Presentation Markers – i.e. with Presentation –random times

Motion Capture – i.e. with PhaseSpace – 400 Hz

Eyetracking – i.e. with EyeLink – 60 Hz

HCI, GSR, Heartrate, Forceplate, etc. ?? Hz

Medine LabStreamingLayer Overivew

Page 18: LabStreamingLayer Overivew

2. Why is LSL Synchronizing

The Challenge: To Synchronize Multiple (Asynchronous) Streams

EEG/EMG/EOG – i.e. with BioSemi – 512 Hz

Stimulus Presentation Markers – i.e. with Presentation –random times

Motion Capture – i.e. with PhaseSpace – 400 Hz

Eyetracking – i.e. with EyeLink – 60 Hz

HCI, GSR, Heartrate, Forceplate, etc. ?? Hz

Medine LabStreamingLayer Overivew

Page 19: LabStreamingLayer Overivew

2. Why is LSL Synchronizing

The Challenge: To Synchronize Multiple (Asynchronous) Streams

EEG/EMG/EOG – i.e. with BioSemi – 512 Hz

Stimulus Presentation Markers – i.e. with Presentation –random times

Motion Capture – i.e. with PhaseSpace – 400 Hz

Eyetracking – i.e. with EyeLink – 60 Hz

HCI, GSR, Heartrate, Forceplate, etc. ?? Hz

Medine LabStreamingLayer Overivew

Page 20: LabStreamingLayer Overivew

2. Why is LSL Synchronizing

The Challenge: To Synchronize Multiple (Asynchronous) Streams

Attach timestamps to data

Attach timestamps to record times

Measue and test EVERYTHING

OVER AND OVER

Medine LabStreamingLayer Overivew

Page 21: LabStreamingLayer Overivew

2. Why is LSL Synchronizing

The Challenge: To Synchronize Multiple (Asynchronous) Streams

Attach timestamps to data

Attach timestamps to record times

Measue and test EVERYTHING

OVER AND OVER

Medine LabStreamingLayer Overivew

Page 22: LabStreamingLayer Overivew

2. Why is LSL Synchronizing

The Challenge: To Synchronize Multiple (Asynchronous) Streams

Attach timestamps to data

Attach timestamps to record times

Measue and test EVERYTHING

OVER AND OVER

Medine LabStreamingLayer Overivew

Page 23: LabStreamingLayer Overivew

2. Why is LSL Synchronizing

The Challenge: To Synchronize Multiple (Asynchronous) Streams

Attach timestamps to data

Attach timestamps to record times

Measue and test EVERYTHING

OVER AND OVER

Medine LabStreamingLayer Overivew

Page 24: LabStreamingLayer Overivew

2. Why is LSL Synchronizing

The Challenge: To Synchronize Multiple (Asynchronous) Streams

Attach timestamps to data

Attach timestamps to record times

Measue and test EVERYTHING OVER AND OVER

Medine LabStreamingLayer Overivew

Page 25: LabStreamingLayer Overivew

3. A Brief Description of LSL Overview

LSL Network View

Medine LabStreamingLayer Overivew

Page 26: LabStreamingLayer Overivew

3. A Brief Description of LSL Overview – liblsl

liblsl

liblsl is an opensource C/C++ library and API for timestampingand streaming multimodal data accross a network.

Basically provides methods for creating 3 objects:

stream info (metadata)stream outlet (send data)stream inlet (receive data)

Medine LabStreamingLayer Overivew

Page 27: LabStreamingLayer Overivew

3. A Brief Description of LSL Overview – liblsl

liblsl

liblsl is an opensource C/C++ library and API for timestampingand streaming multimodal data accross a network.

Basically provides methods for creating 3 objects:

stream info (metadata)

stream outlet (send data)stream inlet (receive data)

Medine LabStreamingLayer Overivew

Page 28: LabStreamingLayer Overivew

3. A Brief Description of LSL Overview – liblsl

liblsl

liblsl is an opensource C/C++ library and API for timestampingand streaming multimodal data accross a network.

Basically provides methods for creating 3 objects:

stream info (metadata)stream outlet (send data)

stream inlet (receive data)

Medine LabStreamingLayer Overivew

Page 29: LabStreamingLayer Overivew

3. A Brief Description of LSL Overview – liblsl

liblsl

liblsl is an opensource C/C++ library and API for timestampingand streaming multimodal data accross a network.

Basically provides methods for creating 3 objects:

stream info (metadata)stream outlet (send data)stream inlet (receive data)

Medine LabStreamingLayer Overivew

Page 30: LabStreamingLayer Overivew

3. A Brief Description of LSL Details – stream info

stream info constructor params

Specifies the nature of an lsl stream (essentials)

name (‘ActiChamp’)

type (‘EEG’)

channel count (‘64’)

sample rate(‘512’)

format (‘float32’)

source Id (‘8JIAes263D’ some such serial number)

append meta-data

XML based information that adheres to a basic template, but canbe extended to contain anything, anyhow

Medine LabStreamingLayer Overivew

Page 31: LabStreamingLayer Overivew

3. A Brief Description of LSL Details – stream info

stream info constructor params

Specifies the nature of an lsl stream (essentials)

name (‘ActiChamp’)

type (‘EEG’)

channel count (‘64’)

sample rate(‘512’)

format (‘float32’)

source Id (‘8JIAes263D’ some such serial number)

append meta-data

XML based information that adheres to a basic template, but canbe extended to contain anything, anyhow

Medine LabStreamingLayer Overivew

Page 32: LabStreamingLayer Overivew

3. A Brief Description of LSL Details – stream info

stream info constructor params

Specifies the nature of an lsl stream (essentials)

name (‘ActiChamp’)

type (‘EEG’)

channel count (‘64’)

sample rate(‘512’)

format (‘float32’)

source Id (‘8JIAes263D’ some such serial number)

append meta-data

XML based information that adheres to a basic template, but canbe extended to contain anything, anyhow

Medine LabStreamingLayer Overivew

Page 33: LabStreamingLayer Overivew

3. A Brief Description of LSL Details – stream outlet

stream outlet constructor params

stream info object

(optionally) chunk size

(optionally) data buffer size

stream outlet life-cycle

once created, a stream outlet object sits on an open socket(a data stream publisher) and can be pinged by clients toinitiate connection (subscription)

a loop on a separate thread can stream data or poll for data(push sample) to output sporadically

destroy after using...(show some code)

Medine LabStreamingLayer Overivew

Page 34: LabStreamingLayer Overivew

3. A Brief Description of LSL Details – stream outlet

stream outlet constructor params

stream info object

(optionally) chunk size

(optionally) data buffer size

stream outlet life-cycle

once created, a stream outlet object sits on an open socket(a data stream publisher) and can be pinged by clients toinitiate connection (subscription)

a loop on a separate thread can stream data or poll for data(push sample) to output sporadically

destroy after using...(show some code)

Medine LabStreamingLayer Overivew

Page 35: LabStreamingLayer Overivew

3. A Brief Description of LSL Details – stream outlet

stream outlet constructor params

stream info object

(optionally) chunk size

(optionally) data buffer size

stream outlet life-cycle

once created, a stream outlet object sits on an open socket(a data stream publisher) and can be pinged by clients toinitiate connection (subscription)

a loop on a separate thread can stream data or poll for data(push sample) to output sporadically

destroy after using...(show some code)

Medine LabStreamingLayer Overivew

Page 36: LabStreamingLayer Overivew

3. A Brief Description of LSL Details – stream outlet

stream outlet constructor params

stream info object

(optionally) chunk size

(optionally) data buffer size

stream outlet life-cycle

once created, a stream outlet object sits on an open socket(a data stream publisher) and can be pinged by clients toinitiate connection (subscription)

a loop on a separate thread can stream data or poll for data(push sample) to output sporadically

destroy after using...(show some code)

Medine LabStreamingLayer Overivew

Page 37: LabStreamingLayer Overivew

3. A Brief Description of LSL Details – stream outlet

stream outlet constructor params

stream info object

(optionally) chunk size

(optionally) data buffer size

stream outlet life-cycle

once created, a stream outlet object sits on an open socket(a data stream publisher) and can be pinged by clients toinitiate connection (subscription)

a loop on a separate thread can stream data or poll for data(push sample) to output sporadically

destroy after using...(show some code)

Medine LabStreamingLayer Overivew

Page 38: LabStreamingLayer Overivew

3. A Brief Description of LSL Details – stream outlet

stream outlet constructor params

stream info object

(optionally) chunk size

(optionally) data buffer size

stream outlet life-cycle

once created, a stream outlet object sits on an open socket(a data stream publisher) and can be pinged by clients toinitiate connection (subscription)

a loop on a separate thread can stream data or poll for data(push sample) to output sporadically

destroy after using...(show some code)

Medine LabStreamingLayer Overivew

Page 39: LabStreamingLayer Overivew

3. A Brief Description of LSL Details – stream outlet

stream outlet constructor params

stream info object

(optionally) chunk size

(optionally) data buffer size

stream outlet life-cycle

once created, a stream outlet object sits on an open socket(a data stream publisher) and can be pinged by clients toinitiate connection (subscription)

a loop on a separate thread can stream data or poll for data(push sample) to output sporadically

destroy after using...(show some code)

Medine LabStreamingLayer Overivew

Page 40: LabStreamingLayer Overivew

3. A Brief Description of LSL Details – stream outlet

stream outlet constructor params

stream info object

(optionally) chunk size

(optionally) data buffer size

stream outlet life-cycle

once created, a stream outlet object sits on an open socket(a data stream publisher) and can be pinged by clients toinitiate connection (subscription)

a loop on a separate thread can stream data or poll for data(push sample) to output sporadically

destroy after using...(show some code)

Medine LabStreamingLayer Overivew

Page 41: LabStreamingLayer Overivew

3. A Brief Description of LSL Details – stream inlet

resolve stream

@param prop (‘name’, ‘type’, etc.)

@param value (‘ActiChamp’, ‘EEG’, etc.)

@return std::vector<stream info>

stream inlet constructor

@param info (one of the stream info objects returned byresolve stream)

(optional) @params buffer length, chunk length, recover (bool)

Medine LabStreamingLayer Overivew

Page 42: LabStreamingLayer Overivew

3. A Brief Description of LSL Details – stream inlet

resolve stream

@param prop (‘name’, ‘type’, etc.)

@param value (‘ActiChamp’, ‘EEG’, etc.)

@return std::vector<stream info>

stream inlet constructor

@param info (one of the stream info objects returned byresolve stream)

(optional) @params buffer length, chunk length, recover (bool)

Medine LabStreamingLayer Overivew

Page 43: LabStreamingLayer Overivew

3. A Brief Description of LSL Details – stream inlet

resolve stream

@param prop (‘name’, ‘type’, etc.)

@param value (‘ActiChamp’, ‘EEG’, etc.)

@return std::vector<stream info>

stream inlet constructor

@param info (one of the stream info objects returned byresolve stream)

(optional) @params buffer length, chunk length, recover (bool)

Medine LabStreamingLayer Overivew

Page 44: LabStreamingLayer Overivew

3. A Brief Description of LSL Details – stream inlet

resolve stream

@param prop (‘name’, ‘type’, etc.)

@param value (‘ActiChamp’, ‘EEG’, etc.)

@return std::vector<stream info>

stream inlet constructor

@param info (one of the stream info objects returned byresolve stream)

(optional) @params buffer length, chunk length, recover (bool)

Medine LabStreamingLayer Overivew

Page 45: LabStreamingLayer Overivew

3. A Brief Description of LSL Details – stream inlet

stream inlet life-cycle

automatically connect to corresponding outlet

launch a ‘listen’ thread to repeatedly call pull sample

if recover=true, automatically respawn if lost

destroy when all done (show some code)

Medine LabStreamingLayer Overivew

Page 46: LabStreamingLayer Overivew

3. A Brief Description of LSL Details – stream inlet

stream inlet life-cycle

automatically connect to corresponding outlet

launch a ‘listen’ thread to repeatedly call pull sample

if recover=true, automatically respawn if lost

destroy when all done (show some code)

Medine LabStreamingLayer Overivew

Page 47: LabStreamingLayer Overivew

3. A Brief Description of LSL Details – stream inlet

stream inlet life-cycle

automatically connect to corresponding outlet

launch a ‘listen’ thread to repeatedly call pull sample

if recover=true, automatically respawn if lost

destroy when all done (show some code)

Medine LabStreamingLayer Overivew

Page 48: LabStreamingLayer Overivew

3. A Brief Description of LSL Details – stream inlet

stream inlet life-cycle

automatically connect to corresponding outlet

launch a ‘listen’ thread to repeatedly call pull sample

if recover=true, automatically respawn if lost

destroy when all done (show some code)

Medine LabStreamingLayer Overivew

Page 49: LabStreamingLayer Overivew

3. A Brief Description of LSL Details – stream inlet

stream inlet life-cycle

automatically connect to corresponding outlet

launch a ‘listen’ thread to repeatedly call pull sample

if recover=true, automatically respawn if lost

destroy when all done (show some code)

Medine LabStreamingLayer Overivew

Page 50: LabStreamingLayer Overivew

3. A Brief Description of LSL Overview – Apps

31 Apps to interface LSL with various devices

Link to LSL and vendor libraries (if any)

Setup ‘streaminfo’ and metadata (xml)

Open LSL ‘outlet’

Launch listener thread to ‘push’ LSL data as it arrives

Medine LabStreamingLayer Overivew

Page 51: LabStreamingLayer Overivew

3. A Brief Description of LSL Overview – Apps

31 Apps to interface LSL with various devices

Link to LSL and vendor libraries (if any)

Setup ‘streaminfo’ and metadata (xml)

Open LSL ‘outlet’

Launch listener thread to ‘push’ LSL data as it arrives

Medine LabStreamingLayer Overivew

Page 52: LabStreamingLayer Overivew

3. A Brief Description of LSL Overview – Apps

31 Apps to interface LSL with various devices

Link to LSL and vendor libraries (if any)

Setup ‘streaminfo’ and metadata (xml)

Open LSL ‘outlet’

Launch listener thread to ‘push’ LSL data as it arrives

Medine LabStreamingLayer Overivew

Page 53: LabStreamingLayer Overivew

3. A Brief Description of LSL Overview – Apps

31 Apps to interface LSL with various devices

Link to LSL and vendor libraries (if any)

Setup ‘streaminfo’ and metadata (xml)

Open LSL ‘outlet’

Launch listener thread to ‘push’ LSL data as it arrives

Medine LabStreamingLayer Overivew

Page 54: LabStreamingLayer Overivew

3. A Brief Description of LSL Overview – Apps

31 Apps to interface LSL with various devices

Link to LSL and vendor libraries (if any)

Setup ‘streaminfo’ and metadata (xml)

Open LSL ‘outlet’

Launch listener thread to ‘push’ LSL data as it arrives

Medine LabStreamingLayer Overivew

Page 55: LabStreamingLayer Overivew

3. A Brief Description of LSL Overview – Apps

Medine LabStreamingLayer Overivew

Page 56: LabStreamingLayer Overivew

3. A Brief Description of LSL Overview – Apps

Medine LabStreamingLayer Overivew

Page 57: LabStreamingLayer Overivew

3. A Brief Description of LSL Overview – LabRecorder and load xdf.m

Typical Setup:

Medine LabStreamingLayer Overivew

Page 58: LabStreamingLayer Overivew

3. A Brief Description of LSL Overview – LabRecorder and load xdf.m

LabRecorder

Locate streams, writemetadata to ouput fileheader

Read streams,write data toouput file

Periodically check clockoffsets (NTP), write offsetsto output file footer

Medine LabStreamingLayer Overivew

load xdf.m

Organize meta-data into adatastructure

Linearize non-sporadictimestamps

Use clock offsets andknown latencies tosynchronize data streams

Page 59: LabStreamingLayer Overivew

3. A Brief Description of LSL Overview – LabRecorder and load xdf.m

LabRecorder

Locate streams, writemetadata to ouput fileheader

Read streams,write data toouput file

Periodically check clockoffsets (NTP), write offsetsto output file footer

Medine LabStreamingLayer Overivew

load xdf.m

Organize meta-data into adatastructure

Linearize non-sporadictimestamps

Use clock offsets andknown latencies tosynchronize data streams

Page 60: LabStreamingLayer Overivew

3. A Brief Description of LSL Overview – LabRecorder and load xdf.m

LabRecorder

Locate streams, writemetadata to ouput fileheader

Read streams,write data toouput file

Periodically check clockoffsets (NTP), write offsetsto output file footer

Medine LabStreamingLayer Overivew

load xdf.m

Organize meta-data into adatastructure

Linearize non-sporadictimestamps

Use clock offsets andknown latencies tosynchronize data streams

Page 61: LabStreamingLayer Overivew

3. A Brief Description of LSL Overview – LabRecorder and load xdf.m

LabRecorder

Locate streams, writemetadata to ouput fileheader

Read streams,write data toouput file

Periodically check clockoffsets (NTP), write offsetsto output file footer

Medine LabStreamingLayer Overivew

load xdf.m

Organize meta-data into adatastructure

Linearize non-sporadictimestamps

Use clock offsets andknown latencies tosynchronize data streams

Page 62: LabStreamingLayer Overivew

3. A Brief Description of LSL Overview – LabRecorder and load xdf.m

LabRecorder

Locate streams, writemetadata to ouput fileheader

Read streams,write data toouput file

Periodically check clockoffsets (NTP), write offsetsto output file footer

Medine LabStreamingLayer Overivew

load xdf.m

Organize meta-data into adatastructure

Linearize non-sporadictimestamps

Use clock offsets andknown latencies tosynchronize data streams

Page 63: LabStreamingLayer Overivew

3. A Brief Description of LSL Overview – LabRecorder and load xdf.m

LabRecorder

Locate streams, writemetadata to ouput fileheader

Read streams,write data toouput file

Periodically check clockoffsets (NTP), write offsetsto output file footer

Medine LabStreamingLayer Overivew

load xdf.m

Organize meta-data into adatastructure

Linearize non-sporadictimestamps

Use clock offsets andknown latencies tosynchronize data streams

Page 64: LabStreamingLayer Overivew

3. A Brief Description of LSL Overview – LabRecorder and load xdf.m

LabRecorder

Locate streams, writemetadata to ouput fileheader

Read streams,write data toouput file

Periodically check clockoffsets (NTP), write offsetsto output file footer

Medine LabStreamingLayer Overivew

load xdf.m

Organize meta-data into adatastructure

Linearize non-sporadictimestamps

Use clock offsets andknown latencies tosynchronize data streams

Page 65: LabStreamingLayer Overivew

3. A Brief Description of LSL Overview – LabRecorder and load xdf.m

LabRecorder

Locate streams, writemetadata to ouput fileheader

Read streams,write data toouput file

Periodically check clockoffsets (NTP), write offsetsto output file footer

Medine LabStreamingLayer Overivew

load xdf.m

Organize meta-data into adatastructure

Linearize non-sporadictimestamps

Use clock offsets andknown latencies tosynchronize data streams

Page 66: LabStreamingLayer Overivew

3. A Brief Description of LSL Synchronization

1: Calculate Clock Offsets

Record clock offsets peridiocally during data acquisition usingclock filter algorithm (NTP) using get time of day to recordsets of 4 timestamps in rapid succession:

send from inlet to outlet (t0)receive from inlet at outlet (t1)immediately send from outlet to inlet (t3)receove from outlet at inlet (t4)

round trip time (RTT) = (t3-t0) - (t2-t1)

clock offset (OFS) = ((t1-t0) + (t2-t3))/2 (for lowest RTT)

Medine LabStreamingLayer Overivew

Page 67: LabStreamingLayer Overivew

3. A Brief Description of LSL Synchronization

1: Calculate Clock Offsets

Record clock offsets peridiocally during data acquisition usingclock filter algorithm (NTP) using get time of day to recordsets of 4 timestamps in rapid succession:

send from inlet to outlet (t0)receive from inlet at outlet (t1)immediately send from outlet to inlet (t3)receove from outlet at inlet (t4)

round trip time (RTT) = (t3-t0) - (t2-t1)

clock offset (OFS) = ((t1-t0) + (t2-t3))/2 (for lowest RTT)

Medine LabStreamingLayer Overivew

Page 68: LabStreamingLayer Overivew

3. A Brief Description of LSL Synchronization

1: Calculate Clock Offsets

Record clock offsets peridiocally during data acquisition usingclock filter algorithm (NTP) using get time of day to recordsets of 4 timestamps in rapid succession:

send from inlet to outlet (t0)

receive from inlet at outlet (t1)immediately send from outlet to inlet (t3)receove from outlet at inlet (t4)

round trip time (RTT) = (t3-t0) - (t2-t1)

clock offset (OFS) = ((t1-t0) + (t2-t3))/2 (for lowest RTT)

Medine LabStreamingLayer Overivew

Page 69: LabStreamingLayer Overivew

3. A Brief Description of LSL Synchronization

1: Calculate Clock Offsets

Record clock offsets peridiocally during data acquisition usingclock filter algorithm (NTP) using get time of day to recordsets of 4 timestamps in rapid succession:

send from inlet to outlet (t0)receive from inlet at outlet (t1)

immediately send from outlet to inlet (t3)receove from outlet at inlet (t4)

round trip time (RTT) = (t3-t0) - (t2-t1)

clock offset (OFS) = ((t1-t0) + (t2-t3))/2 (for lowest RTT)

Medine LabStreamingLayer Overivew

Page 70: LabStreamingLayer Overivew

3. A Brief Description of LSL Synchronization

1: Calculate Clock Offsets

Record clock offsets peridiocally during data acquisition usingclock filter algorithm (NTP) using get time of day to recordsets of 4 timestamps in rapid succession:

send from inlet to outlet (t0)receive from inlet at outlet (t1)immediately send from outlet to inlet (t3)

receove from outlet at inlet (t4)

round trip time (RTT) = (t3-t0) - (t2-t1)

clock offset (OFS) = ((t1-t0) + (t2-t3))/2 (for lowest RTT)

Medine LabStreamingLayer Overivew

Page 71: LabStreamingLayer Overivew

3. A Brief Description of LSL Synchronization

1: Calculate Clock Offsets

Record clock offsets peridiocally during data acquisition usingclock filter algorithm (NTP) using get time of day to recordsets of 4 timestamps in rapid succession:

send from inlet to outlet (t0)receive from inlet at outlet (t1)immediately send from outlet to inlet (t3)receove from outlet at inlet (t4)

round trip time (RTT) = (t3-t0) - (t2-t1)

clock offset (OFS) = ((t1-t0) + (t2-t3))/2 (for lowest RTT)

Medine LabStreamingLayer Overivew

Page 72: LabStreamingLayer Overivew

3. A Brief Description of LSL Synchronization

1: Calculate Clock Offsets

Record clock offsets peridiocally during data acquisition usingclock filter algorithm (NTP) using get time of day to recordsets of 4 timestamps in rapid succession:

send from inlet to outlet (t0)receive from inlet at outlet (t1)immediately send from outlet to inlet (t3)receove from outlet at inlet (t4)

round trip time (RTT) = (t3-t0) - (t2-t1)

clock offset (OFS) = ((t1-t0) + (t2-t3))/2 (for lowest RTT)

Medine LabStreamingLayer Overivew

Page 73: LabStreamingLayer Overivew

3. A Brief Description of LSL Synchronization

1: Calculate Clock Offsets

Record clock offsets peridiocally during data acquisition usingclock filter algorithm (NTP) using get time of day to recordsets of 4 timestamps in rapid succession:

send from inlet to outlet (t0)receive from inlet at outlet (t1)immediately send from outlet to inlet (t3)receove from outlet at inlet (t4)

round trip time (RTT) = (t3-t0) - (t2-t1)

clock offset (OFS) = ((t1-t0) + (t2-t3))/2 (for lowest RTT)

Medine LabStreamingLayer Overivew

Page 74: LabStreamingLayer Overivew

3. A Brief Description of LSL Synchronization

2: Map drifting clock values and fit

This is normally done post-hoc in load xdf.m using a fittingprocedure. Each map is calculated using an ADMM methodincorporating the Huber loss function(http://www.stanford.edu/~boyd/papers/distr_opt_stat_learning_admm.html)

Each map is a DC offset and a slope adjustment(yn = axn + b) for each intermittent OFS record point(default is 5s between queries).

The latest version of LSL has methods for doing this online,but it is not yet validated

Medine LabStreamingLayer Overivew

Page 75: LabStreamingLayer Overivew

3. A Brief Description of LSL Synchronization

2: Map drifting clock values and fit

This is normally done post-hoc in load xdf.m using a fittingprocedure. Each map is calculated using an ADMM methodincorporating the Huber loss function(http://www.stanford.edu/~boyd/papers/distr_opt_stat_learning_admm.html)

Each map is a DC offset and a slope adjustment(yn = axn + b) for each intermittent OFS record point(default is 5s between queries).

The latest version of LSL has methods for doing this online,but it is not yet validated

Medine LabStreamingLayer Overivew

Page 76: LabStreamingLayer Overivew

3. A Brief Description of LSL Synchronization

2: Map drifting clock values and fit

This is normally done post-hoc in load xdf.m using a fittingprocedure. Each map is calculated using an ADMM methodincorporating the Huber loss function(http://www.stanford.edu/~boyd/papers/distr_opt_stat_learning_admm.html)

Each map is a DC offset and a slope adjustment(yn = axn + b) for each intermittent OFS record point(default is 5s between queries).

The latest version of LSL has methods for doing this online,but it is not yet validated

Medine LabStreamingLayer Overivew

Page 77: LabStreamingLayer Overivew

3. A Brief Description of LSL Synchronization

3: Linearize/De-Jitter the timestamps (if appropriate)

Simple linear regression is very robust:

Medine LabStreamingLayer Overivew

This will fail if thesampling rate changes!

Page 78: LabStreamingLayer Overivew

3. A Brief Description of LSL Synchronization

3: Linearize/De-Jitter the timestamps (if appropriate)

Simple linear regression is very robust:

Medine LabStreamingLayer Overivew

This will fail if thesampling rate changes!

Page 79: LabStreamingLayer Overivew

3. A Brief Description of LSL Synchronization

Medine LabStreamingLayer Overivew

This is bad:

Red line is linearized timestamps, blue is raw. On the right is thedifference: between +/-150s !!! .

Page 80: LabStreamingLayer Overivew

3. A Brief Description of LSL Synchronization

Medine LabStreamingLayer Overivew

This is bad:

Red line is linearized timestamps, blue is raw. On the right is thedifference: between +/-150s !!! .

Page 81: LabStreamingLayer Overivew

3. A Brief Description of LSL Synchronization

Medine LabStreamingLayer Overivew

This is bad:

Red line is linearized timestamps, blue is raw. On the right is thedifference: between +/-150s !!! .

Page 82: LabStreamingLayer Overivew

3. A Brief Description of LSL Synchronization

Medine LabStreamingLayer Overivew

Determine device Lag:

To complete thesynchronization, simplysubtract the mean.

Page 83: LabStreamingLayer Overivew

3. A Brief Description of LSL Synchronization

Medine LabStreamingLayer Overivew

Determine device Lag:

To complete thesynchronization, simplysubtract the mean.

Page 84: LabStreamingLayer Overivew

3. A Brief Description of LSL Synchronization

Medine LabStreamingLayer Overivew

Determine device Lag:

To complete thesynchronization, simplysubtract the mean.

Page 85: LabStreamingLayer Overivew

3. A Brief Description of LSL Synchronization

Medine LabStreamingLayer Overivew

Page 86: LabStreamingLayer Overivew

Thank You!

Medine LabStreamingLayer Overivew