mediasoupPowerful WebRTC SFU for Node.jsIaki Baz Castillo

something about itwhat is mediasoup?A WebRTC SFU Selective Forwarding UnitHandles the media layerDoesnt mix audio/video streamsA multi-party video solution for Node.jsNot a standalone media serverA server-side Node.js moduleJavaScript ES6 API (core written in C++)

topologiesmulti-party video conferencing

TopologiesFull MeshEach participant sends his audio and video to all other participantsEach participant receives all the streams from all other participantsNo media server neededLow latencyLot of encodings in each participantHigh uplink bandwidth required

TopologiesMCU: Multipoint Control UnitEach participant sends his streams to a media serverThe server mixes all the streams and composes a single one Example: Asterisk meetmeSimple at client side: single audio/video mixed stream from the serverInteroperability: the server can transcodeLow bandwidth requiredCPU expensive decoding/encoding in server side (high latency)Non flexible client side applications

TopologiesSFU: Selective Forwarding UnitParticipant sends his streams to a media serverThe media server routes the streams to all other participantsEach participant receives many streamsHigh throughput, low latencyLow CPU at server side (no decoding/encoding)Good uplink bandwidth usageThe client side application can render each remote video stream as desiredSimulcast/SVC required for real scenarios

The Man in THE MOUNTAINsSimulcast / SVC

A multi-party family videoconference

ManuAt parents houseAntisystembut owns an iPhone 6Excellent Wi-Fi connection

BeatrizAt homeConservativeMacBook AirExcellent Wi-Fi connection

RobertoIn the mountainsUnknown ideologyHTC Desire (2010)Poor 3G connection

A multi-party family videoconferenceThe problemRoberto has not a good bandwidthRoberto can not receive HD video from Manu and Beatriz

ough !

A multi-party family videoconferencePOSSIBLE SOLUTIONSDrop Roberto from the conferenceLimit the video quality of Manu and Beatriz

A multi-party family videoconferenceSolution: SIMULCAST / SVCManu and Beatriz send different quality video layersThe SFU chooses the appropriate ones for Roberto

TopologiesFULL MESHMCUSFUClient uplinkVery highLowLowClient downlinkVery highLowHighClient CPU usageVery highLowMediumServer CPU usage-Very highVery lowLatencyNoneHighLowCan transcode-YesNoRequires simulcast/SVC--YesFlexible video layoutYesNoYes

mediasoup is an sfuYES!

mediasoup is minimalist

mediasoup is minimalistWhat is *not* mediasoup?It is NOT a standalone serverIt does NOT have init scripts for Debian or CentOSIt does NOT have a config fileIt does NOT implement the SIP protocolIt does NOT implement ANY signaling protocolIt does NOT provide an admin web interfaceIt does NOT provide a client SDK

mediasoup is a Node.js moduleYours trulyRemember

so what?ok

$ npm install save mediasoupmediasoup & Node.jsA Node.js moduleA Node.js module is a dependency/library within a projectYou create your Node.js based project/applicationYou add mediasoup as a module into itAt the end, mediasoup is yet another dependency entry in the package.json of your Node.js application

mediasoup & Node.jsa package.json example

{ "name": "my-amazing-multiconference-server-app", "version": "1.0.0", "description": "Enterprise conferencing application", "main": "index.js", "author": "My Great Company", "license": "UNLICENSED", "dependencies": { "express": "^4.14.0", "mediasoup": "^1.0.0", "socket.io": "^1.5.0" }}

Why Node.js?Ask them

mediasoup exposes a JavaScript es6 api

// Load mediasoup moduleconst mediasoup = require('mediasoup');

// Create a mediasoup Serverlet server = mediasoup.Server();

// Options for the mediasoup Roomconst roomOptions = {mediaCodecs: [{kind : 'audio',name : 'audio/opus',clockRate : 48000},{kind : 'video',name : 'video/vp8',clockRate : 90000}]};

// Create a mediasoup Roomserver.createRoom(roomOptions).then((room) => {// Got the Room instancehandleRoom(room);});

mediasoup APILow level APImediasoup exposes a low level API similar to ORTC// Create a Peerlet peer = room.Peer('alice');

// Create a ICE+DTLS Transportpeer.createTransport(options).then((transport) => {transport.setRemoteDtlsParameters(data);});

// Create a RtpReceiver to handle audio from the browserlet audioReceiver = peer.RtpReceiver('audio', transport);

// Create a RtpReceiver to handle video from the browserlet videoReceiver = peer.RtpReceiver('video', transport);

mediasoup APIHigh level APImediasoup also exposes a high level API similar to WebRTC 1.0// Create a PeerConnectionlet peerconnection = new mediasoup.webrtc.RTCPeerConnection(room, 'alice');

// Set the remote SDP offerpeerconnection.setRemoteDescription(desc).then(() => {return peerconnection.createAnswer();}).then((desc) => {return peerconnection.setLocalDescription(desc);}).then(() => {// Answer the participant request with the SDP answerrequest.accept({sdp: peerconnection.localDescription.sdp});});

mediasoup JUST forwards MEDIARememberSincerely yours

mediasoup APIJust MEDIAmediasoup does NOT talk the SIP protocolnor it talks ANY signaling protocolYou can use socket.io (for example) to communicate with browsers/endpoints via WebSocketor build your own protocolSimilar to the WebRTC API for browsers, mediasoup just handles the media layer

ROADMAP

mediasoup roadmapWorking on it1.0.0RTCP: process RTCP reports to allow mediasoup diagnose per peer uplink/downlink issuesWebRTC 1.0: more API needed2.0.0Simulcast & SVC: handle multiple streams/layers from clients and select which one to route to others

Your turn

The Application

The Application

Ask yourselfWant to build yet another boring enterprise conferencing app?May be a funny social app?Will you build the app on Node.js?Browser app? mobile app? native Android/iOS SDKs?Do you need interoperability with PSTN? Really?

going forwardWebRTC STATUS in browsers

Lets demo !!!

Demo applicationFIRSTSIGHTSee many people, only talk to oneBackendNode.js server running mediasoup and websocket modulesJS logic to handle media rooms and manage participantsFrontendHTML5 application made with React.jsJS logic to handle MediaStream from room participants

MUCHAS GRACIASIaki Baz CastilloTHE ENDhttp://mediasoup.org