Configure and Troubleshoot CMS live streaming with · PDF fileConfigure and Troubleshoot CMS...

Configure and Troubleshoot CMS livestreaming with VBrick DME Contents

IntroductionPrerequisitesRequirementsComponents UsedBackground InformationConfigureNetwork DiagramConfigurationsVerifyTroubleshoot

Introduction

This document describes the steps to configure and troubleshoot Cisco Meeting Server (CMS)integration with VBrick Distributed Media Engine (DME). CMS integration with VBrick has beenadded from version 2.1 and up.

Contributed by Christian Ruiz and Andrea Sancho, Cisco TAC Engineers.

Prerequisites

Requirements

CMS 2.1 and above with 'Recorder' and/or 'Streamer' license(s). (One recorder license willallow you to stream one call)

●

VBrick DME●

VBrick Rev (optional, this is only required if Live Streaming needs to be shared outside theinternal network)

●

Components Used

Software and hardware versions:

1. CMS 2.12. VBrick DME 3.15.0 Rhel7

Tip: Cisco recommends that the CMS VM acting as the streamer should be sized with 1vCPU and 1GB of memory per 6 concurrent streams, with a minimum of 4vCPUs and amaximum of 32vCPUs

The information in this document was created from the devices in a specific lab environment. All ofthe devices used in here started with cleared (default) configurations. If your network is live, makesure that you understand the potential impact of any command.

Background Information

CMS 2.1 and up now support live streaming (webcast) using standard RTMP therefore this featurewill work with all streaming platforms that follow the standard such as YouTube, Facebook,Wowza, etc., however Cisco TAC will only support CMS Live Streaming (Webcast) integration withVBrick DME server. At this moment CMS does not support RTMPS, therefore all traffic betweenthe Streamer and the external server will be unencrypted.

Live Streaming (Webcast) integration with VBrick DME allow users to watch any live streamedCMS conference anywhere inside the network from different devices, additionally when VBrickRev is used along VBrick DME this extends this capability for viewing from outside the internalnetwork for every VBrick Rev authorized user.

Configure

Network Diagram

There are several scenarios supported to deploy Live Streaming with CMS such as: singlecallbridge with multiple streaming servers, a callbridge cluster with a single streaming server andcallbridge cluster with multiple streaming servers. For the purpose of this document it was usedthe most basic deployment with a single callbridge connecting to a single streaming server as allthe configuration steps with this scenario apply to the other scenarios too.

Server A: CMS server with Callbridge and XMPP configured.

Server B: CMS server that will act as the Streamer server.

Note: The CMS server hosting the callbridge is the one that needs to have the StreamingLicense installed and not the CMS server acting as the Streamer server.

Configurations

To begin this configuration it is assumed that you already have a CMS server with a working CallBridge and XMPP server. This is because the streamer server acts as an XMPP client, so theXMPP server needs to be enabled and completely configured on the CMS hosting thecallbridge. Please look at the Troubleshoot section of this document to find common errormessages received when streaming is not working due to XMPP not correctly configured.

Caution: If XMPP server is not correctly configured, stream will not work. XMPP needs to beenabled and completely configured including SRV or DNS resource records (RRs).

1. Certificates: As with all other CMS servers, the streamer server needs to have a valid internalCA signed certificate.

1a Create the files using the 'pki csr' command:

streamer.chrruiz.lab> pki csr streamer CN:streamer.chrruiz.lab O:CHRRUIZLAB

subjectAltName:chrruiz.lab,acano.chrruiz.lab,acano1.chrruiz.lab

1b Retrieve the files using SFTP client:

1c Sign and issue the certificate with your internal local authority, in this example an ADserver:

1d Upload the signed certificate and the callbridge trust bundle certificate to the streamerserver using SFTP:

2.- SSH configuration.

2a Configure interface(s) for the streamer to listen, in this case it was configured interface 'a'only to listen on port 8443.

streamer.chrruiz.lab> streamer listen a:8443

2b Define certificates for the streamer server.

streamer.chrruiz.lab> streamer certs streamer.key streamer.crt

2c Trust the CallBridge certificate bundle.

streamer.chrruiz.lab> streamer trust callbridge.crt

2d Verify that the information entered in the steps above is correct, use the 'streamer'command.

streamer.chrruiz.lab> streamer

Enabled : false

Interface whitelist : a:8443

Key file : streamer.key

Certificate file : streamer.crt

Trust bundle : callbridge.crt

2e If everything shows correct, you can proceed and enable the streamer with the command'streamer enable'.

streamer.chrruiz.lab> streamer enable

3. DNS A record.

3a The DNS A record for the streamer needs to resolve to the IP Address of the Ethernetinterface configured in 2a.

4. API configuration, this configuration is performed in the CMS hosting the CallBridge. You canuse any REST client of your choice to perform this, i.e. POSTMan. For the purpose of thisdocument it was utilized Insomnia REST client: (https://insomnia.rest/)

4a Add the streamer to /streamers, using the HTTPS 'URL' of the streamer server.

https://insomnia.rest/

4b Verify streamer was added using a GET in /streamers.

4c Add the VBrick 'streamURL' to the space(s) that will be used for streaming.

In this example a space called 'astream' was created using the CMS web interface.

The space ID (89d76835-40d6-4d1b-86f5-1b9a2d89041f) was used to PUT the'streamURL' in the following format:

rtmp://<VBrickBroadcastUsername>:<VBrickBroadcastPassword>@<VBrick IP orFQDN>/live/NameoftheStream.

Note: The default username and password for VBrick DME Broadcast is: broadcast /broadcast. Please go to the 'Troubleshoot' section of this document if you have issuessetting up this streamURL.

4d Verify streamURL was added correctly using a GET in /coSpaces/<coSpaceID>

4e Configure 'streamingMode' in the callProfile. Options are automatic, manual or disabled.For this example it was configured for 'Automatic' using the PUT in the callProfile ID(4e5f957c-752d-4456-849c-e594ed792769)

4f Verify 'streamingMode' was added correctly using a GET in /callProfiles/<callProfileID>

4g Verify this callProfile id shows also in /system/profiles, if it is not showing there, streamingwill not work, add it using a PUT if necessary.

4h The parameter 'streamingControlAllowed' in the /callLegProfiles/<callLegProfileid> will giveyou the option to set it to 'true' or 'false' to control users permissions to stream. By default isset to true.

4i If 'manual' option was selected for 'streamingMode' in step 4e, then dtmfProfiles need to beconfigured for starting and stopping streaming. Go to /dtmfProfiles and use the'startStreaming' and 'stopStreaming' parameters to define the DTMF tones to start and stopthe streaming. In this example we used the DTMF profile ID (fd03e044-3079-4e93-8c8a-2fc482623608) to PUT the parameters.

4j Verify the DTMF profile settings were added succesfully using a GET/dtmfProfiles/<dtmfProfileID>

Verify

Use this section in order to confirm that your configuration works properly.

1. CMS event log; In the CMS hosting the callbridge web interface, check that the streamingshows available and streaming, in this example as the streaming is set to automatic you cansee the streaming starts right after the user joins the 'astream' space

2017-05-16 19:11:06.422 Info participant "[email protected]" joined space

89d76835-40d6-4d1b-86f5-1b9a2d89041f (astream's Space)

2017-05-16 19:11:12.434 Info streaming device 2: available (1 streamings)

2. If using a CMA client or WebBridge, you will see an additional 'participant' called'streaming' in the conference. If not using CMA client or WebBridge, please proceed to step2a so you can check it via API.

2a. A GET to /calls/<callid> will show you streaming=true if the call is being streamed, and willalso show one additional participant in the number of participants in the call. In this example itshows two particpants as there was only user 'Christian Ruiz' and 'streaming' in the call.

Tip: If streaming is showing 'true' but the additional participant is not showing, this is mostlikely a XMPP issue where the 'streaming' client is having issues to communicate with theXMPP server. Please refer to the Troubleshoot section of this document to check mostcommon XMPP configuration issues.

3. VBrick DME web interface: Go to Monitor and Logs > Multi-Protocol Connections andcheck that you can see the stream there.

4. Catch the stream: Use the information provided in DME web interface to catch the streamusing a streaming player like VLC media player (http://www.videolan.org/vlc/) to confirm audioand video are working correctly.

http://www.videolan.org/vlc/

Troubleshoot

'Syslog Follow'

Always run the 'syslog follow' command on the streamer server, you should be able to seevery important information and error messages that will help you to know where to start yourtroubleshooting, here is an example of a succesful stream with no error messages shown:

May 8 23:31:51 user.info streamer streamer.1d1a2dda-29a8-47e4-bca1-7cc2c604fdf3[10]:

Connecting to '10.88.246.108', app 'live', stream 'CMSAutomaticStream', port '1935', scheme

'rtmp'

May 8 23:31:51 user.info streamer streamer.1d1a2dda-29a8-47e4-bca1-7cc2c604fdf3[10]: Set

sending chunk size to 4096

May 8 23:31:51 user.info streamer streamer.1d1a2dda-29a8-47e4-bca1-7cc2c604fdf3[10]: Server

window size now set to 16777216

May 8 23:31:51 user.info streamer streamer.1d1a2dda-29a8-47e4-bca1-7cc2c604fdf3[10]: Set peer

bandwidth received (size=2500000, type=2)


Acknowledged window size 2500000

May 8 23:31:51 user.info streamer streamer.1d1a2dda-29a8-47e4-bca1-7cc2c604fdf3[10]: Stream

begin 0


authmod=adobe successful

May 8 23:31:51 user.info streamer streamer.1d1a2dda-29a8-47e4-bca1-7cc2c604fdf3[10]: Ignored

command message 'onBWDone' (['onBWDone', 0.0, None, 8192.0])


unexpected command message (['_result', 2.0, None, None])


unexpected command message (['_result', 3.0, None, None])


command message 'onFCPublish' (['onFCPublish', 0.0, None, {'description': 'CMSAutomaticStream',

'code': 'NetStream.Publish.Start'}])


Successfully created stream with stream id 1

May 8 23:31:51 user.info streamer streamer.1d1a2dda-29a8-47e4-bca1-7cc2c604fdf3[10]: new

status: streaming


Successfully published stream to RTMP server

May 8 23:31:52 user.info streamer streamer.1d1a2dda-29a8-47e4-bca1-7cc2c604fdf3[10]: Client

connected

May 8 23:31:52 user.info streamer streamer.1d1a2dda-29a8-47e4-bca1-7cc2c604fdf3[10]: Call

found

Packet captures:

Packet captures from CMS hosting the callbridge, Streamer and DME will help you in most ofthe issues related to communication. They will be very important to troubleshoot the errormessages:

- "Connecting to RTMP server failed (Timeout)"

- "Initiating RTMP protocol failed (connection closed by far end)"

To take packet captures in:

CMS: Use the 'pcap' command.

DME: Use the web interface in the Diagnostics > Trace Capture, press the 'Start capture'button. Press the 'Stop capture' button to stop the tracing. Press the 'Download trace file' todownload the packet capture.

streamURL configuration issues:

The most common issue is that the Stream Input Authentication username and/or password isincorrect. Verify you are using the correct credentials, Using the VBrick DME web interface,Navigate to User Configuration --> Stream Input Authentication and check you are using thecorrect username andpassword.

Authentication issues against VBrick Stream Input Authentication username and/or password(broadcast user).

1. When using an incomplete format with no user or password, i.e.

rtmp://[email protected]/live/CMSAutomaticStream you will see:

May 26 02:08:43 user.info streamer streamer.bd052ae2-6501-4ae4-ab78-5b94c9a21717[305]:


'rtmp'

May 26 02:08:43 user.info streamer streamer.bd052ae2-6501-4ae4-ab78-5b94c9a21717[305]: Set


May 26 02:08:43 user.info streamer streamer.bd052ae2-6501-4ae4-ab78-5b94c9a21717[305]: Starting

authmod=adobe

May 26 02:08:43 user.err streamer streamer.bd052ae2-6501-4ae4-ab78-5b94c9a21717[305]: No

username or password defined for RTMP authentication

2. When user/password are incorrect, i.ertmp://broadcast:[email protected]/live/CMSAutomaticStream, you will see:

May 26 02:05:16 user.info streamer streamer.5fff36f0-e56d-4d02-9e5e-431b0fba130c[284]:


'rtmp'

May 26 02:05:16 user.info streamer streamer.5fff36f0-e56d-4d02-9e5e-431b0fba130c[284]: Set


May 26 02:05:16 user.err streamer streamer.5fff36f0-e56d-4d02-9e5e-431b0fba130c[284]: RTMP

authentication failed (['_error', 1.0, None, {'description': '[ AccessManager.Reject ] : [

authmod=adobe ] : ?reason=authfailed&opaque=vgoAAA==', 'level': 'error', 'code':

'NetConnection.Connect.Rejected'}])

Additional streamURL related error messages:

- "RTMP stream url has a bad format" - "Connecting to RTMP server failed ([Errno -2] Name or service not known)"

Solutions:

For both error messages, please verify that the streamURL follows exactly this format:rtmp://<VBrickBroadcastUsername>:<VBrickBroadcastPassword>@<VBrick IP orFQDN>/live/NameoftheStream/

1.

Verify that VBrick IP or hostname is resolvable from the streamer server.2.XMPP related issues:

XMPP is required to be enabled, working correctly and completely configured in order forstreaming to work. This includes having correct SRV records or RRs. If they are not configured'streaming' client will not be able to connect to stream. You will see the error message:

May 23 16:20:19 user.err streamer streamer.af28cb0c-08d3-4692-b9e6 Client connect failed

May 23 16:20:19 user.info streamer streamer.af28cb0c-08d3-4692-b9e6 new status: disconnecting

May 23 16:20:19 user.err streamer streamer[1]: Bot af28cb0c-08d3-4692-b9e6-36d7b5b7e149 failed:

CLIENT_CONNECT_FAILED

Solution: Check and correct SRV records or RRs.

Other error messages:

1. "streamerUnavailable"

Error message: "Streamer 'YourStreamerIP' unavailable."

Possible causes: Wrong port was set, port duplicated, port blocked. Streamer server down.

Solution: Verify correct port is configured, that is not in use by other service as 'Recording' andthat is not being blocked between servers. Restart CMS server hosting the callbridge.

Screenshots and logs: The web interface will show the message:

DateTimeFault condition

2017-05-1618:32:12.391Streamer "https://127.0.0.1:8445" unavailable (connect failure)

The same message appears when running a GET to /system/alarms:

<?xml version="1.0"?>

<alarms total="1">

<alarm id="08dd26d9-6c6f-4da2-befb-8ea7d642bd6d">

<type>streamerUnavailable</type>

<activeTimeSeconds>5846</activeTimeSeconds>

</alarm>

</alarms>

2. "streamingLimitReached"

Error message: "start streaming failed: streaming limit reached"

Cause: No enough licenses to stream.

Solution: Verify 'streaming' license(s) is/are installed in the CMS hosting the callbridge and notin the CMS streamer.

Configure and Troubleshoot CMS live streaming with · PDF fileConfigure and Troubleshoot CMS...

Documents

Transcript of Configure and Troubleshoot CMS live streaming with · PDF fileConfigure and Troubleshoot CMS...