QoE Software and Hardware Packaging
Transcript of QoE Software and Hardware Packaging
QoE Software and Hardware Packaging
Jesse Frey
John Marts
Peter Fink
#PSCR2021
DISCLAIMER
Certain commercial entities, equipment, or materials may be identified in this document in order to describe an experimental procedure or concept adequately.
Such identification is not intended to imply recommendation or endorsement by the National Institute of Standards and Technology, nor is it intended to imply that the entities, materials, or equipment are necessarily the best available for the purpose.
* Please note, unless mentioned in reference to a NIST Publication, all information and data presented is preliminary/in-progress and subject to change
#PSCR2021
INTRODUCTION
Jesse FreyJohn MartsPeter Fink
4
Outline
• Background
• Motivation
• Hardware improvements
• Software improvements
• Current status
5
BACKGROUND : OVERVIEW
• Measurement system for quality of experience (QoE)
• QoE key performance indicators (KPIs) for mission critical voice (MCV)• Mouth-to-Ear Latency
• Access Time
• Voice Quality/Intelligibility
• Probability of Successful Delivery
• More information :• MCV QoE Measurement Methods Overview presentation
• Mouth-to-Ear Latency Measurement System paper (NIST IR 8206)
• Access Time Measurement System paper (NIST IR 8328)
6
BACKGROUND : MEASUREMENT CONCEPT
Communications System
KPIs:• Mouth-to-Ear Latency• Access Time• Intelligibility• Probability of
Successful Delivery
7
THE USER EXPERIENCE: PTT COMMUNICATIONS
• Press the Push To Talk (PTT) button and speak into a device
• Listening to speech output from a device
• It’s all about speech
• Goal - Create measurement systems that are: • Based upon the user experience -- speech
• Comparable and fair across technologies
• This is not:• Analyzing internal system design/construction
8
MEASUREMENT SYSTEM
9
HARDWARE ISSUES
• Radio interface• Development board
• Perfboard and parts
• Not plug and play
• Radio cables• Standard headsets
• Modified with connectors
• Also not plug and play
10
SOFTWARE ISSUES
• Original software written in MATLAB• Excellent libraries and support
• Great for initial development
• MATLAB is expensive and less common
• Code re-use across projects is difficult
11
HARDWARE IMPROVEMENTS
• Radio interface• Create a Printed Circuit Board (PCB) design
• Might be purchasable in the future
• Radio cables• Rework hardware to use interoperability cables
• Available for purchase, ready to go
12
NEW HARDWARE
• Same MSP430F5529 processor• All input/output connections are the
same
• Same firmware
• Has 8P8C connectors for radio cables
• Includes transformers for audio• Used to be external
13
RADIO CABLES
• Standard cables for interoperability gateway
• Use 8P8C connector
• Mostly for LMR radios• Working on getting cables for LTE
PTT phones
14
NEW SOFTWARE
• New software written in Python• Popular
• Used in many major software applications
• Supported on a wide variety of platforms
• Common code is placed in packages• General common measurement functions : mcvqoe
• Intelligibility estimator : abcmrt
• Measurement code written as classes• Allows for re-use
• Also allows for simulations
15
PYTHON MODULES
• Packages all code and data together• Tracks dependencies so all needed packages can be installed
• Can be uploaded to the Python package index (PyPI)• Easy to download and install measurement code and dependencies with a single
command
• Packages will be created for measurement system code• Published to PyPI
• Common code shared across measurements• No more duplicated code for each measurement
16
MEASUREMENT CLASSES
• Each measurement has a class
• Settings for test• Used for logging
• Allows for code reuse during reprocessing
• Allows for simulations with test code
• Easy validation of measurement methods
17
STATUS
• Python code • In the final testing stages
• Should be published by the end of 2021
• First Hardware Revision• Complete
• Also needs testing
• Future hardware technology transfer options
18