VoIP on the iPhone: Imagine the PossibilitiesJan Linden, VP of Engineering

VoIP on the iPhoneOutline

• Market trends• Technology trends• Mobile VoIP challenges• Solutions and recommendations

Evolution of IP Voice andVideo Processing

Solutions Designed Specifically for VoIP

Advanced Mobile and Multimedia Capabilities

PSTN Solutions Adapted for VoIP

Mobile Market Trends

• Fixed Mobile Convergence– Mobile phone increasingly perceived as a computer

• Advanced operating systems such as Apple OSX are making this a reality

– Users expect to access the same applications available across multiple platforms

• Increased accessibility and affordability– Network operators are introducing new pricing models to

encourage uptake of data driven services and applications• IP networks enable more cost effective transport

– Handset prices declining• iPhone 3G half the price of 1st Generation

Mobile Technology Trends

• Migration from legacy cellular to next-gen networks– Allows for faster data connection

• Growth of Smartphone adoption– iPhone and other dual-mode phones provide greater flexibility in

connecting to available networks– All-in-one devices able to run multiple applications efficiently

• Consumers can place calls and receive data simultaneously

• Development of 3rd Party Applications– Operating systems, like the Apple OS X, more conducive to

advanced 3rd party applications – VoIP over mobile in high demand

The iPhone as a VoIP Platform• iPhone most VoIP friendly phone on market…

– Open APIs for easy development– Sufficient CPU resources– Support of wideband codecs– Intuitive UI makes for simple navigation between applications– Demand for 3rd party apps is higher than other phones

• …but strengths present unique challenges– Running too many applications can drain resources– Usage scenarios can negatively impact voice quality

With the right voice processing software, developing VoIP applications is a cinch!

Technology Challenges – Network Connections• IP Networks

– Latency– Jitter – Packet loss

• Wireless connections– Low signal strength– Interference from users in

same or neighboring frequencies

– Available bandwidth varies– WiFi access point

bottlenecks

Technology Challenges

• Device Limitations– Limited processing power– Battery life– Data intensive applications can

cause jitter

• Mobile Environments– Tend to have excessive

background noise– Hands Free operation

enhances acoustic echo

Impact of IP Networks

Delay

Packet Loss

Network Jitter

Impact of IP Networks – Latency

• Major effect is “stepping on each other’s talk”

• Usage scenario affects annoyance factor – higher delay can be tolerated for mobile devices

• Long delays make echo more annoying

1011100

0101101

1111010

001100111

Sources of Latency

• Codec• Capture• Playout• Network delay• Jitter buffer• OS interaction• Transcoding

A/DPre-processing

Speech encoding

IP interface

D/APost-processing

Speech decoding

Jitter buffer

IP Network

A/DPre-Processing

Speech Encoding

IP Interface

D/APost-Processing

Speech Decoding

JitterButter

IP Network

Impact of Latency on Voice Quality

• ITU-T (G.114) recommends:– Less than 150 ms one-way delay for most applications (up to 400

ms acceptable in special cases)• Users have become accustomed to longer delays

– Still, low delay very important for high quality

Data from ITU-T G.114

1

2

3

4

0 250 500 750

One-way transmission time [ms]

Mea

n O

pin

ion

Sco

re

Impact of IP Networks – Packet Loss and Jitter

• Packet Loss– Occurs due to flushed buffers in network nodes– Same effect if packets are too late to be used– Smooth concealment necessary

• Network Jitter– Transmission time differs for each packet– Jitter buffer necessary to ensure continuous playout– Trade-off between delay and quality

Technology Challenges – Mobile Environments

• Background Noise– Environments inherently noisy– Usage scenarios can make

differentiating between speech and noise difficult

• Acoustic Echo– Speakerphone operation common for iPhones– Micropohone and speaker close– High delay in VoIP makes echo more prominent

Technology Challenges – Devices• Limited Resources

– Size limits processing power– Battery life also limited

• VoIP Friendliness– Many smartphones limit access to

soundcard– Device buffers add latency– Clock drift

• Too much of a Good Thing– Running too many applications can drain

resources– Multiple applications accessing network

can result in jitter

VoIP Design Considerations

QualityCost

Speech Quality

Time to Market

Flexibility

Ease of Use

Network ImpairmentsPower Consumption

Device Considerations

Cost

Signaling

Features

Infrastructure

Coping with Network Degredation

Power Consumption

Hardware Issues (Processor, OS, Acoustics, etc.)

Echo Cancellation

Additional Voice Processing Components

Environment – Background Noise, Room Acoustics,

etc.

Speech Codec

VoIP DesignChallenges

Network

Codec

Hardware

Echo Power

Voice Environment

Mobile VoIP Design Considerations

Speech Codec

Speech Codec

Packet-loss Robustness

Memory

Sampling Rate

Complexity

Delay

Bit-rate

Quality

Complexity

Bit-rate

• Many conflicting parameters affect choice of codec

• Determines upper limit of quality

• Support of several codecs necessary

– Interoperability

– Usage scenario

• IPR issues a significant concern

Input Signal Robutness

Audio Bandwidth Effect on Quality

• Better than PSTN quality is achievable in VoIP– Utilizing full 0 – 4 kHz band in narrowband– Wideband coding offers more natural and crispier voice

Frequency

4 kHz 6.4-7.0 kHz

GIPS Narrowband Speech

Std Wideband Speech(e.g. G.722.2)

CD -Audio

Std Narrowband Speech Implementation (PSTN)

200 - 3400 Hz

GIPS Wideband Speech

3,4 22.1 kHz8 kHz

Quality

iPhone Specific Challenges

• Applications cannot run in the background• Audio HAL APIs (regular APIs) not available on iPhone. (As on

Mac.) Very iPhone specific solution had to be designed• Must consider change of audio route (e.g. headset in/out) and

handle this• Wideband audio not available via built-in mic

Recommendations

• IP Networks– No need for dedicated bandwidth if solution includes high

performance:• Bandwidth management• Robust codec framework

– Low latency achieved by efficient jitter buffer

• Audio– High quality audio is crucial

• Users expect the same quality they experience on the PC

• Optimized Solutions– Efficiency for ARM processors on iPhones– Reduce CPU constraint and increase battery life

Recommendations

• Adaptive Jitter Buffer and PLC– Manage up to 30% packet loss– Adapt to network busts and excessive mobile jitter

• Codecs Designed for IP Networks– Work in tandem with jitter buffer to overcome packet loss– Can deliver Wideband experience– Robust and bit-rate adaptive codecs necessary

• Efficient use of the total available bandwidth• GIPS iLBC already in the iPhone, GIPS iSAC and other wideband

codecs provide even better user experience

• Aggressive Voice Quality Enhancement– Detect and cancel background noise– Suppress acoustic echo

Recommendations

• VoiceEngine Mobile for iPhone– Comprehensive package of voice processing components

– Handles delay, jitter, packet loss and mobile environments– Optimized for iPhone and Smartphones

– Efficient solution reduces CPU demands– Flexible and easy to integrate