What the bleep!

A serious of unfortunate screeches

Misadventures in sound processing on Android.

28 October 2019

Please ask questions as we go along.

Fair warning: I seldom know what I’m talking about, do your own research.

Pro Tip: Sometimes the sound is loud, be louder.

W A R N I N G

Never have you seen a cheesier presentation.

Some of the demos have no visuals.

It’s just noise.

Some of the visuals in this presentation are, indeed, from 1983.

None of the code you will see follows any kind of best practice. Things like permissions etc. are just casually glossed over. Please don’t try this at home. At least, do not use your own phone.

Topics

• What.• Why.• How.• Resources.• Assorted.• Other.• Miscellaneous.• Terrible Jokes.• Lies.• Alternate truths.

This talk contains language and references that are rated PG-99, or stronger.

This slide has been recycled from some other stupid talk.

• Terrible freelance developer, probably looking for work.

• Has no musical ability, at all. Scored 0% for it. Be warned.

• Has zero presentation skills. Sucks for you.

• Works remotely because I like my dog. A lot.

• Tweets can go to https://twitter.com/EwaldHorn

• Help pay for dog food: https://www.nofuss.co.za/

What?• We’ll take a look at sound on Android.

• How to play some.

• How to record some.

• How to play some we recorded.

• How to mess with it.

• There’ll be demos. Live. Which will fail. Often. Meh.

What is Sound?• Noise. It’s noisy, can be noisy and should be loud. Louder

is better. Ask anyone with a R20k car and a R70k sound system, they know.

• We hear in analogue, stupid computers can only do digital.

• Go from analogue to digital via sampling. If you think you lose data via sampling, well done, you get this.

• If you want the truth, see https://www.merriam-webster.com/dictionary/sound

How does it work?

• Some thingie or the other in your phone reacts to input from the microphone, does some stuff and writes it out to a file. This is the digital presentation of sound. Sort of.

• This is interesting and more accurate : https://en.wikipedia.org/wiki/Sampling_(signal_processing)

• Files saved in various formats, like WAV, MP3, PCM etc.

• Who cares? Let’s see an MP3 DEMO, fall over.

DEMO TIMEBe a hero. Adopt a dog. Or a cat. Or a fish.

Some Random CodemediaRecorder = MediaRecorder() output = getExternalFilesDir(null)?.absolutePath + "/recording.mp3"

mediaRecorder?.setAudioSource(MediaRecorder.AudioSource.MIC) mediaRecorder?.setOutputFormat(MediaRecorder.OutputFormat.MPEG_4) mediaRecorder?.setAudioEncoder(MediaRecorder.AudioEncoder.AAC) mediaRecorder?.setOutputFile(output)

mediaRecorder?.prepare() mediaRecorder?.start()

mediaRecorder?.stop()val intent = Intent() intent.action = Intent.ACTION_VIEW val file = File(getExternalFilesDir(null)?.absolutePath + "/recording.mp3") intent.setDataAndType(Uri.fromFile(file), "audio/*") startActivity(intent)

MP3? Not for us!

• Popular coding format for digital audio

• Super easy to implement and widely supported

• Uses lossy compression to encode signals

• Saves a lot of space, but loses a lot of data

• See https://en.wikipedia.org/wiki/MP3

What else then?

• How about something a bit lower level, more, raw?

• PCM - Pulse Code Modulation

• Samples amplitude of signal at regular intervals

• No compression, raw as it gets

• See it in action - Guess what’s next?

DEMO TIMEBe a hero. Adopt a dog. Or a cat. Or a fish.

Some Random Coderecorder = AudioRecord( MediaRecorder.AudioSource.DEFAULT, SAMPLING_RATE_IN_HZ, CHANNEL_CONFIG, AUDIO_FORMAT, BUFFER_SIZE)

Runnable { override fun run() { val file = File(Environment.getExternalStorageDirectory(), targetFile) val buffer: ByteBuffer = ByteBuffer.allocateDirect(BUFFER_SIZE)

try { FileOutputStream(file).use { outStream -> while (recordingInProgress.get()) { val result = recorder!!.read(buffer, BUFFER_SIZE)

if (result < 0) { throw RuntimeException( "Reading of audio buffer failed: ${getBufferReadFailureReason(result)}" ) }

outStream.write(buffer.array(), 0, BUFFER_SIZE) buffer.clear() } } } catch (e: IOException) { throw RuntimeException("Writing of recorded audio failed", e) } }

PCM? Yes!

• We have access to the raw audio data

• Those ByteBuffers contain the audio samples

• Once you have the raw data, what can you do?

• Demo time!

DEMO TIMEBe a hero. Adopt a dog. Or a cat. Or a fish.

Is that all?

• Since we have access to the raw sound data, we can use it to create a visual representation.

• Basically, read the samples, plot them on a graph.

#wavesmatter

Got a Question?

Messing with Sound

• Android supports sound effects.

• Really!

• Demo time.

DEMO TIMEBe a hero. Adopt a dog. Or a cat. Or a fish.

Android Sound Links

• MediaRecorder - Official Docs

• AudioRecord - Official Docs

• ExoPlayer - Fancy pants media player for AndroidPlenty of videos on YouTube and articles on the web.

https://tears.org.za/