Richard LandauSarah PattenJacob Stewart

BroBotThe Study CompanionGroup 25

BroBot assists in easing the minds of students longing for a worry-free break.

Watches study materials while the user is away from the table. Sends updated images and alerts to user through an Android app on user’s phone.

Will provide real-time updates and will notify user if something is stolen.

Project Overview

Detect theft of an object.

Alert user when a theft

occurs.

Be reliable and simple to

use.

Replaceable or rechargeable

power source

Be lightweight and

affordable.

Goals and Objectives

Specification

Weight Less than 5 lbs

Dimensions Less than 6inX6in base

Communication Range At least 50 ft

Battery Life while in operation

At least 2 hours

Image Comparison Rate 2/sec

Refresh App Picture Every 5 seconds

Design Specifications

Overall Block Diagram

MT9D111 LinkSprite JPEG Color Camera

uCAM-II

Control Communication

Own protocol

UART UART

Data Lines 8 Bit Parallel

UART UART

Picture Type RAW + JPEG

JPEG RAW + JPEG

Voltage 3.1V 5V 5V

Current (Max Operation)

350 mA 100mA 90mA

Max Resolution 1600x1200 640x480 640x480

Cost 20.00 44.95 49.99

Cameras

5 volts neededData is serial dataSupport for fast baud rateSends 8-bit greyscale RAW

ImagesNeed to use RAWs to

analyze pixel values

uCAM-II

All of the computations are on the microcontroller side

A good amount of serial communication protocols (SPI, I2C, UART)

Low power modes including sleep modeGood community backingCheap development board with all the pin outs on

the microcontroller

Processor Considerations

TM4C123GH6PM

STM32F407VG ATSAM4S

Flash Memory 256 KB 1 MB 1 MB

SRAM 32 KB 192KB 128 KB

GPIOs 43 72 47

Max Speed 80 MHz 168 MHz 120 MHz

Package 64 LQFP 100 LQFP 64 LQFP

Dev Price (USD) 12.99 14.90 39.00

Board pluses Not too many output pins but has good community behind it

Comes with many MEMS sensors and all output pins

Comes with 2GB external flash and all output pins

ARM Microcontrollers

3.3V VoltageLow power modesDigital Camera

Interface(DCMI)Good amount of SRAMFast operation speedA lot of serial

communication linesDevelopment board comes

with ST-Link capabilities Good community backing it

with pre-written libraries for most features

STM32F407VG

Wi-Fi direct and Bluetooth are comparable

Wi-Fi direct has longer range than Bluetooth

However, Bluetooth has more support and is easier to implement

Bluetooth is also more common in everyday life so we wanted experience with it

Bluetooth vs WiFi-Direct

Chose RN41 due to low cost, high range, and black box functionality

WT41 strong contender but expensive and drains battery quickly

Bluetooth Module Comparison

Name TI LMX9830

BC04 RN41 BLE112 WT41

Connection Type

UART UART UART UART UART

Range 2 – 10m 1-15m 1 – 100m 2-10m 1-800m

Low Power

Yes Yes Yes Yes No

Price $13.46 $9.96 $21.70 $21.00 $33.00

Alarm

Uses STM32’s on board DAC to create a sine waveSpeed of output of DAC is varied to create

ascending and descending tonesTPA2005D1 – Class-D Amplifier3.3V Low power consumption via Pulse Width

Modulation8-ohm speaker inside BroBot

All components need 3.3-5V Around 750mA at maxPower source needs to be

small and lightweightRechargeable or easily

replaceable power source Two UA78 linear regulators,

one for 3.3V and one for 5V4 AA Batteries held in a

battery holder

Power System Overview

Camera Communication

Camera MUST Sync before further communicationsAfter 15 seconds of no communication, a Sync call

must be given to wake up the camera moduleTo generate an 8-bit grayscale raw picture file:Initial request – Sets resolution and formatReceive data – Starts data streamFinal ACK – Finish transmission

Camera Communication

Project box containing PCB

and batteries

Bluetooth accessible

Sturdy and adjustable

camera arm

Removable cover

BroBot’s Platform

Use computer vision techniques.Compare 2 images from different points in timeDetermine if there has been a significant amount of change

Item watching overview

≠

Store an “original” image

Each cycle, take a new picture

Compare new picture with “original”

If the images differ by too much, sound alarm

Every 20 pictures, replace “original” with the

newest picture taken

Algorithm explanation

Item Watching Flowchart

Android application’s main focus is to alert the user if

something has been stolen and to act as the user

interface

Picture of BroBot’s current view will be refreshed for

the user to view at any time

App will vibrate to alert the user if something is taken

Image causing the alert will be sent

App Summary

Android represents a bigger market shareDoes not require a Mac to codeGood APIs to interface with Bluetooth

Android vs iOS

VS

App Design

Initial screen shown upon startup

Click Connect, and select the device to connect to

If connection fails, user will see an error

If it succeeds…

User will be taken to screen showing BroBot’s field of vision

When his sight is aligned properly, press the button to start the item watching software

This button will be replaced with a “Stop Watching” button, or a “Silence Alarm” button if it is tripped

Bluetooth connection

Once Bluetooth connection is established, app opens an input thread and output thread

Output thread stays idle and waits for signals to be sent to BroBot

Input thread constantly looks for new commands from BroBotEach command received is handled accordingly by the app

and then acknowledged by a return commandBluetooth’s input stream buffer reads in an array of bytesWhen the app sends a command to request a picture (every 5

seconds), BroBot responds with a command, followed by the picture size and then the picture data

Combine these data bytes and restore the original picture

Testing

Each subsystem was tested as soon as it was possible to do so

Bluetooth module was tested from mobile terminal app to test communication

When app and camera were complete, picture testing between the camera, microcontroller, and app was done

Item watching code was tested on the computer before being extended to the embedded environment

After being entirely integrated, general tests were performed

Testing Data

No ChangeAverage change = 12,000

Small Change: Average change = 32,140

Medium Change: Average change = 81,486

Large Change: Average change = 142,987

Testing Data

Used a counter to time the number of comparisons done in 30 seconds

After 5 consistent tests, it was determined as 26/min

Slowly paced away from BroBot to determine range After a few tests in multiple locations, it was

determined as 60ft

Budgeting

Items Predicted Price Actual Price Difference

Camera $20.00 $140.00 -$120.00

Body $40.00 $42.00 -$2.00

Dev Board $15.00 FREE +$15.00

IC $22.00 $60.00 -$38.00

PCB $75.00 $55.00 +$20.00

Batteries $20.00 $6.50 +$13.50

Speaker $0.00 $5.00 -$5.00

Total $192.00 $308.50 -$116.50

Completion of Goals

Specification Results

Weight Less than 5 lbs 2 lbs

Dimensions Less than 6inX6in base

6inX4in base

Communication Range

At least 50 ft 60 ft

Battery Life while in operation

At least 2 hours Over 2 hours

Image Comparison Rate

2/sec .43/sec

Refresh App Picture Every 5 seconds Every 5 seconds