DESIGN AND REALIZATION OF AUTONOMOUS VISION

BASED SEGWAY ROBOT USING ARM CORTEX M4

Srinath V

Department of Electronics and Communication

Dr. NGP Institute of Technology

E-mail: srinath9476@gmail.com

Dr. S Suresh Kumar

Department of Electronics and Communication

Dr. NGP Institute of Technology

E-mail: sskpsg@gmail.com

Abstract. Segway Robot is one of the popular human

transporters, used by major Countries, such as for

police and military patrols in United States and

tourism at red-Fort in India. In the previous version,

the Segway robot featured movement based on

gyroscopic signals, but manual setup of the Segway is

required at initialization. In the proposed version this

obstacle is overcome by the addition of autonomous

features and control through Bluetooth interface. In

addition “Voice and Follow me” command features

can be implemented. As such in this way the Segway

robot can be remotely manipulated to move from one

place to another and also interact with user. To obtain

these features, the Microcontroller can be used with

Dual mode Bluetooth. On the whole, this proposed

product meets the eco-friendly standards with 0%

CO2 emission and acts as an autonomous stand-alone

transport vehicle.

Keywords- Autonomous, Segway, Booster-pack Sens-

Hub, Dual Mode Bluetooth, Mobility Robot- Society

Interaction, Adaptive Control.

I. INTRODUCTION

The exhibition inspection and other

special occasions are being developed for

launching new variety of personal

vehicles.

Among which relatively conventional to

the four wheeled vehicles, the two-

wheeled self-balancing electric robot has

environmental features like energy

saving and environmental protection. In

which, the self-balancing system keeps

the system balance by signal state

detection and analysis of variables.

Through self-balancing vehicle is eco-

friendly flexible movement and

intelligent control it is atypical unstable

and non-linear.

For a long time in self-balancing

robot is used as a transporter named

Segway robot. As well more control

mechanisms with more design, analysis

and modeling are developed in Segway

robot. With a rapid development of

miniature electro- Mechanical devices

Segway robot property are more

outstanding especially in flexibility. By

measuring of compatibility based angle

computing for balanced control in Self-

Balancing vehicles the system attains

more reliable state by following obtain

data from number of sensors and data

fusion[1]. As two wheeled human

transportation vehicle, Segway robot

consists adaptive robust regulation

methods based on pitch and yaw motion

[2] with steering. Based on the battery

drainage prevention, a method Dual-

Redundancy thermal backup control

method are made for its control scheme,

its fault detection and switching method

on power supply system, master control

logics, driver control, power circuit and

actuator [3]. Today`s smart world

robotics arm controlling microcontroller

can be interfaced with Wi-Fi mobile

phone [4]. This controlling based an

application build in the android platform.

As the Segway robot`s popularity

increases, the smart interface system is

essential in the field of IOT, because

every technology needs Real-Time basis.

As the robot`s information in the Real-

Time base is essential the basic needs is

signal processing [5]. Therefore, A signal

patterning and recognition approach is

made for mobile device. The most

significant signal is the brake state

signal, by which it is filtered and

classification module is made accurate

and robust. The most intelligible

technique is visualization, the Vision

based navigation and mapping generally

needs for the robot monitoring system

[6]. In which, the major drawbacks like

forward kinematics and kinetic fusion are

removed by means of Articulated Robot

Motion for Simultaneous Localization

and Mapping (ARM-SLAM).

Today`s part of the world moves

towards Security Process for navigation

and data transferring, which cause of

increasing security growth. The security

rate, needed to be concerned from crime

break action. This crime through data

transfer can be secured by Surveillance

systems. To make a real time based

surveillance system possible within local

area network, so live streaming is

accomplished [7] by using mjpeg

streamer. This system designed with cost

effective and portable to make the IP-

based systems and affordable for the

people having low-budget. It has the

capability of installing and processing

high resource software`s which makes to

accomplish the live streaming and

controlling the robot. The Segway robot

can works with Autonomous mode by

interfacing mobile application control

systems, obstacle avoidance for robot`s

mobility platform should be concerned

[8]. Based on the Home-Automation the

two wheeled inverted pendulum based

Segway Robot can be used. It`s DOF

(Degree of Freedom) arm gripper can be

interfaced for the home application

services, with Inverted pendulum,

variable centroid inverted pendulum

should be balanced in the control way

[9]. The Degree of Freedom of Robust

control is closed-loop system, which

achieves more stability and high

performance in the presence of uncertain

friction coefficients [10].

II. SYSTEM ARCHITECTURE

The basic system needs to be

designed relatively cheap and available

widely on the market, also to be easily

replaceable. As Segway robot works

under Real-time, with proper planning a

standard mechanisms and controllers to

be used for Segway robot systems. By

which, this Segway robot is partitioned

into two units,

1. Self-Balancing unit

2. Autonomous unit.

Fig 2.1.Autonomous System Architecture of Segway Robot

The Architecture of the Autonomous

Segway robot is shown in Fig 2.1. This

proposed Segway robot includes IMU

sensors, Servo motor, motor with

encoding, Arduino Uno, Arm-Cortex,

Grippers, Camera interface, Mobile

application and Bluetooth with Wi-Fi

module based IOT.

The System`s two units are explained

below. The Segway robot`s components

with unit price is shown below of the

table 2.1 and the power consumption of

total board is given in table 2.2.

S.No Components Unit

Price

1. Arduino Uno 500

2. Arm-Cortex 2000

3. IMU sensor 200

4. Motor controller 140

5. Servo Motor 200

6. Bluetooth 400

7. Wi-Fi 800

8. Camera 1000

9. Gripper 2000

10 Motor with Wheel 2700

11 Light 300

Table 2.1 Segway Major Components

DEVICE ACTIVE IDLE

Arduino 12V 5V

Armcortex 5V 3.3V

IMU 5V 3.3V

Motor Controller 12V 5V

Servo-motor 5V 5V

Bluetooth 5V 3.3V

Camera 5V 3.3V

Table 2.2 Power Distribution

A. Self-Balancing Unit

The working of proposed system has

IMU sensor, which helps to obtain the

Pitch, Yaw and Roll angles to detect the

robot`s portion and direction. Arduino uno

microcontroller helps to process the IMU

reading, based on which directs the motor

through motor controller. So, robot can

move either forward or backward direction

based on the manual leaning. The Arduino

microcontroller will also rate the speed the

motor based on tilt angle, because by the

Principle of the inverted pendulum, Speed

is directly proportional to the tilt angle.

The motor encoding will encode the

current speed of the motor rotation. In order

to correct the rated speed in accurately, the

microcontroller will continuously check

and correct the speed of the motor in

particular PWM wavelength.

B. Autonomous Unit

The proposed system designed with

Autonomous mode of operation and

capable of transmitting Segway robot`s

Speed, Status and Location. By using

advance technologies such as mobile

application, robot can be controlled its

directions and also can set the speed

limitation. In this proposed unit it has

additional features such as grippers can be

operated through mobile for any home

applications such as floor cleaning. One

more essential is vision based surveillance

are made with simple camera, which takes

simple pictures and can be transmitted to

mobile phone or Sd-card.

III – PROGRAMMING MODE

PROCESS

The Segway Robot has basic control

unit as Self-Balancing control unit consist

of four main sub units namely Tilt sensor

unit, microcontroller unit, motor control

unit and motor encoder unit. Additionally,

microcontroller will operate two manual

pad inputs. These units are implemented in

the Hardware platform base using Arduino

C programming and it`s flow at each stage

are discussed as follows.

The programming task is divided in

parallel processing, by which is partitioned

in four modes,

1. Self-Balancing mode

2. Synchronization mode

3. Peripherals mode

4. Autonomous mode.

1. Self-Balancing mode Process

Entire self-balancing process is

carried through Arduino programming. The

programming of Arduino should be Real-

time base. In this, the Segway Robot is

driven by manually with two inputs, along

with the IMU6050 and two manual inputs,

the Arduino board is programmed. The

Self-balancing is shown in the Fig 3.1.

Fig 3.1 Self-Balancing schematic

Fig 3.2 Flow chart of Self-Balancing Process

Fig 3.3 Flow chart for manual transporting

2. Synchronous Mode

This mode is data sharing process, in

which communication between Arduino to

Arm microcontroller or vice versa takes

place. Like Speed and direction from Arm

microcontroller will be transferred to

Arduino, based on the rate the Arduino will

direct the motors through motor controller.

In Fig 3.4, the basic synchronous mode

operation is shown.

3. Peripherals mode

In proposed system, Arm

Microcontroller which handles the

peripherals like grippers and camera

positioning control. This control is made

through remote operation. By using

Bluetooth module, interface with mobile

Fig 3.4 Synchronous Mode

Communication

application and control the robot`s gripper

through it. The function from arm

microcontroller to peripherals is shown

below in figure 3.5. The programming flow

diagram is shown along with autonomous

mode in Fig.3.7.

Fig.3.5. Peripheral mode interface with

Lanchpad

4. Autonomous mode

Autonomous control unit has ARM

Cortex m4 microcontroller act as the major

part. This controller has Bluetooth interface

protocol helps to communicate with or

mobile apps. This also helps to direct the

Segway Robot in desired direction by

commanding through mobile or any RF

transceiver. Directing command from

wireless module will be processed by ARM

Cortex. In mobile or any wireless module

the command is divided into two sections.

One section helps to command the wheel

motor and another one helps to command

the ARM grippers. The ARM Cortex have

camera with controller, this helps to capture

videos and images and transfer directly to

our mobile gallery and video clip mass

storage area. Individual operation is shown

in Fig.3.5 with clear flow chart which is

shown below Fig 3.7.

Fig 3.6 Autonomous mode Vision with

Bluetooth and IOT interface

Fig 3.7 Flow chart for autonomous

and peripherals process.

i) Wi-Fi Interface

Wi-Fi ESP8266 module is a self-

contained SOC with integrated TCP/IP

protocol stack that can give any

microcontroller access to Wi-Fi network.

The ESP8266 is capable of either hosting

an application or offloading all Wi-Fi

networking function from another

application processor. This component has

a power full enough on-board processing

and storage capability that allows it to be

included with the sensors and other

application specific devices through its

GPIOs with minimal development up-

front and minimal loading during runtime.

In Segway robot following information

likes robot speed, location, battery status

and photos videos can be enclosed with

home automation, Wi-Fi location devices,

industrial wireless control and security ID

tags are transmitted from transmitter side.

With Wi-Fi, transmitted details about

Segway robot can be received in receiver

side either in remote location or in city

side apart in few kilometers.

ii) Mobile Application

Mobile application is an App build in

the android platform and all smart phones

can uses the android apps. The android

app is built in JAVA programming

language. A signal is generated by clicking

a specific buttons on the android

application, this application is open source

can add, delete and edit the buttons using

eclipse software. By command center in

the mobile can be transmitted to robot

through Bluetooth, as per the commands

the robot microcontroller moves to arm,

grippers, camera position and transport

forward or backward.

Fig 3.8 Autonomous Vision based

Segway Robot Module

IV. CONCLUSION

The Arm Cortex lanch pad now can

be used for the control Robotic peripherals

with Smartphone from a remote area. The

Present Robot with Smartphone is internet

controlled robot has several disadvantages

such as wired restriction and server

problems. Due to delay and server

connection reduction problem date

transmission becomes delay rate. As the

Wi-Fi reduces these problems, because it

is fastest usage of internet, in present

situation, most people uses the

Smartphone worldwide.

REFERENCES

[1] Siyuan Ma1, Chunye Du1, Yang

Zhao1 and Man Wang2,(2016).

“Measure of Compatibility Based

Angle Computing for Balanced

Posture Control on Self-balancing

Vehicles”, Springer International

Publishing Switzerland, pp. 551-

562.

[2] Shui-Chun Lin, Ching-Chih Tsai,

Hsu-Chih Huang, (2011), “Adaptive

Robust Self-Balancing and

Steering of a Two-Wheeled

Human Transportation Vehicle”,

Springer J Intell Robot Syst, pp

103-123.

[3] Yong Tao. Et.al, Tianmiao Wang,

Ye Wang, Jianhong Liang and

Zhongyuan Chen (2016), “ A Dual-

Redundancy Thermal Backup

Control Method for the Two-

Wheeled Self-Balancing Robot”,

Springer Verlag Berlin Heidelberg,

pp. 1029-1038.

[4] Keerthi Premkumar, Mr.K Geatatd

Joe Nigel (2015),“Smart phone

Based Robotic Arm Control Using

Raspberry PI, Android and Wi-Fi”,

IEEE innovation Embedded and

Communication System.

[5] Ali Boyali et.al., Naohisa

Hashimoto, Osamu Matsumoto

(2015), “A Signal Patttern

recognition approach for Mobile

devices and its appliacation to

braking state classification on

robotic mobility devices” Elsevier

Robotics and Autonomous System

(72) pp.37-47.

[6] Matthew Klingensmith et.al,

Siddarth S. Sirinivasa, Michael

Kaess (2015), “Articulated Robot

Motion For Simultaneous Location

and Mapping (ARM-SLAM)”,

IEEE Robotics and Autonomous

letters 26(12), pp.2377-3766.

[7] Rupali Ikhankar, Varun Kuthe,

Sarang Ulabaje, Shruthi Balpande,

Mahendra Dhadwe (2015), “ Pibot:

Raspberry PI Controlled Multi-

Environment Robot For

Surveillance & Live Streaming”,

International conference of

Industrial Instrumentation and

Control, pp.1402-1405.

[8] Cang Ye and Johann Borenstein

(2004), “Obstacle Avoidance for the

Segway Robotic Mobility

Platform”, American Nuclear

Society International Conference on

Robotic System for Hazardous

Environments, Gainesville,

[9] Gang Wang et al, Sang Yong Lee,

Seok Won Kang and Jang-Myung

Lee (2013), “ Balance Control of a

Varible Centroid Inverted

Pendulum Robot”, Springer-Verlag

Berlin Heidelberg Intelligent

Autonomous System 12,(193),

pp.201-208,

[10] Mitual Mathivanan, Mithun

Chakravarti and G.S.Mahalaksmi

(2016), “Semi-Autonomous Self-

Baalancing Robot” IJCTA

9(2),pp.513-519.