Mini Wiki Explorer

Proposal

Jan 31, 2010

WASK while you do task

WASK while you do task

A

B

O

U

T

W

A

S

K What an idea Sirji, a multi task hand held device exclusively used for real-time Wiki ASK

simultaneously at the time of listening someone. This is a project to deliver an

offline/online help anytime wherever you are, just speak into it & leave out all the rest

on it. Generally some words cause over-head transmission (create misunderstanding

about that topic, make it difficult to understand) of that whole lecture. That problem

can be erased using this device. Actually get rid of conventional typing method or

searching for the meaning of words in the dictionary for minutes, just speak & it will do

job for you.

A hand-held device with touch-screen display that enhance the listening and grasping experience for user by performing several tasks. For instance, you are attending a seminar & generally it happens when you are listening to speaker, some of his word lose their importance as they seems to be unknown to you and the only thing you can do at that time is just wrote it down and hope to have some gadget which can search and providing you beam of knowledge. Moreover you don't want to type any word. Then what will one do?

P: Attending a seminar is good for nothing if you don’t remember what you just went through!

S:

Our device converts whatever one say into text which directly enhances

user's interaction with the device & helps you to keep record of the lecture.

P: Generally some words cause over-head

transmission (create misunderstanding about

the topic, making it difficult to understand) of

the whole lecture.

S:

Due to its touchpad, you can easily extract phrases, words from

converted text, user can query Wikipedia (largest online encyclopaedia)

for those popped out texts making things easier to grasp.

P: During listening it’s hard to make

notes in the talk if user feel a need

of it, risk is if he starts making

note a few part of that talk can be

skipped.

S:

We provide user a very efficient way to take notes. As real time speech to text

conversion is very fast of our device, you can simultaneously highlight sentences by

just tapping on it. And you can perform some after talk is over. So take notes, ask for

images if you want to see it, add tags, save, email, print. Isn’t it so simple?

Extra features which could be proved useful for many more people

P: While listening to speaker if one feel need of eBooks related to any topic discussed during talk. It would be difficult to search at that time.

S:

User can search eBooks or books available on any particular. On the backhand our

device will search books from Google Book, Amazon, Flipkart etc. so it would be

efficient and reliable.

Tweet with me:-

If micro-blogging fans just has to say what they

wants to tweet & it will be done then that is

awesome. Just speak what you want to say & it

will be uploaded to your twitter account.

Synchronization:

User can easily synchronize this device with computer and

see saved notes in computer can transfer them from

device to computer & computer to device.

Problem Statements:

WASK while you do task

P

R

O

P

O

S

E

D

S

O

L

U

T

I

O

N

SO we provide you solution of this kind of problem by creating a prototype that

can hear speaker, convert it to text, pop out important words, ask you to

choose any word which you find tough enough to understand. It will search

that word or phrase in Wikipedia and show output of it. According to your

convenience translates it in language you want convert it again to speech and

gives you speech signals.

BD

HS

SS

IAI

TP

WASK while you do task

B

L

O

C

K

D

I

A

G

R

A

M

HS

SS

IAI

TP

Component Descriptions:

Intel® System Controller Hub US15W :

It is a highly integrated chipset that

addresses key requirements of thermally

constrained and fanless embedded

applications. It combines and controls the

Intel® Graphics Media Accelerator 500

(Intel® GMA 500), memory controller, and

I/O controller in a single-chip solution

while featuring advanced 3D graphics and

extensive I/O capabilities such as USB 2.0,

SDIO and PCI Express. It work as main

control unit in the device & controls

various features like graphics and display,

audio, video, interface. The more

information about it can be seen here.

WASK while you do task

B

L

O

C

K

D

I

A

G

R

A

M

HS

SS

IAI

TP

Intel Atom Processor:

A microprocessor incorporates most or all of the functions of a computer's central

processing unit (CPU) on a single integrated circuit (IC, or microchip). It’s an ultra-low-

voltage x86 and x86-64 CPUs (or microprocessors) from Intel. Designed from the ground

up, 45nm Intel® Atom™ processors pack an astounding 47 million transistors on a single

chip measuring less than 26mm², making them Intel's smallest and lowest power

processors.¹ All this while delivering the power and performance you need to access all

Internet capabilities.

Disk Drives:

Solid-State Drive:

A solid-state drive (SSD) is a

data storage device that uses

solid-state memory to store

persistent data & we are using

SODIMM DDR2 400/533 MHz

as memory device in this device

Hard Disk Drive:

A hard disk drive (often

shortened as hard disk, hard

drive, or HDD) is a non-volatile

storage device that stores

digitally encoded data on rapidly

rotating rigid (i.e. hard) platters

with magnetic surfaces.

CFD Connectors:

CFD cables are coaxial

cable used to connect

devices.

A dual inline memory module (DIMM) consists of a number of memory components (usually black) that

are attached to a printed circuit board (usually green). The gold pins on the bottom of the DIMM provide a

connection between the module and a socket on a larger printed circuit board. The pins on the front and

back of a DIMM are not connected to each other.

WASK while you do task

B

L

O

C

K

D

I

A

G

R

A

M

HS

SS

TP

IAI

RAM:

DDR2 SDRAM is a double data rate synchronous dynamic random access memory interface.

DDR2 stores memory in memory cells that are activated with the use of a clock signal to

synchronize their operation with an external data bus.

The memory socket in this device will support at least 2 GB DDRII 400/533 memory which

introduces 166 MHz memory clock, I/O Bus clock 266 MHz with the data transfer rate of 533

million & peak transfer rate 4266 MB/s.

Universal Serial Bus:

:

USB (Universal Serial Bus) is a specification to establish communication between devices and a

host controller (usually personal computers). USB is intended to replace many varieties

of serial and parallel ports. USB can connect computer peripherals such as mice,

keyboards, digital cameras, printers, personal media players, flash drives, and external hard

drives.

In this device at least 4 USB 2.0 ports ( 3 host, 1 client) is used .USB 2.0 has maximum data rate

speed 480 Mbit/s.

Mini Card:

:

PCI Express Mini Card (also

known as Mini PCI Express,

Mini PCIe, and Mini PCI-E) is a

replacement for the Mini

PCI form factor based on PCI

Express. The host device

supports both PCI Express

and USB 2.0 connectivity. Mini

Card edge connector provides

multiple connections and

buses: such as PCIe ×1, USB

2.0.

RJ45:

:

RJ45 is one of the

many registered jacks. It is

often incorrectly used as the

name for the 8P8C modular

connector used to

terminate Ethernet cable. It

specifies both the physical

connector and wiring

pattern.

Intel Gigabit Ethernet chip:

:

Gigabit Ethernet (GbE or 1

GigE) is a term describing

various technologies for

transmitting Ethernet fram

es at a rate of a gigabit per

second, as defined by

the IEEE 802.3-

2008 standard.

We will be using this chip

to enable wired network

facility in the device.

Intel High Definition Auio:

:

:

Also called HD Audio or Azalia. It is used for delivering high-definition audio that is capable of playing back more

channels at higher quality than previous integrated audio codec’s like AC'97.

In this device two audio jacks will be used one for microphone input jack & the other one is speaker output jack.

WASK while you do task

B

L

O

C

K

D

I

A

G

R

A

M

SS

TP

IAI

Serial Digital Video Out:

:

:

:

SDVO makes it possible to use a

16-lane PCI express slot to add

additional video signalling

interfaces such

as VGA and DVI monitor

outputs, SDTV and HDTV televisio

n outputs, or TV tuner inputs to a

system board containing an

integrated Intel 9xx-series

graphics processor. Here it can be

placed on a PCI express card,

allowing video connectors to be

added or exchanged at low cost.

LCD:

:

:

:

A liquid crystal display (LCD) is a

thin, flat panel used for

electronically displaying

information such as text, images,

and moving pictures.

We will be using 7’ or 12 ‘ LCD

display for the output display

device.

Nutshell what it have as its Hardware Specifications are:

N270 Processor and 945GSE Chipset Product: Clientron E830 Sunshine Valley

Features:

Intel® Atom™ N270 processor with 512KB L2 cache 1.6 GHz FSB 533 MT/s at 1.6 GHz

Mobile Intel® 82945GSE Express Chipset (GMCH) 82801GBM (ICH7M)

Supports DDR2 frequency of 533 MT/s or 400 MT/s, single channel, 2GB max

USB ports, 1 PCIe* 1.1 X1 slot, 1 PCI 2.3 slot

Integrated graphics one DVI-I and one VGA Connector

PCI Express* n-Board LAN

PCI Express* n-Board LAN

Audio RealTek* ALC268

ATA/Storage

IDT* ICS9LPRS501 system clock generator

WASK while you do task

S

O

F

T

W

A

R

E

S

P

E

C

I

F

I

C

A

T

I

O

N

S

IAI

TP

Operating System: Moblin OS , short for 'mobile Linux', is an open source operating

system and application stack for Mobile Internet Devices (MIDs),netbooks, nettops and

embedded devices

Software Solution: pytgtk application based on Juilus Open source speech recognition engine. Most important aspect of this device is the software stack we will be using, which includes a voice to text converter, which includes a voice to text converter, after researching we found that work on this has been done on OLPC sugar environment, significant research has been done in this filed regarding the low power machines and efficient speech conversion. We quote here text from http://wiki.laptop.org/go/Speech_to_Text which describes Various free and open source solutions available for this And a nice comparison of them:

Technically, Julius and Sphinx seem to be the best choices. VoxForge supports both of them

and they both are widely used. Sphinx comes in different flavours quite confused by version

numbers. Most notable are Sphinx 3 and 4. Sphinx 3 was written in C and later Sphinx 4 was

released as a complete rewrite in Java. Some points in favour of Julius are:

Julius is better suited for dictation purposes which are what we are looking for here.

Simon project has done some research to rate the Speech to Text engines. Since they

have practically tried it, Julius seems to have scored off well.

Testing of Julius on various machines (and different OSes) showed that Julius needs

no additional configuration for installation.

Some references:

Julius, http://sourceforge.jp/projects/julius

VoxForge, http://www.voxforge.org/

CMU Sphinx, http://cmusphinx.sourceforge.net

4. Simon, http://sourceforge.net/projects/speech2text/

Offline Wikipedia reader The Offline Wikipedia reader is a set of scripts and programmes which can be used to display Wikipedia

pages without an internet connection. The software provides a custom lightweight web server running locally

and uses php to present the pages, which are then viewed using any web browser. All Wikipedia pages are

contained in the Wikipedia page dump. The file needed is called 'pages-articles.xml.bz2', and the most

recent is 4.1GB.

At present, a single tar.bz is downloaded from the site, the pages extract downloaded and copied to the

location, and the indexing process run.

Except this offline reader, online help will also be available in the device for live solution.

WASK while you do task

S

O

F

T

W

A

R

E

S

P

E

C

I

F

I

C

A

T

I

O

N

S

IAI

TP

Wikipedia: Tools Various tools from Wikipedia will be used in this device intended to simplify, make more efficient, or provide

additional functionality to Wikipedians. Like:

Browser & editing

Searching

Downloading

Google Tools

Page histories

Drivers/Development Kits Intel® System Controller Hub US15W drivers

Intel® Atom™ processor Z5xx series and Intel® System Controller Hub US15W development

kit.

Intel® Atom™ processor Z5xx series and Intel® SCH US15W VirtualLab

Intel Compilers

On-chip Debuggers

Chipset Drivers

Intel High definition audio driver

Bios Drivers

Graphics Drivers

QNX* Fastboot Initial Program Loader (IPL)

Except these solutions we need to have Twitter API’s to embed it into application which will

let users to tweet directly through speaking...

& Google AJAX Language API code for the language translation between various languages so

that user can swap between different languages according to their suits.

WASK while you do task

S

O

F

T

W

A

R

E

S

P

E

C

I

F

I

C

A

T

I

O

N

S

IAI

TP

Development and Deployment Environment

We need two Linux environments. We used one machine with dual boot for each environment:

1. Development environment - a Linux development Machine where we'll be doing our coding, compilation, debugging and packaging.

2. Deployment environment - a machine running Moblin We'll deploy our project to this machine for testing and debugging.

Development environment

A Linux machine running an up-to-date Linux distribution is Suitable for Moblin development. Our development environment is: Fedora 12 x86 64 bit

1. Installed the standard set of Linux development tools, as described in Installing Linux development tools.

2. Installed the Moblin SDK components. To set up tools for use in Moblin development, a toolchain containing the required versions of the Moblin libraries, and the Anjuta IDE with Moblin plugin.

Installing the Moblin SDK components

The Moblin SDK components provide assistance at each stage of Moblin application development. They comprise:

Moblin toolchain This is a directory which contains the Moblin operating system, including header files, Libraries, and package information. It allows you to run Moblin applications on your workstation, build and Debug code in an editor of your choice, and do remote debugging. Install them using the Moblin toolchain instructions.

Linux Project Generator This graphical application generates GNU auto tools-enabled projects.. Install it using the Linux Project Generator instructions.

Moblin Package Creator This tool can be used to generate an RPM and/or DEB package from your project, suitable for installing via the Moblin Application Installer. It has both command line and graphical interfaces.

Install them using the Moblin Package Creator instructions.

Anjuta Moblin SDK plugin This provides Anjuta IDE integration for building Moblin applications. Install Anjuta and the plugin using the Anjuta Moblin SDK instructions.

Deployment environment

Since Moblin simulators are yet to be developed, we need to setup a machine with moblin to test application. We dual boot our system for this.

WASK while you do task

I

N

T

E

L

A

T

O

M

I

N

T

E

G

R

A

T

I

O

N

TP

WASK while you do task

T

E

S

T

I

N

G

P

L

A

N

S

Testing is an important part of any project. Testing is an activity that can go on forever so it is important to test the areas that are most important and the areas that are most likely to reveal bugs. Like any project, exhaustive testing is not possible. Testing is a highly strategic activity and different methodologies are used.

Exploratory Testing

Exploratory testing is one of the newer testing methodologies. It is a type of manual testing where no formal plan is made. It involves ‘exploring’ the product, targeting areas that are likely to reveal the most bugs, almost like a mission. It is also known as ad hoc testing but this word is usually viewed with negative confutation portraying a sloppy and careless method of testing.

The Test Plan

A complete test plan discussing the schedule of testing and the strategy to take will be developed at the later stages of the project development life cycle. It is important to priorities the test cases running the ones that are most likely to produce a bug and the ones that are of highest risk if they were to fail. The categories of the test cases run are as follows:

Functionality and System Tests

The test cases in this category deal with testing device’s functionality and ensuring the all the requirements work correctly. This involves testing every area of functionality that the end user can perform.

Unit Test

The first test in the development process is the unit test. The

source code is normally divided into modules, which in turn are

divided into smaller units called units. These units have specific

behaviour. The test done on these units of code is called unit test.

Unit test depends upon the language on which the project is

developed. Here it ensures that each unique path of the project

performs accurately to the documented specifications and

contains clearly defined inputs and expected results.

Stress Tests

It is important to ensure that wikilistener will always have enough resources when it is stressed. Tests in this category dealt with running wikilistener for extended periods of time in an effort to crash it. It is not unusual to see programs crash over time. For example, a memory leak or buffer overflow would Become evident in these circumstances.

Beta or Third Party Tests

We will pick a group of people and tell them to use our device as any normal device and report any bug they find. This can be performed using a bug tracking system where the colleague would report any bugs they find.

Black Box Testing

Testing software without any

knowledge of the inner workings,

structure or language of the module

being tested.

WASK while you do task

W

O

R

K

D

O

N

E

We have studied all the resources materials we can get from internet till now. We know what exactly we are about to made and how we'll achieve our goal. We have read the datasheet of atom processors available on internet. We have done all the research work related to interfacing, programming, and prototyping and how to configure software’s, calibration and working on word sense disambiguation algorithms. What we are up to is to utilize the atom’s processing power to the max level for better performance & usability. Work done so far includes:

According to the give Application References Designs, the proposed device is based on the ‘Media

Phone Reference Design based on Intel Atom Processor.’ This reference is specifically dedicated to

VoIP, video and converged communications providing the suitable design for our device. The

hardware circuit design of the device has been made.

All the necessary hardware requirements have been listed according to the proposed design.

Except hardware solutions, software solution is what we are working mainly on right now. Like

choosing the right development & deployment environment that suits the requirements as is stated

above. Operating system that are to be chosen are Linux, Moblin, & Windows. The choice depends

on the availability of the drivers, BIOS & other development kits what Os they support.

Except it Linux & Moblin seems to be the best choice as they support Intel atom based applications.

Besides these we have tested Julius speech to text conversion application. The snapshots are given

below. We successfully managed to install it on our system & recorded & converted voice with it

into text.

Installed the standard set of Linux development tools, as described in Installing Linux development tools.

We prepared our Fedora machine with the developments application for the better use with the Fedora Electronics Labs as is described here.

Google Labs helps in understanding how things will work to fulfil the translations & image searching.

We worked with existing solutions available in market today for better understanding like:

o Natural speech recorder & speech to text converter:-SONY ICD-SX57 is one of the devices which stores voice & convert it into text. For more reference click here.

o Wikipedia search (Word wise): This feature is present in Mobipocket device which open eBooks & enable word wise wiki search. For more info here.

We used Offline Wikipedia reader which is a set of scripts & programmes used to display Wikipedia content without internet connection.

Using Twitter API’s we managed to tweet or get updates in the account. Info can be found here about API’s. We implemented just for testing twitter API using Firefox extension “JETPACK”. This only gets updates from the account on some specific time interval. Source code can be found here.

For the UI part we will use our coding skill Qt Designer can help in making better user interface. Except it, all the software can be developed using various Free & Open Source Software Solutions available. Most of the prototyping and modelling work has been done so far and refining of the ideas and finding out better ways of implementing is what we are working upon.

WASK while you do task

W

O

R

K

D

O

N

E

Snapshots of work done so far:

/* Adinrec , a component of Julius used to record voice can be seen working */

WASK while you do task

W

O

R

K

D

O

N

E

/* /home/kunal/sounds & /usr/local/bin */

WASK while you do task

W

O

R

K

D

O

N

E

/* Wiki reader Installation */

/* Moblin Installation */

WASK while you do task

W

O

R

K

D

O

N

E

/* Moblin in running state */

WASK while you do task

WASK while you do task

WASK while you do task

WASK while you do task

WASK while you do task