Tech_Days_Yokohama_Workshop_Session.pdf

7/27/2019 Tech_Days_Yokohama_Workshop_Session.pdf

1/103

Survive the test of time


Overview of J2ME and Nokia APIs


2/103


Stephen Neal

[email protected]

[email protected]


3/103


Overview

Part 1 J2ME Overview

Nokia handsets

J2ME Development Application structure

Tools

Available APIs

Part 2 Blackjack case study & networking

J2ME compatibility issues

Efficient MIDP programming


4/103


J2ME Structure

J2ME has two main parts Configurations

Specify minimal operating environments for

applications

Profiles

Complement a particular configuration Provides extra classes for particular vertical

application domains

e.g. RMI Profile, Games Profile


5/103


J2ME Structure

Configurations Specified in terms of

available memory

processor type and speed

network connection types available

basic API functionality Providers must fully implement a spec

developers are guaranteed minimal facilities


6/103


CLDC and MIDP

For wireless devices Commonly use CLDC configuration

Connected Limited Device Configuration

128Kb ROM for JVM and system classes 32Kb RAM (min) for application classes

And the MIDP profile

Mobile Information Device Profile provides:

simple LCD based GUI classes

basic networking facilities persistent data


7/103


Java versions This presentation will focus on CLDC and MIDP

Versions 1.0


8/103


Important CLDC Features

Fundamental packages re-specified Most packages removed

these remain: java.lang

System has no in field

java.io

No buffered classes java.util

Greatly reduced

Just 9 classes

there are 41 in Java 1.4.0


9/103



Restricted low level security model No dynamic byte code verification

J2SE verifier too expensive on memory

Use off-device pre-verification instead

adds extra attribute to basic class file

typically adds 5% to class file size ignored by regular byte code verifier

allows a single in-device linear scan of class file

to perform verification


10/103



Restricted high level security model Verifying class files only ensures that the

classes constitute a valid Java application

Need to restrict the behaviour of valid

applications

Standard security model too memoryintensive

S i th t t f ti


11/103



Simple model employed for J2MEapplication level security

Sandbox model

Limited classes available

User defined class loaders cannot be used

Native code cannot be downloaded Protected system classes

Cannot subclass certain system classes

S i th t t f ti


12/103


Nokia devices

Nokia devices grouped according tocapabilities

Series 30, 40, 60 ,80 UI categories

S i th t t f ti


13/103


Nokia devices

Series 30 Cost driven devices Nokia OS

MIDP 1.0, CLDC 1.0 Nokia UI API

In Nokia 3510i: XHTML, MMS, 4096 colours

Example: Nokia 6310i Maximum size of one MIDlet suite: 30 KB

96x65 pixels

Maximum data storage space for MIDlet: 20 KB



14/103


Nokia devices

Series 40 Size driven colour platform Nokia OS

MIDP 1.0, CLDC 1.0 Nokia UI API

XHTML, MMS, 4-way scrolling

Example: Nokia 7210 Maximum size of one MIDlet suite: 64 KB

128x128 pixels



15/103


Nokia devices

Series 60 One hand operatedfeature platform

Symbian OS

MIDP 1.0, CLDC 1.0

Nokia UI API (not backlight or vibrate)

XHTML, MMS

Example: Nokia 7650 Memory available to Midlets : 4MB (max)

176x208 pixels



16/103


Nokia devices

Series 80 One hand operatedfeature platform

Symbian OS

MIDP 1.0, CLDC 1.0 MIDP SW as a downloadable application

at Forum Nokia web site

Personal Java, JavaPhone API XHTML, MMS

Example: Nokia 9210

Memory available to Midlets : ~14MB 640x200 pixels



17/103


J2ME Development

Can use basic J2ME tools to build apps Tool support makes development simpler

Automated verification and packaging

Sun toolkits

J2ME Wireless Toolkit (1.0.4_01)

Sun One Studio Mobile Edition Borland JBuilder MobileSet Toolkit

Comes bundled with wireless toolkit

Simple integration with Nokia SDK



18/103


Tool support

Nokia Developers Suite Integrates with JBuilder, or Sun One Studio 4

Run as standalone

Includes Nokia emulators Nokia 6310i

Download more emulators

Series 40 and 60, Nokia 7210, Nokia 3510i MIDP SDKs

Others

Jakarta Ant



19/103


Developing with MIDP

MIDP applications are called Midlets Conceptually similar to Applets

Can be downloaded Executed in host environment

Subclass the MIDlet class

Provide no argument constructor

Must not implement main method

Implement lifecycle methods



20/103


Developing Midlets

The Midlet lifecycle A device will call

the methods shown

here as appropriate

Midlet can control

its own lifecycle notifyPaused()

notifyDestroyed()

Destroyed

new

pauseApp()

startApp()

destroyApp()destroyApp()

ActivePaused



21/103


Developing with MIDP Development process

1) Subclass MIDlet class

2) Override lifecycle methods

3) Compilea. Use 1.3+ compiler with modified boot classpath

4) Pre-verify

5) Packagea. Create jar file

b. Create application descriptor (jad file)

6) Install and run, or emulate



22/103


MIDP application structure

A MIDP application comprises two files jad file (application descriptor)

jar file

a Midlet Suite is a collection of related Midlets

Application.jar

classes

images

config details

Application.jad

application info

jar file details



23/103


The jad file

A file describing the Midlet suite Must include:

MIDlet-Name

name of the Midlet suite

MIDlet-

A name for each Midlet in the suite

MIDlet-Version

MIDlet-Vendor

MIDlet-Jar-URL

MIDlet-Jar-Size



24/103

The jad file

Information regarding the Midlet suite May include:

MIDlet-Description

MIDlet-Icon

MIDlet-Info-URL

MIDlet-Data-Size

Minimum persistent storage required by suite

Default is Zero



25/103

The jar file

Follows standard jar file format Manifest to include all data found in jad file

Except: MIDlet-Jar-Size

Can include other resources

E.g: Images

Must also contain: MicroEdition-Profile

MicroEdition-Configuration



26/103

Demo

Hello World Midlet Basic lifecycle methods

Build process

Emulation

From Suns J2ME toolkit

From Nokias J2ME toolkit



27/103

Developing with J2ME

Elementary MIDP User Interface Programming

Persisting Data

Networking

Extension packages

Nokia UI API classes



28/103

User interface programming

Limitation of display devices Colour or Black and White

Display size & device memory

Forget about AWT or Swing!

Too big

Overly complex

Tables, Dialogs, Multiple windows



29/103


New API for user interfaces with MIDP Lightweight Designed for use with small screens

Based upon single display areas

Both high and low level APIs available High level

Abstracts the device

Developer builds forms

Environment influences appearance

Low level Developer has more control over display



30/103


LCD user interface API In package:

javax.microedition.lcdui

Simple structure Add Displayable i tems to a Display!

Each Midlet has a single Display to use Use static method on Display class:

Display.getDisplay();



31/103


User interface class hierarchy Screens are high level Canvas low level (see later)

displaysDisplay Displayable

Screen Canvas

Alert Form List TextBox



32/103


A List example:list = new List("Select artist:", List.EXCLUSIVE);

list.append("B2K",null);

list.append("Babyface",null);

List types include:! EXCLUSIVE

! MULTIPLE

! IMPLICIT



33/103


A TextBox example:

TextBox textBox = new TextBox("Email address:","Text",

30,TextField.EMAILADDR);

Text constraints include:! ANY

! EMAILADDR

! NUMERIC

! PHONENUMBER

! URL



34/103


Forms Like other Screen subclasses

occupies whole screen

can have tickers set

etc...

Allow multiple i tems to be added there are many subclasses of Item



35/103


Items class hierarchy

ImageItem StringItem TextField

Gauge DateField ChoiceGroup

containsForm Item



36/103


A Forms example:form = new Form("Default settings");

form.append(new Gauge("Earpiece volume:",true,10,5));

form.append(new Gauge("Ringer volume:",true,10,5));

code: settingsform



37/103


Another Forms example:form = new Form("Enter flight dates:");

dateField = new DateField("Outbound",

DateField.DATE, timeZone);

form.append(dateField);

Date types include:! DATE

! DATE_TIME

! TIME

code: dateform



38/103

Commands

Notify user generated events J2SE applications use an event delegation

model

Explicitly attached to event sources

Typically results in many anonymous classes

More efficient to re-use objects

J2ME applications use Commands

Many predefined types

Can be extended with custom types



39/103

Commands

General process: Create a command

Assign it to a Displayable thing

The device decides where it will be shown

Menus or soft keys?

Programmer can influence this Set the type and priority of the command



40/103

Commands

Constructor for Commands:public Command(String label, int type, int priority);

Label

Text for the command in the user interface Type

Many predefined values

E.g: OK CANCEL BACK STOP SCREEN EXIT

Priority Influences the device when deciding where to display the

command

Careful: lower priority has precedence!


41/103



42/103

Commands

A Commands example:Form f = new Form(...);

f.addCommand(new Command("Exit", Command.EXIT, 0));

f.setCommandListener(new CommandListener() {

public void commandAction(Command c, Displayable d)

{

if (c.getCommandType() == Command.EXIT) {

notifyDestroyed();

}

}

});



43/103

LCD GUI Programming

Summary Offers both high and low level API

Low level API in part 2

Many useful classes provided

Device determines L&F

Event delegation model Commands



44/103

Persistent storage

Midlets will need to persist data Users name

Mail host address

High score for a game

MIDP includes a specification for PersistentStorage

Record Management System (RMS) Simple database model

Holds unique records for each Midlet suite

Device responsible for storing data



45/103

Persistent storage

Each Midlet suite shares records

Care should be taken with concurrent access

multiple active Midlets

multi threaded Midlets

writes to records guaranteed to be atomic

program defensively

Very simple API javax.microedition.rms

Only contains one class!

and four interfaces



46/103

Persistent storage

Records are stored in a Record Store Use the RecordStore class

Each record store is named max 32 Unicode chars

RecordStore rs = null;

try {

rs = RecordStore.openRecordStore("my-data",false);

} catch(RecordStoreNotFoundException e){

// doesn't exist

} catch(RecordStoreException e){

// some other error

}



47/103

Persistent storage

To create a new record specify the bytes you want to store each record within a store will have a unique

number (id)RecordStore rs = ...;

byte[] data = ...;

int id = -1; //record ids will always be > 0

try {

id = rs.addRecord(data, 0, data.length);

} ...



48/103

Persistent storage Demo: Global Time



49/103

Nokia specific APIs

Extended GUI facilities Sound and device control



50/103

Nokia UI API

MIDP 1.0 UI limited

Nokia address this with custom UI classes

Included in upcoming MIDP2.0 specification

New classes: FullCanvas

DirectGraphics DirectUtils

DeviceControl

Sound



51/103

Nokia UI FullCanvas Softkey options can use large portion of

screen space

FullCanvas removes these

Can no longer add Commands to theCanvas

Use low level API to detect key presses



52/103

Nokia UI DirectGraphics

MIDP 1.0 includes ability to draw and fillrectangles

DirectGraphics provides ability to draw

and fill arbitrary polygonsint[] xPoints = {xLeft, xTail, xRight, xHead};

int[] yPoints = {yLeft, yTail, yRight, yHead};

DirectGraphics dg =DirectUtils.getDirectGraphics(g);

dg.fillPolygon(xPoints, 0, yPoints, 0, 4, argb);

Demo: Boids Full



53/103


DirectGraphics also provides other usefulfeatures

Transparency support

Alpha channel in ARGB format

or, set transparent bit in PNG file



54/103


Improved image rendering facilities Image rotation

Image flipping

Help to decrease application jar size

requires fewer sprites



55/103

Nokia UI DirectGraphics Other features

MIDP1.0 has mutable images no support for transparency

Nokia UI supports mutable transparentimages



56/103

Nokia UI Other

Nokia UI supports other features Sound

Device control

Volume of device

Backlight

Vibration

User preferences may prevent these from

being accessible




57/103




58/103

Part 2 Case study

Blackjack application J2ME application design

Security considerations

Efficient programming

The future for J2ME



59/103

Application design J2ME applications typically consist of a

number of screens

Blackjack application screens Start

Rules

Registration

Game Text or Graphic?

Alerts Win or lose

Splash



60/103

Application design

Screens are organised into a flow: Most require simple behaviour

Some more complex

RegisterSplash Start

Rules

Game



61/103

Application design

High level or Low level API? High level simple Forms based paradigm

Seen already

Low level more complex

Permits explicit draw commands

Allows key listeners



62/103

Application design

Simple screens implemented using Highlevel API E.g: the RegisterScreen class

Can use singleton pattern Complex behaviour required too

Game screens need to adapt to device

Detect display size and create screendynamically

May require graphics capabilities



63/103

Adapting to the device

Different devices have different

capabilities

Must cater for these



64/103

Display factory pattern Alleviates the problem of device configuration

GameScreenFactory

Should detect display capabilities and react

accordingly DeviceCapability

Provides an appropriate GameScreen

implementation for the application Text based

Graphic based (requires low level API)



65/103

Low level user interface API

The high level API provides: Displayable classes

Items

Allows simple GUIs to be created Low level API provides pixel level

access to the display device

Draw charts Write games

Can be combined with high level items



66/103

Low level user interface API

Provides access to

Display objects

Colour or black and white?

Canvasses Discover size of screen and draw accordingly

Subclass Canvas class

implement paint method to

draw: lines, arcs, text images

similar to AWT



67/103

Low level user interface API Canvasses (contd...)

Provides hooks into user events key/button presses

in high level API this is done for you

Override one of the following methods from theCanvas class

protected void keyPressed(int keyCode);

protected void keyReleased(int keyCode);

protected void keyRepeated(int keyCode);

Key codes only cater for numbers, * and # hard to implement text entry

useTextBox andTextField instead



68/103

Graphic Game Screen

Requires use of low level API to allow

drawing directly to canvas

Uses paint method

Unlike JFC & AWT, no concept ofcontainers or layout managers

Must maintain own model of the game andits layout

Use Card, Deck , and Hand classes to

draw appropriate icons



69/103

Graphic Game Screen

Could be extended to provide key

shortcuts to avert over use of soft keys

See GraphicGameScreen class



70/103

Adding Control

Usually better to have a single class for

application logic

Common design patterns

Model View Control

Mediator

In our application we must: Provide the game logic

Switch screens at appropriate times



71/103

The Game Control class

Overall application structure

Game

Contro l

Player

Dealer

comman

ds

cont

rol



72/103

The Dealer

There are two modes for this game

Play against the phone

Play against a virtual dealer on the Internet

Could play for real money

Simple version

All application logic in device Networked version

Need to partition application



73/103

Option 1

Server

Player Dealer

Deck Card

Account

Dealer on phone

reports game status

to server

win, lose, bet amount

server maintains

account

Offload processingfrom server

Reduce network

traffic



74/103

Option 2

Server

Player

Dealer

Deck Card

Account

Dealer on server

all card requests

sent to server

card response sent

to phone

dealer knows game

state on server

Increased network

traffic

More secure!



75/103

Security Issues Downloaded code may be subverted

Modify/replace key classes in device

Introduce biased Dealer

General rule: Keep secure components on server

Demo: decompile



76/103

Networking Mobile devices need to network

MIDP specifies support for HTTP In addition to the GenericConnection class

from CLDC Cut-down HTTP protocol

Device must implement:

HEAD, GET and POST and absolute URIs Not necessarily over TCP/IP

Device should act as a proxy for a gateway service ifit is not IP aware



77/103

Networking CLDC defines the Connector class

Use this class to establish all connections

Example:

c = (HttpConnection) Connector.open(urlStr);c.setRequestProperty("User-Agent",

"Profile/MIDP-1.0 Configuration/CLDC-1.0");

is = c.openInputStream();

Demo: NetworkDealer, Shares Note: use system properties to discover MIDP and Configuration versions dynamically



78/103

Deploying Midlets Midlets can be deployed in a number of

ways Via web server

Over The Air (OTA) Nokia Tradepoint

Download jad file contains URL for Midlet jar

Bluetooth

Infrared

Serial cable



79/103

Deploying Midlets Demo: Deploy

Infra red application deployment



80/103

Mobile Multimedia API An optional API for controlling various media

sources

Sampled audio

MIDI audio

Tone sequences

Video

Not all implementations of MMAPI support allmedia types



81/103

MMAPI Architecture Divided into two key parts

Protocol Handling

Reading data from a source (DS)

File

Server Camera

Content Handling

Rendering of media (Player) Audio playback

Video display

PlayerDS



82/103

Manager Accesses system dependent resources

Dependent upon content type

Wav, AU, MP3, MIDI, MPEG

Uses URI syntax :

E.g: Manager.createPlayer("http://www.nokia.com/tune.mid");

PlayerDS

Manager



83/103

Media player lifecycle Programmer has fine control over the

player lifecycle

Initialisation of player may be expensive

Optimise use of player



84/103

Media player lifecycle Unrealized

Player holds no

resources that it

requires

Realized

Is ready to acquire

media May have contacted

server or opened file

Prefetched

Player has loaded

media Reserves media player

resource

Started

Closed

Audio device acquired

Manager.createPlayer()

realize()

prefetch() start()

deallocate() stop()

close()

close()close()

close()

Prefetched

Started

Closed

Realized

Unrealized



85/103

Controls and Listeners Controls provide media specific capabilities

Different players have different controls Adjust:

playback rate

midi channels video size

pitch

volume

Listeners react to media events

Player lifecycle, media errors, etc

Control

PlayerDS

Content HTTP Server

RMS

JAR

ControlControl

Listener



86/103

Using MMAPI Demo: Music

Adding game-play music to BlackJack


Th F


87/103

The Future MIDP 2.0

Proprietary APIs


MIDP 2 0


88/103

MIDP 2.0 Filed: April 2001

JSR-118

Final release: November 2002

Large expert group (50+)

Goals

Backward compatible with MIDP 1.0

Security Signed Midlet suites

HTTPS


MIDP 2 0


89/103

MIDP 2.0 Goals (contd)

UI improvements Custom items allowed to be created

Finer control over layout and L&F

Game API new Sprite and TiledLayer classes

Includes rotational and flip features

Sound API

Similar features to Nokia UI API

Enhanced application delivery Server notified of application delivery, installation and

deletion


J2ME V i t


90/103

J2ME Variants Implementers provide extra functionality

MIDP 1.0 MIDP 1.0DoJa 2.0

CLDC

Nokia Nokia UI

Polygon

Sound

Device Control

Pixel OperationsFull Canvas

JSR 130SMS send/receive

JSR 135MIDI Sound

Multi channel soundCamera

NTT DoCoMo DoJa 1.0

Device Control

IrDA

Phone call

Browser launchAnimation player

Movie player

Standby application

DoJa 2.0 (optional)Multi channel sound

3D Graphics

Sprites

Camera

J-Phone JSCL 1.1

Multi channel sound

Resident Midlet

3D Graphics

SpritesDevice control

Animation player

Telephony listener

Mail listener


J S ifi ti R t


91/103

Java Specification Requests MIDP and CLDC are two J2ME JSRs

Other J2ME related JSRs include: JSR 75, PDA Support

JSR 190, Events

JSR 172, Web Services

JSR 179, Location Services

JSR 184, 3D API

JSR 177, Security (DRM, Mobile commerce)

JSR 180, Session Initiation Protocol (SIP) (3GPP)

More


O JSR t l th ll!


92/103

One JSR to rule them all! JSR-185

A Roadmap for Mobile Phone related JSRs Java Technology for Mobile Industry (JTWI)

JTWI Compliancy

Specifies mandatory APIs JSR 30 - CLDC 1.0

JSR 118 - MIDP 2.0

JSR 120 WMA

And Conditional APIs

JSR 153 MMAPI (Mobile Multimedia) JSR 139 - CLDC 1.1 (Floating point arithmetic)


JSR 185


93/103

JSR-185 Includes a Technology Compatibility Kit

Reference implementation

Discourages proprietary APIs

Leave for backward compatibility only Deprecate ASAP

Allow software developers to focus on

application development


Implementation g ideline


94/103

Implementation guideline2002 2003 2004

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2

MIDP 1.0 (JSR 37) MIDP2.0 (JSR 118)

CLDC1.0 (JSR 30) CLDC 1.1 (JSR 139)

WMA (JSR 120)

MM API (JSR 135)BT API (JSR 82)

PIM & File API from PDA

Location (J179)

3D API (184)

Nokia UI API

Coreplatform

Productdependentextensions

Temporaryextension

Note: For planning purposes only. Subject to changes without notice. No commercial commitment can be made based on theinformation given in this presentation.




95/103

Efficient MIDP programming Challenge for MIDP programmers

not to learn a massive suite of APIs

fit all required behaviour into 30k !




96/103

Efficient MIDP programming There are a number of techniques for

efficient Java programming It is said that an application spends 90% of

its time in 10% of the code Tune only when the application is near

complete

Much of this time will be spent in libraries extra challenge for the MIDP developer

need to tune for many different libraryimplementations




97/103

Efficient MIDP programming Measuring the program execution speed

Current profiling tools ineffective work with emulators

different characteristics to a real device

Can still use simple performance tests:long startTime = System.currentTimeMillis();doSomething();

long endTime = System.currentTimeMillis();

Beware of: System clock granularity

Garbage collector

KVM collectors are aggressive




98/103

Efficient MIDP programming Other standard Java performance techniques

Reuse objects where possible

Beware of memory leaks

Do not call methods from loop check statements

Multithread where appropriate

Perceived performance

Keep screen active even if only a progress bar

Networking

Avoid inefficient protocols

e.g: XML, SOAP

Minimise HTTP roundtrips




99/103

Efficient MIDP programming Minimise Midlet download time

Keep the jar file small

Typical download time is 1 second per Kilobyte on

GSM network Many WAP gateways wont allow anything over

30k

Phones have restricted storage resources Users may want to install many apps




100/103

Efficient MIDP programming Try the following to minimise your jar files:

Dont write OO code Ouch :o(

Keep the number of entries in the jar file to a minimum

e.g: dont use MVC if single component can do the job

Use interfaces sparingly

Use the unnamed package

Use an obfuscator removes all unnecessary info

10% reduction is typical for a Midlet

Aides security too




101/103

Efficient MIDP programming Minimise your jar files (contd)

If using a library, only include essential classes in jarfile

Keep resources small

Try different image editing tools to see which is mostefficient

Combine all images in to a single file


Summary


102/103

Summary Handheld devices are rapidly becoming

a ubiquitous operating environment

J2ME provides

a platform neutral programming language

JSR-185 addresses portability issues

J2ME enables truly portable pocketable applications




103/103


Tech_Days_Yokohama_Workshop_Session.pdf

Documents

Transcript of Tech_Days_Yokohama_Workshop_Session.pdf