©Zachary Wartell - 6/12/2015 1 2D Graphical User Interface APIs Revision 1.3 Copyright 2006, Dr....

©Zachary Wartell - 04/18/23 1

2D Graphical User Interface APIs

Revision 1.3

Copyright 2006, Dr. Zachary Wartell, UNCC, All Rights Reserved


2D GUI APIs

• API/Library for device input and graphical output– WIMP (Windows,Icons,Menu,Pointer)

• Examples:– MS Windows: Win32 GUI, MFC, .Net C#– Java Swing & AWT – MacOS– Unix: X11, XToolkit, Motif, ….– Cross platform

• QT (TrollTech)• wxWindows• FLTK• GLUT (miniature GUI API)

….dozens and dozens more….

http://java.sun.com/docs/books/tutorial/uiswing/index.html

http://java.sun.com/javase/6/docs/technotes/guides/awt/


2D GUI API: A Software Component

Eye Brain

Display

Input Device

Body

CG System Software

OS

CG 3D API CG GUI (2D) API

ApplicationSoftware

General Computing Hardware

Graphics Computing Hardware

Image Synthesis


Design Parameters in 2D GUIs

• What is variety and sophistication of widgets?• How are widgets composed together?• What input mode modes are supported?• callbacks

– how are they implemented?– how are they registered?– what are there granularities?

• How do events propagate through widget hierarchy?• What data structures are used to represent events?• How do you handle background processing?• How do you do your own drawing to a window?

– What about animation?


Widget hierarchy

• widget – root super class, rectangular region in display, has changeable appearance, has methods for drawing itself and reacting to events– java.awt.component,FLTK Widget, X11 Widget, .Net

Form, etc.

• class hierarchy – labels, buttons, dials, sliders, menu, composition widget, canvas widget – FLTK Widget Hierarchy


FLTK Examples (from FLTK distribution package)

• radio – buttons including “radio” buttons• valuators – map GUI input to real number• tile – subclass of Fl_Group• scroll - subclass of Fl_Group• ..etc..

(Go play with FLTK examples!)


Windows/Widgets

• widgets composed in hierarchy (tree) of parent/child relationship using composition or grouping widget classes

Main Window

MenuBarDrawAreaToolGroup

SelectButton

PencilButton

FillButton

Canvas


Input Modes

• request mode (non-GUI) – program requests input and waits until it is received– C language ‘scanf’ from standard input

• sample mode (GUI) – program tests for input. If input’s available perform some action, but don’t wait for it. if (IsKeyPressed(‘P’)) { computePI(); }

..... position = GetMousePosition(); drawPixel(position);

• event mode (GUI) – program asks GUI Library to call an application function when a particular input event occurs.

GUI Library asynchronously collects input events in a queue and calls application’s function when (1) target event occurs and (2) GUI Library is not performing it’s own internal computation.


Callbacks and Event Handling

• event handling – programming mechanism used by application programmer to tell a library what to do when some event occurs

• callback function – a function written by application programmer that library calls in response to some event. Common steps:

1) register callback function, explicit registeration:• arguments:

1) callback function address 2) specification of under what circumstances (an event) library

should call function3) arbitrary data to pass to function when library calls it

• some OOP API’s don’t require explicit registration

2) call library’s main loop3) (some time later) library calls callback, callback

executes, callback returns to internal library code


void MyApp::handle_error (int error_num,ErrorCallbackData data)

{ …. }void MyApp::handle_net_msg

(Message m,MsgCallbackData data)

{ …. }

Callbacks and Event Handling

void main (…) {…

Library::register_callback (MyApp::handle_error, ERROR_EVENT, &myErrorCallbackData);

Library::register_callback (MyApp::handle_network_message, NETWORK_MESSAGE_EVENT, NULL);…

Library::run_application(); }

Library Code (internal, i.e. hidden from app. programmer)

loop { /* do all the stuff this library does …. */

….. event=detect_event() call_application_functions_triggered_by_event(event);}

Application Code

callbacks


Callback API Characteristics

• Different GUI libraries’ callback handling differ in these characteristics:– callback registration approach – may use

different approaches for different types of events

– callback event granularity – how specific an event is associate with a given callback?

– callback object granularity – for GUI API: Is callback registered for specific window or for entire application?


Callback Registration Approaches

1) call library’s register function (previous slide)

2) initialize array variable whose elements map events to callback functions

3) OOP + constructor automated: – constructor automatically registers certain class

methods as callbacks– developer’s sub-classes implement callback code

4) OOP + explicit registration

- java.util.EventListener, java.awt.util.ActionListener

class Window {virtual whenMouseMove (int x, int y);virtual whenKeyPress (KeyCode c);virtual whenMouseButtonPress (Button b);}


GUI Event Types

• input device: key press, key release, mouse motion

• window: resize, move, expose (“needs redraw”), mouse enter, mouse leave

• widget class specific: value change, menu select, scroll up


Event Propagation

• If child widget doesn’t handle event, should the event be passed up to parent widget?

Main Window

MenuBarDrawAreaToolGroup

SelectButton

PencilButton

FillButton

Canvas


Generic GUI Library Code

void main (…) { GUI::initialize(); win = MyApp::create_main_window(); can = MyApp::create_canvas();

GUI::when(win,KEY_PRESS,’Q’,MyApp::quit); GUI::when(can,MOUSE_MOVE,MyApp::moveBrush); GUI::when(can,WINDOW_EXPOSED,

MyApp::redrawPainting); GUI::run_application(); }

GUI Library (Internal)

loop { event=gather_input_device_events() redraw_all_windows_menus_etc(event); call_application_functions_triggered_by_event(event); call_application_custom_redraw_functions(); }

Application Code

callbacks

void MyApp::quit (…) { …. }void MyApp::moveBrush(…) { …. }

void MyApp::redrawPainting { /* redraw painting area */ }


Default: widget draw callback not called continuously

• GUI library shouldn’t unnecessarily call every widget’s draw function, especially for built-in classes!

• library calls widget draw callback ‘when needed’ only – window open event– expose event

• if developer is drawing his own graphics into a widget, W, then GUI library must be informed to treat W specially– developer can control when widget is redrawn– animation requires mechanism to continuously trigger

W’s draw callback


GLUT Event Handling• callback details – C function pointer, explicit registration, medium event

granularity, coarse object granularity (only one window!)

void keyboard (GLint key, GLint mx, GLint my) {… respond to keyboard input… }void mouse (GLint button, GLint action, GLint mx, GLint my)

{… respond to mouse input … }void display (void)

{... draw my OpenGL stuff…. }

void main (......){

… glutMouseFunc(mouse); glutKeyboardFunc(keyboard);

glutDisplayFunc(display); … }

• calling glutPostRedisplay() triggers GLUT internal loop to call redraw function during next loop iteration


FLTK Event Handling: Uses 3 Approaches

1) mouse, keyboard, most window events: use a C++ virtual member function callback approach:

Fl_Widget::handle(int event)

2) widget specific events: use explicit callback registration– one callback per FLTK widgetvoid Fl_Widget::callback(Fl_Callback*,void*=0)

– triggering event determined by Fl_Whenvoid Fl_Widget::when(Fl_When)


3) window redraw event: use a C++ virtual member function callback approach:

Fl_Widget::draw()

– for built-in widgets FLTK library draws widget– if application needs to draw arbitrary stuff (2D or

3D OpenGL), create subclass and override

Fl_Widget::draw()

– call Fl_Widget::redraw() to trigger FLTK to call ::draw() during next main loop iteration

FLTK Event Handling: Uses 3 Approaches


Java AWT Event Handling - EventListener

• java.awt.event.EventListener interface – input events and most window events– explicit registration, medium granularity, per widget

• related callbacks are grouped into java interfaces derived from EventListener. interface GLEventListener {

void init (GLDrawable);

void display (GLDrawable); void reshape(GLDrawable drawable,

int x, int y, int width, int height); }

• AWT component explicitly registers listener:

glCanvas.addGLEventListener(new MyGLEventListener());

Java AWT Event Handling - ActionListener

• java.awt.event.ActionListener– super-interface EventListener– widget class specific events– explicit registration, medium granularity, per widget

• Basics1) Declare event handler public class MyClass implements ActionListener {…2) Register someComponent.addActionListener(instanceOfMyClass); 3) Write callback code public void actionPerformed(ActionEvent e) { ...//code that reacts to the action... }


Java AWT triggering redraw

• java.awt.component - void repaint() – general

• javax.media.opengl.GLDrawable - void display()

– specific to OpenGL, but avoid for this course



Event Data Structure – variations I and II• What is data structure of API’s “Event”?

I) C struct & union: (X11,XT,Motif) struct EventBase { EventType type; } struct MouseMotion {

EventType type; int x, y;

…etc… }

union Event {

EventType type; struct MouseMotion mouseMouse; …list all types of events… }

II) C++ struct: could reuse C struct & union or use class hierarchy (wxWidgets, MFC (?))


Event Data Structure – variation III

III) int or enum: (Win32 GUI, FLTK)• int/enum only specifies type of event• data related to specific type of event must be queried using

separate functions


Background/Idle Processing – Single-thread

• GUI library’s main loop is hidden from developer; GUI library calls application’s callbacks in response to events.

• Single-thread background processing :– library supports an “idle callback”– GUI library will call the idle callback function

when library isn’t busy doing its other tasks

FLTK:void Fl::add_idle(void (*cb)

(void*),void*=0);

GLUT: void glutIdleFunc(void (*func)(void));

Background/Idle Processing – Multi-threaded

• GUI application automatically has two threads:– internal GUI thread - event processing and

redrawing• executes library’s main loop (internal event processing &

redrawing, application callbacks, application draw functions)• GUI libraries often support idle callback for this thread too…

– standard application thread• cannot draw here!• may contain developer code for application logic, but…

– then thread synchronization is required for objects shared accessed by both threads!


Multi-threaded – Idle Callback

• idle callback executes in event thread!

Java Example:

SwingUtilities.invokeLater(new Runnable() { public void run() { myCallback(); }}));

but to repeat callback you must re-register the callback

public void myCallback(){ //… do stuff… // re-register SwingUtilities.invokeLater(new Runnable() { public void run() { myCallback();}})); }



Example: Animation using idle processing

void main (…) { GUI::initialize(); …. initialize application objects/variables, event callbacks, and GUI widgets …. GUI::idle_callback(MyApp::idle_function); GUI::run_application(); }

GUI Library (Internal)loop {…. see slide 14 …}

Application Code

void MyApp::idle_function (…) { …update application obj.’s/var.’s based on time past since last calling this function GUI::request_widget_redraw (myDrawingWidget); }void MyDrawingWidget::draw(…) { …call drawing API to draw the visual representations of the application objects... }void MyDrawingWidget::whenMouseMove(…) { …update the application obj.’s/var.’s based on mouse motion event… GUI::request_widget_redraw (myDrawingWidget); }

1

3,26

2,4,6,8,10…25,27

5,7,9…

spontaneouscall (expose event, etc.)


RevisionRevision 1.1 various improvements to grammer and structure

- added slide on Event data structure - added slide for running a few FLTK examples

Revision 1.2

1. typos

2. added slide 20

Revision 1.3

1. added “Design Parameters” slide

2. misc. improvements

.

Revisions… to inefficient to maintain this. See SVN logs…

©Zachary Wartell - 6/12/2015 1 2D Graphical User Interface APIs Revision 1.3 Copyright 2006, Dr....

Documents

Transcript of ©Zachary Wartell - 6/12/2015 1 2D Graphical User Interface APIs Revision 1.3 Copyright 2006, Dr....