Building GUIs with Julia, Tk, and Cairo, Part I23 May 2013 | Timothy E. Holy This is the first of two blog posts designed to walk users through the process of creating GUIs in Julia. Those following Julia development will know that plotting in Julia is still evolving, and one could therefore expect that it might be premature to build GUIs with Julia. My own recent experience has taught me that this expectation is wrong: compared with building GUIs in Matlab (my only previous GUI-writing experience), Julia already offers a number of quite compelling advantages. Well see some of these advantages on display below.Well go through the highlights needed to create an image viewer GUI. Before getting into how to write this GUI, first lets play with it to get a sense for how it works. Its best if you just try these commands yourself, because its difficult to capture things like interactivity with static text and pictures.Youll need the ImageView package:Pkg.add("ImageView")Its worth pointing out that this package is expected to evolve over time; however, if things have changed from whats described in this blog, try checking out the blog branch directly from the repository. I should also point out that this package was developed on the authors Linux system, and its possible that things may not work as well on other platforms.First lets try it with a photograph. Load one this way:using Imagesusing ImageViewimg = imread("my_photo.jpg")Any typical image format should be fine, it doesnt have to be a jpg. Now display the image this way:display(img, pixelspacing = [1,1])The basic command to view the image is display. The optional pixelspacing input tells display that this image has a fixed aspect ratio, and that this needs to be honored when displaying the image. (Alternatively, you could set img["pixelspacing"] = [1,1] and then you wouldnt have to tell this to the display function.)You should get a window with your image:photoOK, nice. But we can start to have some fun if we resize the window, which causes the image to get bigger or smaller:photoNote the black perimeter; thats because weve specified the aspect ratio through the pixelspacing input, and when the window doesnt have the same aspect ratio as the image youll have a perimeter either horizontally or vertically. Try it without specifying pixelspacing, and youll see that the image stretches to fill the window, but it looks distorted:display(img)photo(This wont work if youve already defined "pixelspacing" for img; if necessary, use delete!(img, "pixelspacing") to remove that setting.)Next, click and drag somewhere inside the image. Youll see the typical rubberband selection, and once you let go the image display will zoom in on the selected region.photo photoAgain, the aspect ratio of the display is preserved. Double-clicking on the image restores the display to full size.If you have a wheel mouse, zoom in again and scroll the wheel, which should cause the image to pan vertically. If you scroll while holding down Shift, it pans horizontally; hold down Ctrl and you affect the zoom setting. Note as you zoom via the mouse, the zoom stays focused around the mouse pointer location, making it easy to zoom in on some small feature simply by pointing your mouse at it and then Ctrl-scrolling.Long-time users of Matlab may note a number of nice features about this behavior:The resizing and panning is much smoother than Matlabs

Matlab doesnt expose modifier keys in conjunction with the wheel mouse, making it difficult to implement this degree of interactivity

In Matlab, zooming with the wheel mouse is always centered on the middle of the display, requiring you to alternate between zooming and panning to magnify a particular small region of your image or plot.

These already give a taste of some of the features we can achieve quite easily in Julia.However, theres more to this GUI than meets the eye. You can display the image upside-down withdisplay(img, pixelspacing = [1,1], flipy=true)or switch the x and y axes withdisplay(img, pixelspacing = [1,1], xy=["y","x"])photo photoTo experience the full functionality, youll need a 4D image, a movie (time sequence) of 3D images. If you dont happen to have one lying around, you can create one via include("test/test4d.jl"), where test means the test directory in ImageView. (Assuming you installed ImageView via the package manager, you can say include(joinpath(Pkg.dir(), "ImageView", "test", "test4d.jl")).) This creates a solid cone that changes color over time, again in the variable img. Then, type display(img). You should see something like this:GUI snapshotThe green circle is a slice from the cone. At the bottom of the window youll see a number of buttons and our current location, z=1 and t=1, which correspond to the base of the cone and the beginning of the movie, respectively. Click the upward-pointing green arrow, and youll pan through the cone in the z dimension, making the circle smaller. You can go back with the downward-pointing green arrow, or step frame-by-frame with the black arrows. Next, clicking the play forward button moves forward in time, and youll see the color change through gray to magenta. The black square is a stop button. You can, of course, type a particular z, t location into the entry boxes, or grab the sliders and move them.If you have a wheel mouse, Alt-scroll changes the time, and Ctrl-Alt-scroll changes the z-slice.You can change the playback speed by right-clicking in an empty space within the navigation bar, which brings up a popup (context) menu:GUI snapshot

By default, display will show you slices in the xy-plane. You might want to see a different set of slices from the 4d image:display(img, xy=["x","z"])Initially youll see nothing, but thats because this edge of the image is black. Type 151 into the y: entry box (note its name has changed) and hit enter, or move the y slider into the middle of its range; now youll see the cone from the side.GUI snapshotThis GUI is also useful for plain movies (2d images with time), in which case the z controls will be omitted and it will behave largely as a typical movie-player. Likewise, the t controls will be omitted for 3d images lacking a temporal component, making this a nice viewer for MRI scans.Again, we note a number of improvements over Matlab:When you resize the window, note that the controls keep their initial size, while the image fills the window. With some effort this behavior is possible to achieve in Matlab, but (as youll see later in these posts) its essentially trivial with Julia and Tk.

When we move the sliders, the display updates while we drag it, not just when we let go of the mouse button.

If you try this with a much larger 3d or 4d image, you may also notice that the display feels snappy and responsive in a way thats sometimes hard to achieve with Matlab.

Altogether advantages such as these combine to give a substantially more polished feel to GUI applications written in Julia.This completes our tour of the features of this GUI. Now lets go through a few of the highlights needed to create it. Well tackle this in pieces; not only will this make it easier to learn, but it also illustrates how to build re-useable components. Lets start with the navigation frame.First steps: the navigation frameFirst, let me acknowledge that this GUI is built on the work of many people who have contributed to Julias Cairo and Tk packages. For this step, well make particular use of John Verzanis contribution of a huge set of convenience wrappers for most of Tks widget functionality. John wrote up a nice set of examples that demonstrate many of the things you can do with it; this first installment is essentially just a longer example, and wont surprise anyone who has read his documentation.Lets create a couple of types to hold the data well need. We need a type that stores GUI state, which here consists of the currently-viewed location in the image and information needed to implement the play functionality:type NavigationState# Dimensions:zmax::Int # number of frames in z, set to 1 if only 2 spatial dimstmax::Int # number of frames in t, set to 1 if only a single imagez::Int # current position in z-stackt::Int # current moment in time# Other state data:timer # nothing if not playing, TimeoutAsyncWork if we arefps::Float64 # playback speed in frames per secondendNext, lets create a type to hold handles to all the widgets:type NavigationControlsstepup # z buttons...stepdownplayupplaydownstepback # t buttons...stepfwdplaybackplayfwdstopeditz # edit boxeseditttextz # static text (information)texttscalez # scale (slider) widgetsscaletendIt might not be strictly necessary to hold handles to all the widgets (you could do everything with callbacks), but having them available is convenient. For example, if you dont like the icons I created, you can easily initialize the GUI and replace, using the handles, the icons with something better.Well talk about initialization later; for now, assume that we have a variable state of type NavigationState that holds the current position in the (possibly) 4D image, and ctrls which contains a fully-initialized set of widget handles.Each button needs a callback function to be executed when it is clicked. Lets go through the functions for controlling t. First there is a general utility not tied to any button, but it affects many of the controls:function updatet(ctrls, state)set_value(ctrls.editt, string(state.t))set_value(ctrls.scalet, state.t)enableback = state.t > 1set_enabled(ctrls.stepback, enableback)set_enabled(ctrls.playback, enableback)enablefwd = state.t < state.tmaxset_enabled(ctrls.stepfwd, enablefwd)set_enabled(ctrls.playfwd, enablefwd)endThe first two lines synchronize the entry box and slider to the current value of state.t; the currently-selected time can change by many different mechanisms (one of the buttons, typing into the entry box, or moving the slider), so we make state.t be the authoritative value and synchronize everything to it. The remaining lines of this function control which of the t navigation buttons are enabled (if t==1, we cant go any earlier in the movie, so we gray out the backwards buttons).A second utility function modifies state.t:function incrementt(inc, ctrls, state, showframe)state.t += incupdatet(ctrls, state)showframe(state)endNote the call to updatet described above. The new part of this is the showframe function, whose job it is to display the image frame (or any other visual information) to the user. Typically, the actual showframe function will need additional information such as where to render the image, but you can provide this information using anonymous functions. Well see how that works in the next installment; below well just create a simple stub function.Now we get to callbacks which well bind to the step and play buttons:function stept(inc, ctrls, state, showframe)if 1 stept(inc, ctrls, state, showframe))start_timer(state.timer, iround(1000*dt), iround(1000*dt))endstept() increments the t frame by the specified amount (typically 1 or -1), while playt() starts a timer that will call stept at regular intervals. The timer is stopped if play reaches the beginning or end of the movie. The stop_playing! function checks to see whether we have an active timer, and if so stops it:function stop_playing!(state::NavigationState)if !is(state.timer, nothing)stop_timer(state.timer)state.timer = nothingendendAn alternative way to handle playback without a timer would be in a loop, like this:function stept(inc, ctrls, state, showframe)if 1