BASIS IntroductionBuild system And Software Implementation Standard

by Andreas Schuh and Andrew Hundt

Copyright © 2011 University of Pennsylvania. Copyright © 2013 Carnegie Mellon University.

2

BASIS Introduction Outline Introduction

Project Template

CMake Modules

Utilities

Milestones

Tutorials11/22/2013

3

IntroductionWhy BASIS? What is it?

11/22/2013

4

BASIS Goals

The lifetime of your software should be longer than the time you or your team spends on it. Therefore BASIS aims to: establish a unified software structure. ease package integration. streamline maintenance. Make software more robust and reliable. Increase the impact of your work!

11/22/2013

5

Approach

Specify standards/conventions for the software development process.

Standardize project organization and software build including testing and packaging.

Provide basic utility functions in order to promote compliance with lab conventions and to reduce common development burden.

11/22/2013

6

Vision

11/22/2013

BASIS

…

Toolkit

ODVBA

COMPAREPREDICT

GLISTR

MICO

Integrated Distribution Package

Projects• COMPARE• PREDICT• …

Toolkit

BASIS

7

Ah, another Toolkit…

BASIS is not simply a toolkit. It consists of:

Standardized project specifications on the website

A preset project template

Utility functions for integration between multiple supported programming languages.

However, the distinction may be fuzzy at times…11/22/2013

8

BASIS Overview

11/22/2013

Standard Specifications

Template

File

syst

em

H

iera

rchy

Tem

pla

te F

iles

CMake Modules

Basi

sPro

ject

.cm

ake

(CM

ake

)

Basi

sTest

.cm

ake

(CTe

st)

Basi

sPack

.cm

ake

(CPa

ck)

Tools

Pro

ject

Cre

ati

on

Auto

mate

d

Test

ing

Docu

menta

tion

Utilities

Com

mand

Exe

cuti

on

Com

mand-l

ine

Pars

ing

Test

ing

Fram

ew

ork

s

BASIS IntroductionCopyright (c) 2011 University of Pennsylvania. All rights reserved. 9

Project TemplateMad-libs style project creation

11/22/2013

10

basisproject

basisproject is a command line program that: Automatically generates a package source

directory Populates it with starter files using mad-lib

style substitution to customize it for you in seconds.

11/22/2013

11

Template

A directory with all mad-libs style template files, i.e., the project files before substitution

Used by basisproject Follows the BASIS project layout

standard For an example, see the

data/templates subdirectory of the BASIS source package.

11/22/2013

BASIS IntroductionCopyright (c) 2011 University of Pennsylvania. All rights reserved. 12

<source> is the root directory of the source tree.

<project> is the project name in lowercase only.

Source Code Filesystem Hierarchy

<source>/ config/

data/

doc/exampl

e/include/ <projec

t>/src/

test/

11/22/2013

Installation Filesystem Hierarchy

11/22/2013

BASIS Introduction Copyright (c) 2011 University of Pennsylvania. All rights reserved. 13

<prefix> is the installation directory, e.g., /usr/local/ on Unix.

<sinfix> is a project- and possibly version-specific string, e.g., <project>

<prefix>/

bin/ <sinfix>/

include/ sbia/ <project>/

lib/

cmake/ <project>/

<sinfix>/

perl5/ 5.x.x/ <project>/

pythonx.x/ site-packages/ <project>/

share/ <sinfix>/

doc/

example/

…

14

Filesystem Hierarchy

Main executables go in <prefix>/bin/<sinfix>/

Auxiliary executables go in <prefix>/lib/<sinfix>/.

Symbolic links created upon installation by default From

▪ <prefix>/bin/ To (selected) executables in

▪ <prefix>/bin/<sinfix>/.

More details in the specification.11/22/2013

15

Build Standard

11/22/2013

BASIS CMake Modules

16

BASIS extends and replaces most CMake commands. add_executable() basis_add_executable()

Supports multiple languages, not just C++: Java (not yet), Python, Perl, BASH, and MATLAB.

Implements the BASIS standard, adhering guidelines Filesystem hierarchy Software documentation Packaging Testing

CMake Modules

11/22/2013

17

CMake Modules

The main modules are: BasisProject.cmake▪ Must be part of every project. Defines meta-data.

BasisTest.cmake – Extends CTest.cmake.▪ Used for software tests.▪ Included by basis_project_initialize().

BasisPack.cmake – Extends CPack.cmake.▪ Used for packaging.▪ Included by basis_project_finalize().

11/22/2013

18

CMake Modules

The most important functions to know are:

basis_find_package() basis_add_executable() basis_add_library() basis_add_test() basis_add_doc() basis_include_directories() basis_target_link_libraries() basis_set_target_properties() basis_get_target_property() basis_install()

11/22/2013

19

CMake Example

<source>/config/Depends.cmake# find NIfTI-1 tools for MATLABbasis_find_package (MatlabNiftiTools REQUIRED)

basis_add_include_directories (“${MatlabNiftiTools_INCLUDE_DIRS}”)

<source>/src/CMakeLists.txt# MEX-file build from C++basis_add_library (look_for_gold_instead MEX look_for_gold.cxx)

# executable build from MATLAB scriptsbasis_add_executable (miner neurominer.m)basis_target_link_libraries (miner look_for_gold_instead)

11/22/2013

20

CMake Modules

See the API documentation of BASIS for details on the CMake variables, functions, and macros.

Documentation can be automated so it is generated automatically every night (less frequently in later stages) from the in-source code comments using Doxygen.

11/22/2013

21

ToolsWhat tools do we get?

11/22/2013

22

basisproject

Create a new BASIS project

Modify an existing project.

Upgrade a project to a newer version of BASIS.

3-way file merge combines your changes with the changes in the newer BASIS template.

Uses TEMPLATE argument of basis_project to determine currently used template version.

11/22/2013

23

basistest

Wrapper for command-line tools implemented for the automated execution of software tests:

The basistest_cron is executed, e.g., every 5 minutes. It in turn just calls basistest_master with the

arguments specific to the given system. A configuration file tells the basistest_master what

branches of which projects to test at which time interval,…

The actual tests are executed by instances of basistest_slave which are submitted to SGE in our case and am using CTest.11/22/201

3

24

basistest

basistest can create CDash dashboard of test results for two testing regimens: Continuous▪ Executed after a change was committed to the

repository.

Nightly▪ Run once every certain time interval, e.g., every

night.

11/22/2013

25

Documentation Standard

BASIS automates the process of setting up documentation

Doxygen is used for general code documentation Natively, Doxygen knows how to parse C/C++, Java, and Python

(not satisfactory, though)

Filters can be used to translate other languages into something Doxygen understands best: C++. BASIS includes such filters for the missing languages, i.e.(, Python), Perl, BASH, and MATLAB.

This is transparent to the BASIS developer.

A simple basis_add_doc(api GENERATOR Doxygen) in the build CMakeLists.txt is sufficient.

11/22/2013

26

Documentation Standard

See the API documentation of the BASIS utilities for examples of documentation generated fromin-source comments only.

11/22/2013

27

UtilitiesWhat implementations do they provide?

11/22/2013

28

Command Execution Standard

Standardizes application startup and conventions for programs to interact.

Main commands commonly execute several utility commands in *nix and research environments.

Also need to ensure the right commands are executed and the correct libraries are loaded.

11/22/2013

29

Command Execution Standard Calling Conventions aim to ensure

executables are referenced by build target name rather than the filename to avoid ambiguity and errors.

BASIS provides utility functions for each supported language to do just that in a safe and reusable way.

11/22/2013

30

Command-line Parsing

Any command-line tool needs to parse command-line arguments.

BASIS requires each executable to output a help screen on --help (-h)

a shorter usage information on ––helpshort and its version and copyright on --version.

11/22/2013

31

Command-line Parsing

BASIS provides command-line parsing libraries

Implemented for each supported language.

These libraries create a help screen from brief descriptions provided for each option. No need to format the help screen manually. Able to output a man page or XML description. Such help is more likely to be up-to-date rather

then being forgotten when an option was renamed, removed, or added…

11/22/2013

32

Command-line Parsing

C++ TCLAP Gflags Boost::program_options

Java args4j

Python argh (wrapper for argparse)

Perl Getopt::Tabular Getopt::Declare

BASH shflags (extended shFlags)

MATLAB none available yet11/22/201

3

See Command-line Parsing documentation on the BASIS website for more details.

33

Testing Frameworks

C++ gtest gmock libtap++ Boost.Test

Java JUnit JTap

Python unittest PyTap

Perl Test::Simple Test::More Test::MockObject

BASH shtap (slightly modified tap-functions library)

MATLAB MATLAB xUnit Test Framework

11/22/2013

Already part of Perl installation.

Test Anything Protocol (TAP) Producers

Common use and standardized output which can be processed, for example,by Test::Harness written in Perl.

Built into Python language.

34

MilestonesWhat’s changed?

11/22/2013

35

Releases 0.1 through 2.1.4

See ChangeLog of BASIS distributed by SBIA

11/22/2013

36

Future Release: 3.0

BASIS Moves to GitHub and expands beyond the Section of Biomedical Image Analysis (SBIA) at the University of Pennsylvania.

3.0 strips of some SBIA specific defaults to allow more customization required by other organizations.

Everything organization specific shall be separated from the main functionality and be configurable.

Default templates will be available that should provide a good starting point for anyone.

11/22/2013

37

Check GitHub for the latest developments

Read More▪ https://github.com/schuhschuh/cmake-basis

Get the Source▪ git clone https://github.com/schuhschuh/cmake-basis.git

11/22/2013

Beyond the Horizon

38

Feedback

Remember, your participation and feedback is required to improve BASIS!

What do you like? What concerns you? What hinders you? What is missing?

Need tutorial-like examples to go more into details regarding certain aspects with hands-on examples

Post questions and ideas on GitHub Issue tracker▪ https://github.com/schuhschuh/cmake-basis/issues

Wiki▪ https://github.com/schuhschuh/cmake-basis/wiki 11/22/201

3

39

Thank you

11/22/2013