Tracing Software Build Processes to Uncover License Compliance Inconsistencies: A Retrospective

Tracing Software Build Processes to Uncover License Compliance Inconsistencies

Shane McIntosh

Sander van der Burg

Eelco Dolstra

Julius Davies

Daniel M. Germán

Armijn Hemel

TjaldurSoftware Governance

Solutions

@shane_mcintosh

Source code

What is a build system?

Source code

Deliverable

What is a build system?

.tex

.c

.cc

.o

.o

.dvi

.a

.exe

.pdf

.deb

Build systems describe how sources aretranslated into deliverables

3

Continuous Integration:Enabled by the

build system

4

.c .mk


build system

Commit

4

Commit 9719cf0

.c .mk


build system

Commit

4

BuildCommit 9719cf0

.c .mk


build system

Commit

4

Build

Test

Commit 9719cf0

.c .mk


build system

Commit

4

Build

Test

ReportCommit 9719cf0 was successfully integrated

Commit 9719cf0

.c .mk

5

There [is] no such thing as a free lunch

“ ”

5

There [is] no such thing as a free lunch

“ ”An Empirical Study of

Build Maintenance Effort

S. McIntosh, B. Adams, T. H. D. Nguyen,

Y. Kamei, A. E. Hassan [ICSE 2011]

Up to 27% of source changes

are accompanied by build changes

6

Maintenance overhead

6

Maintenance overhead

.c .mk?

Source-build co-change

6

Maintenance overheadBuild technology and maintenance

.c .mk?


6

Maintenance overheadBuild logic

cloningBuild technology and maintenance

.c .mk?


6

Execution overhead



.c .mk?


6

Execution overhead



.c .mk?


Build hotspot detection

6

Execution overhead



.c .mk?


Powerful hotspot indicators


6

Execution overhead



.c .mk?


Powerful hotspot indicators


Build systems also contain useful information!

Reusable components are released under different license terms

7

Reusable components are released under different license terms

7

Apache Public License

Failure to comply with license terms can lead to costly legal issues

8

9

Which source files are enabled?

Ensuring license compliance with reused components

.c.c.c.c

9



.c.c.c

.c

9


Which components are used?


.c.c.c

.c

9



How are they combined?


.c.c.c

.c

Static link

Dynamic link

9



How are they combined?


.c.c.c

.c

Static link

Dynamic link

The build system cananswer these questions!

We use system tracing todiscover build dependencies

Build process

10

Tracelog

OS kernel

open()

We use system tracing todiscover build dependencies

Build process

10

read()

write()

close()

Tracelog

Tracelog

We mine build traces to construct a concrete build dependency graph

patchelf.ccelf.h

patchelf.o

patchelf

/usr/bin/patchelflibstdc++

11

Tracelog

We mine build traces to construct a concrete build dependency graph

patchelf.ccelf.h

patchelf.o

patchelf

/usr/bin/patchelf

g++

libstdc++

g++install

11

patchelf.ccelf.h

patchelf.o

patchelf

/usr/bin/patchelf

g++

libstdc++

g++install

Annotate build graph nodes with license information using Ninka

12

Inconsistency introduced!

patchelf.ccelf.h

patchelf.o

patchelf

/usr/bin/patchelf

g++

libstdc++

g++install

Annotate build graph nodes with license information using Ninka

12

Empirical study

13

Empirical study(RQ1)

Accuracy

13


Accuracy(RQ2)

Practicality

13

14

Measuring the accuracyof our CBDG approach

Included .c.c.c.c

Excluded

14


Included .c.c.c

.cExcluded

14


Included .c.c .c

.cExcluded

Delete

14


Included .c.c .c

.cExcluded

Delete

Execute build

14

Broken means true positive


Included .c.c .c

.cExcluded

Delete

Execute build

14

Clean means false positive



Included .c.c .c

.cExcluded

Delete

Execute build

14




Included .c.c .c

.cExcluded

Delete

Execute build

.c

14




Included .c.c .c

.cExcluded

Delete

Execute build

Execute build

.c

14




Included .c.c .c

.cExcluded

Delete

Execute build

Broken means false negative

Execute build

.c

14




Included .c.c .c

.cExcluded

Delete

Execute build

Clean means true negative

Broken means false negative

Execute build

.c


Accuracy

Precision: 88%-100%

Recall: 98%-100%

(RQ2)Practicality

15

Bugs filed using our approachon multi-licensed packages

FFmpeg

Licensewas updatedwithin 3 days

+

16

Bugs filed using our approachon multi-licensed packages

FFmpeg

Licensewas updatedwithin 3 days

+

CUPS

+

Offending files were removed within 2 days

16


Accuracy

Precision: 88%-100%

Recall: 98%-100%

(RQ2)Practicality

Prompted quick code changes in two systems

17

Tracing Software Build Processes to Uncover License Compliance Inconsistencies: A Retrospective

Software

Transcript of Tracing Software Build Processes to Uncover License Compliance Inconsistencies: A Retrospective