vampyr: Con gurability-Aware Compile-Testing of Source Files · vampyr: Con gurability-Aware...

vampyr: Configurability-AwareCompile-Testing of Source Files

Stefan [email protected]

Daniel [email protected]

System Software GroupFriedrich-Alexander University Erlangen-Nürnberg (FAU)

https://cados.cs.fau.de

LPC ’14

supported by

mailto:[email protected]:[email protected]://cados.cs.fau.de

Kernel Patch Submission Checklist

Before submitting patches kernel developers are expected to follow:

. . .

8 Has been carefully reviewed with respect to relevant Kconfigcombinations. This is very hard to get right with testing – brainpowerpays off here.

. . .

why is that a problem?

undertaker.cs.fau.de vampyr: Configurability-AwareCompile-Testing of Source Files (LPC ’14) 2 – 18

undertaker.cs.fau.de

Configurability-aware compile testing

Compile-test block1 and block2

#i f d e f CONFIG Ab l o c k 1

#e l s eb l o c k 2

#e n d i f

However, one .config cannot cover both blocks

Code is often compile-tested with one allyesconfig

Bugs are easily missed!



How hard is the problem?

Configurability:

#ifdef CONFIG_HOTPLUG_CPU...#endif

Makefilearch/x86/init.carch/x86/entry32.Sarch/x86/...lib/Makefilekernel/sched.c. . .

MEMORY MODEL

FLATMEM

DISCONTIGMEM

SPARSEMEM NUMA

depends on



How hard is the problem?

Configurability:



MEMORY MODEL

FLATMEM

DISCONTIGMEM

SPARSEMEM NUMA

depends on

Still solvable by brainpower?

How much time does it take to get it right?



Configuration Coverage: The vampyr Approach

Idea: Create a maximizing set of configurations

MEMORY MODEL

FLATMEM

DISCONTIGMEM

SPARSEMEM NUMA

depends on



calculateconfigurations

for each source fileConfig 1

Config 2

Config ...

Config n

apply sparsegcc

clangcoccinelle



Evaluation: A Configurability-Aware Tool Driver

Evaluated for Linux/v3.2 a

Number of found compiler warnings and errors increased significantly

Architecture Increase in detected gcc warnings and errors

Linux/x86 176 → 202 (+15%)Linux/mips 158 → 249 (+58%)

Linux/arm: Analysis of Warnings and Errors not found withallyesconfig

Σ Less critical gcc messages 223 → 363 (+63%)

Σ Reported issues by vampyr 254 → 454 (+79%)

Σ Manually validated bugs 31 → 91

ahttps://www4.cs.fau.de/Publications/2014/tartler_14_usenix.pdf


https://www4.cs.fau.de/Publications/2014/tartler_14_usenix.pdfundertaker.cs.fau.de

Evaluation: A Configurability-Aware Tool Driver

Evaluated for Linux/v3.2 a

Number of found compiler warnings and errors increased significantly

Architecture Increase in detected gcc warnings and errors

Linux/x86 176 → 202 (+15%)Linux/mips 158 → 249 (+58%)

Linux/arm: Analysis of Warnings and Errors not found withallyesconfig

Σ Less critical gcc messages 223 → 363 (+63%)

Σ Reported issues by vampyr 254 → 454 (+79%)

Σ Manually validated bugs 31 → 91

ahttps://www4.cs.fau.de/Publications/2014/tartler_14_usenix.pdf

Luckily:The number of found warnings and errors

is lower in Linux/v3.17


https://www4.cs.fau.de/Publications/2014/tartler_14_usenix.pdfundertaker.cs.fau.de

Example 1 (v3.17 - MIPS)

===== Found 1 messages with gcc in arch/mips/ath79/mach-db120.c =====. . . mach-db120.c:132: error: too many arguments to function ’ db120 pci init’(in configs: arch/mips/ath79/mach-db120.c.config1)=============================================

#ifdef CONFIG_PCIstatic void __init db120_pci_init(u8 *eeprom) { [...] }#elsestatic void __init db120_pci_init(void) {}#endif

static void __init db120_setup(void) {[...]db120_pci_init(art + DB120_PCIE_CALDATA_OFFSET);

}

Conclusions

intentional broken compilation?

why not #error?

or model configurability better



Example 2 (v3.17 - ARM)

==== Found 8 messages with gcc in arch/arm/mm/dma-mapping.c ====. . . dma-mapping.c:1358: error: ’ atomic pool’ undeclared (first use in this function)(in configs: arch/arm/mm/dma-mapping.c.config1). . . dma-mapping.c:1369: error: implicit declaration of function ’ in atomic pool’(in configs: arch/arm/mm/dma-mapping.c.config1) . . .==========================================

#ifdef CONFIG_MMUstatic struct gen_pool *atomic_pool;static bool __in_atomic_pool ([...]) {

return addr_in_gen_pool(atomic_pool , [...]);}#endif

static struct page ** __atomic_get_pages(void *addr) {phys = gen_pool_virt_to_phys(atomic_pool , [...]);

}static struct page ** __iommu_get_pages ([...]) {

if (__in_atomic_pool(cpu_addr , PAGE_SIZE))return __atomic_get_pages(cpu_addr);

}

Conclusions

#ifdef around ’uses’ missing or declaration should have been unconditional










}

declared inside #ifdef

used outside #ifdef

Conclusions











}

Conclusions











}

defined inside #ifdef

used outside #ifdef

Conclusions




Example 3 (v3.17 - MIPS)

==== Found 1 messages with gcc in arch/mips/pmcs-msp71xx/msp irq cic.c ====. . . msp irq cic.c:134: error: ’ irq’ undeclared (first use in this function)(in configs: arch/mips/pmcs-msp71xx/msp irq cic.c.config0)=============================================

#ifdef CONFIG_MIPS_MT_SMPstatic int msp_cic_irq_set_affinity(struct irq_data *d,

[...]) {unsigned long imask = (1 irq



Compile Testing

“ Major problem I see is that many architecturemaintainers don’t seem to care about make allmod-config and/or make allyesconfig, meaning thereis no simple means to at least compile-test all codethat can be enabled for a given architecture. Anddon’t even mention make randconfig.”Guenter Roeck



Compile Testing

“ Major problem I see is that many architecturemaintainers don’t seem to care about make allmod-config and/or make allyesconfig, meaning thereis no simple means to at least compile-test all codethat can be enabled for a given architecture. Anddon’t even mention make randconfig.”Guenter Roeck

Integrate vampyr intoyour development workflow!replace brainpower by tools!



Summary and Conclusions

vampyr is a tool to help developers to compile-test their source code

The tool is available under GPLv3!

vampyr has been applied to other configurable system software:busybox, L4/Fiasco

Integration into Undertaker–CheckPatch is on the way



Interested?

Download and try the tool:

https://undertaker.cs.fau.de

More information and papers on the project’s website:

https://cados.cs.fau.de

Questions? Contact me directly ...

[email protected]

... or write to our mailing list!

[email protected]


https://undertaker.cs.fau.dehttps://cados.cs.fau.demailto:[email protected]:[email protected]

Backup Slides Start

Evaluation: Analysis of Warnings and Errors

vampyr reveals issues not covered by allyesconfig:

Less critical gcc messages Compiler DiagnosticsΣ Less critical messages 223 → 363 +140

Manually validated bugsUndeclared types/identifiers 4 → 46 +42Access to possibly uninitialized data 20 → 22 +2Out of bounds array accesses 7 → 13 +4Bad pointer casts 0 → 8Format string warnings 0 → 1Integer overflows 0 → 1Σ Bugs found 31 → 91 +60

Σ Reported issues by vampyr 254 → 454 +200

Seven patches were submitted and accepted by Linux maintainers



Configuration Coverage

Common approach: use a single configuration (i.e. allyesconfig)

Configuration Coverage is the percentage of code that is covered byall tested configurations

This is insufficient because for allyesconfig:

Configuration Coverage on v3.2 v3.17-rc7

Linux/x86 78.6% 77%Linux/arm 59.7% 69.5%

Linux/mips 54.6% 52.4%



Evaluation: Setup and Runtime Requirements

Application of vampyr on all 24 Linux Architectures of Linux v3.2

Used Static Checker: gcc: 4.7

On average, 1.2 compiler invocations per file

Overhead ∼ 20%

Runtime on a Standard Intel Quad-Core Workstation:

Incremental analysis of an individual file: < 1 minute

Generation of 11470 (on Linux/x86) partial configurations in ∼ 4 minutesAnalysis of a full architecture: ∼ 2 hoursMajority of time is spent with activating Kconfig configurations



Why not 100 percent Configuration coverage?

Bugs in Kconfig descriptions in the Linux kernel can causeincorrect expansions of partial configurations.

Imperfect model extractors can also lower the ConfigurationCoverage



Higher Coverage Criteria

vampyr: statement coverage (∼ 20% overhead)

Possibly achievable: decision coverage: (∼ 29% overhead)

Expensive: path coverage: (?? overhead)

Further research: Pairwise testing (cf. related work)


vampyr: Con gurability-Aware Compile-Testing of Source Files · vampyr: Con gurability-Aware...

Documents

Transcript of vampyr: Con gurability-Aware Compile-Testing of Source Files · vampyr: Con gurability-Aware...