MicroTESK: An Extendable Framework for

Test Program GenerationAlexander Kamkin, Tatiana Sergeeva, Andrei Tatarnikov,

Artemiy Utekhin{kamkin, leonsia, andrewt, utekhin}@ispras.ru

Institute for System Programming of the Russian Academy of Sciences (ISPRAS)http://hardware.ispras.ru

SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

CPU design

Hardware Description Languages (HDL) Verilog VHDL

SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

CPU verification

Target CPU(HDL)

lui s1, 0xdeadori s1, s1, 0x0lui s3, 0xbeefori s3, s3, 0xfadd v0, a0, a2sub t1, t3, t5add t7, s1, s3

Test programs(assembler)

Reference simulator (C/C++)

Execution traces (formatted text)

0x2000: lui ...0x2004: ori ...0x2008: ori ...0x200c: lui ...0x2010: add ...0x2014: sub ...0x2018: add ...

Trace comparator

(Perl, Python)

0x2000: lui ...0x2004: ori ...0x2008: ori ...0x200c: lui ...0x2010: add ...0x2014: sub ...0x2018: add ...

SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

Test program generation

• Random (RAVEN)• Combinatorial (MicroTESK v1)• Template-based (Genesys-Pro)• Model-based (research projects)

Requirements

Design ?Test programs

SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

Framework

?

SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

Model-based

Random

Template-based

Combinatorial

Modeling Library

MicroTESK frameworkModeling Framework

Testing Framework

Translator

Coverage Extractor Design Library

Test Template ProcessorTesting Library

Coverage Library

Constraint Solver Engine

Specifications

Test Templates

Test ProgramsExternal Solvers

Test Sequence Generators

Test Data Generators

Model Generator

Coverage ModelDesign ModelModel

MicroTESK plugin

Design Aspect

Instruction Set

Memory Management

Core

Extensions

Framework Plugin

Translator

Solvers

Library

Test Generators

Modeling

Testing

SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

Pipelining

User Defined

Specification exampleop ADD(rd:GPR, rs:GPR, rt:GPR)action = { if(NotWordValue(rs) || NotWordValue(rt)) then UNPREDICTABLE(); endif; tmp_word = rs<31..31>::rs<31..0> + rt<31..31>::rt<31..0>; if(tmp_word<32..32> != tmp_word<31..31>) then SignalException("IntegerOverflow"); else rd = sign_extend(tmp_word<31..0>); endif;}syntax = format("add %s, %s, %s", rd.syntax, rs.syntax, rt.syntax)

op ALU = ADD | SUB | …

Precondition

Test Situations

Equivalence Classes

SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

Template example# Assembly -Style Codeadd r[1], r[2], r[3]sub r[1], r[1], r[4]

# Ruby Control Statements(1..3).each do |i|

add r[i], r[i+1], r[i+2]sub r[i], r[i], r[i+3]

end

# Test Sequence Blockblock (:engine => ”random”, :count => 2013){

add r[1], r[2], r[3]sub r[1], r[2], r[3]

# Test Situation Referencedo overflow end

}

SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

Test sequence generator# Test Sequence Blockblock (:combine => ”product”, :compose => ”random”) { # Nested Block A block (:engine => ”random”, :length => 3, :count => 2) { add r[a], r[b], r[c] sub r[d], r[e], r[f] mult r[g], r[h] div r[i], r[j] } # Nested Block B block (:engine => ”permutate” ) { ld r[k], r[l] st r[m], r[n] }}

# Combination (1, 1)sub r[d], r[e], r[f] # Block Ald r[k], r[l] # Block Bdiv r[i], r[j] # Block Ast r[m], r[n] # Block Badd r[a], r[b], r[c] # Block A# Combination (1, 2)st r[m], r[n] # Block Bsub r[d], r[e], r[f] # Block Ald r[k], r[l] # Block Bdiv r[i], r[j] # Block Aadd r[a], r[b], r[c] # Block A# Combination (2, 1)mult r[g], r[h] # Block Amult r[g], r[h] # Block Ald r[k], r[l] # Block Badd r[a], r[b], r[c] # Block Ast r[m], r[n] # Block B# Combination (2, 2)mult r[g], r[h] # Block Ast r[m], r[n] # Block Bmult r[g], r[h] # Block Ald r[k], r[l] # Block Badd r[a], r[b], r[c] # Block A

SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

• Combinator• Compositor

Constraint solver engine

Collection of solvers:UniversalCustom

SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

MicroTESK framework uses Java Constraint Solver API http://forge.ispras.ru/projects/solver-api

Conclusion

• Framework architecture

• Support for common generation techniques

• Extendibility with other methods

• Flexible open-source environment

SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

Contact information• Institute for System Programming of RAS (ISPRAS) http://www.ispras.ru• MicroTESK Test Program Generator

http://forge.ispras.ru/projects/microtesk• Tatiana Sergeeva

leonsia@ispras.ru

SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

Thank you!Questions?

SYRCoSE 2013 – May 30, 2013 – Kazan, Russia