A Little Language for Surveys: A Little Language for Surveys: Constructing an Internal DSL Constructing an Internal DSL

in Rubyin Ruby

H. Conrad CunninghamH. Conrad Cunningham

Computer and Information ScienceComputer and Information ScienceUniversity of MississippiUniversity of Mississippi

22

Domain-Specific Language (DSL)Domain-Specific Language (DSL)

““programming language or executable programming language or executable specification language that offers, through specification language that offers, through appropriate notations and abstractions, appropriate notations and abstractions, expressive power focused on, and usually expressive power focused on, and usually restricted to, a particular problem domainrestricted to, a particular problem domain”” – van Deursen, Klint, and Visser– van Deursen, Klint, and Visser

““little languagelittle language”” – J. Bentley – J. Bentley

33

Typical DSL Characteristics Typical DSL Characteristics

Not general-purpose (often not Turing-Not general-purpose (often not Turing-complete)complete)

Small (at least initially)Small (at least initially) DeclarativeDeclarative Semantically expressive of domainSemantically expressive of domain Targeted at domain specialists, Targeted at domain specialists,

not necessarily programmersnot necessarily programmers

44

External or InternalExternal or Internal

External:External: different from main different from main programming languageprogramming language– make, yacc, pic, XML-based config filesmake, yacc, pic, XML-based config files

Internal (embedded):Internal (embedded): constrained use of constrained use of the main programming languagethe main programming language– use of Lisp macros, Haskell algebraic types, use of Lisp macros, Haskell algebraic types,

metaprogramming in Ruby, etcmetaprogramming in Ruby, etc..

55

Defining Internal DSLs in Ruby Defining Internal DSLs in Ruby (1 of 2)(1 of 2)

Flexible syntax enables convenient DSL Flexible syntax enables convenient DSL statementsstatements– optional parentheses on method callsoptional parentheses on method calls– variable number of argumentsvariable number of arguments

questionquestion ““Male or female?Male or female?””

# has optional second arg# has optional second arg

66

Defining Internal DSLs in Ruby Defining Internal DSLs in Ruby (2 of 2)(2 of 2)

Blocks (closures) provide DSL structuring Blocks (closures) provide DSL structuring and delayed executionand delayed execution– anonymous function definitionsanonymous function definitions– passed as argumentspassed as arguments

question question ““Male or female?Male or female?”” dodo

response response ““MaleMale””

response response ““FemaleFemale”” do @fem = true enddo @fem = true end

endend

77

Implementing Internal DSLs in Ruby Implementing Internal DSLs in Ruby (1 of 2)(1 of 2)

Reflexive metaprogramming:Reflexive metaprogramming: writing programs that manipulate writing programs that manipulate themselves as data themselves as data

obj.instance_eval(str)obj.instance_eval(str) executes string executes string strstr as Ruby code in context of object as Ruby code in context of object objobj– execute DSL statements dynamicallyexecute DSL statements dynamically

mod.class_eval(str)mod.class_eval(str) executes string executes string str str as Ruby code in context of module as Ruby code in context of module modmod– declare new methods and classes dynamicallydeclare new methods and classes dynamically

88

Implementing Internal DSLs in Ruby Implementing Internal DSLs in Ruby (2 of 2)(2 of 2)

obj.method_missing(sym,*args)obj.method_missing(sym,*args) invoked when undefined method invoked when undefined method symsym called called with arguments with arguments argsargs– take remedial actiontake remedial action

obj.send(sym,*args)obj.send(sym,*args) calls method calls method symsym on object on object objobj with arguments with arguments argsargs– dynamically dynamically ““send a messagesend a message””

99

Little Language for SurveysLittle Language for Surveys

Domain: Specification and presentation Domain: Specification and presentation of simple, multiple-choice surveysof simple, multiple-choice surveys

Tasks:Tasks:

1.1. Analyze domain for language designAnalyze domain for language design

2.2. Design Ruby internal DSLDesign Ruby internal DSL

3.3. Implement DSLImplement DSL

Domain AnalysisDomain Analysis

1111

Commonality/Variability AnalysisCommonality/Variability Analysis

Determine domain boundaries (scope) Determine domain boundaries (scope) Define specialized terms and concepts Define specialized terms and concepts

(terminology)(terminology) Identify unchanging features among domain Identify unchanging features among domain

members (commonalities)members (commonalities) Identify features that may change among Identify features that may change among

domain members (variabilities)domain members (variabilities)

1212

ScopeScope

Support definition of simple, multiple-Support definition of simple, multiple-choice surveyschoice surveys– specification of surveyspecification of survey– presentation of survey and collection of presentation of survey and collection of

responsesresponses Exclude tabulation of aggregate results Exclude tabulation of aggregate results

for nowfor now

1313

Terms and CommonalitiesTerms and Commonalities

1.1. Survey has a Survey has a titletitle2.2. Survey has a Survey has a sequence of questionssequence of questions3.3. Question has a Question has a sequence of responses sequence of responses 4.4. Use ofUse of conditional question conditional question depends upon depends upon

previous responsesprevious responses5.5. Response to Response to silent questionsilent question determined from determined from

previous responsesprevious responses6.6. SurveySurvey execution execution presents appropriate presents appropriate

questions to questions to respondentrespondent and collects choices of and collects choices of responsesresponses

1414

VariabilitiesVariabilities

1.1. Texts for title, questions, and responsesTexts for title, questions, and responses2.2. Number and order of questions within surveyNumber and order of questions within survey3.3. Number of responses expected for questionNumber of responses expected for question4.4. Number and order of responses in questionNumber and order of responses in question5.5. Condition under which question includedCondition under which question included6.6. Method for answering silent questionMethod for answering silent question7.7. Source of survey specificationSource of survey specification8.8. Method for displaying questions and Method for displaying questions and

collecting responses during executioncollecting responses during execution

Internal DSL DesignInternal DSL Design

1616

DSL Design StrategyDSL Design Strategy

Adopt declarative approach – structure Adopt declarative approach – structure explicit but processing implicitexplicit but processing implicit

Use terminology and commonalities to Use terminology and commonalities to suggest language statementssuggest language statements

Represent variabilities as values that Represent variabilities as values that survey programmers supplysurvey programmers supply

1717

Survey DSL Example (1 of 3)Survey DSL Example (1 of 3)

# C1 survey has title (V1)# C1 survey has title (V1)titletitle ““ACMSE Conference SurveyACMSE Conference Survey””# C2 survey has seq of questions (V2)# C2 survey has seq of questions (V2)questionquestion ““Are you an author?Are you an author?”” do do # (V3)# (V3) # C3 question has seq of responses (V4)# C3 question has seq of responses (V4) responseresponse "Yes" do # execute if chosen "Yes" do # execute if chosen @author = true@author = true endend responseresponse ““NoNo”” do # execute if chosen do # execute if chosen @author = false @author = false endendendend

1818

Survey DSL Example (2 of 3)Survey DSL Example (2 of 3)

# C4 conditional question (V5)# C4 conditional question (V5)questionquestion "What type of paper?" do "What type of paper?" do conditioncondition { @author } # when execute? { @author } # when execute? responseresponse "Regular" do @p = :rg end "Regular" do @p = :rg end responseresponse "Student" do @p = :st end "Student" do @p = :st end responseresponse "Work-in-progress" do "Work-in-progress" do @p = :wp @p = :wp endend # V5/V6 block on response & action sets# V5/V6 block on response & action sets # state for conditions & silent choices# state for conditions & silent choices actionaction {@t = 25; @t = 15 if @p == :wp} {@t = 25; @t = 15 if @p == :wp}endend

1919

Survey DSL Example (3 of 3)Survey DSL Example (3 of 3)

# C5 silent question calculates choice (V6)# C5 silent question calculates choice (V6)resultresult "How long is presentation? "How long is presentation?““ dodo conditioncondition { @author } { @author } alternativealternative "25" do # when choose? "25" do # when choose? @p == :rp || @p == :sp @p == :rp || @p == :sp endend alternativealternative ““1515”” do # when choose? do # when choose? @p == :wp @p == :wp endendendend

Internal DSL ImplementationInternal DSL Implementation

1.1. Parse DSL and build abstract syntax Parse DSL and build abstract syntax tree (AST)tree (AST)

2.2. Execute survey by traversing ASTExecute survey by traversing AST

parse execute DSL

AST

2121

Two-Pass ImplementationTwo-Pass Implementation

2222

First Pass: DSL ParsingFirst Pass: DSL Parsing

Use Use instance_evalinstance_eval to execute DSL input to execute DSL input as Ruby code (V7)as Ruby code (V7)

Let Ruby interpreter do most of parsingLet Ruby interpreter do most of parsing Add methods for each DSL statement Add methods for each DSL statement

– – titletitle, , questionquestion, , actionaction, etc., etc. Check specialized syntax and build AST as Check specialized syntax and build AST as

3-level tree (survey, question, response)3-level tree (survey, question, response) Defer conditions and actions by storing Defer conditions and actions by storing

unevaluated blocks (i.e., closures)unevaluated blocks (i.e., closures)

2323

Second Pass: DSL InterpretationSecond Pass: DSL Interpretation

Traverse AST to display questions and Traverse AST to display questions and collect responsescollect responses

Execute deferred blocks needed for Execute deferred blocks needed for conditions and actionsconditions and actions

Use Use missing_methodmissing_method and and class_evalclass_eval to create reader/writer methods for to create reader/writer methods for variables in blocksvariables in blocks

2424

ArchitectureArchitecture

First pass – use Object Scoping, Context First pass – use Object Scoping, Context Variable, and Memento patterns for Variable, and Memento patterns for safety and flexibility, Deferred safety and flexibility, Deferred Evaluation for actionsEvaluation for actions

Second pass – use Visitor pattern for Second pass – use Visitor pattern for flexibility (C6, V8)flexibility (C6, V8)

Object Scoping (1 of 2)Object Scoping (1 of 2)

class SurveyDSL # METHOD BODIES OMITTEDclass SurveyDSL # METHOD BODIES OMITTED

attr_accessor :contextattr_accessor :context

def title(text) def title(text)

def question(text,*args) # text, nsel, blockdef question(text,*args) # text, nsel, block

def result(text,*args) # text, nsel, blockdef result(text,*args) # text, nsel, block

def condition(&check)def condition(&check)

def action(&action)def action(&action)

def response(resp_text,&action)def response(resp_text,&action)

def alternative(alt_text,&guard)def alternative(alt_text,&guard)

end#SurveyDSLend#SurveyDSL

class SurveyBuilder < SurveyDSLclass SurveyBuilder < SurveyDSL

2525

Object Scoping (2 of 2)Object Scoping (2 of 2)

class SurveyBuilder < SurveyDSLclass SurveyBuilder < SurveyDSL

. . .. . .

def read_DSL(rb_dsl_file)def read_DSL(rb_dsl_file)

# Omit checks for file existence# Omit checks for file existence

rb_file = File.new(dsl_file)rb_file = File.new(dsl_file)

instance_eval(rb_file.read, dsl_file) instance_eval(rb_file.read, dsl_file) # load/eval# load/eval

rb_file.close rb_file.close

selfself

end#read_DSLend#read_DSL

. . .. . .

end#SurveyBuilder end#SurveyBuilder

2626

2727

Parser Implementation (1 of 3)Parser Implementation (1 of 3)

def title(text) def title(text)

@context.incr_ql_count @context.incr_ql_count

@context.ql_error = false @context.ql_error = false

if @context.level == :survey_level &&if @context.level == :survey_level &&

@context.survey.title == nil @context.survey.title == nil

@context.survey.title = text.to_s@context.survey.title = text.to_s

else else

illegal_stmt_msg(illegal_stmt_msg(””title")title")

stmt_text_msg("Title",text) stmt_text_msg("Title",text)

if @context.survey.title != nil if @context.survey.title != nil

at_most_one_msg("title","survey") at_most_one_msg("title","survey")

enend d

end end

self self

end#titleend#title

2828

Parser Implementation (2 of 3)Parser Implementation (2 of 3)

def question(text,*args) # text, nsel, blockdef question(text,*args) # text, nsel, block

@context.incr_ql_count@context.incr_ql_count

@context.ql_error = false@context.ql_error = false

if @context.level == :survey_level &&block_given?if @context.level == :survey_level &&block_given?

@context.level = :question_level@context.level = :question_level

@context.qtype = :question_type@context.qtype = :question_type

nsel = 1nsel = 1

nsel = args[0].to_i if args.size > 0nsel = args[0].to_i if args.size > 0

@context.question=QuestionNode.new(text.to_s,nsel)@context.question=QuestionNode.new(text.to_s,nsel)

yieldyield # execute the block on the question call # execute the block on the question call

nresp = @context.question.responses.sizenresp = @context.question.responses.size

if nresp < @context.question.num_to_selif nresp < @context.question.num_to_sel

# error messages omitted# error messages omitted

endend

2929

Parser Implementation (3 of 3)Parser Implementation (3 of 3)

if !@context.ql_errorif !@context.ql_error

@context.survey.add_question(@context.question)@context.survey.add_question(@context.question)

else else

# error messages omitted# error messages omitted

end end

@context.level = :survey_level@context.level = :survey_level

@context.qtype = :no_type@context.qtype = :no_type

elseelse

endend

@context.question = nil@context.question = nil

selfself

end#questionend#question

Did Survey Language Follow Did Survey Language Follow Internal DSL Recommendations?Internal DSL Recommendations?

JMock designers Freeman and PryceJMock designers Freeman and Pryce

1.1. Separate syntax and interpretation into Separate syntax and interpretation into layerslayers

2.2. Use, and abuse, the host languageUse, and abuse, the host language

3.3. Don't trap the user in the internal DSLDon't trap the user in the internal DSL

4.4. Map error reports to the syntax layerMap error reports to the syntax layer

3030

3131

SummarySummary

Illustrated how commonality/variability Illustrated how commonality/variability analysis can be adapted for DSL designanalysis can be adapted for DSL design

Demonstrated how Ruby facilities can Demonstrated how Ruby facilities can be used for internal DSL developmentbe used for internal DSL development

Explored how design patterns can help Explored how design patterns can help lead to safe and flexible DSL processorslead to safe and flexible DSL processors

3232

Future WorkFuture Work

More systematic techniques to explore More systematic techniques to explore domain and discover needed constructsdomain and discover needed constructs

Improved runtime error handling tied Improved runtime error handling tied to DSL inputto DSL input

Better facilities for user extensionBetter facilities for user extension Investigation and comparison of other Investigation and comparison of other

languages – Groovy, Scala, Haskelllanguages – Groovy, Scala, Haskell

3333

AcknowledgementsAcknowledgements

H. C. Cunningham. A little language for H. C. Cunningham. A little language for surveys: Constructing an internal DSL in surveys: Constructing an internal DSL in Ruby, In Ruby, In Proceedings of the ACM SouthEast Proceedings of the ACM SouthEast Conference, Conference, 6 pages, March 2008.6 pages, March 2008.

Members of Fall 2006 graduate class on Ruby Members of Fall 2006 graduate class on Ruby and Software Developmentand Software Development

Suggestions on the paper by Chuck Jenkins, Suggestions on the paper by Chuck Jenkins, Yi Liu, Pallavi Tadepalli, and Jian WengYi Liu, Pallavi Tadepalli, and Jian Weng

3434

QuestionsQuestions