Master Assessment Plan: Analytics...Objective: Marketing mix and web analytics Outcome Data Data...

Master Assessment Plan: Analytics

Outcomes Analysis Years: 20142015 Biennial Report Year/Semester: 2015/Spring

Program(s): All programs Objective: Data mining and machine learning

Outcome Data Data Source Collection DateGraduates should be able to effectively use the SAS Enterprise Minerinterface.

Homework, Test Results, Evaluation Rubric Dickey, MotsingerReif Each Semester

Graduates should be able to recognize and develop association andsequence analyses.


Graduates should be able to explain the role of statistical tests in formingdecision trees.


Graduates should be able to fit decision trees to binary data and interpret theresults.


Graduates should be able to fit regression trees. Homework, Test Results, Evaluation Rubric Dickey, MotsingerReif Each SemesterGraduates should be able to explain discriminant analysis. Homework, Test Results, Evaluation Rubric Dickey, MotsingerReif Each SemesterGraduates should be able to recognize and interpret regression for binaryresponses.


Graduates should be able to divide data using clustering techniques. Homework, Test Results, Evaluation Rubric Dickey, MotsingerReif Each SemesterGraduates should be able to account for oversampling, profits, and losses. Homework, Test Results, Evaluation Rubric Dickey, MotsingerReif Each SemesterGraduates should be able to identify and use techniques for choosing thebest model from many.


Graduates should be able to build models using variable selection techniques. Homework, Test Results, Evaluation Rubric Dickey, MotsingerReif Each SemesterGraduates should be able to recognize and build neural networks. Homework, Test Results, Evaluation Rubric Dickey, MotsingerReif Each Semester

Objective: Data visualizationOutcome Data Data Source Collection Date

Graduates should be able to describe in a way that demonstrates generalunderstanding, the main goals of data visualization.

Homework, Test Results, Evaluation Rubric Healey Each Semester

Graduates should be able to recognize the main types of charts and plots(line, bar, pie, scatter, etc.).


Graduates should be able to Select the type of chart or plot best suited to aparticular type of data and visualization goal.


Graduates should be able to recognize the main types of maps (choropleth,contour, dot, dasymetric, etc.).


Graduates should be able to select the type of map best suited to a particulartype of data and visualization goal.


Objective: Text AnalyticsOutcome Data Data Source Collection Date

Graduates should be able to describe in a way that demonstrates generalunderstanding the main goals of text analytics and text mining.


Graduates should be able to recall the strengths and limitations of differentmethods of systematically representing text and understand how to applythese methods to a text corpus.


Graduates should be able to identify different approaches to computing textsimilarity, and how to use these measures to organize text based on similarityclustering.


Graduates should be able to recognize and describe different models ofemotion or sentiment.


Graduates should be able to employ different approaches to using emotionalmodels to estimate and represent sentiment contained in a text corpus.


Objective: PythonOutcome Data Data Source Collection Date

Graduates should be able to demonstrate a basic understanding of computerprogramming with a common procedural programming language.


Graduates should be able to design, implement, and test small programswritten in Python.


Graduates should be able to perform basic analytic operations with Pythonusing common external libraries (nltk, numpy, pandas, etc.).


Objective: Customer segmentation and positioningOutcome Data Data Source Collection Date

Graduates should be able to recognize real world applications ofsegmentation theory.

Homework, Test Results Dasmohapatra Each Semester

Graduates should be able to employ different techniques and methods forsegmenting various types of data using different statistical software (SAS,SPSS, R).


Graduates should be able to apply different techniques for variable reductionincluding principal components, common factor analysis, etc.


Graduates should be able to solve a segmentation problem using statisticalsoftware (SAS, SPSS, R) and real world data.


Objective: Design of experimentsOutcome Data Data Source Collection Date

Graduates should be able to explain the concept of designing experimentsand its applications beyond laboratories in a business setting.


Graduates should be able to control and vary parameters to get the desiredoutcome in direct marketing experiments.


Graduates should be able to use SAS and JMP software and sample industrydata to demonstrate understanding of techniques used in designing andanalyzing experiments.


Objective: Marketing mix and web analyticsOutcome Data Data Source Collection Date

Graduates should be able to identify different marketing mix models andmarket basket models for use in business settings.


Graduates should be able to develop a pricing model using customer data todemonstrate understanding of marketing mix and market basket models.


Graduates should be able to apply marketbased models to providerecommendations to the customer for product, promotion and pricing changesto its offerings.


Graduates should be able to recognize web data reporting tools (e.g., GoogleAnalytics), ad and campaign testing tools (e.g., optimizely), visualization toolsfor big data, and strategies for integrating web and offline data.


Graduates should be able to complete Google Analytics certification todemonstrate understanding of web analytics basics and applications.


Objective: Logistic regressionOutcome Data Data Source Collection Date

Graduates should be able to identify the key differences between logisticregression and linear regression.

Homework, Test Results LaBarr Each Semester

Graduates should be able to distinguish between nominal and ordinalvariables and the different statistical tests between them.


Graduates should be able to build logistic regression models (in all their forms– binary, ordinal, nominal) using the statistical software packages SAS and R.


Graduates should be able to interpret the output from logistic regressionmodels (in all their forms – binary, ordinal, nominal) using the statisticalsoftware packages SAS and R.


Graduates should be able to interpret the meaning of odds ratios. Homework, Test Results LaBarr Each Semester

Objective: Times series and forecastingOutcome Data Data Source Collection Date

Graduates should be able to decompose a time series into its three basiccomponents – trend, seasonality, and remainder.


Graduates should be able to distinguish between the three differentcorrelation functions – ACF, PACF, and IACF.


Graduates should be able to explain the difference between a stationary andnonstationary time series.


Graduates should be able to build the different classes of time series models(Exponential Smoothing, ARIMA, and Neural Network) using the statisticalsoftware packages SAS and R.


Graduates should be able to interpret the output of different classes of timeseries models (Exponential Smoothing, ARIMA, and Neural Network) using thestatistical software packages SAS and R.


Graduates should be able to diagnose different classes of time series modelsfor accuracy and reliability.


Graduates should be able to forecast different types of time series models. Homework, Test Results LaBarr Each SemesterGraduates should be able to cluster different time series into hierarchicalclusters using one of the three common techniques – bottomup, topdown,middleout.


Objective: Survival analysisOutcome Data Data Source Collection Date

Graduates should be able to build survival curves using both commontechniques – KaplanMeier and LifeTable.


Graduates should be able to interpret the survival and hazard probability of adata series.


Graduates should be able to design a data set that contains both censoredand uncensored observations.


Graduates should be able to build the different classes of survival analysismodels (Accelerated Failure Time, Cox Regression).


Graduates should be able to identify the proper distributional assumption inan Accelerated Failure Time model.


Graduates should be able to identify cases where competing risks areoccurring.


Objective: Exploratory data analytics/fraud detectionOutcome Data Data Source Collection Date

Graduates should be able to build and analyze a social network data set. Homework, Test Results LaBarr Each SemesterGraduates should be able to identify subgroups, centers, closeness, brokers,bridges, diffusion, and adoption in a social network.


Graduates should be able to transform transactional data into a usable formatfor typical forms of analysis.


Graduates should be able to identify common characteristics of fraud in theinsurance industry.


Objective: OptimizationOutcome Data Data Source Collection Date

Graduates should be able to apply integer and mixedinteger optimizationtechniques to identify the product mix and transportation schedules thatmaximize profitability while satisfying demand constraints, capacity constraintsand transportation cost caps.

Homework, Test Results Kyriakoulis Each Semester

Graduates should be able to perform Data Envelopment Analysis to identifythe most efficient unit from a set of many candidates who produce differentfinal products using different inputs mix.


Graduates should be able to identify the optimal stockportfolio allocation thatminimizes risk and at the same time achieves a target return.


Graduates should be able to use nonlinear optimization techniques (Gauss,GaussNewton, NewtonRaphson) to fit the best nonlinear model in a set ofdata.


Objective: Simulation and risk analysisOutcome Data Data Source Collection Date

Graduates should be able to verify the properties of statistical models bysimulating their behavior and identify the impact of violating key modelingassumptions.


Graduates should be able to use the expected value approach to calculatethe expected net present value and potential losses for an investment thatruns across multiple periods.


Graduates should be able to utilize the KolmogorovSmirnov, AndersonDarling and other nonparametric statistics to identify and fit the appropriatedistribution of different real datasets (e.g. oil prices across time, daily oilproduction, lease costs).


Graduates should be able to perform scenario and sensitivity analysis toidentify the most sensitive decision/control variables in a project.


Graduates should be able to use simulation techniques to calculate theexpected Net Present Value, Value at Risk and Expected Shortfall of a projectthat extends across many years; assess the risks and providerecommendations for their reduction.


Graduates should be able to use simulation to value real options, such as"option to abandon project"; identify the optimal option's price and suggest


alternative option contracts that minimize the risk of shareholders.

Objective: Financial AnalyticsOutcome Data Data Source Collection Date

Graduates should be able to evaluate the performance of a portfolio throughthe usage of single factor models and build the optimal (risk vs return) portfolioallocation CAPM, portfolio's alpha, portfolio's beta.


Graduates should be able to analyze portfolio risk and return from thestandpoint of a riskmanager, utilizing a timevarying beta estimation (EWMA).


Graduates should be able to analyze the timevarying volatility of a portfoliothrough the usage of ARCH and GARCH (symmetric and asymmetric) models.


Graduates should be able to build credit scorecards to rank different creditapplicant; utilize clustering, decision trees and logistic regression models.


Graduates should be able to identify the optimal cutoff point for a scorecard ina way that maximizes the profitability of a credit institution.


Graduates should be able to identify the optimal cutoff point for a scorecard ina way that maximizes the approved credit customers while keeping defaultrates the same.


Objective: Linear algebraOutcome Data Data Source Collection Date

Graduates should be able to manipulate and simplify matrix equations usingthe properties of matrix multiplication, addition, inversion, transposition,symmetry.

Homework, Test Results Race Each Semester

Graduates should be able to compute and interpret common vector normsand similarity metrics.


Graduates should be able to solve systems of equations using the methods ofGaussian Elimination and Least Squares.


Graduates should be able to define (mathematically) and describe(geometrically) the notions of linear independence, vector span, vectorspaces, basis vectors, eigenvalues, eigenvectors and projections.


Graduates should be able to use software to find eigenvalues andeigenvectors of a matrix.


Graduates should be able to apply Principal Components Analysis to data forclustering, variable clustering, dimension reduction, and biased regression.


Graduates should be able to determine when Biased Regression isappropriate.


Graduates should be able to apply Biased Regression techniques such asPrincipal Component Regression to solve problems of severe multicollinearity.


Graduates should be able to find dominant topics/themes in text data usingNonnegative Matrix Factorization and the Singular Value Decomposition.


Objective: Project managementOutcome Data Data Source Collection Date

Graduates should be able to demonstrate project management planning andexecution by developing and maintaining a work breakdown structureidentifying all subtasks required to plan and complete a project form start tofinish, assign and track individual accountability for each subtask, scheduleand track work progress as it occurs, and estimate completion of future workbased on previous rate of progress.

Practicum Coaching Rubric West Each Semester

Graduates should be able to demonstrate project management competencyby successfully performing scope feasibility analysis on original project scopeand subsequent scope development/refinement.


Objective: Teamwork/problem solving/conflict resolutionOutcome Data Data Source Collection Date

Graduates should be able to demonstrate professionalism and effectivenessin teambased settings.

Peer Evaluation Rubric West, Rappa Each Semester

Graduates should be able to demonstrate professionalism and effectivenessin resolving conflict associated with creative differences in approachinganalytic problems/projects and with workload distribution and management.


Graduates should be able to demonstrate leadership skills in planning andarticulating a vision for homework team projects, organizing workloadassignments, controlling for performance variation amongst teammates, andmotivating teammates to succeed.


Graduates should be able to demonstrate followership in supporting andenabling team leads by contributing effectively to the work plan, visiondevelopment and organization, subordinating/aligning individual goals withthose of the team, executing assignments, and motivating teammates andteam leader.


Graduates should be able to demonstrate problem solving competencies bybeing able to effectively resolve project issues focused on understandingunderlying datasets to be analyzed, understanding context of the underlyingbusiness problem, articulatingrefiningbounding assumptions and projectproblem statements, and then addressing scope feasibility and development.


Objective: Communication skillsOutcome Data Data Source Collection Date

Graduates should be able to prepare documents for use in business settingsthat are direct, concise, professional, easily skimmable, and grammaticallycorrect; e.g. apply conventions and strategies to business emails, design andwrite appropriate memos, create an audience centered executive summary,create an audiencecentered, persuasive resume.

Presentation Rubrics, Brody Communication Rubric West, Rappa, EganWarren Each Semester

Graduates should be able to apply strategies for editing and proofreading thatdemonstrate an understanding of grammar.


Graduates should be able to identify, understand, and use the components ofan effective presentation.


Graduates should be able to assess the audience for a presentation and usea variety of modes to present.


Graduates should be able to develop strategies for preparing and structuringpresentations to communicate, motivate, and/or persuade listeners.


Graduates should be able to use techniques for connecting with an audience,including voice projection, pacing, body language, eye contact, gestures,humor, personality and other performance skills.


Graduates should be able to use media and visual aids effectively inpresentations.


Graduates should be able to develop strategies to handle fear of publicspeaking, minimize distracting mannerisms, e.g. verbal pauses, and projectconfidence.


Graduates should be able to handle audience questions. Presentation Rubrics, Brody Communication Rubric West, Rappa, EganWarren Each Semester

Program(s): Analytics Objective: To meet enrollment and admissions targets

Outcome Data Data Source Collection DateMeet student enrollment target, which is set at 100percent of operatingcapacity for the program (currently 80 students/year)

Enrollment statistics DGP Annually

Meet application pool target of 4:1 (applicants to available seats), or better Application statistics DGP AnnuallyMeet acceptance rate target of below 30percent Enrollment statistics DGP AnnuallyMeet enrollment rate target of greater than 80percent Enrollment statistics DGP AnnuallyMeet student attrition target of below 10percent Enrollment statistics DGP AnnuallyMeet admissions target for average undergraduate GPA (UGPA) of 3.50 orgreater

Admissions statistics DGP Annually

Meet admission rank index target of less than 75percent (ratio of the averageundergraduate institution rank to NCSU rank a ratio below 100percent isbetter)


Maintain an admissions profile (acceptance rate, enrollment rate, and UGPA)that is equivalent to, or better than comparablysized MBA programs at publicuniversities ranked in the Top10


Objective: To achieve an excellent placement of graduates and a return on investment for the analytics degreeOutcome Data Data Source Collection Date

Maintain job placement outcomes (placement rates and salaries) that areequivalent to, or better than comparablysized MBA programs at publicuniversities ranked in the Top10.

Placement statistics DGP Annually

Maintain job placement outcomes (placement rates and salaries) that areequivalent to, or better than comparablysized, 1year quantbased MSprograms at four benchmark schools (Berkeley, MIT, Cornell and CarnegieMellon


Attain a job placement rate of 90percent or higher by graduation Placement statistics DGP AnnuallyAttain a average ROI payback period of less than 36 months ROI statistics DGP AnnuallyAttain an average ROI payback period that is better (shorter) thancomparablysized MBA programs at public universities ranked in the Top10

ROI statistics DGP Annually

Attain a ratio of job offers per candidate of 2.0 or greater Job offer statistics DGP AnnuallyAttain a ratio of job interviews per candidate of 10.0 or greater Job interview data DGP Annually

Objective: To maintain the program as effective, efficient, and competitive with similar programsOutcome Data Data Source Collection Date

Maintain the program as a selfcontained and sustainable business model (i.e.operate within the budgetary cost constraints provided by tuition revenuegenerated for the program)

Budget statistics DGP Annually

Maintain resident and nonresident tuition at or below the average for othersimilar MS degree programs

Tuition comparisons DGP Annually







































Graduates should be able to demonstrate a basic understanding of computer Homework, Test Results, Evaluation Rubric Healey Each Semester

programming with a common procedural programming language.Graduates should be able to design, implement, and test small programswritten in Python.








































































Graduates should be able to transform transactional data into a usable format Homework, Test Results LaBarr Each Semester

for typical forms of analysis.Graduates should be able to identify common characteristics of fraud in theinsurance industry.






















Graduates should be able to use simulation to value real options, such as"option to abandon project"; identify the optimal option's price and suggestalternative option contracts that minimize the risk of shareholders.












































Graduates should be able to demonstrate leadership skills in planning andarticulating a vision for homework team projects, organizing workload


assignments, controlling for performance variation amongst teammates, andmotivating teammates to succeed.Graduates should be able to demonstrate followership in supporting andenabling team leads by contributing effectively to the work plan, visiondevelopment and organization, subordinating/aligning individual goals withthose of the team, executing assignments, and motivating teammates andteam leader.
























































Graduates should be able to divide data using clustering techniques. Homework, Test Results, Evaluation Rubric Dickey, MotsingerReif Each SemesterGraduates should be able to account for oversampling, profits, and losses. Homework, Test Results, Evaluation Rubric Dickey, MotsingerReif Each Semester

Graduates should be able to identify and use techniques for choosing thebest model from many.








































































Graduates should be able to distinguish between the three different Homework, Test Results LaBarr Each Semester

correlation functions – ACF, PACF, and IACF.Graduates should be able to explain the difference between a stationary andnonstationary time series.










































































































































Graduates should be able to use the expected value approach to calculate Homework, Test Results Kyriakoulis Each Semester

the expected net present value and potential losses for an investment thatruns across multiple periods.Graduates should be able to utilize the KolmogorovSmirnov, AndersonDarling and other nonparametric statistics to identify and fit the appropriatedistribution of different real datasets (e.g. oil prices across time, daily oilproduction, lease costs).





























































































































Graduates should be able to recall the strengths and limitations of different Homework, Test Results, Evaluation Rubric Healey Each Semester

methods of systematically representing text and understand how to applythese methods to a text corpus.Graduates should be able to identify different approaches to computing textsimilarity, and how to use these measures to organize text based on similarityclustering.












































































































































Graduates should be able to demonstrate project management planning andexecution by developing and maintaining a work breakdown structureidentifying all subtasks required to plan and complete a project form start tofinish, assign and track individual accountability for each subtask, scheduleand track work progress as it occurs, and estimate completion of future work


based on previous rate of progress.Graduates should be able to demonstrate project management competencyby successfully performing scope feasibility analysis on original project scopeand subsequent scope development/refinement.























































Outcome Data Data Source Collection Date

Graduates should be able to effectively use the SAS Enterprise Minerinterface.


























































































Graduates should be able to cluster different time series into hierarchicalclusters using one of the three common techniques – bottomup, topdown,middleout.





















































Master Assessment Plan: Analytics...Objective: Marketing mix and web analytics Outcome Data Data...

Documents

Transcript of Master Assessment Plan: Analytics...Objective: Marketing mix and web analytics Outcome Data Data...