Risk Poster Spring2014 48x42

1
7/23/2019 Risk Poster Spring2014 48x42 http://slidepdf.com/reader/full/risk-poster-spring2014-48x42 1/1 Trade Study Results Trade study results indicate that changing the Life Cycle combination factor and the launch indicators were worse fits than the baseline assumptions. The overall conclusion of these trade studies was that the baseline u-curve function and role value scheme should be replaced by:  With this research and its associated software tool, CubeSat mission designers will be able to input mission characteristics, such as their CubeSat form factor, mass, development cycle, and launch information, in order to determine the mission risk root causes which are of the highest concern for their mission. Historical data has been gathered from the CubeSat community and analyzed in order to provide a statistical background to characterize these Risk Estimating Relationships (RERs). The RER development uses general error regression models to obtain the best fit relationship between root cause consequence and likelihood values and the input factors of interest. These root causes are combined into seven overall CubeSat mission risks which are then graphed on the industry-standard 5x5 Likelihood- Consequence (L-C) chart to help mission designers quickly identify areas of concern within their mission. Decision analysis methods will be available for spacecraft designers to choose risk mitigation strategies which optimize their preferences. CubeSat Mission Design Software Tool for Risk Estimating Relationships Katharine Brumbaugh Gamble Advisor: Dr. Glenn Lightsey References: “Agency Risk Management ProceduralRequirements,”  (2008): NASA Procedural Requirements, NPR 8000.4A. Brumbaugh, K.M., Lightsey, E.G., "Systematic Approach to Risk Management for Small Satellites," Journal of Small Satellites. JoSS, Vol. 2, No. 1, p. 147-160, July 2013. Snee, R. (1977). "Validation of Regression Models: Methods and Examples." Technometrics Vol. 19 (No. 4). Stone, M. (1974). Cross-validating choice and assessment of statistical predictions (with discussion). J. Roy. Statist. Soc. B, 36, 111-147. What is Risk? “Afuture event with a negative consequence that has some probability of occurring. A risk can also be defined as the potential for performance shortfalls . A risk poses a threat to the spacecraft safety, program cost, schedule, or major mission objective. An item whose resolution is unlikely without focused management attention.” (NASA, 2008) What is Risk Management? Risk Management has been applied on larger s/c o Much too detailed, too time intensive for low-cost missions o Often hard to get access to risk database Basic Risk Management Plan follows these steps (Brumbaugh, 2013): o  Risk Identification o  Determine Mitigation Techniques o  Track Progress Risk is typically shown on a 5x5 L-C chart o The Good: Quick and visual explanation of spacecraft status, especially with tracking o The Bad: Likelihood and Consequence values are subjectively determined based on the systems engineer’s perspective of things. Image credit: Wikipedia Mathematical Background – Regression Analysis Given a set of data, regression analysis finds the line of best fit to describe the data Regression techniques include: o Ordinary Least Squares Traditionally used for linear models and additive models Minimizes square standard error o General Error Regression Techniques: Can use additive for or multiplicative error models. Minimum Percentage Error (MPE) Iterated Least Squares / Minimum Unbiased Percentage Error (IRLS / MUPE) Minimum Percentage Error – Zero Percentage Bias (MPE-ZPE) Background Research Plan Choosing Risk Functions Once the data has been processed, it can be run through the Minimum PercentageError – Zero Percentage Bias (MPE-ZPB) General Error Regression (GER) technique to obtain the 68 risk estimating relationships. The risk function forms were chosen based on engineering judgment and experience with cost estimating relationships. The primary factors of interest are form factor (ff), whether the mission has launched (launch), and the time in each life cycle phase – development (dev), integration (int), spacecraft functional testing (scfunc), environmental testing (environ), waiting for launch (wait), and operations (ops). Verification & Validation Trade studies were completed to show that assumptions made in the development of the Risk Estimating Relationships (RERs) were reasonable. Four trade studies were completed by changing the: Risk Analysis Software Tool This inputs page is considered a necessity for the development of the risk management tool in an effort to make the tool as user-friendly as possible. The mission designer simply has to input their form factor, mass, select a launch option from four choices, input their launch date, and input the months in development, integration, functional testing, environmental testing, awaiting launch, and operations. The internal calculations of the tool will then output the root cause likelihood and consequence values on the outputs page. The user also has the ability to track the spacecraft risks at multiple milestones by indicating on the Options bar to which milestone the current inputs correspond. Data Gathering & Processing The data then needed to be processed before being run through regression techniques. This processing included: 1. Combining expert opinions through the use of a u-curve function and the evaluation of the respondent’s role on the mission. 2. Combining life cycle duration estimates based on predicated and actual values Acknowledgements:  Funded by the National Defense Science and Engineering Graduate Fellowship (NDSEG) from the Department of Defense. Texas Spacecraft Lab and all the team members for their hard work and dedication. The Aerospace Engineering Department for their continued support Dr. Glenn Lightsey for his continued support and attitude as a great advisor. Future Work: The next, and final, phase of this research is to generate the risk acceptance and decision analysis methodology. That is, what should the user do once the risk has been identified and given a likelihood and consequence value? Based on a historical database of lessons learned and mitigation techniques, the user will be able to select the method by which they hope to increase their likelihood of mission success. CubeSat Mission Risk Survey Response 1 Years experience Role on team U-value Response n Years experience Role on team U-value Normalize U-values to become weights Weighted average of root cause response values and life cycle estimates become combined response Remove duplicate mission entries in matrices/vectors, add combinedresponse Mass Form factor Launch choice Launch date Processed life cycle estimates Parameter Input  Actual or Predicted? Description Form factor 3 Entera numericvaluecorrespondi ng to thenumberof U's yourspacecraft design uses (e.g. 3U wouldbeenteredas"3") Mass 4 Enteranumericvalueofthemasslimit(inkg) Launched? No,butwehavealaunchpromised (ELaNaorsimilar) Selectananswerusingthedrop-downmenu:Yes,thes/chaslaunched;No,butwe'vebeen manifested;No,butwehavealaunchpromised(ELaNaorsimilar);No,we havenotbeen manifested orgiven a promiseof a launch Launch Date 2014 Givethedateofthelaunch;Ifthes/c hasyettobelaunched,givetheprojecteddate.(Canbein MM/DD/YYYYorMM/YYYYformat) Months in Development 24Actual Entera numericvaluecorrespondi ng to thenumberof months in s/cdesign and development, includin g everything up untilflight integration ; Indicatewhetherthis valueis actualorpredict ed Months in Integration 6Actual Entera numericvaluecorresponding to thenumberof months taken fors/cintegration; Indicate whetherthis valueis actualorpredicted Months in S/C Functional Testing 4Actual Entera numericvaluecorrespondi ng to thenumberof months spent on integrated s/ctesting at theorganization level, including functionaltesting; Indicatewhetherthis valueis actualor  predicted Months in S/C Environmental Testing 6Actual Entera numericvaluecorrespondi ng to thenumberof months spent on necessary testing to satisfy launch providerrequirements (usually includes thermalvac, vib tables, and mass  properties t esting); Indi catewhetherthis valu eis actualorpredicted Months S/Cis awaiting launch 1Actual Entera numericvaluecorrespondi ng to thenumberof months thespacecraft was "on theshelf" waiting forlaunch afteralltesting had been completed; Indicatewheth erthis valueis actualor  predicted MonthsS/Cisin operations 12Predicted Entera numericvaluecorresponding to thenumberof months thespacecraft was operationalin orbit;Indicatewheth erthis valueis actualorpredicted Milestone LVINT Enterthenameof themilestoneforwhich thesenumbers reflect thestatus Options: CalculateL-Cvalues for Milestone1 CalculateL-Cvalues for Milestone2 CalculateL-Cvalues for Milestone3 Current factors of interest in regression analysis Macro buttons will calculate the L-C values for multiple milestones Life cycle values may not be based on experience, but could be predicted Miss io n R isk R o o t Cau se Consequence value Likelihood value Consequence value Likelihood value Consequence value Likelihood value Schedule  3.31 8047 82 4.487 23012 3 3 .2902 8484 9 4 .4713 7785 1 3.177 3318 6 4. 5357 4229 6 1. Inability tofind desired spacecraft components  2.21 2231 943 4 .352 60911 7  2.2122 3194 3 4.39 2936 827 2.21 2231 943 4.55 7244 851 2. Mechanical design delays (such as issues with the CADordrawings)  2.57 656724 4.40 9706 951  2.4989 8297 3 4.36 2327 041 2.22 0156 905 4.53 4473 471 3. Software design delays (such as basic component functionality orembedded codingissues)  3.94 1404 598 4. 7744 5070 1  3.8691798 7 4.69644 896 3.43 1920 442 4.69 3574 931 4. Delay due toissuse with payload provider(may be related todelivery of EDUorflight unit, documentation, or interface issues)  3.56 4926 097 4.2 42310 987  3.5649 2609 7 4.29 3131 741 3.56 4926 097 4.40 5439 749 5. Delay due toinadequate documentation  2.20 2138 245 3.9 60906 613  2.3759 1863 1 4.05460 527 3.13 0304 667 4.24 2332 588 Payload  3.34 7989 002 4.47 8869 868 3.317 5176 4.53 8279 758 3.43 5401 292 4.70 980103 1. Software interface issues between payload and spacecraft bus  3.31 9286 913 4.71 6094 717  3.0217 4766 4 4.71 3521 567 2.93 1091 293 4.85 3501 468 2. Hardware/electrical interface issues between payload and spacecraft bus  2.94 5127 6 4.45 7863 799  2.9246 8945 8 4.581 5059 2 3.14 8496 586 4.75 7762 118 3. Payload malfunction due tomechanical issues  3.10044 056 4.26 7186 994  3.3497 6483 2 4.42110 122 3.68 4628 181 4.67 8684 548 4. Payload malfunction due tosoftware issues  3.54 5703 713 4.43 606919  3.4981 9099 8 4.52 7144 099 3.38 0372 317 4.66 82102 Mi lest one 1 Mi le st one 2 Mi lest one 3 Rootcauses calculated via VBA-programmed functions Mission risk L-C values calculated via rank reciprocal weighting scheme (see JoSSpaper) Currently up to 3 milestones can be tracked at one time Milestone 1 C L SCH 3. 32 4. 49 PER 2. 65 4. 46 PAY 3. 35 4. 48 SC-1 2.6 4. 56 SC-2 2. 9 4. 4 SC-3 2. 58 4. 46 COST 3. 16 4. 85 C L C L SCH 3. 29 4. 471  SCH 3.177 4.536       1       2       3       4       L       I       K       E       L       I       H       O       O       D       5 Mi lest one 2 Mi lest one 3 4 5 CONSEQUENCE 1 2 3 Select missionriskto bringtofront onplot: Whichmilestones wouldyoulike toplot? PER PAY SC1 COST Delete all riskshapes Plotall risks SCH SC1 Milestone 1 PER SC2 Milestone 2 PAY SC3 Milestone 3 COST All Mil es tones PLOT!  Ability to choose which risks and milestones to plot Rank reciprocal weighted mission risks (same as on outputs page)  Ability to bringdesired risk to front  Ability to clear plot  Ability to plot multiple milestones (all risks) Once the option to calculate the L-C values is selected, a VBA program takes the user inputs and calculates the consequence and likelihood values for each root cause based on the formulas establishedfrom the regression techniques. The plots page provides a quick and easy way to view the mission risks on a traditionalL-C chart. The mission risk consequence is plotted on the x- axis while the likelihood is plotted on the y-axis. Excel macro calculates combined rank and outputs best function Minimize SSD while keeping Bias = 0 General R^2 value allows comparison between any model (not just linear) The MPE-ZPB technique calls for minimizing the Sum of Squared Deviations (SSD) while keeping the bias (B M ) equal to zero. Excel Solver was used to accomplish this task while also comparing the generalized R 2  value for each function form under consideration.            n i i i i  M a  x  f a  x  f  y SSD 1 2 2 ) , ( ) , (            n i i i i  M a  x  f  y a  x  f n  B 1 ) , ( ) , ( 1 } ) ( }{ ) ( { } ) )( ( { 2 2 2 2 2 2 i i i i i i i i  y  y n  x  x n  y  x  y  x n  R Risk Estimating Relationship function forms: (1) L1 = a + b*ff (2) L2 = a + b*ff + cc*dev + d*int + e*scfunc + f*environ + g*wait + h*ops (3) L3 = a + b*ff + cc*launch (4) L4 = a + b*launch (5) L5 = a + b*dev + cc*int + d*scfunc + e*environ + f*wait + g*ops (6) E1 = a + b*cc^ff (7) E2 = a + b*cc^launch (8) E3 = a + b*cc^(dev+int+scfunc+environ+wait+ops) (9) T1 = a + b*ff^cc (10) T2 = a + b*dev^cc + d*int^e + f*scfunc^g + h*environ^I + j*wait^k + l*ops^m (11) T3 = a + b*launch^cc (12) T4 = a + b*ff^cc + d*launch^e Model Numberof RCs <0 % RCs <0 TOTAvg<0 Avgof Avg uTrial_1 51 75 . 000% 14-1.445229864 uTrial_2 43 63 . 235% 12-0.367804957 uTrial_3 4464. 706% 10 -0 . 53987402 uTrial_4 47 69 . 118% 12-0.835219003 uTrial_5 5175. 000% 11 -1 . 02684192 rTrial_1 43 63.235% 9 -0 .29767468 rTrial_2 42 61 . 765% 10-0.696299316 rTrial_3 46 67 . 647% 11-0.562529072 lcTrial_1 26 38.235% 7 0.3652918 lcTrial_2 1725. 000% 2 1. 728000384 lcTrial_3 1623. 529% 2 2. 207289738 lcTrial_4 1927. 941% 4 1. 911504421 laTrial_1 1 1.471% 1 0.424133482 laTrial_2 15 22.059% 7 -0 .44827084 laTrial_3 4 5.882% 2 0.376422029 urTrial_1 54 79 . 412% 13-1.464243642 urTrial_2 51 75 . 000% 12-1.117280153 1. U-curve function (uTrial) 2. Role numerical values (rTrial) 3. Life cycle combination factor (lcTrial) 4. Launch indicator numerical mapping values (laTrial) Role Weight Principal Investigator 0.5 P rogram Ma na ge r 1 S yste msEn g ine er 0.75 Ch ief E ng i nee r 0. 75 SubsystemLead 0.25 Team me mber 0 .0 5 (fore ac h subsystem) 6 / 1 4 / 1 ) , (  role  yrs role  yrs u    Leave-Out-One Model Validation (Stone, 1974) was used to validate the functions chosen in the GER MPE-ZPB algorithm. This model validation involves: The method shows that a majority of the cases were accurate within +/- 50% for both the Consequence and Likelihood calculations. It is concluded that the model is validated. The model equations were also validated by moving outside the data range (Snee, 1977) and the results indicate a high correlation of a decreased input value leading to large deviations from expected values. Increasing values, though, rarely cause large variations in output values. It is concluded that this is not an issue and the user will be made aware of the data ranges and the danger of moving outside of the limits. Range C L (+/-) 50% 64. 046% 70. 648% (+/-) 100% 80. 492% 76. 531% (+/-) 150% 91. 116% 81. 873% To have the statistical analysis based on historical data, it was necessary to collect data from the C ubeSat community through the use of an online survey form. These results were collected over the course of 8 months, with a total of 66 responses. Demographic analysis of the results showed that mostly university missions completed the survey, with CubeSats of the 3U form factor. Additionally, the respondents tended to have 0-5 years experience in building spacecraft. CubeSat Mission Risk Survey April 2013 – November 2013: Gathered data December 2013: High-level demographic analysis of data Risk Estimating Relationships September 2013: General Error Regression using MPE-ZPB technique to determine risk function form December 2013: Trade studies and model validation of chosen risk function form Data processing techniques September 2013: Initial algorithms completed with artificial data December 2013: Algorithms run on actual data CubeSat Risk Analysis Tool September 2013: Macro-enabled Excel interface with inputs, outputs, and plots pages April 2014: Release to CubeSat Community for feedback Spring 2014: Establish risk acceptance techniques Summer 2014: Incorporate feedback and develop risk mitigation database 4 5 CONSEQUENCE 1 2 3 i ils t s ul uli t lt Inputs page Outputs page Plots page 0 5 10 15 20 25 30 35 40 0 2.5 5 Modified response values Modified Conseq Responsefor Cases Case 14 Conseq Case 13 Conseq Case 12 Conseq Case 11 Conseq Case 10 Conseq Case 9 Conseq Case 8 Conseq Case 7 Conseq Case 6 Conseq Case 5 Conseq Case 4 Conseq Case 3 Conseq Case 2 Conseq Case 1 Conseq 0 5 10 15 20 25 30 35 40 45 0 2.5 5 Modified response values Modified Likel ResponseforCases Case 14 Conseq Case 13 Conseq Case 12 Conseq Case 11 Conseq Case 10 Conseq Case 9 Conseq Case 8 Conseq Case 7 Conseq Case 6 Conseq Case 5 Conseq Case 4 Conseq Case 3 Conseq Case 2 Conseq Case 1 Conseq What is a CubeSat? California Polytechnic State University (Cal Poly) established a standard launch mechanism called the Poly-Picosatellite Orbital Deployer (P-POD) The P-POD holds 10 cm cubed satellites  – called 1-Unit (1U) CubeSats Common configurations are in 1U, 2U, 3U, and now 6U and even 12U The first six CubeSats were deployed from a P- POD in June 2003 CubeSats fly as secondary cargo aboard any launch available, depending on the mission requirements for orbit parameters Bevo-2 (left) and RACE (right) 3U CubeSats at UT-Austin Typical 5x5 L-C chart Example regression line High-level demographic analysis of survey results Combining experts diagram Example GER spreadsheet This plot shows the RERs have a strong preferencefor function form 10 Moving Outside Data Range Results

Transcript of Risk Poster Spring2014 48x42

Page 1: Risk Poster Spring2014 48x42

7/23/2019 Risk Poster Spring2014 48x42

http://slidepdf.com/reader/full/risk-poster-spring2014-48x42 1/1

Trade Study Results

Trade study results indicate that

changing the Life Cycle combination

factor and the launch indicators

were worse fits than the baseline

assumptions.

The overall conclusion of these

trade studies was that the baseline

u-curve function and role value

scheme should be replaced by:  

With this research and its associated software tool, CubeSat mission designers will be able to input mission characteristics, such as their CubeSat form factor, mass, development cycle, and launch information, in order to determine the mission risk root causes which are of

the highest concern for their mission. Historical data has been gathered from the CubeSat community and analyzed in order to provide a statistical background to characterize these Risk Estimating Relationships (RERs). The RER development uses general error regression

models to obtain the best fit relationship between root cause consequence and likelihood values and the input factors of interest. These root causes are combined into seven overall CubeSat mission risks which are then graphed on the industry-standard 5x5 Likelihood-

Consequence (L-C) chart to help mission designers quickly identify areas of concern within their mission. Decision analysis methods will be available for spacecraft designers to choose risk mitigation strategies which optimize their preferences.

CubeSat Mission Design Software Tool for Risk Estimating RelationshipsKatharine Brumbaugh Gamble

Advisor: Dr. Glenn Lightsey

References:“Agency Risk Management ProceduralRequirements,”  (2008): NASA Procedural Requirements, NPR 8000.4A.

Brumbaugh, K.M., Lightsey, E.G., "Systematic Approach to Risk Management for Small Satellites," Journal of Small Satellites. JoSS, Vol.

2, No. 1, p. 147-160, July 2013.

Snee, R. (1977). "Validation of Regression Models: Methods and Examples." Technometrics Vol. 19 (No. 4).

Stone, M. (1974). Cross-validating choice and assessment of statistical predictions (with discussion). J. Roy. Statist. Soc. B, 36, 111-147.

What is Risk?

“A future event with a negative consequence that has some probability

of occurring. A risk can also be defined as the potential for performance

shortfalls . A risk poses a threat to the spacecraft safety, program cost,

schedule, or major mission objective. An item whose resolution is

unlikely without focused management attention.” (NASA, 2008) 

What is Risk Management?

•Risk Management has been applied on larger s/c

o Much too detailed, too time intensive for low-cost missions

o Often hard to get access to risk database

•Basic Risk Management Plan follows these steps (Brumbaugh, 2013):

o  Risk Identification

o  Determine Mitigation Techniques

o  Track Progress

•Risk is typically shown on a 5x5 L-C chart

o The Good: Quick and visual explanation of spacecraft status,

especially with tracking

o The Bad: Likelihood and Consequence values are subjectively

determined based on the systems engineer’s perspective of

things.Image credit: Wikipedia

Mathematical Background – Regression Analysis

•Given a set of data, regression analysis finds the

line of best fit to describe the data

•Regression techniques include:

o Ordinary Least Squares

Traditionally used for linear models and

additive models

Minimizes square standard erroro General Error Regression Techniques:

Can use additive for or multiplicative

error models.

Minimum Percentage Error (MPE)

Iterated Least Squares / Minimum

Unbiased Percentage Error (IRLS / MUPE)

Minimum Percentage Error – Zero

Percentage Bias (MPE-ZPE)

Background Research Plan

Choosing Risk Functions

Once the data has been processed, it can be run through the MinimumPercentage Error – Zero Percentage Bias (MPE-ZPB) General Error

Regression (GER) technique to obtain the 68 risk estimating relationships.

The risk function forms were chosen based on engineering judgment and

experience with cost estimating relationships. The primary factors of

interest are form factor (ff), whether the mission has launched (launch),

and the time in each life cycle phase – development (dev), integration (int),

spacecraft functional testing (scfunc), environmental testing (environ),

waiting for launch (wait), and operations (ops).

Verification & Validation

Trade studies were completed to show that assumptions made in thedevelopment of the Risk Estimating Relationships (RERs) were reasonable.

Four trade studies were completed by changing the:

Risk Analysis Software Tool

This inputs page is considered a necessity for the development of the riskmanagement tool in an effort to make the tool as user-friendly as possible.

The mission designer simply has to input their form factor, mass, select a

launch option from four choices, input their launch date, and input the

months in development, integration, functional testing, environmental

testing, awaiting launch, and operations. The internal calculations of the

tool will then output the root cause likelihood and consequence values on

the outputs page. The user also has the ability to track the spacecraft risks

at multiple milestones by indicating on the Options bar to which milestone

the current inputs correspond.

Data Gathering & Processing

The data then needed to be processed before being run through regression

techniques. This processing included:

1. Combining expert opinions through the use of a u-curve function and

the evaluation of the respondent’s role on the mission. 

2. Combining life cycle duration estimates based on predicated and actual

values

Acknowledgements: •Funded by the National Defense Science and Engineering Graduate Fellowship (NDSEG) from the Department of Defense.

•Texas Spacecraft Lab and all the team members for their hard work and dedication.

•The Aerospace Engineering Department for their continued support

•Dr. Glenn Lightsey for his continued support and attitude as a great advisor.

Future Work: The next, and final, phase of this research is to generate the risk acceptance and decision analysis methodology. That is,

what should the user do once the risk has been identified and given a likelihood and consequence value? Based on a

historical database of lessons learned and mitigation techniques, the user will be able to select the method by which they

hope to increase their likelihood of mission success.

CubeSat Mission

Risk Survey

Response 1

Years

experience

Role on

team

U-value

Response n

Years

experience

Role on

team

U-value

Normalize U-values to become weights

Weighted average of root cause response

values and life cycle estimates become

combined response

Remove duplicate mission entries in

matrices/vectors, add combined response

Mass

Form

factor

Launch

choice

Launch

date

Processedlife cycle

estimates

Parameter Input  Actual or

Predicted? Description

Form factor 3

Entera numericvaluecorresponding to thenumberof U's yourspacecraft design uses (e.g. 3U

would beentered as "3")

Mass 4 Entera numericvalueof themass limit (in kg)

Launched?

No, but we have alaunch promised

(ELaNaorsimilar)

Select an answerusing thedrop-down menu: Yes, thes/chas launched; No, but we'vebeen

manifested; No, but wehavea launch promised (ELaNa orsimilar); No, we havenot been

manifested orgiven a promiseof a launch

Launch Date 2014

Givethedateof thelaunch; If thes/c has yet to belaunched, givetheprojected date. (Can bein

MM/DD/YYYYorMM/YYYYformat)

Months in

Development 24Actual

Entera numericvaluecorresponding to thenumberof months in s/cdesign and development,

including everything up untilflight integration; Indicatewhetherthis valueis actualorpredicted 

Months in Integration 6Actual

Entera numericvaluecorresponding to thenumberof months taken fors/cintegration; Indicate

whetherthis valueis actualorpredicted 

Months in S/C

Functional Testing 4Actual

Entera numericvaluecorresponding to thenumberof months spent on integrated s/ctesting at

theorganization level, including functionaltesting; Indicatewhetherthis valueis actualor 

 predicted 

Months in S/C

Environmental Testing 6Actual

Entera numericvaluecorresponding to thenumberof months spent on necessary testing to

satisfy launch providerrequirements (usually includes thermalvac, vib tables, and mass

 properties testing); Indicatewhetherthis valueis actualorpredicted 

Months S/Cis awaiting

launch 1Actual

Entera numericvaluecorresponding to thenumberof months thespacecraft was "on theshelf"

waiting forlaunch afteralltesting had been completed; Indicatewhetherthis valueis actualor 

 predicted 

Months S/Cis in

operations 12Predicted

Entera numericvaluecorresponding to thenumberof months thespacecraft was operationalin

orbit;Indicatewhetherthis valueis actualorpredicted 

Milestone LVINT Enterthenameof themilestoneforwhich thesenumbers reflect thestatus

Options:

CalculateL-Cvalues

for Milestone1

CalculateL-Cvalues

for Milestone2

CalculateL-Cvalues

for Milestone3

Current factors of interest

in regression analysis

Macro buttons will calculate the L-Cvalues for multiple milestones

Life cycle values may not

be based on experience, but

could be predicted

Missio nRisk Ro o tCause

Consequence

value

Likelihood

value

Consequence

value

Likelihood

value

Consequence

value

Likelihood

valueSchedule   3. 31804782 4. 487230123 3. 290284849 4. 471377851 3. 17733186 4.535742296

1. Inability tofind desired spacecraft

components   2. 212231943 4. 352609117   2. 212231943 4. 392936827 2. 212231943 4. 557244851

2. Mechanical design delays (such as

issues with the CADordrawings)   2. 576567724 4. 409706951   2. 498982973 4. 362327041 2. 220156905 4. 534473471

3. Software design delays (such as basic

component functionality orembedded

codingissues)   3. 941404598 4.774450701   3. 869117987 4. 699644896 3. 431920442 4. 693574931

4. Delay due toissuse with payload

provider(may be related todelivery of

EDUorflight unit, documentation, or

interface issues)   3. 564926097 4. 242310987   3. 564926097 4. 293131741 3. 564926097 4. 405439749

5. Delay due toinadequate

documentation   2. 202138245 3. 960906613   2. 375918631 4. 055460527 3. 130304667 4. 242332588

Payload   3. 347989002 4. 478869868 3. 3175176 4. 538279758 3. 435401292 4. 709801103

1. Software interface issues between

payload and spacecraft bus   3. 319286913 4. 716094717   3. 021747664 4. 713521567 2. 931091293 4. 853501468

2. Hardware/electrical interface issues

between payload and spacecraft bus   2. 9451276 4. 457863799   2. 924689458 4. 58150592 3. 148496586 4. 757762118

3. Payload malfunction due tomechanical

issues   3. 100044056 4. 267186994   3. 349764832 4. 422110122 3. 684628181 4. 678684548

4. Payload malfunction due tosoftware

issues   3. 545703713 4. 436069919   3. 498190998 4. 527144099 3. 380372317 4. 66821002

M il es to ne 1 M il es to ne 2 M il es to ne 3

Root causes calculatedvia VBA-programmed

functions

Mission risk L-C values calculatedvia rank reciprocal weighting

scheme (see JoSSpaper)

Currently up to 3 milestones can betracked at one time

Milestone

1 C L

SCH 3 . 32 4 . 49

PER 2 . 65 4 . 46

PAY 3 . 35 4 . 48

SC-1 2 . 6 4 . 56

SC-2 2 .9 4 .4

SC-3 2 . 58 4 . 46

COST 3 . 16 4 . 85

C L C L

SCH 3 . 29 4 . 47 1   SCH 3.177 4.536

      1

      2

      3

      4

      L      I      K      E      L      I      H      O      O      D

      5

M il es to ne 2 M il es to ne 3

4 5

CONSEQUENCE

1 2 3

Select missionriskto

bringtofront onplot:

Whichmilestones

wouldyoulike toplot?

P E R P A YSC1

COST

Delete all riskshapes

Plotall risks

SCH

SC1

Milestone 1

PER

SC2

Milestone 2

PAY

SC3

Milestone 3

COST All Miles tones

PLOT!

 Ability to choose which risks andmilestones to plot

Rank reciprocal weighted missionrisks (same as on outputs page)

 Ability to bring desired risk to front

 Ability to clear plot

 Ability to plot multiplemilestones (all risks)

Once the option to

calculate the L-C

values is selected, a

VBA program takes

the user inputs and

calculates the

consequence and

likelihood values for

each root cause

based on the

formulas

established from

the regression

techniques.

The plots page

provides a quick

and easy way to

view the mission

risks on a

traditional L-C

chart. The mission

risk consequence is

plotted on the x-

axis while the

likelihood is plotted

on the y-axis.

Excel macro calculatescombined rank and outputsbest function

Minimize SSD whilekeeping Bias = 0

General R^2 value allowscomparison between any model

(not just linear)

The MPE-ZPB technique calls for minimizing the

Sum of Squared Deviations (SSD) while keeping

the bias (BM) equal to zero. Excel Solver was used

to accomplish this task while also comparing the

generalized R2 value for each function form under

consideration.

   

  

   

  n

ii

ii M 

a x f 

a x f  ySSD

1

2

2

),(

),(

   

  

   

  n

ii

ii M 

a x f 

 ya x f 

n B

1 ),(

),(1

})(}{)({

}))(({2222

22

iiii

iiii

 y yn x xn

 y x y xn R

Risk Estimating Relationship function forms:

(1) L1 = a + b*ff

(2) L2 = a + b*ff + cc*dev + d*int + e*scfunc + f*environ + g*wait + h*ops

(3) L3 = a + b*ff + cc*launch

(4) L4 = a + b*launch

(5) L5 = a + b*dev + cc*int + d*scfunc + e*environ + f*wait + g*ops

(6) E1 = a + b*cc^ff

(7) E2 = a + b*cc^launch

(8) E3 = a + b*cc^(dev+int+scfunc+environ+wait+ops)

(9) T1 = a + b*ff^cc

(10) T2 = a + b*dev^cc + d*int^e + f*scfunc^g + h*environ^I + j*wait^k + l*ops^m

(11) T3 = a + b*launch^cc

(12) T4 = a + b*ff^cc + d*launch^e

Model Numberof RCs <0 % RCs <0 TOTAvg<0 Avgof Avg

uTrial_1 5 1 7 5. 00 0% 1 4 - 1 .4 45 22 98 64

uTrial_2 4 3 6 3. 23 5% 1 2 - 0 .3 67 80 49 57

uTrial_3 4 4 6 4. 70 6% 1 0 - 0. 53 98 74 02

uTrial_4 4 7 6 9. 11 8% 1 2 - 0 .8 35 21 90 03

uTrial_5 5 1 7 5. 00 0% 1 1 - 1. 02 68 41 92

rTrial_1 43 63. 235% 9 - 0. 29767468

rTrial_2 4 2 6 1. 76 5% 1 0 - 0 .6 96 29 93 16

rTrial_3 4 6 6 7. 64 7% 1 1 - 0 .5 62 52 90 72

lcTrial_1 26 38.235% 7 0.3652918

lcTrial_2 1 7 2 5. 00 0% 2 1 .7 28 000 38 4

lcTrial_3 1 6 2 3. 52 9% 2 2 .2 07 289 73 8

lcTrial_4 1 9 2 7. 94 1% 4 1 .9 11 504 42 1

laTrial_1 1 1.471% 1 0.424133482

laTrial_2 15 22. 059% 7 - 0. 44827084

laTrial_3 4 5.882% 2 0.376422029

urTrial_1 5 4 7 9. 41 2% 1 3 - 1 .4 64 24 36 42

urTrial_2 5 1 7 5. 00 0% 1 2 - 1 .1 17 28 01 53

1. U-curve function (uTrial)

2. Role numerical values (rTrial)

3. Life cycle combination factor

(lcTrial)

4. Launch indicator numerical

mapping values (laTrial)

Role Weight

Principal

Investigator

0.5

P ro gr am Ma na ge r 1

S ys te ms E n gi ne er 0 .7 5

C hi ef E ng in ee r 0 .7 5

SubsystemLead 0.25

T ea m me mb er 0 .0 5 (f or e ac h

subsystem)

6/14/1),(   role yrsrole yrsu  

 

Leave-Out-One Model

Validation (Stone, 1974) was

used to validate the functions

chosen in the GER MPE-ZPB

algorithm. This model validation

involves:

The method shows that a

majority of the cases were

accurate within +/- 50% for both

the Consequence and Likelihood

calculations. It is concluded that

the model is validated.

The model equations were also

validated by moving outside the

data range (Snee, 1977) and the

results indicate a high correlation

of a decreased input valueleading to large deviations from

expected values. Increasing

values, though, rarely cause large

variations in output values. It is

concluded that this is not an

issue and the user will be made

aware of the data ranges and

the danger of moving outside of

the limits. 

Range C L

(+ /- ) 5 0 % 6 4 .0 4 6% 7 0 .6 4 8%

(+ /- ) 1 0 0 % 8 0 .4 9 2% 7 6 .5 3 1%

(+ /- ) 1 5 0 % 9 1 .1 1 6% 8 1 .8 7 3%

To have the statistical analysis based onhistorical data, it was necessary to

collect data from the C ubeSat

community through the use of an

online survey form. These results were

collected over the course of 8 months,

with a total of 66 responses.

Demographic analysis of the

results showed that mostly

university missions completed

the survey, with CubeSats of

the 3U form factor.

Additionally, the respondents

tended to have 0-5 years

experience in building

spacecraft.

CubeSat Mission Risk Survey

April 2013 – November 2013:

Gathered data

December 2013:

High-level demographic

analysis of data

Risk Estimating Relationships

September 2013:

General Error Regression using

MPE-ZPB technique to

determine risk function form

December 2013:

Trade studies and model

validation of chosen risk

function form

Data processing

techniques

September 2013:

Initial algorithms

completed with

artificial data

December 2013:

Algorithms run on

actual data

CubeSat Risk Analysis Tool

September 2013:

Macro-enabled Excel interface

with inputs, outputs, and plots

pages

April 2014:

Release to CubeSat Community

for feedback

Spring 2014:

Establish risk acceptance

techniques

Summer 2014:

Incorporate feedback and

develop risk mitigation database

      1

      2

      3

      4

      L      I      K      E      L      I      H      O      O      D

      5

4 5

CONSEQUENCE

1 2 3

i i l st s

u l u l i t l t

P E R P A Y

S C1

COST

lotall risks

SCH

E

Y

CST

Inputs page

Outputs page

Plots page

0

5

10

15

20

25

30

35

40

0 2.5 5

Modified response values

Modified Conseq Responsefor Cases

Case 14 Conseq

Case 13 Conseq

Case 12 Conseq

Case 11 Conseq

Case 10 Conseq

Case 9 Conseq

Case 8 Conseq

Case 7 Conseq

Case 6 Conseq

Case 5 Conseq

Case 4 Conseq

Case 3 Conseq

Case 2 Conseq

Case 1 Conseq

0

5

10

15

20

25

30

35

40

45

0 2.5 5

Modified response values

Modified Likel Responsefor Cases

Case 14 Conseq

Case 13 Conseq

Case 12 Conseq

Case 11 Conseq

Case 10 Conseq

Case 9 Conseq

Case 8 Conseq

Case 7 Conseq

Case 6 Conseq

Case 5 Conseq

Case 4 Conseq

Case 3 Conseq

Case 2 Conseq

Case 1 Conseq

What is a CubeSat?

•California Polytechnic State University (Cal Poly)

established a standard launch mechanism called

the Poly-Picosatellite Orbital Deployer (P-POD)

•The P-POD holds 10 cm cubed satellites – called

1-Unit (1U) CubeSats

•Common configurations are in 1U, 2U, 3U, and

now 6U and even 12U•The first six CubeSats were deployed from a P-

POD in June 2003

•CubeSats fly as secondary cargo aboard any

launch available, depending on the mission

requirements for orbit parameters

Bevo-2 (left) and RACE (right)

3U CubeSats at UT-Austin

Typical 5x5 L-C chart

Example regression line

High-level demographic

analysis of survey results

Combining

experts

diagram

Example GER spreadsheet

This plot shows the

RERs have a strong

preference for

function form 10

Moving Outside Data Range Results