Ask Good Questions

of 43 /43
1 Ask Good Questions Allan J. Rossman Dept of Statistics Cal Poly – San Luis Obispo (USA) 8 th International Conference on Teaching Statistics

Embed Size (px)


Ask Good Questions. Allan J. Rossman Dept of Statistics Cal Poly – San Luis Obispo (USA) 8 th International Conference on Teaching Statistics. Frank Sinatra. What’s the secret to success as a singer? “Sing Good Songs”. 2. My similarly succinct suggestion. - PowerPoint PPT Presentation

Transcript of Ask Good Questions

  • Ask Good QuestionsAllan J. RossmanDept of StatisticsCal Poly San Luis Obispo (USA)

    8th International Conference on Teaching Statistics

    *Ask Good Questions (ICOTS-8)

  • Ask Good Questions (ICOTS-8)*Frank SinatraWhats the secret to success as a singer?Sing Good SongsRossman

    Ask Good Questions (ICOTS-8)

  • *My similarly succinct suggestionWhats the secret to success at teaching statistics?

    Ask Good QuestionsRossmanAsk Good Questions (ICOTS-8)

  • No hidden messageMy take-home messageAsk Good QuestionsQuiz at end of presentationQ: What was my point?A: Ask Good QuestionsI may email you in 10 yearsQ: What do you remember?A: Ask Good QuestionsAsk Good Questions (ICOTS-8)Rossman

    *Ask Good Questions (ICOTS-8)

  • *George CobbJudge a textbook by its exercises, and you cannot go far wrong.RossmanAsk Good Questions (ICOTS-8)

  • *My corollaryJudge a teacher by the questions that he/she asks of students, and you cannot go far wrong.RossmanAsk Good Questions (ICOTS-8)

  • *What kinds/purposes of questions?Guide students to develop their understanding and skillsLearning activitiesAssess how well students have learnedAssessment questions

    RossmanAsk Good Questions (ICOTS-8)

  • *What makes a question good?I respectfully decline to answerFor nowInstead Ill present some examples of both kind of questions (learning activities, assessment questions)Taken from Stat 101 (introductory, algebra-based, service course at tertiary level)Then Ill revisit this questionRossmanAsk Good Questions (ICOTS-8)

  • *Four examples (learning activities)RossmanAsk Good Questions (ICOTS-8)

  • *George Cobb (again)Shorn of all subtlety and led naked out of the protective fold of educational research literature, there comes a sheepish little fact: lectures dont work nearly as well as many of us would like to think.RossmanAsk Good Questions (ICOTS-8)

  • Example 1: TVs and life expectancyIs there an association between a countrys life expectancy and its number of televisions per person?Rossman*Ask Good Questions (ICOTS-8)

  • Example 1: TVs and life expectancyDoes the scatterplot reveal an association? Which direction? How strong? Linear?Based on these data, would you conclude that sending TVs to Haiti would cause Haitians to start living longer?Identify a confounding variable that explains the associationDoes a strong association between variables imply a cause/effect relationship? Rossman*Ask Good Questions (ICOTS-8)

  • *Example 2: Reading cancer pamphletsAre pamphlets containing information for cancer patients written at an appropriate level that cancer patients can understand?

    Ask Good Questions (ICOTS-8)Rossman

  • Example 2: Reading cancer pamphletsExplain why the mean cannot be calculated for patients reading levelsDetermine median reading level of patients, median readability level of pamphletsAre the pamphlets readability levels well-matched to the patients reading levels?What proportion of patients have reading level below that of simplest pamphlet?

    Rossman*Ask Good Questions (ICOTS-8)

  • Example 2: Reading cancer pamphletsAsk Good Questions (ICOTS-8)Rossman

    *Ask Good Questions (ICOTS-8)

  • ***Example 3: Sleep deprivationDoes sleep deprivation have harmful effects on cognitive functioning three days later?21 subjects; random assignment

    Identify type of study, observational units, explanatory and response variables

    Ask Good Questions (ICOTS-8)Rossman

  • ***Example 3: Sleep deprivationStudents use simulation (tactile, then computer-based) to investigate core question of statistical inference:Is such an extreme difference unlikely to occur by chance (random assignment) alone (if there were no treatment effect)? Ask Good Questions (ICOTS-8)Rossman

  • Example 3: Sleep deprivation

    Summarize conclusion, and explain reasoning process that supports your conclusion

    Ask Good Questions (ICOTS-8)Rossman

    *Ask Good Questions (ICOTS-8)

  • Example 4: Which tire?Legendary campus storyWhich tire would you pick?Research question: Do people tend to pick right front tire more often than random chance?Again, students investigate the question of how surprising the observed class result would be under the null model with probabilityRossman*Ask Good Questions (ICOTS-8)

  • Example 4: Which tire?What if 32% of a random sample selects right front? Is this a significant/convincing result?Or, what additional information would you need?Make prediction for how p-value will change (if at all) as sample size increasesCalculate p-value for many different sample sizesSummarize how sample size affects p-value, strength of evidenceExplain why this makes intuitive sense (in hindsight, if not in foresight)Rossman*Ask Good Questions (ICOTS-8)

  • *Five examples (assessment items)RossmanAsk Good Questions (ICOTS-8)

  • *Joan Garfield, Beth ChanceBecause students learn to value what they know they will be tested on, we should assess what we value. - JGThe number one mantra to remember when designing assessment instruments is: Assess what you value. - BCRossmanAsk Good Questions (ICOTS-8)

  • Assessment example 1For each of the following quantities, indicate whether it can NEVER be negative or can SOMETIMES be negative.

    a) Sample size b) Sample proportion c) Standard deviationd) Inter-quartile range e) Difference in sample means f) Odds ratio g) Total sum of squares h) Slope coefficienti) Coefficient of determination j) Correlation coefficient k) ANOVA F-test statistic l) p-value

    Rossman*Ask Good Questions (ICOTS-8)

  • Assessment example 2The U.S. has slightly more than 300 million residents. In order to estimate the proportion of U.S. residents who have a facebook account to within about 3 percentage points with 95% confidence, about how many people should be randomly sampled?


    Rossman*Ask Good Questions (ICOTS-8)

  • Assessment example 3Are people more likely to lie with email than with pencil-and-paper communication? A recent study involved 48 graduate students in business at a particular university who participated in a bargaining game. Researchers kept track of whether the student lied about the amount of money involved when negotiating with another player. Some of the participants were randomly assigned to use email for their communication, while others used paper-and-pencil. It turned out that 24 of 26 who used email were guilty of lying about the amount of money involved, compared to 14 of 22 who used paper-and-pencil.Rossman*Ask Good Questions (ICOTS-8)

  • Assessment example 3 (cont.)Use a simulation analysis to investigate whether these data provide strong evidence in support of the research conjecture in the first sentence above. Summarize the conclusions that you draw from your analysis. Be sure to address issues of statistical significance, causation, and generalizability. Also explain the reasoning process and justification for your conclusions. Rossman*Ask Good Questions (ICOTS-8)

  • Assessment example 4State a research question, and describe a data collection plan to address it, for which a two-proportion z-test would be the relevant inference procedure. Identify the explanatory and response variables, and also state the relevant hypotheses, defining the parameter values appropriately. Be sure to indicate whether the data collection plan involves random sampling or random assignment.Rossman*Ask Good Questions (ICOTS-8)

  • Assessment example 5 (from 2009 AP Statistics exam)Consider the statistic mean / median. What values of this statistic might indicate that the population distribution is skewed to the right?Consider simulation results for values of mean / median, based on a normal population, and also the observed value of mean / median for given sample data. Do the simulation results suggest that the underlying population is skewed to the right? Explain.Rossman*Ask Good Questions (ICOTS-8)

  • Assessment example 5 (from AP Statistics, cont.)

    Use only statistics in the five-number summary (min, Q1, median, Q3, max) to propose your own measure of skewness. Indicate values of this statistic that would suggest skewness to the right. Explain.

    Rossman*Ask Good Questions (ICOTS-8)

  • Oh, by the way .Notice that this talk is trying to model question-based pedagogyFrom these specific questions/examples we can extract principles of good questionsAsk Good Questions (ICOTS-8)Rossman

    *Ask Good Questions (ICOTS-8)

  • *So, what makes a question good?Makes students thinkGoes beyond their starting pointChallenges without overwhelming studentsEngages students interestTo put forth effort to learnTo care about the course material being studiedAddresses important ideasIndicates to students whats valuedRossmanAsk Good Questions (ICOTS-8)

  • *What else makes a question good?Fits within logical progressionBuilds on students knowledgeAsks students to make/check predictionsConfronts students misconceptionsInspires students to ask their own questionsAbout course materialAbout real-world phenomena that can be investigated using intellectual skills being learnedRossmanAsk Good Questions (ICOTS-8)

  • One vehicle for students posing and investigating own questionsStudents use gapminder software (Hans Rosling) toPose research questions about the worldInvestigate those questions with animated graphicsWrite report of their findings

    Rossman*Ask Good Questions (ICOTS-8)

  • *Some questions from students projectsAre students more likely to recycle water bottles depending on the proximity of recycle bin and waste basket?Can people better answer math problems if they are presented with Roman letter notation as opposed to Greek letter notation?Are people who walk into a clothing store more likely to purchase something when the weather is rainy as opposed to sunny?Is balsa wood less elastic after it has been immersed in water?Are students reaction times affected by whether theyve just completed exercising?Are faculty more likely to drive a foreign car than students?Do guests at a dinner respond more positively to spaghetti sauce that has been sweetened or not sweetened? Do college students spend more money at a local fast food restaurant if they are under the influence of alcohol?

    RossmanAsk Good Questions (ICOTS-8)

  • More unsolicited adviceICOTS: Teaching StatisticsMy experience at introductory tertiary levelAny advice for teachers of younger students?Ask Good QuestionsAny advice for teachers of graduate students?Ask Good QuestionsAny advice for statistics education researchers?Ask Good Questions

    Ask Good Questions (ICOTS-8)Rossman

    *Ask Good Questions (ICOTS-8)

  • Examples of interest to meDoes presenting activities/examples based on real data from genuine studiesImprove student learning of statistical ideas?Increase student interest in statistics?Enhance transferability of skills?Do students learn more if tactile simulations are presented before technology ones?Does it matter whether students construct or simply consume simulation results?Ask Good Questions (ICOTS-8)Rossman

    *Ask Good Questions (ICOTS-8)

  • *Some final questions for youHave you ever attended a presentation titledAsk Bad Questions, orDont Ask Questions?Of course not! So, Why am I wasting your time offering such obvious advice as Ask Good Questions??RossmanAsk Good Questions (ICOTS-8)

  • *Yeah, why am I wasting your time?I respectfully suggest that the next time you or I feel compelled to invest some of our valuable time, thought, energy, and creativity on Preparing crystal-clear lecturesWriting lucid paragraphs of expositionDeveloping software illustrationsCrafting beautiful presentations

    RossmanAsk Good Questions (ICOTS-8)

  • *What should we do?RossmanAsk Good Questions (ICOTS-8)

    *Ask Good Questions (ICOTS-8)

  • *What should we do?Resist this temptation!RossmanAsk Good Questions (ICOTS-8)

    *Ask Good Questions (ICOTS-8)

  • So, what do I suggest instead?Instead we should invest these precious commodities (time, thought, energy, creativity) onDeveloping engaging classroom activitiesPreparing thought-provoking assignmentsWriting lab or project expectationsCrafting effective assessment itemsIn other words, we should focus more of our attention on making sure that we

    *RossmanAsk Good Questions (ICOTS-8)

  • *Whats my point again?Ask Good Questions!RossmanAsk Good Questions (ICOTS-8)

  • Thanks very [email protected]:// Good Questions (ICOTS-8)Rossman

    *Ask Good Questions (ICOTS-8)