Probability and Sampling
| Assessment vs. Evaluation |
Probability and Sampling |
The Significance of Significance |
Classic vs. Official View |
|---|
Probability and Sampling
Assessing Learning
Having said all of that, the issue at hand is.. how DO we assess whether learning is taking place?
Let’s assume for the sake of argument, that we have decided (or your client/subject matter expert (SME) has decided) that assessment plays a major role in the design and that we are dealing with less than precise metrics to measure so-called “success”.
What can be done?
Well, it is not easy. And there are many ways it can be done. So that will help us to make some predictions/assumptions, even though they are less than precise/finite… Their power to do so draws from their combination (the outcome is greater than the sum of their individual parts).
Before We Begin, One Huge Reality Check is Warranted Here…
Not to offend anyone but you may have noticed that, even though school districts and state departments of education want to believe it, all the standardized testing in the world may never yield the level of granularity that their proponents say or believe they do. There are simply too many variables. And, as you may have gleaned from any educational stats courses you have taken, the error term/significance level (.05) is rather large. It simply would not work if we were dealing with a cancer medicine, or building structures in civil engineering, etc. As an example, in Program Evaluation courses we are often introduced to white papers that describe how that once S.E.S. Providers (supplemental after school interventions) that can assist them with drop out prevention and remediation are approved, they are almost never later disapproved because in a court of law it is almost impossible to “prove’ that the do not work. Those who have been in education for a while realize that what is being done is about as best as can be done because there are simply too many variable and confounds that make most analyses less than precise. Having said all the above, we have plenty of tools at our disposal to make some well-thought-out and informed decisions, including statistics and probability to help us make some fairly accurate predictions. An integral part of our ‘science’ of learning is of course, our two instructional cornerstones: ASSURE and ADDIE models, both of which were borrowed from software engineering and other applied sciences.
How does Probability and Sampling Fit into the Concept of Informal Learning?
Dr. Jordan Ellenberg, in His Book “How Not to Be Wrong: The Power of Mathematical Thinking”, he introduces the reader to several examples of utilizing statistical probability and sampling to allow one to make informed choices regarding various topics, including how many affinity groups became savvy enough to beat the odds in several state’s Powerball Lotteries. (Some of you may have also saw the movie “21” with Kevin Spacey (based on the book “Bringing Down the House)) where a group of MIGHT students used statistics and card counting in Las Vegas. NO, we are not endorsing card counting, but simply helping you understand that these statistical measurement (such as the KR-20 model) can be our friend here, especially if we have enough data at our disposal… We could now diverge and discuss/debate the pros and cons of big data in education, but no time here… safe to say, however, that the more data you have the more power it brings to the table, and better predictions can be made… Back to the situation at hand…. current.
Probability and Sampling
In case you are unfamiliar with the concepts of probability and sampling, we can offer you two videos from the Annenberg Foundation. Here’s an overview of probability and how it works:
Against All Odds: Introduction to Probability
Video URL= https://www.learner.org/courses/againstallodds/unitpages/unit18.html
What is Probability
Proper sampling allows us to measure only a part of the population to be able to make some inferences about the whole:
Video URL= https://www.learner.org/courses/againstallodds/unitpages/unit17.html
What is Sampling
To fully understand sampling, we also need to understand distributions
Video =”https://www.learner.org/courses/againstallodds/unitpages/unit22.html
Sampling Distributions
The key to making all of this work is what is referred to as internal validity… the better you design your model the more accurate your inferences will be… all of this is the subject of the Program and Evaluation and Analysis courses mentioned above.