Similar to last week’s implementation post, when you are evaluating a study that tout’s a program’s effectiveness you need to find out if the sample of students they studied are similar to the students you teach. Were the participants in the study primarily one ethnicity? Did they have a high or low income? Were they a similar age to your students?
Different people respond to things differently. For instance, students with a high-income background may not understand a program created with low-income students in mind. Problems may be tailored to a different set of experiences.
To make sure a program will work as cited, it’s important to match the study demographics to your students demographics.
One major caveat to this is that often studies that are well-done will strive for a cross-section of students. That is, they will recruit participants from all ethnic backgrounds and socio-economic statuses. In this case, the study write up will explain that in their sampling methodology. However, it is important to ask if there were any unique results patterns for demographic groups. For example, did white students show only a small amount of growth while black students showed dramatic growth? If so, and you have primarily white students, you may want to look for another program.
One final note on sample is that you want the sample to be large. In fact, the larger the better! It’s also important to note the sample size of the demographic you are interested in. If there are 200 high income students in the study, but 5 low income students and you teach primarily low income students, there aren’t enough in the sample for your to draw conclusions about the program’s success.
