Validity And Reliability in Research
Validity And Reliability in Research
Empirically sound research includes data that is both valid and reliable. In broad terms, validity and reliability are measures of accuracy in health education research. The study design, including sample size, methodology, and analyses must be carefully monitored to accurately assess the soundness of the results. Highly credible research is built on the basis of controlling both internal and external validity, in order to generalize the study results to other similar populations (Cottrell & McKenzie, 2011). In epidemiological studies, validity of screenings and tests is defined as the “ability to distinguish between who has a disease and who does not" (Gordis, 2009, p. 89). Validity is measured by sensitivity and specificity, that measure the tests rate of correctly identifying those who have or do not have the disease (Gordis, 2009). Reliability refers to the repeatability of a test, as sound research must be replicable in order to be useful (Gordis, 2009). Intrasubject, intraobserver, and interobserver variations may occur that affect study reliability, as naturally occurring changes in subjects or biases within the observation process may alter study outcomes. Controlling for both valid and reliability requires the use of specific statistical tests, such as the kappa statistic, to ensure the study provides credible results (Rosner, 2011).
As an example, a recent study of the benefits of chocolate milk on football player cognition, provides multiple blatant examples of study invalidity. The study examined one brand, which ultimately was discovered to be a corporate partner of the University of Maryland. The study was conducted with no treatment and control group, no additional brands included in the study, nor publication of results before it was publicized as a successful study (Belluz, 2016). Rather, the first publication of results was located on the milk producer’s website, therefore no peer review occurred before it the product marketing had begun. The university is currently conducting an internal investigation after media criticism brought attention to the lack of validity. A reliability error would include a study with a sample size that is too small to produce consistent results. For example, a small study measuring the mean cholesterol level in older adults taking a particular statin drug would need to have a power-based sample size test to determine the number of participants necessary to be deemed reliable (Rosner, 2011). If only 20 older adults are measured, it would be highly improbable that the mean rate would remain constant if another 20 participants were measured. In other words, it is not a repeatable test because it does not meet the sample size criteria for such a study.
The differences between the two measures of research quality are significant and fairly distinct. In order to better explain the concepts, a simpler definition may be that validity measures the strength and accuracy of the test, while reliability measures test consistency over multiple tests. Student researchers, in particular, must understand the concepts of validity and reliability, and identify the appropriate tests for each aspect, in order to conduct a sound study. When conducting dissertation research, the dissertation committee members will oversee the validity and reliability of the study, monitoring for a sound plan and study design, including appropriate methods, sample size, analysis, and reporting. These steps must be developed in the early stages of planning to ensure the data and outcome can be evaluated for accuracy.
As an example, a quantitative dissertation research project that examines specific health determinants of midlife women would require a valid methodology, such as an acceptable questionnaire, void of intentional bias and clearly written in language that is appropriate for the sample group, in order to accurately measure the associated variables. The questionnaire would have to be distributed to a large enough sample group in order to be considered reliable, and analyzed as determined in the study design, with oversight from the dissertation committee. If appropriate, the results may then be generalized to similar groups or to the larger population represented by the sample.
References
Belluz, J. (2016, January 16). The incredible tale of irresponsible chocolate milk research at the University of Maryland. Vox Science & Health. Retrieved from http://www.vox.com/ 2016/1/16/10777050/university-of-maryland-chocolate-milk
Cottrell, R. R., & McKenzie, J. F. (2011). Health promotion & education research methods: Using the five-chapter thesis/dissertation model (2nd ed.). Sudbury, MA: Jones and Bartlett.
Gordis, L. (2009). Epidemiology. (5th ed.). Philadelphia, PA: Saunders Elsevier.
Rosner, B. (2011). Fundamentals of biostatistics (7th ed.). Boston, MA: Brooks/Cole.