The 4 Differences Between Reliability And Validity (in Science)
Given that in colloquial language they have very similar meanings, it is easy to confuse the terms reliability and validity when we talk about science and, specifically, psychometrics.
With this text we intend to elucidate the main differences between reliability and validity We hope you find it useful to clarify this very common doubt.
In psychometrics, the concept “reliability” refers to the precision of an instrument ; Specifically, the reliability coefficients inform us of the consistency and stability of the measurements taken with said tool.
The greater the reliability of an instrument, the fewer random and unpredictable errors that will appear when using it to measure certain attributes. Reliability excludes predictable errors, that is, those that are subject to experimental control.
According to classical test theory, reliability is the proportion of the variance that is explained by the true scores. Thus, the direct score on a test would be composed of the sum of the random error and the true score.
The two main components of reliability are temporal stability and internal consistency The first concept indicates that the scores change little when measured on different occasions, while internal consistency refers to the degree to which the items that make up the test measure the same psychological construct.
Therefore, a high reliability coefficient indicates that scores on a test fluctuate little internally and over time and, in summary, that the instrument is free of measurement errors
When we talk about validity we refer to whether the test correctly measures the construct it aims to measure. This concept is defined as the relationship between the score obtained on a test and another related measure ; The degree of linear correlation between both elements determines the validity coefficient.
Likewise, in scientific research, high validity indicates the degree to which the results obtained with a certain instrument or in a study can be generalized.
There are different types of validity, which depend on the way it is calculated; This makes it a term with very diverse meanings. Fundamentally we can distinguish between content validity, criterion (or empirical) validity and construct validity
Content validity defines to what extent the items of a psychometric test are a representative sample of the elements that make up the construct to be evaluated. The instrument must include all the fundamental aspects of the construct; For example, if we want to make an adequate test to measure depression, we must necessarily include items that evaluate mood and decreased pleasure.
Criterion validity measures the ability of the instrument to predict aspects related to the trait or area of interest. Finally, construct validity aims determine if the test measures what it purports to measure for example from the convergence with the scores obtained in similar tests.
Differences between reliability and validity
Although these two psychometric properties are closely related, the truth is that they refer to clearly differentiated aspects. Let’s see what these differences consist of
1. The object of analysis
Reliability is a characteristic of the instrument, in the sense that it measures the properties of the items that compose it. On the other hand, validity does not refer exactly to the instrument but to generalizations made from the results obtained through it.
Although it is a somewhat simplistic way of putting it, broadly speaking, it is usually stated that validity indicates that a psychometric tool actually measures the construct it aims to measure, while reliability refers to whether it measures it correctly, without errors.
3. The way they are calculated
To measure reliability, three procedures are fundamentally used: the method of two halves, the method of parallel forms and the test-retest The most used is the two-halves procedure, in which the items are divided into two groups once the test has been answered; The correlation between the two halves is then analyzed.
The method of parallel or alternative forms consists of creating two equivalent tests to measure the extent to which the items correlate with each other. The test-retest is simply based on passing the test twice, under conditions that are as similar as possible. Both procedures can be combined, giving rise to the test-retest with parallel forms, which consists of leaving a time interval between the first form of the test and the second.
For its part, the validity It is calculated in different ways depending on the type, but in general all methods are based on the comparison between the score on the objective test and other data from the same subjects in relation to similar traits; The goal is for the test to act as a predictor of the trait.
Among the methods used to evaluate validity we find factor analysis and the multimethod-multitrait matrix technique. Likewise, content validity is often determined through rational, not statistical, analyses; For example, it includes face validity, which refers to the subjective judgment of experts about the validity of the test.
The reliability of a psychometric instrument influences its validity: The more reliable it is, the greater its validity will also be Therefore, the validity coefficients of a tool are always lower than the reliability coefficients, and validity indirectly informs us about reliability.
