Empirical research with quality guarantee – Learn how

Welcome to the tutorial “Know the 6 quality criteria of empirical theses“. The quality or practical relevance of empirical research results is measured according to several mutually dependent standards, so-called quality criteria. In order to meet these standards, you must follow them when collecting, analyzing, and interpreting your data. The specific criteria you have to observe depend on whether your work is qualitative or quantitative. But in any case, it is essential for a high-quality research work to familiarize yourself with each quality criterion in advance. In this tutorial we will give you a short and concise overview of all criteria of quantitative and qualitative research, so that nothing stands in the way of the quality of your research. Let's go!

The 3 quality criteria in quantitative research

Let us start with the quality criteria of quantitative research. Three essential criteria are distinguished, namely: Objectivity, Reliability, and Validity. Each criterion has its own reference. While objectivity is primarily concerned with who conducts the research, reliability and validity refer to how and what is studied respectively.

In the context of quantitative research, research is objective, that is, neutral or factual, if the results are independent of confounding influences. Possible confounding factors are unequal experimental conditions or bias and conflicts of interest on the part of the persons conducting the research. Although complete objectivity is often impossible to achieve in practice, you can mitigate confounding influences by ensuring three types of objectivity, namely: implementation objectivity, evaluation objectivity, and interpretation objectivity.
Implementation objectivity is fulfilled if you do not proceed arbitrarily, but according to standardized rules that do not allow individual leeway. Accordingly, it must be clearly regulated, for example in a manual, how the data collection is to be carried out. In addition, the conditions of the data collection situation must be the same for all participants.
Evaluation objectivity exists if the data evaluation is independent of the respective evaluating person. Thus, different evaluators should arrive at the same or at least largely consistent results. For this purpose, clear evaluation guidelines and standardized evaluation instruments must be available.
Interpretation objectivity is given when the interpretation of the evaluated data is independent of the respective interpreter and therefore different persons ideally arrive at the same interpretation results. Standardized analysis instruments such as clearly defined benchmarks and norm tables can serve this purpose.

The reliability criterion focuses on the reliability of the research results. A research is reliable if the same results are obtained when it is repeatedly carried out, independent researchers arrive at identical or similar results under the same conditions, and the conclusions are comprehensible. Reliability is higher the more standardized and controlled the test conditions are.

In total, five types of reliability are distinguished: Interrater reliability, test-retest reliability, parallel-forms reliability, split-half reliability, and internal consistency reliability.
Interrater reliability occurs when data are analyzed by different researchers and consistent results are obtained.
Test-retest reliability means that repeated performance of a research investigation at different times leads to a correlation between the results.
Parallel-forms reliability is about obtaining the same or similar results with two different measurement instruments that measure the same variable.
Split-half reliability determines whether the results of a test divided into two halves, in which each item is intended to measure the same feature, correlate with each other. If not, the measurement is inconsistent.
Internal consistency reliability exists when, within a test, the individual items measure approximately the same thing. For example, a survey is internally consistent if the result on a single question correlates with the overall survey result.

The quality criterion of validity concerns the validity of a research or a measuring instrument. A measurement is valid if it measures what it is intended to measure. A general distinction is made between internal and external validity.
Internal validity is given if the occurrence of the measured values can be clearly explained and no alternative explanations exist, since possible confounding influences are excluded. A subcategory of internal validity is content validity. It is given when the survey method validly measures the content of a specific feature that cannot be measured directly, such as an attitude or belief, or when the measurement is aligned with the content of the feature of interest.
External validity means the generalizability and representativeness of the measurement results. The results are therefore externally valid if they are also valid outside a specific study and can be transferred to other situations, persons, and points in time. Subcategories of external validity are construct validity and criterion validity.
Construct validity means the consistency of a pre-defined theoretical construct, such as 'customer trust', with existing similar construct definitions from other research.
Criterion validity exists when the feature of interest is measured in such a way that there is consistency with relevant external criteria. In other words, the measurement results correlate with external results, insofar as these are based on the measurement of a similar construct.

The 3 quality criteria in qualitative research

Let us now turn to the quality criteria of qualitative research. Since the results of qualitative research show a higher degree of subjectivity, the quality criteria of quantitative research usually cannot be transferred here completely, but only approximately. For qualitative research, the following three criteria are often considered: Transparency, intersubjectivity, and reach.

Qualitative research is considered to be transparent if every important step of the research process is documented in detail and comprehensibly. This includes, among other things, clearly formulating the research question, presenting the survey context, justifying the methods used and describing their implementation, and explaining the data evaluation and interpretation. The criterion of transparency is similar to the criterion of validity in that it is about making visible whether what is actually being surveyed is what is intended to be surveyed.

The criterion of intersubjectivity focuses on making the research results available for discussion within the scientific community. It is fulfilled if the subjectively obtained results are comprehensible and understandable for external parties. The decisive factor here is to critically reflect on the subjectively obtained results and to offer alternative interpretations. The intersubjectivity criterion is based on the quality criterion of objectivity in that the subjective opinion of the researcher is not presented as the only correct one and outsiders can form their own judgment.

The criterion of reach refers to the generalizability of the research results. In order to fulfill it, it must be ensured that the repeated implementation of a survey procedure, such as an expert interview, leads to similar results in a similar study. For this purpose, it is necessary to define the scope of application precisely and to indicate to what extent the results can be generalized. Insofar as the criterion of scope is concerned with the repeatability of a research, a similarity to the quantitative criterion of reliability can be seen.