Showing posts with label methodology. Show all posts
Showing posts with label methodology. Show all posts

Sunday, February 24, 2013

Epistemological Pragmatism: The Function of Empirical Experiments

The various research methodologies provide multiple means with which an individual may assess a situation. The most appropriate procedures for generating and interpreting information are often determined by the values of the researcher and the specific purpose of a study. However, experimental methods are the most effective means with which one may evaluate theoretical relationships. Although many research strategies generate meaningful information, hypotheses can only be conclusively refuted by contrary evidence (Kabadayi, 2007). While other methods may indicate the nature of a connection between variables, experimentation is the most direct means with which one may determine the significance of antecedent conditions.

Creswell (2009) describes a continuum of research methods within which each specific procedure emphasizes a particular degree of quantitative and qualitative elements. The most appropriate methodology will generate data that is relevant to the purpose of the study. Qualitative studies attempt to explore and represent the general subject of the study as accurately and completely as possible. While case studies explore a specific topic in depth, ethnography and phenomenology focus more on the interpretation of subjective perceptions. However, although qualitative research methods provide the means with which one may construct a representation of some phenomenon, the validity of any inductive generalizations must be evaluated through deductive strategies.

Creswell (2009) explains that quantitative methods attempt to assess the validity of a general theoretical relationship through particular comparisons between hypothetical predictions and objective measurements. Theories are interpretations that explain the relationship observed between objective variables. Scientific disciplines establish theories that are founded on empirical evidence (Çakir, 2012). In quantitative studies, the significance of a theory is evaluated in the context of a relationship that is quantified through the statistical interpretation of numerical values (Creswell, 2009). While surveys assess theories in comparison to collected data, experiments directly test theoretical relationships through comparisons between experimental and control groups. In an experiment, the effect of some manipulation, the independent variable, is quantified in reference to the differentiation it produces, the dependent variable, relative to a control group that is not does not receive the treatment.

Experiments are superior to other forms of research, because this methodology transcends mere correlations through the direct evaluation of the causal relationship between variables. While other methods may indicate some type of connection, experimentation is the only means with which one may determine the causal mechanism that produces observed correlations. Mere observation does not confirm any theoretical interpretation regarding causality. In order to understand the nature of an objective relationship, a researcher must interact with variables to determine what antecedent conditions result in dependent change.

The accuracy of scientific knowledge is increased through the elimination of false theories (Kabadayi, 2007). The scientific function of qualitative studies is limited because no number of finite observations is conclusive evidence to verify a theory with infinite applications. Any research methodology can only conditionally confirm a theory (Çakir, 2012). Regardless of the quantity of evidence that may support an idea, a scientific theory is perpetually open to the possibility of disproval. Any general theory may be conclusively refuted by contrary data. While other strategies may provide information that is relevant to a theory, falsification is most conclusive when founded on a comparison between experimental and control groups that differ in only a single feature.

Although experimentation is the most effective method for testing a theory, other methods are more suitable for alternative agendas. The most appropriate strategy of inquiry must be selected based on the purpose of a research project (Creswell, 2009). Scientific experimentation will produce little progress in the exploration of subjective meanings and interpretations. Qualitative studies like phenomenology and ethnography are much more appropriate for research that is intended to discover and describe human perceptions rather than assess their objective truth. Similarly, case studies are more appropriate when a researcher is seeking to represent only a specific situation. Additionally, the establishment of control and experimental groups may not be possible or morally permissible in many circumstances. Quantitative surveys are often more appropriate in situations involving large populations or when experimental manipulations are not practical.

Experiments do not produce absolute knowledge. Rather, any assessment is limited by the degree to which the strategy of inquiry compares the hypothetical significance of a theory to objective variables (Çakir, 2012). While experimental falsification is conclusive evidence for rejecting a theory, measurements that are consistent with the hypothesis do not necessarily confirm any truth. Ladyman, Douven, Horsten, and van Fraassen, (1997) explain that science functions to construct theories that are only sufficiently accurate to represent truth in practice. Empirical verification is only a means with which a researcher may evaluate the consistency between their theoretical understanding and empirical measurements. This method of comparison is the foundation for valid theories that enable accurate interpretations of objective relationships between variables (Creswell, 2009). Experiments are superior to other research methods, because they explicitly establish conditions that enable direct comparisons between a theory and empirical measurements. 

References

Çakir, M. (2012). Epistemological dialogue of validity: Building validity in educational and social research. Education, 132(3), 664-674.

Creswell, J. W. (2009). Research Design (3rd Ed.). Los Angeles, CA: SAGE Publications.

Kabadayi, T. (2007). The second verificationists. Eurasian Journal of Educational Research, 29, 35-43.

Ladyman, J., Douven, I., Horsten, L., & van Fraassen, B. C. (1997). A defence of van Fraassen’s critique of abductive inference: Reply to Psillos. Philosophical Quarterly, 47, 305–21.

Wednesday, February 6, 2013

Meta-Analysis and Post-Modernism

The paradigms that characterize particular academic disciplines often orient research towards a style of design (Creswell, 2009). While all research contains both quantitative and qualitative elements, most studies emphasize one approach more than the other (Creswell, 2009). These research designs are not dichotomous opposites (Creswell, 2009). Rather, quantitative and qualitative designs represent the extremes of a continuum of possibilities (Creswell, 2009). For example, in mixed methods research, qualitative and quantitative forms are combined in tandem to strengthen any conclusions beyond the capability of a limited design (Creswell, 2009). By embracing a variety of assessment types, a mixed methodology transcends the limitations of a single research paradigm. However, a research design must be valid in the context of the specific problem. Researchers must justify the design of their study by demonstrating the functionality of selected means (Creswell, 2009).

Qualitative research is consistent with orientations are more concerned the exploration of meanings or the generation of change (Creswell, 2009). Constructivism is one such type of paradigm that focuses on the subjective interpretation of meaning (Creswell, 2009). Rather than test the validity of a perception, constructivist research explores the variety of subjective concepts and often attempts to describe the diverse interpretations of individuals. Other orientations toward advocacy emphasize collaborative participation to generate positive change (Creswell, 2009). This type of research attempts to understand a problem and identify the most appropriate means with which the conditions may be improved for individuals. Because of these exploratory functions, qualitative research tends to examine open ended questions (Creswell, 2009). Rather than adhere to a specific research design, qualitative studies must continuously develop in the context of the situation (Creswell, 2009). This ensures that the methods and interpretations are appropriate for the situation (Creswell, 2009). In qualitative research, themes are inductively interpreted from data gathered in an emerging design (Creswell, 2009). However, although this type of study may produce information that accurately represents a particular situation, the validity of any generalizations is limited because of the specificity of collected data.

In contrast, quantitative studies typically attempt to evaluate objective theories through deductive tests of specific hypotheses (Creswell, 2009). This type of research is often scientifically oriented. Postpositivism is one such type of orientation that emphasizes the empirical examination of relationships between variables (Creswell, 2009). From the postpositivist perspective, validity is defined as the degree to which perceptions are consistent with objectivity. Valid theories accurately represent the nature of a relationship between variables. Similarly, pragmatism is another paradigm that often emphasizes quantitative research (Creswell, 2009). Pragmatist research derives conclusions from functional applications of a theory (Creswell, 2009). Functional theories are the foundation for hypotheses that accurately predict the consequences of specific events. In quantitative research, the resulting measurements are interpreted through statistical analysis to enable conclusions regarding the hypothesis (Creswell, 2009). While this research design may generate conclusions regarding the validity of a theory in a particular case, it does little to explore the diverse possibilities of meanings and perceptions regarding a situation.

Meta-analysis is one type of quantitative design that integrates information from multiple sources (Whiston & Li, 2011). In this type of research, empirical information is interpreted through the statistical analysis of existing studies (Whiston & Li, 2011). The meta-analytic research of Cheng, Cheung, Chio and Chan (2013) collected data regarding psychological symptoms from a variety of prior studies. This study examined the relationship between locus of control and symptoms of depression and anxiety in order to quantify the effect of cultural orientations (Cheng et al., 2013). Their conclusions represent a quantitative interpretation of 152 studies including 33,224 individuals total (Cheng et al., 2013). While meta-analysis is an appropriate means with which to assess the validity of a theory across a large sample of the population, the degree to which particular cases are accurately represented in the study is reduced by the lack of depth in this type of data collection.

Specific operational definitions are established to select relevant information that is then coded into a system of calculations to generate a numeric value (Whiston & Li, 2011). These definitions are directly related to the variables that constitute a hypothesis of a relationship. Locus of control refers to a subjective perception regarding the causal origin of occurrences (Cheng et al., 2013). Rotter (1966) explained that the cultural norms in individualist societies emphasize independence and self reliance. In contrast, Bond and Smith (1996) describe the values of interdependence and harmony that are prioritized in collectivist societies. Because individuals in collectivist cultures are socialized to accommodate changes rather than resist them, as described by Morling, Kitayama, and Miyamoto (2002), Cheng et al (2013) predict that these societies will exhibit symptoms of depression and anxiety at a reduced frequency compared to individualist cultures.

Cheng et al. (2013) noted that individuals within collectivist cultures may be more prepared to endorse an external locus of control. Their hypothesis proposes that the relationship between locus of control and negative psychological symptoms is stronger in individualist societies. Their data analysis indicated that perceptions of an external locus of control are moderately related to symptoms of depression and anxiety (Cheng et al., 2013). This relationship was stronger in societies that are oriented toward individualism (Cheng et al., 2013). The results of their meta-analysis are consistent with the hypothetical relationship between cultural orientations and psychological symptoms (Cheng et al., 2013).

References

Bond, M. H., & Smith, P. B. (1996). Cross-cultural social and organizational psychology. Annual Review of Psychology, 47, 205–235. doi: 10.1146/annurev.psych.47.1.205

Cheng, C., Cheung, S., Chio, J. H., & Chan, M. S. (2013). Cultural meaning of perceived control: A meta-analysis of locus of control and psychological symptoms across 18 cultural regions. Psychological Bulletin, 39(1), 152-188. doi: 10.1037/a0028596

Creswell, J. W. (2009). Research Design (3rd Ed.). Los Angeles, CA: SAGE Publications.

Morling, B., Kitayama, S., & Miyamoto, Y. (2002). Cultural practices emphasize influence in the United States and adjustment in Japan. Personality and Social Psychology Bulletin, 28, 311–323. doi:10.1177/ 0146167202286003

Rotter, J. B. (1966). Generalized expectancies for internal versus external control of reinforcement. Psychological Monographs: General and Applied, 80, 1–28. doi:10.1037/h0092976

Whiston, S. C., & Li P. (2011). Meta-analysis: A systemic model for synthesizing counseling research. Journal of Counseling & Development. 89(3), 273-281.

Sunday, February 3, 2013

Theoretical Validity: Induction and Deduction

In his literature review, Çakir (2012) explores the foundations of knowledge in education and the social sciences. The social sciences attempt to construct theories that are founded on empirical evidence (Çakir, 2012). These concepts enable meaningful interpretations of events. However, a theory is itself only a subjective interpretation that specifies a causal interrelation (Creswell, 2009). The validity of any proposition is a measure of the degree to which the perceptual construct represents the relationship between variables (Creswell, 2009). Because all such knowledge is established through inductive generalizations, the validity of these interpretations is limited (Çakir, 2012). Any knowledge is only a conditional hypothesis that may eventually be falsified. Science progresses through the replacement such theories as they are invalidated by empirical evidence (Kabadayi, 2007).

Although positivists assert that meaningful principles may be derived from empirical verifications, scientific systems must remain open to the possibility of falsification (Çakir, 2012). No finite number of observations can conclusively prove a theory with infinite applications (Kabadayi, 2007). This limitation of inductionist reasoning invalidates epistemological assertions founded on verification (Kabadayi, 2007). While logical principles may be confirmed through assessments in the context of objectivity, multiple theoretical constructs may often be applied to a single collection of data (Çakir, 2012). Without some form of neutral appraisal, the value of a paradigm is relative to a particular subjective perspective (Çakir, 2012). Because any learning represents an interpretation of novelty in the context of existing understanding, the validity of a theory is never independent of personal values (Çakir, 2012). Methods of verification are never able to transcend subjectivity and resolve perceptual incommensurabilities.

In contrast, deductive falsifications conclusively invalidate any theory (Kabadayi, 2007). A theory provides a rationale for hypothetical predictions (Creswell, 2009). A general theory is only conditionally validated through applications test derivative hypotheses in particular situations (Creswell, 2009). However, the process of evaluation is necessarily embedded within an existing set of practical methods (Çakir, 2012). In order to ensure validity, a theory must remain open to all deductive forms of criticism (Çakir, 2012). Because theories must be assessed through application, validity is defined in reference to specific a methodology rather than intrinsic metaphysical truth (Creswell, 2009). In order to maintain validity, a theory must not be exempt from empirical contradictions. Rather, by eliminating invalidated concepts, the degree of confidence regarding invulnerable ideas is increased. Although absolute certainty may never be achieved, the functionality of a theory in particular instances is ensured by continuously testing hypotheses in all possible applications.

References

Creswell, J. W. (2009). Research Design (3rd Ed.). Los Angeles, CA: SAGE Publications.

Çakir, M. (2012). Epistemological dialogue of validity: Building validity in educational and social research. Education, 132(3), 664-674.

Kabadayi, T. (2007). The second verificationists. Eurasian Journal of Educational Research, 29, 35-43.