Educational Research

From E-Learning Faculty Modules

Contents

Module Summary

There are a number of reasons for conducting educational research. Foremost, it is to improve the practice of teaching and learning, in the hopes of training up more and more effective instructors and instruction for the future. For online practitioners-in-the-field and other researchers, there are various areas of research interest:

  • Various methodologies for teaching and learning
  • Ways to improve online group learning
  • Design methods for various types of digital learning objects, learning sequences, and online courses
  • Working with the human factors in learning (perception, cognition, various abilities, and other details)
  • The integration and application of new technologies in teaching online
  • Fresh insights about online teaching and learning in particular domain fields

There are many ways to conduct educational research. Based on some of the literature and research practices, this module will highlight some points of consideration. This module is too brief to do more than provide a basic overview to a highly complex topic.

Takeaways

Learners will...

  • Define domain-field standards for the conducting of educational research and analysis.
  • Identify some of the types of educational research being practiced today. Further, consider when a particular type of educational research may be applied to a particular situation.
  • Define some of the methodologies for conducting educational research today along with some of the strengths and weaknesses of the various educational research approaches.
  • Explain some approaches to data analysis.
  • Consider some publication outlets for educational research.


Module Pretest

1. What are some domain-field standards for the conducting and analysis of educational research? How are these standards applied to the work?

2. What are some types of educational research being practiced today? When are particular types of research applied to particular situations? What are some methodologies for conducting educational research? What are the respective strengths and weaknesses of these various types of educational research approaches?

3. What are some common approaches to data analysis?

4. What are some publication outlets for educational research? How do these various types of publications differ?


Main Contents

This main section will highlight some basic information about educational research. This module will be updated as new information is available and as time allows.

1. Some Standards for Conducting and Analyzing Educational Research

POLICY STANDARDS

Human Subjects Review

There are a thicket of legal and policy standards that define the values around human subjects research. “Human subjects research” involve a panoply of human well-being concerns from physical and emotional health to other concerns. Professionals in the university’s compliance office help evaluate research proposals to ensure that the research fits within these guidelines.

Learner Privacy Protections

Learners have various personal privacy protections under the federal Family Education Rights and Privacy Act (FERPA). This restricts faculty and staff from releasing student information (beyond publicly-released catalog information). This means that if a faculty members wants to capture a student’s likeness—in photos, in video captures, in audio, and in multimedia, they need written and express permission (in “media releases”) by each of the students. The students have to make the decision to release this information of their own free will.

Intellectual Property Protections

Students also own de facto right to their own work products. These include their homework, portfolios, and other work products. Instructors cannot generally usurp university students’ homework for research or even future classes. Rather, faculty members must request permission to use the student work for a particular purpose and attain a legal release to do so.

The exception is as follows: Student work done in a university lab generally does not belong to the student. Rather, the intellectual property (IP) belongs partially or totally to the university—based on the university’s IP policy and case law.

EDUCATIONAL RESEARCH STANDARDS

Alignment with all Applicable Domain-Field Standards

Any educational research has to generally align with both the standards of the literature as well as the particular specific domain field’s standards and practices. In other words, if an educational research project is done in the field of public health, then the resulting work has to align with both the values of the educational research literature and of public health.

Review of the Literature

Educational research requires a solid review of the literature. This is to not only contextualize the issue for the readers, but it’s to ensure that the research actually is an outgrowth of what is already know but also that it contributes to the existing knowledge base. This review should be relevant, comprehensive (with a thorough vetting of the literature), updated to the present (with current-year citations), and accurate. This review offers a digital trail to show the researcher’s diligence in integrating prior research.

A further assumption of this review is that the research topic cannot be totally outside of the researcher’s area of expertise. Rather, the idea is that the research sprang from the researcher’s areas of interest and expertise. The data should at least not totally be outside of the researchers’ areas of expertise.

An often unspoken issue is that research can generally not go against the understood political values of the society. Or rather, if such research does go against mainline ideas, such works will likely not appear in mainline publications but may find an audience in more peripheral publications.

Transparent Research Methodology

The research methodology should be defined in such depth that the research should be repeatable by others. Or, if the situation is not repeatable, then an outside evaluator should at least be able to discern where the data came from from which certain conclusions were drawn.

Sufficient Data

Another standard for educational research involves the amount of data from which conclusions are drawn—whether the research is qualitative, quantitative, or mixed method in approaches. For statistical significance, there needs to be a certain number of participants in a research study. If a certain number was not reached, then that shortfall should be reflected in terms of the data analysis—with a pull-back from the generalizability of the data.

Another standard, then, is the protection of data for a certain period of time. This often ranges from between 3 – 7 years.

Quality Data Analysis

Researchers who are examining a particular data set have to analyze it for research value. They cannot over-reach or over-assert. They should be able to identify anomalies in the data. They should be able to draw some conclusions from the information—based on their initial research designs. Further, they need to be able to extract ideas for future research—that builds on the extant work.

Meritocracy in Academic Publishing

Researchers should only submit one manuscript to one publisher at a time. Even if the publisher takes a year or two to decide on the merits of publishing or not publishing, a researcher cannot bypass that wait by offering multiple submissions to multiple publishers.

In academic publishing, the gold standard is to have a double-blind review of all manuscripts, with a detailed rubric explaining the various variables that should be analyzed. There is an ideal to maintain a meritocracy for work and to avoid nepotism. Usually, the peer reviews or critiques result in one of three decisions: outright acceptance, conditional acceptance, or outright rejection.

While multiple publishable papers may be written from one research project, the researchers have to be careful not to have much overlap between such papers. Academic publishers want to debut new works, and they do not want to have a work that is a close copy of another. Also, unpublished publications will “date out” and lose value over time unless it can find a publication which will accept it for publishing.


2. Some Types of Educational Research Practiced Today

There are various types of educational research practiced today. It is beyond the purview of this short article to summarize the topics. However, there are some basics about educational research that may be summarized.

A Relevant Research Question

A research question emerges from the totality of the researcher’s professional experience, his / her reading of the research in the field, and the needs of the domain field. In a sense, a research question has to bridge the past and the future. This is where a review of the literature is necessary to give a historical context. It is also critical to identify relevance of the research findings for the future (research and practice)…to bolster the value of a research question.

A research question may evoke a variety of hypotheses. These hypotheses will have to be tested with the research.

The Design of the Research

The way research is designed depends on what the researcher wants to know. Then, there is strategizing on how to elicit that information (in as unbiased a way as possible) in order to answer the research question(s). The point is to arrive at that information in an untainted and accurate way. Research is a highly formalized process with various conventions in higher education.

Design of the Test Instrument

Sometimes, test instruments are created to elicit information. These may be surveys, or simulated spaces, or an experimental design setting (like a mock classroom). The point is to build an instrument that will surface the information desired. To this end, test instruments have to be tested and validated for the particular desired use.

Analysis and Discussion

The value of research comes not only by how it is conceptualized and conducted but also by the strength of the analysis. This section has to show the relevance of the research and findings. This section should highlight new information and the implications of that new data.

THREE GENERAL CATEGORIES OF RESEARCH TYPES

1. Quantitative

Quantitative or positivist approaches have been used as a traditional research approach for millennia, with researchers formulating hypotheses and testing these in experimental designs (whether in laboratories or in the world). Quantitative research surfaces statistical data that may be analyzed for various insights.

Surveys are another form of quantitative analysis.

In online learning, back-end data analysis of socio-technical systems (like learning / course management systems) or data-mining may surface other insights.

Another form of quantitative analysis involves pre- and post-test testing to see if a particular learning intervention may have affected measurable learning.

To reiterate, quantitative methods include the following:

  • experimental methods
  • surveys
  • data mining socio-technical systems
  • pre- and post-test testing

and other forms.

2. Qualitative (Post-Positivist Research)

Qualitative (or post-positivist) research refers to a more relativistic sense of the world in which realities are co-created and perceived through people. In this context, empirics are not used to arrive at Newtonian facts of the universe but to surface subjective realities.

Common tools in the qualitative research toolset include the following:

  • surveys (which include open-ended questions which enable text analysis or content analysis);
  • interviews (conducted in the field in natural environments to surface specific information and “rich descriptive data” (Taylor & Bogdan, 1998, p. 135) while using “analytic induction” (Taylor & Bogdan, 1998, p. 139)
  • narratives (a form of research inquiry in which the research participants / subjects respond with personal stories and oral histories)
  • phenomenologies (the identification of the “essence” of a lived human experience based on a certain phenomenon as observed by the qualitative researcher) (Creswell, 2003, p. 15)
  • ethnographies (the study of an intact cultural group in a natural setting over a long period of time to observe and collect data) (Creswell, 2003, p. 14)
  • case studies (real-world incidences and experiences that inform a particular professional practice)
  • heuristic research / “discovery” inquiry [in which the researcher immerses himself / herself into a particular context for deep experiential observations and reflections (and data-gathering) and deep self-reflections for self-discovery / self-processes]. Here, heuristic research is a way of deep immersion of the self—in which the researcher is the research tool, honed to be self-aware and surroundings-aware, for meaning-making of direct and lived experiences (with the initial data beginning inside the self, according to C.E. Moustakas, who originated this approach)
  • grounded theory (in which theories are surfaced by deep immersions in particular experiences and spaces and data, and with the use of abductive logic and with observed data disciplining the theory and theory formation—and without the uses of a priori assumptions) (Glaser & Strauss, 1967/1999). Here, researchers actively search for disconfirming or negative cases. They delimit the theory after datamining in-depth and considering various degrees of plausibility.
  • a Delphi study (in which a group of experts are asked to respond to questions about their domain field separately, and the contents are collated)

The idea is that the research method should not limit the potential findings. Rather, the research method should open up particular insights to the researcher. Sufficient immersion should lead to a “conceptual leap” or insight by qualitative researchers (Piantanida & Garman, 1999, p. 172).

Mixed Methods

Creswell (2003) suggests that mixed methods research involves a combination of quantitative and qualitative research and analysis…that may be “sequential, concurrent, (and) transformative” (p. 13).


3. Common Approaches to Data Analysis

The methods of data analysis relate closely to the research method, the software used to analyze the data, the particular research questions asked, and even the findings that the researchers find.

An initial (and general) approach may consider some of the following questions.

  • What did the data find?
  • How solid was the research behind that data? Was there a sufficient number of participants for statistical significance? Were there any gaps in the data? (If so, how may that be rectified—or at least acknowledged?)
  • Were there any anomalies (or surprises) to the data? If so, what?
  • What are some possible interpretations of the data? What are some of the possible implications?
  • What are some relevant questions to ask in future research?
  • What improvements could have been made on the actual research to attain the information needed?


4. Some Publication Outlets for Educational Research

Proprietary and Open-Source Publishers

There are multiple types of publishers for academic writing. The traditional type involves proprietary and commercial publishers. Many of these have gone to using information technologies to “ingest” manuscripts and to take these through the editorial, review, and revision processes.

Another type involves open-source publishers, which ask writers to release their works into the public domain, through various licenses (like the Creative Commons licensure).

Both types of publishers require that the authors own rights to their contents. However, open-source publishers allow the integration of open-source digital contents from third-party sources that have been released through an open-source license.

The reputation of a publisher in the academic community is an important aspect to consider, too. The point of publishing is to “signal” the relevance of the research work, and it is also to reach a wider readership / audience with similar interests.

Editorial Vision

Generally, it’s a good idea to make sure that the written work aligns with the editorial vision of the publishers—at least in terms of themes. Publishers have their own niches and angles in relation to a domain field. Some editors provide more support to authors than others, and that should be an important point of consideration.

Publishing Platform

Various publishers have different publishing platforms, with different affordances (enablements) and looks-and-feels. If an article is one that would be best presented in a high-tech forum, then certain publications may be selected and others discarded. Some publications offer works in multiple formats (like HTML and pdf).

Types of Formalized Articles

  • Research papers—These are the most traditional kinds of forms of academic publications for research. These follow a basic structure: abstract (or executive summary), key words, introduction, review of the literature, research, analysis of the research, discussion, and conclusions (with a references list at the end).
  • Case studies—Case studies are research situations with an n = 1. From the one from-life experience, various observations are made that may have transferability or applicability to others.
  • Concept papers—A concept paper is a research-based “thought experiment” that follows through on an idea that may be applicable to the field. This is a form of educated speculation--that is informed by experience, reflection, and valid data.
  • Position papers—A position paper takes an informed position on a particular issue, or it offers a proposed solution to a known problem in the field. The position paper cannot be mere opinion. Rather, it should cite the relevant research literature and apply logical concepts to the analysis of the research. These are a kind of formalized editorial.
  • Book review—A book review basically contains a summary of the book contents and then analysis of the work. The research here involves a thorough reading of the original work, any cross-referenced works, and the analytical mind of the reviewer.

Examples

Educause Review Online

The International Review of Research in Open and Distance Learning

Journal of Online Learning and Teaching (of MERLOT)

How To

The setting up of effective educational research is non-trivial. Researchers earn graduate degrees in order to learn how to conduct research well. In that spirit, the steps to conducting educational research will not be summarized here. Rather, readers are asked to do sufficient research and learning about methodologies before beginning research.

Possible Pitfalls

There are several areas of concern for those conducting educational research. This segment will highlight some areas of concern.

Sufficient Resourcing

First, for research to be effectively conducted, it is critical that the researcher and / or research team have the sufficient resources to follow through. Resources include such things as human resources (hours), funding, hardware and software, and participants for the research. There also has to be sufficient political will in the organization.

A Relevant Research Question

The research question has to be sufficiently relevant to the organization and to the larger public sphere. This assumes that the researchers have in-depth knowledge of the extant information in the research literature in order to know what to pursue.

Clear Planning

One of the most critical phases of educational research involves the setting up of the research (the research design). How this is set up often determines whether the research has relevance, whether the research is conducted intelligently, and whether the data from the research has any value.

If the research will be continued over many numbers of years, the same general information has to be collected in a particular way for equivalence and applicability.

This phase is critical because any problems which may be anticipated should be raised and addressed early on. It helps to have sufficient professional eyes on the project in order to identify potential challenges. (A doctoral committee serves this function for doctoral students.)

Human Protections

Institutions of higher education have compliance offices to ensure that institutional review board (IRB) standards are followed. IRB standards have to be considered for any research that can have any sort of effect on human subjects. At Kansas State University, this work is analyzed by the University Research Compliance Office. Vulnerable populations require special protections (Gall, Gall, & Borg, 2003, p. 69). Attaining the proper permissions is a critical part of the research.

Students also have privacy protection rights. They have a right not to be videotaped or recorded generally in their classes. They have a right not to release any of their personal information for publication (particularly not in any sort of personally identifiable way). Further, students have intellectual property rights to their own creations. This means that their homework and other elements cannot be used for research or publication, generally, without their written and express permission. To keep the lines of decision-making clean, most faculty are not allowed to require release of such privacy rights and of intellectual property rights as a precondition of the student taking the class.

Further, any research that has implications on people must have data that is protected and kept for a certain length of time (generally speaking, 3 - 7 years, depending on the context). When it is time to expunge research data, it has to be done in a way that protects human participants from the leakage of personal information.

Follow-Through

Once the research has been approved, the researcher and / or team have to follow the steps very methodically. The research should also be journaled for further learning.

Data Analysis

The types of information gathered from various research methods will vary. Quantitative research results in raw numbers that may be statistically analyzed by various software packages for value. Data sets usually have to go through much information scrubbing before the data may even be minimally comparable, if data sets are mixed.

Qualitative information may be analyzed through various text analysis software packages as well. In some circumstances, such information may be analyzed manually—depending on the amount of data. Heuristic research, grounded theory, and case studies may be sufficiently small and focused to not necessary require the use of qualitative data software analysis.

For mixed methods, various tools are usually applied to analyze the data.

The pitfalls in data analysis include the following questions:

  • Are there sufficient findings from the research conducted? Are there any gaps in the data?
  • Was the research conducted in a way that bias was not introduced into the data?
  • What exactly do the findings say?
  • What anomalies were found in the data? Any plausible interpretations?
  • How do the research findings correlate with the extant research in the field?
  • How generalizable are the data? Has the researcher over-reached or over-asserted?

The Dissemination of Research

A majority of funders of educational research have stipulations in the grants that the findings of the research be broadly disseminated. The idea is to magnify the potential benefits of the learning to a broad spectrum of users.

A challenge then is to find a publication venue that is legitimate, so the article fits with the publication’s focus and readership. If a work is not published within a certain window of time, the information dates out and no longer has value. A work has to be sufficiently competitive in order to appear in published form. In other words, it has to be fairly unique and insightful. It has to add to the body of literature in a refreshing way.

Module Post-Test

1. What are some domain-field standards for the conducting and analysis of educational research? How are these standards applied to the work?

2. What are some types of educational research being practiced today? When are particular types of research applied to particular situations? What are some methodologies for conducting educational research? What are the respective strengths and weaknesses of these various types of educational research approaches?

3. What are some common approaches to data analysis?

4. What are some publication outlets for educational research? How do these various types of publications differ?


References

Creswell, J.W. (2003). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. 2nd Ed. Thousand Oaks: SAGE Publications.

Gall, M.D., Gall, J.P., & Borg, W.R. (2003). Educational Research: An Introduction. 7th Ed. Boston: Pearson Education.

Glaser, B.G. & Strauss, A.L. (1967/1999). The Discovery of Grounded Theory: Strategies for Qualitative Research. New York: Aldine de Gruyter.

Piantanida, M. & Garman, N.B. (1999). The Qualitative Dissertation: A Guide for Students and Faculty. Thousand Oaks: Corwin Press, Inc.

Taylor, S.J. & Bogdan, R. (1998). Introduction to Qualitative Research Methods: A Guidebook and Resource. New York: John Wiley & Sons, Inc.

Extra Resources

Gall, M.D., Gall, J.P., & Borg, W.R. (2003). Educational Research: An Introduction. 7th Ed. Boston: Pearson Education.