Article Critique – Task Stream Assignment
Florida International University
Article Critique – Task Stream Assignment
Introduction Summary
The article I critiqued is called “Interactions Among Online Learners: A Quantitative Interdisciplinary Study” by Pawan Jain, Sachin Jain, and Smita Jain, 2011. The study focuses on the amount of interaction students have with each other and with their professors to identify if interaction levels differ among disciplines.
Research Problem
The major research problem identified in the study is that there is not enough research concerning the matter of online interaction. The majority of the completed research is very discipline specific and cannot be generalized. The author’s justification for conducting the study was to remedy the lack of prior research on this increasingly important topic. The purpose is “to fill the gap and try to understand the relationship between the interaction and differences in discipline;” however, the authors also noted that this was “one” of the purposes of the study and failed to mention any other purpose.
While the authors did not specifically discuss their decision to utilize a quantitative approach in this study, it was clearly justified by their need to examine the relationship between discipline and study interaction. The theoretical basis that is used as the basis for this study is that increased interaction within an online course will ultimately lead to a better designed course and better outcomes for students grades. Further, the authors attest that much of the literature is inadequate for their study due to the specificity of the studies, disallowing them to be universally applied.
The research question is “do the differences in the discipline area impact the overall interaction among learners as defined by the number of learner-learner interactions?” The hypothesis that was tested is “there is no significant relationship between the number of learner-learner interactions and discipline area.” The only noted relationship that could be inferred between the theory and the research question/hypotheses is that there are not any studies that have been conducted that measure said information in a way that can be used for their purposes.
They note the importance of learner-learner interaction as a major pedagogical design; however, they continued to note the lack of available research.
METHODS SUMMARY
Measurement
The primary concepts/variables in this study were the observations of the amount of discussion posts by students in 39 different courses across 4 disciplines (College of Education, College of Business, College of Arts and Sciences and the College of Health Sciences). The identified independent variable was the 4 categories of discipline; the dependent variable was the overall interactions per student per week. The conceptual or operational definition of these variables was not mentioned beyond the above listed information. The authors did not address whether the measures were valid or reliable.
Research Design
Although the authors in this study did not explicitly state the specific research design, one can infer the researchers used a Randomized-to Groups Pretest-Posttest Design. Maturation could be a threat to the internal validity because of the time elapsed between the pretest and between posttest 1 and 2. Experimenter effects could also be a threat to internal validity because the study does not assure the reader that experimenter bias has not influenced the results. Subject effects could also be a threat to internal validity. Students may have responded differently knowing they were part of a study. The authors did not address whether or not the children knew they were being videotaped or used for a research study, which may have affected their behavior and responses. Because this study only included a sample from the majority one race/ethnic group (White), and was not specific to the SES of the participants, there is a clear threat to external validity. The study may not be generalized appropriately to a larger population.
Sampling
The population that the authors wanted to study was interdisciplinary students enrolled in online courses. The population from which the sample was taken does not represent all of the appropriate people. The sample was taken from Graduate online courses at a major university in the Rocky mountain region from four different disciplines. The sample is limited in that it excludes all students besides graduate students. There was no mention of a specific sampling technique that was used. I presume that it was a convenience sample. Additionally, there was not any address of the implications of the sampling technique for the findings.
There may be issues with external validity based on the limited population of the sample. Since they only sampled graduate courses, it cannot be generalized that the results would be similar for other class levels. The difference in intensity for graduate courses compared to undergraduate courses is not noted as it should be.
Data Collection
The researchers collected data based on observations of the number of discussion posts per student per week in each course across the four disciplines. The advantage of this data collection is that the information from the courses was archived and researchers needed only to count the amount of discussion posts. The disadvantages to this use of data collection include human error in counting, as well as the lack of distinction between quality and quantity of the posts.
Data Analysis Critique
The data was organized using SPSS 15.0 statistical software. Descriptive statistics were utilized to summarize, organize and simplify the data collected for the study Additionally, a one-way analysis of variance was used to find the relationship between the dependent variable and the nominal independent variable, discipline. Follow-up tests were conducted to analyze the pairwise differences among the mean and Scheffe’s post hoc comparison test was employed for this purpose.
Results Critique
The major findings of the study were that “the mean number of interactions per student per week was 4.76; the standard deviation for this variable was 3.89” (Jain et al, 2011, p. 541). The use of a one way analysis of variance was conducted and concluded that “differences in discipline accounted for 22% of the variance of the dependent variable” (Jain et al, 2011, p. 542). The results showed that the overall interaction in Arts & Science courses was significantly different than the overall interaction in Health Science courses. The interaction in Business courses was significantly different than both, the interaction in Health Science courses and the interaction in education courses. But no significant differences were found between the interactions in Arts & Sciences courses and interaction in Business courses and interaction in health Science courses and between interaction in education courses and the interaction in Arts & Sciences courses. Overall, the health sciences courses had a higher interaction rate than the rest, leading the authors to conclude that the “interactivity in an online class depends on the discipline it belongs to” (Jain et al, 2011, p. 543). I am not highly confident in the results because of the nature of the data collection as well as the limited sample used. Additionally, they do not provide for a practical use.
Implications of the Findings Critique
The conclusions that the authors reached were that interactivity was different based on the discipline of the course; these conclusions are appropriate but the author did not provide practical implications of the study. The only implications the author noted was that additional research would need to be conducted. The authors did not provide any practical information on how the conclusions of this study could be used to further develop courses. In my opinion, the significance of findings was minimal for my area of interest. Online education design and implementation is my area of interest and I did not find this study to be particularly helpful or beneficial.
My Contribution
Overall the study did not provide any practical information. I would suggest adjusting the data collection method to include additional information about the particular students from whom data was being collected (Are they full time or part time? Are they married? Do they have children? Do they work? If yes, do they work part time or full time?). The information gathered as to which students provided higher levels of interaction would be far more beneficial in designing online courses. I would keep the data collection of the graduate students, but also expand it to include undergraduate as well. I feel it is important to note the difference in interaction between graduate level business courses and undergraduate level business courses. Simply knowing that there is a difference among interactions does not provide any beneficial information in developing courses and activities designed to increase interaction. The authors noted on more than one occasion the lack of research already in this field; however, I felt that their contribution should have been more significant. They conducted this study, which in my opinion, still leads to a lack of necessary research.
Additionally, the actual layout of the information in their paper is not easy to navigate through. There was key information that was left out such as the sampling technique used, as well as any external factors that were not controllable. I would suggest changing the layout of the article and address key information in a clear and easy to follow way. The independent and dependent variables were not clearly addressed until the Results section of the article, when they should have been listed at the beginning of the Methods section.
The Theoretical Framework section was well laid out and the authors did a good job of defining any new terms that they used; however, they lacked a clear connection between the question and the theoretical framework. They were not clear as to specifically what the theoretical framework had to do with the questions at hand. Further, the purpose of the study was mentioned in the theoretical framework section as opposed to the opening portion of the study. The paper also noted in the theoretical framework that said purpose was “one of the purposes” (2011, p. 541), but another purpose was never clearly identified. I think it is important to lay out a clear purpose toward the beginning of the article so that the reader can easily identify what they will get out of the article.
As mentioned above, there was a significant lack of practical implementations. The results failed to provide information and suggestions on how to move forward. They simply suggest that interaction is good based on their theoretical framework and that in fact there is a difference in interaction amongst disciplines. I would change the procedures of the study to be able to provide practical uses, such as what can be done to increase interaction for students who have additional responsibilities such as full time jobs, spouses, and children? Information on when the discussion posts were completed as well as the timeframe as to how long the students were given to complete each post, would also be beneficial. I also feel that information should be provided on the length of posts, to determine if the students in each discipline are contributing quality comments and discussion or are they simply replying with “yes, I agree”.
In conclusion, I thought that the study lacked focus, and implications. I felt as though the authors took the easy way out in stating that “more research was necessary”, which is generally true of most studies. I felt as though the authors could have completed additional data collection in relation to the particular students in each course as well as expanded the study across undergraduate courses as well. There was no mention in the article about specifically studying graduate students. I was disappointed in the article; I did not feel as though I gained any information from reading it, other than a conclusion that I could have inferred on my own.
References
Jain, P., Jain, S., Jain, S., (2011). Interactions among online learners: a quantitative
interdisciplinary study. Education, 131, 3, p. 538-544.
References
Walker, D. D., Stephens, R., Roffman, R., DeMarce, J., Lozano, B., Towe, S., & Berg, B.
(2011). Randomized controlled trial of motivational enhancement therapy with
nontreatment-seeking adolescent cannabis users: A further test of the teen marijuana check-
up. Psychology of Addictive Behaviors, 25(3), 474-484. doi:10.1037/a0024076