E-mail comments to the Editor

Download the complete PDF of this issue

Editor's Note: We are pleased to present research that focuses upon immediate evaluation of student engagement in online learning. Drs. Dereshiwsky and Moan raise some interesting concerns about validity of identification of a student's learning style with student success in Distance Learning.

Identifying Factors that Predict Student Engagement in Web-based Coursework

Eugene R. Moan and Mary I. Dereshiwsky

Introduction

"To thine own self be true." Little could William Shakespeare have anticipated the soundness of his advice for learners in today's Information Age. The traditional face-to-face group teaching/learning setting has now been supplanted with a plethora of technologically enhanced alternatives. These include interactive television (IITV), Web-enhanced courses which consist of a combination of live or IITV-based group instruction supplemented with Internet interaction, and Web-based courses which involve instructional interaction conducted entirely over the Internet.

Such expanded instructional choices have led to concerns regarding the best fit of classroom type for a given student. To this end, a variety of learning style self-assessment instruments have been developed. Some examples of surveys are provided and discussed in McVay, 2000; Western Cooperative for Educational Telecommunications, 1999; and on the Western Governors University Website 2001 (http://www.wgu.edu/self_assessment.asp). The stated objective of such surveys is to help students understand their predominant individual learning style and how this learning style may or may not correlate to success in online learning. Despite the growing popularity of such self-assessment scales, however, not all distance education experts are convinced of their validity in predicting student success in online learning. Draves (2000), for example, believes that the only relevant factor in distance-based student success is the individual student's desire to succeed in his or her Web course.

This paper is a continuation of the initial studies conducted on student engagement in Web teaching and learning by the authors (Dereshiwsky, 2001; Dereshiwsky and Moan, 2000). Specifically, its goal is to identify which facets of online learning, as measured by a popularly used distance education instrument, predict levels and types of student engagement in Web coursework.

Methods and Procedures

The population for this study consisted of the students in the second five-week summer session of Dr. Dereshiwsky's Web-based Introduction to Research course. This course is a master's level requirement intended to provide education majors with an overview of the research process and its applications.

A total of 18 students completed all course requirements. They represented majors in educational leadership, counseling, elementary education, bilingual and multicultural education, and special education. Three of the 18 students were classified as 'non-degree' status.

All students were asked to complete the Western Governors University (WGU) "Is Distance Education for Me?" self-test during the first week of the course. The instructor asked students to e-mail their answers to her during this time. She explained to the students that the purpose was not to screen them into or out of the course, but rather to promote student awareness of the unique aspects of online learning. It was hoped that each student, including first-time Web course learners, thought deeply about his/her situation and whether an online course would be a good fit.

The WGU distance education self-test consists of a series of 10 questions designed to tap various facets of online teaching and learning. The responses consist of a three-point ordinal fixed-choice scale. After completing the self-test online, students may then access a Web page that analyzes their responses in terms of the fit of online learning to the needs and learning style of the student.

Table 1 below summarizes the 10 facets of teaching and learning measured by the WGU self-test. The complete instrument may be found at http://www.wgu.edu/wgu/self_assessment.asp.

Table 1.

Themes of Teaching and Learning Measured by WGU Distance Education Self-Test

The researchers sought to discover if there was a predictive relationship between the 10 items on the WGU self-test and four measures of student engagement in the course. These measures are outlined in Table 2 and explained more fully below.

Table 2.

Measures of Student Engagement in Web Course

Instruction of the course took place in an asynchronous posting area entitled the Virtual Conference Center (VCC). This area of cyberspace is akin to a classroom meeting place.

The VCC is organized by the instructor into a series of folders. Some folders, such as "Announcements and Updates" as well as the "More Words To Lead By" newsletter, are intended solely for instructor posting. Students may access the posts made by the instructor but may not make any posts of their own in these read-only folders. Other folders, such as "Questions and Answers," are places where students are invited to post any concerns or needs they have throughout the course. Both the instructor and other students then read and post responses to any questions in this folder.

Students were required to visit the VCC on a daily basis and check the VCC "Announcements and Updates" and "Questions and Answers" folders for new postings. "More Words To Lead By" was presented to students as a voluntary item to read in the VCC. Approximately every other day, the instructor posted a cluster of positive-thinking stories, poems and/or quotations centered on a key theme in this folder for student enjoyment.

In addition, the instructor posed two open-ended discussion questions in specially marked VCC folders during the five weeks of the course. Students were asked to comment on special needs of practitioners (teachers, counselors, other professionals) as researchers in Discussion Item #1. In Discussion Item #2, they were asked to brainstorm and share examples of potentially biased or misleading examples of reported research from the media with their classmates. The syllabus required students to visit each folder during a designated one-week time span, read the postings of their classmates and post at least three substantive messages during this time. This meant either their own opinions on the topic at hand, or a response containing supporting rationale to a classmate's post.

The instructor was able to monitor the number of visitors to a given VCC folder at any given point in time. These were compiled per student at the end of the semester, along with each student's individual responses to the 10 fixed-choice items on the WGU distance education self-test.

The ordinal fixed choices of responses were reverse-scored by the researchers upon their entry into the data base. This was done so that higher totals were reflective of relatively better fit of respondent to distance learning environments.

The primary research questions investigated by the researchers in the analysis are as follows:

1.     What is the predictive relationship, if any, between each item of the WGU distance education self-test and the following three measures of required student engagement in the Introduction to Research Web course:

a.     accessing posted "Announcements and Updates;"

b.     accessing posted "Questions and Answers;"

c.     accessing posted responses to both discussion items

2.     What is the predictive relationship, if any, between each item of the WGU distance education self-test and the discretionary measure of student engagement in the Introduction to Research Web course: accessing posted "More Words To Lead By" Newsletter items?

Due to the ordinal nature of the responses to the WGU distance education self-test, as well as the relatively small sample (18 subjects), the researchers analyzed the data by computing the non-parametric Spearman correlation coefficient (popularly known as rho) in order to answer the above research questions. The results appear in the following section.

Findings and Results

Table 3 contains the Spearman rho values and associated significance levels (p-values) for the pairwise correlations between the ten distance education self-test responses (rows) and the three required measures of student engagement (columns).

Table 3.

Spearman Correlation Coefficients: Individual Distance Education Self-Test Items
Related to Required Activities of Online Course Engagement

* : statistically significant at a= 0.05 when actual value is rounded to two decimal places.

As shown in Table 3, two of the 10 topic areas of the distance education self-test were significantly correlated to all three required student engagement indices. These two areas are perceived helpfulness of class discussion and amount of time available to devote to one's coursework. In addition, the ability to prioritize tasks was significantly correlated to students' engagement (number of posts read) with the VCC "Questions and Answers" and "Discussion Items" folder postings.

Table 4 contains Spearman rho values and associated levels of significance for the 10 areas of the distance education self-test (rows) as related to the one discretionary measure of student engagement, number of posts accessed in the VCC newsletter folder (column).

Table 4.

Spearman Correlation Coefficients: Individual Distance Education Self-Test Items Related to Discretionary Activity of Online Course Engagement

In contrast to the preceding results for the three required student engagement measures, none of the10 distance education self-test measures was significantly related to the voluntary engagement measure (number of posts accessed by students in the "More Words To Lead By" VCC folder).

Discussion

It is important to keep the exploratory nature of this study in mind in interpreting the results. This study is a preliminary attempt to identify indicators of student engagement in Web coursework. Nonetheless, the consistency of significant results was striking, even within the one disciplinary area (master's level Web-based Introduction to Research course) as well as the relatively small sample size.

The amount of time that a student has to devote to his or her coursework is consistently related to all three required indices of student engagement in this study setting. In addition, the ability to prioritize one's learning tasks is linked to two of these three required indices of student engagement. Finally, the less helpful a student found class discussion to be, the more likely he or she was to engage in required Web course outputs. These results suggest the value and importance of time-management skills for students interested in learning via the Internet. Perhaps screening instruments can be developed to help students assess their time-management skills. In addition, first-time Web course students might benefit from orientation and training in such time-management techniques designed to maximize their chances of success in learning on the Internet.

It is also interesting to note what did not seem to predict required student engagement in Web coursework. Surprisingly, the importance of face-to-face interaction was not a significant predictor of any of the three required engagement outcomes. This seems to run counter to prevailing stereotypes about Web courses and beliefs that "they won't work (or "students won't like them") because students need and want face-to-face interaction."

Another prevalent belief is that many students enroll in Web-based courses because they have no other alternative. This may indeed be true for some busy working professionals who cannot afford to make a time commitment to intensive periodic on-site classes. It may also be true in the case of students who are located in remote areas not readily accessible to a campus-based or satellite class meeting. But even if this is the case, such "Web-by-necessity" enrollment does not seem to be significantly related to subsequent student engagement in the form of completion of required Web course assignments. This conclusion seems to be further confirmed by the lack of association between the predictability of one's personal or professional schedule, and likelihood to engage in required Web course activities.

The above conclusions may also suggest a shift in direction of the marketing of such Web-based coursework. In much current campus advertising, such courses are heavily marketed to distance-based students, or to those with heavy outside job responsibilities. Perhaps such marketing needs to be redirected and aimed instead at students who are 'self-starters' and who possess above-average time-management skills…regardless of their location or other professional commitments. These two target market segments are sometimes popularly referred to as "non-traditional students." But given the results of this study, the 'ideal' student for Web instruction may indeed not necessarily be the 'non-traditional' student, but rather may well include a more conventional 18-to-22-year-old non-working full-time college student who lives on campus and who possesses superior levels of time-management skills.

Finally, it was somewhat surprising to discover that a student's predominant learning style did not predict any of the three required indices of student engagement in Web learning. This appears to fly in the face of the plethora of literature and popular belief that students can be classified into a predominant learning style which, in turn, is closely linked to a particular format of teaching and learning. Instead, the findings and conclusions of this study seem to support the conviction of Draves (2001) that the key criteria for student success may simply be motivation and desire to do well in a Web course.

It was also surprising to discover that noneof the characteristics of online learning contained in the distance education self-test appear to predict voluntary student engagement in Web coursework. This suggests that further research is needed to identify those student learning characteristics, if any, that predict "who will go the extra mile" in terms of his/her Web course learning opportunities.

As mentioned above, this study should be replicated in other course settings to see if the content area of instruction, semester of enrollment, or other situational factors appear to influence the relationship between student learning characteristics and type as well as level of engagement in Web coursework. Ideally, with larger sample sizes, researchers may unearth a profile analysis to identify which combinations of student characteristics link to which combination of course behaviors and engagement outcomes, and under which contextual circumstances (i.e., content area of course, time of enrollment, etc.). Taking such a multivariate approach would allow for potential overlap between and among these three sets of indices.

Concluding Comments

Engaging student interest and participation is a challenge faced by every educator, regardless of format of classroom setting. The Internet has provided both unique opportunities and creative challenges with respect to designing instruction in order to maximize student engagement.

An understanding of student attitudes, perceptions and readiness regarding different facets of distance-based instruction is integral to predicting such maximum levels of student engagement in instructional activities. Continued exploration of the factors that link to student engagement is a worthwhile activity for all key stakeholders interested in optimally effective distance education strategies.

References

Connick, G. P. (Ed.). (1999). The distance learner's guide. Western Cooperative for Educational Telecommunications (WICHE). Upper Saddle River, NJ: Prentice-Hall.

Dereshiwsky, M. I. (2001). E.N.G.A.G.E. 'Em: Strategies for effective involvement of online students. Virtual University Gazette. http://www.geteducated.com/vugaz.htm

Dereshiwsky, M.I. & Moan, E.R. (2000). Good connections: Strategies to maximize student engagement. Education at a Distance. < http://www.usdla.org/ED_magazine/illuminactive/NOV00_Issue/story04.htm>

Draves, W.A. (2000). LearningontheNet. River Falls, WI: Learning Resources Network.

McVay, M. (2000). How to be a successful distance student: Learning on the internet. (2^nd ed.). Upper Saddle River, NJ: Prentice-Hall.

Western Governors University (2001). Self-assessment quiz. Retrieved from http://www.wgu.edu/self_assessment.asp)

About the Authors:

Dr. Eugene R. Moan is Professor of Educational Psychology at the Center for Excellence in Education, Northern Arizona University, Flagstaff, AZ. His telephone number is (928) 523-9604 and his e-mail address is Eugene.Moan@nau.edu.

Dr. Mary I. Dereshiwsky is Associate Professor of Educational Leadership at the Center for Excellence in Education, Northern Arizona University, Flagstaff, AZ. Her telephone number is (928) 523-1892 and her e-mail address is LDRSPETSCherry@aol.com.