Editor’s Note: Technology tools designed for distance learning are frequently adopted to enhance on-campus classes. This study compares the acceptance of web enhancements to learning for on-campus and off-campus learners, and acceptance based on amount of use. Blended learning in this instance means a combination of face-to-face instruction ((f2f) with online learning using a learning management system. The perceived value of web resources was significant, especially for off-campus students, and there was a positive correlation with degree of use. Speed of the internet connection was not significant in this study.

 

Blended Learning and Learner Satisfaction:
Keys to User Acceptance?

Larry R. Irons, Robert Keel, Cheryl L. Bielema

 

Abstract

User acceptance of blended learning instructional strategies is an important benchmark in ongoing efforts to develop distance learning programs. Student satisfaction is a key component in developing user acceptance. This paper reports on a study of blended learning in a state university in the Midwest. The particular blend of learning modalities reported here consists of 1) face-to-face instruction and 2) a learning management system (MyGateway) that provides asynchronous coordination and 3) a variety of group communication tools for students.

The research uses a survey instrument administered to a stratified random sample of 666 students. The analysis of the data uses the independent samples t-test to assess the relationship of two independent variables (access location and use) to three dependent variables (learning activity, satisfaction, student/teacher communication). Access location is defined in terms of on-campus access or off-campus access. Use is defined in terms of the degree (high use/low use) to which the co-present instructor actually incorporates MyGateway into the course.

The analysis finds that access location does not predict student perceptions of the three dependent variables. Degree of use, however, does predict student perceptions of the three dependent variables. The more students use the learning management system the greater their likelihood of agreeing with positive statements about items in the survey relating to each of the dependent variables.

 

Introduction

Discussions of blended learning are starting to examine the benefits derived from learning situations characterized by interactive media and mixed modalities of instruction. Particular attention was recently brought to bear on the results of online learning when mixed with co-present instructors and/or a learning community. The research and discussion focuses on differentiating blended learning by comparing outcomes between it and traditional, classroom learning, or e-learning alone (Barbian, 2002). Regardless of the comparisons made by researchers and developers, those studying blended learning agree that learner acceptance is a baseline requirement for a successful implementation.

This paper assumes that satisfaction, as opposed to frustration, with blended learning provides a basis for learner acceptance and repeated use (Hara & Kling, 1999). We provide an analysis of learner satisfaction[1] with a learning portal implemented at a mid-western university that is predominantly a commuter campus. The learning portal is referred to herein as “MyGateway.” It is based on the Blackboard Learning System. [2] The faculty use MyGateway largely as part of a hybrid, or blended, course delivery strategy to enhance teaching. Typical uses involve posting announcements, assignments, discussion forums, online tutorials, and chat sessions. In addition to these learning resources learners can manage their calendar, check grades, etc. So, the study reported here deals with only one modality of blended learning, i.e., combining a learning management system with traditional classroom instruction. It does not examine learner satisfaction with blended learning in any of the mixed modalities that classes can use when blending e-learning courses unto themselves. As Table 1 shows, a variety of mixed modalities of learning are possible in “blended” learning. This analysis focuses on only one (grayed cells) of several potential blends.

Table 1

* A plus sign (+) indicates a possible combination whereas a minus sign (–) indicates no possible combination. We do not include asynchronous (e.g. cassette tapes, self-paced manuals) and synchronic (e.g. teleconferencing) distance learning resources that are not computer-mediated in this typology.

We do not claim that the findings reported here are relevant to choosing among learning management systems. Rather, our purpose is more limited in scope, and analytic in nature. This paper reports on a study of student class participation, satisfaction, and communication with the teacher in one specific type of blended learning in a university class. Our analysis considers both access (the primary location where students access MyGateway) and use (low use or high use) of the learning portal (MyGateway) as key independent variables in predicting learner satisfaction, participation, and communication within distance learning classes.

We focus on access because of the longstanding assumption that one of distance education’s most promising features is the way it makes learning resources flexible and available to learners regardless of location (Valore & Diehl, 1987). Access has always been an issue for distance learning, from early reliance on the postal system, distributing books, self-paced manuals and other learning materials like audio-cassettes or albums, to later reliance on the general availability of analog technologies like television and telephony. In each case, access to learning resources was important. The same concern for access to learning resources appears relevant to interactive video classes, especially those integrating learning management system portals such as MyGateway into the curriculum. For instance, roughly equal access (between the host and remote sites) to studio classrooms equipped with interactive video feeds does not ensure equal access to Internet connectivity, especially in rural areas (Irons, Jung, & Keel, 2002). On the other hand, computer-mediated communication via the Internet can also provide a base for engaged students, even at remote sites, to increase their sense of involvement by using discussion groups, chat rooms, and other asynchronous technologies (Bielema, 1996). We use access location to the learning portal, MyGateway, as an independent variable in our analysis of student satisfaction.

We also focus on use because, flexibility of access aside, the ability of learners to apply technology in support of their learning activity depends largely on their ongoing response to using it (Storey, Phillips, Maczewski & Wang, 2002). If using a particular technology is not relatively straightforward and demonstrably effective in ways that are relevant to the student’s learning activity, users (in this case students) will not continue using the technology when presented with other options. Thus, we predict that such students will agree with negative statements about MyGateway and not agree with positive statements about MyGateway.[3] Our concern with use is different from a concern with usability per se. We are focusing on the results of sustained exposure (high use) to a learning management system in a blended learning class, as opposed to limited or incidental exposure (low use). Further, our concern is for how students perceive the learning management system to affect their participation in the class (learning activity), satisfaction with the class, and communication with the teacher as compared to a traditional face-to-face class in which a learning management system is not used.

 

Methodology

A stratified random sample (10 percent of the courses using MyGateway) was selected according to levels of faculty use of the learning management tool, MyGateway (N = 45). The courses were designated low use if faculty had logged 9 days or less in Control Panel accesses; high use if faculty had logged 10 or more days of Control Panel accesses by the middle of the semester (sample median was 8, mean 11). Questionnaires were completed in the class during the evaluation period at the end of the semester.

Completed surveys by undergraduate and graduate students numbered 666 with a 70% response rate. Students responding in high use courses numbered 318, while 348 students in low use courses responded. Given our interest in access, the survey also asked students about the computing and network resources available from their primary access location.

 

Questionnaire

The questionnaire for the current study consisted of 46 items, with 16 of those items designed for the data analysis reported here using a seven point Likert scale (see Table 2). The Likert items asked students to indicate their agreement on a seven-point scale ranging from (1) strongly agree to (7) strongly disagree.

 

Table 2. Questionnaire lead-ins and question text

Computers and Access for Students

Students were asked about their primary computer access location and capabilities available to them. Over 60 percent (61.6%) of the respondents reported owning a new computer (e.g., Pentium). Slightly fewer students in low use courses than those in the high use courses indicated having the newer computers. Only 9.8 percent of students reported that they do not own a computer.

 

Table 3. Primary Access Location

If a student’s primary access was from off-campus, they were asked specifics about the type of Internet connection: dial-up telephone via modem, dial up with DSL, cable modem, or other. See Table 2 for connection details of both groups.

Again, the two groups of students were similar, with cable modems used a little more frequently by the high use students than those in the low use group.

 

Table 4. Type of Internet Connection to University Servers

Blended Learning: Location and Use

We analyze the relationship of access location and use to learner satisfaction with MyGateway by:

1.      distinguishing students who primarily use it on campus from those who primarily use it off campus

2.      distinguishing low use from high use classes.

An independent samples t-test was used to measure each of these independent variables’ (access location and use) predictive power in relation to specific items from a survey instrument. The items used in the full survey were intended to assess the learning portal from a larger array of concerns. The items studied herein were developed to measure the degree to which two independent variables (access and use) are significant predictors for increased learner satisfaction (questions 33, 35, 36, 37, 38, see Table 2), increased learning activity (questions 9, 11 – 16, see Table 2), and increased student/teacher communication (questions 8, 10, 17, 32, see Table 2) in the courses studied. We examine each of these below and assume that as the degree of satisfaction with a technology increases the more learning activity occurs, along with increased student/teacher communication.

 

Access Location and MyGateway Student Evaluations

Our first concern in analyzing access location is to estimate its ability to predict the responses offered by students using MyGateway. To the extent that students have to travel to campus to use computers with access to the network, we expect those students to experience disadvantage in learning activities, lower satisfaction, and poorer student/teacher communication. So, our first three hypotheses test these relationships:

H1: Access to learning resources over the Internet from off-campus locations increases student participation in learning activities in blended learning classes.

H2: Access to learning resources over the Internet from off-campus locations increases student satisfaction in blended learning classes.

H3: Access to learning resources over the Internet from off-campus locations increases student/teacher communication in blended learning classes.

Specifically, we measured the relationship of access location (either on campus or off-campus) to students’ responses to items involving learning activity, student satisfaction, and student/teacher communication.

 

Table 5. Access Location Items (independent samples t-test)

Question 35 was the only item from the survey that indicated a significant relationship (p < .05) between the access location of students and their response. Question 35, a student satisfaction item, stated: “It takes more effort to complete a course that uses online resources like MyGateway.” Judgments about the success or failure of those efforts are not predicted by access location. The type of off-campus connectivity available to the student does not affect the relationship, i.e. whether the connection was dial-up or broadband (DSL or cable modem). [4] Students accessing MyGateway from home, regardless of connectivity, were statistically more likely to disagree with Question 35. Alternatively, those students accessing MyGateway at campus locations were more likely to agree with Question 35. Thus, we find only limited support for the hypothesis that students who commute to campus to use MyGateway resources experience lower satisfaction. Qualitative comments from students on the personal benefits of using MyGateway support the notion that flexible access to materials, support staff, and teachers was a source of satisfaction. One comment in particular was instructive on this point:

Especially because [the university] is a commuter campus, it is useful to be able to access course information, contact instructors and stimulate “community” interaction (via discussion boards).

 

Degree of Use and MyGateway Student Evaluations

In addition to access location, the survey items allow us to analyze differences between students in classes with low use of MyGateway and students in classes with high use. Our working hypothesis is that increased exposure to classes using MyGateway results in higher degrees of learning activity, satisfaction, and student/teacher communication. To the degree that the hypothesis is not supported we would expect lower acceptance of blended learning courses.

H3: Increased use of the learning portal increases student participation in learning activities in blended learning classes.

H4: Increased use of the learning portal increases student satisfaction in blended learning classes.

H5: Increased use of the learning portal increases student/teacher communication in blended learning classes.

The hypotheses, simply stated, claim that the more people use the learning portal, MyGateway, the higher their participation in learning activities, satisfaction with their classes, and student/teacher communication when compared to traditional classes—rather than the other way around.

 

Use of MyGateway in Relation to Learning Activity

Every learning activity item measured by the MyGateway survey indicates that students in low use classes are more likely (p < .05) to disagree with positive statements about learning activities in MyGateway classes. Thus, students in classes with high use of MyGateway were more likely to review lecture notes for clarification (Question 9), discuss ideas with other students (Question 11), actively participate in the course (Question 12), work on assignments with other students (Question 13), complete assignments on time (Question 14), access other online materials related to course content (Question 15), and spend more time studying for the course (Question 16).

 

Table 6. Learning Activity Items (independent samples t-test)

It is notable that even though students agreed that they were spending more time studying for the course, they nonetheless recognized the benefits of greater use of MyGateway in their courses. Students responding to open-ended questions on the benefits of using MyGateway reinforced the answers to the survey items. The two most often mentioned benefits were the availability of grades and course materials. The ability to easily communicate with instructors and other students was also emphasized, along with convenience of web access. Specific pluses mentioned included help in staying connected to the class, getting instructor feedback quicker, increased organization and opportunities for clarification.

 

Use of MyGateway in Relation to Student Satisfaction

Two items related to student satisfaction with courses using MyGateway indicate that students in low use classes are more likely (p < .05) to disagree with positive statements about their satisfaction with MyGateway classes. Thus, students in classes with high use of MyGateway were more likely to agree that they are very satisfied with the course (Question 33) and that they would like to have MyGateway used in other courses (Question 36). Interestingly, responses to Question 35, which are significant when measuring access location are not significant when measuring use. Even though students who travel to campus to use the Internet think more effort is involved, their responses are not affected by degree of use. In comparing high and low use groups, MyGateway has no discernable effect on re-enrollment (Question 37) or student retention (Question 38).

 

Table 7. Satisfaction Items (independent samples t-test)
 

Use of MyGateway in Relation to Student/Teacher Communication

Three items related to student/teacher communication in courses using MyGateway indicate that students in low use classes are more likely (p < .05) to disagree with statements about their degree of communication with faculty in MyGateway classes.

 

Table 8. Student/Teacher Communication Items
(independent samples t-test)

Thus, students in classes with high use of MyGateway were more likely to agree that they were able to seek clarification (Question 8), receive instructor comments on assignments quickly (Question 10), and communicate with their instructor (Question 17). Additionally, students did not express (Question 32) a preference for using their personal email to communicate with the instructor, implying that the MyGateway learning portal’s communication resources are sufficient.

 

Conclusions

The data presented above indicate that increased use of the Blackboard Learning System implemented as a learning portal, MyGateway, results in higher estimates of learning activity, higher degrees of satisfaction, and higher student/teacher communication than in courses not using the portal. Thus, the data from the current survey lend support to the contention that the more students are exposed to learning portals like MyGateway the higher their estimation of its positive affects. We suggest that the findings here are not only applicable to the MyGateway implementation of the Blackboard Learning System. Rather, we contend that similar findings are likely in any learning portal implementation when students get accustomed to using it (i.e., use it in direct support of the course work), especially when the course consists of a blend between traditional co-present classrooms and asynchronous, computer-mediated learning portals like MyGateway.

 

References

Barbian, J. (2002) “Here’s Proof” online learning magazine 6,6, pp. 27-31.

Bielema, C. L. (1996) Factors affecting implementation of interactive, computer-mediated instructional techniques for instructors and learners at a distance (Doctoral dissertation, University of Illinois, Urbana-Champaign) Dissertation Abstracts International, 57(11), 4703A.

Hara, Noriko & Kling, Rob (1999) “Students' Frustrations with a Web-Based Distance Education Course” First Monday 4, 12. Retrieved from http://firstmonday.org/issues/issue4_12/hara/index.html

Irons, L., Jung, D. & Keel, R. (2002) “Interactivity in Distance Learning: The Digital Divide and Student Satisfaction” Educational Technology & Society 5, 3, pp. 175-188. Retrieved from http://ifets.ieee.org/periodical/vol_3_2002/jung.pdf

Storey, M.A., Phillips, B., Maczewski, M. & Wang, M. (2002) “Evaluating the usability of Web-based learning tools,” Educational Technology & Society 5, 3, pp. 91-100. Retrieved from  http://ifets.ieee.org/periodical/vol_3_2002/storey.pdf

Valore, L. & Diehl, G.E. (1987) The effectiveness and acceptance of home study. Washington, DC: National Home Study Council.

About the Authors

Larry Irons combines industry and academic livelihoods in his professional career. He is Managing Partner of I.C. Technologies, a consulting services company focused on using performance support, eLearning and communities of practice to achieve client goals. Larry has researched distance learning since the early 1990s, participating in several industry/academic/government-sponsored projects. His consulting and research interests both focus on how communication technologies are designed and used. Larry received his Ph.D. in Sociology from Washington University in St. Louis in 1992.

Larry can be contacted at: larryi@ic-t.com

 

Robert Keel has taught sociology for the past 26 years. He currently is a lecturer in the Department of Sociology at the University of Missouri-St. Louis. For the past 8 years he has worked to develop Internet-based class resources for his classes at UM-St. Louis. He also works as a faculty liaison and specialist in information technology for the UM-St. Louis Information Technology Services department.

Robert can be contacted at rok@umsl.edu.

 

Cheryl Bielema, Instructional Designer, manages the Faculty Resource Center, University of Missouri-St. Louis, consulting with and training faculty in the integration of technology. She collaborates with the Center for Teaching and Learning, jointly sponsoring teaching and technology workshops for faculty and staff. Bielema worked previously at the University of Illinois, Urbana-Champaign, in distance learning as well as in UI Extension. Her teaching experience is in Curriculum and Instruction and Educational Technology. Her Ph.D. is in Human Resource Development, University of Illinois.

Cheryl can be contacted at bielema@umsl.edu.

Endnotes