1. Introduction
As part of various educational attempts to popularize higher education, Massive Open Online Courses (MOOCs) have attracted a lot of attention as a way to further expand higher educational opportunities and improve the quality of instructional methods. MOOCs are large-scale online learning environments, which allow learners to take high-quality courses offered by universities around the world without separate admission procedures or tuition fees. MOOCs have many participants per class, being open to all who want to take them, providing learning experiences without restrictions on time and space or specific course start and end points. Despite increasing interests and expectations, there are debates and doubts regarding learning outcomes and the effectiveness of MOOCs. In general, participation tends to decline rapidly during the first week and continues to decline during the later weeks of a course [1]. Consequently, fewer than 10% of enrollees complete MOOCs [2]. High dropout and low completion rates have been a real challenge for the design and development of MOOCs. Moreover, due to the large study body, their diverse support needs, and a unique learning culture, facilitation of engagement in MOOCs has been very challenging.
Looking at MOOCs through the lens of engagement may help identify learner groups with different engagement tendencies and allow for a better understanding of the learning processes in MOOCs. It may help us construct personalized instructional strategies and interventions to help students learn more effectively and complete the MOOC. Yet, many previous studies on MOOCs dealt with heterogeneous learners as a single group or mainly investigated their behavioral participation [2]. Therefore, this study aimed to cluster MOOC learners via multidimensional aspects of engagement and to provide strategies to customize interventions for each clustered group. We clustered MOOC learners in terms of their behavioral, emotional, and cognitive participation. We then examined the differences in demographical characteristics, such as sex, age, and reasons for taking courses in MOOCs, across all clusters. We also examined differences in perceptions of teaching presence and learning persistence. Based on the results, we provided empirical and practical implications for instructional and learning strategies that can be used to facilitate engagement and persistence in MOOCs. The followings are the research questions examined in the study:
1. Do MOOC learners cluster into different groups based on their engagement?
2. Do the learners in different engagement cluster groups share similar demographic characteristics?
3. Is there a significant difference in perceived teaching presence across the engagement clusters?
4. Is there a significant difference in learning persistence across the engagement clusters?
2. Theoretical background
2.1 Engagement in MOOCs
Engagement is highly associated with retention, learning, achievement, and graduation [3]. Engagement in a MOOC is defined by “the willingness and extent to which people are active in a MOOC as displayed through their interaction with the content and people in the course” [4]. Although engagement is critical for all learning [20], it is particularly so in MOOCs, where learners voluntarily self-enroll and learn at their own pace; it is a prerequisite for learning and course completion. For this research, we perceive engagement as a multidimensional composite embracing cognitive, emotional, and behavioral aspects. Behavioral engagement was defined as participating in activities such as paying attention to learning itself, asking questions, and participating in discussions during the learning processes [3]. Emotional engagement was defined as the energy that learners put into having positive attitudes toward their instructors, peer learners, or the MOOC itself. Cognitive engagement was defined as the efforts made by learners to invest in acquiring complex content or skills in the MOOC learning process.
Recent studies have delineated strategies to amplify engagement in MOOCs. [4] reported that engagement in MOOCs could be leveraged by focusing on three pedagogical concepts: teacher presence, social learning, and peer learning. [5] investigated three top-rated MOOCs and identified five strategies of engagement: (1) a problem- centric approach with explanations that are clear and easy to understand, (2) an accessible and passionate instructor, (3) peer interaction, (4) active learning and learning-by-doing, and (5) a wide variety of useful course resources and activities. However, all of these strategies aim to provide general support and fail to take individual differences into consideration. Personal variables that influence engagement in MOOCs have not been fully examined.
2.2. Engagement and Perceived Teaching Presence in MOOCs
Teaching presence contributes to deep and meaningful learning by means of social interaction in the cognitive/inquiry-based process [6]. Teaching presence also helps learners reach a high level of critical thinking and knowledge construction, and it can be manifested by three types of approaches: design, facilitation, and direction [7]. In particular, teaching presence manifested through structured and cohesive facilitation is highly associated with the success of online learning. A sustained and learner-centered teaching presence may encourage student participation in online environments.
MOOCs normally provide access at scale. Given the nature of MOOC courses, interactions with instructors are very limited, and automated feedback is often conceived as the instructor talking. The high dropout rates in MOOCs highlight the necessity of personalized design, facilitation, and direction; in other words, tailored support of teaching presence is necessary for successful learning in a MOOC. Moreover, the presence of instructor is one of the primary predictors on MOOC completion along with learner perception of peer-instructor interactions [8].
In MOOCs, learners perceive teaching presence in terms of direct instruction offering clarification, examples, and resources; facilitation using a summary; and assessment of formative discussion [9]. Whereas the interaction with instructors is more effective than interaction with peers in terms of student satisfaction and perceived learning in online education [10], no statistically significant differences in course completion and participation in MOOCs has been found between a control group and an intervention group, in which teachers and teaching assistants post on online discussions and provide weekly feedback [11]. Because of the novel characteristics of MOOCs, instructors must deal with a diverse learner group and instructional challenges, meaning this is a much-needed topic of exploration. Moreover, most previous studies that investigated the relationship between teaching presence and online participation targeted a whole learner group instead of taking learner differences into consideration.
2.3. Engagement and Learning Persistence in MOOCs
Learning persistence refers to the willingness to complete the course that a learner is currently enrolled in [12]. With online learning, the term persistence is frequently used interchangeably with retention, completion/non- completion, dropouts, and withdrawal when discussing learners’ intent to continue their online learning studies [13]. In MOOC settings, personal traits reflecting behavioral engagement are associated with learner persistence: students who completed a pre-course survey or took a quantitative track (instead of a qualitative or auditing track) had a stronger tendency to complete the course [1]. Most students who received a certificate in a MOOC actively uploaded postings in their forums, suggesting that such learner behavioral engagement is a better indicator of course completion and learner persistence [14]. Lastly, a function of network benefit, user preference, and motivation to achieve are highly associated with persistence in MOOCs [15]. Although the findings on MOOCs are consistent, they are limited to specific aspects of participation such as behavioral, cognitive, or emotional participation and do not address comprehensive understanding.
To deal with the high dropout rates in MOOCs, tailored support on learning persistence grounded in multidimensional engagement patterns is essential for individual learners to successfully complete MOOCs. Although instructional interventions to decrease dropouts have been historically less studied [13], in recent years more efforts have been made to examine instructional strategies to promote student persistence. However, these studies mainly targeted all learner groups in online learning environments, disregarding individual differences and MOOC learning settings. Overall, we aimed to expand and deepen the current understanding of engagement through the perspective of multidimensional engagement in MOOCs. Our study clustered learners by using their engagement patterns in a MOOC course and explored cluster-specific personal characteristics, perceived teaching presence, and learning persistence. Ultimately, we hope to guide the design and delivery of tailored instruction for learners in MOOCs. Investigation of the research questions addressed above can inform the prescription of course activities per engagement style and guide the course design of MOOCs.
3. Methods
3.1 Research Context and Participants
Through convenience sampling, enrollees in a MOOC called Digital Storytelling were recruited for the study. The course was developed by Ewha Womens University in Korea. The main learning materials in the course consisted of several short lecture videos and weekly learning activities including quizzes or discussions. Demographics and characteristics of the participants are shown in Table 1. College students and graduates were asked to name their majors; high-school students or participants without a college degree were asked to select “none.”
Table 1. Demographics of Participants
3.2. Measurements
An online questionnaire of 35 questions was used to measure variables of interest. As shown in Table 2, four demographic questions were followed by two MOOC-related questions about reasons for participation and prior experience in MOOCs. The other 29 items dealt with three constructs: engagement, teaching presence, and learning persistence. Items validated in the existing literature were translated and used in this study. Four professionals in the field of educational technology ensured face validity of all items through expert review; they each had more than five years of experience in the field as a researcher or professor. Reliability tests using Cronbach’s alpha were conducted for each scale. Each item was on a five-point Likert scale that ranged from ‘strongly disagree’ to ‘strongly agree.’
Table 2. Measurements for Learning Engagement and Learner Characteristics
Thirteen items consisting three scales - behavioral, emotional, and cognitive engagement – from [16] were revised for the MOOC setting and used to measure engagement in this study. Three items measured behavioral engagement; for example, “I follow the rules of the MOOC class.” Cronbach’s alpha for behavioral engagement was .837. Five items were used to measure emotional engagement; for example, “I feel happy when taking the MOOC class.” Cronbach’s alpha was .916. Another five items were used to measure cognitive engagement; for example, “If I do not understand what I learned in the MOOC class, I go back to watch the recorded session and learn again.” Cronbach’s alpha for cognitive engagement was .841. Cronbach’s alpha for the construct of engagement was .892.
Teaching presence was measured by 12 items representing two factors: instructional design and organization, and direct facilitation. The items used by [17] were edited and used in the study. For instructional design and organization, there were three items; for example, “The instructor clearly communicated important course goals.” Cronbach’s alpha was .919. For direct facilitation, there were nine items; for example, “The instructor helped me stay on task in a way that helped me to learn.” Cronbach’s alpha for direct facilitation was .938. Cronbach’s alpha for the construct of teaching presence was .955.
Learning persistence was measured using four items. Items from [18] research were revised and used in this study; for example “I will finish my studies in this MOOC class no matter how difficult it may be.” Cronbach’s alpha for learning persistence was .759.
3.3. Data Collection and Analysis
Teaching presence and engagement should be measured after learner perceptions of instructor facilitation and learner involvement are well established. Thus, we eliminated data on students who had been involved in the course for less than the first four weeks. To accurately reflect general learner characteristics of MOOCs, we included data on the rest of the students (i.e., those who had been enrolled in the course for more than four weeks), disregarding their actual course completion. Data collection was conducted during a 16-week course. Emails with a link to an online survey were sent to the enrollees twice, at the fifth and ninth weeks. Among the 179 course enrollees, after eliminating responses with missing data, data from 170 research participants were finally selected for the analysis.
We analyzed the data as follows. First, internal consistency reliability of the data was confirmed. Second, descriptive statistical analysis was performed. Third, we implemented cluster analysis with the K-means algorithm. We attempted to categorize students with similar engagement styles, and the prospect of using the k-means clustering for this purpose was significantly promising. Cluster analysis classified the subjects who had similar characteristics into a group based on the extent to which they were similar to each other [46]. By means of K-means cluster analysis, the data were randomly divided into k clusters, keeping the clustering in a direction that did not cause errors [19]. Fourth, cross-analysis with a chi-square test was done on the nominal data to speculate on the attributes of the clusters. We specifically examined the learners’ characteristics by occupation, age, and reasons for taking MOOCs. Finally, we conducted a set of analyses of variance and following post-hoc tests to identify differences in teaching presence and learning persistence.
4. Results
4.1 Cluster Analysis
The descriptive statistical analysis was conducted to test normality of data as shown in Table 3. K-means cluster analysis was carried out by setting the reference variable as the respondent in choosing the cluster method as the "farthest item" and applying the "squared Euclidian distance" of the isometric index (See Table 4). The number of clusters was decided based on the "clustering schedule" and the "proximity matrix table."; The best number of clusters turned out to be three, considering the meaning of each profile, the ratio of cases, and the purpose of tailored instructional design. Based on these criteria, the engagement styles of each cluster were pinpointed, and the clusters were named Cluster1, Cluster2, and Cluster3, as shown in Table 4.
Table 3. Results of Descriptive Statistical Analysis
Table 4. Summary table of K-means cluster analysis
4.2 Cross- Analysis of Cluster Characteristics
Cross-analysis using chi-square tests examined differences in age, occupation, prior experience, and reasons for taking MOOCs across clusters, as shown in Table 5. Whereas the differences in sex, age, occupation, major, and prior experience were not statistically significant, the differences in reasons for taking the MOOC were significant. When asked about why they enrolled in the MOOC, participants were allowed to choose multiple responses. The Cluster1 reported the most numerous reasons to participate in the class (39.41%, n = 67), followed by the Cluster2 (34.12%, n = 58), and the Cluster3, who chose the fewest reasons (26.47%, n = 45). The Cluster1 reported the highest score on ‘to pursue my personal interests and academic curiosity’ (47.67%) as a reason to take the course. Cluster1 also ranked high on other reasons to take the course: ‘to earn academic credits’ (32.00%), ‘to supplement and further my study’ (30.0%), and ‘to prepare for the exams or interviews for achieving jobs’ (31.03%). The Cluster2 scored highest on ‘to earn academic credits’ as their reason for the course-taking (44.0%). The next highest reason for their enrollment was to pursue personal interests and academic curiosity (37.21%). Last, Cluster3 were likely to take the course for preparation for job application (44.83%) and to do in-depth study (43.33%). Compared to students in other clusters, they were less likely to report their reasons for MOOC-taking as pursuing personal interests and academic curiosity (15.12%).
Table 5. Results of Cross Analysis on Reasons for taking the course
We conducted a set of one-way ANOVAs to make a clear identification for the characteristics of each cluster regarding their perceptions of teaching presence and learning persistence. Levene’s test confirmed the homogeneity of variance (p > .05). The results of ANOVA indicated clusters present significantly different profiles on perceived teaching presence on design & organization (F = 15.536, p < .05), perceived teaching presence on direct facilitation (F = 20.326, p < .05) and learning persistence (F = 23.487, p < .05) (see Table 6). Post-hoc comparisons on perceived teaching presence indicated that, for design and organization, the Cluster1 (mean = 4.64) and the Cluster2 (mean = 4.4) reported a higher perception on teaching presence than did the Cluster3 (mean = 3.93, p < .05). For direct facilitation, the Cluster1 (mean = 4.36) showed a significantly higher perception than did the Cluster2 (mean = 3.84, p < .05) or the Cluster3 (mean = 3.53, p < .05). In addition, the Cluster1 (mean = 4.48) reported a stronger learning persistence than did the Cluster2 (mean = 4.24, p < .05) or the Cluster3 (mean = 3.78, p < .05). The Cluster3 ranked the lowest in learning persistence (mean = 3.78, p < .05) (see Table 7).
Table 6. Results of ANOVA on Perceived Teaching Presence and Learning Persistence
Table 7. Results of Post-Hoc Tests on Perceived Teaching Presence and Learning Persistence (n = 170)
5. Discussion and Conclusion
5.1 Discussion
The current study aimed to provide a direction for understanding learners’ engagement patterns related to motivation, teaching presence, and learning persistence in MOOCs. This study clustered 170 MOOC learners into three groups based on their patterns of engagement in order to draw design and implementation implications for MOOCs: Cluster1, Cluster2, and Cluster3. The results of this study were as follows.
First, distinct and authentic engagement patterns were found for each of the three clusters. The first type of learners, the Cluster1, showed high levels in all the areas of engagement. Given the instrument items describing behavioral, emotional, and cognitive engagement, the Cluster1 followed the course policies; completed assignments in a timely manner; self-checked their own learning progress; enjoyed learning in the MOOC course; were interested in the learning activities; read additional resources; self-questioned to assure their own understanding of the topics; continued studying when there was no quiz to take; tried to find additional information in TV, journals, articles, and books; and took further action when a concept covered in the course was hard to understand.
Meanwhile, the Cluster2 and the Cluster3 presented different levels of participation in different areas. The Cluster2 were likely to have relatively high behavioral engagement but comparably low emotional and cognitive engagement in terms of regulating course schedule, working on assignments, and monitoring their own learning. The Cluster3 were likely to present relatively high emotional engagement, showing that they enjoyed and were interested in taking the course, yet they showed low behavioral and cognitive commitment by not following the lecture schedule, not working on course assignments, and not making additional efforts to learn and understand the topics on their own. These results together suggest that learners pursued their learning through different learning approaches in MOOCs. Whereas [20] explored behavioral engagement patterns using trace data on a MOOC, our findings show that, besides behavioral aspects, different engagement tendencies are found for cognitive and emotional aspects.
Second, our findings indicate that the reason for taking a MOOC has a significant effect on engagement. This is meaningful in distinguishing characteristics of each cluster because learning motivations and goals shape the purpose of taking a MOOC and reflect on their learning process. As the primary reason for course enrollment, most of the Cluster1 chose personal interests, whereas many from the Cluster2 tended to choose academic credit as their reason. Meanwhile, many among the Cluster3 indicated that they wanted to prepare for future jobs or for in-depth study. The results indicate that engagement, particularly a high degree of involvement in cognitive, emotional, and behavioral aspects, is highly related to learners’ personal interests and academic curiosity. This finding seems to be consistent with [21], who found that MOOC completers chose intrinsic motivation as a prominent reason for course completion.
Since continuance intention to use MOOCs is significantly predicted by the quality of system, course, and service [22], our results suggest a need to provide tailored instruction to make connections with personal interests and provoke academic curiosity. Our findings meaningfully compare with those of [23]. They found that the motivation to earn a certificate, to meet new people, to take a course with others, or to deepen one’s knowledge on one’s job or one’s school is associated with higher self-regulated learning skills whereas the motivation for general interests, for career change, for fun and challenge, or for personal growth is associated with lower self-regulated learning skills.
Third, learners of the three clusters reported different perceptions of teaching presence. For instructional design and organization, which constitute an aspect of teaching presence, the Cluster1 and the Cluster2 presented relatively higher perceptions than the Cluster3. The survey items indicate that the Cluster1 and the Cluster2 had better opinions than the Cluster3 regarding whether the instructor clearly communicated key topics, learning objectives, and course activities. For direct facilitation, another aspect of teaching presence, the Cluster1 reported significantly higher perceptions of the instructor’s direct facilitation than the Cluster2 and the Cluster3. The survey items showed that the Cluster1 felt that the instructors made informed and personalized decisions on their learning, offered timely feedback, and provided guidance to keep them focused on learning, whereas the Cluster2 and the Cluster3 reported less effective and personalized feedback. Thus, it seems reasonable to infer that different engagement patterns may have exposed the learners to different sets of course activities in the MOOC. Divergent accumulated experience may have influenced their perceptions of teaching presence and consequently affected their self-reported learning outcomes. Given that the Cluster3, who ranked the lowest in engagement in all three dimensions, reported the lowest level of perceptions on both factors of teaching presence; and that the Cluster2, who ranked the second-highest in engagement, had the second-highest perceptions of teaching presence; and that the Cluster1, who ranked the highest in engagement, had the highest perceptions of teaching presence, it seems reasonable to suppose that the pattern of engagement has a meaningful connection with learner perceptions of teaching presence. Hence, the distinctive features of each cluster should be considered in planning customized treatment to facilitate their perceptions of teaching presence, since this is a significant factor in learning outcome.
Our findings also indicate that linearly structured courses, like the MOOC used in this study, seem to be effective for the Cluster1, given their perceptions of teaching. In general, learners in MOOCs process their learning via their own paths and are led by their own decisions based on their specific learning goals. This means that a linear course design may not be most effective for all learning objectives or all learner groups. For instance, the Cluster2 and the Cluster3 in this study might respond more favorably to a different type of course design.
MOOCs can be constructed using a linear and straightforward design or using a nonlinear structure. MOOCs can also offer personalized and flexible learning paths. The effectiveness of different instructional approaches in MOOCs needs further exploration to come to a definitive conclusion [24]. In the meantime, in courses designed hierarchically, learning guidance or cues could be added so that all learners can be well acquainted with key concepts or major course activities in order to progress their learning to the next level. Additional cues may be embedded in existing materials; for instance, partitioning lecture videos into self- contained segments to be useful for nonlinear navigation. In less prescriptive course design, diagnostic measures of prior knowledge at the time of course enrollment may help learners self-evaluate their competency and make more informed decisions in their learning process as well as help instructors make necessary and timely interventions [25].
Finally, learners of the three clusters reported different perceptions of their own learning persistence. Engagement appears to be positively correlated with perceptions of learning persistence. The learners with the highest engagement, the Cluster1, reported the highest persistence. The individuals in the Cluster2, who had the second-highest engagement, had the next highest persistence. The learners of the Cluster3, who presented the least engagement, reported the lowest persistence. The findings of this study suggest that engagement style can be used as a framework to personalize course design to foster learner persistence. Integration of our results offers both a theoretically and an empirically grounded framework for instructional design and delivery of MOOCs.
5.2 Implications
Understanding engagement, personal characteristics, influences on teaching presence, and influences on learning persistence is important in the design and development of learning environments where initial enrollment and continuous participation is mainly decided by the learners, as in MOOCs. Given the results of our study that elucidate our understanding of engagement patterns, there are several considerations that may help to personalize MOOCs.
Engagement is grounded on learner characteristics, such as their reasons for taking a MOOC. Accumulated learner data collected at the beginning of a course may inform the designers or the instructors about how to build learner profiles and make appropriate customizations. For example, course activities that stimulate academic curiosity and link to personal interests may help the Cluster2 and the Cluster3 leverage their levels of engagement as well as strengthen the motivation of the Cluster1.
Engagement patterns seem to lead learners along different learning pathways in MOOCs. Consequently, the learning experience on the different pathways may result in divergent perceptions of teaching presence and learning persistence. To offer a high-quality experience, several sets of learning paths may be recommended by matching the attributes of each cluster to each learning path. For Cluster2 and Cluster3, their learning path might highlight the relevance of course materials for academic success (for Cluster2) or for job settings (for Cluster3). Particularly for the Cluster3, a path highlighting real-world problems and addressing work applications may help keep them engaged. A path clarifying milestones in the course may help Cluster3 proceed with their learning without missing core course activities. Moreover, highlighting the interesting endpoints of a course early in the semester may help Cluster3 and Cluster2 achieve their learning goals.
Engagement might be stimulated through self-assessment of the current learning process. Continuous assessment of learning via quizzes or constant monitoring of learning progress may particularly help Cluster3 strengthen their motivation by seeing the gap between the job that needs to be done and the job as currently completed. Having an opportunity to reflect on their own learning progress may help Cluster2 foster higher cognitive and emotional engagement.
Last, engagement may be improved by social learning, where students observe the social presence and visible activities of peer learners. Thus, matching student groups for collaborative course activities should take engagement patterns into consideration. For example, by reading the discussions posted by Cluster1, Cluster2 may attempt to model similarly high cognitive and emotional commitment. In another example, matching learners with different expertise may help Cluster3 develop relational expertise and stay engaged. In addition, because learner’s preferred communication mode is associated with their level of English proficiency, gender, education, and age, these demographical characteristics may be also considered in student grouping.
5.3 Limitations and Future Research
This study had several limitations and resulted in multiple suggestions for future studies. First, self- reported data were used to assess behavioral, emotional, and cognitive participation in the study. Since most learning processes are stored as log data in MOOCs, future studies may use the log data as objective behavioral measures.
Second, data were collected using a convenience sampling approach from enrollees of a MOOC in Korea. Generalization to a global audience of MOOCs is therefore somewhat limited. The sample consisted of 76.5% female undergraduates, and over 40% had prior experience with MOOCs. In this study, there were no significant differences in engagement in terms of gender, age, major, and prior MOOC experience. However, further studies should confirm the generalizability of the findings for a learner group with demographic and geographical diversity.
Third, the sample size was 170. A follow-up study should be performed with a larger sample size to obtain stronger statistical power.
Fourth, participants completed the survey at either the fifth or the ninth week. However, differences between responses from the fifth week versus those from the ninth week were not examined. Whether student perceptions change during a four-week gap has not been investigated, as far as we know; our findings may not be identical to findings from a single-time-point data collection.
Last, futures research may develop a prediction model for each cluster. For example, factors like self- regulation, self-efficacy, and cognitive load could be included in the prediction model to examine their effects on perceptions of teaching presence and learning persistence and to prescribe more customized and cluster- specific feedback.
Conflicts of Interest
The authors declare no conflict of interest.
References
- B. J. Evans, R. B. Baker, and T. S. Dee, "Persistence patterns in massive open online courses (MOOCs)," The Journal of Higher Education, vol. 87, no. 2, pp. 206-242, 2016, doi: 10.1080/00221546.2016.11777400.
- K. Jordan, "Initial trends in enrolment and completion of massive open online courses," The International Review of Research in Open and Distance Learning, vol. 15, no. 1, pp. 133-169, 2014, doi: https://doi.org/10.19173/irrodl.v15i1.1651.
- J. A. Fredricks, P. C. Blumenfeld, and A. Paris, "School engagement: potential of the concept: state of the evidence," Review of Educational Research, vol. 74, pp. 59-119, 2004. https://doi.org/10.3102/00346543074001059
- S. Walji, A. Deacon, J. Small, and L. Czerniewicz, "Learning through engagement: MOOCs as an emergent form of provision," Distance Education, vol. 37, no. 2, pp. 208-223, 2016. https://doi.org/10.1080/01587919.2016.1184400
- K. F. Hew, "Promoting engagement in online courses: What strategies can we learn from three highly rated MOOCS," British Journal of Educational Technology, vol. 47, no. 2, pp. 320-341, 2016, doi: 10.1111/bjet.12235.
- A. G. Picciano, "Beyond student perceptions: Issues of interaction, presence, and performance in an online course," Journal of Asynchronous Learning Networks, vol. 6, no. 1, pp. 21-40, 2002.
- D. R. Garrison, M. Cleveland-Innes, and T. Fung, "Student role adjustment in online communities of inquiry: Model and instrument validation," Journal of Asynchronous Learning Networks, vol. 8, no. 2, pp. 61-74, 2004.
- E. B. Gregori, J. Zhang, C. Galvan-Fernandez, and F. de Asis Fernandez-Navarro, "Learner support in MOOCs: Identifying variables linked to completion," Computers & Education, vol. 122, pp. 153-168, 2018, doi: 10.1016/j.compedu.2018.03.014.
- S. L. Watson, W. R. Watson, J. Richardson, and J. Loizzo, "Instructor's use of social presence, teaching presence, and attitudinal dissonance: A case study of an attitudinal change MOOC," The International Review of Research in Open and Distributed Learning, vol. 17, no. 3, 2016.
- K. Swan, "Virtual interaction: Design factors affecting student satisfaction and perceived learning in asynchronous online courses," Distance Education, vol. 22, no. 2, pp. 306-331, 2001. https://doi.org/10.1080/0158791010220208
- J. H. Tomkin and D. Charlevoix, "Do professors matter?: Using an a/b test to evaluate the impact of instructor involvement on MOOC student outcomes," In Proceedings of the first ACM conference on Learning@ scale conference, pp. 71-78, Mar. 2014.
- Y. Joo, N. Kim, and G. Kim. "The Structural relationship among self-efficacy, internal locus of control, school support, learning flow, satisfaction and learning persistence in cyber education," Korean Journal of Educational Technology, vol. 26, no. 1, pp. 25-55, 2010.
- Y. Lee and J. Choi, "A review of online course dropout research: Implications for practice and future research," Educational Technology Research and Development, vol. 59, no. 5, pp. 593-618, 2011, doi: 10.1007/s11423-010-9177-y.
- L. Breslow, D. E. Pritchard, J. DeBoer, G. S. Stump, A. D. Ho, and D. T. Seaton, "Studying learning in the worldwide classroom: Research into edX's first MOOC," Research & Practice in Assessment, vol. 8, pp. 13-25, 2013.
- B. Li, X. Wang and S. C. Tan, "What makes MOOC users persist in completing MOOCs? A perspective from network externalities and human factors," Computers in Human Behavior, vol. 85, pp. 385-395, 2018, doi: 10.1016/j.chb.2018.04.028.
- J. C. Y. Sun and R. Rueda, "Situational interest, computer self-efficacy and self-regulation: Their impact on student engagement in distance education," British Journal of Educational Technology, vol. 43, no. 2, pp. 191-204, 2012, doi: 10.1111/j.1467-8535.2010.01157.x.
- K. P. Swan, J. C. Richardson, P. Ice, D. R. Garrison, M. Cleveland-Innes, and J. B. Arbaugh, "Validating a measurement tool of presence in online communities of inquiry," E-Mentor, vol. 2, no. 24, pp. 88-94, 2008.
- N. Shin, "Transactional presence as a critical predictor of success in distance learning," Distance Education, vol. 24, no. 1, pp. 69-86, 2003. https://doi.org/10.1080/01587910303048
- H. Roh, Statistical Analysis of Multivariate Data by SPSS, Seoul: Hyosung, 2007.
- B. Chen, Y. Fan, G. Zhang, and Q. Wang, "Examining motivations and self-regulated learning strategies of returning MOOCs learners," In Proceedings of the Seventh International Learning Analytics & Knowledge Conference, pp. 542-543, Mar. 2017.
- M. Barak, A. Watted, and H. Haick, "Motivation to learn in massive open online courses: Examining aspects of language and social engagement," Computers & Education, vol. 94, pp. 49-60, 2016, doi: 10.1016/j.compedu.2015.11.010.
- M. Yang, Z. Shao, Q. Liu, and C. Liu, "Understanding the quality factors that influence the continuance intention of students toward participation in MOOCs," Educational Technology Research and Development, vol. 65, no. 5, pp. 1195-1214, 2017, doi: 10.1007/s11423-017-9513-6.
- R. F. Kizilcec, M. Perez-Sanagustin, and J. J. Maldonado, "Self-regulated learning strategies predict learner behavior and goal attainment in Massive Open Online Courses," Computers & Education, vol. 104, pp. 18-33, 2017, doi: 10.1016/j.compedu.2016.10.001.
- K. F. Hew and W. S. Cheung, "Students' and instructors' use of massive open online courses (MOOCs): Motivations and challenges," Educational Research Review, vol. 12, pp. 45-58, 2014, doi: 10.1016/j.edurev.2014.05.001.
- G. Kennedy, C. Coffrin, P. De Barba, and L. Corrin, "Predicting success: how learners' prior knowledge, skills and activities predict MOOC performance," In Proceedings of the Fifth International Conference on Learning Analytics And Knowledge, pp. 136-140, Mar. 2015.