MOOC completion and course activities

There has been a lot of discussion about MOOC completion, most of which has focused on completion rates: what percent of people complete a MOOC, and how should we calculate that number?  However, what has drawn less attention, but is potentially more interesting, is what in-course activities impact completion.  Understanding whether or not different course elements in a MOOC affect completion can potentially help us better understand best practices in MOOC design.

We used data from three of Duke’s MOOCs that recently concluded – The History and Future of (Mostly) Higher Education, Introduction to Genetics and Evolution, and Image and Video Processing: From Mars to Hollywood with a Stop at the Hospital – all three of which began in January and ended in March.  A total of 4,973 students successfully completed one of these three classes.

We asked students to complete a survey after the course ended.  A total of 1,979 students across the three courses completed a survey.  Note that this includes both students who received a Statement of Accomplishment and those who did not.  The surveys are roughly split 50/50 between those who completed the course and those who did not.

We were interested to find out what course activities increased the likelihood that someone would finish the course.  We asked students whether they participated in various activities – things like completing optional readings, participating in forums, or editing a course wiki.  The full list of activities we asked about is shown in the table below.  We then ran a statistical model predicting whether a student had completed a course based on their involvement in each of the activities.

Slide1

We found three especially interesting things in our analysis.  First, “Watch lectures” is omitted from most of the models.  This is because watching the lectures perfectly predicts completion in two of the three classes.  Unsurprisingly, no one completed the courses without watching the lectures in either Genetics of Image and Video Processing.  What is interesting is that some students in the Higher Education course did complete the course without watching lectures.  This is likely because the Higher Education course focused on delivering content in a variety of new and innovative ways using different mediums.  Because the “watch lectures” variable is not significant in that model, we believe this suggests that content can be effectively delivered in MOOCs in ways other than only through video lectures.

Second, we find that viewing the course wiki in the Higher Education course was significantly associated with completion.  Students who used the course wiki were 75% more likely to complete the course.  This may be because this course had a very extensive and active course wiki; 73% of survey respondents viewed the wiki at least once.  Our analysis indicates that participating in a student group in this course is associated with a lower likelihood of completion.  However, this is likely not a robust finding since less than 10% of the sample reported participating in a study group.

Third, there is a significant relationship between forum activity and completion, but only in the Genetics class.  In that class, students who participated in the forums were twice as likely to complete the course.  This could be due to the level of difficulty in this course; students who were actively discussing the material with other students may have learned more.  The survey respondents from the Genetics class were especially active in the forums; 46% reported some forum activity.

Note – while taking quizzes is obviously the strongest predictor of course completion, this is essentially a non-finding because the quizzes are the main graded elements of the courses.  Therefore, completing the course means, by definition, that students completed the quizzes.  We kept the variable in the analysis simply to confirm that the analysis was accurately modeling course completion.