Empirical Analysis of the Impact of Openness on Paying Enrollments in Distance Education Courses and Implications for Sustainability

Session Title: Empirical Analysis of the Impact of Openness on Paying Enrollments in Distance Education Courses and Implications for Sustainability

Presenters: David Wiley, Associate Professor, Brigham Young University
Justin Johansen, Director of BYU Independent Study, Brigham Young University

Download Video

Time & Date: 10:15 A.M. - 11:00 A.M.

Location: Rm. C100

Session Description: For this study three university-level and three high school courses will be opened. The actual costs of opening each course will be carefully tracked and reported. The effect of opening the courses on paid enrollments will be examined. An analysis of the financial self-sustainability of a program of opening BYU IS courses will then be conducted using the cost and enrollment data.

A cost tracking system has been developed to help determine how much it costs to develop each course produced by BYU IS. Those within BYU IS who work on opening courses will bill their time accordingly, and their reported time will be used to determine the costs of converting a course to the open format. Existing technical infrastructure (e.g., servers) will be used so that no additional overhead costs are incurred.

The research will employ an interrupted time-series quasi-experimental design using equivalent, no-treatment control groups (Shadish, Cook, and Campbell, 2002). The open publication of courses is the treatment or “interruption”. After at least three months have passed after opening of courses, the enrollment data for each course will be analyzed. Specifically, comparisons of the slope and intercept of the time series trend lines before the treatment will be compared with the slope and intercept of the trend line after the treatment.

Because this study uses a shorter time series, a non-equivalent control group will also be used to improve the interpretability of the study’s results as suggested by (Shadish et al., 2002). For each of the three university-level and three high school courses to be opened, at least one additional course with similar enrollment trends will be identified as a control. Similarity here means similarity of slope of the enrollment trend line over a period of the preceding six months, as determined through a process of intraocular inspection of visualized data. There is no test of significance for use with interrupted time series data. For each course in the treatment group, the slope and intercept of the trend lines after the treatment will be compared with the slope and intercept of the appropriate control group course trend lines after treatment.

In addition to the data coming from the interrupted time-series design, additional data will be gathered to triangulate findings. There are four OCW user groups of interest: (1) OCW users who never click the “click to register” button and never register for a course, (2) OCW users who never click the “click to register” button but register via another path, (3) OCW users who click the “click to register” button, but never actually complete registration, and (4) those OCW users who both click the “click to register” button and register. Web analytics data can give us insight into groups (1), (3), and (4); giving a direct measure of how many OCW users convert via online registration. We would expect any additional group (2) enrollments to show up (indirectly) in the time series data.
Using cost, income, and enrollment data gathered during the study, an analysis using the above formula will be performed to make an initial estimate of the financial sustainability of a long-term open publishing initiative within BYU IS.