This past fall I took part in Stanford’s online learning experiment. For those of you who are not aware, Stanford gave anyone with Internet access the opportunity to enroll in three different online courses. This was free of charge, not even books were required. Each was an introductory level class, with the three subjects being artificial intelligence, machine learning, and databases (links may die as offered classes change). As far as I know, this was the first time that lecture videos have been paired with scheduled homework, quizzes, and programming assignments at this scale. I only vaguely remember numbers for the AI (artificial intelligence) and ML (machine learning) classes but I believe before classes began there were greater than 100,000 students enrolled apiece. The number of active students was approximately a third of the enrolled number. Even still, this is teaching at an unheard of scale.

I enrolled in both the AI and ML courses. Enrolling in two was more work, but I’m glad I did it. The classes followed slightly different formats and taking both provided a perspective in what style worked better for me.

AI Class

The AI class had video lectures, in video quizzes, homework, and two exams. The videos for this class were short. Each video lasted approximately one to four minutes. This was a nice length as you could easily go back and pick out specific topics to re-watch without jumping around a longer video. The videos showed the professor hand writing notes on pieces of paper. This style made it seem like you were receiving one-on-one tutoring.

At the end of some of the videos a quiz question was asked. These questions did not count towards your final grade. They existed to help you think and learn about the material as it was being presented.

The homework usually consisted of a fewer than 10 questions. You had until the due date to submit or change your answers. You did not get any feedback on the homework until it was graded.

For both of the exams you had about three days to finish the 15 or so questions that were asked.

ML Class

The ML class had video lectures, in video quizzes, homework, and programming assignments. In contrast to the AI class, the ML video lectures were usually five to thirteen minutes long. This made it harder to re-watch specific parts as you had to jump around the video to find different topics. On the plus side, the ML videos could be easily downloaded and watched on any device (you could not do this with the AI lectures). The ML videos also had controls for speeding up playback. I watched the vast majority of the videos at 1.5 times the normal speed.

The video questions for the ML class were similar to the AI class in that they didn’t count towards a final grade and there was usually a max of one per video. Because of the fewer number of videos, there were fewer quizzes than in the AI class. More quizzes would be good, as it forces the student to think instead of simply zoning out while watching the lectures.

The homework in the ML class was different from the AI class in a couple of ways. The largest difference was that you received feedback immediately when you submitted your answers. You were allowed to attempt the homework as often as you liked and your highest score became your score for that assignment. The questions were somewhat different between different attempts to minimize memorization of answers. There was also a minimum waiting period of ten minutes between attempts, but in order to reduce reliance on simply remembering previous attempts I usually waited a few hours to a day between attempts.

The ML class had programming assignments. In these exercises you filled in some Octave code to implement an aspect of machine learning. These exercises where great. Most of the time they made you think about the techniques you were learning that week. It was also nice to get some hands-on experience solving sample machine learning problems. It was rewarding watching my simple spam filter flag email as spam. The programming exercises were best when they weren’t simply a task in translating math to code.

Thoughts on the differences

I enjoyed the ML class homework style, instant feedback upon submission, better than the AI because it provided a quicker feedback cycle. Instead of submitting homework answers and waiting to review mistakes you were able to review instantly. I found this, and being able to repeat homework, to be more effective for learning than the single shot style of the AI class.

I preferred the shortness of the AI lectures. It allowed for easier repetition of lectures and provided more opportunities for quizzes. I also preferred the handwritten style of the AI lectures. The ML lectures felt like I was back in a classroom watching a professor go through some power point slides, adding handwritten notes as the lecture progressed. That isn’t very engaging. The one-on-one style of the AI lectures was more engaging.

I enjoyed both the programming exercises of ML class and the exams in the AI class. Both added an interesting way of learning to their respective class.

Recommendations

If I were designing a course for myself, I found the short videos with many quiz questions style of lecturing to be the more effective style. I would offer homework in a fashion similar to the ML class. Submit as many times as you want, but with a minimum time between submissions. I would up that time to be at least one hour to minimize memorization of answers. My ideal class would also have both programming exercises and exams.

Both classes had some student run question and answer boards (similar to stackoverflow). This was alright but I would look into adding a type of forum that is more suited towards discussions. A Q&A style board is not a great environment for having a discussion that is ordered by time. I think normal forums style with replies ordered by time would be effective and worth an experiment.

Conclusion

I greatly enjoyed taking these two Stanford classes. I think online lectures and homework were an extremely effective way of delivering information and reinforcing learning. I found taking these two classes to be a manageable amount of work. I did pretty much stop reading books while taking these classes and instead focused on watching lectures and doing homework. Had I only taken one class or spent breaks at work watching videos, I think I would have been able to maintain my other activities while taking these classes.

There are quite a few classes being offered at Stanford that start in the near future. Scroll to the bottom of this page to take a look at what Stanford is offering starting in January. I would highly recommend taking a class. MIT also just announced MITx. I haven’t heard a ton of information about it, but it sounds similar and I look forward to its launch.