Overview

Autolab is a CMU course management service that enables instructors to offer programming assignments (labs) to their students over the Internet. It is based on autograding, that is, programs evaluating other programs. Each time a student submits their work, the system autogrades it on a secure virtual machine instance running in the cloud, stores the rsulting scores in a gradebook, and returns feedback to the student. The autograded scores for each student can be displayed, rank ordered, on a real-time scoreboard.

Autograding and the scoreboard are two key ideas in Autolab:

• Autograding. The model for a traditional programming class is that students work on their code, hand it in once, and then get feedback a week or two later, at which point they've already moved on to the next assignment. Autograding, on the other hand, allows students to get immediate continuous and immediate feedback on their performance, improve it, and as a result, learn more.
• Scoreboard. The scoreboard is a fun and powerful motivation for students. When coupled with autograding, it creates a sense of community and a healthy competition that seems to benefit everyone. Students anonymize themselves on the scoreboard by giving themselves nicknames (two of our all-time favorites are "kill -9 15213" and "213 makes me ANSI"). A mix of curiosity and competitiveness drives the stronger students to be at the top of the scoreboard, and all students have a clear idea of what they need for full credit and can see firsthand that good solutions are indeed possible. In our experience, everyone wins.

Autolab also provides the other services that instructors expect in a course management system, including gradebooks, rosters, handins/handouts, code annotation, manual grading, cheat checking, and bulk emails.

The first Autolab prototype was written in 2002 by Dave O'Hallaron for 15-213: Introduction to Computer Systems. It was reimplemented in 2010 in a modern Rails form, with a secure and scalable backend, by Hunter Pitelka, Dave O'Hallaron, and Kelly Rivers, with contributions from some talented students, including Abhay Buch, Tom Abraham, and Steven Fackler. In a typical semester, Autolab serves over 1,400 students in 5-6 courses, handles over 1M page views, and autogrades over 60K jobs. We're actively developing the system, and we hope to make it available to other schools soon, using Amazon's EC2 as the backend host.

Blogroll

CMU Courses

Resources

Instructor Guide: Describes how to use Autolab for your course.

Documentation: Complete info on the system. Includes an Instructor Guide, Lab Author Guide, Developer Guide, and a detailed Reference Guide.

Autolab overview. Dave did an interview with IEEE Internet Computing that explains the background, motivation, and goals for the Autolab project.

Bugtracker: Submit any bugs or feature requests you find, and watch for the development team to update the items you're waiting for.

Job opportunities: We're always looking for top undergraduate programmers to work with us. If you're passionate about improving the learning experience for students and are good at design, Rails programming, Javascript, Python, or systems level programming, please contact us.

The Autolab Team