Overview

Autolab is a course management service at CMU that enables instructors to offer programming labs to their students over the Internet. The two key ideas in Autolab are autograding, that is, programs evaluating other programs, and scoreboards:

Autograding. Each time a student submits their work, the system autogrades it on a secure Linux virtual machine, and stores the results in a gradebook. Autograders and the programs they evaluate can be written in any language and use any software packages. The model for a traditional programming class is that students work on their code, hand it in once, and then get feedback a week or two later, at which point they've already moved on to the next assignment. Autograding, on the other hand, allows students to get immediate feedback on their performance.

Scoreboard. The latest autograded scores for each student are displayed, rank ordered, on a real-time scoreboard. The scoreboard is a fun and powerful motivation for students. When coupled with autograding, it creates a sense of community and a healthy competition that seems to benefit everyone. Students anonymize themselves on the scoreboard by giving themselves nicknames. A mix of curiosity and competitiveness drives the stronger students to be at the top of the scoreboard, and all students have a clear idea of what they need for full credit. In our experience, everyone wins.

Autolab also provides other services that instructors expect in a course management system, including gradebooks, rosters, handins/handouts, lab writeups, code annotation, manual grading, late penalties, grace days, cheat checking, meetings, partners, and bulk emails.

Each semester, Autolab serves over 2,500 CMU students in many different courses. In a typical semester it autogrades over 100K jobs. We're still actively developing the system, and we hope to eventually make it available to other schools.

The Autolab Team

Tom Abraham: Lead developer.

Daniel Bucci: Front-end developer.

Abhay Buch: Front-end developer and deployment guru.

Xuetao Guan: Lead developer (PKU, Beijing).

Ashley Kumar: Front-end developer.

Dave O'Hallaron: Project lead.

Mihir Pandya: Back-end developer and systems guru.

Dylan Swen: Front-end developer.

An Wu: Front-end developer.

Peter Xiao: Front-end developer.

Jimmy Zong: Front-end developer.

Distinguished Alumni

Autolab Courses

Here are some of the courses at CMU that are using Autolab. Legend: f=Fall, s=Spring, m=Summer.

02-713: Algs and Data Structs for Scientists | s13
15-110: Principles of Computing | f10| s11| m11| s12| f12| s13
15-112: Fundamentals of Programming | f11| s12| m12| f12| s13
15-122: Imperative Programming | s12|m12| f12| s13
15-122: Imperative Programming (Qatar) | s13
15-150: Functional Programming | s13
15-210: Parallel and Sequential Data Structs and Algs | s13
15-213: Intro to Computer Systems | f10| s11| m11| f11| s12| m12| f12| s13
15-213z: Intro to Comp Sys (Peking Univ) | n12 | f12
15-237: Cross-Platform Mobile Web Apps | f12 | s13
15-359: Probability and Computing | s13
15-411: Compiler Design | f11| f12
15-440: Distributed Systems | f11| f12
15-440w: Distributed Systems (Qatar) | f12
15-732: Secure Software Systems | s13
15-746: Storage Systems | s12| s13

Resources

Documentation: Complete info on the system. Includes an Instructor Guide, Lab Author Guide, Developer Guide, and a detailed Reference Guide.

Overview: A recent talk by Dave O that describes the system and the vision for the future.

Interview. Dave O did an interview with IEEE Internet Computing that explains the background, motivation, and goals for the Autolab project.

Job opportunities: We're always looking for top undergraduate programmers to work with us. If you're passionate about improving the learning experience for students and are good at design, Rails programming, Javascript, Python, or systems level programming, please contact us at autolab-dev@andrew.cmu.edu.

Contact Us : If you have any questions or suggestions, you can contact us at autolab-dev@andrew.cmu.edu.