Automated Grading with the Canary Framework
Canary Framework Overview
The Canary Framework is a set of open source software engineering tools and supplemental resources that support automated grading. Details about how the Canary Framework is used at NC State Computer Science in our SIGCSE 2018 paper.
Students push their code submissions to [GitHub](https://enterprise.github.com/home_. The continuous integration system, Jenkins, pulls their code and evaluates it as appropriate for the class. We run the Jenkins master and executors on the Virtual Computing Lab (VCL).
Setup Instructions
Tooling
We have created (or are creating) tooling to support the Canary Framework automated grading system and other teaching related tasks. Information about each tool are set of tools are below:
- GitHubManager: tool that creates GitHub repositories, adds students to teams, and updates repository permissions from push (write) to pull (read) when a project is closed.
- JenkinsJobResources: tools to create and close Jenkins jobs.
- BuildResources: scripts and build configuration files to evaluate student submissions.
- JenkinsWebScraper: Selenium program that scrapes results from the most recent build on Jenkins for use when grading.
- BuildDataCollector: Forthcoming tooling that supports the collection of student build information into a database for data analysis and grading.
- FeedbackFileGenerator: generates a PDF feedback file with information about a student’s grade on an assignment from a grade spreadsheet.