PyBryt GitHub Action

GitHub Actions allow you to automate workflows for repositories for tasks like CI/CD. With GitHub Actions, or indeed any CI/CD service, you can automate PyBryt to run on repositories of student code.

PyBryt has a pre-made action that can be used in GitHub Actions workflows called microsoft/pybryt-action. With this action, you can set up PyBryt to run automatically on students’ submissions whenever they push to their repositories.

Let’s consider an example assignment with student repositories structured like:

.
├── refs
│   ├── ref-1.pkl
│   └── ref-2.pkl
└── submission.ipynb

To automate PyBryt for the submission submission.ipynb, you would add a new GitHub Actions workflow file in .github/workflows:

.
├── .github
│   └── workflows
│       └── run-pybryt.yml
├── refs
│   ├── ref-1.pkl
│   └── ref-2.pkl
└── submission.ipynb

Inside this file, define the following action configuration:

name: Run PyBryt

on:
  push:
    branches: [ main ]
  workflow_dispatch:

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Run PyBryt
        uses: microsoft/pybryt-action@v0.1.1
        with:
          submission-path: submission.ipynb
          references: |
            refs/ref-1.pkl
            refs/ref-2.pkl

This configuration tells the PyBryt action that it should execute the student implementation submission.ipynb against the references listed in the references key. (Elements of the references key can be either paths to files or URLs.) It is set up to run whenever a commit is pushed to the main branch or when triggered manually by the user.

When this action runs, PyBryt will execute the submission specified to generate a student implementation and then it will run that implementation against the provided references. It will then print a report to the console containing the details of the passing and failing of the implementation for each reference as well as any messages generated by the reference.

In some cases, you may want a record of the objects PyBryt generates, like the report and student implementation and results objects. The PyBryt action stores these artifacts in files that can be found by looking at the outputs of the step that runs PyBryt.

For example, to save all of these artifacts as files in a results directory, you could use a configuration like this:

name: Run PyBryt

on:
  push:
    branches: [ main ]
  workflow_dispatch:

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Run PyBryt
        id: pybryt
        uses: microsoft/pybryt-action@v0.1.1
        with:
          submission-path: submission.ipynb
          references: |
            refs/ref-1.pkl
            refs/ref-2.pkl

      - name: Save, commit, and push results
        run: |
          mkdir -p results
          cp ${{ steps.pybryt.outputs.report-path }} results/report.txt
          cp ${{ steps.pybryt.outputs.results-path }} results/results.pkl
          cp ${{ steps.pybryt.outputs.student-implementation-path }} results/student-implementation.pkl
          git add results
          git commit -m "PyBryt results for ${{ github.sha }}"
          git push

The second step of the action above copies the files from the paths listed in the outputs from the PyBryt action into files in the results directory before committing and pushing them. The two .pkl files can be unpickled with the dill library for further use.

For more information about the PyBryt action and how to use it, see its documentation.