Agile Development Considerations for ML Projects
Overview
When running ML projects, we follow the Agile methodology for software development with some adaptations, as we acknowledge that research and experimentation are sometimes difficult to plan and estimate.
Goals
- Run and manage ML projects effectively
- Create effective collaboration between the ML team and the other teams working on the project
To learn more about how ISE runs the Agile process for software development teams, refer to this doc.
Within this framework, the team follows these Agile ceremonies:
- Backlog management
- Retrospectives
- Scrum of Scrums (where applicable)
- Sprint planning
- Stand-ups
- Working agreement
Agile Process During Exploration and Experimentation
-
While acknowledging the fact that ML user stories and research spikes are less predictable than software development ones, we strive to have a deliverable for every user story in every sprint.
-
User stories and spikes are usually estimated using T-shirt sizes or similar, and not in actual days/hours.
-
ML design sessions should be included in each sprint.
Examples of ML Deliverables for each Sprint
- Working code (e.g. models, pipelines, exploratory code)
- Documentation of new hypotheses, and the acceptance or rejection of previous hypotheses as part of a Hypothesis Driven Analysis (HDA). For more information see Hypothesis Driven Development on Barry Oreilly's website
- Exploratory Data Analysis (EDA) results and learnings documented
Collaboration Between Data Scientists and Software Developers
- Data scientists and software developers work together on the project. The team uses one backlog and attend the same Agile ceremonies. In cases where the project has many participants, we will divide into working groups, but still have the entire team join the Agile ceremonies.
- If possible, feasibility study and initial model experimentation takes place before the operationalization work kicks off.
- Everyone shares the accountability for the MLOps solution.
- The ML model interface (API) is determined as early as possible, to allow the developers to consider its integration into the production pipeline.
- MLOps artifacts are developed with a continuous collaboration and review of the data scientists, to ensure the appropriate approaches for experimentation and productization are used.
- Retrospectives and sprint planning are performed on the entire team level, and not the specific work groups level.
Last update:
August 26, 2024