Skip to content

rats.projects

Interact with the various components in the repository during development.

A project is loosely tied to one repository, which is made up of one or more components. Each component is a separate entity that can be built, tested, and released independently. This module provides a handful of convenience APIs to query for project and component information, along with exposing a few methods to run commands within the context of a component's development environment. It's common for python environments to be misconfigured, and as we run commands in multiple components, it's easy to accidentally run commands in a component with the wrong virtual environment being activated, with unpredictable results. The libraries in this module try to alleviate some of these pain points while trying to remain agnostic of env management tools and other component specific choices.

Warning

Python environments are never trivial to manage, and are a common source of complexity because of the many ways to define your development environment. We've tested the functionality in this module heavily for the set of tools we use to develop rats, but some of the contained functionality might not work fully within components that use a different set of tool, like conda.

__all__ = ['ComponentId', 'ComponentNotFoundError', 'ComponentTools', 'PluginConfigs', 'PluginContainer', 'PluginServices', 'ProjectConfig', 'ProjectNotFoundError', 'ProjectTools', 'UnsetComponentTools', 'find_nearest_component', 'find_repo_root'] module-attribute

ComponentId

Bases: NamedTuple

A simple wrapper around the name of a component, for typing convenience.

name instance-attribute

ComponentTools(path)

A small collection of operations commonly done on components.

This class might contain unrelated things like poetry and docker specific methods that we can hopefully move to better components in the future.

Source code in src/rats/projects/_component_tools.py 
32
33
def __init__(self, path: Path) -> None:
    self._path = path

component_name()

The component's name, as defined by [project.name] in pyproject.toml.

Source code in src/rats/projects/_component_tools.py 
35
36
37
def component_name(self) -> str:
    """The component's name, as defined by `[project.name]` in `pyproject.toml`."""
    return self._load_pyproject()["project"]["name"]

Create a symlink in the component directory.

The destination path must be relative to the component directory.

Parameters:

Name Type Description Default
src Path

the existing file or directory to link to

 required
dst Path

the symlink path to be created

 required
Source code in src/rats/projects/_component_tools.py 
39
40
41
42
43
44
45
46
47
48
49
50
51
def symlink(self, src: Path, dst: Path) -> None:
    """
    Create a symlink in the component directory.

    The destination path must be relative to the component directory.

    Args:
        src: the existing file or directory to link to
        dst: the symlink path to be created
    """
    self._validate_component_path(dst)

    symlink(src, dst)

copy(src, dst)

Copy a file into the component.

The destination path must be relative to the component directory.

Parameters:

Name Type Description Default
src Path

the existing file to copy

 required
dst Path

the destination of the copied file

 required
Source code in src/rats/projects/_component_tools.py 
53
54
55
56
57
58
59
60
61
62
63
64
65
66
def copy(self, src: Path, dst: Path) -> None:
    """
    Copy a file into the component.

    The destination path must be relative to the component directory.

    Args:
        src: the existing file to copy
        dst: the destination of the copied file
    """
    # we expect this instance to create files in the matching component
    self._validate_component_path(dst)

    copy(src, dst)

copy_tree(src, dst)

Copy a directory into the component.

The destination path must be relative to the component directory.

Parameters:

Name Type Description Default
src Path

the existing directory to copy

 required
dst Path

the destination of the copied directory

 required
Source code in src/rats/projects/_component_tools.py 
68
69
70
71
72
73
74
75
76
77
78
79
80
def copy_tree(self, src: Path, dst: Path) -> None:
    """
    Copy a directory into the component.

    The destination path must be relative to the component directory.

    Args:
        src: the existing directory to copy
        dst: the destination of the copied directory
    """
    self._validate_component_path(dst)

    copytree(src, dst, dirs_exist_ok=True)

create_or_empty(directory)

Ensure a directory in the component exists and is empty.

Source code in src/rats/projects/_component_tools.py 
82
83
84
85
86
87
88
def create_or_empty(self, directory: Path) -> None:
    """Ensure a directory in the component exists and is empty."""
    self._validate_component_path(directory)
    if directory.exists():
        rmtree(directory)

    directory.mkdir(parents=True)

find_path(name)

Given a path, relative to the root of the component, return the full path.

All paths are expected to be within the directory of the component.

Source code in src/rats/projects/_component_tools.py 
90
91
92
93
94
95
96
97
98
def find_path(self, name: str) -> Path:
    """
    Given a path, relative to the root of the component, return the full path.

    All paths are expected to be within the directory of the component.
    """
    path = (self._path / name).resolve()
    self._validate_component_path(path)
    return path

pytest()

Source code in src/rats/projects/_component_tools.py 
105
106
def pytest(self) -> None:
    self.run("pytest")

ruff(*args)

Source code in src/rats/projects/_component_tools.py 
108
109
def ruff(self, *args: str) -> None:
    self.run("ruff", *args)

pyright()

Source code in src/rats/projects/_component_tools.py 
111
112
def pyright(self) -> None:
    self.run("pyright")

poetry(*args)

Source code in src/rats/projects/_component_tools.py 
114
115
116
117
118
119
120
121
122
123
def poetry(self, *args: str) -> None:
    warnings.warn(
        "The ComponentTools.poetry() method is deprecated. Use ComponentTools.run() instead.",
        DeprecationWarning,
        stacklevel=2,
    )
    if not self.is_poetry_detected():
        raise RuntimeError(f"cannot run poetry commands in component: {self.component_name()}")

    self.exe("env", "-u", "POETRY_ACTIVE", "-u", "VIRTUAL_ENV", "poetry", *args)

run(*args)

Tries to run a command within the component's venv.

Source code in src/rats/projects/_component_tools.py 
125
126
127
128
129
130
131
132
133
def run(self, *args: str) -> None:
    """Tries to run a command within the component's venv."""
    # generally try to unset any package manager venv specific details
    if self.is_poetry_detected():
        self.exe("env", "-u", "POETRY_ACTIVE", "-u", "VIRTUAL_ENV", "poetry", "run", *args)
    elif self._is_uv_detected():
        self.exe("env", "-u", "UV_ACTIVE", "-u", "VIRTUAL_ENV", "uv", "run", *args)
    else:
        self.exe(*args)

exe(*cmd)

Run a command from the root of a component.

Source code in src/rats/projects/_component_tools.py 
135
136
137
138
139
140
141
142
def exe(self, *cmd: str) -> None:
    """Run a command from the root of a component."""
    logger.debug(f"executing in {self._path}/: {' '.join(cmd)}")
    try:
        subprocess.run(cmd, cwd=self._path, check=True)
    except subprocess.CalledProcessError as e:
        logger.error(f"failure detected: {' '.join(cmd)}")
        sys.exit(e.returncode)

is_poetry_detected()

Returns true if we think this component might be managed by poetry.

Since PEP 621 is gaining adoption, including by poetry, we should be able to remove most of the complexity in trying to parse details out of pyproject.toml. This method is here until we can fully delete any non PEP 621 code since we initially started as poetry-specific.

Source code in src/rats/projects/_component_tools.py 
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
def is_poetry_detected(self) -> bool:
    """
    Returns true if we think this component might be managed by poetry.

    Since PEP 621 is gaining adoption, including by poetry, we should be able to remove most of
    the complexity in trying to parse details out of pyproject.toml. This method is here until
    we can fully delete any non PEP 621 code since we initially started as poetry-specific.
    """
    data = self._load_pyproject()
    if "tool" in data and "poetry" in data["tool"]:
        # we found some poetry values in the toml file
        return True

    # make double sure by checking if we see a lockfile for poetry
    return self.find_path("poetry.lock").is_file()

UnsetComponentTools(path)

Bases: ComponentTools

A stub component tools without any implemented operations.

All methods within this class raise a NotImplementedError.

Source code in src/rats/projects/_component_tools.py 
32
33
def __init__(self, path: Path) -> None:
    self._path = path

copy_tree(src, dst)

Source code in src/rats/projects/_component_tools.py 
186
187
def copy_tree(self, src: Path, dst: Path) -> None:
    raise NotImplementedError("no component selected")

create_or_empty(directory)

Source code in src/rats/projects/_component_tools.py 
189
190
def create_or_empty(self, directory: Path) -> None:
    raise NotImplementedError("no component selected")

find_path(name)

Source code in src/rats/projects/_component_tools.py 
192
193
def find_path(self, name: str) -> Path:
    raise NotImplementedError("no component selected")

exe(*cmd)

Source code in src/rats/projects/_component_tools.py 
195
196
def exe(self, *cmd: str) -> None:
    raise NotImplementedError("no component selected")

PluginConfigs

Configurable services within the rats.projects module.

PROJECT = apps.ServiceId[ProjectConfig]('project') class-attribute instance-attribute

Main config object to change the behavior of rats.projects libraries.

PluginContainer(app)

Bases: apps.Container, apps.PluginMixin

Source code in rats/apps/_app_containers.py 
90
91
def __init__(self, app: Container) -> None:
    self._app = app

PluginServices

Services made available from the [rats.projects] module.

CWD_COMPONENT_TOOLS = apps.ServiceId[ComponentTools]('cwd-component-tools') class-attribute instance-attribute

The component tools instance for the component the command was run within.

PROJECT_TOOLS = apps.ServiceId[ProjectTools]('project-tools') class-attribute instance-attribute

Main project library to interact with the repository and the contained components.

CONFIGS = PluginConfigs class-attribute instance-attribute

Alias to the rats.projects.ProjectConfig class.

ComponentNotFoundError

Bases: ValueError

ProjectConfig

Bases: NamedTuple

Settings used by the rats.projects.ProjectTools libraries.

name instance-attribute

The name of your project.

In monorepos, this is typically the name of the repository. In single-component repositories, we generally expect the name of the repo to match the name of the component/package.

path instance-attribute

The path to the root of the project.

image_registry instance-attribute

The name of the container image registry built images will be tagged with.

image_push_on_build instance-attribute

When enabled, images are automatically pushed to the defined registry when built.

image_tag = None class-attribute instance-attribute

The version tag of the container image built images will be tagged with.

ProjectNotFoundError

Bases: ValueError

ProjectTools(config)

Small collection of methods to operate on the project and access component tools.

A config is required to specify the behavior of this instance.

Parameters:

Name Type Description Default
config apps.Provider[ProjectConfig]

the configuration of the project we are operating within.

 required
Source code in src/rats/projects/_project_tools.py 
45
46
47
48
49
50
51
52
def __init__(self, config: apps.Provider[ProjectConfig]) -> None:
    """
    A config is required to specify the behavior of this instance.

    Args:
        config: the configuration of the project we are operating within.
    """
    self._config = config

build_component_images()

Sequentially builds container images for every component in the project.

Source code in src/rats/projects/_project_tools.py 
54
55
56
57
def build_component_images(self) -> None:
    """Sequentially builds container images for every component in the project."""
    for c in self.discover_components():
        self.build_component_image(c.name)

build_component_image(name)

Builds the container image for a given component in the project.

Parameters:

Name Type Description Default
name str

the name of the component to be built.

 required
Source code in src/rats/projects/_project_tools.py 
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
def build_component_image(self, name: str) -> None:
    """
    Builds the container image for a given component in the project.

    Args:
        name: the name of the component to be built.
    """
    component_tools = self.get_component(name)
    file = component_tools.find_path("Containerfile")
    if not file.exists():
        file = component_tools.find_path("Dockerfile")

    if not file.exists():
        raise RuntimeError(f"Containerfile/Dockerfile not found in component {name}")

    config = self._config()
    image = self.container_image(name)

    print(f"building docker image: {image.full}")
    component_tools.exe(
        "docker",
        "build",
        "-t",
        image.full,
        "--file",
        str(file),
        str(self.repo_root()),
    )

    if image.name.split("/")[0].split(".")[1:3] == ["azurecr", "io"]:
        acr_registry = image.name.split(".")[0]
        if os.environ.get("DEVTOOLS_K8S_SKIP_LOGIN", "0") == "0":
            component_tools.exe("az", "acr", "login", "--name", acr_registry)

    if config.image_push_on_build:
        component_tools.exe("docker", "push", image.full)

container_image(name)

Get the calculated container image for the given component.

If a tag is not present in the rats.projects.ProjectConfig, one is calculated using rats.projects.ProjectTools.image_context_hash.

Parameters:

Name Type Description Default
name str

the name of the component for which we want the image.

 required
Source code in src/rats/projects/_project_tools.py 
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
def container_image(self, name: str) -> ContainerImage:
    """
    Get the calculated container image for the given component.

    If a tag is not present in the [rats.projects.ProjectConfig][], one is calculated using
    [rats.projects.ProjectTools.image_context_hash][].

    Args:
        name: the name of the component for which we want the image.
    """
    config = self._config()
    return ContainerImage(
        name=f"{config.image_registry}/{name}",
        tag=config.image_tag or self.image_context_hash(),
    )

image_context_hash() cached

Calculates a hash based on all the files available to the container build context.

The hash is calculated by ignoring all files selected by .gitignore configs, and hashing all remaining files from the root of the project, giving us a unique hash of all possible contents that can be copied into an image.

Source code in src/rats/projects/_project_tools.py 
112
113
114
115
116
117
118
119
120
121
122
@cache  # noqa: B019
def image_context_hash(self) -> str:
    """
    Calculates a hash based on all the files available to the container build context.

    The hash is calculated by ignoring all files selected by `.gitignore` configs, and hashing
    all remaining files from the root of the project, giving us a unique hash of all possible
    contents that can be copied into an image.
    """
    manifest = self.image_context_manifest()
    return sha256(manifest.encode()).hexdigest()

image_context_manifest() cached

Calculates a manifest of the files in the image context.

When building container images, this hash can be used to determine if any of the files in the image might have changed. This manifest is used by methods like rats.projects.ProjectTools.image_context_hash.

Inspired by https://github.com/5monkeys/docker-image-context-hash-action

Source code in src/rats/projects/_project_tools.py 
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
@cache  # noqa: B019
def image_context_manifest(self) -> str:
    """
    Calculates a manifest of the files in the image context.

    When building container images, this hash can be used to determine if any of the files in
    the image might have changed. This manifest is used by methods like
    [rats.projects.ProjectTools.image_context_hash][].

    Inspired by https://github.com/5monkeys/docker-image-context-hash-action
    """
    containerfile = dedent("""
        FROM mcr.microsoft.com/mirror/docker/library/ubuntu:24.04
        COPY . /image-context
        WORKDIR /image-context

        CMD ["bash", "-c", "find . -type f | sort"]
    """)

    subprocess.run(
        ["docker", "build", "-t", "image-context-hasher", "-f-", "."],
        input=containerfile,
        check=True,
        cwd=self.repo_root(),
        capture_output=True,
        text=True,
    )

    output = subprocess.run(
        [
            "docker",
            "run",
            "--pull",
            "never",
            "--rm",
            "image-context-hasher",
        ],
        check=True,
        cwd=self.repo_root(),
        capture_output=True,
        text=True,
    ).stdout

    def _file_hash(p: str) -> str:
        contents = (self.repo_root() / p).read_bytes()
        return f"{sha256(contents).hexdigest()}\t{p}"

    lines = [f"{_file_hash(line[2:])}" for line in sorted(output.strip().split("\n"))]

    return "\n".join(lines)

project_name()

The name of the project, as defined by the provided rats.projects.ProjectConfig.

Source code in src/rats/projects/_project_tools.py 
175
176
177
def project_name(self) -> str:
    """The name of the project, as defined by the provided [rats.projects.ProjectConfig][]."""
    return self._config().name

discover_components() cached

Looks through the code base for any components containing a pyproject.toml file.

Source code in src/rats/projects/_project_tools.py 
179
180
181
182
@cache  # noqa: B019
def discover_components(self) -> tuple[ComponentId, ...]:
    """Looks through the code base for any components containing a `pyproject.toml` file."""
    return tuple(self._component_paths().keys())

get_component(name)

Get the component tools for a given component.

Parameters:

Name Type Description Default
name str

the name of the component within the project.

 required
Source code in src/rats/projects/_project_tools.py 
184
185
186
187
188
189
190
191
192
193
194
195
def get_component(self, name: str) -> ComponentTools:
    """
    Get the component tools for a given component.

    Args:
        name: the name of the component within the project.
    """
    cid = ComponentId(name)
    if cid not in self._component_paths():
        raise ComponentNotFoundError(f"component {name} is not a valid python component")

    return ComponentTools(self._component_paths()[cid])

repo_root()

The path to the root of the repository.

Source code in src/rats/projects/_project_tools.py 
197
198
199
200
201
202
203
204
205
206
207
208
def repo_root(self) -> Path:
    """The path to the root of the repository."""
    p = Path(self._config().path).resolve()
    # 99% of the time we just want the root of the repo
    # but in tests we use sub-projects to create fake scenarios
    # better test tooling can probably help us remove this later
    if not (p / ".git").exists() and not (p / ".rats-root").exists():
        raise ProjectNotFoundError(
            f"repo root not found: {p}. devtools must be used on a project in a git repo."
        )

    return p

find_nearest_component(cwd=None)

Try to find the path to the root of the nearest component.

This method traverses up the directory tree, starting from the working directory, and looks for the first directory that contains a pyproject.toml file.

Parameters:

Name Type Description Default
cwd Path | None

optionally provide a starting search directory

 None
Source code in src/rats/projects/_plugin.py 
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
def find_nearest_component(cwd: Path | None = None) -> Path:
    """
    Try to find the path to the root of the nearest component.

    This method traverses up the directory tree, starting from the working directory, and looks for
    the first directory that contains a `pyproject.toml` file.

    Args:
        cwd: optionally provide a starting search directory
    """
    if cwd is None:
        cwd = Path.cwd()

    guess = cwd.resolve()
    while str(guess) != "/":
        if (guess / "pyproject.toml").exists() and (guess / "pyproject.toml").is_file():
            return guess
        guess = guess.parent

    raise ComponentNotFoundError(f"component root not found from cwd: {cwd.as_posix()}.")

find_repo_root(cwd=None)

Try to find the path to the root of the repo.

This method traverses up the directory tree, starting from the working directory, and looks for the first directory that contains a .git directory. This behavior can be overwritten by defining a DEVTOOLS_PROJECT_ROOT environment variable.

Parameters:

Name Type Description Default
cwd Path | None

optionally provide a starting search directory

 None
Source code in src/rats/projects/_plugin.py 
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
def find_repo_root(cwd: Path | None = None) -> Path:
    """
    Try to find the path to the root of the repo.

    This method traverses up the directory tree, starting from the working directory, and looks for
    the first directory that contains a `.git` directory. This behavior can be overwritten by
    defining a `DEVTOOLS_PROJECT_ROOT` environment variable.

    Args:
        cwd: optionally provide a starting search directory
    """
    env = os.environ.get("DEVTOOLS_PROJECT_ROOT")
    if env:
        return Path(env)

    if cwd is None:
        cwd = Path.cwd()

    guess = cwd.resolve()
    while str(guess) != "/":
        if (guess / ".git").exists():
            return guess
        guess = guess.parent

    raise ProjectNotFoundError(
        "repo root not found. devtools must be used on a project in a git repo."
    )