sammo.store

Implements two different types of dictionaries that can store either data in memory or on disk. Allows keys to be arbitrary JSON-serializable objects that get rendered to byte strings for indexing. Mainly used to cache LLM API calls, but can be used for other purposes as well.

Module Contents

Classes

PersistentDict	Implements a dictionary that is persisted to disk. Entries are appended to the end of the file, with later entries overwriting earlier ones. The file is read into memory on initialization to allow for fast lookups. Write and delete operations are thread-safe.
InMemoryDict	Implements a dictionary that lives only in memory. Entries are not persisted to disk unless persist is called.

API

class sammo.store.PersistentDict(filename: os.PathLike | str)

Bases: collections.abc.MutableMapping, pyglove.JSONConvertible

Implements a dictionary that is persisted to disk. Entries are appended to the end of the file, with later entries overwriting earlier ones. The file is read into memory on initialization to allow for fast lookups. Write and delete operations are thread-safe.

Parameters:

filename – path for the stored data. Loads the dictionary from the given file, or creates a new one if it doesn’t exist.

Initialization

vacuum() → None

Removes all deleted entries from the file.

to_json(**kwargs)

classmethod from_json(json_value, **kwargs)

class sammo.store.InMemoryDict

Bases: sammo.store.PersistentDict

Implements a dictionary that lives only in memory. Entries are not persisted to disk unless persist is called.

Initialization

persist(filename: os.PathLike | str)

Persists the dictionary to disk.

Parameters:

filename – path for the stored data.