Ledger
======

.. note:: See the :doc:`/operations/ledger_snapshot` section to find out more about how ledger files are generated by CCF nodes and how they should be managed by operators.

The entire service and application state is contained in the ledger (including both governance and application transactions). The ledger contains regular signature transactions (``public:ccf.internal.signatures`` map) which sign the root of the :term:`Merkle Tree` at the time the signature transaction is emitted.

.. tip:: Use the ``read_ledger.py`` utility available as part of the :doc:`/audit/python_library` to read the public content of a CCF ledger.


Ledger Format
-------------

The ledger is split into multiple files (or chunks). Each ledger file starts with an 8-byte offset pointing to the end of file where a vector of positions is stored for fast fetching of transactions (only set when a file is complete, internal use only).

The serialised transactions follow the 8-byte offset, as described in :ref:`architecture/ledger:Transaction Format`.

.. note:: A complete ledger file always ends on a signature transaction.

Transaction Format
------------------

The following table describes the structure of a serialised transaction as it is stored in the ledger. 

.. note:: These serialised transactions in CCF currently describes only the final write set rather than the internal execution path of a transaction; writes are reordered, only the final writes are present, and reads are omitted (read count will be 0).

+----------+------------------------------------------+-------------------------------------------------------------------------+
|          | Field Type                               | Description                                                             |
+==========+==========================================+=========================================================================+
|          |                                          | | - Version (1 byte)                                                    |
|          | Transaction Header                       | | - Flags (1 byte)                                                      |
|          |                                          | | - Entry size (6 bytes)                                                |
+  Header  +------------------------------------------+-------------------------------------------------------------------------+
|          | AES GCM Header                           | IV and tag fields required to decrypt and verify integrity              |
+          +------------------------------------------+-------------------------------------------------------------------------+
|          | uint64_t                                 | Length of serialised public domain                                      |
+----------+------------------------------------------+-------------------------------------------------------------------------+
|          | :cpp:type:`ccf::kv::EntryType`           | Snapshot, or a WriteSet variant                                         |
+          +------------------------------------------+-------------------------------------------------------------------------+
|          | :cpp:type:`ccf::kv::Version`             | Transaction version                                                     |
+          +------------------------------------------+-------------------------------------------------------------------------+
|          | :cpp:type:`ccf::crypto::Sha256Hash`      | User-defined claims digest, when entry type is WriteSetWith.*Claims     |
+          +------------------------------------------+-------------------------------------------------------------------------+
|          | :cpp:type:`ccf::crypto::Sha256Hash`      | Commit evidence digest, when entry type is WriteSetWithCommitEvidence.* |
+          +------------------------------------------+-------------------------------------------------------------------------+
|          | :cpp:type:`ccf::kv::Version`             | Unused, reserved for compatibility                                      |
+          +------------------------------------------+-------------------------------------------------------------------------+
|          | **Repeating [0..n]**                     | With ``n`` the number of maps in the transaction                        |
+          +-----+------------------------------------+-------------------------------------------------------------------------+
|          |     | std::string                        | Name of the serialised :cpp:type:`ccf::kv::Map`                         |
|          +-----+------------------------------------+-------------------------------------------------------------------------+
|          |     | :cpp:type:`ccf::kv::Version`       | Read version                                                            |
|          +-----+------------------------------------+-------------------------------------------------------------------------+
|          |     | uint64_t                           | Read count                                                              |
|          |     +------------------------------------+-------------------------------------------------------------------------+
|          |     | **Repeating [0..read count]**                                                                                |
+          |     +---+--------------------------------+-------------------------------------------------------------------------+
| | Public |     |   | | uint64_t                     | | Key length                                                            |
| | Domain |     |   | | K                            | | Key                                                                   |
|          |     |   | | Ver                          | | Version                                                               |
+          +-----+---+--------------------------------+-------------------------------------------------------------------------+
|          |     | uint64_t                           | Write count                                                             |
+          |     +------------------------------------+-------------------------------------------------------------------------+
|          |     | **Repeating [0..write count]**                                                                               |
+          |     +---+--------------------------------+-------------------------------------------------------------------------+
|          |     |   | | uint64_t                     | | Key length                                                            |
|          |     |   | | K                            | | Key                                                                   |
|          |     |   | | uint64_t                     | | Value length                                                          |
|          |     |   | | V                            | | Value                                                                 |
+          +-----+---+--------------------------------+-------------------------------------------------------------------------+
|          |     | | uint64_t                         | Remove count                                                            |
+          +     +------------------------------------+-------------------------------------------------------------------------+
|          |     | **Repeating [0..remove count]**                                                                              |
+          +     +---+--------------------------------+-------------------------------------------------------------------------+
|          |     |   | | uint64_t                     | | Key length                                                            |
|          |     |   | | K                            | | Key                                                                   |
+----------+-----+---+--------------------------------+-------------------------------------------------------------------------+
| | Private| **Optional**                                                                                                       |
| | Domain | | Encrypted serialised private domain blob.                                                                        |
+----------+--------------------------------------------------------------------------------------------------------------------+

Transaction Encryption
----------------------

Each entry in the ledger corresponds to a transaction committed by the primary node.

When a transaction is committed, each ``Store::Map`` containing writes is serialised in different security domains (i.e. public or private), based on the name of the Map when it was created (default is private). A public ``Store::Map`` (i.e. one whose name starts with "public:") is serialised and stored in the ledger as plaintext while a private ``Store::Map`` is serialised and encrypted before being stored.

Ledger entries are integrity-protected and encrypted using a symmetric key shared by all trusted nodes (see :doc:`/architecture/cryptography`). This key is kept secure inside each enclave. See :ref:`governance/common_member_operations:Rekeying Ledger` for details on how members can rotate the ledger encryption key.

Note that even if a transaction only writes to a private ``Store::Map``, unencrypted information such as the sequence number is always present in the serialised entry.