Python Library

This page describes the Python API of the ccf.ledger module which can be used by auditors to parse a CCF ledger.

The latest version of the CCF Python tools package is available on PyPi and can be installed as follows:

$ pip install ccf

Tip

Use the read_ledger.py command line utility to display the content of a CCF ledger from the command line.

Tutorial

This tutorial demonstrates how to parse the ledger produced by a CCF node. It shows a very basic example which loops through all transactions in the ledger and displays the content of all updates to a well-known built-in table.

First, the paths to the ledger directories should be set:

ledger_dirs = ["</path/to/ledger/dir>"] # List of a single ledger directory

Note

By default, ledger directories are created under the node directory (See operations/configuration:``ledger`` configuration entry).

Then, import the ledger module and instantiate a ccf.ledger object:

ledger = ccf.ledger.Ledger(ledger_dirs, committed_only=False)

By default, non-committed ledger files are ignored, unless the committed_only argument is set to False.

In this particular example, a target table is set. This is a public table that can be read and audited from the ledger directly. In this example, the target table is the well-known public:ccf.gov.nodes.info table that keeps track of all nodes in the network.

target_table = "public:ccf.gov.nodes.info"

Note

In practice, it is likely that auditors will want to run more elaborate logic when parsing the ledger. For example, this might involve auditing governance operations and looping over multiple tables.

Finally, the ledger can be iterated over. For each transaction in the ledger, the public tables changed within that transaction are observed. If the target table is included, we loop through all keys and values modified in that transaction.

for chunk in ledger:
    for transaction in chunk:
        # Retrieve all public tables changed in transaction
        public_tables = transaction.get_public_domain().get_tables()

        if target_table in public_tables:
            # Ledger verification is happening implicitly in ccf.ledger.Ledger()
            for key, value in public_tables[target_table].items():
                # Note: `key` and `value` are raw bytes here.
                # This code needs to have knowledge of the serialisation format for each table.
                # In this case, the target table 'public:ccf.gov.nodes.info' is raw bytes to JSON.
                LOG.info(f"{key.decode()} : {json.loads(value)}")

Tip

The integrity of the ledger is automatically verified when iterating over transactions.

read_ledger.py command line utility

The read_ledger.py command line utility can be used to parse, verify the integrity and display the public content of the ledger directory:

$ read_ledger.py /path/to/ledger/dir
Reading ledger from ['/path/to/ledger/dir']
Contains 9 chunks
chunk /path/to/first/ledger/chunk (committed)
  seqno 1 (9 public tables) # Number of tables written at seqno 1
    table "public:ccf.gov.constitution" (1 write): # 1 write to this table at seqno 1
      <u64: 0>: # key
        "class Action ..." # value
    table "public:ccf.gov.members.acks" (2 writes): # 2 writes to this table at seqno 1
        ...
  seqno 31 (0 public tables) # transaction at seqno 31 is private only
  -- private: 94 bytes # size of private entry
...
Ledger verification complete. Found 15 signatures, and verified until 2.52 # Ledger integrity verified up to seqno 2.52

By default, non-committed ledger files are ignored, unless the --uncommitted command line argument is specified.

Alternatively, read_ledger.py can parse the content of a snapshot file:

$ read_ledger.py --snapshot /path/to/snapshot/file
Reading snapshot from /path/to/snapshot/file (committed)
  seqno 12 (15 public tables)
...

Note

The output of the read_ledger.py utility can be filtered by key-value store table using the --tables <regex> arguments, e.g.:

$ read_ledger.py /path/to/ledger/dir --tables ".*ccf.gov.proposals.*"
# Only displays governance proposal related tables

Example

An example of how to read and verify entries on the ledger can be found in governance_history.py, which verifies the member voting history for a short-lived service.

Since every vote request is signed by the voting member, verified by the primary node and then stored on the ledger, the test performs the following (this sequence of operations is performed sequentially per transaction):

1. Read and store the member certificates

2. Read an entry from the public:ccf.gov.history table (each entry in the table contains the member id of the voting member, along with their latest signed request)

3. Create a public key using the certificate of the voting member (which was stored on step 1)

4. Verify the signature using the public key and the raw request

5. Repeat steps 2 - 4 until all voting history entries have been read

ledger_code.py command line utility

The ledger_code.py command line utility can be used to display the history of code changes in a ledger:

$ ledger_code.py -s /path/to/ledger/dir
2.1 Introduced : [('OE_SGX_v1', '3258fb')]
2.103 Introduced : [('OE_SGX_v1', '048b8c')]
3.175 Removed : [('OE_SGX_v1', '3258fb')]

$ ledger_code.py -vs /path/to/ledger/dir
2.1: {('OE_SGX_v1', '3258fb'): ['9b013f']}
2.3: {('OE_SGX_v1', '3258fb'): ['9b013f', 'beb114']}
2.5: {('OE_SGX_v1', '3258fb'): ['9b013f', 'beb114', '6ff2f4']}
2.103: {('OE_SGX_v1', '3258fb'): ['9b013f', 'beb114', '6ff2f4'], ('OE_SGX_v1', '048b8c'): ['3ebd58']}
2.119: {('OE_SGX_v1', '3258fb'): ['9b013f', 'beb114', '6ff2f4'], ('OE_SGX_v1', '048b8c'): ['3ebd58', '53cb2a']}
2.146: {('OE_SGX_v1', '3258fb'): ['9b013f', 'beb114', '6ff2f4'], ('OE_SGX_v1', '048b8c'): ['3ebd58', '53cb2a', '1d1396']}
2.152: {('OE_SGX_v1', '3258fb'): ['beb114', '6ff2f4'], ('OE_SGX_v1', '048b8c'): ['3ebd58', '53cb2a', '1d1396']}
3.156: {('OE_SGX_v1', '3258fb'): ['beb114', '6ff2f4'], ('OE_SGX_v1', '048b8c'): ['3ebd58', '53cb2a', '1d1396']}
3.165: {('OE_SGX_v1', '3258fb'): ['6ff2f4'], ('OE_SGX_v1', '048b8c'): ['3ebd58', '53cb2a', '1d1396']}
3.168: {('OE_SGX_v1', '3258fb'): ['6ff2f4'], ('OE_SGX_v1', '048b8c'): ['3ebd58', '53cb2a', '1d1396']}
3.175: {('OE_SGX_v1', '048b8c'): ['3ebd58', '53cb2a', '1d1396']}
4.179: {('OE_SGX_v1', '048b8c'): ['3ebd58', '53cb2a', '1d1396']}

As with read_ledger.py, non-committed ledger files are ignored, unless the --uncommitted command line argument is specified.

API

class ccf.ledger.Ledger(paths: List[str], committed_only: bool = True, read_recovery_files: bool = False, validator: LedgerValidator | None = None)

Class used to iterate over all ccf.ledger.LedgerChunk stored in a CCF ledger folder.

Parameters:

name (str) – Ledger directory for a single CCF node.

get_transaction(seqno: int) → Transaction

Return the ccf.ledger.Transaction recorded in the ledger at the given sequence number.

Note that the transaction returned may not yet be verified by a signature transaction nor committed by the service.

Parameters:

seqno (int) – Sequence number of the transaction to fetch.

Returns:

ccf.ledger.Transaction

get_latest_public_state() → Tuple[dict, int]

Return the current public state of the service.

Note that the public state returned may not yet be verified by a signature transaction nor committed by the service.

Returns:

Tuple[Dict, int]: Tuple containing a dictionary of public tables and their values and the seqno of the state read from the ledger.

class ccf.ledger.LedgerChunk(name: str, ledger_validator: LedgerValidator | None = None)

Class used to parse and iterate over ccf.ledger.Transaction in a CCF ledger chunk.

Parameters:

• name (str) – Name for a single ledger chunk.

• ledger_validator (LedgerValidator) – LedgerValidator instance used to verify ledger integrity.

class ccf.ledger.Transaction(filename: str, ledger_validator: LedgerValidator | None = None)

A transaction represents one entry in the CCF ledger.

get_raw_tx() → bytes

Return raw transaction bytes.

Returns:

Raw transaction bytes.

get_private_domain_size() → int

Retrieve the size of the private (i.e. encrypted) domain for that transaction.

get_public_domain() → PublicDomain

Retrieve the public (i.e. non-encrypted) domain for that entry.

Note: Even if the entry is private-only, an empty ccf.ledger.PublicDomain object is returned.

Returns:

ccf.ledger.PublicDomain

class ccf.ledger.Snapshot(filename: str)

Utility used to parse the content of a snapshot file.

get_private_domain_size() → int

Retrieve the size of the private (i.e. encrypted) domain for that transaction.

get_public_domain() → PublicDomain

Retrieve the public (i.e. non-encrypted) domain for that entry.

Note: Even if the entry is private-only, an empty ccf.ledger.PublicDomain object is returned.

Returns:

ccf.ledger.PublicDomain

class ccf.ledger.PublicDomain(buffer: BytesIO)

All public tables within a ccf.ledger.Transaction.

get_tables() → dict

Return a dictionary of all public tables (with their content) in a ccf.ledger.Transaction.

Returns:

Dictionary of public tables with their content.

get_seqno() → int

Return the sequence number at which the transaction was recorded in the ledger.

get_claims_digest() → bytes | None

Return the claims digest when there is one

get_commit_evidence_digest() → bytes | None

Return the commit evidence digest when there is one

ccf.receipt.root(leaf: str, proof: List[dict])

Recompute root of Merkle tree from a leaf and a proof of the form: [{“left”: digest}, {“right”: digest}, …]

ccf.receipt.verify(root: str, signature: str, cert: Certificate)

Verify signature over root of Merkle Tree

ccf.receipt.check_endorsement(endorsee: Certificate, endorser: Certificate)

Check endorser has endorsed endorsee

ccf.receipt.check_endorsements(node_cert: Certificate, service_cert: Certificate, endorsements: List[Certificate])

Check a node certificate is endorsed by a service certificate, transitively through a list of endorsements.