Verifying Transactions

Checking for Commit

Because of the decentralised nature of CCF, a request is committed to the ledger only once a number of nodes have agreed on that request.

To guarantee that their request is successfully committed to the ledger, a user should issue a GET /app/tx request, specifying the version received in the response. This version is constructed from a view and a sequence number.

$ curl -X GET "https://<ccf-node-address>/app/tx?transaction_id=2.18" --cacert service_cert.pem --key user0_privk.pem --cert user0_cert.pem -i
HTTP/1.1 200 OK
content-length: 23
content-type: application/json
x-ms-ccf-transaction-id: 5.42

{"status":"Committed"}

This example queries the status of Transaction ID 2.18 (constructed from view 2 and sequence number 18). The response indicates this was successfully committed. The headers also show that the service has since made progress with other requests and changed view (x-ms-ccf-transaction-id: 5.42).

The possible statuses returned by GET /app/tx are:

  • Unknown - this node has not received a transaction with the given ID

  • Pending - this node has received a transaction with the given ID, but does not yet know if the transaction has been committed

  • Committed - this node knows that this transaction is committed, it is an irrevocable and durable part of the service’s transaction history

  • Invalid - this node knows that the given transaction cannot be committed. This occurs when the view changes, and some pending transactions may be lost and must be resubmitted, but also applies to IDs which are known to be impossible given the current committed IDs

On a given node, the possible transitions between states are described in the following diagram:

        stateDiagram
    Unknown --> Pending
    Pending --> Unknown
    Pending --> Committed
    Pending --> Invalid

It is possible that intermediate states are not visible (e.g. a transition from Unknown to Committed may never publically show a Pending result). Nodes may disagree on the current state due to communication delays, but will never disagree on transitions (in other words, they may believe a COMMITTED transaction is still Unknown or Pending, but will never report it as Invalid). A transition from Pending to Unknown can only occur immediately after an election, while the node is confirming where the new view starts, and will usually resolve to Committed or Pending quickly afterwards.

Note that transaction IDs are uniquely assigned by the service - once a request has been assigned an ID, this ID will never be associated with a different write transaction. In normal operation, the next requests will be given versions 2.19, then 2.20, and so on, and after a short delay 2.18 will be committed. If requests are submitted in parallel, they will be applied in a consistent order indicated by their assigned versions.

If the network is unable to reach consensus, it will trigger a leadership election which increments the view. In this case the user’s next request may be given a version 3.16, followed by 3.17, then 3.18. The sequence number is reused, but in a different view; the service knows that 2.18 can never be assigned, so it can report this as an invalid ID. Read-only transactions are an exception - they do not get a unique Transaction ID but instead return the ID of the last write transaction whose state they may have read.

Write Receipts

Once a transaction has been committed, it is possible to get a cryptographic receipt over the entry produced in the ledger. That receipt can be verified offline.

To obtain a receipt, a user needs to call a GET /app/receipt for a particular Transaction ID. Because fetching the information necessary to produce a receipt likely involves a round trip to the ledger, the endpoint is implemented as a historical query. This means that the request may return 202 Accepted at first, with a suggested Retry-After header. A subsequent call will return the actual receipt, for example:

$ curl -X GET "https://<ccf-node-address>/app/receipt?transaction_id=2.643" --cacert service_cert.pem --key user0_privk.pem --cert user0_cert.pem

{'cert': '-----BEGIN CERTIFICATE-----\n'
        'MIIBzzCCAVWgAwIBAgIRANoNTgf6hwugwIZgb/vrSXcwCgYIKoZIzj0EAwMwFjEU\n'
        'MBIGA1UEAwwLQ0NGIE5ldHdvcmswHhcNMjIwNDI1MTUwMTMzWhcNMjIwNDI2MTUw\n'
        'MTMyWjATMREwDwYDVQQDDAhDQ0YgTm9kZTB2MBAGByqGSM49AgEGBSuBBAAiA2IA\n'
        'BCbH0j1tcyHeyBVxThuzwyk6IesLFe6CXBqOYBbEvrLCrdR+wj400UR0xWBcVroY\n'
        '8GqF+tLS8YQPt+kepG82lZuRjAbaBn2BL8g6cdKGOew5fo9LCL9+NlH+W6bz83E/\n'
        'Z6NqMGgwCQYDVR0TBAIwADAdBgNVHQ4EFgQUlMCr5/1LIzGegZPJfHwzFl5GCwUw\n'
        'HwYDVR0jBBgwFoAUuzKYZ+UZum8NaytcWRfirw8TA+MwGwYDVR0RBBQwEocEfwAA\n'
        'AYcEf7i1MYcEfwAAAjAKBggqhkjOPQQDAwNoADBlAjEAyLPRWCR4Nn0Z3q/1rKCc\n'
        'ppiSwPrMQ/asHoUAUgmnAGkxgJ6A8iGJXa/WFal/PDtdAjBfp+AIkuVcBjmWfLxF\n'
        'MKN7njz6Q9m10APn+KigsvS4aIRbHZHbYTkAOyFEdipm5gA=\n'
        '-----END CERTIFICATE-----\n',
'leaf_components': {'claims_digest': '0000000000000000000000000000000000000000000000000000000000000000',
                    'commit_evidence': 'ce:2.500:a4e526cef8d1155e8a8c5f4c0dd8d1220c924cea620e1cc737c4039c8abc8d6b',
                    'write_set_digest': '56f1f53e61a80606fec984cbfadaf833b3bb2e68f464f3b7c641e163fcb1f647'},
'node_id': 'c0718b67b986748d074a293201fa2f9bc3a29e07d204ccc24167afdc59bdf0c3',
'proof': [{'left': '26e7edf1065e23a387b8d6fd3743b4885aae1919c4c347d73b792e3e77b57695'},
        {'left': '76c79df9146479c0aebdd51f228e1d304d60f382f61571a729f2fba7d93ce2a7'},
        {'left': '6b54d109e3f474f9f546d5d462de40440ca335b133d3899c228532fcc535e732'},
        {'left': 'bf3e529174209df204342224fc39ffee68189c30db03e53ba1d4baa053e02471'},
        {'left': '740dcdb82afe5f2b311056c96bd6afcc1b571e1cdb8e918bfe24eda48213eb1a'},
        {'left': '05efd90dfd3bf86a3efa185c6a869342cb0d9af4dd92256d37dc826b86e99684'}],
'signature': 'MGYCMQCI1yY5xhGNFXyKjO6mFhxZCjAsuTsquO1agIzKkxh/Uf9Em+P7AwnxuQ2uC7WU++QCMQCSzpE9k8xkSku5BlkUelDPBHxLuokLIpKAyn73HXtXiz+zXOVWIZkYftUEoQ31pG0='}

cert contains the certificate of the signing node, endorsed by the Service Identity. node_id is the node’s ID inside CCF, a digest of its public key.

Note that receipts over signature transactions are a special case, for example:

$ curl -X GET "https://<ccf-node-address>/app/receipt?transaction_id=2.35" --cacert service_cert.pem --key user0_privk.pem --cert user0_cert.pem

{'leaf': 'fdc977c49d3a8bdf986176984e9432a09b5f6fe0c04e0b1c2dd177c03fdca9ec',
 'node_id': '06fef62c80b6471c7005c1b114166fd1b0e077845f5ad544ad4eea4fb1d31f78',
 'proof': [],
 'signature': 'MGQCMACklXqd0ge+gBS8WzewrwtwzRzSKy+bfrLZVx0YHmQvtsqs7dExYESsqrUrB8ZcKwIwS3NPKaGq0w2QlPlCqUC3vQoQvhcZgPHPu2GkFYa7JEOdSKLknNPHaCRv80zx2RGF',
 'cert': '<PEM string>'}

The proof is empty, and the leaf field is set to the value being signed, which is the root of the Merkle Tree covering all transactions until the signature. This allows writing verification code that handles both regular and signature receipts similarly, but it is worth noting that the ‘leaf’ value for signatures is not the digest of the signature transaction itself.

From version 2.0, CCF also includes endorsement certificates for previous service identities, by the current Service Identity, in service_endorsements. Thus, after at least one recovery, the endorsement check now takes the form of a certificate chain verification instead of a single endorsement check.

Receipt Verification

Verifying a receipt consists of the following steps:

  1. Digest commit_evidence to produce commit_evidence_digest and claims to produce claims_digest when applicable.

  2. If the receipt contains leaf_components, digest the concatenation write_set_digest + commit_evidence_digest + claims_digest to produce leaf.

  3. Combine leaf with the successive elements in proof to calculate the value of root. See ccf.receipt.root() for a reference implementation.

  4. Verify signature over the root using the certificate of the node identified by node_id and cert. See ccf.receipt.verify() for a reference implementation.

  5. Check that the certificate cert of node_id used to sign the receipt is endorsed by the CCF network. See ccf.receipt.check_endorsements() for a reference implementation.

Note that since a receipt is a committment by a service to a transaction, a verifier must know the Service Identity, and provide it as an input to step 5.

Application Claims

CCF allows application code to attach arbitrary claims to a transaction, via the enclave::RpcContext::set_claims_digest() API, as illustrated in User-Defined Claims in Receipts.

This is useful to allow the reveal and verification of application-related claims offline, ie. without access to the CCF network. For example, a logging application may choose to set the digest of the payload being logged as claims_digest. A user who logs a payload can then present the receipt and the payload to a third party, who can confirm that they match, having verified the receipt. They can perform this verification without access to the service.

Multiple claims can be registered by storing them in a collection or object whose digest is set as claims_digest. It is possible to reveal them selectively, by capturing their digest in turn, rather than their raw value directly, eg:

claims_digest = hash( hash(claim_a) + hash(claim_b) )

Revealing hash(claim_a) and claim_b allows verification without revealing claim_a in this case.

Although CCF takes the approach of concatenating leaf components to keep its implementation simple and format-agnostic, an application may choose to encode its claims in a structured way for convenience, for example as JSON, CBOR etc.

Applications may wish to expose dedicated endpoints, besides CCF’s built-in GET /node/receipt, in which they can selectively expand claims, as illustrated in User-Defined Claims in Receipts. If some claims must stay confidential, applications should encrypt them rather than merely digest them. They key can be kept in a private table for example, which like the claim will be available through the historical query API. The application logic can then decide whether to decrypt the claim for the caller depending on its authorisation policy.

Commit Evidence

The commit_evidence field in receipts fulfills two purposes:

  1. It exposes the full Transaction ID in a format that is easy for a user to extract, and does not require parsing the ledger entry.

  2. Because it cannot be extracted from the ledger without access to the ledger secrets, it guarantees the transaction is committed.

Entries are written out to the ledger as early as possible, to relieve memory pressure inside the enclave. If receipts could be produced from these entries regardless of their replication status, a malicious actor could emit them for transactions that have been tentatively run by a primary, appended to its local ledger, but since rolled back. By including a committment to the digest of commit_evidence as a leaf component in the Merkle Tree, which is effectively a nonce derived from ledger secrets and TxID, we ensure that only receipts produced by nodes that can reveal this nonce are verifiable.