Developer API

A CCF application is composed of the following:

Application Entry Point

std::unique_ptr<ccf::endpoints::EndpointRegistry> ccf::make_user_endpoints(ccf::AbstractNodeContext &context)

To be implemented by the application. Creates a collection of endpoints which will be exposed to callers under /app.

Parameters:

context – Access to node and host services

Returns:

Unique pointer to the endpoint registry instance

Application Endpoint Registration

struct Endpoint : public ccf::endpoints::EndpointDefinition

An Endpoint represents a user-defined resource that can be invoked by authorised users via HTTP requests, over TLS. An Endpoint is accessible at a specific verb and URI, e.g. POST /app/accounts or GET /app/records.

Endpoints can read from and mutate the state of the replicated key-value store.

A CCF application is a collection of Endpoints recorded in the application’s EndpointRegistry.

Subclassed by ccf::endpoints::PathTemplatedEndpoint, ccf::js::CustomJSEndpoint

Public Functions

Endpoint &set_openapi_description(const std::string &description)

Set the OpenAPI description for the endpoint.

Returns:

This Endpoint for further modification

Endpoint &set_openapi_summary(const std::string &summary)

Set the OpenAPI summary for the endpoint.

Returns:

This Endpoint for further modification

Endpoint &set_openapi_deprecated(bool is_deprecated)

Set the endpoint as deprecated.

Returns:

This Endpoint for further modification

Endpoint &set_openapi_deprecated_replaced(const std::string &deprecation_version, const std::string &replacement)

Set the endpoint as deprecated and overwrites the description to include deprecation version and point to a replacement endpoint.

Returns:

This Endpoint for further modification

Endpoint &set_openapi_hidden(bool hidden)

Whether the endpoint should be omitted from the OpenAPI document.

Returns:

This Endpoint for further modification

Endpoint &set_params_schema(const nlohmann::json &j)

Sets the JSON schema that the request parameters must comply with.

Parameters:

j – Request parameters JSON schema

Returns:

This Endpoint for further modification

Endpoint &set_result_schema(const nlohmann::json &j, std::optional<http_status> status = std::nullopt)

Sets the JSON schema that the request response must comply with.

Parameters:

  • j – Request response JSON schema

  • status – Request response status code

Returns:

This Endpoint for further modification

template<typename In, typename Out>
inline Endpoint &set_auto_schema(std::optional<http_status> status = std::nullopt)

Sets the schema that the request and response bodies should comply with. These are used to populate the generated OpenAPI document, but do not introduce any constraints on the actual types that are parsed or produced by the handling functor.

Note

See DECLARE_JSON_ serialisation macros for serialising user-defined data structures.

Template Parameters:

  • In – Request body JSON-serialisable data structure

  • Out – Response body JSON-serialisable data structure

Parameters:

status – Response status code

Returns:

This Endpoint for further modification

template<typename T>
inline Endpoint &set_auto_schema(std::optional<http_status> status = std::nullopt)

Sets schemas for request and response bodies using typedefs within T.

See also

set_auto_schema

Note

T data structure should contain two nested In and Out structures for request parameters and response format, respectively.

Template Parameters:

T – Type containing In and Out typedefs with JSON-schema description specialisations

Parameters:

status – Request response status code

Returns:

This Endpoint for further modification

template<typename T>
inline Endpoint &add_query_parameter(const std::string &param_name, QueryParamPresence presence = RequiredParameter)

Add OpenAPI documentation for a query parameter which can be passed to this endpoint.

Template Parameters:

T – Type with appropriate ds::json specialisations to generate a JSON schema description

Parameters:

  • param_name – Name to be used for the query parameter to this Endpoint

  • presence – Enum value indicating whether this parameter is required or optional

Returns:

This Endpoint for further modification

Endpoint &set_forwarding_required(ForwardingRequired fr)

Overrides whether a Endpoint is always forwarded, or whether it is safe to sometimes execute on followers.

Parameters:

fr – Enum value with desired status

Returns:

This Endpoint for further modification

struct Installer

Subclassed by ccf::endpoints::EndpointRegistry

class EndpointRegistry : public ccf::endpoints::Endpoint::Installer

The EndpointRegistry records the user-defined endpoints for a given CCF application.

This is the abstract base for several more complete registrys. For a versioned API wrapping access to common CCF properties, see BaseEndpointRegistry. For implementation of several common endpoints see CommonEndpointRegistry.

Subclassed by ccf::BaseEndpointRegistry

Public Functions

virtual void install(Endpoint &endpoint) override

Install the given endpoint, using its method and verb

If an implementation is already installed for this method and verb, it will be replaced.

Parameters:

endpoint – Endpoint object describing the new resource to install

void set_default(EndpointFunction f, const AuthnPolicies &ap)

Set a default EndpointFunction

The default EndpointFunction is only invoked if no specific EndpointFunction was found.

Parameters:

  • f – Method implementation

  • ap – Authentication policy

class CommonEndpointRegistry : public ccf::BaseEndpointRegistry

Subclassed by ccf::ACMERpcEndpoints, ccf::GovEndpointRegistry, ccf::NodeEndpoints, ccf::UserEndpointRegistry

struct EndpointMetricsEntry

Public Members

std::string path

Endpoint path.

std::string method

Endpoint method.

size_t calls = 0

Number of calls since node start.

size_t errors = 0

Number of errors (4xx) since node start.

size_t failures = 0

Number of failures (5xx) since node start.

size_t retries = 0

Number of transaction retries caused by conflicts since node start

struct EndpointMetrics

Public Members

std::vector<EndpointMetricsEntry> metrics

Metrics for all endpoints in the frontend.

class BaseEndpointRegistry : public ccf::endpoints::EndpointRegistry

Extends the basic ccf::endpoints::EndpointRegistry with helper API methods for retrieving core CCF properties.

The API methods are versioned with a _vN suffix. App developers should use the latest version which provides the values they need. Note that the N in these versions is specific to each method name, and is not related to a specific CCF release version. These APIs will be stable and supported for several CCF releases.

The methods have a consistent calling pattern, taking their arguments first and setting results to the later out-parameters, passed by reference. All return an ApiResult, with value OK if the call succeeded.

Subclassed by ccf::CommonEndpointRegistry, nobuiltins::NoBuiltinsRegistry

Public Functions

ApiResult get_view_history_v1(std::vector<ccf::TxID> &history, ccf::View since = 1)

Get the history of the consensus view changes.

Returns the history of view changes since the given view, which defaults to the start of time.

A view change is characterised by the first sequence number in the new view.

ApiResult get_view_history_v2(std::vector<ccf::TxID> &history, ccf::View since, ccf::InvalidArgsReason &reason)

Get the history of the consensus view changes.

Returns the history of view changes since the given view, which defaults to the start of time.

A view change is characterised by the first sequence number in the new view.

ApiResult get_status_for_txid_v1(ccf::View view, ccf::SeqNo seqno, ccf::TxStatus &tx_status)

Get the status of a transaction by ID, provided as a view+seqno pair.

Note that this value is the node’s local understanding of the status of that transaction in the network at call time. For a given TxID, the initial status is always UNKNOWN, and eventually becomes COMMITTED or INVALID. See the documentation section titled “Verifying Transactions” for more detail. 

    UNKNOWN [Initial status]
     v  ^
   PENDING
   v     v
COMMITTED INVALID [Final statuses]

This status is not sampled atomically per handler: if this is called multiple times in a transaction handler, later calls may see more up to date values than earlier calls. Once a final state (COMMITTED or INVALID) has been reached, no further changes are possible.

See also

ccf::TxStatus

ApiResult get_last_committed_txid_v1(ccf::View &view, ccf::SeqNo &seqno)

Get the ID of latest transaction known to be committed.

ApiResult generate_openapi_document_v1(ccf::kv::ReadOnlyTx &tx, const std::string &title, const std::string &description, const std::string &document_version, nlohmann::json &document)

Generate an OpenAPI document describing the currently installed endpoints.

The document is compatible with OpenAPI version 3.0.0 - the _v1 suffix describes the version of this API, not the returned document format. Similarly, the document_version argument should be used to version the returned document itself as the set of endpoints or their APIs change, it does not affect the OpenAPI version of the format of the document.

ApiResult get_quote_for_this_node_v1(ccf::kv::ReadOnlyTx &tx, QuoteInfo &quote_info)

Get a quote attesting to the hardware this node is running on.

ApiResult get_id_for_this_node_v1(NodeId &node_id)

Get the id of the currently executing node.

ApiResult get_quotes_for_all_trusted_nodes_v1(ccf::kv::ReadOnlyTx &tx, std::map<NodeId, QuoteInfo> &quotes)

Get quotes attesting to the hardware that each node in the service is running on.

ApiResult get_view_for_seqno_v1(ccf::SeqNo seqno, ccf::View &view)

Get the view associated with a given seqno, to construct a valid TxID.

ApiResult get_user_data_v1(ccf::kv::ReadOnlyTx &tx, const UserId &user_id, nlohmann::json &user_data)

Get the user data associated with a given user id.

ApiResult get_member_data_v1(ccf::kv::ReadOnlyTx &tx, const MemberId &member_id, nlohmann::json &member_data)

Get the member data associated with a given member id.

ApiResult get_user_cert_v1(ccf::kv::ReadOnlyTx &tx, const UserId &user_id, ccf::crypto::Pem &user_cert_pem)

Get the certificate (PEM) of a given user id.

ApiResult get_member_cert_v1(ccf::kv::ReadOnlyTx &tx, const MemberId &member_id, ccf::crypto::Pem &member_cert_pem)

Get the certificate (PEM) of a given member id.

ApiResult get_untrusted_host_time_v1(::timespec &time)

Get untrusted time from the host of the currently executing node.

RPC Context

class RpcContext

Describes the currently executing RPC.

Subclassed by ccf::RpcContextImpl

Public Functions

virtual std::shared_ptr<SessionContext> get_session_context() const = 0

Return information about the persistent session which this request was received on. Allows correlation between multiple requests coming from the same long-lived session.

virtual const PathParams &get_request_path_params() = 0

Returns a map of all PathParams parsed out of the original query path. For instance if this endpoint was installed at /foo/{name}/{age}, and the request path /foo/bob/42, this would return the map: {“name”: “bob”, “age”: “42”}

virtual const http::HeaderMap &get_request_headers() const = 0

Returns map of all headers found in the request.

virtual std::optional<std::string> get_request_header(const std::string_view &name) const = 0

Returns value associated with named header, or nullopt of this header was not present.

virtual const std::string &get_request_url() const = 0

Returns full URL provided in request, rather than split into path + query.

virtual void set_claims_digest(ccf::ClaimsDigest::Digest &&digest) = 0

Sets the application claims digest associated with this transaction. This digest is used to construct the Merkle tree leaf representing this transaction. This allows a transaction to make specific, separately-revealable claims in each transaction, without being bound to the transaction serialisation format or what is stored in the KV. The digest will be included in receipts issued for that transaction.

Authentication

Policies

static std::shared_ptr<EmptyAuthnPolicy> ccf::empty_auth_policy = std::make_shared<EmptyAuthnPolicy>()

Perform no authentication

static std::shared_ptr<UserCertAuthnPolicy> ccf::user_cert_auth_policy = std::make_shared<UserCertAuthnPolicy>()

Authenticate using TLS session identity, and public:ccf.gov.users.certs table

static std::shared_ptr<MemberCertAuthnPolicy> ccf::member_cert_auth_policy = std::make_shared<MemberCertAuthnPolicy>()

Authenticate using TLS session identity, and public:ccf.gov.members.certs table

static std::shared_ptr<MemberCOSESign1AuthnPolicy> ccf::member_cose_sign1_auth_policy = std::make_shared<MemberCOSESign1AuthnPolicy>()

Authenticate using COSE Sign1 payloads, and public:ccf.gov.members.certs table

static std::shared_ptr<UserCOSESign1AuthnPolicy> ccf::user_cose_sign1_auth_policy = std::make_shared<UserCOSESign1AuthnPolicy>()

Authenticate using COSE Sign1 payloads, and public:ccf.gov.users.certs table

static std::shared_ptr<JwtAuthnPolicy> ccf::jwt_auth_policy = std::make_shared<JwtAuthnPolicy>()

Authenticate using JWT, validating the token using the public:ccf.gov.jwt.public_signing_keys_metadata table

class TypedUserCOSESign1AuthnPolicy : public ccf::UserCOSESign1AuthnPolicy

Typed User COSE Sign1 Authentication Policy

Extends UserCOSESign1AuthPolicy, to require that a specific message type is present in the corresponding protected header.

Identities

struct UserCertAuthnIdentity : public ccf::AuthnIdentity

Public Members

UserId user_id

CCF user ID

struct MemberCertAuthnIdentity : public ccf::AuthnIdentity

Public Members

MemberId member_id

CCF member ID

struct UserCOSESign1AuthnIdentity : public ccf::COSESign1AuthnIdentity

Public Members

UserId user_id

CCF user ID

ccf::crypto::Pem user_cert

User certificate, used to sign this request, described by keyId

TimestampedProtectedHeader protected_header

COSE Protected Header

struct MemberCOSESign1AuthnIdentity : public ccf::COSESign1AuthnIdentity

Public Members

MemberId member_id

CCF member ID

ccf::crypto::Pem member_cert

Member certificate, used to sign this request, described by keyId

GovernanceProtectedHeader protected_header

COSE Protected Header

struct JwtAuthnIdentity : public ccf::AuthnIdentity

Public Members

std::string key_issuer

JWT key issuer, as defined in public:ccf.gov.jwt_public_signing_key_issuer

nlohmann::json header

JWT header

nlohmann::json payload

JWT payload

Supporting Types

enum class ccf::TxStatus

Describes the status of a transaction, as seen by this node.

Values:

enumerator Unknown

This node has not received this transaction, and knows nothing about it

enumerator Pending

This node has this transaction locally, but has not yet heard that the transaction has been committed by the distributed consensus

enumerator Committed

This node has seen that this transaction is committed, it is an irrevocable and durable part of the service’s transaction history

enumerator Invalid

This node knows that the given transaction cannot be committed. This may mean there has been a view change, and a previously pending transaction has been lost (the original request should be resubmitted and will be given a new Transaction ID). This also describes IDs which are known to be impossible given the currently committed IDs

using ccf::View = uint64_t

Transactions occur within a fixed View. Each View generally spans a range of transactions, though empty Views are also possible. The View is advanced by the consensus protocol during election of a new leader, and a single leader is assigned in each View. View and Term are synonymous.

using ccf::SeqNo = uint64_t

Each transaction is assigned a unique incrementing SeqNo, maintained across View transitions. This matches the order in which transactions are applied, where a higher SeqNo means that a transaction executed later. SeqNos are unique during normal operation, but around elections it is possible for distinct transactions in separate Views to have the same SeqNo. Only one of these transactions will ever commit, and the others are ephemeral.

struct TxID
enum class ccf::ApiResult

Lists the possible return codes from the versioned APIs in ccf::BaseEndpointRegistry

Values:

enumerator OK

Call was successful, results can be used

enumerator Uninitialised

The node is not yet initialised, and doesn’t have access to the service needed to answer this call. Should only be returned if the API is called too early, during node construction.

enumerator InvalidArgs

One of the arguments passed to the function is invalid. It may be outside the range of known values, or not be in the expected format.

enumerator NotFound

The requsted value was not found.

enumerator InternalError

General error not covered by the cases above. Generally means that an unexpected exception was thrown during execution.

Historical Queries

ccf::endpoints::EndpointFunction ccf::historical::adapter_v3(const HandleHistoricalQuery &f, ccf::AbstractNodeContext &node_context, const CheckHistoricalTxStatus &available, const TxIDExtractor &extractor = txid_from_header)
class AbstractStateCache : public ccf::AbstractNodeSubSystem

Stores the progress of historical query requests.

A request will generally need to be made multiple times (with the same handle and range) before the response is available, as the historical state is asynchronously retrieved from the ledger and then validated. If the same handle is used for a new range, the state for the old range will be discarded. State is also discarded after the handle’s expiry duration, or when drop_cached_states is called for a given handle. The management of requests (how many unique handles are concurrently active, how they are correlated across HTTP requests, how the active quota is divided between callers) is left to the calling system.

Subclassed by ccf::historical::StateCache

Public Functions

virtual void set_default_expiry_duration(ExpiryDuration seconds_until_expiry) = 0

Set the default time after which a request’s state will be deleted, and will not be accessible without retrieving it again from the ledger. Any call to get_store_XXX which doesn’t pass an explicit seconds_until_expiry will reset the timer to this default duration.

virtual void set_soft_cache_limit(CacheSize cache_limit) = 0

Set the cache limit (in bytes) to evict least recently used requests from the cache after its size grows beyond this limit. The limit is not strict. It is estimated based on serialized states’ sizes approximation and is checked once per tick, and so it can overflow for a short time.

virtual ccf::kv::ReadOnlyStorePtr get_store_at(RequestHandle handle, ccf::SeqNo seqno, ExpiryDuration seconds_until_expiry) = 0

Retrieve a Store containing the state written at the given seqno.

See get_store_range for a description of the caching behaviour. This is equivalent to get_store_at(handle, seqno, seqno), but returns nullptr if the state is currently unavailable.

virtual ccf::kv::ReadOnlyStorePtr get_store_at(RequestHandle handle, ccf::SeqNo seqno) = 0

Same as get_store_at but uses default expiry value.

See also

get_store_at

virtual StatePtr get_state_at(RequestHandle handle, ccf::SeqNo seqno, ExpiryDuration seconds_until_expiry) = 0

Retrieve a full state at a given seqno, including the Store, the TxID assigned by consensus, and an offline-verifiable receipt for the Tx.

virtual StatePtr get_state_at(RequestHandle handle, ccf::SeqNo seqno) = 0

Same as get_state_at but uses default expiry value.

See also

get_state_at

virtual std::vector<ccf::kv::ReadOnlyStorePtr> get_store_range(RequestHandle handle, ccf::SeqNo start_seqno, ccf::SeqNo end_seqno, ExpiryDuration seconds_until_expiry) = 0

Retrieve a range of Stores containing the state written at the given indices.

If this is not currently available, this function returns an empty vector and begins fetching the ledger entry asynchronously. This will generally be true for the first call for a given seqno, and it may take some time to completely fetch and validate. The call should be repeated later with the same arguments to retrieve the requested entries. This state is kept until it is deleted for one of the following reasons:

  • A call to drop_cached_states

  • seconds_until_expiry seconds elapse without calling this function

  • This handle is used to request a different seqno or range

The range is inclusive of both start_seqno and end_seqno. If a non-empty vector is returned, it will always contain the full requested range; the vector will be of length (end_seqno - start_seqno + 1) and will contain no nullptrs.

virtual std::vector<ccf::kv::ReadOnlyStorePtr> get_store_range(RequestHandle handle, ccf::SeqNo start_seqno, ccf::SeqNo end_seqno) = 0

Same as get_store_range but uses default expiry value.

See also

get_store_range

virtual bool drop_cached_states(RequestHandle handle) = 0

Drop state for the given handle.

May be used to free up space once a historical query has been resolved, more aggressively than waiting for the states to expire.

struct State

Public Members

ccf::kv::ReadOnlyStorePtr store = nullptr

Read-only historical store at transaction_id.

TxReceiptImplPtr receipt = nullptr

Receipt for ledger entry at transaction_id.

ccf::TxID transaction_id

View and Sequence Number for the State.

class Receipt

Subclassed by ccf::ProofReceipt, ccf::SignatureReceipt

Indexing

class Strategy

The base class for all indexing strategies.

Sub-class this and override handle_committed_transaction to implement your own indexing strategy. Create an instance of this on each node, and then install it with context.get_indexing_strategies().install_strategy(). It will then be given each committed transaction shortly after commit. You should build some aggregate/summary from these transactions, and return that to endpoint handlers in an efficient format.

Subclassed by ccf::indexing::strategies::VisitEachEntryInMap, loggingapp::CommittedRecords

Public Functions

virtual void handle_committed_transaction(const ccf::TxID &tx_id, const ccf::kv::ReadOnlyStorePtr &store) = 0

Receives every committed transaction, in-order, shortly after commit.

The given store contains only the changes that occured in the current transaction.

virtual std::optional<ccf::SeqNo> next_requested() = 0

Returns next tx for which this index should be populated, or nullopt if it wants none. Allows indexes to be populated lazily on-demand, or out-of-order, or reset

class SeqnosByKey_Bucketed_Untyped : public ccf::indexing::strategies::VisitEachEntryInMap

Subclassed by ccf::indexing::strategies::SeqnosByKey_Bucketed< M >, ccf::indexing::strategies::SeqnosForValue_Bucketed< V >

struct Impl

JavaScript FFI Plugins

std::vector<ccf::js::FFIPlugin> ccf::get_js_plugins()

To be implemented by the application.

Returns:

Vector of JavaScript FFI plugins

HTTP Entity Tags Matching

class Matcher

Utility class to resolve If-Match and If-None-Match as described in https://www.rfc-editor.org/rfc/rfc9110#field.if-match

Public Functions

inline Matcher(const std::string &match_header)

Construct a Matcher from a match header

Note: Weak tags are not supported.

inline bool matches(const std::string &etag) const

Check if a given ETag matches the If-Match/If-None-Match header.

inline bool is_any() const

Check if the header will match any ETag (*)