scitt-ccf-ledger

Development guidelines

The following explains how to build, run, and test scitt-ccf-ledger.

Development environment

scitt-ccf-ledger uses a Trusted Execution Environment (TEE) to provide strong security guarantees. This means TEE hardware (AMD SEV-SNP) is required to run and test scitt-ccf-ledger in full.

However, scitt-ccf-ledger also supports running in virtual mode which does not require TEE hardware and is generally sufficient for local development.

Develop within Codespaces

For virtual mode development only, instead of following the steps below, you can also use GitHub Codespaces and then continue with the “Building” section:

Develop within a running Docker image

Similar to Codespaces you could build and test the application within the running docker image:

docker build -t mytestimg -f .devcontainer/Dockerfile .
docker run --rm -it --volume $(pwd):/opt/app --workdir /opt/app --entrypoint /bin/bash mytestimg
# workaround to make git happy in a running docker image
/opt/app# git config --global --add safe.directory "*"

## ready to build and test now, see below commands

Develop within a host machine

It is expected that you have Azure Linux 3.0. Follow the steps below to setup your development environment:

1. Set up your host machine:
   • If using virtual mode, running Azure Linux 3.0 on any platform (WSL, VM, etc.) is enough
   • If using SNP, you should use a machine with SNP hardware support and a platform that allows to enforce security policies for containers running on it (e.g., Confidential Containers on AKS, Confidential Containers on ACI)
2. Install dependencies:

    wget https://github.com/microsoft/CCF/archive/refs/tags/ccf-7.0.10.tar.gz
    tar xvzf ccf-7.0.10.tar.gz
    cd CCF-ccf-7.0.10/scripts/
    ./setup-dev.sh
   

Compiling

Using Docker build container

When you need to quickly build it and do not have the configured development environment yet.

./docker/build.sh

Using your development environment

This will expect all of the required dependencies to be set correctly.

Build scitt-ccf-ledger by running:

./build.sh

Running

Using Docker development script

The script is used in testing, it starts the docker image and sets basic configuration. For more details refer to docker/README.md.

./docker/run-dev.sh

To start a multi-node network instead, set NODE_COUNT:

NODE_COUNT=3 ./docker/run-dev.sh

Requests go to the nginx round robin load balancer on https://localhost:8000, which forwards them to the nodes; node i itself is reachable on https://localhost:$((8001 + i)). See docker/README.md for the available tuning variables.

In your development environment

1. Build first (see above)

2. Start a single-node CCF network running the scitt-ccf-ledger application:

    ./start.sh
   

3. Before claims can be submitted, the scitt-ccf-ledger application needs to be configured. For local development purposes, the following command will setup the service appropriately.

   ./pyscitt.sh governance local_development --url https://127.0.0.1:8000
   

   Note this command should not be used for a production instance, as it will leave the service open to all.

Configuring

The application expects the configuration to be submitted via the CCF proposals, for that you could use the CLI.

To set a JavaScript policy:

echo <<< EOL
{
    "policy": {
        "policyScript": "export function apply(phdr) { if (!phdr.issuer) {return 'Issuer not found'} else if (phdr.issuer !== 'did:x509:0:sha256:HnwZ4lezuxq/GVcl/Sk7YWW170qAD0DZBLXilXet0jg=::eku:1.3.6.1.4.1.311.10.3.13') { return 'Invalid issuer'; } return true; }"
    },
    "authentication": {
        "allowUnauthenticated": true
    }
}
EOL >> test-config.json;

Alternatively, to set a Rego policy:

echo <<< EOL
{
    "policy": {
        "policyRego": "\npackage policy\ndefault allow := false\nissuer_allowed if {\n    input.phdr[\"CWT Claims\"].iss == \"did:x509:0:sha256:HnwZ4lezuxq_GVcl_Sk7YWW170qAD0DZBLXilXet0jg::eku:1.3.6.1.4.1.311.10.3.13\"\n}\nseconds_since_epoch := time.now_ns() / 1000000000\niat_in_the_past if {\n    input.phdr[\"CWT Claims\"].iat < seconds_since_epoch\n}\nsvn_positive if {\n    input.phdr[\"CWT Claims\"]._svn >= 0\n}\nallow if {\n    issuer_allowed\n    iat_in_the_past\n    svn_positive\n}\n"
    },
    "authentication": {
        "allowUnauthenticated": true
    }
}
EOL >> test-config.json;

The configuration can be pushed with the following command:

./pyscitt.sh governance propose_configuration -k --url https://localhost:8000 --member-key workspace/member0_privk.pem --member-cert workspace/member0_cert.pem --configuration test-config.json

Above you can see a special workspace directory which would have been created when running with docker/run-dev.sh and would contain the member keys.

Testing

scitt-ccf-ledger has unit tests, covering individual components of the source code, and functional tests, covering end-to-end use cases of scitt-ccf-ledger.

Unit tests

Unit tests will be run as part of the build workflow in cmake.

Using your host environment

./build.sh

*Using Docker

The command below uses the built Docker image mytestimg to run the tests (see above how to build the image).

docker run --rm -it --volume $(pwd):/opt/app --workdir /opt/app --entrypoint /bin/bash mytestimg -c 'git config --global --add safe.directory "*" && ./build.sh'

Functional (e2e) tests

To start the tests you need to use the script run_functional_tests.sh.

Specific functional test can also be run by passing additional pytest arguments, e.g. ./run_functional_tests.sh -k test_use_cacert_submit_verify_x509_signature

Note: the functional tests will launch their own CCF network on a randomly assigned port. You do not need to start an instance beforehand.

Using Docker

The script will launch the built Docker image and will execute tests against it:

./docker/build.sh
DOCKER=1 ./run_functional_tests.sh

By default this runs against a single node. Set NODE_COUNT to run the same tests against a multi-node network instead, reached through the round robin load balancer, which additionally covers request distribution and the redirection of writes to the primary:

DOCKER=1 NODE_COUNT=3 ./run_functional_tests.sh

See docker/README.md for the cluster details.

Using your host environment

./build.sh
./run_functional_tests.sh

Performance regression tests

A basic set of performance indicators can be obtained by building the project, and running:

./run_functional_tests.sh -m bencher

Also see .github/workflow/bencher.yml, and the dashboard. This is useful to understand the potential performance impact of changes.

Load tests

HTTP based

To run load tests, you can use the test_load test in test/test_load.py (with the load driver under test/load_test/). It simulates a high load on the scitt-ccf-ledger application and measures its performance under stress.

./docker/build.sh
DOCKER=1 NODE_COUNT=3 ./run_functional_tests.sh -m perf -k test_load --enable-perf

The output will be stored in the test/load_test/locust_stats.json file, and the chart images generated in test/load_test/charts.

When NODE_COUNT is greater than 1 the load is driven through the load balancer, and the resource usage of every node and of the load balancer is collected: summary.txt and docker_resource_usage.png report the cluster totals along with a per-container breakdown.

.NET SDK based

To run the same load through the .NET SDK (single process, concurrent, waitForCommit):

./docker/build.sh
DOCKER=1 ENABLE_DOTNET_TESTS=1 ./run_functional_tests.sh -m perf -k test_dotnet_load --enable-perf

The run stats are stored in test/load_test/dotnet_load_stats.json (throughput, latency percentiles, per-second completions), and chart images are generated in test/load_test/charts (dotnet_throughput.png, dotnet_latency.png, and dotnet_docker_resources.png in Docker mode).

Address sanitization

To enable ASan it is necessary to build CCF from source:

BUILD_DEBUG_CCF_FROM_SOURCE=ON ./build.sh
./run_functional_tests.sh

Fuzzing

Run HTTP API fuzzing tests after building the application:

Using Docker

DOCKER=1 ./run_fuzz_tests.sh

Using your host environment

./run_fuzz_tests.sh

AMD SEV-SNP platform

To use AMD SEV-SNP as a platform, it is required to pass additional configuration values required by CCF for the attestation on AMD SEV-SNP hardware. These values may differ depending on which SNP platform you are using (e.g., Confidential Containers on ACI, Confidential Containers on AKS).

The required configs can be set using the SNP_ATTESTATION_CONFIG environment variable in any of the development and test scripts. The variable should be set to the path of a JSON file containing the CCF SNP attestation configuration. An example file would look like this:

{
    "snp_endorsements_servers": [
      {
        "type": "AMD",
        "url": "kdsintf.amd.com"
      }
    ],
    "snp_security_policy_file": "/path/to/security-policy-base64",
    "snp_uvm_endorsements_file": "/path/to/reference-info-base64"
}

Please refer to the CCF documentation on the AMD SEV-SNP platform for more details on how to set these values in each platform.

To start SCITT on SNP, you would run:

SNP_ATTESTATION_CONFIG=/path/to/snp-attestation-config.json ./start.sh

To run the SCITT functional tests on SNP, you would run:

SNP_ATTESTATION_CONFIG=/path/to/snp-attestation-config.json ./run_functional_tests.sh

This site is open source. Improve this page.