DeathStarBench
DeathStarBench is an open-source benchmark suite for cloud microservices. DeathStarBench includes six end-to-end services, four for cloud systems, and one for cloud-edge systems running on drone swarms.
- Social Network
- Media Service
- Hotel Reservation
- E-Commerce Site (not yet onboarded)
- Banking System (not yet onboarded)
- Drone coordination system (not yet onboarded)
In VC, we have integrated 3 end-to-end services that are released (i.e, SocialNetwork, Media Service, Hotel Reservation). We use DeathStarBench to study the architectural characteristics of microservices, their implications in networking and operating systems, their challenges with respect to cluster management, and their trade-offs in terms of application design and programming frameworks.
This toolset was compiled from the github repo to integrate into VirtualClient.
What is Being Measured?
DeathStarBench is designed to anlayze network communication latencies and throughput between client and server systems. It produces different percentile latencies for network communication and also data transfer/sec for each HTTP request.
Workload Metrics
The following metrics are examples of those captured by the Virtual Client when running the DeathStarBench workload.
| Scenario Name | Metric Name | Example Value (min) | Example Value (max) | Example Value (avg) | Unit |
|---|---|---|---|---|---|
| hotelReservation_MixedWorkload | 100% Network Latency | 8.67 | 208.64 | 21.41504 | ms |
| hotelReservation_MixedWorkload | 50% Network Latency | 320.0 | 596.0 | 459.076 | us |
| hotelReservation_MixedWorkload | 75% Network Latency | 362.0 | 687.0 | 541.96 | us |
| hotelReservation_MixedWorkload | 90% Network Latency | 0.86 | 1.06 | 0.925 | ms |
| hotelReservation_MixedWorkload | 90% Network Latency | 431.0 | 846.0 | 667.4628099173554 | us |
| hotelReservation_MixedWorkload | 99% Network Latency | 0.87 | 4.8 | 2.1194799999999995 | ms |
| hotelReservation_MixedWorkload | 99% Req/sec | 300.0 | 300.0 | 300.0 | |
| hotelReservation_MixedWorkload | 99.99% Network Latency | 5.0 | 23.98 | 10.10556 | ms |
| hotelReservation_MixedWorkload | Avg Req/sec | 53.05 | 53.82 | 53.33760000000001 | |
| hotelReservation_MixedWorkload | Avg network Latency | 368.63 | 713.42 | 533.17708 | us |
| hotelReservation_MixedWorkload | Stdev Req/sec | 74.3 | 77.5 | 74.93035999999998 | |
| hotelReservation_MixedWorkload | Stdev network Latency | 171.58 | 821.33 | 422.1610080645161 | us |
| hotelReservation_MixedWorkload | Stdev network Latency | 0.86 | 1.01 | 0.935 | ms |
| hotelReservation_MixedWorkload | Transfer/sec | 1.61 | 1.62 | 1.6100399999999998 | MB |
| mediaMicroservices_ComposeReviews | 100% Network Latency | 8.57 | 216.7 | 45.076381322957207 | ms |
| mediaMicroservices_ComposeReviews | 100% Network Latency | 3.14 | 3.14 | 3.14 | s |
| mediaMicroservices_ComposeReviews | 50% Network Latency | 377.0 | 619.0 | 470.984496124031 | us |
| mediaMicroservices_ComposeReviews | 75% Network Latency | 0.87 | 0.93 | 0.8966666666666667 | ms |
| mediaMicroservices_ComposeReviews | 75% Network Latency | 481.0 | 846.0 | 597.156626506024 | us |
| mediaMicroservices_ComposeReviews | 90% Network Latency | 594.0 | 848.0 | 699.4677419354839 | us |
| mediaMicroservices_ComposeReviews | 90% Network Latency | 0.87 | 1.71 | 1.1991666666666666 | ms |
| mediaMicroservices_ComposeReviews | 99% Network Latency | 0.96 | 82.18 | 3.6118992248062016 | ms |
| mediaMicroservices_ComposeReviews | 99% Req/sec | 300.0 | 300.0 | 300.0 | |
| mediaMicroservices_ComposeReviews | 99.99% Network Latency | 5.4 | 122.75 | 18.143735408560315 | ms |
| mediaMicroservices_ComposeReviews | 99.99% Network Latency | 1.69 | 1.69 | 1.69 | s |
| mediaMicroservices_ComposeReviews | Avg Req/sec | 53.05 | 53.58 | 53.26624031007752 | |
| mediaMicroservices_ComposeReviews | Avg network Latency | 448.1 | 849.99 | 577.9506779661016 | us |
| mediaMicroservices_ComposeReviews | Avg network Latency | 0.86 | 8.99 | 1.2863636363636364 | ms |
| mediaMicroservices_ComposeReviews | Stdev Req/sec | 74.2 | 77.74 | 74.93906976744185 | |
| mediaMicroservices_ComposeReviews | Stdev network Latency | 202.22 | 841.95 | 457.2306629834254 | us |
| mediaMicroservices_ComposeReviews | Stdev network Latency | 0.87 | 99.82 | 2.4854545454545455 | ms |
| mediaMicroservices_ComposeReviews | Transfer/sec | 214.41 | 215.28 | 214.56081395348839 | KB |
| socialNetwork_ComposePost | 100% Network Latency | 1.03 | 1.03 | 1.03 | m |
| socialNetwork_ComposePost | 100% Network Latency | 9.18 | 206.46 | 26.08708904109589 | ms |
| socialNetwork_ComposePost | 50% Network Latency | 314.0 | 536.0 | 431.31849315068498 | us |
| socialNetwork_ComposePost | 50% Network Latency | 14.08 | 14.08 | 14.08 | s |
| socialNetwork_ComposePost | 75% Network Latency | 369.0 | 707.0 | 517.6746575342465 | us |
| socialNetwork_ComposePost | 75% Network Latency | 18.38 | 18.38 | 18.38 | s |
| socialNetwork_ComposePost | 90% Network Latency | 0.86 | 1.1 | 0.943 | ms |
| socialNetwork_ComposePost | 90% Network Latency | 461.0 | 848.0 | 640.9929078014185 | us |
| socialNetwork_ComposePost | 90% Network Latency | 23.51 | 23.51 | 23.51 | s |
| socialNetwork_ComposePost | 99% Network Latency | 841.0 | 841.0 | 841.0 | us |
| socialNetwork_ComposePost | 99% Network Latency | 0.86 | 5.59 | 2.0380068728522335 | ms |
| socialNetwork_ComposePost | 99% Network Latency | 35.36 | 35.36 | 35.36 | s |
| socialNetwork_ComposePost | 99% Req/sec | 300.0 | 300.0 | 300.0 | |
| socialNetwork_ComposePost | 99.99% Network Latency | 0.93 | 0.93 | 0.93 | m |
| socialNetwork_ComposePost | 99.99% Network Latency | 5.24 | 131.71 | 10.476369863013696 | ms |
| socialNetwork_ComposePost | Avg Req/sec | 50.66 | 53.62 | 53.25143344709897 | |
| socialNetwork_ComposePost | Avg network Latency | 357.53 | 750.97 | 502.6953082191782 | us |
| socialNetwork_ComposePost | Avg network Latency | 14.75 | 14.75 | 14.75 | s |
| socialNetwork_ComposePost | Stdev Req/sec | 74.21 | 76.43 | 74.77890784982935 | |
| socialNetwork_ComposePost | Stdev network Latency | 178.71 | 842.99 | 402.9131468531469 | us |
| socialNetwork_ComposePost | Stdev network Latency | 0.9 | 2.3 | 1.1783333333333333 | ms |
| socialNetwork_ComposePost | Stdev network Latency | 6.96 | 6.96 | 6.96 | s |
| socialNetwork_ComposePost | Transfer/sec | 203.52 | 215.21 | 214.512457337884 | KB |
| socialNetwork_MixedWorkload | 100% Network Latency | 10.57 | 212.74 | 34.00373188405797 | ms |
| socialNetwork_MixedWorkload | 50% Network Latency | 260.0 | 545.0 | 423.28985507246378 | us |
| socialNetwork_MixedWorkload | 75% Network Latency | 319.0 | 649.0 | 509.4528985507246 | us |
| socialNetwork_MixedWorkload | 90% Network Latency | 0.86 | 1.07 | 0.9120000000000001 | ms |
| socialNetwork_MixedWorkload | 90% Network Latency | 397.0 | 837.0 | 634.6826568265683 | us |
| socialNetwork_MixedWorkload | 99% Network Latency | 0.92 | 4.74 | 2.2688043478260875 | ms |
| socialNetwork_MixedWorkload | 99% Req/sec | 300.0 | 300.0 | 300.0 | |
| socialNetwork_MixedWorkload | 99.99% Network Latency | 5.2 | 61.57 | 12.163768115942029 | ms |
| socialNetwork_MixedWorkload | Avg Req/sec | 52.93 | 53.59 | 53.20576086956522 | |
| socialNetwork_MixedWorkload | Avg network Latency | 318.75 | 679.17 | 503.808695652174 | us |
| socialNetwork_MixedWorkload | Stdev Req/sec | 74.21 | 75.97 | 74.77231884057972 | |
| socialNetwork_MixedWorkload | Stdev network Latency | 186.21 | 837.75 | 450.1571102661597 | us |
| socialNetwork_MixedWorkload | Stdev network Latency | 0.87 | 1.08 | 0.9830769230769232 | ms |
| socialNetwork_MixedWorkload | Transfer/sec | 2.94 | 2.95 | 2.9403985507246377 | MB |
| socialNetwork_ReadHomeTimeline | 100% Network Latency | 9.78 | 231.93 | 30.178617021276595 | ms |
| socialNetwork_ReadHomeTimeline | 50% Network Latency | 335.0 | 559.0 | 455.40070921985815 | us |
| socialNetwork_ReadHomeTimeline | 75% Network Latency | 396.0 | 760.0 | 547.6205673758865 | us |
| socialNetwork_ReadHomeTimeline | 90% Network Latency | 503.0 | 849.0 | 670.3127413127413 | us |
| socialNetwork_ReadHomeTimeline | 90% Network Latency | 0.85 | 1.2 | 0.9400000000000002 | ms |
| socialNetwork_ReadHomeTimeline | 99% Network Latency | 0.91 | 4.76 | 2.20468085106383 | ms |
| socialNetwork_ReadHomeTimeline | 99% Req/sec | 300.0 | 300.0 | 300.0 | |
| socialNetwork_ReadHomeTimeline | 99.99% Network Latency | 5.18 | 28.01 | 10.59241134751773 | ms |
| socialNetwork_ReadHomeTimeline | Avg Req/sec | 53.01 | 53.69 | 53.300567375886519 | |
| socialNetwork_ReadHomeTimeline | Avg network Latency | 383.4 | 765.05 | 533.2132269503546 | us |
| socialNetwork_ReadHomeTimeline | Stdev Req/sec | 74.34 | 76.32 | 74.8363829787234 | |
| socialNetwork_ReadHomeTimeline | Stdev network Latency | 186.09 | 826.21 | 439.895 | us |
| socialNetwork_ReadHomeTimeline | Stdev network Latency | 0.91 | 1.42 | 1.0525 | ms |
| socialNetwork_ReadHomeTimeline | Transfer/sec | 213.99 | 215.23 | 214.5518439716312 | KB |
| socialNetwork_ReadUserTimeline | 100% Network Latency | 9.57 | 211.58 | 31.61867383512545 | ms |
| socialNetwork_ReadUserTimeline | 50% Network Latency | 333.0 | 558.0 | 455.64874551971328 | us |
| socialNetwork_ReadUserTimeline | 75% Network Latency | 392.0 | 763.0 | 547.831541218638 | us |
| socialNetwork_ReadUserTimeline | 90% Network Latency | 495.0 | 848.0 | 668.3110236220473 | us |
| socialNetwork_ReadUserTimeline | 90% Network Latency | 0.85 | 1.22 | 0.9456 | ms |
| socialNetwork_ReadUserTimeline | 99% Network Latency | 0.9 | 5.04 | 2.2501075268817205 | ms |
| socialNetwork_ReadUserTimeline | 99% Req/sec | 300.0 | 300.0 | 300.0 | |
| socialNetwork_ReadUserTimeline | 99.99% Network Latency | 5.07 | 33.31 | 11.192293906810037 | ms |
| socialNetwork_ReadUserTimeline | Avg Req/sec | 53.02 | 53.68 | 53.3015770609319 | |
| socialNetwork_ReadUserTimeline | Avg network Latency | 380.75 | 765.2 | 535.2530107526883 | us |
| socialNetwork_ReadUserTimeline | Stdev Req/sec | 74.36 | 76.3 | 74.83716845878137 | |
| socialNetwork_ReadUserTimeline | Stdev network Latency | 0.89 | 1.03 | 0.95375 | ms |
| socialNetwork_ReadUserTimeline | Stdev network Latency | 185.79 | 848.64 | 446.7977490774908 | us |
| socialNetwork_ReadUserTimeline | Transfer/sec | 214.48 | 215.13 | 214.55741935483872 | KB |