Platform Overview
The Virtual Client is a unified workload and system monitoring platform for running customer-representative scenarios on virtual machines or physical hosts/blades in the Azure Cloud. The platform supports a wide range of different industry standard/benchmark workloads used to measuring various aspects of the system under test (e.g. CPU, I/O, network performance, power consumption). The platform additionally provides the ability to capture important performance and reliability measurements from the underlying system. The platform supports all business-critical Azure environments including guest/VM systems, host/blade systems and data center/DC lab systems. The platform additionally supports both x64 and ARM64 compute architectures.
Team Contacts
Downloads
Release versions of the Virtual Client are available on public NuGet.org. Note that versions that are not tagged as 'Release' are considered beta quality. Production quality releases will be tagged appropriately.
- NuGet Releases
- https://www.nuget.org/packages/VirtualClient
- Please note we will be renaming package to Microsoft.VirtualClient in the future pending MSFT internal processes.
Platform/Architectures
In the workload and profile sections below, the following designations are used to indicate the OS platform and CPU architecture on which the workloads are supported:
- linux-x64
The workload is supported on the Linux operating system for Intel and AMD x64 architectures.
- linux-arm64
The workload is supported on the Linux operating system for ARM64/AARCH64 architectures.
- win-x64
The workload is supported on the Windows operating system for Intel and AMD x64 architectures.
- win-arm64 The workload is supported on the Windows operating system for ARM64/AARCH64 architectures.
Supported Workloads/Benchmarks
The following list of workloads are used by Virtual Client profiles to exercise the system components in a consistent way required to measure performance baselines and differences.
VirtualClient handles the installation and execution of various tools. Individual license files are not prompted for each workload. By using VirtualClient, users accept the license of each of the benchmarks individually, comply to the terms for the tool you are using, and take responsibility for using them.
| Workload/Benchmark | Specialization | Supported Platforms/Architectures | License(s) |
|---|---|---|---|
| 7zip | Compression | linux-x64, linux-arm64 | GNU LGPL |
| AspNetBench | ASP.NET Web Server. | linux-x64, linux-arm64, win-x64, win-arm64 | MIT (ASP.NET) MIT (Bombardier) |
| BlenderBenchmark | GPU/Graphics Rendering Performance | win-x64 | GNU LGPL |
| CoreMark | CPU Performance | linux-x64, linux-arm64 | Apache+Custom |
| CoreMark Pro | Precision CPU | linux-x64, linux-arm64, win-x64, win-arm64 | Apache+Custom |
| CPS | Network Connection Reliability | linux-x64, linux-arm64, win-x64, win-arm64 | Microsoft-Developed |
| DCGMI | GPU Qualification | linux-x64 | Apache-2.0 |
| DeathStarBench | Docker Swarm/Container Microservices | linux-x64, linux-arm64, win-x64, win-arm64 | Apache-2.0 |
| DiskSpd | Disk I/O Performance | win-x64, win-arm64 | MIT |
| Flexible IO Tester (FIO) | Disk I/O Performance | linux-x64, linux-arm64, win-x64 | GPL-2.0 |
| GeekBench5 | CPU Performance | linux-x64, win-x64, win-arm64 | End User License Required |
| Graph500 | 3D Simulation | linux-x64, linux-arm64 | Custom |
| Gzip | Compression | linux-x64, linux-arm64 | GPL |
| HPCG | High Performance Compute (HPC) | linux-x64, linux-arm64 | Custom |
| HPLinpack | Linear Equations | linux-x64, linux-arm64 | IBM |
| LAPACK | Linear Equations | linux-x64, linux-arm64, win-x64, win-arm64 | Custom |
| Latte | Network Latencies | win-x64, win-arm64 | MIT |
| LMbench | Memory Performance | linux-x64, linux-arm64 | GPL-2.0 |
| LZBench | Compression/Streaming | linux-x64, linux-arm64, win-x64, win-arm64 | None |
| Memcached | In-Memory Data Cache | linux-x64, linux-arm64 | BSD-3 (Memcached) GPL-2.0 (Memtier) |
| MLPerf | Machine Learning | linux-x64 | Custom |
| NAS Parallel | High Performance Compute (HPC) | linux-x64, linux-arm64 | NASA-1.3 |
| Network ICMP Ping | Network Latencies | linux-x64, linux-arm64, win-x64, win-arm64 | MIT |
| NTttcp | Network Bandwidth | linux-x64, linux-arm64, win-x64, win-arm64 | MIT |
| OpenFOAM | Computational Fluid Dynamics | linux-x64, linux-arm64 | Custom |
| OpenSSL | Cryptography/Encryption | linux-x64, linux-arm64, win-x64 | Apache-2.0 |
| Pbzip2 | Compression | linux-x64, linux-arm64 | BSD |
| PostgreSQL | Relational Database Performance | linux-x64, linux-arm64, win-x64 | PostgreSQL |
| Prime95 | CPU Stress | linux-x64 | Custom |
| Redis | In-Memory Data Cache | linux-x64, linux-arm64 | BSD-3 (Redis) GPL-2.0 (Memtier) |
| SockPerf | Network Latencies | linux-x64, linux-arm64 | Custom |
| SPEC CPU 2017, SPECrate Integer | Precision CPU, Integer Calculations | linux-x64, linux-arm64 | End User License Required |
| SPEC CPU 2017, SPECrate Floating Point | Precision CPU, Floating-point Calculations | linux-x64, linux-arm64 | End User License Required |
| SPEC CPU 2017, SPECspeed Integer | Precision CPU, Integer Calculations | linux-x64, linux-arm64 | End User License Required |
| SPEC CPU 2017, SPECspeed Floating Point | Precision CPU, Floating-point Calculations | linux-x64, linux-arm64 | End User License Required |
| SPEC JBB 2015, SPECjbb | Java Server | linux-x64, linux-arm64, win-x64, win-arm64 | End User License Required |
| SPEC JVM 2008, SPECjvm | Java Runtime Performance | linux-x64, linux-arm64, win-x64, win-arm64 | SPEC |
| SPEC Power 2008, SPECpower | High precision, steady-state CPU usage | linux-x64, linux-arm64, win-x64, win-arm64 | End User License Required |
| SPECviewperf 2020, SPECview | 3D graphics performance | win-x64 | SPEC |
| Stressapptest | Fault Tolerance | linux-x64, linux-arm64 | Apache-2.0 |
| Stress-ng | Fault Tolerance | linux-x64, linux-arm64 | GPL-2.0 |
| SuperBench | Machine Learning | linux-x64 | MIT |
| Sysbench OLTP w/MySQL | Relational Database Performance | linux-x64, linux-arm64 | GPL-2.0 (Sysbench) GPL-2.0 (MySQL) |
Supported System Monitoring Facilities
The platform supports capturing information from the system in the background while workloads are running. The following list of monitoring facilities are available in the Virtual Client.
Certain monitoring facilities are only available on specific hardware because they expect specific tools/hardware on the system (e.g. ipmiutil, nvidia monitors).
| Monitor | Specialization | Supported Platforms/Architectures | License(s) |
|---|---|---|---|
| Nvidia SMI | Nvidia GPUs | linux-x64, linux-arm64 | |
| Performance Counters | Performance Counters | linux-x64, linux-arm64, win-x64, win-arm64 |
Data Collection Notice
The software may collect information about you and your use of the software and send it to Microsoft. Microsoft may use this information to provide services and improve our products and services. You may turn off the telemetry as described in the repository. There are also some features in the software that may enable you and Microsoft to collect data from users of your applications. If you use these features, you must comply with applicable law, including providing appropriate notices to users of your applications together with a copy of Microsoft’s privacy statement. Our privacy statement is located at https://go.microsoft.com/fwlink/?LinkID=824704. You can learn more about data collection and use in the help documentation and our privacy statement. Your use of the software operates as your consent to these practices.
Trademarks
This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.