WAFBench

WAF Bench (WB) Tool Suits

Build Status


/\ \  _ \ \   /\  __ \   /\  ___\    /\  == \   /\  ___\   /\ "-.\ \   /\  ___\   /\ \_\ \
\ \ \/ ".\ \  \ \  __ \  \ \  __\    \ \  __<   \ \  __\   \ \ \-.  \  \ \ \____  \ \  __ \  
 \ \__/".~\_\  \ \_\ \_\  \ \_\       \ \_____\  \ \_____\  \ \_\\"\_\  \ \_____\  \ \_\ \_\
  \/_/   \/_/   \/_/\/_/   \/_/        \/_____/   \/_____/   \/_/ \/_/   \/_____/   \/_/\/_/

WAF (Web Application Firewall) Bench tool suits is designed to verify the correctness and measure the performance of WAF.

Motivation

Real Traffic Performance Testing

The WAF’s performance fluctuates greatly as the input traffic varies. The existing tool, i.e. ab, can test the performance under only one kind of customized request, which can not reproduce the real traffic scenario. In addition, ab can only customize some fields of the request, which is inconvenient for testing.

In order to get the performance experienced by the real customer and further improve WAF’s efficiency, we build wb, an ab-like tool, which could send multiple and fully-customized packets in one invoking. Besides, WAF Bench tool suits accepts YAML-based input, which can customize your packet and reproduce multiple different packets easily.

Automatic WAF Correctness Testing

Automatic WAF correctness testing can relieve developers from the heavy and tedious verifying works. There are two important factors for conducting a WAF correctness testing: the testing framework and test cases.

For the testing framework, we provide a FTW-Compatible Tool which leverages wb as its underlying tool. Its performance is better than FTW.

For test cases, we provide several generators for different purposes:

Currently, they are still in development.

Feature

Feature

WAF Bench

WAF Bench (wb) is the ab-like tool for conducting performance testing.

Python-based WAF Bench

Python WAF Bench (pywb) is an enhanced tools of wb.

FTW-compatible Tool

Auto-Test Generator

Usage

Using WAFBench at Docker environment is the easiest and recommended method.

# Install Docker
curl -fsSL get.docker.com -o get-docker.sh
sudo sh get-docker.sh
# Clone this repository
git clone https://github.com/microsoft/WAFBench.git
# Build WAFBench
docker build -t wafbench WAFBench

If you install WAFBench by Dockerfile, wb, pywb, ftw_compatible_tool has been install in image wafbench. You can use them just as a local application by below commands:

docker run -ti --rm wafbench wb
docker run -ti --rm wafbench pywb
docker run -ti --rm wafbench ftw_compatible_tool

The detail usage can be found at their corresponding sections.

The black box regression test with crs-v3.1 (please replace hostname and port to the service under test)

docker run -ti --rm -v `pwd`:/data --rm wafbench ftw_compatible_tool -d /data/regression.db -x "load util/regression-test/crs-v3.1/black-box/ | gen | start hostname:port| report | exit"

The result file is regression.db with sqlite3 format, you can open it by DB Browser for SQLite.

Install at native machine

Prerequisites

Some software or libraries may be necessary for further build / usage. All of them are listed below:

The WB tool suites are developed and tested under CentOS 7 (Linux version 3.10, AMD 64 architecture) in a 32 core (Intel Xeon E5 @ 2.30GHz) Server.

Note: If you don’t aim to conduct complex WAF testing tasks there is a cheat sheet below summarizing common testing instructions ranging from install to usage, by which you can focus on your goal; The Advance Usage section just gives you more detailed information of WB if considering it as a black box cannot satisfy your demand.

Cheat Sheet

This tutorial is based on CentOS 7 (Linux version 3.10, AMD 64 architecture). WB also can work on other Linux distribution since there is no dependencies on CentOS.

Install Dependencies

On CentOS:

Before installing dependencies using yum in CentOS, we recommend you to enable the EPEL repo to find some packages listed below (yum --enablerepo=extras install epel-release).

However you can find these packages in your own source using ‘yum search …’

sudo yum install gcc gcc-c++ make                # Install build-essential
sudo yum install libev-devel.x86_64              # Install development headers for libev
sudo yum install cmake                           # Install CMake
sudo yum install boost-devel.x86_64              # Install boost libraries
sudo yum install python2                         # Install python2
sudo yum install python2-pip.noarch              # Install python2 pip
sudo pip install ftw                             # Install ftw module
sudo yum install wget.x86_64                     # Install wget
sudo yum install expat-devel                     # Install expat
sudo yum install openssl-devel                   # Install openssl

or just type

sudo yum install gcc gcc-c++ make libev-devel.x86_64 cmake boost-devel.x86_64 python2 python2-pip.noarch wget.x86_64 expat-devel openssl-devel
sudo pip install ftw

Download WB tools suits

Just clone this repo to your machines.

git clone git@github.com:Microsoft/WAFBench.git

Install wb

Please refer to wb Readme

cd wb
make
make install

Conduct Performance Test / AB-like Test

Assuming that the server is at 10.0.1.1:18081 running we can:

./pywb/main.py -t 10 -c 20  10.0.1.1:18081

Or send requests from a file such as requests.dat:

./pywb/main.py -t 10 -c 25 -F ./example/packets/test-2-packets.yaml 10.0.1.1:18081

Advance Usage

Since the components are independent to each other, the detailed build, install and use tutorials are maintained in their own folder. You can access them at:

Attributions

WB uses the following libraries.

Framework for Testing WAFs (FTW!)

https://github.com/fastly/ftw

Copyright 2016 Fastly
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Changelog

For changelog, you may refer to CHANGELOG.md.