Generator¶
Overview¶
This component is responsible for generating requests and storing them in a format that could be submitted to the server. The user can declare the requests by leveraging the functions inside the library in tests/infra/piccolo/generator.py. The user can generate requests from the library by either calling the command line tool in tests/infra/piccolo/generate_packages.py or by creating a script calling the functions of the library, such as the tests/infra/piccolo/logging_generator.py which contains a sample generation of requests for the logging CCF application.
Run generator¶
The generator component offers a command-line tool for basic scenarios as well as a sample for an application for more complex scenarios.
Command-Line Tool¶
For the command line option you need to run the following:
$ python3 generate_packages.py
After executing this command there will be a .parquet
file produced in the same directory
containing the data with the requests. You can configure the generated requests using the
following arguments:
-h, --help show this help message and exit
-hs HOST, --host HOST The host to submit the request. (default: 127.0.0.1:8000)
-p PATH, --path PATH The relative path to submit the request. (default: /app/log/private)
-vr VERB, --verb VERB The request action. (default: POST)
-r ROWS, --rows ROWS The number of requests to send. (default: 16)
-rt REQUEST_TYPE, --request_type REQUEST_TYPE The transfer protocol for the request. (default: HTTP/1.1)
-pf PATH_TO_PARQUET, --path_to_parquet PATH_TO_PARQUET Path to the parquet file to store the generated requests (default: ./requests.parquet)
-ct CONTENT_TYPE, --content_type CONTENT_TYPE The Content-Type representation header is used to indicate the original media type of the resource. (default: application-json)
-d DATA, --data DATA A string with the data to be sent with a request (default: {"id": 1, "msg": "Send message with id 1"})
Scripting Requests¶
To script the requests, a Messages
object must be first created and the two main functions
append()
and parquet_to_file()
get called. The append()
function will create a dataframe
containing an id for each request and the request in a raw format. The generated-dataframe will first be
concatenated to the object-dataframe and then the generated-dataframe will be returned to the user.
The user could then, based on the previously returned dataframes, interleave requests or
implement as complex scenarios as he wants. The append()
requires the server, the path and the
request method to create the requests, if a method that requires data is specified then the user
can declare the data as well. The content-tye and the request version can also be specified
if the defaults are not preferred. Lastly, the user can also declare the number of times the
same request is generated by providing the number on the iterations
argument.
After finishing the generation of the requests, the to_parquet_file()
function must be called
to write the concatenated object-dataframe on the file specified in the arguments. Then you
can run your script as you would run any python file:
$ python3 logging_generator.py
Parquet files are an easy and well-compressed way of storing requests generated from this component to run the same generated requests multiple times on the same submitter under different circumstances.