HTTP Engine
Scenario actions
This section describes functionality available in scenarios described in the scenarios section of a test script.
GET / POST / PUT / PATCH / DELETE requests
An HTTP request object may have the following attributes:
url- The request URL; will be appended to the target URL set inconfig.targetbut can also be a fully qualified URL.json- A JSON object to send in the request body.body- Arbitrary data to send in the request body.headers- A JSON object describing header key-value pairs.cookie- A JSON object describing cookie key-value pairs.capture- Set this to capture values from the response body of a request and store those in variables.
Example:
config: target: 'https://example.com' phases: - duration: 10 arrivalRate: 1 scenarios: - flow: - get: url: '/' - post: url: '/resource' json: hello: 'world'Configuration
TLS/SSL
By default, Artillery will reject SSL certificates that it’s unable to validate. Typically, these errors occur when testing services with a self-signed certificate or an expired SSL certificate. You may see errors such as UNABLE_TO_GET_ISSUER_CERT_LOCALLY, UNABLE_TO_VERIFY_LEAF_SIGNATURE, CERT_UNTRUSTED or one of the other validation error codes when that happens.
To get around these issues when using Artillery, you can disable certificate validation with one of the following two options:
- Pass the
-kor--insecureflag to theartillery runorartillery quickcommands. - Set
rejectUnauthorized: falseunder theconfig.tlssetting in your test script:
config: target: 'https://myapp.staging:3002' tls: rejectUnauthorized: falseIgnoring certificate errors can be useful for testing in a development or staging environment. However, you should never use this setting on a production environment. Ignoring certificate errors in a production system can lead to potential security vulnerabilities, like man-in-the-middle attacks.
Request timeouts
If a response to any request takes longer than 10 seconds, Artillery will abort the request and raise an ETIMEDOUT error.
To increase or decrease the default timeout period, set the config.http.timeout setting to the number of seconds for timing out the request.
config: target: 'http://my.app' http: # Responses have to be sent within 10 seconds, or an `ETIMEDOUT` error gets raised. timeout: 10Max sockets per virtual user
By default, Artillery creates one TCP connection per virtual user. To allow for multiple sockets per virtual user (to mimic the behavior of a web browser, for example), specify the number of connections with the config.http.maxSockets setting.
config: target: 'http://my.app' http: # Set a max of five TCP connections per virtual user. maxSockets: 5This setting is per virtual user, not for the total number of sockets. To limit the total number of sockets, use the config.http.pool setting.
Proxy support
Artillery can connect to the internet through a forward proxy , like a corporate proxy.
To send all traffic through a proxy, set the HTTP_PROXY environment variable to the proxy URL. The proxy URL itself may be HTTP or HTTPS. Both HTTP and HTTPS requests will be sent via the proxy.
HTTP_PROXY='http://my.proxy.app:3128' artillery run my-script.yamlTo send HTTPS traffic through a different proxy, set the HTTPS_PROXY environment variable. All HTTPS traffic will use this proxy.
HTTPS_PROXY='https://secure.proxy.app:3128' artillery run my-script.yamlAdditional performance metrics
The HTTP engine can be configured to track additional performance metrics by setting config.http.extendedMetrics to true:
config: http: extendedMetrics: trueThe engine will then report the additional metrics listed below.
cookieJarOptions override
Added inv2.0.0-24 Cookie parsing behavior may be customized via config.http.cookieJarOptions by passing options to the underlying ToughCookie instance .
Default Configuration
Added inv2.0.0-36You can set default configuration for the HTTP engine with config.http.defaults options.
For example, to set a default header:
config: http: defaults: headers: X-My-Header: '123'You can set the following default configuration options:
| Name | Valid Options | Description |
|---|---|---|
| headers | any key:value pair | Default headers to be used in all requests. |
| cookie | any key:value pair | Default cookies to be used in all requests. |
| strictCapture |
| Whether to turn on strict capture by default for all captures. |
| think | jitter: number or percentage | Sets jitter to simulate real-world random variance into think time pauses. |
Logging
You can print messages to the console for each scenario using the log action:
config: target: 'https://example.com' phases: - duration: 10 arrivalRate: 1 scenarios: - flow: - log: 'New virtual user running' - get: url: '/' - post: url: '/resource' json: hello: 'world'In the example above, Artillery will print “New virtual user running” every time it generates a virtual user for the scenario.
The string argument to log may also include variables:
config: target: 'https://example.com' phases: - duration: 10 arrivalRate: 1 scenarios: - flow: - log: 'Current environment is set to: {{ $environment }}' - get: url: '/' - post: url: '/resource' json: hello: 'world'Debug messages will get printed to the console even when running the tests in “quiet” mode (using the --quiet or -q flag with the run command).
Setting Headers
Arbitrary headers may be sent under the headers option for a request:
- get: url: '/test' headers: X-My-Header: '123'Compressed Responses (gzip)
Artillery automatically decompresses responses encoded with gzip, by adding an Accept-Encoding header to the request.
Set gzip to false to disable automatic decompression if required to improve performance.
- post: url: '/test' json: foo: bar gzip: falseBasic Authentication
If your request requires Basic HTTP authentication , set your username and password under the auth option:
- get: url: '/protected/resource' auth: user: myusername pass: mypasswordQuery Strings
Query strings can be appended directly to the url or set with qs:
- get: url: '/products' qs: search_keyword: 'coffee' page_size: 25The above request is identical to the following:
- get: url: '/products?search_keyword=coffee&page_size=25'Redirects
Artillery follows redirects by default. To stop Artillery from following redirects, set the followRedirect option on a request to false:
- get: url: '/test' followRedirect: falseForms
URL-encoded forms (application/x-www-form-urlencoded)
Use the form attribute to send an URL-encoded form :
- post: url: '/submit' form: name: 'Homer Simpson' favorite_food: 'donuts'Multipart forms (multipart/form-data)
Use the formData attribute to send a multipart/form-data form (forms containing files, non-ASCII data, and binary data).
formData is an object with fieldName: fieldValue pairs, each of them representing a form field:
- post: url: '/upload' formData: name: 'Homer Simpson' favorite_food: 'donuts'Uploading files
Added inv2.0.17To upload a file, set the value of the form field in formData to fromFile: './path/to/your-file'. The file path is resolved relative to the test script file.
- post: url: '/upload' formData: name: 'Homer Simpson' favorite_food: 'donuts' image: fromFile: './path/to/homer.jpg'You can set content type for a file field explicitly with the contentType attribute:
- post: url: '/upload' formData: name: 'Homer Simpson' favorite_food: 'donuts' image: # Set content-type for this binary file explicitly fromFile: './path/to/binary-file' contentType: 'image/jpeg'If you need to set contentType on a non-file field, use contentType and value attributes:
- post: url: '/upload' formData: name: value: '{"name":"Tiki","species":"pony","color":"brown","age":3}' contentType: 'application/json'If you are running distributed tests on AWS Fargate or Lambda you need to provide the path to the file in the includeFiles section of the config.
config: target: "http://my.shop.app" phases: - duration: 1 arrivalRate: 1 includeFiles: - ./path/to/invoice.pdf scenarios: - name: "order" flow: - post: url: '/upload' formData: id: abc123456 item: 'donuts' invoice: fromFile: ./path/to/invoice.pdf Setting the Content-Length header for file uploads
In some cases you may need to explicitly set the Content-Length header for a file upload (e.g. with S3/CloudFront pre-signed URLs). You can do this by setting the setContentLengthHeader attribute on the request level to true as seen below. This attribute will only work for file uploads.
- post: url: '/upload' setContentLengthHeader: true formData: name: 'Homer Simpson' favorite_food: 'donuts' image: fromFile: './path/to/homer.jpg'Extracting and re-using parts of a response (request chaining)
You can parse responses and re-use those values in subsequent requests using the capture option in your requests.
Syntax
To tell Artillery to parse a response, add the capture option to a request:
- get: url: '/' capture: json: '$.id' as: 'id'The capture option must always have an as attribute, which names the value for use in subsequent requests. It also requires one of the following attributes:
json- Allows you to define a JSONPath expression.xpath- Allows you to define an XPath expression.regexp- Allows you to define a regular expression that gets passed to a RegExp constructor . A specific capturing group to return may be set with thegroupattribute (set to an integer index of the group in the regular expression). Flags for the regular expression may be set with theflagsattribute.header- Allows you to set the name of the response header whose value you want to capture.selector- Allows you to define a Cheerio element selector. Theattrattribute will contain the name of the attribute whose value we want. An optionalindexattribute may be set to a number to grab an element matching a selector at the specified index,"random"to grab an element at random, or"last"to grab the last matching element. If theindexattribute is not specified, the first matching element will get captured.
Turn off strict capture
By default, captures are strict. If a capture rule fails because nothing matches, any subsequent requests in the scenario will not run, and that virtual user will stop making requests. This behavior is the default since subsequent requests typically depend on captured values and fail when one is not available.
In some cases, it may be useful to turn this behavior off. To make virtual users continue with running requests even after a failed capture, set strict to false:
- get: url: '/' capture: json: '$.id' as: 'id' strict: false # We don't mind if `id` can't be captured and the next requests 404s - get: url: '/things/{{ id }}'Capturing multiple values
Multiple values can be captured in a single request with an array of capture specs:
- get: url: '/journeys' capture: - xpath: '(//Journey)[1]/JourneyId/text()' as: 'JourneyId' - header: 'x-my-custom-header' as: 'headerValue'Examples
The following example grabs a matching a element at random and uses the value of its href attribute in the next request:
- get: url: '/some/page' capture: - selector: 'a[class^=productLink]' index: 'random' attr: 'href' as: 'productUrl' - get: url: '{{ productUrl }}'Cookies
Cookies are remembered and re-used by individual virtual users. The cookie option can set custom cookies in individual requests.
The following example sets a cookie with the key saved with the value of tapir,sloth, which can be accessed in subsequent requests:
- get: url: '/pets/' cookie: saved: 'tapir,sloth' - get: url: '/pets/saved'Pausing execution with think
To pause the virtual user for N seconds, use a think action. The argument to think is the number of second to pause for. Floating numbers are supported, e.g. 0.5 pauses for half a second.
- post: url: '/pets' json: name: 'Mali' species: 'dog' capture: json: '$.id' as: 'id' # wait for 5 seconds: - think: 5 - get: url: '/pets/{{ id }}'Using time units for think
Added inv2.0.1 As an alternative to using seconds, you can also provide any human-readable format from the ms package . For example:
- post: url: '/pets' json: name: 'Mali' species: 'dog' capture: json: '$.id' as: 'id' # wait for two minutes: - think: 2min - get: url: '/pets/{{ id }}'Conditional Requests
The ifTrue option can execute a request in a flow only when meeting a condition. ifTrue may take two forms:
- Set to a name of a scenario variable. If the variable is set (and is not explicitly set to
false), the request will be executed. - Set to a simple conditional expression, which may refer to any of the scenario variables and use one or more of:
- A numeric operation with
+,-,*,/,%(modulo), or^(power). - A comparison with
==,<,>,<=, or>=. - A boolean operation with
or,and, ornot. If the variable is explicitly set to a boolean, make sure to use this instead of comparison operators.
- A numeric operation with
Examples
Make a GET request only if the keyword scenario variable is set to a value:
- get: url: '/search?keyword={{ keyword }}' ifTrue: 'keyword'Make a GET request only if the pageNumber scenario variable is less than 10:
- get: url: '/pages/{{ pageNumber }}' ifTrue: 'pageNumber < 10'Make a GET request only if the admin and user scenario variables are set to true:
- get: url: '/admin' ifTrue: 'admin and user'Loops
You can use the loop construct to repeat through several requests in a scenario.
Looping through a number of requests
You can loop through a specific number of requests using the count attribute.
The following example sends 100 GET requests to / for each virtual user:
scenarios: - flow: - loop: - get: url: '/' count: 100loop is an array - any number of requests can be specified. Variables, cookies, and response parsing will work as expected.
Accessing the loop count
You can access the current step of the loop by using the $loopCount variable, starting from 1. This variable is only available inside of the loop.
The following loop will first make a request to /pages/1, then /pages/2, and so on, up to /pages/100:
scenarios: - flow: - loop: - get: url: '/pages/{{ $loopCount }}' count: 100If the count option is omitted, the loop will run indefinitely.
Looping through an array
You can loop through an array of values by setting the over property to a specified array or by using the name of the variable containing an array of values. The loop can then access the value through the $loopElement variable.
In the following example, each value of the array defined in over will make one request for each value specified in the array, for a total of 3 requests:
scenarios: - flow: - loop: - get: url: '/products/{{ $loopElement }}' over: - id123 - id456 - id789You can also iterate over different arrays defined as variables in the config section.
The following example uses the productIds variable inside of a loop. Each virtual user will make three requests for one of the arrays:
config: target: 'https://my.app.local' phases: - duration: 600 arrivalRate: 10 variables: productIds: - ['id1', 'id2', 'id3'] - ['id4', 'id5', 'id6'] scenarios: - flow: - loop: - get: url: '/products/{{ $loopElement }}' over: productIdsIn this example, a virtual user will make one of the following three requests in their loop:
/products/id1,/products/id2and/products/id3/products/id4,/products/id5and/products/id6
Looping with custom logic
You can further control when to continue looping through a custom function defined in the whileTrue option.
For instance, let’s say you want to poll an endpoint until it returns a JSON response with the top-level status attribute set to "ready":
config: target: 'https://my.app.local' phases: - duration: 600 arrivalRate: 10 processor: './my-functions.js' scenarios: - flow: - loop: - think: 5 - get: url: '/some/endpoint' capture: - json: $.status as: 'status' whileTrue: 'statusReady'The whileTrue option uses a JavaScript function loaded from my-functions.js containing the logic that determines when the loop stops:
module.exports = { statusReady: statusReady, }; function statusReady(context, next) { const continueLooping = context.vars.status !== 'ready'; // While `continueLooping` is true, the `next` function will // continue the loop in the test scenario. return next(continueLooping); }The whileTrue option takes precedence over count and over if either of those is specified.
Parallel requests
Added inv2.0.10Artillery can send multiple requests in parallel. The following scenario sends three requests in parallel:
scenarios: - flow: - parallel: - get: url: '/resource1' - get: url: '/resource2' - get: url: '/resource3'Writing custom logic in JS / TS
The HTTP engine has support for “hooks”, which allow running custom code at specific points during the execution of a scenario.
beforeScenarioandafterScenario- Called before/after a virtual user executes a scenario.beforeRequest- Called before sending a request; request parameters like the URL, cookies, headers, and body can be customized here.afterResponse- Called after a response has been received; the response can be inspected, and custom variables can be set here.
Custom functions may also be run at any point in a scenario with the function action.
Loading custom code
You can load custom code with config.processor attribute in your test script.
beforeRequest hooks
A beforeRequest hook gets the following arguments:
requestParams- Use this parameter to customize what to send in the request (headers, body, cookies, etc.).context- The virtual user’s context.context.varsis a dictionary containing all defined variables.context.scenariois the scenario definition for the scenario currently being run by the virtual user. Added inv2.0.0-38
events- An event emitter that can be used to communicate with Artillery.
afterResponse hooks
A function invoked in an afterResponse gets the following arguments:
requestParams- Use this parameter to customize what to send in the request (headers, body, cookies, etc.).response- Contains the response details (headers, body, etc.).context- The virtual user’s context.context.varsis a dictionary containing all defined variables.context.scenariois the scenario definition for the scenario currently being run by the virtual user. Added inv2.0.0-38
events- An event emitter that can be used to communicate with Artillery.
function actions and beforeScenario / afterScenario hooks
A function invoked as an action in a scenario definition, either through function, beforeScenario and afterScenario gets the following arguments:
context- The virtual user’s context.context.varsis a dictionary containing all defined variables.context.scenariois the scenario definition for the scenario currently being run by the virtual user. Added inv2.0.0-38
events- An event emitter that can be used to communicate with Artillery.
Specifying a function to run
beforeRequest and afterResponse hooks can be set in a request:
- post: url: '/some/route' beforeRequest: 'setJSONBody' afterResponse: 'logHeaders'In this example, Artillery runs the setJSONBody function before making the request and the logHeaders function after receiving the response.
Specifying multiple functions
You can also specify an array of function names, which run sequentially, one after another.
- post: url: '/products' beforeRequest: - 'setApiKey' - 'setJSONBody' afterResponse: 'logHeaders'Setting scenario-level hooks
Similarly, a scenario definition can have beforeScenario and afterScenario attributes, which will make the specified functions run at the start, or the end of a scenario.
scenarios: - beforeScenario: 'setApiKey' afterScenario: 'logResults' flow: - get: url: '/search?q={{ query }}' - get: url: '/products'Function steps in a flow
A function may be run at any point in a scenario with a function action:
scenarios: - flow: - get: url: '/search?q={{ query }}' - function: 'setupSomeData' - get: url: '/some/url?q={{ query }}'Hook functions invoked with function action have full access to the virtual user’s context:
async function setupSomeData(context, events) { // Set the "query" variable for the virtual user. context.vars['query'] = 'foo'; }Metrics reported by the engine
In addition to the default metrics reported by Artillery, the HTTP engine reports the following metrics:
| Metric | Type | Description |
|---|---|---|
http.requests | Counter (count) | Number of HTTP requests made. |
http.responses | Counter (count) | Number of HTTP responses received. |
http.codes.<status_code> | Counter (count) | Number of codes received for each specific status code. |
http.downloaded_bytes | Counter (bytes) | Sum of downloaded_bytes of all responses received during this period. |
http.response_time.<aggregation> | Histogram (milliseconds) | Response time (measured as TTFB - time-to-first-byte) aggregation for requests during this period. |
http.request_rate | Rate (req/sec) | Rate of http requests done over the time period. |
http.response_time.2xx Available inv2.0.21 | Histogram (milliseconds) | Response time distribution for 2xx responses |
http.response_time.3xx Available inv2.0.21 | Histogram (milliseconds) | Response time distribution for 3xx responses |
http.response_time.4xx Available inv2.0.21 | Histogram (milliseconds) | Response time distribution for 4xx responses |
http.response_time.5xx Available inv2.0.21 | Histogram (milliseconds) | Response time distribution for 5xx responses |
Extended metrics
If extendedMetrics is enabled, the following additional metrics will be reported:
| Metric | Type | Description |
|---|---|---|
http.dns.<aggregation> | Histogram (milliseconds) | Time taken in the DNS phase of requests (i.e. time taken by DNS lookups). |
http.tcp.<aggregation> | Histogram (milliseconds) | Time taken in the TCP phase of requests (i.e. time taken to establish TCP connections). |
http.tls.<aggregation> | Histogram (milliseconds) | Time taken in the TLS phase of requests (i.e. time taken by completing TLS handshakes). |
http.total.<aggregation> | Histogram (milliseconds) | Time for the entire response to be downloaded (i.e. time between when request started and it finished due to ending, aborting or erroring). |
Debugging
If you’re having issues getting your test scenarios to complete successfully, you can print out helpful debugging messages using the DEBUG environment variable.
On macOS and Linux systems, you can temporarily set the DEBUG environment variable by setting its value when running your Artillery test script:
DEBUG=http artillery run my-script.yamlFor the Windows Command Prompt, you first need to set the DEBUG environment variable using the set command before running the test script:
set DEBUG=http artillery run my-script.yamlIf you use PowerShell on Windows, you can set the DEBUG environment variable using $Env:DEBUG before running the test script:
$Env:DEBUG = 'http' artillery run my-script.yamlThe following examples use the macOS/Linux format for setting environment variables, but you can use any of the DEBUG values on Windows for your preferred command line interface.
Print request details, errors, and capture values
Set DEBUG=http when running your tests to view details about each request (URL, method, headers, etc.), values captured during a request, and any errors that occurred during the test run.
DEBUG=http artillery run my-script.yamlPrint request URL
Set DEBUG=http:request when running your tests to view the URL for each request made.
DEBUG=http:request artillery run my-script.yamlPrint response headers and body
Set DEBUG=http:response when running your tests to view each request’s response headers and body.
DEBUG=http:response artillery run my-script.yamlCombining debugging messages
You can combine different debugging modes in a single test run by setting a comma-separated list of values for DEBUG. For instance, to print the request URL and the response headers and body:
DEBUG=http:request,http:response artillery run my-script.yamlYou can also display all HTTP debugging messages with DEBUG=http*:
DEBUG=http* artillery run my-script.yaml