Author: roast chicken Prince
Source: Hang Seng LIGHT cloud community
This year's company developer conference is in the form of online. According to the Convention, in order to ensure the service, it is necessary to evaluate the performance of the whole system temporarily. The more commonly used tools are jmeter and Apache Bench. Finally, Apache Bench (AB for short) was selected between them, and some conclusions were made for ab tool.
Open the official website to see the following paragraph:
ab is a tool for benchmarking your Apache Hypertext Transfer Protocol (HTTP) server. It is designed to give you an impression of how your current Apache installation performs. This especially shows you how many requests per second your Apache installation is capable of serving.
Apache bench is a web stress testing tool provided with Apache server, abbreviated as ab. AB is also a line tool with low local requirements for initiating load. According to AB, many concurrent access threads can be created to simulate multiple visitors accessing a URL address at the same time. Therefore, it can be used to test the load pressure of the target server. In general, AB tool is small and simple, and can learn quickly. It can provide the required basic performance indicators, but it has no graphical results and cannot be monitored.
Comparison of jmeter and ab
There are a lot of introductions on the Internet. I collected and found the summary of a great God (I won't say much about it, you can baidu by yourself)
1. JMeter is a complete request and return, while AB only sends the request and does not process the return, but the request is sent successfully or failed. Therefore, in terms of accuracy, JMeter is more accurate, while AB is faster, which can generate more access requests with the least machine resources;
2. Jmeter itself supports assertions, variable parameters and input of CSV data sets, and can set more flexible test scenarios, while AB does not support it (I didn't expect it at the moment);
3. Jmeter can provide more detailed statistical result data, such as interface error information, single thread request time, etc., while AB does not support it;
4. Jmeter does not support accurate time pressure measurement, such as pressure measurement for 10 minutes, but AB supports it;
5. Jmeter supports distributed pressure measurement clusters and functions, but AB does not;
6. The software itself consumes resources: Jmeter takes more time and resources than AB because it is heavy and counts a lot of result data. AB is ultra lightweight and is very suitable for single interface pressure testing in the process of development and testing.
Because this test is only conducted for a single interface, and there is just a free linux machine in hand, AB is selected after comprehensive consideration. There is no more nonsense. Here are some explanations on the use of ab.
Use of AB
The official website introduces the use of ab in detail. We can check the official website address:
Here are some excerpts. English is relatively simple, so there is no translation.
ab [ -A auth-username:password ] [ -b windowsize ] [ -B local-address ] [ -c concurrency ] [ -C cookie-name=value ] [ -d ] [ -e csv-file ] [ -E client-certificate file ] [ -f protocol ] [ -g gnuplot-file ] [ -h ] [ -H custom-header ] [ -i ] [ -k ] [ -l ] [ -m HTTP-method ] [ -n requests ] [ -p POST-file ] [ -P proxy-auth-username:password ] [ -q ] [ -r ] [ -s timeout ] [ -S ] [ -t timelimit ] [ -T content-type ] [ -u PUT-file ] [ -v verbosity] [ -V ] [ -w ] [ -x <table>-attributes ] [ -X proxy[:port] ] [ -y <tr>-attributes ] [ -z <td>-attributes ] [ -Z ciphersuite ] [http[s]://]hostname[:port]/path
- -A auth-username:password
Supply BASIC Authentication credentials to the server. The username and password are separated by a single : and sent on the wire base64 encoded. The string is sent regardless of whether the server needs it (i.e., has sent an 401 authentication needed).
- -b windowsize
Size of TCP send/receive buffer, in bytes.
- -B local-address
Address to bind to when making outgoing connections.
- -c concurrency
Number of multiple requests to perform at a time. Default is one request at a time.
- -C cookie-name=value
Add a Cookie: line to the request. The argument is typically in the form of a name=value pair. This field is repeatable.
Do not display the "percentage served within XX [ms] table". (legacy support).
- -e csv-file
Write a Comma separated value (CSV) file which contains for each percentage (from 1% to 100%) the time (in milliseconds) it took to serve that percentage of the requests. This is usually more useful than the 'gnuplot' file; as the results are already 'binned'.
- -E client-certificate-file
When connecting to an SSL website, use the provided client certificate in PEM format to authenticate with the server. The file is expected to contain the client certificate, followed by intermediate certificates, followed by the private key. Available in 2.4.36 and later.
- -f protocol
Specify SSL/TLS protocol (SSL2, SSL3, TLS1, TLS1.1, TLS1.2, or ALL). TLS1.1 and TLS1.2 support available in 2.4.4 and later.
- -g gnuplot-file
Write all measured values out as a 'gnuplot' or TSV (Tab separate values) file. This file can easily be imported into packages like Gnuplot, IDL, Mathematica, Igor or even Excel. The labels are on the first line of the file.
Display usage information.
- -H custom-header
Append extra headers to the request. The argument is typically in the form of a valid header line, containing a colon-separated field-value pair (i.e., "Accept-Encoding: zip/zop;8bit").
Do HEAD requests instead of GET.
Enable the HTTP KeepAlive feature, i.e., perform multiple requests within one HTTP session. Default is no KeepAlive.
Do not report errors if the length of the responses is not constant. This can be useful for dynamic pages. Available in 2.4.7 and later.
- -m HTTP-method
Custom HTTP method for the requests. Available in 2.4.10 and later.
- -n requests
Number of requests to perform for the benchmarking session. The default is to just perform a single request which usually leads to non-representative benchmarking results.
- -p POST-file
File containing data to POST. Remember to also set -T.
- -P proxy-auth-username:password
Supply BASIC Authentication credentials to a proxy en-route. The username and password are separated by a single : and sent on the wire base64 encoded. The string is sent regardless of whether the proxy needs it (i.e., has sent an 407 proxy authentication needed).
When processing more than 150 requests, ab outputs a progress count on stderr every 10% or 100 requests or so. The -q flag will suppress these messages.
Don't exit on socket receive errors.
- -s timeout
Maximum number of seconds to wait before the socket times out. Default is 30 seconds. Available in 2.4.4 and later.
Do not display the median and standard deviation values, nor display the warning/error messages when the average and median are more than one or two times the standard deviation apart. And default to the min/avg/max values. (legacy support).
- -t timelimit
Maximum number of seconds to spend for benchmarking. This implies a -n 50000 internally. Use this to benchmark the server within a fixed total amount of time. Per default there is no timelimit.
- -T content-type
Content-type header to use for POST/PUT data, eg. application/x-www-form-urlencoded. Default is text/plain.
- -u PUT-file
File containing data to PUT. Remember to also set -T.
- -v verbosity
Set verbosity level - 4 and above prints information on headers, 3 and above prints response codes (404, 200, etc.), 2 and above prints warnings and info.
Display version number and exit.
Print out results in HTML tables. Default table is two columns wide, with a white background.
- -x -attributes
String to use as attributes for table. Attributes are inserted table here.
- -X proxy[:port]
Use a proxy server for the requests.
- -y -attributes
String to use as attributes for tr.
- -z -attributes
String to use as attributes for tb
- -Z ciphersuite
Specify SSL/TLS cipher suite (See openssl ciphers)
The value, if any, returned in the server HTTP header of the first successful response. This includes all characters in the header from beginning to the point a character with decimal value of 32 (most notably: a space or CR/LF) is detected.
The DNS or IP address given on the command line
The port to which ab is connecting. If no port is given on the command line, this will default to 80 for http and 443 for https.
The protocol parameters negotiated between the client and server. This will only be printed if SSL is used.
The request URI parsed from the command line string.
This is the size in bytes of the first successfully returned document. If the document length changes during testing, the response is considered an error.
The number of concurrent clients used during the test
Time taken for tests
This is the time taken from the moment the first socket connection is created to the moment the last response is received
The number of successful responses received
The number of requests that were considered a failure. If the number is greater than zero, another line will be printed showing the number of requests that failed due to connecting, reading, incorrect content length, or exceptions.
The number of errors that failed during write (broken pipe).
The number of responses that were not in the 200 series of response codes. If all responses were 200, this field is not printed.
The number of connections that resulted in Keep-Alive requests
Total body sent
If configured to send data as part of the test, this is the total number of bytes sent during the tests. This field is omitted if the test did not include a body to send.
The total number of bytes received from the server. This number is essentially the number of bytes sent over the wire.
The total number of document bytes received from the server. This number excludes bytes received in HTTP headers
Requests per second
This is the number of requests per second. This value is the result of dividing the number of requests by the total time taken
Time per request
The average time spent per request. The first value is calculated with the formula concurrency * timetaken * 1000 / done while the second value is calculated with the formula timetaken * 1000 / done
The rate of transfer as calculated by the formula totalread / 1024 / timetaken
give an example
As can be seen from the above, ab supports many parameters, but generally speaking, only - n (total number of requests sent) and - c (number of concurrent threads requested) parameters are necessary.
As can be seen from the columns in the figure, 200 threads were started concurrently this time, running a total of 2000 times (in order to avoid disclosing information security, the interface was treated with privacy).
As a result, we observed
Requests for second is 722.77, that is, 722.77 requests are processed every second;
Time per request is 276.714, that is, the average time of the request is 276.714ms;
In addition, the time-consuming indicators of 99 lines, 98 lines and 95 lines can also be used;
In particular, if the NON-2xx responses are 2000 and the total number of requests is 2000, it means that all the 2000 requests returned are non-2000. It is necessary to consider whether the requests returned are all wrong;