This demo is a dellr620 physical server with centos7.6 x86_64-bit
The hardware-related information is as follows:
[root@mysql-redis scripts]# sh hw_msg1.sh +--------------------------------------------------------------+ | This Machine's Hyper-Threading is Enabled(recommend disable) | +--------------------------------------------------------------+ Systembit : 64 MEM info : 20*4096 MB Disk_totle : Pro_SN_name : Product Name: PowerEdge R620 Serial Number: 5RV2962 System name : \S Board_SN name: Product Name: Serial Number: CPU model : 32 Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz CPU_phy_num : 2 CPU_core_num : 8 CPU_logic_num: 16 Netcard info : Broadcom Inc. and subsidiaries NetXtreme BCM5720 Gigabit Ethernet PCIe +++++++++++++++++++++++++++++++++++ <<raid informations>> Virtual Drive: 0 (Target Id: 0) Name : RAID Level : Primary-0, Secondary-0, RAID Level Qualifier-0 Size : 893.75 GB State : Optimal Strip Size : 64 KB Number Of Drives : 1 -- Virtual Drive Information: Virtual Drive: 1 (Target Id: 1) Name : RAID Level : Primary-0, Secondary-0, RAID Level Qualifier-0 Size : 223.0 GB State : Optimal Strip Size : 64 KB Number Of Drives : 1 +++++++++++++++++++++++++++++++++++ <<Virtual Drive Information>> Device Id: 3 PD Type: SATA Raw Size: 894.252 GB [0x6fc81ab0 Sectors] Inquiry Data: 56AB609UKCDU TOSHIBA Q300. SAFM12.3 Device Speed: 6.0Gb/s Drive Temperature : N/A Device Id: 0 PD Type: SATA Raw Size: 223.570 GB [0x1bf244b0 Sectors] Inquiry Data: PHDV723500E8240AGN INTEL SSDSC2BB240G7 N2010112 Device Speed: 6.0Gb/s Drive Temperature :25C (77.00 F)2. Install fio
1.redhat linux system direct yum installation
yum install fio.x86_64
With direct yum installation on centos7.6, the default version is fio.3.7
2. Source Compilation and Installation
In order to test asynchronous io, libaio needs to be installed
yum install libaio-devel wget -P /data/soft http://github.com/axboe/fio/archive/fio-3.7.tar.gz tar xf /data/soft/fio.3.7.tar.gz -C /usr/local/ cd /usr/local/fio.3.7 ./configure make && make install3. Introduction of Important Parameters
Fio's parameters vary from release to release, but here's only a list of the most important parameters that are commonly used everyday
filename=/dev/sdb1 or/data/soft/test1: Specify the name of the test hard disk or the name of the test file (never select a system device test, which will cause the system to crash)
direct=1: The test process bypasses the machine's own buffer to make the test results more accurate.
rw=randwrite:Write randomly.rw Available Values: randwrite/randread/read/write/randrw/trim/randtrim/trim write; Random Write, Random Read, Sequential Read, Sequential Write, Mixed Random Read and Write,
bs=16k: The size of a single I/O block is 16KB. The size of a single I/O block has a greater impact on IOPS. Generally speaking, to get the largest IOPS, the smaller the bs, the better.Of course, this is also related to the smallest size of the file system.Generally, the smallest file partition size under linux is 512B,1KB,2KB,4KB,8KB and so on, which can be selected according to the file system type.
size=5g: Test file size is 5GB
Ioengine=libaio: The I/O engine uses libaio, an asynchronous I/O engine that submits a batch of I/Os at a time and then waits for them to complete, which reduces the number of interactions and is more efficient.FIO supports at least 13 different I/O engines.sync,mmap,libaio,network, etc.
Iodepth=4: The I/O queue depth is adjusted based on the degree of parallelism of the device.There are usually two ways of I/O access.That is, synchronous I/O, and asynchronous I/O. Synchronous I/O can only send one I/O request at a time and wait for the kernel to complete before returning.This is always less than 1 for a single thread. If you want iodepth to be greater than 1, you can do so through multithreaded concurrent execution.Asynchronous I/O submits multiple I/O requests at once.Wait for I/O to finish or harvest once at intervals.Iodepth is commonly used in asynchronous I/O models (libaio) to specify how many I/O requests are initiated at a time.Flash storage generally has multiple parallel units.Supports multiple I/O concurrent execution.I/O for better asynchronization
-numjobs=4:Use four threads to test the hard disk device For Raid devices, such as raid1,raid5, or devices with high parallelism, consider increasing the number of test threads-numjobs to test appropriately.
runtime=1000: The test time is 1000s. Works with the parameter size, which is a double limit, i.e. stops testing if any limit is reached.
rwmixread=70: In mixed read-write mode, 70% read
group_reporting: Set the display results, specify group_reporting summarizes information for each process.
-name iops_randwrite: Rename the test iops_randwrite
--outputTestResult.log: Test results specify output to fileTestResult.login
You can use fio simulation to test different types of io requests, including random write, sequential write, random read, sequential read.Mixed Random Read-Write
4.1 Write randomly:
time fio -rw=randwrite -bs=4k -runtime=60 -iodepth 1 -numjobs=4 -size=5G -filename /data/soft/test1 -ioengine libaio -direct=1 -group_reporting -name iops_randwrite --output TestResult.log
Command description:
A direct asynchronous random write (libaio) test with 4 threads on the / data/soft/test1 file for 60s, a queue depth of 1, and a block size of 4 kb, named iops_randwrite.Output results are not displayed by four jobs but are summarized by group, resulting in IOPS randomly written from this file system under this pressure.For Raid devices, such as raid1,raid5, or devices with high parallelism, consider appropriately increasing the number of test threads - numjobs to test.
--outputTestResult.logOutput to specified log
Here is a random write of the specific execution process:
[root@mysql-redis soft]# time fio -rw=randwrite -bs=4k -runtime=60 -iodepth 1 -numjobs=4 -size=5G -filename /data/soft/test1 -ioengine libaio -direct=1 -group_reporting -name iops_randwrite iops_randwrite: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1 ... fio-3.7 Starting 4 processes iops_randwrite: Laying out IO file (1 file / 5120MiB) Jobs: 4 (f=4): [w(4)][100.0%][r=0KiB/s,w=68.2MiB/s][r=0,w=17.5k IOPS][eta 00m:00s] iops_randwrite: (groupid=0, jobs=4): err= 0: pid=13505: Thu May 21 18:50:33 2020 write: IOPS=39.2k, BW=153MiB/s (160MB/s)(9182MiB/60001msec) slat (usec): min=6, max=27813, avg=17.42, stdev=36.21 clat (nsec): min=1177, max=123217k, avg=82183.17, stdev=323538.76 lat (usec): min=53, max=123230, avg=99.84, stdev=325.69 clat percentiles (usec): | 1.00th=[ 47], 5.00th=[ 51], 10.00th=[ 57], 20.00th=[ 62], | 30.00th=[ 65], 40.00th=[ 70], 50.00th=[ 75], 60.00th=[ 79], | 70.00th=[ 84], 80.00th=[ 91], 90.00th=[ 103], 95.00th=[ 117], | 99.00th=[ 161], 99.50th=[ 206], 99.90th=[ 330], 99.95th=[ 619], | 99.99th=[14615] bw ( KiB/s): min=17536, max=45304, per=25.12%, avg=39354.01, stdev=5145.44, samples=476 iops : min= 4384, max=11326, avg=9838.50, stdev=1286.36, samples=476 lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=4.22% lat (usec) : 100=84.07%, 250=11.50%, 500=0.12%, 750=0.03%, 1000=0.01% lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.03%, 50=0.01% lat (msec) : 250=0.01% cpu : usr=4.33%, sys=19.34%, ctx=2824889, majf=0, minf=1185 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,2350467,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 Run status group 0 (all jobs): WRITE: bw=153MiB/s (160MB/s), 153MiB/s-153MiB/s (160MB/s-160MB/s), io=9182MiB (9628MB), run=60001-60001msec Disk stats (read/write): sda: ios=0/2349456, merge=0/0, ticks=0/167181, in_queue=166516, util=96.46% real 1m0.706s user 0m11.167s sys 0m47.017s
4.2 Random reading:
time fio -readonly -rw=randread -bs=4k -runtime=60 -iodepth 1 -numjobs=2 -filename /data/soft/test2 -ioengine libaio -direct=1 -group_reporting -name iops_randread
Instructions on command parameters:
Command description: A direct asynchronous random read (libaio) test named iops_with a duration of 60s, a queue depth of 1, and a block size of 4 KB is performed on the / data/soft/test1 file using two threadsRandread. Output results are not displayed by two jobs, but are summarized by the group, resulting in IOPS randomly read from this file system under this pressure
Here is a random reading of the specific execution process:
[root@mysql-redis39 ~]# time fio -readonly -rw=randread -bs=4k -runtime=60 -iodepth 1 -numjobs=2 -filename /data/soft/test2 -ioengine libaio -direct=1 -group_reporting -name iops_randread iops_randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1 ... fio-3.7 Starting 2 processes Jobs: 2 (f=2): [r(2)][100.0%][r=36.7MiB/s,w=0KiB/s][r=9389,w=0 IOPS][eta 00m:00s] iops_randread: (groupid=0, jobs=2): err= 0: pid=24566: Fri May 22 06:21:46 2020 read: IOPS=9358, BW=36.6MiB/s (38.3MB/s)(2193MiB/60001msec) slat (usec): min=7, max=1480, avg=23.09, stdev= 8.09 clat (usec): min=8, max=123984, avg=185.97, stdev=570.65 lat (usec): min=136, max=123999, avg=209.50, stdev=570.70 clat percentiles (usec): | 1.00th=[ 139], 5.00th=[ 141], 10.00th=[ 145], 20.00th=[ 169], | 30.00th=[ 174], 40.00th=[ 178], 50.00th=[ 182], 60.00th=[ 188], | 70.00th=[ 198], 80.00th=[ 204], 90.00th=[ 212], 95.00th=[ 219], | 99.00th=[ 255], 99.50th=[ 273], 99.90th=[ 322], 99.95th=[ 537], | 99.99th=[ 1549] bw ( KiB/s): min=13824, max=19616, per=49.99%, avg=18713.87, stdev=1060.66, samples=238 iops : min= 3456, max= 4904, avg=4678.43, stdev=265.16, samples=238 lat (usec) : 10=0.01%, 100=0.01%, 250=98.87%, 500=1.08%, 750=0.02% lat (usec) : 1000=0.01% lat (msec) : 2=0.02%, 4=0.01%, 20=0.01%, 250=0.01% cpu : usr=4.11%, sys=15.44%, ctx=561546, majf=0, minf=498 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=561503,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 Run status group 0 (all jobs): READ: bw=36.6MiB/s (38.3MB/s), 36.6MiB/s-36.6MiB/s (38.3MB/s-38.3MB/s), io=2193MiB (2300MB), run=60001-60001msec Disk stats (read/write): sda: ios=560393/2, merge=0/0, ticks=102314/0, in_queue=101910, util=97.80% real 1m0.706s user 0m5.649s sys 0m19.196s
The above indicators need attention:
Read: iops=9358, BW=36.6MiB/s (38.3MB/s) (2193MiB/60001msec) io=2193MiB (2300MB) These parameters give you insight into iops and traffic
Lat(usec): min=136, max=123999, avg=209.50, stdev=570.70 These parameters also need to be noted: the io delay averages 209.5 subtle.
4.3 Write in sequence:
time fio -rw=write -bs=1m -runtime=60 -iodepth 1 -numjobs=5 -size=5G -filename /data/soft/test3 -ioengine libaio -direct=1 -group_reporting -name bw_write
Description: A direct asynchronous sequential write (libaio) stress test with a block size of 1 M and a queue depth of 1 is named bw_using four threads for 60s Write.Output results are not shown separately by four job s, but are presented as a group summary.
[root@mysql-redis ~]# time fio -rw=write -bs=1m -runtime=60 -iodepth 1 -numjobs=5 -size=5G -filename /data/soft/test3 -ioengine libaio -direct=1 -group_reporting -name bw_write bw_write: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=1 ... fio-3.7 Starting 5 processes bw_write: Laying out IO file (1 file / 5120MiB) Jobs: 5 (f=5): [W(5)][100.0%][r=0KiB/s,w=184MiB/s][r=0,w=184 IOPS][eta 00m:00s] bw_write: (groupid=0, jobs=5): err= 0: pid=26278: Fri May 22 06:44:38 2020 write: IOPS=319, BW=320MiB/s (335MB/s)(18.7GiB/60009msec) slat (usec): min=56, max=303, avg=140.02, stdev=29.86 clat (msec): min=3, max=1092, avg=15.49, stdev=67.68 lat (msec): min=3, max=1093, avg=15.63, stdev=67.68
4.4 Sequential reading:
time fio -readonly -rw=read -bs=1m -runtime=60 -iodepth 1 -numjobs=2 -filename /data/soft/test2 -ioengine libaio -direct=1 -group_reporting -name bw_read
[root@mysql-redis ~]# time fio -readonly -rw=read -bs=1m -runtime=60 -iodepth 1 -numjobs=2 -filename /data/soft/test2 -ioengine libaio -direct=1 -group_reporting -name bw_read bw_read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=1 ... fio-3.7 Starting 2 processes Jobs: 2 (f=2): [R(2)][100.0%][r=259MiB/s,w=0KiB/s][r=259,w=0 IOPS][eta 00m:00s] bw_read: (groupid=0, jobs=2): err= 0: pid=26789: Fri May 22 06:51:01 2020 read: IOPS=255, BW=256MiB/s (268MB/s)(10.0GiB/40034msec) slat (usec): min=95, max=1149, avg=245.64, stdev=42.23 clat (msec): min=4, max=130, avg= 7.57, stdev= 2.98 lat (msec): min=4, max=130, avg= 7.81, stdev= 2.98
4.5 Mixed Read-Write
time fio -rw=randrw -rwmixread=70 -bs=16k -runtime=60 -iodepth 1 -numjobs=2 -filename /data/soft/test2 -ioengine libaio -direct=1 -group_reporting -name iops_randrw
Command description:
Two threads did a direct asynchronous mixed read-write (libaio) stress test on the / data/soft/test2 file for 60s with a queue depth of 1 and a block size of 16 kb. The read-write ratio was pseudo 7:3 and the test was named iops_randrw
rwmixwrite=30 means that in mixed read-write mode, writing accounts for 30%.
-rwmixread=70 means 70% reading in mixed read-write mode.
Mixed Read-Write output:
[root@mysql-redis ~]# time fio -rw=randrw -rwmixread=70 -bs=16k -runtime=60 -iodepth 1 -numjobs=2 -filename /data/soft/test2 -ioengine libaio -direct=1 -group_reporting -name iops_randrw iops_randrw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=libaio, iodepth=1 ... fio-3.7 Starting 2 processes Jobs: 2 (f=2): [m(2)][100.0%][r=62.0MiB/s,w=27.2MiB/s][r=4030,w=1740 IOPS][eta 00m:00s] iops_randrw: (groupid=0, jobs=2): err= 0: pid=27095: Fri May 22 06:55:08 2020 read: IOPS=3931, BW=61.4MiB/s (64.4MB/s)(3686MiB/60001msec) slat (usec): min=8, max=6353, avg=36.20, stdev=19.77 clat (nsec): min=1648, max=123466k, avg=346175.56, stdev=725094.33 lat (usec): min=12, max=123516, avg=382.99, stdev=725.55 clat percentiles (usec): | 1.00th=[ 215], 5.00th=[ 241], 10.00th=[ 255], 20.00th=[ 277], | 30.00th=[ 293], 40.00th=[ 310], 50.00th=[ 322], 60.00th=[ 334], | 70.00th=[ 351], 80.00th=[ 375], 90.00th=[ 412], 95.00th=[ 449], | 99.00th=[ 1221], 99.50th=[ 1467], 99.90th=[ 1631], 99.95th=[ 1827], | 99.99th=[16712] bw ( KiB/s): min=23328, max=33792, per=49.97%, avg=31431.39, stdev=1649.11, samples=238 iops : min= 1458, max= 2112, avg=1964.44, stdev=103.06, samples=238 write: IOPS=1686, BW=26.4MiB/s (27.6MB/s)(1581MiB/60001msec) slat (usec): min=12, max=2005, avg=42.06, stdev=16.09 clat (usec): min=3, max=123270, avg=225.73, stdev=432.84 lat (usec): min=86, max=123320, avg=268.44, stdev=433.42 clat percentiles (usec): | 1.00th=[ 78], 5.00th=[ 85], 10.00th=[ 100], 20.00th=[ 129], | 30.00th=[ 149], 40.00th=[ 174], 50.00th=[ 202], 60.00th=[ 233], | 70.00th=[ 265], 80.00th=[ 306], 90.00th=[ 359], 95.00th=[ 408], | 99.00th=[ 510], 99.50th=[ 1090], 99.90th=[ 1549], 99.95th=[ 1614], | 99.99th=[ 5080] bw ( KiB/s): min= 9152, max=15329, per=49.97%, avg=13484.37, stdev=892.91, samples=238 iops : min= 572, max= 958, avg=842.76, stdev=55.80, samples=238 lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 50=0.01%, 100=3.06% lat (usec) : 250=22.22%, 500=72.91%, 750=0.76%, 1000=0.05% lat (msec) : 2=0.96%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01% lat (msec) : 250=0.01% cpu : usr=4.64%, sys=15.91%, ctx=346233, majf=0, minf=266 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=235884,101198,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 Run status group 0 (all jobs): READ: bw=61.4MiB/s (64.4MB/s), 61.4MiB/s-61.4MiB/s (64.4MB/s-64.4MB/s), io=3686MiB (3865MB), run=60001-60001msec WRITE: bw=26.4MiB/s (27.6MB/s), 26.4MiB/s-26.4MiB/s (27.6MB/s-27.6MB/s), io=1581MiB (1658MB), run=60001-60001msec Disk stats (read/write): sda: ios=274477/101294, merge=0/0, ticks=88860/19119, in_queue=107576, util=95.82% real 1m0.690s user 0m6.300s sys 0m19.718s
4.6SSD Performance Test Profile
The configuration file is as follows:
[root@mysql-redis scripts]# cat /data/soft/ssd-test.fio ##2020-05-24 [global] bs=4k ioengine=libaio iodepth=4 numjobs=1 group_reporting size=5g direct=1 runtime=60 directory=/data/soft filename=ssd.test.file #inflate-log=/data/soft/TestResult.log #output=/data/soft/TestResult.log [seq-read] name=iops_seqread rw=read stonewall [rand-read] name=iops_randread rw=randread stonewall [seq-write] name=iops_seqwrite rw=write stonewall [rand-write] name=iops_randwrite rw=randwrite stonewall [randrw] name=iops_randrw rwmixread=70 rw=randrw stonewall
Here is what the fio test specifies for the profile output:
[root@mysql-redis soft]# fio ssd-test.fio iops_seqread: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4 iops_randread: (g=1): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4 iops_seqwrite: (g=2): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4 iops_randwrite: (g=3): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4 iops_randrw: (g=4): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4 fio-3.7 Starting 5 processes iops_seqread: Laying out IO file (1 file / 5120MiB) Jobs: 1 (f=1): [_(4),m(1)][90.5%][r=47.8MiB/s,w=20.7MiB/s][r=12.2k,w=5308 IOPS][eta 00m:21s] iops_seqread: (groupid=0, jobs=1): err= 0: pid=2142: Sun May 24 20:06:20 2020 read: IOPS=57.5k, BW=225MiB/s (236MB/s)(5120MiB/22782msec) slat (usec): min=3, max=1943, avg= 5.13, stdev= 2.21 clat (usec): min=25, max=12467, avg=63.13, stdev=202.65 lat (usec): min=40, max=12472, avg=68.41, stdev=202.72 clat percentiles (usec): | 1.00th=[ 45], 5.00th=[ 53], 10.00th=[ 55], 20.00th=[ 56], | 30.00th=[ 56], 40.00th=[ 56], 50.00th=[ 57], 60.00th=[ 57], | 70.00th=[ 57], 80.00th=[ 58], 90.00th=[ 59], 95.00th=[ 62], | 99.00th=[ 69], 99.50th=[ 79], 99.90th=[ 2073], 99.95th=[ 5080], | 99.99th=[10814] bw ( KiB/s): min=43579, max=193812, per=75.80%, avg=174446.82, stdev=47419.87, samples=45 iops : min=10894, max=48453, avg=43611.36, stdev=11855.06, samples=45 lat (usec) : 50=3.39%, 100=96.25%, 250=0.08%, 500=0.07%, 750=0.04% lat (usec) : 1000=0.02% lat (msec) : 2=0.06%, 4=0.04%, 10=0.05%, 20=0.01% cpu : usr=19.83%, sys=39.29%, ctx=1222141, majf=0, minf=114 IO depths : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1310720,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=4 iops_randread: (groupid=1, jobs=1): err= 0: pid=2170: Sun May 24 20:06:20 2020 read: IOPS=21.7k, BW=84.9MiB/s (89.0MB/s)(5094MiB/60001msec) slat (usec): min=3, max=1068, avg=12.62, stdev= 6.14 clat (usec): min=42, max=38568, avg=167.76, stdev=315.76 lat (usec): min=62, max=38588, avg=180.75, stdev=316.07 clat percentiles (usec): | 1.00th=[ 130], 5.00th=[ 135], 10.00th=[ 137], 20.00th=[ 141], | 30.00th=[ 143], 40.00th=[ 147], 50.00th=[ 151], 60.00th=[ 155], | 70.00th=[ 161], 80.00th=[ 172], 90.00th=[ 192], 95.00th=[ 206], | 99.00th=[ 251], 99.50th=[ 273], 99.90th=[ 3982], 99.95th=[ 7308], | 99.99th=[14091] bw ( KiB/s): min= 5397, max=74196, per=79.00%, avg=68682.74, stdev=15713.88, samples=119 iops : min= 1349, max=18549, avg=17170.32, stdev=3928.48, samples=119 lat (usec) : 50=0.01%, 100=0.02%, 250=98.94%, 500=0.82%, 750=0.02% lat (usec) : 1000=0.02% lat (msec) : 2=0.03%, 4=0.04%, 10=0.07%, 20=0.02%, 50=0.01% cpu : usr=11.52%, sys=37.46%, ctx=668232, majf=0, minf=319 IO depths : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1304064,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=4 iops_seqwrite: (groupid=2, jobs=1): err= 0: pid=2231: Sun May 24 20:06:20 2020 write: IOPS=47.1k, BW=184MiB/s (193MB/s)(5120MiB/27848msec) slat (usec): min=4, max=139, avg= 7.50, stdev= 2.96 clat (usec): min=27, max=15374, avg=76.14, stdev=278.53 lat (usec): min=49, max=15381, avg=83.78, stdev=279.26 clat percentiles (usec): | 1.00th=[ 56], 5.00th=[ 59], 10.00th=[ 60], 20.00th=[ 61], | 30.00th=[ 62], 40.00th=[ 64], 50.00th=[ 65], 60.00th=[ 67], | 70.00th=[ 68], 80.00th=[ 71], 90.00th=[ 73], 95.00th=[ 74], | 99.00th=[ 81], 99.50th=[ 89], 99.90th=[ 3326], 99.95th=[ 3392], | 99.99th=[15139] bw ( KiB/s): min= 3769, max=175262, per=80.83%, avg=152166.87, stdev=43009.10, samples=55 iops : min= 942, max=43815, avg=38041.35, stdev=10752.22, samples=55 lat (usec) : 50=0.24%, 100=99.43%, 250=0.07%, 500=0.01%, 750=0.01% lat (usec) : 1000=0.01% lat (msec) : 2=0.01%, 4=0.21%, 10=0.01%, 20=0.03% cpu : usr=11.79%, sys=43.48%, ctx=652945, majf=0, minf=95 IO depths : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,1310720,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=4 iops_randwrite: (groupid=3, jobs=1): err= 0: pid=2262: Sun May 24 20:06:20 2020 write: IOPS=47.2k, BW=185MiB/s (194MB/s)(5120MiB/27743msec) slat (usec): min=4, max=415, avg= 7.49, stdev= 2.82 clat (usec): min=26, max=21032, avg=75.63, stdev=276.91 lat (usec): min=48, max=21058, avg=83.27, stdev=277.60 clat percentiles (usec): | 1.00th=[ 56], 5.00th=[ 60], 10.00th=[ 60], 20.00th=[ 62], | 30.00th=[ 62], 40.00th=[ 64], 50.00th=[ 65], 60.00th=[ 67], | 70.00th=[ 68], 80.00th=[ 70], 90.00th=[ 73], 95.00th=[ 74], | 99.00th=[ 80], 99.50th=[ 86], 99.90th=[ 3326], 99.95th=[ 3392], | 99.99th=[15139] bw ( KiB/s): min= 3175, max=142274, per=68.21%, avg=128901.20, stdev=33671.41, samples=55 iops : min= 793, max=35568, avg=32224.95, stdev=8417.81, samples=55 lat (usec) : 50=0.21%, 100=99.50%, 250=0.04%, 500=0.01%, 750=0.01% lat (usec) : 1000=0.01% lat (msec) : 2=0.01%, 4=0.20%, 10=0.01%, 20=0.02%, 50=0.01% cpu : usr=12.97%, sys=43.05%, ctx=654916, majf=0, minf=258 IO depths : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,1310720,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=4 iops_randrw: (groupid=4, jobs=1): err= 0: pid=2292: Sun May 24 20:06:20 2020 read: IOPS=11.3k, BW=44.3MiB/s (46.5MB/s)(2659MiB/60001msec) slat (usec): min=4, max=594, avg=13.08, stdev= 6.88 clat (usec): min=61, max=45073, avg=244.45, stdev=441.39 lat (usec): min=134, max=45115, avg=257.84, stdev=441.61 clat percentiles (usec): | 1.00th=[ 137], 5.00th=[ 143], 10.00th=[ 151], 20.00th=[ 176], | 30.00th=[ 190], 40.00th=[ 202], 50.00th=[ 212], 60.00th=[ 221], | 70.00th=[ 233], 80.00th=[ 258], 90.00th=[ 330], 95.00th=[ 359], | 99.00th=[ 474], 99.50th=[ 1270], 99.90th=[ 6521], 99.95th=[10028], | 99.99th=[17695] bw ( KiB/s): min= 3072, max=51824, per=99.92%, avg=45336.55, stdev=11250.66, samples=119 iops : min= 768, max=12958, avg=11334.12, stdev=2812.66, samples=119 write: IOPS=4860, BW=18.0MiB/s (19.9MB/s)(1139MiB/60001msec) slat (usec): min=6, max=874, avg=16.94, stdev= 7.42 clat (usec): min=40, max=31265, avg=192.80, stdev=413.19 lat (usec): min=56, max=31306, avg=210.09, stdev=413.32 clat percentiles (usec): | 1.00th=[ 52], 5.00th=[ 58], 10.00th=[ 62], 20.00th=[ 78], | 30.00th=[ 145], 40.00th=[ 169], 50.00th=[ 188], 60.00th=[ 202], | 70.00th=[ 212], 80.00th=[ 227], 90.00th=[ 251], 95.00th=[ 277], | 99.00th=[ 619], 99.50th=[ 1205], 99.90th=[ 5997], 99.95th=[ 9634], | 99.99th=[16581] bw ( KiB/s): min= 1184, max=22176, per=99.94%, avg=19428.64, stdev=4847.41, samples=119 iops : min= 296, max= 5544, avg=4857.13, stdev=1211.85, samples=119 lat (usec) : 50=0.12%, 100=7.22%, 250=74.36%, 500=17.22%, 750=0.24% lat (usec) : 1000=0.10% lat (msec) : 2=0.52%, 4=0.06%, 10=0.11%, 20=0.04%, 50=0.01% cpu : usr=7.88%, sys=29.46%, ctx=340101, majf=0, minf=374 IO depths : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=680630,291634,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=4 Run status group 0 (all jobs): READ: bw=225MiB/s (236MB/s), 225MiB/s-225MiB/s (236MB/s-236MB/s), io=5120MiB (5369MB), run=22782-22782msec Run status group 1 (all jobs): READ: bw=84.9MiB/s (89.0MB/s), 84.9MiB/s-84.9MiB/s (89.0MB/s-89.0MB/s), io=5094MiB (5341MB), run=60001-60001msec Run status group 2 (all jobs): WRITE: bw=184MiB/s (193MB/s), 184MiB/s-184MiB/s (193MB/s-193MB/s), io=5120MiB (5369MB), run=27848-27848msec Run status group 3 (all jobs): WRITE: bw=185MiB/s (194MB/s), 185MiB/s-185MiB/s (194MB/s-194MB/s), io=5120MiB (5369MB), run=27743-27743msec Run status group 4 (all jobs): READ: bw=44.3MiB/s (46.5MB/s), 44.3MiB/s-44.3MiB/s (46.5MB/s-46.5MB/s), io=2659MiB (2788MB), run=60001-60001msec WRITE: bw=18.0MiB/s (19.9MB/s), 18.0MiB/s-18.0MiB/s (19.9MB/s-19.9MB/s), io=1139MiB (1195MB), run=60001-60001msec Disk stats (read/write): sda: ios=3294785/2912828, merge=0/3, ticks=430527/238028, in_queue=666582, util=99.09% [root@mysql-redis soft]#V. Summary
Recommendations for fio test hard disks:
1. It is recommended that sequential I/O and larger blocksize be used to test the vomiting volume and delay of the device
2. It is recommended to use random I/O and smaller blocksize to test the IOPS and latency of the device
3. When configuring numjobs and io depth to test the underlying storage performance, it is recommended that you have a better understanding of whether synchronous or asynchronous I O is used (whether it is a multi-process concurrent i/o request or a batch of i/o requests submitted at once)
Tip: If you are interested in fio measuring disk performance, you can refer to ezfio (directly atGithub.comSearch ezfio)
Important note: fio supports direct I/O testing of bare devices. When testing bare devices, it is important to avoid the bare devices where the system is located, otherwise the system will run.
For example:
/dev/sdb/dev/sdb5/dev/sdb1 These devices are absolutely prohibited from fio testing.
At the same time, when testing the bare equipment of other devices, it is necessary to determine in advance whether there is important data on the bare equipment, and make a good backup of the important equipment in advance.
This is my simple learning summary of fio test hard disk for reference only. If there are users who directly copy the application causing losses, this blog does not bear any responsibility.
Reference documents:
http://www.ssdfans.com/?p=6087
https://linux.die.net/man/1/fio
https://blog.csdn.net/don_chiang709/article/details/92628623