Меряем производительность накопителей или снова про IOPS

Навеяно постом уважаемого amarao о том, как надо измерять производительность дисков.

Цель:


Протестировать производительность имеющихся в наличии средств хранения информации и убедиться в верности выбранной методики, а также понять разницу в производительности между разными видами накопителей, а также enterprise-level и consumer-level жёсткими дисками.

Оборудование:


  1. SD-карта Sandisk Class 10 UHS 1 Extreme Pro 8 GB (до 95 Мбайт/с чтение, до 90 Мбайт/с запись)
  2. SD-карта Team Class 10 32 GB (до 20 Мбайт/с)
  3. SD-карта Transcend 2GB без класса скорости
  4. SSD-диск OCZ-AGILITY3 60 GB
  5. SATA-диск consumer-level Hitachi Deskstar HDS723020BLA642 2 ТБ 7200 об/мин, 64 Мбайт
  6. SATA-диск enterprise-level Western Digital RE3 WD2502ABYS-23B7A0 250 GB 7200 об/мин 16 Мбайт
  7. SATA-диск consumer-level Seagate Barracuda 7200.11 ST3320613AS 320 GB 7200 об/мин 16 Mбайт
  8. CD-ROM
  9. RAM-диск /dev/ram в Linux


Методика тестирования:


Методика полностью описана в посте. Есть правда несколько не совсем понятных моментов:
Мы подбираем такую глубину параллельности операций, чтобы latency оставалось в разумных пределах.
Задача подобрать такой iodepth, чтобы avg.latency была меньше 10мс.

Так как в тестировании используется не СХД и не диски SAS, а различные накопители SATA, то параллельность нам измерять нету смысла.
Очищать диск перед каждым тестированием (dd if=/dev/zero of=/dev/sdz bs=2M oflag=direct) очень времязатратно, поэтому будем это делать перед тестированием один раз на каждый накопитель.
Тестировать весь диск полностью очень времязатратно, поэтому будем использовать тестирование в течении 30 секунд.
Итак, сформулируем методику тестирования для нашего случая:
Получить значение IOPS, выдаваемое накопителем при произвольном чтении и записи блоками по 4 Кбайт и задержке avg.latency не более 10 мс за время теста в 30 секунд. Также для полноты картины измерим скорость линейной записи.

Конфигурация fio на чтение:
[readtest]
blocksize=4k
filename=/dev/sda
rw=randread
direct=1
buffered=0
ioengine=libaio
iodepth=1024

runtime=30


Конфигурация fio на запись:
[writetest]
blocksize=4k
filename=/dev/sdz
rw=randwrite
direct=1
buffered=0
ioengine=libaio
iodepth=1024

runtime=30


Результаты:


SD-карта Sandisk Class 10 UHS 1 Extreme Pro 8 GB (до 95 Мбайт/с чтение, до 90 Мбайт/с запись)


Чтение: iodepth=1024, bw=4528.8KB/s, iops=1132, avg= 894.10 msec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1024
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [4484K/0K /s] [1121 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=3805
read: io=136464KB, bw=4528.8KB/s, iops=1132, runt= 30133msec
slat (usec): min=3, max=31548, avg=877.11, stdev=4880.99
clat (msec): min=119, max=1819, avg=894.10, stdev=157.70
lat (msec): min=149, max=1819, avg=894.98, stdev=157.76
clat percentiles (msec):
| 1.00th=[ 437], 5.00th=[ 742], 10.00th=[ 775], 20.00th=[ 807],
| 30.00th=[ 824], 40.00th=[ 840], 50.00th=[ 865], 60.00th=[ 881],
| 70.00th=[ 906], 80.00th=[ 963], 90.00th=[ 1090], 95.00th=[ 1205],
| 99.00th=[ 1450], 99.50th=[ 1549], 99.90th=[ 1663], 99.95th=[ 1696],
| 99.99th=[ 1778]
bw (KB/s): min= 5, max= 5089, per=98.25%, avg=4448.82, stdev=632.04
lat (msec): 250=0.38%, 500=0.85%, 750=5.19%, 1000=77.27%, 2000=16.31%
cpu: usr=0.31%, sys=0.57%, ctx=1195, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=99.8%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
issued: total=r=34116/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=136464KB, aggrb=4528KB/s, minb=4528KB/s, maxb=4528KB/s, mint=30133msec, maxt=30133msec

Disk stats (read/write):
sdb: ios=33865/0, merge=4/0, ticks=4258187/0, in_queue=4284190, util=99.72%

Чтение: iodepth=64, bw=4375.1KB/s, iops=1093, avg= 58.49 msec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=64
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [4252K/0K /s] [1063 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=3745
read: io=131544KB, bw=4375.1KB/s, iops=1093, runt= 30061msec
slat (usec): min=3, max=33, avg= 5.40, stdev= 2.58
clat (msec): min=1, max=358, avg=58.49, stdev=33.30
lat (msec): min=1, max=358, avg=58.50, stdev=33.30
clat percentiles (msec):
| 1.00th=[ 4], 5.00th=[ 8], 10.00th=[ 14], 20.00th=[ 25],
| 30.00th=[ 36], 40.00th=[ 47], 50.00th=[ 59], 60.00th=[ 70],
| 70.00th=[ 81], 80.00th=[ 92], 90.00th=[ 103], 95.00th=[ 112],
| 99.00th=[ 122], 99.50th=[ 125], 99.90th=[ 157], 99.95th=[ 198],
| 99.99th=[ 289]
bw (KB/s): min= 3752, max= 5024, per=99.93%, avg=4372.10, stdev=221.75
lat (msec): 2=0.01%, 4=1.90%, 10=5.34%, 20=8.70%, 50=26.80%
lat (msec): 100=44.58%, 250=12.65%, 500=0.02%
cpu: usr=0.60%, sys=1.54%, ctx=32894, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=99.8%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued: total=r=32886/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=131544KB, aggrb=4375KB/s, minb=4375KB/s, maxb=4375KB/s, mint=30061msec, maxt=30061msec

Disk stats (read/write):
sdb: ios=32731/0, merge=1/0, ticks=1911326/0, in_queue=1913834, util=99.71%

Чтение: iodepth=32, bw=4222.4KB/s, iops=1055, avg=30.31 msec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [4112K/0K /s] [1028 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=3764
read: io=126804KB, bw=4222.4KB/s, iops=1055, runt= 30032msec
slat (usec): min=3, max=40, avg= 4.54, stdev= 2.16
clat (msec): min=1, max=78, avg=30.31, stdev=16.37
lat (msec): min=1, max=78, avg=30.31, stdev=16.37
clat percentiles (usec):
| 1.00th=[ 2864], 5.00th=[ 5024], 10.00th=[ 7840], 20.00th=[13504],
| 30.00th=[19072], 40.00th=[24704], 50.00th=[30080], 60.00th=[35584],
| 70.00th=[41216], 80.00th=[46336], 90.00th=[52480], 95.00th=[56576],
| 99.00th=[62208], 99.50th=[63744], 99.90th=[68096], 99.95th=[70144],
| 99.99th=[75264]
bw (KB/s): min= 3904, max= 4808, per=100.00%, avg=4222.38, stdev=211.15
lat (msec): 2=0.01%, 4=3.33%, 10=10.52%, 20=17.80%, 50=54.29%
lat (msec): 100=14.06%
cpu: usr=0.68%, sys=1.01%, ctx=31715, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=99.9%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=31701/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=126804KB, aggrb=4222KB/s, minb=4222KB/s, maxb=4222KB/s, mint=30032msec, maxt=30032msec

Disk stats (read/write):
sdb: ios=31575/0, merge=0/0, ticks=956196/0, in_queue=956760, util=99.70%

Чтение: iodepth=16, bw=4097.2KB/s, iops=1024, avg=15.61 msec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=16
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [3976K/0K /s] [994 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=3771
read: io=122984KB, bw=4097.2KB/s, iops=1024, runt= 30017msec
slat (usec): min=3, max=34, avg= 4.53, stdev= 2.15
clat (msec): min=1, max=40, avg=15.61, stdev= 7.83
lat (msec): min=1, max=40, avg=15.62, stdev= 7.83
clat percentiles (usec):
| 1.00th=[ 2736], 5.00th=[ 3760], 10.00th=[ 5024], 20.00th=[ 7648],
| 30.00th=[10304], 40.00th=[12864], 50.00th=[15424], 60.00th=[18048],
| 70.00th=[20608], 80.00th=[23168], 90.00th=[26240], 95.00th=[28288],
| 99.00th=[32128], 99.50th=[33536], 99.90th=[36096], 99.95th=[37120],
| 99.99th=[39168]
bw (KB/s): min= 3944, max= 4728, per=100.00%, avg=4103.32, stdev=220.20
lat (msec): 2=0.01%, 4=6.07%, 10=22.92%, 20=39.08%, 50=31.93%
cpu: usr=0.59%, sys=1.15%, ctx=30756, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=30746/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=122984KB, aggrb=4097KB/s, minb=4097KB/s, maxb=4097KB/s, mint=30017msec, maxt=30017msec

Disk stats (read/write):
sdb: ios=30641/0, merge=0/0, ticks=478130/0, in_queue=478256, util=99.70%е

Чтение: iodepth=8, bw=4020.6KB/s, iops=1005, avg= 7.95 msec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=8
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [3888K/0K /s] [972 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=3778
read: io=120648KB, bw=4020.6KB/s, iops=1005, runt= 30008msec
slat (usec): min=3, max=124, avg= 5.39, stdev= 3.01
clat (msec): min=1, max=22, avg= 7.95, stdev= 3.37
lat (msec): min=1, max=22, avg= 7.96, stdev= 3.37
clat percentiles (usec):
| 1.00th=[ 2608], 5.00th=[ 2992], 10.00th=[ 3504], 20.00th=[ 4512],
| 30.00th=[ 5728], 40.00th=[ 6816], 50.00th=[ 7840], 60.00th=[ 8768],
| 70.00th=[ 9920], 80.00th=[11072], 90.00th=[12480], 95.00th=[13632],
| 99.00th=[15936], 99.50th=[16768], 99.90th=[18304], 99.95th=[18816],
| 99.99th=[20096]
bw (KB/s): min= 3872, max= 4640, per=100.00%, avg=4026.14, stdev=215.32
lat (msec): 2=0.01%, 4=13.98%, 10=57.35%, 20=28.66%, 50=0.01%
cpu: usr=0.96%, sys=1.63%, ctx=30168, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=30162/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=120648KB, aggrb=4020KB/s, minb=4020KB/s, maxb=4020KB/s, mint=30008msec, maxt=30008msec

Disk stats (read/write):
sdb: ios=30068/0, merge=0/0, ticks=238837/0, in_queue=238884, util=99.72%

Чтение: iodepth=4, bw=3977.8KB/s, iops=994, avg=4016.37 usec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=4
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [3860K/0K /s] [965 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=3785
read: io=119328KB, bw=3977.8KB/s, iops=994, runt= 30004msec
slat (usec): min=3, max=118, avg= 4.38, stdev= 2.36
clat (usec): min=1057, max=9386, avg=4016.37, stdev=983.10
lat (usec): min=1094, max=9390, avg=4021.16, stdev=983.13
clat percentiles (usec):
| 1.00th=[ 2608], 5.00th=[ 2736], 10.00th=[ 2896], 20.00th=[ 3120],
| 30.00th=[ 3248], 40.00th=[ 3728], 50.00th=[ 3984], 60.00th=[ 4128],
| 70.00th=[ 4320], 80.00th=[ 4896], 90.00th=[ 5280], 95.00th=[ 5984],
| 99.00th=[ 7008], 99.50th=[ 7264], 99.90th=[ 8256], 99.95th=[ 8512],
| 99.99th=[ 9152]
bw (KB/s): min= 3824, max= 4584, per=100.00%, avg=3983.19, stdev=212.25
lat (msec): 2=0.01%, 4=51.13%, 10=48.86%
cpu: usr=0.54%, sys=1.12%, ctx=29841, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=29832/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=119328KB, aggrb=3977KB/s, minb=3977KB/s, maxb=3977KB/s, mint=30004msec, maxt=30004msec

Disk stats (read/write):
sdb: ios=29742/0, merge=0/0, ticks=119303/0, in_queue=119300, util=99.71%

Чтение: iodepth=1, bw=3886.7KB/s, iops=971, avg= 1018.74 usec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [3752K/0K /s] [938 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=3796
read: io=116604KB, bw=3886.7KB/s, iops=971, runt= 30001msec
slat (usec): min=4, max=133, avg= 7.87, stdev= 3.84
clat (usec): min=676, max=1644, avg=1018.74, stdev=103.74
lat (usec): min=691, max=1660, avg=1027.08, stdev=104.32
clat percentiles (usec):
| 1.00th=[ 844], 5.00th=[ 852], 10.00th=[ 868], 20.00th=[ 884],
| 30.00th=[ 980], 40.00th=[ 996], 50.00th=[ 1012], 60.00th=[ 1096],
| 70.00th=[ 1112], 80.00th=[ 1112], 90.00th=[ 1128], 95.00th=[ 1144],
| 99.00th=[ 1192], 99.50th=[ 1224], 99.90th=[ 1240], 99.95th=[ 1352],
| 99.99th=[ 1416]
bw (KB/s): min= 3704, max= 4408, per=100.00%, avg=3892.61, stdev=197.78
lat (usec): 750=0.56%, 1000=46.23%
lat (msec): 2=53.21%
cpu: usr=0.54%, sys=1.95%, ctx=29155, majf=0, minf=0
IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=29151/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=116604KB, aggrb=3886KB/s, minb=3886KB/s, maxb=3886KB/s, mint=30001msec, maxt=30001msec

Disk stats (read/write):
sdb: ios=29065/0, merge=0/0, ticks=29446/0, in_queue=29433, util=98.12%


Измерение последовательного чтения:
root@host:~# dd if=/dev/sdb  of=/dev/null iflag=direct bs=2M
3790+0 records in
3790+0 records out
7948206080 bytes (7.9 GB) copied, 419.551 s, 18.9 MB/s


Анализ чтения:

При чтении чётко прослеживается закономерность: чем больше размер очереди, тем больше IOPS и больше задержки. Соответственно производительность накопителя на чтение: iodepth=8, bw=4020.6KB/s, iops=1005, avg= 7.95 msec.

Запись: iodepth=1024, bw=358153 B/s, iops=87, avg=10136.40 msec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1024
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [0.1% done] [0K/4K /s] [0 /1 iops] [eta 09h:47m:09s]
writetest: (groupid=0, jobs=1): err= 0: pid=3817
write: io=11136KB, bw=358153 B/s, iops=87, runt= 31839msec
slat (usec): min=3, max=596124, avg=10899.63, stdev=65581.18
clat (msec): min=1487, max=16067, avg=10136.40, stdev=2980.73
lat (msec): min=1954, max=16067, avg=10147.30, stdev=2981.04
clat percentiles (msec):
| 1.00th=[ 1958], 5.00th=[ 3392], 10.00th=[ 4948], 20.00th=[ 8356],
| 30.00th=[ 9896], 40.00th=[10421], 50.00th=[10814], 60.00th=[11207],
| 70.00th=[11731], 80.00th=[12387], 90.00th=[13304], 95.00th=[13698],
| 99.00th=[14353], 99.50th=[15008], 99.90th=[16057], 99.95th=[16057],
| 99.99th=[16057]
bw (KB/s): min= 0, max= 450, per=93.22%, avg=325.33, stdev=106.70
lat (msec): 2000=1.19%, >=2000=98.81%
cpu: usr=0.06%, sys=0.03%, ctx=211, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.3%, 16=0.6%, 32=1.1%, >=64=97.7%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
issued: total=r=0/w=0/d=2784, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=11136KB, aggrb=349KB/s, minb=349KB/s, maxb=349KB/s, mint=31839msec, maxt=31839msec

Disk stats (read/write):
sdb: ios=84/2770, merge=0/0, ticks=70/4360043, in_queue=4409040, util=99.94%

Запись: iodepth=64, bw=480082 B/s, iops=117, avg=545.79 msec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=64
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [0.2% done] [0K/36K /s] [0 /9 iops] [eta 04h:40m:56s]
writetest: (groupid=0, jobs=1): err= 0: pid=3849
write: io=14500KB, bw=480082 B/s, iops=117, runt= 30928msec
slat (usec): min=3, max=32, avg=10.09, stdev= 2.61
clat (msec): min=1, max=2803, avg=545.79, stdev=512.00
lat (msec): min=1, max=2803, avg=545.80, stdev=512.00
clat percentiles (msec):
| 1.00th=[ 6], 5.00th=[ 12], 10.00th=[ 23], 20.00th=[ 120],
| 30.00th=[ 155], 40.00th=[ 251], 50.00th=[ 371], 60.00th=[ 553],
| 70.00th=[ 734], 80.00th=[ 963], 90.00th=[ 1319], 95.00th=[ 1565],
| 99.00th=[ 2024], 99.50th=[ 2180], 99.90th=[ 2507], 99.95th=[ 2606],
| 99.99th=[ 2802]
bw (KB/s): min= 81, max= 861, per=99.37%, avg=465.04, stdev=170.84
lat (msec): 2=0.03%, 4=0.03%, 10=4.39%, 20=4.77%, 50=5.63%
lat (msec): 100=2.34%, 250=22.79%, 500=17.38%, 750=13.46%, 1000=10.01%
lat (msec): 2000=18.15%, >=2000=1.02%
cpu: usr=0.32%, sys=0.41%, ctx=3626, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.4%, 32=0.9%, >=64=98.3%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued: total=r=0/w=0/d=3625, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=14500KB, aggrb=468KB/s, minb=468KB/s, maxb=468KB/s, mint=30928msec, maxt=30928msec

Disk stats (read/write):
sdb: ios=84/3619, merge=0/0, ticks=67/1952100, in_queue=1959030, util=99.94%

Запись: iodepth=32, bw=536911 B/s, iops=131, avg=244.01 msec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/308K /s] [0 /77 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=3880
write: io=15960KB, bw=536911 B/s, iops=131, runt= 30439msec
slat (usec): min=3, max=29, avg=10.21, stdev= 2.29
clat (msec): min=1, max=1277, avg=244.01, stdev=222.90
lat (msec): min=1, max=1277, avg=244.02, stdev=222.90
clat percentiles (msec):
| 1.00th=[ 6], 5.00th=[ 10], 10.00th=[ 16], 20.00th=[ 33],
| 30.00th=[ 117], 40.00th=[ 141], 50.00th=[ 167], 60.00th=[ 241],
| 70.00th=[ 326], 80.00th=[ 408], 90.00th=[ 570], 95.00th=[ 685],
| 99.00th=[ 963], 99.50th=[ 1037], 99.90th=[ 1205], 99.95th=[ 1254],
| 99.99th=[ 1270]
bw (KB/s): min= 111, max= 814, per=99.65%, avg=522.18, stdev=146.44
lat (msec): 2=0.03%, 4=0.03%, 10=5.61%, 20=8.30%, 50=9.85%
lat (msec): 100=3.58%, 250=33.58%, 500=25.91%, 750=9.72%, 1000=2.63%
lat (msec): 2000=0.75%
cpu: usr=0.23%, sys=0.61%, ctx=3993, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.4%, 32=99.2%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=3990, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=15960KB, aggrb=524KB/s, minb=524KB/s, maxb=524KB/s, mint=30439msec, maxt=30439msec

Disk stats (read/write):
sdb: ios=84/3987, merge=0/0, ticks=70/966153, in_queue=967613, util=99.95%

Запись: iodepth=16, bw=578004 B/s, iops=141, avg=113.33 msec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=16
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/292K /s] [0 /73 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=3913
write: io=17028KB, bw=578004 B/s, iops=141, runt= 30167msec
slat (usec): min=3, max=33, avg= 9.80, stdev= 2.56
clat (msec): min=1, max=703, avg=113.33, stdev=112.35
lat (msec): min=1, max=703, avg=113.34, stdev=112.35
clat percentiles (msec):
| 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 10], 20.00th=[ 16],
| 30.00th=[ 24], 40.00th=[ 38], 50.00th=[ 109], 60.00th=[ 131],
| 70.00th=[ 143], 80.00th=[ 172], 90.00th=[ 277], 95.00th=[ 347],
| 99.00th=[ 469], 99.50th=[ 482], 99.90th=[ 611], 99.95th=[ 685],
| 99.99th=[ 701]
bw (KB/s): min= 224, max= 894, per=100.00%, avg=565.96, stdev=161.13
lat (msec): 2=0.02%, 4=0.02%, 10=11.28%, 20=14.02%, 50=20.39%
lat (msec): 100=3.24%, 250=39.53%, 500=11.18%, 750=0.31%
cpu: usr=0.16%, sys=0.69%, ctx=4259, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=99.6%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=4257, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=17028KB, aggrb=564KB/s, minb=564KB/s, maxb=564KB/s, mint=30167msec, maxt=30167msec

Disk stats (read/write):
sdb: ios=84/4255, merge=0/0, ticks=73/480767, in_queue=481510, util=99.96%

Запись: iodepth=8, bw=592348 B/s, iops=144, avg=55.27 msec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=8
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/328K /s] [0 /82 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=3945
write: io=17428KB, bw=592348 B/s, iops=144, runt= 30128msec
slat (usec): min=3, max=60, avg=10.30, stdev= 2.35
clat (msec): min=1, max=439, avg=55.27, stdev=74.63
lat (msec): min=1, max=439, avg=55.28, stdev=74.63
clat percentiles (msec):
| 1.00th=[ 6], 5.00th=[ 6], 10.00th=[ 8], 20.00th=[ 10],
| 30.00th=[ 12], 40.00th=[ 15], 50.00th=[ 17], 60.00th=[ 21],
| 70.00th=[ 29], 80.00th=[ 121], 90.00th=[ 130], 95.00th=[ 212],
| 99.00th=[ 322], 99.50th=[ 326], 99.90th=[ 433], 99.95th=[ 433],
| 99.99th=[ 441]
bw (KB/s): min= 248, max= 918, per=100.00%, avg=583.55, stdev=168.80
lat (msec): 2=0.02%, 4=0.09%, 10=23.96%, 20=34.11%, 50=12.55%
lat (msec): 100=1.54%, 250=24.37%, 500=3.35%
cpu: usr=0.39%, sys=0.53%, ctx=4363, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=99.8%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=4357, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=17428KB, aggrb=578KB/s, minb=578KB/s, maxb=578KB/s, mint=30128msec, maxt=30128msec

Disk stats (read/write):
sdb: ios=84/4347, merge=0/0, ticks=70/238777, in_queue=239110, util=99.95%

Запись: iodepth=4, bw=605445 B/s, iops=147, avg=27.05 msec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=4
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/220K /s] [0 /55 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=3977
write: io=17756KB, bw=605445 B/s, iops=147, runt= 30031msec
slat (usec): min=3, max=27, avg=10.13, stdev= 2.30
clat (msec): min=3, max=429, avg=27.05, stdev=51.71
lat (msec): min=3, max=429, avg=27.06, stdev=51.71
clat percentiles (msec):
| 1.00th=[ 6], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 6],
| 30.00th=[ 8], 40.00th=[ 8], 50.00th=[ 8], 60.00th=[ 8],
| 70.00th=[ 10], 80.00th=[ 11], 90.00th=[ 113], 95.00th=[ 117],
| 99.00th=[ 233], 99.50th=[ 306], 99.90th=[ 314], 99.95th=[ 314],
| 99.99th=[ 429]
bw (KB/s): min= 160, max= 902, per=100.00%, avg=592.20, stdev=151.52
lat (msec): 4=0.14%, 10=79.41%, 20=5.54%, 100=1.04%, 250=12.93%
lat (msec): 500=0.95%
cpu: usr=0.26%, sys=0.68%, ctx=4445, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=4439, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=17756KB, aggrb=591KB/s, minb=591KB/s, maxb=591KB/s, mint=30031msec, maxt=30031msec

Disk stats (read/write):
sdb: ios=84/4432, merge=0/0, ticks=70/119270, in_queue=119460, util=99.95%

Запись: iodepth=2, bw=613056 B/s, iops=149, avg=13.34 msec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=2
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/316K /s] [0 /79 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=4008
write: io=17984KB, bw=613056 B/s, iops=149, runt= 30039msec
slat (usec): min=4, max=131, avg=12.93, stdev= 4.61
clat (msec): min=1, max=306, avg=13.34, stdev=35.30
lat (msec): min=2, max=306, avg=13.36, stdev=35.30
clat percentiles (msec):
| 1.00th=[ 4], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 4],
| 30.00th=[ 4], 40.00th=[ 4], 50.00th=[ 4], 60.00th=[ 4],
| 70.00th=[ 4], 80.00th=[ 4], 90.00th=[ 5], 95.00th=[ 109],
| 99.00th=[ 194], 99.50th=[ 200], 99.90th=[ 233], 99.95th=[ 302],
| 99.99th=[ 306]
bw (KB/s): min= 242, max= 862, per=100.00%, avg=599.47, stdev=132.12
lat (msec): 2=0.02%, 4=89.03%, 10=2.54%, 20=0.80%, 100=0.29%
lat (msec): 250=7.23%, 500=0.09%
cpu: usr=0.18%, sys=0.85%, ctx=4500, majf=0, minf=0
IO depths: 1=0.1%, 2=100.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=4496, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=17984KB, aggrb=598KB/s, minb=598KB/s, maxb=598KB/s, mint=30039msec, maxt=30039msec

Disk stats (read/write):
sdb: ios=84/4485, merge=0/0, ticks=74/59624, in_queue=59697, util=99.96%

Запись: iodepth=1, bw=618686 B/s, iops=151, avg= 6.60 msec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/456K /s] [0 /114 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=4039
write: io=18128KB, bw=618686 B/s, iops=151, runt= 30004msec
slat (usec): min=5, max=42, avg=14.46, stdev= 3.39
clat (msec): min=1, max=293, avg= 6.60, stdev=24.74
lat (msec): min=1, max=293, avg= 6.62, stdev=24.74
clat percentiles (usec):
| 1.00th=[ 1720], 5.00th=[ 1752], 10.00th=[ 1800], 20.00th=[ 1832],
| 30.00th=[ 1848], 40.00th=[ 1864], 50.00th=[ 1912], 60.00th=[ 1944],
| 70.00th=[ 1976], 80.00th=[ 1992], 90.00th=[ 2024], 95.00th=[ 2448],
| 99.00th=[109056], 99.50th=[187392], 99.90th=[197632], 99.95th=[228352],
| 99.99th=[292864]
bw (KB/s): min= 207, max= 896, per=100.00%, avg=607.58, stdev=153.16
lat (msec): 2=83.94%, 4=11.50%, 10=0.84%, 20=0.02%, 100=0.13%
lat (msec): 250=3.55%, 500=0.02%
cpu: usr=0.13%, sys=0.91%, ctx=4540, majf=0, minf=0
IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=4532, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=18128KB, aggrb=604KB/s, minb=604KB/s, maxb=604KB/s, mint=30004msec, maxt=30004msec

Disk stats (read/write):
sdb: ios=84/4524, merge=0/0, ticks=70/29663, in_queue=29770, util=99.24%


Измерение последовательной записи:
root@host:~# dd if=/dev/zero  of=/dev/sdb oflag=direct bs=1M  
dd: writing `/dev/sdb': No space left on device
7581+0 records in
7580+0 records out
7948206080 bytes (7.9 GB) copied, 555.815 s, 14.3 MB/s


Анализ записи:

С записью всё иначе. Чем больше размер очереди, тем ниже скорость в Мбайт/с и IOPS, и выше задержки. Соответственно производительность накопителя на запись: iodepth=1, bw=618686 B/s, iops=151, avg= 6.60 msec.
При последовательной записи даже близко не удалось приблизиться к обещанной производителем 90 Мбайт/с на запись — максимальная скорость 14,3 Мбайт/с. Разница в скорости между произвольной и линейной записью ощутимая — более, чем в 6 раз.

SD-карта Team Class 10 32 GB (до 20 Мбайт/с)

Чтение: iodepth=1024, bw=5240.2KB/s, iops=1310, avg=773.34 msec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1024
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [5248K/0K /s] [1312 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=4616
read: io=157824KB, bw=5240.2KB/s, iops=1310, runt= 30118msec
slat (usec): min=2, max=25379, avg=758.25, stdev=4208.51
clat (msec): min=98, max=1956, avg=773.34, stdev=139.05
lat (msec): min=122, max=1956, avg=774.10, stdev=139.16
clat percentiles (msec):
| 1.00th=[ 416], 5.00th=[ 685], 10.00th=[ 685], 20.00th=[ 709],
| 30.00th=[ 709], 40.00th=[ 734], 50.00th=[ 734], 60.00th=[ 758],
| 70.00th=[ 783], 80.00th=[ 807], 90.00th=[ 906], 95.00th=[ 1057],
| 99.00th=[ 1319], 99.50th=[ 1434], 99.90th=[ 1598], 99.95th=[ 1663],
| 99.99th=[ 1811]
bw (KB/s): min= 5, max= 5280, per=98.38%, avg=5155.14, stdev=688.17
lat (msec): 100=0.01%, 250=0.49%, 500=0.81%, 750=51.20%, 1000=40.98%
lat (msec): 2000=6.53%
cpu: usr=0.33%, sys=0.66%, ctx=1365, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=99.8%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
issued: total=r=39456/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=157824KB, aggrb=5240KB/s, minb=5240KB/s, maxb=5240KB/s, mint=30118msec, maxt=30118msec

Disk stats (read/write):
sdb: ios=39187/0, merge=0/0, ticks=4274397/0, in_queue=4285950, util=99.72%

Чтение: iodepth=32, bw=5107.6KB/s, iops=1276, avg=25052.12 usec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [5112K/0K /s] [1278 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=4622
read: io=153360KB, bw=5107.6KB/s, iops=1276, runt= 30026msec
slat (usec): min=3, max=118, avg= 4.74, stdev= 2.30
clat (usec): min=900, max=66226, avg=25052.12, stdev=13451.25
lat (usec): min=936, max=66230, avg=25057.28, stdev=13451.24
clat percentiles (usec):
| 1.00th=[ 2384], 5.00th=[ 4016], 10.00th=[ 6368], 20.00th=[11072],
| 30.00th=[15808], 40.00th=[20352], 50.00th=[24960], 60.00th=[29568],
| 70.00th=[34048], 80.00th=[38656], 90.00th=[43264], 95.00th=[46336],
| 99.00th=[50944], 99.50th=[52480], 99.90th=[55552], 99.95th=[56064],
| 99.99th=[60672]
bw (KB/s): min= 4864, max= 5120, per=100.00%, avg=5108.00, stdev=32.89
lat (usec): 1000=0.01%
lat (msec): 2=0.01%, 4=4.91%, 10=12.15%, 20=22.02%, 50=59.51%
lat (msec): 100=1.40%
cpu: usr=0.73%, sys=1.43%, ctx=38345, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=99.9%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=38340/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=153360KB, aggrb=5107KB/s, minb=5107KB/s, maxb=5107KB/s, mint=30026msec, maxt=30026msec

Disk stats (read/write):
sdb: ios=38192/0, merge=1/0, ticks=956237/0, in_queue=956730, util=99.71%

Чтение: iodepth=16, bw=4941.1KB/s, iops=1235, avg=12942.28 usec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=16
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [4944K/0K /s] [1236 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=4626
read: io=148328KB, bw=4941.1KB/s, iops=1235, runt= 30014msec
slat (usec): min=3, max=118, avg= 4.83, stdev= 2.38
clat (usec): min=905, max=37620, avg=12942.28, stdev=6474.59
lat (usec): min=940, max=37624, avg=12947.54, stdev=6474.59
clat percentiles (usec):
| 1.00th=[ 2352], 5.00th=[ 3152], 10.00th=[ 4128], 20.00th=[ 6368],
| 30.00th=[ 8640], 40.00th=[10560], 50.00th=[12864], 60.00th=[14912],
| 70.00th=[17024], 80.00th=[19328], 90.00th=[21632], 95.00th=[23424],
| 99.00th=[26240], 99.50th=[27264], 99.90th=[29568], 99.95th=[30080],
| 99.99th=[31872]
bw (KB/s): min= 4920, max= 4976, per=100.00%, avg=4946.44, stdev=10.84
lat (usec): 1000=0.01%
lat (msec): 2=0.01%, 4=9.13%, 10=28.24%, 20=45.49%, 50=17.14%
cpu: usr=0.66%, sys=1.63%, ctx=37089, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=37082/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=148328KB, aggrb=4941KB/s, minb=4941KB/s, maxb=4941KB/s, mint=30014msec, maxt=30014msec

Disk stats (read/write):
sdb: ios=36956/0, merge=0/0, ticks=478014/0, in_queue=478134, util=99.71%

Чтение: iodepth=8, bw=4762.9KB/s, iops=1190, avg=6711.70 usec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=8
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [4756K/0K /s] [1189 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=4635
read: io=142896KB, bw=4762.9KB/s, iops=1190, runt= 30007msec
slat (usec): min=3, max=122, avg= 5.18, stdev= 2.62
clat (usec): min=895, max=18630, avg=6711.70, stdev=2806.36
lat (usec): min=932, max=18635, avg=6717.32, stdev=2806.35
clat percentiles (usec):
| 1.00th=[ 2384], 5.00th=[ 2480], 10.00th=[ 3216], 20.00th=[ 4080],
| 30.00th=[ 4960], 40.00th=[ 5792], 50.00th=[ 6624], 60.00th=[ 7520],
| 70.00th=[ 8384], 80.00th=[ 9280], 90.00th=[10432], 95.00th=[11584],
| 99.00th=[13248], 99.50th=[14016], 99.90th=[15040], 99.95th=[15680],
| 99.99th=[17536]
bw (KB/s): min= 4744, max= 4808, per=100.00%, avg=4766.51, stdev=12.91
lat (usec): 1000=0.01%
lat (msec): 2=0.01%, 4=19.26%, 10=66.98%, 20=13.76%
cpu: usr=0.84%, sys=1.30%, ctx=35731, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=35724/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=142896KB, aggrb=4762KB/s, minb=4762KB/s, maxb=4762KB/s, mint=30007msec, maxt=30007msec

Disk stats (read/write):
sdb: ios=35613/0, merge=0/0, ticks=238850/0, in_queue=238857, util=99.71%

Чтение: iodepth=4, bw=4626.6KB/s, iops=1156, avg=3450.49 usec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=4
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [4628K/0K /s] [1157 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=4640
read: io=138816KB, bw=4626.6KB/s, iops=1156, runt= 30004msec
slat (usec): min=3, max=111, avg= 5.17, stdev= 2.53
clat (usec): min=888, max=8495, avg=3450.49, stdev=810.23
lat (usec): min=924, max=8499, avg=3456.10, stdev=810.16
clat percentiles (usec):
| 1.00th=[ 2448], 5.00th=[ 2480], 10.00th=[ 2576], 20.00th=[ 2608],
| 30.00th=[ 2640], 40.00th=[ 3376], 50.00th=[ 3472], 60.00th=[ 3504],
| 70.00th=[ 3504], 80.00th=[ 4256], 90.00th=[ 4384], 95.00th=[ 5152],
| 99.00th=[ 5984], 99.50th=[ 6112], 99.90th=[ 6880], 99.95th=[ 7520],
| 99.99th=[ 7776]
bw (KB/s): min= 4616, max= 4656, per=100.00%, avg=4630.92, stdev= 8.59
lat (usec): 1000=0.01%
lat (msec): 2=0.01%, 4=74.99%, 10=25.00%
cpu: usr=0.60%, sys=1.50%, ctx=34710, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=34704/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=138816KB, aggrb=4626KB/s, minb=4626KB/s, maxb=4626KB/s, mint=30004msec, maxt=30004msec

Disk stats (read/write):
sdb: ios=34598/0, merge=0/0, ticks=119153/0, in_queue=119123, util=99.71%

Чтение: iodepth=2, bw=4608.8KB/s, iops=1152, avg=1727.08 usec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=2
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [4608K/0K /s] [1152 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=4646
read: io=138268KB, bw=4608.8KB/s, iops=1152, runt= 30001msec
slat (usec): min=3, max=125, avg= 6.25, stdev= 3.06
clat (usec): min=884, max=2286, avg=1727.08, stdev=53.30
lat (usec): min=920, max=2291, avg=1733.79, stdev=52.82
clat percentiles (usec):
| 1.00th=[ 1576], 5.00th=[ 1608], 10.00th=[ 1624], 20.00th=[ 1720],
| 30.00th=[ 1720], 40.00th=[ 1736], 50.00th=[ 1736], 60.00th=[ 1752],
| 70.00th=[ 1752], 80.00th=[ 1752], 90.00th=[ 1768], 95.00th=[ 1768],
| 99.00th=[ 1864], 99.50th=[ 1864], 99.90th=[ 2008], 99.95th=[ 2224],
| 99.99th=[ 2256]
bw (KB/s): min= 4600, max= 4624, per=100.00%, avg=4612.75, stdev= 8.15
lat (usec): 1000=0.01%
lat (msec): 2=99.90%, 4=0.10%
cpu: usr=0.61%, sys=1.60%, ctx=34578, majf=0, minf=0
IO depths: 1=0.1%, 2=100.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=34567/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=138268KB, aggrb=4608KB/s, minb=4608KB/s, maxb=4608KB/s, mint=30001msec, maxt=30001msec

Disk stats (read/write):
sdb: ios=34465/0, merge=0/0, ticks=59313/0, in_queue=59287, util=99.71%

Чтение: iodepth=1, bw=4593.6KB/s, iops=1148, avg=860.36 usec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [4592K/0K /s] [1148 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=4650
read: io=137812KB, bw=4593.6KB/s, iops=1148, runt= 30001msec
slat (usec): min=4, max=119, avg= 7.87, stdev= 3.64
clat (usec): min=688, max=6641, avg=860.36, stdev=54.23
lat (usec): min=704, max=6655, avg=868.71, stdev=54.21
clat percentiles (usec):
| 1.00th=[ 724], 5.00th=[ 748], 10.00th=[ 836], 20.00th=[ 852],
| 30.00th=[ 852], 40.00th=[ 868], 50.00th=[ 868], 60.00th=[ 868],
| 70.00th=[ 868], 80.00th=[ 884], 90.00th=[ 884], 95.00th=[ 900],
| 99.00th=[ 980], 99.50th=[ 988], 99.90th=[ 1096], 99.95th=[ 1112],
| 99.99th=[ 1144]
bw (KB/s): min= 4568, max= 4624, per=100.00%, avg=4597.69, stdev=12.16
lat (usec): 750=4.97%, 1000=94.82%
lat (msec): 2=0.20%, 10=0.01%
cpu: usr=0.69%, sys=1.75%, ctx=34459, majf=0, minf=0
IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=34453/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=137812KB, aggrb=4593KB/s, minb=4593KB/s, maxb=4593KB/s, mint=30001msec, maxt=30001msec

Disk stats (read/write):
sdb: ios=34352/0, merge=0/0, ticks=29370/0, in_queue=29347, util=97.83%


Запись: iodepth=32, bw=5960 B/s, iops=1, avg=21703.81 msec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [0.0% done] [0K/0K /s] [0 /0 iops] [eta 107d:03h:32m:23s]
writetest: (groupid=0, jobs=1): err= 0: pid=11326
write: io=307200 B, bw=5960 B/s, iops=1, runt= 51535msec
slat (usec): min=3, max=26, avg=11.08, stdev= 6.44
clat (msec): min=684, max=49451, avg=21703.81, stdev=13250.41
lat (msec): min=684, max=49451, avg=21703.82, stdev=13250.41
clat percentiles (msec):
| 1.00th=[ 685], 5.00th=[ 2057], 10.00th=[ 4146], 20.00th=[ 8225],
| 30.00th=[11731], 40.00th=[16712], 50.00th=[16712], 60.00th=[16712],
| 70.00th=[16712], 80.00th=[16712], 90.00th=[16712], 95.00th=[16712],
| 99.00th=[16712], 99.50th=[16712], 99.90th=[16712], 99.95th=[16712],
| 99.99th=[16712]
bw (KB/s): min= 0, max= 5, per=97.78%, avg= 4.89, stdev= 0.75
lat (msec): 750=1.33%, 2000=2.67%, >=2000=96.00%
cpu: usr=0.01%, sys=0.00%, ctx=76, majf=0, minf=0
IO depths: 1=1.3%, 2=2.7%, 4=5.3%, 8=10.7%, 16=21.3%, 32=58.7%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=97.8%, 8=0.0%, 16=0.0%, 32=2.2%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=75, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=300KB, aggrb=5KB/s, minb=5KB/s, maxb=5KB/s, mint=51535msec, maxt=51535msec

Disk stats (read/write):
sdb: ios=84/74, merge=0/0, ticks=70/1283090, in_queue=1308454, util=100.00%

Запись: iodepth=16, bw=5956 B/s, iops=1, avg=10826.78 msec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=16
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [0.0% done] [0K/0K /s] [0 /0 iops] [eta 84d:00h:53m:38s]
writetest: (groupid=0, jobs=1): err= 0: pid=18036
write: io=241664 B, bw=5956 B/s, iops=1, runt= 40571msec
slat (usec): min=3, max=25, avg=12.86, stdev= 5.71
clat (msec): min=684, max=24058, avg=10826.78, stdev=5773.72
lat (msec): min=684, max=24058, avg=10826.79, stdev=5773.72
clat percentiles (msec):
| 1.00th=[ 685], 5.00th=[ 1401], 10.00th=[ 2089], 20.00th=[ 4817],
| 30.00th=[ 6915], 40.00th=[ 9634], 50.00th=[11731], 60.00th=[13042],
| 70.00th=[14484], 80.00th=[15139], 90.00th=[16712], 95.00th=[16712],
| 99.00th=[16712], 99.50th=[16712], 99.90th=[16712], 99.95th=[16712],
| 99.99th=[16712]
bw (KB/s): min= 0, max= 5, per=97.78%, avg= 4.89, stdev= 0.75
lat (msec): 750=1.69%, 2000=3.39%, >=2000=94.92%
cpu: usr=0.01%, sys=0.00%, ctx=60, majf=0, minf=0
IO depths: 1=1.7%, 2=3.4%, 4=6.8%, 8=13.6%, 16=74.6%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=97.8%, 8=0.0%, 16=2.2%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=59, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=236KB, aggrb=5KB/s, minb=5KB/s, maxb=5KB/s, mint=40571msec, maxt=40571msec

Disk stats (read/write):
sdb: ios=84/58, merge=0/0, ticks=67/552507, in_queue=566187, util=99.97%

Запись: iodepth=8, bw=5958 B/s, iops=1, avg=5404.96 msec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=8
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [0.0% done] [0K/0K /s] [0 /0 iops] [eta 73d:12h:46m:55s]
writetest: (groupid=0, jobs=1): err= 0: pid=22634
write: io=208896 B, bw=5958 B/s, iops=1, runt= 35061msec
slat (usec): min=3, max=27, avg=15.22, stdev= 4.75
clat (msec): min=685, max=10998, avg=5404.96, stdev=2526.07
lat (msec): min=685, max=10998, avg=5404.97, stdev=2526.07
clat percentiles (msec):
| 1.00th=[ 685], 5.00th=[ 1369], 10.00th=[ 1401], 20.00th=[ 2737],
| 30.00th=[ 4113], 40.00th=[ 4817], 50.00th=[ 6194], 60.00th=[ 6849],
| 70.00th=[ 6915], 80.00th=[ 7570], 90.00th=[ 8225], 95.00th=[ 8225],
| 99.00th=[10945], 99.50th=[10945], 99.90th=[10945], 99.95th=[10945],
| 99.99th=[10945]
bw (KB/s): min= 0, max= 5, per=97.78%, avg= 4.89, stdev= 0.75
lat (msec): 750=1.96%, 2000=9.80%, >=2000=88.24%
cpu: usr=0.00%, sys=0.01%, ctx=52, majf=0, minf=0
IO depths: 1=2.0%, 2=3.9%, 4=7.8%, 8=86.3%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=97.8%, 8=2.2%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=51, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=204KB, aggrb=5KB/s, minb=5KB/s, maxb=5KB/s, mint=35061msec, maxt=35061msec

Disk stats (read/write):
sdb: ios=84/50, merge=0/0, ticks=70/255730, in_queue=261170, util=99.97%

Запись: iodepth=4, bw=5956 B/s, iops=1, avg=2706.33 msec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=4
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [0.0% done] [0K/0K /s] [0 /0 iops] [eta 67d:05h:30m:54s]
writetest: (groupid=0, jobs=1): err= 0: pid=26634
write: io=192512 B, bw=5956 B/s, iops=1, runt= 32322msec
slat (usec): min=3, max=25, avg=15.77, stdev= 3.33
clat (msec): min=684, max=4825, avg=2706.33, stdev=852.43
lat (msec): min=684, max=4825, avg=2706.34, stdev=852.43
clat percentiles (msec):
| 1.00th=[ 685], 5.00th=[ 1369], 10.00th=[ 1385], 20.00th=[ 2057],
| 30.00th=[ 2089], 40.00th=[ 2737], 50.00th=[ 2737], 60.00th=[ 2737],
| 70.00th=[ 3425], 80.00th=[ 3425], 90.00th=[ 3458], 95.00th=[ 4113],
| 99.00th=[ 4817], 99.50th=[ 4817], 99.90th=[ 4817], 99.95th=[ 4817],
| 99.99th=[ 4817]
bw (KB/s): min= 1, max= 5, per=98.22%, avg= 4.91, stdev= 0.60
lat (msec): 750=2.13%, 2000=10.64%, >=2000=87.23%
cpu: usr=0.01%, sys=0.00%, ctx=48, majf=0, minf=0
IO depths: 1=2.1%, 2=4.3%, 4=93.6%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=47, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=188KB, aggrb=5KB/s, minb=5KB/s, maxb=5KB/s, mint=32322msec, maxt=32322msec

Disk stats (read/write):
sdb: ios=84/46, merge=0/0, ticks=67/121626, in_queue=124990, util=99.94%

Запись: iodepth=2, bw=5961 B/s, iops=1, avg=1358.91 msec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=2
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [0.0% done] [0K/4K /s] [0 /1 iops] [eta 65d:03h:05m:34s]
writetest: (groupid=0, jobs=1): err= 0: pid=30700
write: io=184320 B, bw=5961 B/s, iops=1, runt= 30919msec
slat (usec): min=5, max=25, avg=16.40, stdev= 2.69
clat (msec): min=686, max=1400, avg=1358.91, stdev=102.85
lat (msec): min=686, max=1400, avg=1358.93, stdev=102.85
clat percentiles (msec):
| 1.00th=[ 685], 5.00th=[ 1369], 10.00th=[ 1369], 20.00th=[ 1369],
| 30.00th=[ 1369], 40.00th=[ 1369], 50.00th=[ 1369], 60.00th=[ 1369],
| 70.00th=[ 1369], 80.00th=[ 1369], 90.00th=[ 1385], 95.00th=[ 1401],
| 99.00th=[ 1401], 99.50th=[ 1401], 99.90th=[ 1401], 99.95th=[ 1401],
| 99.99th=[ 1401]
bw (KB/s): min= 5, max= 5, per=100.00%, avg= 5.00, stdev= 0.00
lat (msec): 750=2.22%, 2000=97.78%
cpu: usr=0.00%, sys=0.01%, ctx=46, majf=0, minf=0
IO depths: 1=2.2%, 2=97.8%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=45, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=180KB, aggrb=5KB/s, minb=5KB/s, maxb=5KB/s, mint=30919msec, maxt=30919msec

Disk stats (read/write):
sdb: ios=84/44, merge=0/0, ticks=66/59776, in_queue=61210, util=99.94%

Запись: iodepth=1, bw=5960 B/s, iops=1, avg=687.15 msec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/4K /s] [0 /1 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=1796
write: io=180224 B, bw=5960 B/s, iops=1, runt= 30237msec
slat (usec): min=19, max=26, avg=20.86, stdev= 1.44
clat (msec): min=684, max=715, avg=687.15, stdev= 6.12
lat (msec): min=684, max=715, avg=687.17, stdev= 6.12
clat percentiles (msec):
| 1.00th=[ 685], 5.00th=[ 685], 10.00th=[ 685], 20.00th=[ 685],
| 30.00th=[ 685], 40.00th=[ 685], 50.00th=[ 685], 60.00th=[ 685],
| 70.00th=[ 685], 80.00th=[ 685], 90.00th=[ 685], 95.00th=[ 693],
| 99.00th=[ 717], 99.50th=[ 717], 99.90th=[ 717], 99.95th=[ 717],
| 99.99th=[ 717]
bw (KB/s): min= 5, max= 5, per=100.00%, avg= 5.00, stdev= 0.00
lat (msec): 750=100.00%
cpu: usr=0.01%, sys=0.00%, ctx=45, majf=0, minf=0
IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=44, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=176KB, aggrb=5KB/s, minb=5KB/s, maxb=5KB/s, mint=30237msec, maxt=30237msec

Disk stats (read/write):
sdb: ios=84/43, merge=0/0, ticks=67/29550, in_queue=30230, util=99.94%


Измерение последовательной записи:
root@host:~# dd if=/dev/zero of=/dev/sdb oflag=direct bs=2M
dd: writing `/dev/sdb': No space left on device
15600+0 records in
15599+0 records out
32714522624 bytes (33 GB) copied, 2212.11 s, 14.8 MB/s


Анализ записи:

На записи ни при каких условиях не удалось выжать более 1 IOPS. Соответственно производительность накопителя на запись не соответствует нашим требованиям по avg.latency < 10 мс, так как его показатели: iodepth=1, bw=5960 B/s, iops=1, avg=687.15 msec. Хотя при последовательной записи накопитель действительно соответствует Class 10 — 14,8 Мбайт/с.

SD-карта Transcend 2GB

Чтение: iodepth=1024, bw=3800.9KB/s, iops=950, avg=1059.66 msec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1024
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [5.6% done] [3204K/0K /s] [801 /0 iops] [eta 08m:38s]
readtest: (groupid=0, jobs=1): err= 0: pid=4182
read: io=114640KB, bw=3800.9KB/s, iops=950, runt= 30162msec
slat (usec): min=2, max=68303, avg=1045.05, stdev=5824.60
clat (msec): min=136, max=1823, avg=1059.66, stdev=161.31
lat (msec): min=170, max=1823, avg=1060.71, stdev=161.40
clat percentiles (msec):
| 1.00th=[ 437], 5.00th=[ 938], 10.00th=[ 938], 20.00th=[ 971],
| 30.00th=[ 979], 40.00th=[ 1004], 50.00th=[ 1020], 60.00th=[ 1074],
| 70.00th=[ 1106], 80.00th=[ 1172], 90.00th=[ 1270], 95.00th=[ 1352],
| 99.00th=[ 1516], 99.50th=[ 1549], 99.90th=[ 1647], 99.95th=[ 1680],
| 99.99th=[ 1795]
bw (KB/s): min= 3, max= 4008, per=98.71%, avg=3750.79, stdev=502.34
lat (msec): 250=0.34%, 500=0.78%, 750=0.89%, 1000=34.54%, 2000=63.45%
cpu: usr=0.23%, sys=0.46%, ctx=1029, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=99.8%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
issued: total=r=28660/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=114640KB, aggrb=3800KB/s, minb=3800KB/s, maxb=3800KB/s, mint=30162msec, maxt=30162msec

Disk stats (read/write):
sdb: ios=28639/0, merge=20/0, ticks=4312340/0, in_queue=4312670, util=99.72%

Чтение: iodepth=32, bw=3610.3KB/s, iops=902, avg=35.45 msec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [3608K/0K /s] [902 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=4187
read: io=108436KB, bw=3610.3KB/s, iops=902, runt= 30036msec
slat (usec): min=3, max=122, avg= 5.34, stdev= 2.71
clat (msec): min=2, max=94, avg=35.45, stdev=19.17
lat (msec): min=2, max=94, avg=35.45, stdev=19.17
clat percentiles (usec):
| 1.00th=[ 3024], 5.00th=[ 5600], 10.00th=[ 9024], 20.00th=[15680],
| 30.00th=[22144], 40.00th=[28800], 50.00th=[35584], 60.00th=[41728],
| 70.00th=[48384], 80.00th=[54528], 90.00th=[61696], 95.00th=[65280],
| 99.00th=[72192], 99.50th=[74240], 99.90th=[78336], 99.95th=[79360],
| 99.99th=[84480]
bw (KB/s): min= 3584, max= 3640, per=100.00%, avg=3614.12, stdev=14.16
lat (msec): 4=2.71%, 10=8.99%, 20=14.90%, 50=45.97%, 100=27.43%
cpu: usr=0.61%, sys=1.22%, ctx=27123, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=99.9%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=27109/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=108436KB, aggrb=3610KB/s, minb=3610KB/s, maxb=3610KB/s, mint=30036msec, maxt=30036msec

Disk stats (read/write):
sdb: ios=26993/0, merge=3/0, ticks=955993/0, in_queue=956693, util=99.71%

Чтение: iodepth=16, bw=3283.4KB/s, iops=820, avg=19.48 msec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=16
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [3268K/0K /s] [817 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=4193
read: io=98564KB, bw=3283.4KB/s, iops=820, runt= 30019msec
slat (usec): min=3, max=121, avg= 5.26, stdev= 2.57
clat (msec): min=2, max=50, avg=19.48, stdev= 9.76
lat (msec): min=2, max=50, avg=19.49, stdev= 9.76
clat percentiles (usec):
| 1.00th=[ 3024], 5.00th=[ 4576], 10.00th=[ 6112], 20.00th=[ 9536],
| 30.00th=[12736], 40.00th=[16064], 50.00th=[19328], 60.00th=[22656],
| 70.00th=[25728], 80.00th=[29056], 90.00th=[32640], 95.00th=[35072],
| 99.00th=[39168], 99.50th=[40704], 99.90th=[43776], 99.95th=[44800],
| 99.99th=[45824]
bw (KB/s): min= 3224, max= 3328, per=100.00%, avg=3286.44, stdev=21.13
lat (msec): 4=4.12%, 10=17.36%, 20=30.77%, 50=47.74%, 100=0.01%
cpu: usr=0.72%, sys=1.05%, ctx=24650, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=99.9%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=24641/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=98564KB, aggrb=3283KB/s, minb=3283KB/s, maxb=3283KB/s, mint=30019msec, maxt=30019msec

Disk stats (read/write):
sdb: ios=24552/0, merge=0/0, ticks=477986/0, in_queue=478184, util=99.71%

Чтение: iodepth=8, bw=2836.8KB/s, iops=709, avg=11.27 msec
root@host:~# fio ./read.ini
\readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=8
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [2828K/0K /s] [707 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=4198
read: io=85132KB, bw=2836.8KB/s, iops=709, runt= 30011msec
slat (usec): min=3, max=124, avg= 5.84, stdev= 2.78
clat (msec): min=1, max=30, avg=11.27, stdev= 4.81
lat (msec): min=2, max=30, avg=11.28, stdev= 4.81
clat percentiles (usec):
| 1.00th=[ 3536], 5.00th=[ 4048], 10.00th=[ 4832], 20.00th=[ 6432],
| 30.00th=[ 8032], 40.00th=[ 9536], 50.00th=[11072], 60.00th=[12608],
| 70.00th=[14144], 80.00th=[15808], 90.00th=[17792], 95.00th=[19328],
| 99.00th=[22144], 99.50th=[23168], 99.90th=[25216], 99.95th=[26240],
| 99.99th=[27520]
bw (KB/s): min= 2752, max= 2920, per=100.00%, avg=2839.36, stdev=32.02
lat (msec): 2=0.01%, 4=4.87%, 10=38.22%, 20=53.46%, 50=3.44%
cpu: usr=0.78%, sys=1.11%, ctx=21293, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=21283/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=85132KB, aggrb=2836KB/s, minb=2836KB/s, maxb=2836KB/s, mint=30011msec, maxt=30011msec

Disk stats (read/write):
sdb: ios=21212/0, merge=0/0, ticks=238853/0, in_queue=238904, util=99.71%

Чтение: iodepth=4, bw=2447.4KB/s, iops=611, avg= 6.53 msec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=4
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [2440K/0K /s] [610 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=4205
read: io=73440KB, bw=2447.4KB/s, iops=611, runt= 30008msec
slat (usec): min=3, max=133, avg= 6.30, stdev= 3.41
clat (msec): min=1, max=15, avg= 6.53, stdev= 1.66
lat (msec): min=1, max=15, avg= 6.53, stdev= 1.66
clat percentiles (usec):
| 1.00th=[ 3632], 5.00th=[ 4256], 10.00th=[ 4512], 20.00th=[ 5024],
| 30.00th=[ 5408], 40.00th=[ 5984], 50.00th=[ 6432], 60.00th=[ 6816],
| 70.00th=[ 7264], 80.00th=[ 7840], 90.00th=[ 8768], 95.00th=[ 9536],
| 99.00th=[11200], 99.50th=[11840], 99.90th=[12992], 99.95th=[13632],
| 99.99th=[15168]
bw (KB/s): min= 2408, max= 2530, per=100.00%, avg=2450.12, stdev=23.74
lat (msec): 2=0.01%, 4=2.61%, 10=94.20%, 20=3.18%
cpu: usr=0.93%, sys=0.84%, ctx=18373, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=18360/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=73440KB, aggrb=2447KB/s, minb=2447KB/s, maxb=2447KB/s, mint=30008msec, maxt=30008msec

Disk stats (read/write):
sdb: ios=18300/0, merge=0/0, ticks=119210/0, in_queue=119227, util=99.71%

Чтение: iodepth=2, bw=2387.9KB/s, iops=596, avg=3337.08 usec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=2
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [2416K/0K /s] [604 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=4209
read: io=71640KB, bw=2387.9KB/s, iops=596, runt= 30002msec
slat (usec): min=3, max=35, avg= 9.64, stdev= 3.52
clat (usec): min=1185, max=5737, avg=3337.08, stdev=456.90
lat (usec): min=1213, max=5749, avg=3347.25, stdev=456.89
clat percentiles (usec):
| 1.00th=[ 1992], 5.00th=[ 2608], 10.00th=[ 2640], 20.00th=[ 2928],
| 30.00th=[ 3184], 40.00th=[ 3280], 50.00th=[ 3440], 60.00th=[ 3504],
| 70.00th=[ 3568], 80.00th=[ 3664], 90.00th=[ 3824], 95.00th=[ 3920],
| 99.00th=[ 4704], 99.50th=[ 4832], 99.90th=[ 4896], 99.95th=[ 4960],
| 99.99th=[ 5728]
bw (KB/s): min= 2352, max= 2432, per=100.00%, avg=2389.15, stdev=20.15
lat (msec): 2=1.14%, 4=96.04%, 10=2.82%
cpu: usr=0.73%, sys=1.76%, ctx=17923, majf=0, minf=0
IO depths: 1=0.1%, 2=100.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=17910/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=71640KB, aggrb=2387KB/s, minb=2387KB/s, maxb=2387KB/s, mint=30002msec, maxt=30002msec

Disk stats (read/write):
sdb: ios=17854/0, merge=0/0, ticks=59277/0, in_queue=59274, util=99.71%

Чтение: iodepth=1, bw=2362.2KB/s, iops=590, avg=1680.19 usec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [2364K/0K /s] [591 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=4216
read: io=70868KB, bw=2362.2KB/s, iops=590, runt= 30001msec
slat (usec): min=4, max=125, avg=10.06, stdev= 4.19
clat (usec): min=929, max=3056, avg=1680.19, stdev=318.18
lat (usec): min=942, max=3071, avg=1690.77, stdev=318.27
clat percentiles (usec):
| 1.00th=[ 972], 5.00th=[ 988], 10.00th=[ 1096], 20.00th=[ 1592],
| 30.00th=[ 1608], 40.00th=[ 1624], 50.00th=[ 1640], 60.00th=[ 1848],
| 70.00th=[ 1864], 80.00th=[ 1880], 90.00th=[ 1976], 95.00th=[ 1992],
| 99.00th=[ 2864], 99.50th=[ 2864], 99.90th=[ 2992], 99.95th=[ 2992],
| 99.99th=[ 3024]
bw (KB/s): min= 2320, max= 2408, per=100.00%, avg=2364.36, stdev=22.11
lat (usec): 1000=8.00%
lat (msec): 2=88.82%, 4=3.18%
cpu: usr=0.58%, sys=1.23%, ctx=17729, majf=0, minf=0
IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=17717/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=70868KB, aggrb=2362KB/s, minb=2362KB/s, maxb=2362KB/s, mint=30001msec, maxt=30001msec

Disk stats (read/write):
sdb: ios=17664/0, merge=0/0, ticks=29536/0, in_queue=29517, util=98.39%


Измерение последовательного чтения:
root@host:~# dd if=/dev/sdb  of=/dev/null iflag=direct bs=2M
957+1 records in
957+1 records out
2007498752 bytes (2.0 GB) copied, 195.504 s, 10.3 MB/s


Анализ чтения:

Скорость линейного чтения соответствует Class 10, хотя флешка очень давно покупалась и была «простенькой». Производительность накопителя на чтение: iodepth=4, bw=2447.4KB/s, iops=611, avg= 6.53 msec.

Запись: iodepth=1024, bw=36269 B/s, iops=8, avg=37126.84 msec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1024
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [0.0% done] [0K/0K /s] [0 /0 iops] [eta 49710d:06h:27m:29s]
writetest: (groupid=0, jobs=1): err= 0: pid=4234
write: io=1660.0KB, bw=36269 B/s, iops=8, runt= 46867msec
slat (usec): min=3, max=4088.5K, avg=77160.94, stdev=520319.60
clat (msec): min=14843, max=46865, avg=37126.84, stdev=10034.11
lat (msec): min=18784, max=46865, avg=37204.00, stdev=9985.34
clat percentiles (msec):
| 1.00th=[16712], 5.00th=[16712], 10.00th=[16712], 20.00th=[16712],
| 30.00th=[16712], 40.00th=[16712], 50.00th=[16712], 60.00th=[16712],
| 70.00th=[16712], 80.00th=[16712], 90.00th=[16712], 95.00th=[16712],
| 99.00th=[16712], 99.50th=[16712], 99.90th=[16712], 99.95th=[16712],
| 99.99th=[16712]
bw (KB/s): min= 0, max= 0, per=0.00%, avg= 0.00, stdev= 0.00
lat (msec): >=2000=100.00%
cpu: usr=0.00%, sys=0.03%, ctx=138, majf=0, minf=0
IO depths: 1=0.2%, 2=0.5%, 4=1.0%, 8=1.9%, 16=3.9%, 32=7.7%, >=64=84.8%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=100.0%
issued: total=r=0/w=0/d=415, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=1660KB, aggrb=35KB/s, minb=35KB/s, maxb=35KB/s, mint=46867msec, maxt=46867msec

Disk stats (read/write):
sdb: ios=84/413, merge=3/0, ticks=694/5472010, in_queue=5509883, util=100.00%

Запись: iodepth=32, bw=35509 B/s, iops=8, avg=3679.60 msec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [0.1% done] [0K/0K /s] [0 /0 iops] [eta 17h:24m:15s]
writetest: (groupid=0, jobs=1): err= 0: pid=4264
write: io=1156.0KB, bw=35509 B/s, iops=8, runt= 33336msec
slat (usec): min=3, max=24, avg=11.15, stdev= 3.16
clat (msec): min=116, max=8382, avg=3679.60, stdev=1873.21
lat (msec): min=116, max=8382, avg=3679.62, stdev=1873.21
clat percentiles (msec):
| 1.00th=[ 255], 5.00th=[ 660], 10.00th=[ 1004], 20.00th=[ 1647],
| 30.00th=[ 2409], 40.00th=[ 3228], 50.00th=[ 3916], 60.00th=[ 4686],
| 70.00th=[ 5014], 80.00th=[ 5342], 90.00th=[ 5669], 95.00th=[ 6128],
| 99.00th=[ 7767], 99.50th=[ 7963], 99.90th=[ 8356], 99.95th=[ 8356],
| 99.99th=[ 8356]
bw (KB/s): min= 3, max= 53, per=98.61%, avg=33.53, stdev= 8.02
lat (msec): 250=0.69%, 500=3.11%, 750=2.42%, 1000=3.46%, 2000=14.88%
lat (msec): >=2000=75.43%
cpu: usr=0.00%, sys=0.06%, ctx=292, majf=0, minf=0
IO depths: 1=0.3%, 2=0.7%, 4=1.4%, 8=2.8%, 16=5.5%, 32=89.3%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=99.6%, 8=0.0%, 16=0.0%, 32=0.4%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=289, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=1156KB, aggrb=34KB/s, minb=34KB/s, maxb=34KB/s, mint=33336msec, maxt=33336msec

Disk stats (read/write):
sdb: ios=84/287, merge=3/0, ticks=114/1004880, in_queue=1013170, util=100.00%

Запись: iodepth=16, bw=35257 B/s, iops=8, avg=1852.79 msec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=16
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [0.1% done] [0K/4K /s] [0 /1 iops] [eta 16h:24m:47s]
writetest: (groupid=0, jobs=1): err= 0: pid=4295
write: io=1088.0KB, bw=35257 B/s, iops=8, runt= 31599msec
slat (usec): min=3, max=23, avg=11.47, stdev= 2.19
clat (msec): min=257, max=3878, avg=1852.79, stdev=850.88
lat (msec): min=257, max=3878, avg=1852.81, stdev=850.88
clat percentiles (msec):
| 1.00th=[ 293], 5.00th=[ 498], 10.00th=[ 619], 20.00th=[ 938],
| 30.00th=[ 1336], 40.00th=[ 1598], 50.00th=[ 1958], 60.00th=[ 2311],
| 70.00th=[ 2474], 80.00th=[ 2606], 90.00th=[ 2868], 95.00th=[ 3032],
| 99.00th=[ 3720], 99.50th=[ 3785], 99.90th=[ 3884], 99.95th=[ 3884],
| 99.99th=[ 3884]
bw (KB/s): min= 7, max= 49, per=98.50%, avg=33.49, stdev= 7.39
lat (msec): 500=5.15%, 750=9.93%, 1000=5.51%, 2000=29.78%, >=2000=49.63%
cpu: usr=0.00%, sys=0.06%, ctx=274, majf=0, minf=0
IO depths: 1=0.4%, 2=0.7%, 4=1.5%, 8=2.9%, 16=94.5%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=99.6%, 8=0.0%, 16=0.4%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=272, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=1088KB, aggrb=34KB/s, minb=34KB/s, maxb=34KB/s, mint=31599msec, maxt=31599msec

Disk stats (read/write):
sdb: ios=84/270, merge=3/0, ticks=107/486823, in_queue=490777, util=100.00%

Запись: iodepth=8, bw=35293 B/s, iops=8, avg=925.17 msec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=8
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/36K /s] [0 /9 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=4327
write: io=1064.0KB, bw=35293 B/s, iops=8, runt= 30871msec
slat (usec): min=4, max=18, avg=11.48, stdev= 1.40
clat (msec): min=214, max=2006, avg=925.17, stdev=385.05
lat (msec): min=214, max=2006, avg=925.18, stdev=385.05
clat percentiles (msec):
| 1.00th=[ 245], 5.00th=[ 281], 10.00th=[ 400], 20.00th=[ 519],
| 30.00th=[ 652], 40.00th=[ 848], 50.00th=[ 971], 60.00th=[ 1074],
| 70.00th=[ 1205], 80.00th=[ 1303], 90.00th=[ 1385], 95.00th=[ 1483],
| 99.00th=[ 1729], 99.50th=[ 1926], 99.90th=[ 2008], 99.95th=[ 2008],
| 99.99th=[ 2008]
bw (KB/s): min= 13, max= 47, per=99.46%, avg=33.81, stdev= 6.65
lat (msec): 250=1.13%, 500=18.05%, 750=15.79%, 1000=18.05%, 2000=46.62%
lat (msec): >=2000=0.38%
cpu: usr=0.04%, sys=0.02%, ctx=269, majf=0, minf=0
IO depths: 1=0.4%, 2=0.8%, 4=1.5%, 8=97.4%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=99.6%, 8=0.4%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=266, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=1064KB, aggrb=34KB/s, minb=34KB/s, maxb=34KB/s, mint=30871msec, maxt=30871msec

Disk stats (read/write):
sdb: ios=84/264, merge=3/0, ticks=110/240687, in_queue=242880, util=100.00%

Запись: iodepth=4, bw=35262 B/s, iops=8, avg=462.96 msec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=4
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/40K /s] [0 /10 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=4359
write: io=1052.0KB, bw=35262 B/s, iops=8, runt= 30549msec
slat (usec): min=4, max=22, avg=11.73, stdev= 1.45
clat (msec): min=153, max=909, avg=462.96, stdev=145.01
lat (msec): min=153, max=909, avg=462.98, stdev=145.01
clat percentiles (msec):
| 1.00th=[ 200], 5.00th=[ 258], 10.00th=[ 281], 20.00th=[ 334],
| 30.00th=[ 379], 40.00th=[ 412], 50.00th=[ 453], 60.00th=[ 494],
| 70.00th=[ 537], 80.00th=[ 578], 90.00th=[ 660], 95.00th=[ 734],
| 99.00th=[ 824], 99.50th=[ 889], 99.90th=[ 914], 99.95th=[ 914],
| 99.99th=[ 914]
bw (KB/s): min= 21, max= 47, per=99.67%, avg=33.89, stdev= 6.47
lat (msec): 250=4.18%, 500=58.17%, 750=33.46%, 1000=4.18%
cpu: usr=0.00%, sys=0.06%, ctx=266, majf=0, minf=0
IO depths: 1=0.4%, 2=0.8%, 4=98.9%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=263, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=1052KB, aggrb=34KB/s, minb=34KB/s, maxb=34KB/s, mint=30549msec, maxt=30549msec

Disk stats (read/write):
sdb: ios=84/262, merge=3/0, ticks=113/120700, in_queue=121243, util=100.00%

Запись: iodepth=2, bw=35341 B/s, iops=8, avg=231.40 msec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=2
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/40K /s] [0 /10 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=4389
write: io=1040.0KB, bw=35341 B/s, iops=8, runt= 30133msec
slat (usec): min=5, max=38, avg=13.54, stdev= 2.05
clat (msec): min=68, max=434, avg=231.40, stdev=66.36
lat (msec): min=68, max=434, avg=231.41, stdev=66.36
clat percentiles (msec):
| 1.00th=[ 109], 5.00th=[ 137], 10.00th=[ 151], 20.00th=[ 174],
| 30.00th=[ 194], 40.00th=[ 208], 50.00th=[ 225], 60.00th=[ 239],
| 70.00th=[ 260], 80.00th=[ 281], 90.00th=[ 330], 95.00th=[ 351],
| 99.00th=[ 404], 99.50th=[ 408], 99.90th=[ 437], 99.95th=[ 437],
| 99.99th=[ 437]
bw (KB/s): min= 22, max= 48, per=100.00%, avg=34.09, stdev= 6.12
lat (msec): 100=0.77%, 250=65.00%, 500=34.23%
cpu: usr=0.00%, sys=0.05%, ctx=263, majf=0, minf=0
IO depths: 1=0.4%, 2=99.6%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=260, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=1040KB, aggrb=34KB/s, minb=34KB/s, maxb=34KB/s, mint=30133msec, maxt=30133msec

Disk stats (read/write):
sdb: ios=84/258, merge=3/0, ticks=110/59687, in_queue=59927, util=100.00%

Запись: iodepth=1, bw=35230 B/s, iops=8, avg=116.23 msec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/40K /s] [0 /10 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=4419
write: io=1036.0KB, bw=35230 B/s, iops=8, runt= 30112msec
slat (usec): min=13, max=33, avg=16.24, stdev= 2.64
clat (msec): min=24, max=279, avg=116.23, stdev=46.96
lat (msec): min=24, max=279, avg=116.25, stdev=46.96
clat percentiles (msec):
| 1.00th=[ 35], 5.00th=[ 46], 10.00th=[ 56], 20.00th=[ 74],
| 30.00th=[ 92], 40.00th=[ 101], 50.00th=[ 114], 60.00th=[ 127],
| 70.00th=[ 139], 80.00th=[ 155], 90.00th=[ 176], 95.00th=[ 200],
| 99.00th=[ 262], 99.50th=[ 269], 99.90th=[ 281], 99.95th=[ 281],
| 99.99th=[ 281]
bw (KB/s): min= 21, max= 47, per=99.56%, avg=33.85, stdev= 6.35
lat (msec): 50=7.34%, 100=31.66%, 250=59.85%, 500=1.16%
cpu: usr=0.02%, sys=0.04%, ctx=265, majf=0, minf=0
IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=259, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=1036KB, aggrb=34KB/s, minb=34KB/s, maxb=34KB/s, mint=30112msec, maxt=30112msec

Disk stats (read/write):
sdb: ios=84/257, merge=3/0, ticks=110/29853, in_queue=30003, util=99.98%


Измерение последовательной записи:
root@host:~# dd if=/dev/zero  of=/dev/sdb oflag=direct bs=2M
dd: writing `/dev/sdb': No space left on device
958+0 records in
957+0 records out
2007498752 bytes (2.0 GB) copied, 392.872 s, 5.1 MB/s


Анализ записи:

Не удалось получить от накопителя заданный уровень задержки. Интересен факт, что независимо от длины очереди, значение IOPS было постоянным — 8. Последовательная запись на носитель медленнее, чем чтение, и не соответствует Class 10. Наиболее близкий уровень: iodepth=1, bw=35230 B/s, iops=8, avg=116.23 msec.

SSD-диск OCZ-AGILITY3 60 GB


Чтение: iodepth=1024, bw=54321KB/s, iops=13580, avg=75.25 msec
root@host# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1024
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [55834K/0K /s] [13.7K/0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=25162
read: io=1592.6MB, bw=54321KB/s, iops=13580, runt= 30012msec
slat (usec): min=2, max=3715, avg=71.91, stdev=384.44
clat (msec): min=11, max=184, avg=75.25, stdev= 6.46
lat (msec): min=11, max=184, avg=75.32, stdev= 6.47
bw (KB/s): min=46128, max=55432, per=99.85%, avg=54239.90, stdev=1183.86
cpu: usr=2.08%, sys=5.65%, ctx=12877, majf=0, minf=536
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
issued r/w/d: total=407567/0/0, short=0/0/0

lat (msec): 20=0.03%, 50=0.10%, 100=99.16%, 250=0.71%

Run status group 0 (all jobs):
READ: io=1592.6MB, aggrb=54320KB/s, minb=55624KB/s, maxb=55624KB/s, mint=30012msec, maxt=30012msec

Disk stats (read/write):
sdd: ios=406300/0, merge=7/0, ticks=4261556/0, in_queue=4262475, util=99.75%

Чтение: iodepth=128, bw=49115KB/s, iops=12278, avg=10416.41 usec
root@host# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [48656K/0K /s] [11.9K/0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=25422
read: io=1439.5MB, bw=49115KB/s, iops=12278, runt= 30011msec
slat (usec): min=2, max=141, avg= 5.67, stdev= 2.73
clat (usec): min=213, max=110027, avg=10416.41, stdev=5804.65
lat (usec): min=217, max=110030, avg=10422.32, stdev=5804.64
bw (KB/s): min= 0, max=54056, per=32.25%, avg=15840.70, stdev=23083.02
cpu: usr=6.68%, sys=15.09%, ctx=309545, majf=0, minf=151
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
issued r/w/d: total=368494/0/0, short=0/0/0
lat (usec): 250=0.01%, 500=0.03%, 750=0.13%, 1000=0.22%
lat (msec): 2=1.38%, 4=8.11%, 10=40.02%, 20=47.88%, 50=1.99%
lat (msec): 100=0.24%, 250=0.01%

Run status group 0 (all jobs):
READ: io=1439.5MB, aggrb=49114KB/s, minb=50293KB/s, maxb=50293KB/s, mint=30011msec, maxt=30011msec

Disk stats (read/write):
sdd: ios=367410/0, merge=5/0, ticks=3824886/0, in_queue=3825505, util=99.76%

Чтение: iodepth=64, bw=52705KB/s, iops=13176, avg=4849.37 usec
root@host# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [54562K/0K /s] [13.4K/0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=25203
read: io=1544.5MB, bw=52705KB/s, iops=13176, runt= 30006msec
slat (usec): min=2, max=162, avg= 5.55, stdev= 2.82
clat (usec): min=173, max=28179, avg=4849.37, stdev=2171.93
lat (usec): min=194, max=28185, avg=4855.17, stdev=2171.92
bw (KB/s): min= 0, max=53784, per=49.18%, avg=25921.64, stdev=26461.21
cpu: usr=7.18%, sys=16.42%, ctx=328087, majf=0, minf=87
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued r/w/d: total=395367/0/0, short=0/0/0
lat (usec): 250=0.01%, 500=0.17%, 750=0.64%, 1000=0.99%
lat (msec): 2=5.23%, 4=29.45%, 10=61.81%, 20=1.65%, 50=0.04%

Run status group 0 (all jobs):
READ: io=1544.5MB, aggrb=52705KB/s, minb=53969KB/s, maxb=53969KB/s, mint=30006msec, maxt=30006msec

Disk stats (read/write):
sdd: ios=394198/0, merge=2/0, ticks=1909902/0, in_queue=1910098, util=99.75%

Чтение: iodepth=32, bw=47885KB/s, iops=11971, avg=2664.11 usec
root@host# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [43970K/0K /s] [10.8K/0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=25226
read: io=1403.3MB, bw=47885KB/s, iops=11971, runt= 30003msec
slat (usec): min=3, max=121, avg= 6.51, stdev= 2.75
clat (usec): min=94, max=9940, avg=2664.11, stdev=1397.42
lat (usec): min=109, max=9945, avg=2670.88, stdev=1397.38
bw (KB/s): min=36160, max=55408, per=100.16%, avg=47963.66, stdev=5087.19
cpu: usr=5.78%, sys=14.95%, ctx=302737, majf=0, minf=55
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued r/w/d: total=359175/0/0, short=0/0/0
lat (usec): 100=0.01%, 250=2.09%, 500=7.79%, 750=4.82%, 1000=2.75%
lat (msec): 2=5.04%, 4=53.76%, 10=23.75%

Run status group 0 (all jobs):
READ: io=1403.3MB, aggrb=47885KB/s, minb=49034KB/s, maxb=49034KB/s, mint=30003msec, maxt=30003msec

Disk stats (read/write):
sdd: ios=358185/0, merge=0/0, ticks=953875/0, in_queue=953911, util=99.74%

Чтение: iodepth=16, bw=48238KB/s, iops=12059, avg=1316.15 usec
root@host# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=16
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [53555K/0K /s] [13.8K/0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=25280
read: io=1413.4MB, bw=48238KB/s, iops=12059, runt= 30002msec
slat (usec): min=4, max=118, avg= 8.38, stdev= 3.34
clat (usec): min=84, max=52237, avg=1316.15, stdev=743.98
lat (usec): min=104, max=52246, avg=1324.79, stdev=744.00
bw (KB/s): min=34160, max=53896, per=99.86%, avg=48168.54, stdev=6020.67
cpu: usr=5.31%, sys=15.42%, ctx=306267, majf=0, minf=39
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=361811/0/0, short=0/0/0
lat (usec): 100=0.02%, 250=5.06%, 500=9.22%, 750=5.69%, 1000=6.44%
lat (msec): 2=57.38%, 4=16.17%, 10=0.01%, 50=0.01%, 100=0.01%

Run status group 0 (all jobs):
READ: io=1413.4MB, aggrb=48238KB/s, minb=49395KB/s, maxb=49395KB/s, mint=30002msec, maxt=30002msec

Disk stats (read/write):
sdd: ios=360786/0, merge=0/0, ticks=475948/0, in_queue=475940, util=99.77%

Чтение: iodepth=8, bw=47698KB/s, iops=11924, avg=660.16 usec
root@host# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=8
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [50597K/0K /s] [12.4K/0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=25299
read: io=1397.5MB, bw=47698KB/s, iops=11924, runt= 30001msec
slat (usec): min=3, max=143, avg= 8.44, stdev= 3.31
clat (usec): min=64, max=52445, avg=660.16, stdev=445.75
lat (usec): min=73, max=52454, avg=668.87, stdev=445.74
bw (KB/s): min=26688, max=51840, per=100.06%, avg=47725.07, stdev=6097.29
cpu: usr=5.36%, sys=15.34%, ctx=300814, majf=0, minf=31
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=357747/0/0, short=0/0/0
lat (usec): 100=0.06%, 250=7.35%, 500=19.37%, 750=44.03%, 1000=17.18%
lat (msec): 2=11.77%, 4=0.23%, 10=0.01%, 50=0.01%, 100=0.01%

Run status group 0 (all jobs):
READ: io=1397.5MB, aggrb=47698KB/s, minb=48842KB/s, maxb=48842KB/s, mint=30001msec, maxt=30001msec

Disk stats (read/write):
sdd: ios=356802/0, merge=0/0, ticks=236589/0, in_queue=236581, util=99.74%

Чтение: iodepth=4, bw=44827KB/s, iops=11206, avg=345.88 usec
root@host# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=4
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [47525K/0K /s] [11.7K/0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=25319
read: io=1313.4MB, bw=44827KB/s, iops=11206, runt= 30001msec
slat (usec): min=3, max=128, avg= 8.73, stdev= 3.41
clat (usec): min=60, max=44760, avg=345.88, stdev=197.72
lat (usec): min=69, max=44769, avg=354.90, stdev=197.62
bw (KB/s): min=37792, max=47280, per=99.96%, avg=44805.83, stdev=2312.44
cpu: usr=5.01%, sys=15.25%, ctx=282863, majf=0, minf=28
IO depths: 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=336211/0/0, short=0/0/0
lat (usec): 100=0.09%, 250=19.26%, 500=71.01%, 750=9.05%, 1000=0.02%
lat (msec): 2=0.55%, 4=0.01%, 10=0.01%, 50=0.01%

Run status group 0 (all jobs):
READ: io=1313.4MB, aggrb=44826KB/s, minb=45902KB/s, maxb=45902KB/s, mint=30001msec, maxt=30001msec

Disk stats (read/write):
sdd: ios=335261/0, merge=0/0, ticks=116934/0, in_queue=116921, util=99.75%

Чтение: iodepth=2, bw=32546KB/s, iops=8136, avg=234.51 usec
root@host# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=2
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [34082K/0K /s] [8321 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=25331
read: io=976420KB, bw=32546KB/s, iops=8136, runt= 30001msec
slat (usec): min=4, max=135, avg= 8.94, stdev= 3.14
clat (usec): min=82, max=44081, avg=234.51, stdev=146.58
lat (usec): min=101, max=44089, avg=243.73, stdev=146.52
bw (KB/s): min=27816, max=33888, per=99.96%, avg=32532.88, stdev=1031.17
cpu: usr=3.82%, sys=11.54%, ctx=214442, majf=0, minf=25
IO depths: 1=0.1%, 2=100.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=244105/0/0, short=0/0/0
lat (usec): 100=0.32%, 250=63.16%, 500=36.21%, 750=0.01%, 1000=0.02%
lat (msec): 2=0.28%, 4=0.01%, 10=0.01%, 50=0.01%

Run status group 0 (all jobs):
READ: io=976420KB, aggrb=32546KB/s, minb=33327KB/s, maxb=33327KB/s, mint=30001msec, maxt=30001msec

Disk stats (read/write):
sdd: ios=243444/0, merge=0/0, ticks=57717/0, in_queue=57707, util=99.70%

Чтение: iodepth=1, bw=19336KB/s, iops=4834, avg=196.02 usec
root@host# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [19505K/0K /s] [4762 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=25357
read: io=580104KB, bw=19336KB/s, iops=4834, runt= 30001msec
slat (usec): min=4, max=103, avg= 8.59, stdev= 2.31
clat (usec): min=52, max=4069, avg=196.02, stdev=45.26
lat (usec): min=58, max=4078, avg=204.88, stdev=45.45
bw (KB/s): min=18624, max=19936, per=100.05%, avg=19346.31, stdev=295.36
cpu: usr=2.32%, sys=6.80%, ctx=145027, majf=0, minf=25
IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=145026/0/0, short=0/0/0
lat (usec): 100=0.27%, 250=98.77%, 500=0.85%, 750=0.01%, 1000=0.02%
lat (msec): 2=0.09%, 4=0.01%, 10=0.01%

Run status group 0 (all jobs):
READ: io=580104KB, aggrb=19336KB/s, minb=19800KB/s, maxb=19800KB/s, mint=30001msec, maxt=30001msec

Disk stats (read/write):
sdd: ios=144605/0, merge=0/0, ticks=28537/0, in_queue=28528, util=95.12%


Измерение последовательного чтения:
root@host# dd if=/dev/sdd  of=/dev/null iflag=direct bs=2M 
28620+1 records in
28620+1 records out
60022480896 bytes (60 GB) copied, 238.535 s, 252 MB/s


Анализ чтения:

Закономерность также видна: чем больше размер очереди, тем больше IOPS и больше задержки. Соответственно производительность накопителя на чтение: Чтение: iodepth=64, bw=52705KB/s, iops=13176, avg=4849.37 usec. Измеренная скорость линейного чтения 252 Мбайт/с далеко не дотягивает до заявленной производителем (Max Read: up to 525MB/s).

Запись: iodepth=1024, bw=71064KB/s, iops=17765, avg=57.55 msec
root@host# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1024
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/71737K /s] [0 /17.6K iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=25502
write: io=2082.6MB, bw=71064KB/s, iops=17765, runt= 30008msec
slat (usec): min=2, max=49835, avg=54.11, stdev=424.63
clat (msec): min=7, max=163, avg=57.55, stdev= 9.97
lat (msec): min=7, max=163, avg=57.60, stdev= 9.98
bw (KB/s): min=61960, max=74416, per=100.02%, avg=71074.07, stdev=3099.18
cpu: usr=2.65%, sys=10.56%, ctx=16632, majf=0, minf=23
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
issued r/w/d: total=0/533119/0, short=0/0/0

lat (msec): 10=0.01%, 20=0.04%, 50=6.73%, 100=91.88%, 250=1.35%

Run status group 0 (all jobs):
WRITE: io=2082.6MB, aggrb=71063KB/s, minb=72769KB/s, maxb=72769KB/s, mint=30008msec, maxt=30008msec

Disk stats (read/write):
sdd: ios=0/531357, merge=0/14, ticks=0/4256081, in_queue=4256786, util=99.74%

Запись: iodepth=512, bw=70673KB/s, iops=17668, avg=28.92 msec
root@host# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=512
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/71751K /s] [0 /17.6K iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=25524
write: io=2070.2MB, bw=70673KB/s, iops=17668, runt= 30007msec
slat (usec): min=2, max=48135, avg=54.43, stdev=429.52
clat (msec): min=5, max=138, avg=28.92, stdev= 7.88
lat (msec): min=7, max=138, avg=28.97, stdev= 7.88
bw (KB/s): min=66355, max=73584, per=100.10%, avg=70740.36, stdev=2833.32
cpu: usr=2.69%, sys=10.19%, ctx=16510, majf=0, minf=22
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
issued r/w/d: total=0/530174/0, short=0/0/0

lat (msec): 10=0.01%, 20=0.17%, 50=97.07%, 100=2.74%, 250=0.01%

Run status group 0 (all jobs):
WRITE: io=2070.2MB, aggrb=70673KB/s, minb=72369KB/s, maxb=72369KB/s, mint=30007msec, maxt=30007msec

Disk stats (read/write):
sdd: ios=0/528541, merge=0/12, ticks=0/4257407, in_queue=4258063, util=99.75%

Запись: iodepth=256, bw=70731KB/s, iops=17682, avg=14.42 msec
root@host# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=256
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/71315K /s] [0 /17.5K iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=25546
write: io=2072.1MB, bw=70731KB/s, iops=17682, runt= 30010msec
slat (usec): min=2, max=49650, avg=54.45, stdev=426.61
clat (msec): min=2, max=119, avg=14.42, stdev= 6.41
lat (msec): min=2, max=119, avg=14.48, stdev= 6.42
bw (KB/s): min=65992, max=73437, per=100.02%, avg=70748.44, stdev=2926.02
cpu: usr=2.60%, sys=9.86%, ctx=16555, majf=0, minf=22
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
issued r/w/d: total=0/530661/0, short=0/0/0

lat (msec): 4=0.01%, 10=17.81%, 20=76.24%, 50=4.75%, 100=1.18%
lat (msec): 250=0.01%

Run status group 0 (all jobs):
WRITE: io=2072.1MB, aggrb=70731KB/s, minb=72428KB/s, maxb=72428KB/s, mint=30010msec, maxt=30010msec

Disk stats (read/write):
sdd: ios=0/528998, merge=0/11, ticks=0/4278359, in_queue=4278963, util=99.75%

Запись: iodepth=128, bw=70591KB/s, iops=17647, avg=7244.71 usec
root@host# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/71909K /s] [0 /17.6K iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=25576
write: io=2068.7MB, bw=70591KB/s, iops=17647, runt= 30008msec
slat (usec): min=2, max=163, avg= 5.88, stdev= 4.40
clat (usec): min=484, max=108710, avg=7244.71, stdev=5168.71
lat (usec): min=487, max=108714, avg=7250.84, stdev=5168.65
bw (KB/s): min=65860, max=73405, per=100.14%, avg=70691.51, stdev=2874.92
cpu: usr=7.44%, sys=17.22%, ctx=387944, majf=0, minf=22
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
issued r/w/d: total=0/529576/0, short=0/0/0
lat (usec): 500=0.01%, 750=0.01%, 1000=0.31%
lat (msec): 2=5.26%, 4=15.84%, 10=60.46%, 20=16.73%, 50=1.00%
lat (msec): 100=0.39%, 250=0.01%

Run status group 0 (all jobs):
WRITE: io=2068.7MB, aggrb=70591KB/s, minb=72285KB/s, maxb=72285KB/s, mint=30008msec, maxt=30008msec

Disk stats (read/write):
sdd: ios=0/527794, merge=0/8, ticks=0/3821818, in_queue=3822381, util=99.72%

Запись: iodepth=64, bw=70635KB/s, iops=17658, avg=3616.35 usec
root@host# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/71446K /s] [0 /17.5K iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=25602
write: io=2069.8MB, bw=70635KB/s, iops=17658, runt= 30005msec
slat (usec): min=2, max=160, avg= 5.85, stdev= 4.73
clat (usec): min=487, max=61425, avg=3616.35, stdev=2845.27
lat (usec): min=504, max=61431, avg=3622.46, stdev=2845.16
bw (KB/s): min=66011, max=73784, per=100.03%, avg=70656.08, stdev=2903.02
cpu: usr=7.83%, sys=16.87%, ctx=383027, majf=0, minf=22
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued r/w/d: total=0/529849/0, short=0/0/0
lat (usec): 500=0.01%, 750=0.03%, 1000=0.99%
lat (msec): 2=16.88%, 4=46.75%, 10=34.56%, 20=0.51%, 50=0.25%
lat (msec): 100=0.03%

Run status group 0 (all jobs):
WRITE: io=2069.8MB, aggrb=70634KB/s, minb=72329KB/s, maxb=72329KB/s, mint=30005msec, maxt=30005msec

Disk stats (read/write):
sdd: ios=0/528157, merge=0/3, ticks=0/1908453, in_queue=1908547, util=99.74%

Запись: iodepth=32, bw=70448KB/s, iops=17611, avg=1808.25 usec
root@host# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/72024K /s] [0 /17.6K iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=25630
write: io=2064.2MB, bw=70448KB/s, iops=17611, runt= 30003msec
slat (usec): min=3, max=472, avg= 6.86, stdev= 4.85
clat (usec): min=361, max=64001, avg=1808.25, stdev=1900.05
lat (usec): min=381, max=64006, avg=1815.36, stdev=1899.96
bw (KB/s): min=64232, max=73584, per=100.13%, avg=70539.47, stdev=2978.82
cpu: usr=7.51%, sys=16.93%, ctx=376150, majf=0, minf=22
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued r/w/d: total=0/528410/0, short=0/0/0
lat (usec): 500=0.08%, 750=1.38%, 1000=25.44%
lat (msec): 2=27.79%, 4=44.06%, 10=1.10%, 20=0.01%, 50=0.14%
lat (msec): 100=0.01%

Run status group 0 (all jobs):
WRITE: io=2064.2MB, aggrb=70447KB/s, minb=72138KB/s, maxb=72138KB/s, mint=30003msec, maxt=30003msec

Disk stats (read/write):
sdd: ios=0/526823, merge=0/0, ticks=0/951988, in_queue=951967, util=99.74%

Запись: iodepth=16, bw=70562KB/s, iops=17640, avg=897.30 usec
root@host# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=16
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/71696K /s] [0 /17.6K iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=25666
write: io=2067.4MB, bw=70562KB/s, iops=17640, runt= 30001msec
slat (usec): min=4, max=160, avg= 7.91, stdev= 4.52
clat (usec): min=134, max=48672, avg=897.30, stdev=1390.50
lat (usec): min=143, max=48678, avg=905.46, stdev=1390.18
bw (KB/s): min=65424, max=74296, per=100.03%, avg=70583.68, stdev=2998.17
cpu: usr=6.75%, sys=17.48%, ctx=376455, majf=0, minf=22
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=0/529233/0, short=0/0/0
lat (usec): 250=0.52%, 500=46.44%, 750=16.68%, 1000=8.85%
lat (msec): 2=14.06%, 4=13.36%, 10=0.01%, 50=0.07%

Run status group 0 (all jobs):
WRITE: io=2067.4MB, aggrb=70562KB/s, minb=72255KB/s, maxb=72255KB/s, mint=30001msec, maxt=30001msec

Disk stats (read/write):
sdd: ios=0/527771, merge=0/0, ticks=0/474172, in_queue=474137, util=99.74%

Запись: iodepth=8, bw=68056KB/s, iops=17014, avg=460.47 usec
root@host# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=8
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/60592K /s] [0 /14.8K iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=25718
write: io=1993.1MB, bw=68056KB/s, iops=17014, runt= 30001msec
slat (usec): min=4, max=153, avg= 7.96, stdev= 4.44
clat (usec): min=45, max=55053, avg=460.47, stdev=1040.65
lat (usec): min=50, max=55059, avg=468.67, stdev=1040.38
bw (KB/s): min=47096, max=74344, per=100.44%, avg=68355.95, stdev=6706.70
cpu: usr=6.40%, sys=17.64%, ctx=373251, majf=0, minf=22
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=0/510439/0, short=0/0/0
lat (usec): 50=0.01%, 100=1.25%, 250=65.56%, 500=6.91%, 750=11.48%
lat (usec): 1000=2.55%
lat (msec): 2=9.06%, 4=3.14%, 10=0.02%, 20=0.01%, 50=0.03%
lat (msec): 100=0.01%

Run status group 0 (all jobs):
WRITE: io=1993.1MB, aggrb=68056KB/s, minb=69689KB/s, maxb=69689KB/s, mint=30001msec, maxt=30001msec

Disk stats (read/write):
sdd: ios=0/509333, merge=0/0, ticks=0/234860, in_queue=234810, util=99.74%

Запись: iodepth=4, bw=54118KB/s, iops=13529, avg=285.05 usec
root@host# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=4
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/69414K /s] [0 /16.1K iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=25739
write: io=1585.6MB, bw=54118KB/s, iops=13529, runt= 30001msec
slat (usec): min=4, max=148, avg= 8.58, stdev= 3.86
clat (usec): min=22, max=53292, avg=285.05, stdev=732.96
lat (usec): min=36, max=53299, avg=293.89, stdev=732.88
bw (KB/s): min=41992, max=71504, per=99.55%, avg=53871.80, stdev=6853.35
cpu: usr=5.90%, sys=16.46%, ctx=341954, majf=0, minf=22
IO depths: 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=0/405896/0, short=0/0/0
lat (usec): 50=2.69%, 100=43.57%, 250=1.37%, 500=46.21%, 750=1.71%
lat (usec): 1000=0.58%
lat (msec): 2=2.73%, 4=1.05%, 10=0.07%, 20=0.01%, 50=0.02%
lat (msec): 100=0.01%

Run status group 0 (all jobs):
WRITE: io=1585.6MB, aggrb=54117KB/s, minb=55416KB/s, maxb=55416KB/s, mint=30001msec, maxt=30001msec

Disk stats (read/write):
sdd: ios=0/404392, merge=0/0, ticks=0/116249, in_queue=116226, util=99.72%

Запись: iodepth=2, bw=55449KB/s, iops=13862, avg=133.14 usec
root@host# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=2
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/52981K /s] [0 /12.1K iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=25799
write: io=1624.6MB, bw=55449KB/s, iops=13862, runt= 30001msec
slat (usec): min=4, max=121, avg= 9.00, stdev= 3.99
clat (usec): min=8, max=47598, avg=133.14, stdev=495.36
lat (usec): min=38, max=47603, avg=142.43, stdev=495.31
bw (KB/s): min=41704, max=74514, per=100.17%, avg=55540.66, stdev=8889.68
cpu: usr=6.80%, sys=17.36%, ctx=355805, majf=0, minf=22
IO depths: 1=0.1%, 2=100.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=0/415881/0, short=0/0/0
lat (usec): 10=0.01%, 20=0.06%, 50=50.54%, 100=3.83%, 250=42.08%
lat (usec): 500=0.83%, 750=0.57%, 1000=0.13%
lat (msec): 2=1.64%, 4=0.29%, 10=0.02%, 50=0.01%

Run status group 0 (all jobs):
WRITE: io=1624.6MB, aggrb=55448KB/s, minb=56779KB/s, maxb=56779KB/s, mint=30001msec, maxt=30001msec

Disk stats (read/write):
sdd: ios=0/414844, merge=0/0, ticks=0/56656, in_queue=56639, util=99.60%

Запись: iodepth=1, bw=52320KB/s, iops=13079, avg=65.35 usec
root@host# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/52297K /s] [0 /12.8K iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=25822
write: io=1532.9MB, bw=52320KB/s, iops=13079, runt= 30001msec
slat (usec): min=5, max=99, avg= 8.85, stdev= 2.53
clat (usec): min=0, max=48971, avg=65.35, stdev=311.65
lat (usec): min=37, max=48981, avg=74.47, stdev=311.67
bw (KB/s): min=41920, max=73824, per=99.98%, avg=52310.10, stdev=6877.76
cpu: usr=6.74%, sys=17.84%, ctx=392397, majf=0, minf=22
IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=0/392410/0, short=0/0/0
lat (usec): 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=47.18%
lat (usec): 100=51.47%, 250=0.59%, 500=0.18%, 750=0.19%, 1000=0.17%
lat (msec): 2=0.18%, 4=0.02%, 10=0.01%, 50=0.01%

Run status group 0 (all jobs):
WRITE: io=1532.9MB, aggrb=52319KB/s, minb=53575KB/s, maxb=53575KB/s, mint=30001msec, maxt=30001msec

Disk stats (read/write):
sdd: ios=0/391034, merge=0/0, ticks=0/26366, in_queue=26344, util=87.84%


Измерение последовательной записи:
root@host# dd if=/dev/zero of=/dev/sdd oflag=direct bs=2M
dd: writing `/dev/sdd': No space left on device
28621+0 records in
28620+0 records out
60022480896 bytes (60 GB) copied, 238.875 s, 251 MB/s


Анализ записи:

Следует отметить нелинейную закономерность IOPS от задержек avg.latency. Запись: iodepth=128, bw=70591KB/s, iops=17647, avg=7244.71 usec. Измеренная скорость линейной записи 251 Мбайт/с далеко не дотягивает до заявленной производителем (Max Write: up to 500MB/s ).

UPDATE: Как подсказал amarao тестирование записи SSD надо проводить на более длинном промежутке времени. Поэтому был запущен дополнительный тест на 10 часов. Число iodepth выбрано 64, так как на более коротком промежутке времени эта длина очереди оказалась самой эффективной. Файл конфигурации таков:
write.ini
[writetest]
blocksize=4k
filename=/dev/sdd
rw=randwrite
direct=1
buffered=0
ioengine=libaio
iodepth=64

runtime=36000
time_based


В результате показатели IOPS и bw снизились более, чем в 2 раза:

Запись 10 часов: iodepth=64, bw=32310KB/s, iops=8077, avg=7912.32
root@host]# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/42369K /s] [0 /10.4K iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=25904
write: io=1109.3GB, bw=32310KB/s, iops=8077, runt=36000021msec
slat (usec): min=2, max=54737, avg= 8.69, stdev= 7.66
clat (usec): min=273, max=1637.4K, avg=7912.32, stdev=7961.38
lat (usec): min=280, max=1637.5K, avg=7921.36, stdev=7961.69
bw (KB/s): min= 974, max=70008, per=100.06%, avg=32326.89, stdev=10116.94
cpu: usr=2.87%, sys=10.90%, ctx=139461266, majf=0, minf=22
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued r/w/d: total=0/290788467/0, short=0/0/0
lat (usec): 500=0.01%, 750=0.09%, 1000=0.27%
lat (msec): 2=2.71%, 4=19.19%, 10=61.65%, 20=9.32%, 50=6.23%
lat (msec): 100=0.52%, 250=0.02%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec): 2000=0.01%

Run status group 0 (all jobs):
WRITE: io=1109.3GB, aggrb=32309KB/s, minb=33085KB/s, maxb=33085KB/s, mint=36000021msec, maxt=36000021msec

Disk stats (read/write):
sdd: ios=1743/290784750, merge=0/3511, ticks=8629/2295779056, in_queue=2295739130, util=100.00%


Так как fio не позволяет видеть зависимость показателей от времени «из коробки» — не было возможности оценить, по истечению какого времени производительность стала падать.

SATA-диск consumer-level Hitachi Deskstar HDS723020BLA642 2 ТБ 7200 об/мин, 64 Мбайт

Чтение: iodepth=128, bw=558713 B/s, iops=136, avg=937.95 msec
root@host# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [r] [0.0% done] [73K/0K /s] [18 /0 iops] [eta 42d:18h:13m:10s]
readtest: (groupid=0, jobs=1): err= 0: pid=26563
read: io=16900KB, bw=558713 B/s, iops=136, runt= 30974msec
slat (usec): min=3, max=124, avg= 9.05, stdev= 4.35
clat (msec): min=17, max=3368, avg=937.95, stdev=589.91
lat (msec): min=18, max=3368, avg=937.96, stdev=589.91
bw (KB/s): min= 0, max= 625, per=31.90%, avg=173.88, stdev=255.59
cpu: usr=0.13%, sys=0.28%, ctx=4225, majf=0, minf=152
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.4%, 32=0.8%, >=64=98.5%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
issued r/w/d: total=4225/0/0, short=0/0/0

lat (msec): 20=0.02%, 50=0.26%, 100=1.63%, 250=8.99%, 500=18.11%
lat (msec): 750=16.00%, 1000=13.78%, 2000=36.31%, >=2000=4.90%

Run status group 0 (all jobs):
READ: io=16900KB, aggrb=545KB/s, minb=558KB/s, maxb=558KB/s, mint=30974msec, maxt=30974msec

Disk stats (read/write):
sdb: ios=4213/0, merge=0/0, ticks=3883587/0, in_queue=3900496, util=99.60%

Чтение: iodepth=64, bw=518986 B/s, iops=126, avg=504.75 msec
root@host# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [540K/0K /s] [132 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=26631
read: io=15600KB, bw=518986 B/s, iops=126, runt= 30780msec
slat (usec): min=4, max=45, avg=10.96, stdev= 2.37
clat (msec): min=8, max=2411, avg=504.75, stdev=311.77
lat (msec): min=8, max=2411, avg=504.76, stdev=311.77
bw (KB/s): min= 0, max= 565, per=49.08%, avg=248.34, stdev=256.27
cpu: usr=0.19%, sys=0.28%, ctx=3900, majf=0, minf=87
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.4%, 32=0.8%, >=64=98.4%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued r/w/d: total=3900/0/0, short=0/0/0

lat (msec): 10=0.03%, 50=0.36%, 100=3.44%, 250=18.69%, 500=34.36%
lat (msec): 750=22.77%, 1000=12.59%, 2000=7.74%, >=2000=0.03%

Run status group 0 (all jobs):
READ: io=15600KB, aggrb=506KB/s, minb=518KB/s, maxb=518KB/s, mint=30780msec, maxt=30780msec

Disk stats (read/write):
sdb: ios=3892/0, merge=0/0, ticks=1928873/0, in_queue=1936814, util=99.61%

Чтение: iodepth=32, bw=499089 B/s, iops=121, avg=262.47 msec
root@host# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [511K/0K /s] [124 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=26812
read: io=14748KB, bw=499089 B/s, iops=121, runt= 30259msec
slat (usec): min=5, max=36, avg=14.30, stdev= 2.71
clat (msec): min=13, max=2281, avg=262.47, stdev=229.50
lat (msec): min=13, max=2281, avg=262.49, stdev=229.50
bw (KB/s): min= 448, max= 542, per=100.14%, avg=487.68, stdev=20.30
cpu: usr=0.16%, sys=0.31%, ctx=3687, majf=0, minf=55
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.4%, 32=99.2%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued r/w/d: total=3687/0/0, short=0/0/0

lat (msec): 20=0.57%, 50=12.96%, 100=14.94%, 250=29.51%, 500=28.53%
lat (msec): 750=9.79%, 1000=2.52%, 2000=1.11%, >=2000=0.05%

Run status group 0 (all jobs):
READ: io=14748KB, aggrb=487KB/s, minb=499KB/s, maxb=499KB/s, mint=30259msec, maxt=30259msec

Disk stats (read/write):
sdb: ios=3673/0, merge=0/0, ticks=956588/0, in_queue=962866, util=99.62%

Чтение: iodepth=16, bw=446150 B/s, iops=108, avg=146.79 msec
root@host# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=16
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [421K/0K /s] [102 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=26830
read: io=13144KB, bw=446150 B/s, iops=108, runt= 30168msec
slat (usec): min=5, max=93, avg=17.37, stdev= 3.35
clat (msec): min=8, max=920, avg=146.79, stdev=122.95
lat (msec): min=8, max=921, avg=146.81, stdev=122.95
bw (KB/s): min= 402, max= 496, per=100.24%, avg=436.03, stdev=16.06
cpu: usr=0.14%, sys=0.28%, ctx=3286, majf=0, minf=39
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=99.5%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=3286/0/0, short=0/0/0

lat (msec): 10=0.15%, 20=6.18%, 50=18.72%, 100=19.84%, 250=37.61%
lat (msec): 500=15.86%, 750=1.43%, 1000=0.21%

Run status group 0 (all jobs):
READ: io=13144KB, aggrb=435KB/s, minb=446KB/s, maxb=446KB/s, mint=30168msec, maxt=30168msec

Disk stats (read/write):
sdb: ios=3283/0, merge=0/0, ticks=479629/0, in_queue=481207, util=99.64%

Чтение: iodepth=8, bw=379415 B/s, iops=92, avg=86.29 msec
root@host# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=8
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [380K/0K /s] [92 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=26848
read: io=11152KB, bw=379415 B/s, iops=92, runt= 30098msec
slat (usec): min=5, max=154, avg=17.66, stdev= 4.06
clat (msec): min=9, max=502, avg=86.29, stdev=67.28
lat (msec): min=9, max=502, avg=86.31, stdev=67.28
bw (KB/s): min= 340, max= 406, per=100.17%, avg=370.64, stdev=15.20
cpu: usr=0.10%, sys=0.25%, ctx=2788, majf=0, minf=31
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=99.7%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=2788/0/0, short=0/0/0

lat (msec): 10=0.04%, 20=6.74%, 50=32.89%, 100=27.19%, 250=30.42%
lat (msec): 500=2.69%, 750=0.04%

Run status group 0 (all jobs):
READ: io=11152KB, aggrb=370KB/s, minb=379KB/s, maxb=379KB/s, mint=30098msec, maxt=30098msec

Disk stats (read/write):
sdb: ios=2770/0, merge=0/0, ticks=238618/0, in_queue=239079, util=99.65%

Чтение: iodepth=4, bw=301030 B/s, iops=73, avg=54.37 msec
root@host# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=4
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [299K/0K /s] [73 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=26863
read: io=8836.0KB, bw=301030 B/s, iops=73, runt= 30057msec
slat (usec): min=6, max=41, avg=17.78, stdev= 3.12
clat (msec): min=7, max=355, avg=54.37, stdev=38.77
lat (msec): min=7, max=355, avg=54.39, stdev=38.77
bw (KB/s): min= 193, max= 331, per=100.53%, avg=294.55, stdev=20.61
cpu: usr=0.09%, sys=0.20%, ctx=2209, majf=0, minf=27
IO depths: 1=0.1%, 2=0.1%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=2209/0/0, short=0/0/0

lat (msec): 10=0.14%, 20=5.57%, 50=55.82%, 100=27.89%, 250=10.10%
lat (msec): 500=0.50%

Run status group 0 (all jobs):
READ: io=8836KB, aggrb=293KB/s, minb=301KB/s, maxb=301KB/s, mint=30057msec, maxt=30057msec

Disk stats (read/write):
sdb: ios=2196/0, merge=0/0, ticks=119319/0, in_queue=119501, util=99.63%

Чтение: iodepth=2, bw=278016 B/s, iops=67, avg=29.43 msec
root@host# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=2
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [290K/0K /s] [71 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=26875
read: io=8148.0KB, bw=278016 B/s, iops=67, runt= 30011msec
slat (usec): min=7, max=41, avg=18.08, stdev= 2.98
clat (msec): min=4, max=249, avg=29.43, stdev=24.66
lat (msec): min=4, max=249, avg=29.45, stdev=24.66
bw (KB/s): min= 246, max= 309, per=100.03%, avg=271.07, stdev=12.89
cpu: usr=0.09%, sys=0.19%, ctx=2037, majf=0, minf=26
IO depths: 1=0.1%, 2=100.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=2037/0/0, short=0/0/0

lat (msec): 10=9.33%, 20=38.49%, 50=37.60%, 100=12.37%, 250=2.21%

Run status group 0 (all jobs):
READ: io=8148KB, aggrb=271KB/s, minb=278KB/s, maxb=278KB/s, mint=30011msec, maxt=30011msec

Disk stats (read/write):
sdb: ios=2027/0, merge=0/0, ticks=59697/0, in_queue=59716, util=99.60%

Чтение: iodepth=1, bw=258021 B/s, iops=62, avg=15.85 msec
root@host# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [266K/0K /s] [65 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=26893
read: io=7564.0KB, bw=258021 B/s, iops=62, runt= 30019msec
slat (usec): min=9, max=39, avg=17.66, stdev= 3.15
clat (msec): min=2, max=250, avg=15.85, stdev= 6.91
lat (msec): min=2, max=250, avg=15.87, stdev= 6.91
bw (KB/s): min= 167, max= 291, per=100.32%, avg=251.80, stdev=18.56
cpu: usr=0.09%, sys=0.16%, ctx=1891, majf=0, minf=24
IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=1891/0/0, short=0/0/0

lat (msec): 4=0.16%, 10=9.62%, 20=73.61%, 50=16.55%, 500=0.05%

Run status group 0 (all jobs):
READ: io=7564KB, aggrb=251KB/s, minb=258KB/s, maxb=258KB/s, mint=30019msec, maxt=30019msec

Disk stats (read/write):
sdb: ios=1883/0, merge=0/0, ticks=29818/0, in_queue=29829, util=99.45%


Измерение последовательного чтения:
root@host# dd if=/dev/sdb  of=/dev/null iflag=direct bs=2M
953864+1 records in
953864+1 records out
2000398934016 bytes (2.0 TB) copied, 15221 s, 131 MB/s


Анализ чтения:

Скорость линейного чтения очень высокая — 131 Мбайт/с. Не удалось добиться нужного уровня avg.latency. Наиболее близкие результаты: iodepth=1, bw=258021 B/s, iops=62, avg=15.85 msec.

Запись: iodepth=1024, bw=643734 B/s, iops=157, avg=6071.14 msec
root@home# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1024
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [w] [0.0% done] [0K/527K /s] [0 /128 iops] [eta 45d:14h:53m:02s]
writetest: (groupid=0, jobs=1): err= 0: pid=26924
write: io=19456KB, bw=643734 B/s, iops=157, runt= 30949msec
slat (usec): min=2, max=282743, avg=6210.03, stdev=35626.77
clat (msec): min=727, max=11001, avg=6071.14, stdev=1458.55
lat (msec): min=962, max=11001, avg=6077.35, stdev=1458.99
bw (KB/s): min= 0, max= 728, per=3.71%, avg=23.31, stdev=118.34
cpu: usr=0.06%, sys=0.07%, ctx=276, majf=0, minf=22
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.3%, 32=0.7%, >=64=98.7%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
issued r/w/d: total=0/4864/0, short=0/0/0

lat (msec): 750=0.02%, 1000=0.66%, 2000=3.29%, >=2000=96.03%

Run status group 0 (all jobs):
WRITE: io=19456KB, aggrb=628KB/s, minb=643KB/s, maxb=643KB/s, mint=30949msec, maxt=30949msec

Disk stats (read/write):
sdb: ios=83/4855, merge=0/0, ticks=131/4370631, in_queue=4378949, util=100.00%

Запись: iodepth=32, bw=589269 B/s, iops=143, avg=222319.82 usec
root@host# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/581K /s] [0 /141 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=26956
write: io=17380KB, bw=589269 B/s, iops=143, runt= 30202msec
slat (usec): min=4, max=64, avg=15.65, stdev= 2.85
clat (usec): min=497, max=398588, avg=222319.82, stdev=37626.92
lat (usec): min=519, max=398600, avg=222335.96, stdev=37627.65
bw (KB/s): min= 486, max= 1093, per=99.97%, avg=574.85, stdev=77.20
cpu: usr=0.21%, sys=0.36%, ctx=4345, majf=0, minf=22
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.4%, 32=99.3%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued r/w/d: total=0/4345/0, short=0/0/0
lat (usec): 500=0.02%
lat (msec): 2=0.14%, 4=0.30%, 10=1.17%, 50=0.09%, 100=0.16%
lat (msec): 250=83.87%, 500=14.25%

Run status group 0 (all jobs):
WRITE: io=17380KB, aggrb=575KB/s, minb=589KB/s, maxb=589KB/s, mint=30202msec, maxt=30202msec

Disk stats (read/write):
sdb: ios=83/4336, merge=0/0, ticks=64/961069, in_queue=962665, util=99.84%

Запись: iodepth=16, bw=590036 B/s, iops=144, avg=110994.39 msec
root@host# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=16
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/573K /s] [0 /140 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=26980
write: io=17364KB, bw=590036 B/s, iops=144, runt= 30135msec
slat (usec): min=4, max=53, avg=19.06, stdev= 3.23
clat (usec): min=355, max=222467, avg=110994.39, stdev=20437.11
lat (usec): min=381, max=222490, avg=111013.98, stdev=20437.75
bw (KB/s): min= 491, max= 1109, per=100.16%, avg=576.90, stdev=78.30
cpu: usr=0.14%, sys=0.43%, ctx=4342, majf=0, minf=22
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=99.7%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=0/4341/0, short=0/0/0
lat (usec): 500=0.02%, 1000=0.02%
lat (msec): 2=0.16%, 4=1.41%, 10=0.02%, 50=0.12%, 100=19.79%
lat (msec): 250=78.46%

Run status group 0 (all jobs):
WRITE: io=17364KB, aggrb=576KB/s, minb=590KB/s, maxb=590KB/s, mint=30135msec, maxt=30135msec

Disk stats (read/write):
sdb: ios=83/4341, merge=0/0, ticks=67/480952, in_queue=481017, util=99.84%

Запись: iodepth=8, bw=584675 B/s, iops=142, avg=55996.99 usec
root@host# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=8
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/626K /s] [0 /152 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=27008
write: io=17160KB, bw=584675 B/s, iops=142, runt= 30054msec
slat (usec): min=5, max=44, avg=19.01, stdev= 3.15
clat (usec): min=278, max=95168, avg=55996.99, stdev=11865.20
lat (usec): min=307, max=95187, avg=56016.54, stdev=11865.82
bw (KB/s): min= 458, max= 1099, per=100.08%, avg=570.47, stdev=79.55
cpu: usr=0.17%, sys=0.40%, ctx=4291, majf=0, minf=22
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=99.8%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=0/4290/0, short=0/0/0
lat (usec): 500=0.02%, 750=0.02%, 1000=0.02%
lat (msec): 2=1.56%, 4=0.02%, 20=0.02%, 50=25.78%, 100=72.54%

Run status group 0 (all jobs):
WRITE: io=17160KB, aggrb=570KB/s, minb=584KB/s, maxb=584KB/s, mint=30054msec, maxt=30054msec

Disk stats (read/write):
sdb: ios=83/4264, merge=0/0, ticks=59/238810, in_queue=239070, util=99.83%

Запись: iodepth=4, bw=583096 B/s, iops=142, avg=28063.83 msec
root@home# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=4
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/573K /s] [0 /140 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=27030
write: io=17100KB, bw=583096 B/s, iops=142, runt= 30030msec
slat (usec): min=5, max=37, avg=18.82, stdev= 3.02
clat (usec): min=274, max=53388, avg=28063.83, stdev=6724.53
lat (usec): min=296, max=53410, avg=28083.17, stdev=6725.00
bw (KB/s): min= 492, max= 1109, per=100.14%, avg=569.81, stdev=78.93
cpu: usr=0.17%, sys=0.39%, ctx=4275, majf=0, minf=22
IO depths: 1=0.1%, 2=0.1%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=0/4275/0, short=0/0/0
lat (usec): 500=0.02%, 750=1.54%, 1000=0.07%
lat (msec): 4=0.02%, 20=5.38%, 50=92.82%, 100=0.14%

Run status group 0 (all jobs):
WRITE: io=17100KB, aggrb=569KB/s, minb=583KB/s, maxb=583KB/s, mint=30030msec, maxt=30030msec

Disk stats (read/write):
sdb: ios=83/4256, merge=0/0, ticks=60/119372, in_queue=119517, util=99.82%

Запись: iodepth=2, bw=585825 B/s, iops=143, avg=13954.43 usec
root@host# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=2
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/626K /s] [0 /152 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=27064
write: io=17172KB, bw=585825 B/s, iops=143, runt= 30016msec
slat (usec): min=6, max=40, avg=18.67, stdev= 3.01
clat (usec): min=158, max=27959, avg=13954.43, stdev=3872.39
lat (usec): min=186, max=27980, avg=13973.63, stdev=3872.96
bw (KB/s): min= 476, max= 1087, per=99.98%, avg=571.86, stdev=78.33
cpu: usr=0.21%, sys=0.36%, ctx=4293, majf=0, minf=22
IO depths: 1=0.1%, 2=100.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=0/4293/0, short=0/0/0
lat (usec): 250=0.02%, 500=1.54%, 750=0.09%
lat (msec): 10=10.16%, 20=84.30%, 50=3.89%

Run status group 0 (all jobs):
WRITE: io=17172KB, aggrb=572KB/s, minb=585KB/s, maxb=585KB/s, mint=30016msec, maxt=30016msec

Disk stats (read/write):
sdb: ios=83/4273, merge=0/0, ticks=64/59670, in_queue=59743, util=99.80%

Запись: iodepth=1, bw=580929 B/s, iops=141, avg=7025.40 usec
root@host# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1
fio 1.57
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/569K /s] [0 /139 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=27088
write: io=17020KB, bw=580929 B/s, iops=141, runt= 30001msec
slat (usec): min=6, max=74, avg=18.97, stdev= 3.19
clat (usec): min=137, max=18822, avg=7025.40, stdev=2379.35
lat (usec): min=151, max=18841, avg=7044.92, stdev=2379.66
bw (KB/s): min= 459, max= 1117, per=100.04%, avg=567.25, stdev=81.49
cpu: usr=0.18%, sys=0.38%, ctx=4255, majf=0, minf=22
IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=0/4255/0, short=0/0/0
lat (usec): 250=1.60%, 500=0.07%
lat (msec): 2=0.05%, 4=5.69%, 10=87.99%, 20=4.61%

Run status group 0 (all jobs):
WRITE: io=17020KB, aggrb=567KB/s, minb=580KB/s, maxb=580KB/s, mint=30001msec, maxt=30001msec

Disk stats (read/write):
sdb: ios=83/4235, merge=0/0, ticks=62/29802, in_queue=29863, util=99.56%


Анализ записи:

С записью, на удивление, оказалось всё лучше. Удалось добиться 141 IOPS при 7 мс latency. iodepth=1, bw=580929 B/s, iops=141, avg=7025.40 usec.

SATA-диск enterprise-level Western Digital RE3 WD2502ABYS-23B7A0 250 GB 7200 об/мин 16 Мбайт

Чтение: iodepth=1024, bw=598599 B/s, iops=146, avg=6409.46 msec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1024
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [0.0% done] [516K/0K /s] [129 /0 iops] [eta 06d:06h:39m:59s]
readtest: (groupid=0, jobs=1): err= 0: pid=3813
read: io=18048KB, bw=598599 B/s, iops=146, runt= 30874msec
slat (usec): min=2, max=287017, avg=6648.57, stdev=37692.92
clat (msec): min=863, max=9575, avg=6409.46, stdev=1605.46
lat (msec): min=1077, max=9575, avg=6416.11, stdev=1605.43
clat percentiles (msec):
| 1.00th=[ 1303], 5.00th=[ 2573], 10.00th=[ 4080], 20.00th=[ 6128],
| 30.00th=[ 6194], 40.00th=[ 6325], 50.00th=[ 6587], 60.00th=[ 6783],
| 70.00th=[ 7177], 80.00th=[ 7635], 90.00th=[ 8160], 95.00th=[ 8356],
| 99.00th=[ 8717], 99.50th=[ 8848], 99.90th=[ 9110], 99.95th=[ 9372],
| 99.99th=[ 9634]
bw (KB/s): min= 0, max= 644, per=95.81%, avg=559.53, stdev=117.13
lat (msec): 1000=0.02%, 2000=3.55%, >=2000=96.43%
cpu: usr=0.03%, sys=0.08%, ctx=265, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.4%, 32=0.7%, >=64=98.6%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
issued: total=r=4512/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=18048KB, aggrb=584KB/s, minb=584KB/s, maxb=584KB/s, mint=30874msec, maxt=30874msec

Disk stats (read/write):
sda: ios=4482/0, merge=0/0, ticks=4311286/0, in_queue=4353880, util=99.73%

Чтение: iodepth=32, bw=512118 B/s, iops=125, avg=255.83 msec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [472K/0K /s] [118 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=3826
read: io=15136KB, bw=512118 B/s, iops=125, runt= 30265msec
slat (usec): min=4, max=129, avg=23.86, stdev= 4.10
clat (msec): min=8, max=1241, avg=255.83, stdev=212.30
lat (msec): min=8, max=1241, avg=255.86, stdev=212.30
clat percentiles (msec):
| 1.00th=[ 21], 5.00th=[ 34], 10.00th=[ 45], 20.00th=[ 71],
| 30.00th=[ 99], 40.00th=[ 133], 50.00th=[ 180], 60.00th=[ 247],
| 70.00th=[ 338], 80.00th=[ 449], 90.00th=[ 586], 95.00th=[ 685],
| 99.00th=[ 840], 99.50th=[ 889], 99.90th=[ 996], 99.95th=[ 1045],
| 99.99th=[ 1237]
bw (KB/s): min= 438, max= 538, per=100.00%, avg=500.36, stdev=23.40
lat (msec): 10=0.03%, 20=0.79%, 50=11.44%, 100=18.02%, 250=30.02%
lat (msec): 500=23.65%, 750=13.48%, 1000=2.51%, 2000=0.05%
cpu: usr=0.31%, sys=0.48%, ctx=3784, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.4%, 32=99.2%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=3784/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=15136KB, aggrb=500KB/s, minb=500KB/s, maxb=500KB/s, mint=30265msec, maxt=30265msec

Disk stats (read/write):
sda: ios=3771/0, merge=0/0, ticks=958434/0, in_queue=963377, util=99.72%

Чтение: iodepth=16, bw=474046 B/s, iops=115, avg=138.18 msec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=16
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [460K/0K /s] [115 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=3832
read: io=13964KB, bw=474046 B/s, iops=115, runt= 30164msec
slat (usec): min=4, max=48, avg= 9.72, stdev= 2.26
clat (msec): min=11, max=677, avg=138.18, stdev=104.64
lat (msec): min=11, max=677, avg=138.19, stdev=104.64
clat percentiles (msec):
| 1.00th=[ 21], 5.00th=[ 29], 10.00th=[ 37], 20.00th=[ 55],
| 30.00th=[ 72], 40.00th=[ 90], 50.00th=[ 108], 60.00th=[ 129],
| 70.00th=[ 159], 80.00th=[ 210], 90.00th=[ 289], 95.00th=[ 359],
| 99.00th=[ 474], 99.50th=[ 545], 99.90th=[ 652], 99.95th=[ 660],
| 99.99th=[ 676]
bw (KB/s): min= 422, max= 507, per=100.00%, avg=463.22, stdev=21.17
lat (msec): 20=0.92%, 50=16.47%, 100=28.87%, 250=39.01%, 500=14.01%
lat (msec): 750=0.72%
cpu: usr=0.16%, sys=0.20%, ctx=3491, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=99.6%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=3491/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=13964KB, aggrb=462KB/s, minb=462KB/s, maxb=462KB/s, mint=30164msec, maxt=30164msec

Disk stats (read/write):
sda: ios=3490/0, merge=0/0, ticks=481087/0, in_queue=481253, util=99.72%

Чтение: iodepth=16, bw=474046 B/s, iops=115, avg=138.18 msec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=16
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [460K/0K /s] [115 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=3832
read: io=13964KB, bw=474046 B/s, iops=115, runt= 30164msec
slat (usec): min=4, max=48, avg= 9.72, stdev= 2.26
clat (msec): min=11, max=677, avg=138.18, stdev=104.64
lat (msec): min=11, max=677, avg=138.19, stdev=104.64
clat percentiles (msec):
| 1.00th=[ 21], 5.00th=[ 29], 10.00th=[ 37], 20.00th=[ 55],
| 30.00th=[ 72], 40.00th=[ 90], 50.00th=[ 108], 60.00th=[ 129],
| 70.00th=[ 159], 80.00th=[ 210], 90.00th=[ 289], 95.00th=[ 359],
| 99.00th=[ 474], 99.50th=[ 545], 99.90th=[ 652], 99.95th=[ 660],
| 99.99th=[ 676]
bw (KB/s): min= 422, max= 507, per=100.00%, avg=463.22, stdev=21.17
lat (msec): 20=0.92%, 50=16.47%, 100=28.87%, 250=39.01%, 500=14.01%
lat (msec): 750=0.72%
cpu: usr=0.16%, sys=0.20%, ctx=3491, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=99.6%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=3491/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=13964KB, aggrb=462KB/s, minb=462KB/s, maxb=462KB/s, mint=30164msec, maxt=30164msec

Disk stats (read/write):
sda: ios=3490/0, merge=0/0, ticks=481087/0, in_queue=481253, util=99.72%

Чтение: iodepth=8, bw=443783 B/s, iops=108, avg=73.81 msec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=8
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [448K/0K /s] [112 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=3836
read: io=13024KB, bw=443783 B/s, iops=108, runt= 30052msec
slat (usec): min=4, max=47, avg= 9.73, stdev= 1.99
clat (msec): min=7, max=211, avg=73.81, stdev=31.56
lat (msec): min=7, max=211, avg=73.82, stdev=31.56
clat percentiles (msec):
| 1.00th=[ 21], 5.00th=[ 27], 10.00th=[ 33], 20.00th=[ 43],
| 30.00th=[ 52], 40.00th=[ 63], 50.00th=[ 74], 60.00th=[ 83],
| 70.00th=[ 93], 80.00th=[ 104], 90.00th=[ 117], 95.00th=[ 126],
| 99.00th=[ 143], 99.50th=[ 149], 99.90th=[ 178], 99.95th=[ 180],
| 99.99th=[ 212]
bw (KB/s): min= 391, max= 463, per=100.00%, avg=433.20, stdev=16.73
lat (msec): 10=0.03%, 20=0.71%, 50=27.12%, 100=49.57%, 250=22.57%
cpu: usr=0.12%, sys=0.22%, ctx=3257, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=99.8%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=3256/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=13024KB, aggrb=433KB/s, minb=433KB/s, maxb=433KB/s, mint=30052msec, maxt=30052msec

Disk stats (read/write):
sda: ios=3240/0, merge=0/0, ticks=238833/0, in_queue=239203, util=99.70%

Чтение: iodepth=4, bw=361840 B/s, iops=88, avg=45.25 msec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=4
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [356K/0K /s] [89 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=3840
read: io=10616KB, bw=361840 B/s, iops=88, runt= 30043msec
slat (usec): min=5, max=46, avg= 9.62, stdev= 1.88
clat (msec): min=12, max=102, avg=45.25, stdev=12.46
lat (msec): min=13, max=102, avg=45.26, stdev=12.46
clat percentiles (msec):
| 1.00th=[ 23], 5.00th=[ 28], 10.00th=[ 30], 20.00th=[ 35],
| 30.00th=[ 38], 40.00th=[ 42], 50.00th=[ 45], 60.00th=[ 48],
| 70.00th=[ 51], 80.00th=[ 56], 90.00th=[ 63], 95.00th=[ 68],
| 99.00th=[ 80], 99.50th=[ 84], 99.90th=[ 97], 99.95th=[ 99],
| 99.99th=[ 103]
bw (KB/s): min= 322, max= 395, per=100.00%, avg=353.08, stdev=17.19
lat (msec): 20=0.26%, 50=66.84%, 100=32.86%, 250=0.04%
cpu: usr=0.18%, sys=0.11%, ctx=2654, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=2654/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=10616KB, aggrb=353KB/s, minb=353KB/s, maxb=353KB/s, mint=30043msec, maxt=30043msec

Disk stats (read/write):
sda: ios=2642/0, merge=0/0, ticks=119520/0, in_queue=119583, util=99.72%

Чтение: iodepth=2, bw=351144 B/s, iops=85, avg=23.29 msec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=2
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [320K/0K /s] [80 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=3844
read: io=10296KB, bw=351144 B/s, iops=85, runt= 30025msec
slat (usec): min=8, max=46, avg=20.46, stdev= 5.06
clat (msec): min=7, max=42, avg=23.29, stdev= 5.06
lat (msec): min=7, max=42, avg=23.32, stdev= 5.06
clat percentiles (usec):
| 1.00th=[11968], 5.00th=[15168], 10.00th=[16768], 20.00th=[19072],
| 30.00th=[20608], 40.00th=[21888], 50.00th=[23168], 60.00th=[24448],
| 70.00th=[25984], 80.00th=[27520], 90.00th=[30080], 95.00th=[31872],
| 99.00th=[35584], 99.50th=[37120], 99.90th=[39168], 99.95th=[39168],
| 99.99th=[42240]
bw (KB/s): min= 314, max= 415, per=100.00%, avg=342.71, stdev=18.16
lat (msec): 10=0.16%, 20=25.41%, 50=74.44%
cpu: usr=0.14%, sys=0.35%, ctx=2574, majf=0, minf=0
IO depths: 1=0.1%, 2=100.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=2574/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=10296KB, aggrb=342KB/s, minb=342KB/s, maxb=342KB/s, mint=30025msec, maxt=30025msec

Disk stats (read/write):
sda: ios=2563/0, merge=0/0, ticks=59727/0, in_queue=59753, util=99.71%

Чтение: iodepth=1, bw=349284 B/s, iops=85, avg=11.69 msec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [328K/0K /s] [82 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=3852
read: io=10236KB, bw=349284 B/s, iops=85, runt= 30009msec
slat (usec): min=8, max=50, avg=31.99, stdev= 3.37
clat (msec): min=2, max=22, avg=11.69, stdev= 3.49
lat (msec): min=2, max=22, avg=11.72, stdev= 3.49
clat percentiles (usec):
| 1.00th=[ 3984], 5.00th=[ 5920], 10.00th=[ 7200], 20.00th=[ 8512],
| 30.00th=[ 9792], 40.00th=[10816], 50.00th=[11584], 60.00th=[12608],
| 70.00th=[13504], 80.00th=[14784], 90.00th=[16320], 95.00th=[17536],
| 99.00th=[19584], 99.50th=[20096], 99.90th=[21376], 99.95th=[21376],
| 99.99th=[22400]
bw (KB/s): min= 310, max= 390, per=100.00%, avg=341.07, stdev=16.19
lat (msec): 4=1.02%, 10=31.11%, 20=67.37%, 50=0.51%
cpu: usr=0.06%, sys=0.41%, ctx=2559, majf=0, minf=0
IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=2559/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=10236KB, aggrb=341KB/s, minb=341KB/s, maxb=341KB/s, mint=30009msec, maxt=30009msec

Disk stats (read/write):
sda: ios=2552/0, merge=0/0, ticks=29780/0, in_queue=29786, util=99.29%


Измерение последовательного чтения:
root@host:~# dd if=/dev/sda  of=/dev/null iflag=direct bs=2M
119237+1 records in
119237+1 records out
250059350016 bytes (250 GB) copied, 2603.86 s, 96.0 MB/s


Анализ чтения:

Скорость линейного чтения ниже, чем у предыдущего диска consumer-level. Не удалось получить результат при заданном уровне avg.latency < 10 мс. Наиболее близкий результат на чтение: iodepth=1, bw=349284 B/s, iops=85, avg=11.69 msec.

Запись: iodepth=1024, bw=792709 B/s, iops=193, avg=5036.55 msec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1024
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [0.0% done] [0K/644K /s] [0 /161 iops] [eta 04d:10h:34m:55s]
writetest: (groupid=0, jobs=1): err= 0: pid=3862
write: io=23820KB, bw=792709 B/s, iops=193, runt= 30770msec
slat (usec): min=3, max=226929, avg=5052.81, stdev=28899.87
clat (msec): min=665, max=7288, avg=5036.55, stdev=1103.42
lat (msec): min=877, max=7288, avg=5041.60, stdev=1103.79
clat percentiles (msec):
| 1.00th=[ 1057], 5.00th=[ 2376], 10.00th=[ 3982], 20.00th=[ 4686],
| 30.00th=[ 4817], 40.00th=[ 4948], 50.00th=[ 5145], 60.00th=[ 5276],
| 70.00th=[ 5538], 80.00th=[ 5866], 90.00th=[ 6194], 95.00th=[ 6390],
| 99.00th=[ 6718], 99.50th=[ 6783], 99.90th=[ 6980], 99.95th=[ 7111],
| 99.99th=[ 7308]
bw (KB/s): min= 1, max= 850, per=93.87%, avg=726.55, stdev=140.26
lat (msec): 750=0.02%, 1000=0.54%, 2000=3.22%, >=2000=96.22%
cpu: usr=0.03%, sys=0.12%, ctx=311, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.3%, 32=0.5%, >=64=98.9%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
issued: total=r=0/w=0/d=5955, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=23820KB, aggrb=774KB/s, minb=774KB/s, maxb=774KB/s, mint=30770msec, maxt=30770msec

Disk stats (read/write):
sda: ios=84/5933, merge=0/0, ticks=230/4329180, in_queue=4359370, util=100.00%

Запись: iodepth=32, bw=782310 B/s, iops=190, avg=167484.47 usec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/736K /s] [0 /184 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=3885
write: io=23056KB, bw=782310 B/s, iops=190, runt= 30179msec
slat (usec): min=3, max=132, avg=19.51, stdev= 8.95
clat (usec): min=356, max=866732, avg=167484.47, stdev=145167.58
lat (usec): min=361, max=866744, avg=167504.66, stdev=145168.06
clat percentiles (msec):
| 1.00th=[ 4], 5.00th=[ 18], 10.00th=[ 28], 20.00th=[ 48],
| 30.00th=[ 71], 40.00th=[ 93], 50.00th=[ 117], 60.00th=[ 151],
| 70.00th=[ 210], 80.00th=[ 277], 90.00th=[ 396], 95.00th=[ 474],
| 99.00th=[ 603], 99.50th=[ 635], 99.90th=[ 725], 99.95th=[ 791],
| 99.99th=[ 865]
bw (KB/s): min= 646, max= 1865, per=100.00%, avg=764.76, stdev=150.29
lat (usec): 500=0.05%, 750=0.03%, 1000=0.07%
lat (msec): 2=0.33%, 4=0.57%, 10=1.56%, 20=3.54%, 50=14.50%
lat (msec): 100=23.30%, 250=32.43%, 500=19.69%, 750=3.85%, 1000=0.07%
cpu: usr=0.24%, sys=0.69%, ctx=5767, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.3%, 32=99.5%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=5764, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=23056KB, aggrb=763KB/s, minb=763KB/s, maxb=763KB/s, mint=30179msec, maxt=30179msec

Disk stats (read/write):
sda: ios=84/5760, merge=0/0, ticks=223/960820, in_queue=962817, util=100.00%

Запись: iodepth=16, bw=769140 B/s, iops=187, avg=85163.46 usec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=16
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/712K /s] [0 /178 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=3907
write: io=22592KB, bw=769140 B/s, iops=187, runt= 30078msec
slat (usec): min=3, max=43, avg=19.45, stdev= 8.87
clat (usec): min=355, max=515043, avg=85163.46, stdev=55854.61
lat (usec): min=360, max=515074, avg=85183.85, stdev=55855.21
clat percentiles (usec):
| 1.00th=[ 1720], 5.00th=[15936], 10.00th=[23424], 20.00th=[37120],
| 30.00th=[50432], 40.00th=[64256], 50.00th=[78336], 60.00th=[91648],
| 70.00th=[105984], 80.00th=[122368], 90.00th=[150528], 95.00th=[183296],
| 99.00th=[284672], 99.50th=[313344], 99.90th=[391168], 99.95th=[440320],
| 99.99th=[514048]
bw (KB/s): min= 576, max= 1860, per=100.00%, avg=751.53, stdev=151.70
lat (usec): 500=0.05%, 750=0.19%, 1000=0.19%
lat (msec): 2=0.80%, 4=1.10%, 10=0.30%, 20=4.98%, 50=22.17%
lat (msec): 100=36.03%, 250=32.74%, 500=1.43%, 750=0.02%
cpu: usr=0.26%, sys=0.65%, ctx=5649, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=99.7%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=5648, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=22592KB, aggrb=751KB/s, minb=751KB/s, maxb=751KB/s, mint=30078msec, maxt=30078msec

Disk stats (read/write):
sda: ios=84/5613, merge=0/0, ticks=210/477223, in_queue=478567, util=100.00%

Запись: iodepth=8, bw=780963 B/s, iops=190, avg=41927.87 usec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=8
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/708K /s] [0 /177 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=3929
write: io=22908KB, bw=780963 B/s, iops=190, runt= 30037msec
slat (usec): min=3, max=133, avg=16.44, stdev= 7.30
clat (usec): min=356, max=166901, avg=41927.87, stdev=19827.30
lat (usec): min=360, max=166930, avg=41946.17, stdev=19827.88
clat percentiles (usec):
| 1.00th=[ 932], 5.00th=[13888], 10.00th=[17280], 20.00th=[23168],
| 30.00th=[29568], 40.00th=[35072], 50.00th=[40704], 60.00th=[46848],
| 70.00th=[52992], 80.00th=[59648], 90.00th=[68096], 95.00th=[75264],
| 99.00th=[88576], 99.50th=[95744], 99.90th=[113152], 99.95th=[116224],
| 99.99th=[166912]
bw (KB/s): min= 632, max= 1936, per=100.00%, avg=763.63, stdev=161.74
lat (usec): 500=0.30%, 750=0.40%, 1000=0.44%
lat (msec): 2=1.10%, 4=0.23%, 10=0.16%, 20=11.96%, 50=50.29%
lat (msec): 100=34.78%, 250=0.35%
cpu: usr=0.31%, sys=0.61%, ctx=5728, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=99.9%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=5727, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=22908KB, aggrb=762KB/s, minb=762KB/s, maxb=762KB/s, mint=30037msec, maxt=30037msec

Disk stats (read/write):
sda: ios=84/5703, merge=0/0, ticks=96/238850, in_queue=239140, util=100.00%

Запись: iodepth=4, bw=773543 B/s, iops=188, avg=21153.83 usec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=4
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/768K /s] [0 /192 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=3951
write: io=22676KB, bw=773543 B/s, iops=188, runt= 30018msec
slat (usec): min=3, max=44, avg=15.74, stdev= 6.49
clat (usec): min=306, max=66557, avg=21153.83, stdev=7431.97
lat (usec): min=335, max=66586, avg=21171.31, stdev=7432.45
clat percentiles (usec):
| 1.00th=[ 540], 5.00th=[12352], 10.00th=[13632], 20.00th=[15424],
| 30.00th=[17280], 40.00th=[18816], 50.00th=[20352], 60.00th=[22144],
| 70.00th=[23936], 80.00th=[26752], 90.00th=[30592], 95.00th=[34048],
| 99.00th=[41728], 99.50th=[45824], 99.90th=[52992], 99.95th=[55552],
| 99.99th=[66048]
bw (KB/s): min= 630, max= 1904, per=100.00%, avg=756.66, stdev=156.44
lat (usec): 500=0.88%, 750=1.18%, 1000=0.37%
lat (msec): 2=0.11%, 4=0.02%, 10=0.19%, 20=44.47%, 50=52.55%
lat (msec): 100=0.23%
cpu: usr=0.33%, sys=0.56%, ctx=5671, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=5669, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=22676KB, aggrb=755KB/s, minb=755KB/s, maxb=755KB/s, mint=30018msec, maxt=30018msec

Disk stats (read/write):
sda: ios=84/5652, merge=0/0, ticks=100/119303, in_queue=119453, util=100.00%

Запись: iodepth=2, bw=769245 B/s, iops=187, avg=10622.52 usec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=2
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/724K /s] [0 /181 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=3974
write: io=22544KB, bw=769245 B/s, iops=187, runt= 30010msec
slat (usec): min=4, max=46, avg=18.56, stdev= 7.38
clat (usec): min=157, max=28381, avg=10622.52, stdev=3481.52
lat (usec): min=190, max=28406, avg=10642.92, stdev=3482.38
clat percentiles (usec):
| 1.00th=[ 290], 5.00th=[ 7200], 10.00th=[ 7904], 20.00th=[ 8640],
| 30.00th=[ 9024], 40.00th=[ 9536], 50.00th=[ 9920], 60.00th=[10432],
| 70.00th=[11072], 80.00th=[12096], 90.00th=[16320], 95.00th=[18048],
| 99.00th=[20096], 99.50th=[20608], 99.90th=[25472], 99.95th=[26496],
| 99.99th=[28288]
bw (KB/s): min= 664, max= 1920, per=100.00%, avg=751.66, stdev=158.62
lat (usec): 250=0.71%, 500=1.81%, 750=0.05%
lat (msec): 2=0.04%, 10=48.49%, 20=47.91%, 50=0.99%
cpu: usr=0.25%, sys=0.66%, ctx=5639, majf=0, minf=0
IO depths: 1=0.1%, 2=100.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=5636, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=22544KB, aggrb=751KB/s, minb=751KB/s, maxb=751KB/s, mint=30010msec, maxt=30010msec

Disk stats (read/write):
sda: ios=84/5618, merge=0/0, ticks=97/59580, in_queue=59680, util=100.00%

Запись: iodepth=1, bw=768989 B/s, iops=187, avg=5294.30 usec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/768K /s] [0 /192 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=3997
write: io=22532KB, bw=768989 B/s, iops=187, runt= 30004msec
slat (usec): min=6, max=182, avg=26.05, stdev=11.19
clat (usec): min=113, max=17523, avg=5294.30, stdev=2342.17
lat (usec): min=120, max=17539, avg=5321.15, stdev=2342.65
clat percentiles (usec):
| 1.00th=[ 141], 5.00th=[ 2864], 10.00th=[ 3472], 20.00th=[ 3856],
| 30.00th=[ 4320], 40.00th=[ 4576], 50.00th=[ 4896], 60.00th=[ 5280],
| 70.00th=[ 5600], 80.00th=[ 6112], 90.00th=[ 7264], 95.00th=[11712],
| 99.00th=[13760], 99.50th=[14144], 99.90th=[16512], 99.95th=[16768],
| 99.99th=[17536]
bw (KB/s): min= 624, max= 1892, per=100.00%, avg=751.07, stdev=156.08
lat (usec): 250=2.52%, 500=0.04%, 750=0.04%, 1000=0.04%
lat (msec): 2=0.25%, 4=20.34%, 10=71.10%, 20=5.68%
cpu: usr=0.12%, sys=0.66%, ctx=5636, majf=0, minf=0
IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=5633, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=22532KB, aggrb=750KB/s, minb=750KB/s, maxb=750KB/s, mint=30004msec, maxt=30004msec

Disk stats (read/write):
sda: ios=84/5613, merge=0/0, ticks=100/29717, in_queue=29817, util=99.39%


Измерение последовательной записи:
root@host:~# dd if=/dev/zero of=/dev/sda oflag=direct bs=2M   
dd: writing `/dev/sda': No space left on device
119238+0 records in
119237+0 records out
250059350016 bytes (250 GB) copied, 2615.61 s, 95.6 MB/s


Анализ записи:

Следует отметить стабильность IOPS от размера очереди на протяжении всего теста. Характеристика накопителя: iodepth=1, bw=768989 B/s, iops=187, avg=5294.30 usec.

SATA-диск consumer-level Seagate Barracuda 7200.11 ST3320613AS 320 GB 7200 об/мин 16 Mбайт

Чтение: iodepth=1024, bw=452666 B/s, iops=110, avg=8229.64 msec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1024
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [0.0% done] [260K/0K /s] [65 /0 iops] [eta 11d:12h:33m:43s]
readtest: (groupid=0, jobs=1): err= 0: pid=3795
read: io=13824KB, bw=452666 B/s, iops=110, runt= 31272msec
slat (usec): min=3, max=411507, avg=8701.29, stdev=49676.88
clat (msec): min=1190, max=12289, avg=8229.64, stdev=2180.39
lat (msec): min=1445, max=12289, avg=8238.34, stdev=2180.71
clat percentiles (msec):
| 1.00th=[ 1860], 5.00th=[ 3064], 10.00th=[ 4490], 20.00th=[ 7767],
| 30.00th=[ 8029], 40.00th=[ 8225], 50.00th=[ 8586], 60.00th=[ 8848],
| 70.00th=[ 9372], 80.00th=[ 9896], 90.00th=[10552], 95.00th=[10814],
| 99.00th=[11207], 99.50th=[11600], 99.90th=[11994], 99.95th=[11994],
| 99.99th=[12256]
bw (KB/s): min= 0, max= 506, per=96.51%, avg=426.56, stdev=97.53
lat (msec): 2000=1.88%, >=2000=98.12%
cpu: usr=0.02%, sys=0.07%, ctx=232, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.5%, 32=0.9%, >=64=98.2%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
issued: total=r=3456/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=13824KB, aggrb=442KB/s, minb=442KB/s, maxb=442KB/s, mint=31272msec, maxt=31272msec

Disk stats (read/write):
sda: ios=3444/0, merge=0/0, ticks=4365220/0, in_queue=4391680, util=99.72%

Чтение: iodepth=32, bw=370844 B/s, iops=90, avg=353.27 msec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [388K/0K /s] [97 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=3799
read: io=11004KB, bw=370844 B/s, iops=90, runt= 30385msec
slat (usec): min=4, max=140, avg=18.36, stdev= 3.36
clat (msec): min=15, max=1357, avg=353.27, stdev=264.64
lat (msec): min=15, max=1357, avg=353.29, stdev=264.64
clat percentiles (msec):
| 1.00th=[ 29], 5.00th=[ 47], 10.00th=[ 66], 20.00th=[ 106],
| 30.00th=[ 149], 40.00th=[ 208], 50.00th=[ 277], 60.00th=[ 375],
| 70.00th=[ 494], 80.00th=[ 611], 90.00th=[ 750], 95.00th=[ 840],
| 99.00th=[ 1012], 99.50th=[ 1106], 99.90th=[ 1319], 99.95th=[ 1352],
| 99.99th=[ 1352]
bw (KB/s): min= 264, max= 411, per=100.00%, avg=362.59, stdev=30.10
lat (msec): 20=0.07%, 50=5.85%, 100=12.72%, 250=27.59%, 500=24.50%
lat (msec): 750=19.12%, 1000=9.05%, 2000=1.09%
cpu: usr=0.09%, sys=0.41%, ctx=2751, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.3%, 16=0.6%, 32=98.9%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=2751/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=11004KB, aggrb=362KB/s, minb=362KB/s, maxb=362KB/s, mint=30385msec, maxt=30385msec

Disk stats (read/write):
sda: ios=2732/0, merge=0/0, ticks=953790/0, in_queue=963950, util=99.71%

Чтение: iodepth=16, bw=335270 B/s, iops=81, avg=195368.11 usec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=16
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [300K/0K /s] [75 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=3817
read: io=9880.0KB, bw=335270 B/s, iops=81, runt= 30176msec
slat (usec): min=4, max=50, avg=17.58, stdev= 4.03
clat (usec): min=217, max=755355, avg=195368.11, stdev=148034.81
lat (usec): min=269, max=755375, avg=195386.26, stdev=148035.38
clat percentiles (usec):
| 1.00th=[ 1896], 5.00th=[30080], 10.00th=[45824], 20.00th=[71168],
| 30.00th=[95744], 40.00th=[120320], 50.00th=[150528], 60.00th=[187392],
| 70.00th=[246784], 80.00th=[313344], 90.00th=[423936], 95.00th=[501760],
| 99.00th=[602112], 99.50th=[643072], 99.90th=[700416], 99.95th=[724992],
| 99.99th=[757760]
bw (KB/s): min= 278, max= 1078, per=100.00%, avg=327.22, stdev=102.27
lat (usec): 250=0.04%, 500=0.04%, 750=0.16%, 1000=0.16%
lat (msec): 2=0.69%, 4=1.70%, 10=0.97%, 20=0.16%, 50=7.69%
lat (msec): 100=20.12%, 250=38.70%, 500=24.37%, 750=5.14%, 1000=0.04%
cpu: usr=0.18%, sys=0.25%, ctx=2470, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.2%, 8=0.3%, 16=99.4%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=2470/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=9880KB, aggrb=327KB/s, minb=327KB/s, maxb=327KB/s, mint=30176msec, maxt=30176msec

Disk stats (read/write):
sda: ios=2468/0, merge=0/0, ticks=480783/0, in_queue=481256, util=99.71%

Чтение: iodepth=8, bw=304153 B/s, iops=74, avg=107.66 msec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=8
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [304K/0K /s] [76 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=3821
read: io=8944.0KB, bw=304153 B/s, iops=74, runt= 30112msec
slat (usec): min=4, max=139, avg=18.18, stdev= 3.87
clat (msec): min=13, max=307, avg=107.66, stdev=48.10
lat (msec): min=13, max=307, avg=107.68, stdev=48.10
clat percentiles (msec):
| 1.00th=[ 29], 5.00th=[ 38], 10.00th=[ 47], 20.00th=[ 61],
| 30.00th=[ 75], 40.00th=[ 93], 50.00th=[ 108], 60.00th=[ 120],
| 70.00th=[ 135], 80.00th=[ 151], 90.00th=[ 169], 95.00th=[ 186],
| 99.00th=[ 243], 99.50th=[ 251], 99.90th=[ 289], 99.95th=[ 297],
| 99.99th=[ 310]
bw (KB/s): min= 264, max= 327, per=100.00%, avg=297.14, stdev=15.40
lat (msec): 20=0.04%, 50=11.99%, 100=33.23%, 250=54.16%, 500=0.58%
cpu: usr=0.10%, sys=0.31%, ctx=2236, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.2%, 8=99.7%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=2236/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=8944KB, aggrb=297KB/s, minb=297KB/s, maxb=297KB/s, mint=30112msec, maxt=30112msec

Disk stats (read/write):
sda: ios=2224/0, merge=0/0, ticks=238710/0, in_queue=239166, util=99.71%

Чтение: iodepth=4, bw=233640 B/s, iops=57, avg=70.06 msec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=4
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [236K/0K /s] [59 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=3825
read: io=6860.0KB, bw=233640 B/s, iops=57, runt= 30066msec
slat (usec): min=5, max=49, avg=18.10, stdev= 2.69
clat (msec): min=18, max=151, avg=70.06, stdev=19.49
lat (msec): min=18, max=151, avg=70.08, stdev=19.49
clat percentiles (msec):
| 1.00th=[ 35], 5.00th=[ 42], 10.00th=[ 47], 20.00th=[ 53],
| 30.00th=[ 59], 40.00th=[ 64], 50.00th=[ 70], 60.00th=[ 75],
| 70.00th=[ 80], 80.00th=[ 87], 90.00th=[ 95], 95.00th=[ 104],
| 99.00th=[ 124], 99.50th=[ 130], 99.90th=[ 151], 99.95th=[ 151],
| 99.99th=[ 151]
bw (KB/s): min= 201, max= 251, per=100.00%, avg=228.00, stdev=13.05
lat (msec): 20=0.06%, 50=15.57%, 100=77.73%, 250=6.65%
cpu: usr=0.10%, sys=0.21%, ctx=1715, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=99.8%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=1715/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=6860KB, aggrb=228KB/s, minb=228KB/s, maxb=228KB/s, mint=30066msec, maxt=30066msec

Disk stats (read/write):
sda: ios=1706/0, merge=0/0, ticks=119393/0, in_queue=119530, util=99.70%

Чтение: iodepth=2, bw=224993 B/s, iops=54, avg=36.37 msec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=2
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [212K/0K /s] [53 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=3829
read: io=6596.0KB, bw=224993 B/s, iops=54, runt= 30020msec
slat (usec): min=9, max=48, avg=22.16, stdev= 2.08
clat (msec): min=15, max=149, avg=36.37, stdev= 9.31
lat (msec): min=15, max=149, avg=36.40, stdev= 9.31
clat percentiles (msec):
| 1.00th=[ 19], 5.00th=[ 23], 10.00th=[ 26], 20.00th=[ 29],
| 30.00th=[ 32], 40.00th=[ 35], 50.00th=[ 37], 60.00th=[ 38],
| 70.00th=[ 41], 80.00th=[ 44], 90.00th=[ 48], 95.00th=[ 51],
| 99.00th=[ 58], 99.50th=[ 60], 99.90th=[ 145], 99.95th=[ 151],
| 99.99th=[ 151]
bw (KB/s): min= 167, max= 262, per=100.00%, avg=219.48, stdev=17.43
lat (msec): 20=1.76%, 50=92.12%, 100=6.00%, 250=0.12%
cpu: usr=0.08%, sys=0.23%, ctx=1650, majf=0, minf=0
IO depths: 1=0.1%, 2=99.9%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=1649/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=6596KB, aggrb=219KB/s, minb=219KB/s, maxb=219KB/s, mint=30020msec, maxt=30020msec

Disk stats (read/write):
sda: ios=1641/0, merge=0/0, ticks=59690/0, in_queue=59730, util=99.70%

Чтение: iodepth=1, bw=227153 B/s, iops=55, avg=18.00 msec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [224K/0K /s] [56 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=3833
read: io=6656.0KB, bw=227153 B/s, iops=55, runt= 30005msec
slat (usec): min=12, max=145, avg=27.24, stdev= 3.99
clat (msec): min=3, max=35, avg=18.00, stdev= 5.54
lat (msec): min=3, max=35, avg=18.02, stdev= 5.54
clat percentiles (usec):
| 1.00th=[ 6752], 5.00th=[ 8896], 10.00th=[10816], 20.00th=[12992],
| 30.00th=[14656], 40.00th=[16192], 50.00th=[18048], 60.00th=[19584],
| 70.00th=[21120], 80.00th=[22912], 90.00th=[25472], 95.00th=[27264],
| 99.00th=[30592], 99.50th=[31104], 99.90th=[32384], 99.95th=[35584],
| 99.99th=[35584]
bw (KB/s): min= 182, max= 254, per=100.00%, avg=221.51, stdev=16.49
lat (msec): 4=0.06%, 10=7.57%, 20=55.05%, 50=37.32%
cpu: usr=0.10%, sys=0.16%, ctx=1664, majf=0, minf=0
IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=1664/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=6656KB, aggrb=221KB/s, minb=221KB/s, maxb=221KB/s, mint=30005msec, maxt=30005msec

Disk stats (read/write):
sda: ios=1657/0, merge=0/0, ticks=29793/0, in_queue=29800, util=99.34%


Анализ чтения:

Не удалось добиться нужного уровня avg.latency. Наиболее близкие результаты: iodepth=1, bw=227153 B/s, iops=55, avg=18.00 msec.

Запись: iodepth=1024, bw=454850 B/s, iops=111, avg=8344.81 msec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1024
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [0.0% done] [0K/132K /s] [0 /33 iops] [eta 11d:09h:04m:56s]
writetest: (groupid=0, jobs=1): err= 0: pid=3838
write: io=13948KB, bw=454850 B/s, iops=111, runt= 31401msec
slat (usec): min=3, max=878364, avg=8661.37, stdev=51377.60
clat (msec): min=1188, max=12599, avg=8344.81, stdev=2246.20
lat (msec): min=1426, max=12599, avg=8353.47, stdev=2245.59
clat percentiles (msec):
| 1.00th=[ 1827], 5.00th=[ 2933], 10.00th=[ 4555], 20.00th=[ 7570],
| 30.00th=[ 8160], 40.00th=[ 8455], 50.00th=[ 8717], 60.00th=[ 9110],
| 70.00th=[ 9503], 80.00th=[10028], 90.00th=[10683], 95.00th=[11076],
| 99.00th=[11600], 99.50th=[11731], 99.90th=[12256], 99.95th=[12387],
| 99.99th=[12649]
bw (KB/s): min= 0, max= 815, per=94.25%, avg=418.49, stdev=125.16
lat (msec): 2000=1.86%, >=2000=98.14%
cpu: usr=0.01%, sys=0.08%, ctx=235, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.5%, 32=0.9%, >=64=98.2%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
issued: total=r=0/w=0/d=3487, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=13948KB, aggrb=444KB/s, minb=444KB/s, maxb=444KB/s, mint=31401msec, maxt=31401msec

Disk stats (read/write):
sda: ios=85/3455, merge=0/0, ticks=3414/4329547, in_queue=4392543, util=100.00%

Запись: iodepth=32, bw=438475 B/s, iops=107, avg=298800.84 usec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/428K /s] [0 /107 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=3861
write: io=12968KB, bw=438475 B/s, iops=107, runt= 30285msec
slat (usec): min=3, max=138, avg=19.15, stdev= 3.63
clat (usec): min=419, max=1160.5K, avg=298800.84, stdev=230187.03
lat (usec): min=471, max=1160.6K, avg=298820.55, stdev=230187.44
clat percentiles (msec):
| 1.00th=[ 11], 5.00th=[ 32], 10.00th=[ 52], 20.00th=[ 86],
| 30.00th=[ 126], 40.00th=[ 176], 50.00th=[ 231], 60.00th=[ 310],
| 70.00th=[ 408], 80.00th=[ 519], 90.00th=[ 652], 95.00th=[ 734],
| 99.00th=[ 873], 99.50th=[ 930], 99.90th=[ 1057], 99.95th=[ 1123],
| 99.99th=[ 1156]
bw (KB/s): min= 337, max= 881, per=100.00%, avg=428.34, stdev=73.09
lat (usec): 500=0.03%, 1000=0.03%
lat (msec): 2=0.09%, 4=0.15%, 10=0.68%, 20=1.30%, 50=7.25%
lat (msec): 100=14.53%, 250=28.16%, 500=25.72%, 750=17.46%, 1000=4.32%
lat (msec): 2000=0.28%
cpu: usr=0.22%, sys=0.38%, ctx=3242, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.5%, 32=99.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=3242, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=12968KB, aggrb=428KB/s, minb=428KB/s, maxb=428KB/s, mint=30285msec, maxt=30285msec

Disk stats (read/write):
sda: ios=84/3228, merge=0/0, ticks=24/959137, in_queue=963340, util=99.78%

Запись: iodepth=16, bw=442678 B/s, iops=108, avg=147974.19 usec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=16
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/428K /s] [0 /107 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=3883
write: io=13024KB, bw=442678 B/s, iops=108, runt= 30127msec
slat (usec): min=3, max=47, avg=19.10, stdev= 2.97
clat (usec): min=434, max=622395, avg=147974.19, stdev=107052.31
lat (usec): min=484, max=622416, avg=147993.84, stdev=107052.73
clat percentiles (msec):
| 1.00th=[ 8], 5.00th=[ 26], 10.00th=[ 37], 20.00th=[ 58],
| 30.00th=[ 77], 40.00th=[ 98], 50.00th=[ 121], 60.00th=[ 145],
| 70.00th=[ 182], 80.00th=[ 231], 90.00th=[ 306], 95.00th=[ 367],
| 99.00th=[ 474], 99.50th=[ 498], 99.90th=[ 586], 99.95th=[ 619],
| 99.99th=[ 619]
bw (KB/s): min= 333, max= 948, per=100.00%, avg=432.32, stdev=76.93
lat (usec): 500=0.03%, 1000=0.03%
lat (msec): 2=0.18%, 4=0.31%, 10=1.04%, 20=1.72%, 50=13.14%
lat (msec): 100=24.29%, 250=42.29%, 500=16.46%, 750=0.49%
cpu: usr=0.15%, sys=0.45%, ctx=3258, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=99.5%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=3256, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=13024KB, aggrb=432KB/s, minb=432KB/s, maxb=432KB/s, mint=30127msec, maxt=30127msec

Disk stats (read/write):
sda: ios=84/3231, merge=0/0, ticks=17/476620, in_queue=478430, util=99.77%

Запись: iodepth=8, bw=424831 B/s, iops=103, avg=77074.76 usec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=8
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/416K /s] [0 /104 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=3906
write: io=12484KB, bw=424831 B/s, iops=103, runt= 30091msec
slat (usec): min=3, max=46, avg=19.05, stdev= 2.84
clat (usec): min=388, max=290503, avg=77074.76, stdev=39278.20
lat (usec): min=436, max=290523, avg=77094.38, stdev=39278.62
clat percentiles (msec):
| 1.00th=[ 5], 5.00th=[ 23], 10.00th=[ 30], 20.00th=[ 41],
| 30.00th=[ 51], 40.00th=[ 63], 50.00th=[ 74], 60.00th=[ 86],
| 70.00th=[ 97], 80.00th=[ 112], 90.00th=[ 131], 95.00th=[ 145],
| 99.00th=[ 182], 99.50th=[ 202], 99.90th=[ 225], 99.95th=[ 245],
| 99.99th=[ 289]
bw (KB/s): min= 332, max= 952, per=100.00%, avg=415.36, stdev=80.17
lat (usec): 500=0.03%, 750=0.03%
lat (msec): 2=0.45%, 4=0.42%, 10=0.87%, 20=2.47%, 50=24.64%
lat (msec): 100=43.10%, 250=27.97%, 500=0.03%
cpu: usr=0.14%, sys=0.44%, ctx=3121, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=0.1%, 8=99.8%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=3121, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=12484KB, aggrb=414KB/s, minb=414KB/s, maxb=414KB/s, mint=30091msec, maxt=30091msec

Disk stats (read/write):
sda: ios=84/3106, merge=0/0, ticks=20/238780, in_queue=239160, util=99.78%

Запись: iodepth=4, bw=438823 B/s, iops=107, avg=37297.06 usec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=4
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/436K /s] [0 /109 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=3928
write: io=12872KB, bw=438823 B/s, iops=107, runt= 30037msec
slat (usec): min=4, max=49, avg=19.17, stdev= 2.65
clat (usec): min=369, max=102682, avg=37297.06, stdev=15596.69
lat (usec): min=418, max=102702, avg=37316.80, stdev=15597.26
clat percentiles (msec):
| 1.00th=[ 3], 5.00th=[ 16], 10.00th=[ 20], 20.00th=[ 25],
| 30.00th=[ 28], 40.00th=[ 32], 50.00th=[ 36], 60.00th=[ 41],
| 70.00th=[ 45], 80.00th=[ 50], 90.00th=[ 58], 95.00th=[ 66],
| 99.00th=[ 80], 99.50th=[ 88], 99.90th=[ 97], 99.95th=[ 99],
| 99.99th=[ 103]
bw (KB/s): min= 309, max= 955, per=100.00%, avg=428.51, stdev=82.23
lat (usec): 500=0.03%, 750=0.03%
lat (msec): 2=0.75%, 4=0.71%, 10=0.84%, 20=8.70%, 50=68.92%
lat (msec): 100=19.98%, 250=0.03%
cpu: usr=0.19%, sys=0.41%, ctx=3218, majf=0, minf=0
IO depths: 1=0.1%, 2=0.1%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=3218, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=12872KB, aggrb=428KB/s, minb=428KB/s, maxb=428KB/s, mint=30037msec, maxt=30037msec

Disk stats (read/write):
sda: ios=84/3206, merge=0/0, ticks=20/119400, in_queue=119513, util=99.78%

Запись: iodepth=2, bw=436981 B/s, iops=106, avg=18715.34 usec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=2
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/416K /s] [0 /104 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=3950
write: io=12812KB, bw=436981 B/s, iops=106, runt= 30023msec
slat (usec): min=5, max=155, avg=22.97, stdev= 3.82
clat (usec): min=393, max=73421, avg=18715.34, stdev=9080.87
lat (usec): min=441, max=73443, avg=18738.87, stdev=9081.44
clat percentiles (usec):
| 1.00th=[ 1048], 5.00th=[ 5280], 10.00th=[ 7968], 20.00th=[11200],
| 30.00th=[12992], 40.00th=[15424], 50.00th=[18048], 60.00th=[20096],
| 70.00th=[22656], 80.00th=[26496], 90.00th=[30848], 95.00th=[35072],
| 99.00th=[42752], 99.50th=[44288], 99.90th=[51456], 99.95th=[73216],
| 99.99th=[73216]
bw (KB/s): min= 317, max= 922, per=100.00%, avg=426.53, stdev=73.58
lat (usec): 500=0.03%, 750=0.06%, 1000=0.72%
lat (msec): 2=1.03%, 4=0.59%, 10=12.55%, 20=44.21%, 50=40.65%
lat (msec): 100=0.16%
cpu: usr=0.20%, sys=0.40%, ctx=3203, majf=0, minf=0
IO depths: 1=0.1%, 2=100.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=3203, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=12812KB, aggrb=426KB/s, minb=426KB/s, maxb=426KB/s, mint=30023msec, maxt=30023msec

Disk stats (read/write):
sda: ios=84/3189, merge=0/0, ticks=17/59607, in_queue=59646, util=99.76%

Запись: iodepth=1, bw=430597 B/s, iops=105, avg=9477.07 usec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1
2.0.8
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/416K /s] [0 /104 iops] [eta 00m:00s]
writetest: (groupid=0, jobs=1): err= 0: pid=3972
write: io=12616KB, bw=430597 B/s, iops=105, runt= 30002msec
slat (usec): min=6, max=48, avg=27.49, stdev= 3.66
clat (usec): min=342, max=35159, avg=9477.07, stdev=7171.91
lat (usec): min=358, max=35173, avg=9505.14, stdev=7172.29
clat percentiles (usec):
| 1.00th=[ 470], 5.00th=[ 564], 10.00th=[ 660], 20.00th=[ 820],
| 30.00th=[ 4576], 40.00th=[ 5920], 50.00th=[ 7776], 60.00th=[11968],
| 70.00th=[13760], 80.00th=[15424], 90.00th=[18816], 95.00th=[22912],
| 99.00th=[27008], 99.50th=[30080], 99.90th=[35072], 99.95th=[35072],
| 99.99th=[35072]
bw (KB/s): min= 319, max= 915, per=100.00%, avg=420.32, stdev=76.45
lat (usec): 500=2.09%, 750=16.52%, 1000=1.71%
lat (msec): 2=0.19%, 4=6.21%, 10=27.01%, 20=37.86%, 50=8.40%
cpu: usr=0.13%, sys=0.36%, ctx=3155, majf=0, minf=0
IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=3154, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=12616KB, aggrb=420KB/s, minb=420KB/s, maxb=420KB/s, mint=30002msec, maxt=30002msec

Disk stats (read/write):
sda: ios=84/3141, merge=0/0, ticks=20/29754, in_queue=29787, util=99.29%


Анализ записи:

Как уже стало закономерностью — с записью на диск дела обстоят лучше. Удалось добиться 105 IOPS при 9,5 мс latency. iodepth=1, bw=430597 B/s, iops=105, avg=9477.07 usec.

CD-ROM

Подробно тестировать нет необходимости, так как цифры нужны больше для ориентации. Накопитель не позволил выполнить условие avg.latency < 10 мс. Наиболее близкий результат чтения: iodepth=1, bw=23655 B/s, iops=5.
Чтение: iodepth=1, bw=23655 B/s, iops=5, avg=172.92 msec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=1
2.0.8
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [32K/0K /s] [8 /0 iops] [eta 00m:00s]
readtest: (groupid=0, jobs=1): err= 0: pid=4518
read: io=712704 B, bw=23655 B/s, iops=5, runt= 30128msec
slat (usec): min=13, max=25, avg=14.97, stdev= 2.39
clat (msec): min=48, max=3651, avg=172.92, stdev=269.66
lat (msec): min=48, max=3651, avg=172.94, stdev=269.66
clat percentiles (msec):
| 1.00th=[ 52], 5.00th=[ 82], 10.00th=[ 92], 20.00th=[ 109],
| 30.00th=[ 121], 40.00th=[ 137], 50.00th=[ 151], 60.00th=[ 163],
| 70.00th=[ 180], 80.00th=[ 202], 90.00th=[ 223], 95.00th=[ 243],
| 99.00th=[ 269], 99.50th=[ 3654], 99.90th=[ 3654], 99.95th=[ 3654],
| 99.99th=[ 3654]
bw (KB/s): min= 1, max= 40, per=100.00%, avg=25.22, stdev= 5.62
lat (msec): 50=0.57%, 100=13.79%, 250=82.18%, 500=2.87%, >=2000=0.57%
cpu: usr=0.00%, sys=0.03%, ctx=182, majf=0, minf=0
IO depths: 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=174/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=696KB, aggrb=23KB/s, minb=23KB/s, maxb=23KB/s, mint=30128msec, maxt=30128msec

Disk stats (read/write):
sr0: ios=172/0, merge=0/0, ticks=29810/0, in_queue=29834, util=99.57%


RAM-диск /dev/ram в Linux

Чтение: iodepth=2, bw=852105KB/s, iops=213026, avg= 5.51 usec
root@host:~# fio ./read.ini
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=2
2.0.8
Starting 1 process

readtest: (groupid=0, jobs=1): err= 0: pid=4262
read: io=97140KB, bw=852105KB/s, iops=213026, runt= 114msec
slat (usec): min=2, max=118, avg= 2.47, stdev= 0.92
clat (usec): min=3, max=121, avg= 5.51, stdev= 0.94
lat (usec): min=6, max=124, avg= 8.34, stdev= 1.21
clat percentiles (usec):
| 1.00th=[ 5], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 5],
| 30.00th=[ 5], 40.00th=[ 5], 50.00th=[ 6], 60.00th=[ 6],
| 70.00th=[ 6], 80.00th=[ 6], 90.00th=[ 6], 95.00th=[ 6],
| 99.00th=[ 6], 99.50th=[ 6], 99.90th=[ 9], 99.95th=[ 13],
| 99.99th=[ 15]
lat (usec): 4=0.01%, 10=99.91%, 20=0.07%, 50=0.01%, 250=0.01%
cpu: usr=34.51%, sys=64.60%, ctx=1, majf=0, minf=0
IO depths: 1=0.1%, 2=100.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=24285/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=97140KB, aggrb=852105KB/s, minb=852105KB/s, maxb=852105KB/s, mint=114msec, maxt=114msec

Disk stats (read/write):
ram1: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%


Анализ чтения:

Вроде всё верно, измерение чтения RAM-диска соответствует измеренному, но есть один нюанс:
  cpu          : usr=34.51%, sys=64.60%, ctx=1, majf=0, minf=0

иначе говоря, мы упёрлись в производительность процессора. И не факт, что самого RAM-диска. Результат по чтению на данный момент: iodepth=2, bw=852105KB/s, iops=213026, avg= 5.51 usec.

Запись: iodepth=2, bw=693857KB/s, iops=173464, avg= 6.44 usec
root@host:~# fio ./write.ini
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=2
2.0.8
Starting 1 process

writetest: (groupid=0, jobs=1): err= 0: pid=4293
write: io=97140KB, bw=693857KB/s, iops=173464, runt= 140msec
slat (usec): min=3, max=120, avg= 3.35, stdev= 0.96
clat (usec): min=4, max=124, avg= 6.44, stdev= 1.25
lat (usec): min=9, max=128, avg=10.12, stdev= 1.50
clat percentiles (usec):
| 1.00th=[ 6], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 6],
| 30.00th=[ 6], 40.00th=[ 6], 50.00th=[ 6], 60.00th=[ 7],
| 70.00th=[ 7], 80.00th=[ 7], 90.00th=[ 7], 95.00th=[ 7],
| 99.00th=[ 8], 99.50th=[ 8], 99.90th=[ 10], 99.95th=[ 14],
| 99.99th=[ 52]
lat (usec): 10=99.80%, 20=0.19%, 100=0.01%, 250=0.01%
cpu: usr=20.86%, sys=76.26%, ctx=1, majf=0, minf=0
IO depths: 1=0.1%, 2=100.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete: 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued: total=r=0/w=0/d=24285, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=97140KB, aggrb=693857KB/s, minb=693857KB/s, maxb=693857KB/s, mint=140msec, maxt=140msec

Disk stats (read/write):
ram1: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%


Анализ записи:

То же самое с записью на RAM-диск. Мы упёрлись в процессор:
  cpu          : usr=20.86%, sys=76.26%, ctx=1, majf=0, minf=0

Результат по записи на данный момент: iodepth=2, bw=693857KB/s, iops=173464, avg= 6.44 usec

Сводная таблица результатов тестирования:
image

Появившиеся вопросы:


  1. Как же можно представить размер очереди в реальной жизни и что она говорит о носителе? К примеру на SD-картах размер очереди 1, а на SSD-дисках — 64.
  2. Достаточно ли 30 секунд для теста IOPS или надо тестировать весь диск целиком?
  3. Тест производительности RAM-диска упёрся в CPU. Является ли это настоящей производительностью диска или всё же ограничено процессором?


ВЫВОДЫ:


  1. Методика похоже верная и точная. Расхождений в тестировании не было.
  2. UPDATE: Предварительно 30 секунд достаточно для RAM-дисков, чтобы получить более ли менее точную оценку IOPS. Для SSD-дисков время замера должно быть однозначно больше.
  3. Измерение производительности на запись и чтение всё же лучше производить отдельно друг от друга.
  4. SSD-диски вне конкуренции. Как по IOPS и скорости линейного чтения, так и по latency.
  5. Разница в производительности между enterprise и consumer дисками есть в значениях IOPS и Latency. Велика ли разница — решать каждому под конкретные задачи. А вообще недостаточно данных для более точного анализа.
  6. Скорость линейного чтения enterprise дисков не выше consumer дисков.
  7. Жёсткие диски выдают очень низкую скорость при случайном чтении. Не более полумегабайта в секунду.
  8. Производительность IOPS на чтение у измеренных жёстких дисков была ниже, чем на запись. Вероятно это сделано с целью «сравнять» производительность чтения и записи при учёте пенальти RAID, при этом низкий показатель IOPS чтения компенсируется встроенным в ОС кешем.
  9. Линейная скорость протестированных дисков выше произвольной на величину до 500 и более раз.

P.S. Хотелось бы ещё результатов тестирования enterprise-level дисков, но более не оказалось под рукой.

Хорошего дня!
Share post

Comments 21

    +16
    Графики где? Где сводные таблицы? Где аналитика?
    Портянка сырых данных — это не аналитика, это я вообще не знаю что.
      –1
      Аналитика в выводах в соответствии с поставленной целью. Таблица да, нужна, но её вставить сюда нетривиально.
        +1
        один из вариантов:
        1. график и таблицы делаем в Excel
        2. снимаем и обрезаем (PrintScreen + Paint)
        3. вставляем картинку в пост
          0
          Спасибо за совет. Сделал.
        0
        Добавил сводную таблицу. Можете посмотреть более наглядно.
        0
        А где данные из (Чтение: iodepth=1024, bw=4528.8KB/s, iops=1132, avg= 894.10 msec)
          0
          Вставил. Видать где-то затерялось по ходу написания статьи.
          0
          У меня тут сейчас как раз по этому методу тестируется ioDrive2 1,2Tb и полка HP P2000 12х146@15k RAID10 :)
            0
            Было бы интересно узнать результаты. Мог бы их вставить в текст. Думаю нет необходимости тестировать сразу все 1,2 Тб для такого накопителя. Результат будет схожим, как и за 30 секунд.
              0
              Я буквально на днях консультировался с amarao, по его совету — SSD тестирую около получаса, т.к. в пике у него значение на много выше (хотя так сходу могу уже сказать что значения там очень даже вкусные :)). Данные просто так отдать мне, к сожалению, не даст руководство, так что буду писать отдельную статью и по полке и по иодрайву.
            0
            Чтобы понять, упираетесь ли вы в процессор или в память в тесте с ram-диском, попробуйте увеличить размер блока в несколько раз.
              0
              Простите, но статья фигня полная. Хотите посмотреть как должны быть проводиться серьезные измерения и быть оформлены результаты? Примерно так: это энтерпрайз SSD www.storagereview.com/micron_realssd_p320h_enterprise_pcie_review, это бытовой (они там для сравнения прогоняют его дополнительно энтерпрайз тестами) — www.storagereview.com/corsair_neutron_gtx_ssd_review
                0
                Вы считаете, что надо было также расписывать, как по указанной ссылке, если до начала тестирования не было уверенности, что методика вообще покажет верный порядок цифр а целью статьи было как раз-таки убедиться в её верности?
                Да и не претендовала статья на серьёзные измерения — иначе для чего надо было бы менять SD-флешки или CD-ROM? А нужно это было чтобы понять, стоит ли сравнивать разные накопители этим методом и каких цифр вообще ожидать. Все «неожидаемые» цифры как раз приведены в выводах.
                  0
                  То есть вы придумали абстрактную методику, затем провели тесты и убедились, что для абстрактной методики цифры при нескольких измерениях совпадают и сделали из этого вывод, что методика верна?
                  При этом вы даже не понимаете понятия, которыми оперируете во время тестов — что такое размер очереди и на что он влияет, почему он разный, почему цифра latency взята именно такая и почему она разная для разных устройств, как устроены накопители, которые вы тестируете и почему у них чтение/запись выдает разные цифры.

                  Графики из статей, ссылки на которые я привёл наглядно отвечают на некоторые ваши вопросы — нет, 30 секунд недостаточно для тестировани, как вы можете увидеть, некоторые накопители резко снижают производительность и так же резко повышают задержку только минут через 20 после начала теста, с чем это связано, как вы думаете?

                  А если ещё глобальнее — что именно вы измерили, кроме того, что вы измерили производительность ваших устройств при искусственных условиях? Будет ли производительность хотя бы так же соотноситься при реальной работе? Например, в качестве рабочей станции или сервера или при использовании в качестве носителя для фотоаппарата?
                    0
                    Методика не придумана, а взята из другого поста и об этом прямо указано. О верности методики говорят логичное изменение цифр при тестировании (к примеру размер очереди всегда негативно сказывается на задержках и увеличении IOPS), а также соизмеримость с общеизвестными данными о производительности.

                    Я понимаю понятия, которые используются в тесте. Единственное, я спросил как представить в реальной жизни размер очереди. Ну для примера где она находится — в ОС или на самом накопителе, какова её длина для разных типов накопителей и на что это влияет «в реальной работе». Цифра latency взята из поста на который приведена ссылка в начале статьи. В то же время появляется вопрос — 10 мс оказалось недостаточно даже для дисков 7200 об/мин enterprise на чтение — насколько верно эта цифра выбрана?

                    Вы привели ссылки с целью показать, как должны проводиться серьёзные измерения, но при этом не учитываете тот факт, что в данном тесте протестировано 9 накопителей и если разрисовывать их графиками как в ссылках, то это будет портянка на два дня чтива. Хорошо, 30 секунд недостаточно. Для SSD, к примеру. А для RAM-диска чем мало время? А для SD-карт?

                    Да, я измерил производительность накопителей при искусственных условиях. Этого требовала поставленная цель. Не было целью описать, как накопители поведут себя в реальной работе где бы то ни было. Если Вы прольёте свет на все эти вопросы — это будет только плюсом.
                    Хорошего дня!
                      0
                      Для RAM-диска достаточно несколько секунд теста, т.к. у него константные показатели. У магнитных дисков так же (если исключить разницу между short hand областью и остальным). У SSD, SD, usb-flash, аппаратных рейдов с wb и т.д. нужно длительное тестирование для того, чтобы пробить все уровни локальной оптимизации всплесков.

                      Откуда данные? Эксперименты. У забытого на неделю на тесте HDD показатели не поменялись. У SSD показатели явно падают в течение нескольких часов с момента начала теста.
                        0
                        Спасибо за совет. Дополнительно провёл тест этого же SSD в течении 10 часов вместо 30 секунд. В результате показатели IOPS и bw снизились более, чем в 2 раза (см.update).
                        0
                        Методика в посте, откуда вы ее взяли, преследовала вполне конкретные цели — определить худшую производительность устройств.
                        Про длину очереди — объясню на пальцах: это количество команд чтения/записи, которое мы отправляем накопителю (строго говоря операционной системе, а она уже дальше накопителю, а может через еще сто слоев типа рейд-контроллеров, хба-адаптера и тд) ДО того, как получили ответ на первую команду. То есть, даем сразу Х команд, ждем результатов, пришли данные — добавляем еще и тд, так, чтобы у нас всегда было Х команд без ответа. Почему это важно? Потому что разные задачи имеют различный паттерн доступа — флешка в фотоаппарате никогда не столкнется с ситуацией параллельной записи мелкими блоками в сто мест, домашний винчестер — очень редко, серверный — круглые сутки молотит именно в таком режиме. И мозгов внутри накопителей достаточно, чтобы определить характер доступа — разложить (псевдо)случайные записи/чтения более-менее в линию, прочитать на несколько блоков больше, если есть вероятность, что они понадобятся и тд. Но! Тут есть одно огромное но: разные задачи и классы задач требуют зачастую противоположной оптимизации! То есть научив накопитель работать с абсолютно случайной нагрузкой на запись по 4кб мы на порядок ухудшим например типичные задачи типа запуска игрушки или копирования пары сотен файлов, и наоборот.
                        К вопросу, как это влияет при «реальной работе» — флешка отлично пишет и читает линейно, так как именно на это и заточена, больше одной операции параллельно выполнить не может, так как у нее нет мозгов для этого — ни буферной памяти, ни процессора. Ссд из таких же флешек прекрасно справляется со случайной и параллельной нагрузкой, так как в нем есть эдак полгига оперативной памяти и возможность писать параллельно в десяток(-тки) микросхем флеша. Жесткий диск — ограничен механикой, поделите секунду на скорость доступа (фактически ограниченную скоростью вращения) и вы получите предельные операции ввода-вывода, однако, если не морочить винту голову и подать ту нагрузку, которую он ждет и на которую он рассчитан — эти цифры могут увеличиться в несколько раз. Кстати, диск, который у вас типа энтерпрайз, это просто обычный винт, с парой твиков, так что ждать от него чудес бессмысленно.
                        Думаю обсуждать сд-диски с временем доступа в 200-300мс нет смысла, а рам-диск вообще не является диском и накопителем, сплошная абстракция, думаю очевидно, что его ограничения связаны исключительно с накладными расходами самого линукса на эмуляцию блочного устройства и цифры показывают пальцем в небо.

                        Теперь к цифре 10мс — вот об этом я и говорю — вы ее взяли из другой статьи, совершенно не осмысливая и не понимая. Что это за 10мс, почему именно 10мс? Много это или мало? Для каких ситуаций какие цифры подходят? А ведь это важнейший параметр, но подходить к нему нужно с ясным пониманием. О каких 10мс можно вести речь, если у дисков 7200об время доступа среднее порядка 12-15мс? Скажите, для записи фотографии снятой на флешку разница между 10мс и 100мс будет иметь значение? А при просмотре фильма задержка 1000мс повредит? При копировании 150 тысяч файлов будет задержка 150мс?
                          0
                          Кстати, в графиках по моим ссылкам протестировано по 6-7 накопителей, не сильно меньше, чем 9.
                    +1
                    Результаты теста с 30с временем выполнения для SSD не применимы.

                    Причина — специфичное поведение алгоритма housekeeeping'а. Когда идёт всплеск нагрузки, он его обрабатывает быстро, а потом «доделывает» в бэкграунде.

                    Самым ярким примером являются злосчастные 320ые SSD'шки от интела. Если их поставить в fio, то в начале на случайную запись там будет порядка 2000 IOPS, а спустя 2 часа теста — 400.

                    И это суровая реальность. Минимальный срок тестирования скорости для SSD — хотя бы час. (Особенно, если вы эту SSD хотите использовать непрерывно больше часа в продакте). При этом желательно наиболее выдающиеся показатели (1-2 строки) проверить хотя бы часа 3 на тему «не снижаются ли показатели во времени».
                      0
                      Да, ещё. Я не до конца уверен, но на lwn писали об исследованиях производительности флешек и карт памяти (не путать с ssd-дисками) — там проблема в том, что ftl (трансляции) в них оптимизируют под fat32 и даже используют чипы разной производительности для области FAT (которая часто меняется мелкими блоками) и остальной области (где пишут обычно большими блоками и последовательно).

                      Вот ссылка для начала чтения: lwn.net/Articles/528617/ (сама статья и всё по ссылкам)

                      Only users with full accounts can post comments. Log in, please.