Pull to refresh
EPAM
Компания для карьерного и профессионального роста

Тесты производительности EBS

EPAM corporate blog Amazon Web Services *
Привет! image

Очень много в интернете позитива/негатива об EBS. Так же не мало тестов на их производительность. Я решил сам провести несколько тестов и сделать небольшое исследование по этому поводу.

Итак, в тесте участвовала машинка m1.large к которой были примаунчены диски:
  • EBS Standard, 100 GB
  • EBS IO-1 500 IOPS, 100 GB
  • EBS IO-1 1000 IOPS, 100 GB
  • EBS IO-1 2000 IOPS, 200 GB
  • 8xEBS Standard, 30 GB, RAID 10
  • Ephemeral, 450 GB

Были проведены несколько тестов:

# hdparm -tT /dev/xcdX

# dd if=/dev/zero of=tempfile bs=5M count=1024 conv=fdatasync,notrunc
# echo 3 > /proc/sys/vm/drop_caches
# dd if=tempfile of=/dev/null bs=5M count=1024

# sysbench --num-threads=16 --test=fileio --file-total-size=5G --file-test-mode=rndrw prepare
# sysbench --num-threads=16 --test=fileio --file-total-size=5G --file-test-mode=rndrw run

Результаты вывода в консоль:

EBS Standard, 100 GB
# hdparm -tT /dev/xvdj

/dev/xvdj:
 Timing cached reads:   4866 MB in  2.00 seconds = 2428.53 MB/sec
 Timing buffered disk reads:  242 MB in  3.00 seconds =  80.54 MB/sec

 Timing cached reads:   5146 MB in  2.00 seconds = 2579.25 MB/sec
 Timing buffered disk reads:  294 MB in  3.01 seconds =  97.59 MB/sec

 Timing cached reads:   4870 MB in  2.00 seconds = 2440.55 MB/sec
 Timing buffered disk reads:  306 MB in  3.00 seconds = 101.89 MB/sec

# dd if=/dev/zero of=tempfile bs=5M count=1024 conv=fdatasync,notrunc
1024+0 records in
1024+0 records out
5368709120 bytes (5.4 GB) copied, 161.222 s, 33.3 MB/s

# echo 3 > /proc/sys/vm/drop_caches

# dd if=tempfile of=/dev/null bs=5M count=1024
1024+0 records in
1024+0 records out
5368709120 bytes (5.4 GB) copied, 86.4683 s, 62.1 MB/s


#  sysbench --num-threads=16 --test=fileio --file-total-size=5G --file-test-mode=rndrw run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 16

Extra file open flags: 0
128 files, 40Mb each
5Gb total file size
Block size 16Kb
Number of random requests for random IO: 10000
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Threads started!
Done.

Operations performed:  6000 Read, 4000 Write, 12800 Other = 22800 Total
Read 93.75Mb  Written 62.5Mb  Total transferred 156.25Mb  (69.816Mb/sec)
 4468.25 Requests/sec executed

Test execution summary:
    total time:                          2.2380s
    total number of events:              10000
    total time taken by event execution: 1.5942
    per-request statistics:
         min:                                  0.01ms
         avg:                                  0.16ms
         max:                                 65.00ms
         approx.  95 percentile:               0.02ms

Threads fairness:
    events (avg/stddev):           625.0000/249.48
    execution time (avg/stddev):   0.0996/0.03


EBS IO-1 500 IOPS, 100 GB
# hdparm -tT /dev/xvdh

/dev/xvdh:
 Timing cached reads:   4314 MB in  2.00 seconds = 2161.08 MB/sec
 Timing buffered disk reads:   72 MB in  3.05 seconds =  23.57 MB/sec

 Timing cached reads:   3646 MB in  2.00 seconds = 1826.09 MB/sec
 Timing buffered disk reads:   76 MB in  3.04 seconds =  25.01 MB/sec

 Timing cached reads:   4346 MB in  2.00 seconds = 2175.61 MB/sec
 Timing buffered disk reads:   76 MB in  3.03 seconds =  25.12 MB/sec

# dd if=/dev/zero of=tempfile bs=5M count=1024 conv=fdatasync,notrunc
1024+0 records in
1024+0 records out
5368709120 bytes (5.4 GB) copied, 271.993 s, 19.7 MB/s

# echo 3 > /proc/sys/vm/drop_caches

# dd if=tempfile of=/dev/null bs=5M count=1024
1024+0 records in
1024+0 records out
5368709120 bytes (5.4 GB) copied, 182.106 s, 29.5 MB/s


#  sysbench --num-threads=16 --test=fileio --file-total-size=5G --file-test-mode=rndrw run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 16

Extra file open flags: 0
128 files, 40Mb each
5Gb total file size
Block size 16Kb
Number of random requests for random IO: 10000
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Threads started!
Done.

Operations performed:  6000 Read, 4000 Write, 12800 Other = 22800 Total
Read 93.75Mb  Written 62.5Mb  Total transferred 156.25Mb  (16.794Mb/sec)
 1074.78 Requests/sec executed

Test execution summary:
    total time:                          9.3042s
    total number of events:              10000
    total time taken by event execution: 0.2975
    per-request statistics:
         min:                                  0.01ms
         avg:                                  0.03ms
         max:                                 30.70ms
         approx.  95 percentile:               0.02ms

Threads fairness:
    events (avg/stddev):           625.0000/553.34
    execution time (avg/stddev):   0.0186/0.02


EBS IO-1 1000 IOPS, 100 GB
# hdparm -tT /dev/xvdf

/dev/xvdf:
 Timing cached reads:   5090 MB in  2.00 seconds = 2550.81 MB/sec
 Timing buffered disk reads:  104 MB in  3.03 seconds =  34.30 MB/sec

 Timing cached reads:   5000 MB in  2.00 seconds = 2505.62 MB/sec
 Timing buffered disk reads:   98 MB in  3.10 seconds =  31.64 MB/sec

 Timing cached reads:   5046 MB in  2.01 seconds = 2507.34 MB/sec
 Timing buffered disk reads:   98 MB in  3.04 seconds =  32.19 MB/sec


# dd if=/dev/zero of=tempfile bs=5M count=1024 conv=fdatasync,notrunc
1024+0 records in
1024+0 records out
5368709120 bytes (5.4 GB) copied, 167.252 s, 32.1 MB/s

# echo 3 > /proc/sys/vm/drop_caches

# dd if=tempfile of=/dev/null bs=5M count=1024
1024+0 records in
1024+0 records out
5368709120 bytes (5.4 GB) copied, 126.793 s, 42.3 MB/s

#  sysbench --num-threads=16 --test=fileio --file-total-size=5G --file-test-mode=rndrw run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 16

Extra file open flags: 0
128 files, 40Mb each
5Gb total file size
Block size 16Kb
Number of random requests for random IO: 10000
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Threads started!
Done.

Operations performed:  6001 Read, 4000 Write, 12800 Other = 22801 Total
Read 93.766Mb  Written 62.5Mb  Total transferred 156.27Mb  (37.871Mb/sec)
 2423.73 Requests/sec executed

Test execution summary:
    total time:                          4.1263s
    total number of events:              10001
    total time taken by event execution: 0.8466
    per-request statistics:
         min:                                  0.01ms
         avg:                                  0.08ms
         max:                                 22.31ms
         approx.  95 percentile:               0.02ms

Threads fairness:
    events (avg/stddev):           625.0625/318.25
    execution time (avg/stddev):   0.0529/0.02


EBS IO-1 2000 IOPS, 200 GB
# hdparm -tT /dev/xvdi

/dev/xvdi:
 Timing cached reads:   4846 MB in  2.00 seconds = 2428.51 MB/sec
 Timing buffered disk reads:   90 MB in  3.02 seconds =  29.80 MB/sec

 Timing cached reads:   5122 MB in  2.05 seconds = 2503.64 MB/sec
 Timing buffered disk reads:  100 MB in  3.07 seconds =  32.56 MB/sec

 Timing cached reads:   4330 MB in  2.04 seconds = 2123.52 MB/sec
 Timing buffered disk reads:  102 MB in  3.05 seconds =  33.41 MB/sec



# dd if=/dev/zero of=tempfile bs=5M count=1024 conv=fdatasync,notrunc
1024+0 records in
1024+0 records out
5368709120 bytes (5.4 GB) copied, 161.549 s, 33.2 MB/s

# echo 3 > /proc/sys/vm/drop_caches

# dd if=tempfile of=/dev/null bs=5M count=1024
1024+0 records in
1024+0 records out
5368709120 bytes (5.4 GB) copied, 155.51 s, 34.5 MB/s


#  sysbench --num-threads=16 --test=fileio --file-total-size=5G --file-test-mode=rndrw prepare
sysbench 0.4.12:  multi-threaded system evaluation benchmark

128 files, 40960Kb each, 5120Mb total
Creating files for the test...
[root@ip-10-98-91-92 4]#  sysbench --num-threads=16 --test=fileio --file-total-size=5G --file-test-mode=rndrw run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 16

Extra file open flags: 0
128 files, 40Mb each
5Gb total file size
Block size 16Kb
Number of random requests for random IO: 10000
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Threads started!
Done.

Operations performed:  6000 Read, 4000 Write, 12801 Other = 22801 Total
Read 93.75Mb  Written 62.5Mb  Total transferred 156.25Mb  (74.645Mb/sec)
 4777.28 Requests/sec executed

Test execution summary:
    total time:                          2.0932s
    total number of events:              10000
    total time taken by event execution: 1.0015
    per-request statistics:
         min:                                  0.01ms
         avg:                                  0.10ms
         max:                                 10.10ms
         approx.  95 percentile:               0.02ms

Threads fairness:
    events (avg/stddev):           625.0000/177.29
    execution time (avg/stddev):   0.0626/0.02


8xEBS Standard, 30 GB, RAID 10
# hdparm -tT /dev/md0

/dev/md0:
 Timing cached reads:   3964 MB in  1.93 seconds = 2048.84 MB/sec
 Timing buffered disk reads:  230 MB in  3.53 seconds =  65.13 MB/sec

 Timing cached reads:   3994 MB in  1.99 seconds = 2002.16 MB/sec
 Timing buffered disk reads:  398 MB in  3.00 seconds = 132.64 MB/sec

 Timing cached reads:   4334 MB in  2.03 seconds = 2138.00 MB/sec
 Timing buffered disk reads:  302 MB in  3.02 seconds =  99.84 MB/sec


# dd if=/dev/zero of=tempfile bs=5M count=1024 conv=fdatasync,notrunc
1024+0 records in
1024+0 records out
5368709120 bytes (5.4 GB) copied, 113.234 s, 47.4 MB/s

# echo 3 > /proc/sys/vm/drop_caches

# dd if=tempfile of=/dev/null bs=5M count=1024
1024+0 records in
1024+0 records out
5368709120 bytes (5.4 GB) copied, 97.9346 s, 54.8 MB/s

# sysbench --num-threads=16 --test=fileio --file-total-size=5G --file-test-mode=rndrw run
sysbench 0.5:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 16
Random number generator seed is 0 and will be ignored


Extra file open flags: 0
128 files, 40Mb each
5Gb total file size
Block size 16Kb
Number of IO requests: 10000
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Threads started!

Operations performed:  5998 reads, 4002 writes, 12800 Other = 22800 Total
Read 93.719Mb  Written 62.531Mb  Total transferred 156.25Mb  (87.287Mb/sec)
 5586.40 Requests/sec executed

General statistics:
    total time:                          1.7901s
    total number of events:              10000
    total time taken by event execution: 1.1625s
    response time:
         min:                                  0.01ms
         avg:                                  0.12ms
         max:                                  8.99ms
         approx.  95 percentile:               0.03ms

Threads fairness:
    events (avg/stddev):           625.0000/171.83
    execution time (avg/stddev):   0.0727/0.01


Ephemeral, 450 GB
# hdparm -tT /dev/xvdb 

/dev/xvdb:
 Timing cached reads:   4048 MB in  2.00 seconds = 2027.97 MB/sec
 Timing buffered disk reads:  1794 MB in  3.12 seconds = 575.84 MB/sec

 Timing cached reads:   4854 MB in  2.00 seconds = 2432.18 MB/sec
 Timing buffered disk reads:  1830 MB in  3.00 seconds = 609.94 MB/sec

 Timing cached reads:   3434 MB in  2.00 seconds = 1719.73 MB/sec
 Timing buffered disk reads:  770 MB in  3.13 seconds = 245.97 MB/sec

# dd if=/dev/zero of=tempfile bs=5M count=1024 conv=fdatasync,notrunc
1024+0 records in
1024+0 records out
5368709120 bytes (5.4 GB) copied, 95.9093 s, 56.0 MB/s

# echo 3 > /proc/sys/vm/drop_caches
# dd if=tempfile of=/dev/null bs=5M count=1024
1024+0 records in
1024+0 records out
5368709120 bytes (5.4 GB) copied, 55.5027 s, 96.7 MB/s


# sysbench --num-threads=16 --test=fileio --file-total-size=5G --file-test-mode=rndrw run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 16

Extra file open flags: 0
128 files, 40Mb each
5Gb total file size
Block size 16Kb
Number of random requests for random IO: 10000
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Threads started!
Done.

Operations performed:  6000 Read, 4000 Write, 12800 Other = 22800 Total
Read 93.75Mb  Written 62.5Mb  Total transferred 156.25Mb  (11.263Mb/sec)
  720.82 Requests/sec executed

Test execution summary:
    total time:                          13.8731s
    total number of events:              10000
    total time taken by event execution: 0.1603
    per-request statistics:
         min:                                  0.01ms
         avg:                                  0.02ms
         max:                                  2.34ms
         approx.  95 percentile:               0.02ms

Threads fairness:
    events (avg/stddev):           625.0000/117.83
    execution time (avg/stddev):   0.0100/0.00



Обработка результатов


1. Сравнения скорости чтения/записи через dd, Mb/s:
image

2. Количество запросов в секунду через sysbench, Requests/s:
image

Итоги


Итак, похоже быстрее всех работает именно Ephemeral, как и ожидалось, но IOPS к нему самый низкий.

Скорость соединения со стандартными EBS выше, чем к оптимизированным, даже с 2к IOPS. Но количество IOPS больше у оптимизированных EBS, хотя RAID по IOPS всех победил.

В итоге, как и следовало ожидать, по количеству операций в секунду и простоте настроек оптимизированные EBS лучше, хотя использование рейдов из них будет ещё быстродейственнее.

Если вам важна скорость обращения к файлу — используйте ephemeral или стандартные EBS.
Tags: awsebsstoragebenchmark
Hubs: EPAM corporate blog Amazon Web Services
Total votes 24: ↑20 and ↓4 +16
Comments 13
Comments Comments 13

Information

Founded
1993
Location
США
Website
www.epam.com
Employees
over 10,000 employees
Registered
Representative
vesyolkinaolga