Oracle-RAC集群网络 IO性能测试

前言:

Oracle RAC集群对于网络，IO性能有一定的要求，特别是心跳网络，需要千兆以上，Oracle官方最佳实践建议配置万兆网络，而IO性能则是直接影响数据库的用户操作执行，因此，在进行RAC集群的部署时，我们需要对集群网络，IO性能进行测试，以确保性能指标满足集群运行要求。

本文接下来的内容将讲述Oracle RAC集群网络，IO性能常用的测试工具以及测试方法，希望对大家有所帮助。

测试工具:

网络:netpref

IO:orion,DBMS_RESOURCE_MANAGER.CALIBRATE_IO,fio

网络性能压测步骤(netpref)

安装编译netpref工具

tar -zxvf netperf-2.7.0.tar.gzcd netperf-netperf-2.7.0/./configuremakemake install

验证安装成功

[root@rac1 ~]# netperf -h​Usage: netperf [global options] -- [test options] ​Global options:-a send,recvSet the local send,recv buffer alignment-A send,recvSet the remote send,recv buffer alignment-B brandstr Specify a string to be emitted with brief output-c [cpu_rate]Report local CPU usage-C [cpu_rate]Report remote CPU usage-dIncrease debugging output-D time,[units] * Display interim results at least every time intervalusing units as the initial guess for units per secondA negative value for time will make heavy use of thesystem's timestamping functionality-f G|M|K|g|m|k Set the output units-F lfill[,rfill]* Pre-fill buffers with data from specified file-hDisplay this text-H name|ip,fam * Specify the target machine and/or local ip and family-i max,min Specify the max and min number of iterations (15,1)-I lvl[,intvl] Specify confidence level (95 or 99) (99) and confidence interval in percentage (10)-jKeep additional timing statistics-l testlen Specify test duration (>0 secs) (<0 bytes|trans)-L name|ip,fam * Specify the local ip|name and address family-o send,recvSet the local send,recv buffer offsets-O send,recvSet the remote send,recv buffer offset-n numcpu Set the number of processors for CPU util-NEstablish no control connection, do 'send' side only-p port,lport* Specify netserver port number and/or local port-P 0|1 Don't/Do display test headers-rAllow confidence to be hit on result only-s seconds Wait seconds between test setup and test start-SSet SO_KEEPALIVE on the data connection-t testname Specify test to perform-T lcpu,rcpuRequest netperf/netserver be bound to local/remote cpu-v verbositySpecify the verbosity level-W send,recvSet the number of send,recv buffers-v levelSet the verbosity level (default 1, min 0)-VDisplay the netperf version and exit-y local,remote Set the socket priority-Y local,remote Set the IP_TOS. Use hexadecimal.-Z passphraseSet and pass to netserver a passphrase​For those options taking two parms, at least one must be specified;specifying one value without a comma will set both parms to thatvalue, specifying a value with a leading comma will set just the secondparm, a value with a trailing comma will set just the first. To seteach parm to unique values, specify both and separate them with acomma.​* For these options taking two parms, specifying one value with no commawill only set the first parms and will leave the second at the defaultvalue. To set the second value it must be preceded with a comma or be acomma-separated pair. This is to retain previous netperf behaviour.

进行网络性能压测，netpref通过模拟服务端与客户端的TCP/UDP包网络发送，来测试主机之间的网络性能，这里测试的RAC为两节点的集群，所以节点一与节点二将分别作为服务段以及客户端进行测试

节点一RAC1作为服务端，启动netserver

[root@rac1 tmp]# netserver Starting netserver with host 'IN(6)ADDR_ANY' port '12865' and family AF_UNSPEC#关闭netserver​,可以通过kill方式

节点二RAC2作为客户端，发送TCP/UDP进行测试

进行TCP协议测试

参数说明#TCP_RR 在一个TCP连接中进行多次TCP request和response的交易过程，这种模式常常出现在数据库应用中#TCP_STREAM: TCP批量传输，在测试过程中，netperf向netserver发送批量的TCP数据分组，以确定数据传输过程中的吞吐量#-H 主机名IP#-l 时长[root@rac2 netperf-netperf-2.7.0]# netperf -t TCP_RR -H rac1-priv -l 60MIGRATED TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to rac1-priv () port 0 AF_INET : first burst 0Local /RemoteSocket Size Request Resp. Elapsed Trans.Send Recv SizeSize TimeRate bytes Bytes bytes bytes secs. per sec ​16384 87380 1 1 60.00 1721.92 16384 87380 #Trans Rate per sec 每秒的交易1721/s[root@rac2 netperf-netperf-2.7.0]# netperf -t TCP_STREAM -H rac1-priv -l 60MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to rac1-priv () port 0 AF_INETRecv Send SendSocket Socket Message Elapsed Size Size SizeTimeThroughput bytes bytes bytes secs. 10^6bits/sec ​87380 16384 16384 60.00 2808.18 #Throughput 10^6bits/sec 网络吞吐量每秒2808Mbps/s

进行UDP协议测试

参数说明#UDP_RR 进行多次UDP request和response的交易过程，这种模式常常出现在数据库应用中#UDP_STREAM: UDP批量传输，在测试过程中，netperf向netserver发送批量的UDP数据分组，以确定数据传输过程中的吞吐量#-H 主机名IP#-l 时长[root@rac2 netperf-netperf-2.7.0]# netperf -t UDP_RR -H rac1-priv -l 60MIGRATED UDP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to rac1-priv () port 0 AF_INET : first burst 0Local /RemoteSocket Size Request Resp. Elapsed Trans.Send Recv SizeSize TimeRate bytes Bytes bytes bytes secs. per sec ​262144 262144 1 1 60.00 1727.69 262144 262144#Trans Rate per sec 每秒的交易1727/s[root@rac2 netperf-netperf-2.7.0]# netperf -t UDP_STREAM -H rac1-priv -l 60MIGRATED UDP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to rac1-priv () port 0 AF_INETSocket Message ElapsedMessagesSize SizeTime Okay Errors Throughputbytes bytes secs ## 10^6bits/sec​262144 65507 60.00 441940386.00262144 60.00 44188 385.95#Throughput 10^6bits/sec 网络吞吐量每秒385Mbps/s

总结:通过netpref的测试，我们可以看到TCP_RR的指标为每秒1721/s,TCP_STREAM每秒的吞吐量为2808Mbps/s,UDP_RR的指标为每秒1727/s,UDP_STREAM每秒的吞吐量为385Mbps/s,这大概就是千兆网络的性能

IO性能压测步骤(orion)

ORION (Oracle I/O Calibration Tool)是校准用于Oracle数据库的存储系统I/O性能的独立工具。校准结果对于了解存储系统的性能有很大帮助，不仅可以找出影响Oracle数据库性能的问题，还能测量新数据库安装的大小。由于ORION是一个Oracle软件自带的独立工具，用户不需要创建和运行Oracle数据库

查看工具orion

[root@rac1 tmp]# /u01/app/oracle/product/11.2.0/db_1/bin/orion -helpORION: ORacle IO Numbers -- Version 11.2.0.4.0​ORION runs IO performance tests that model Oracle RDBMS IO workloads.It measures the performance of small (2-32K) IOs and large (128K+) IOsat various load levels. ​Each Orion data point is a test for a specific mix of small and large IO loads sustained for a duration. An Orion test consists of multiple data point tests. These data point tests can be represented as a two-dimensional matrix. Each column in the matrix represents data point tests with the same small IO load, but varying large IO loads. Each row represents data point tests with the same large IO load, but varying small IO loads. An Orion test can be for a single point, a single row, a single column, or the whole matrix.​The 'run' parameter is the only mandatory parameter. Defaultsare indicated for all other parameters. For additional information onthe user interface, see the Orion User Guide.​<testname> is the prefix used for all input and output filenames. Bydefault, it is 'orion'. It can be specified with the 'testname' parameter.​<testname>.lun should contain a carriage-return-separated list of LUNs.​The output files for a test run are prefixed by <testname>_<date> wheredate is "yyyymmdd_hhmm".​The output files are: <testname>_<date>_summary.txt - Summary of the input parameters, along with the minimum small IO latency (in usecs), the maximum MBPS, and the maximum IOPS observed. <testname>_<date>_mbps.csv - Performance results of large IOs in MBPS.<testname>_<date>_iops.csv - Performance results of small IOs in IOPS.<testname>_<date>_lat.csv - Latency of small IOs in microseconds.<testname>_<date>_hist.csv - Histogram of IO latencies.<testname>_<date>_trace.txt - Extended, unprocessed output.​WARNING: IF YOU ARE PERFORMING WRITE TESTS, BE PREPARED TO LOSE ANY DATA STOREDON THE LUNS.​Mandatory parameters:runType of workload to run (simple, normal, advanced, dss, oltp).simple - Tests random small (8K) IOs at various loads,then random large (1M) IOs at various loads.normal - Tests combinations of random small (8K) IOs and random large (1M) IOs.advanced - Tests the workload specified by the user using optional parameters.dss- Tests with random large (1M) IOs at increasing loads to determine the maximum throughput.oltp- Tests with random small (8K) IOs at increasing loads to determine the maximum IOPS.​Optional parameters:testname Name of the test run.​num_disks Number of disks (physical spindles). This number is used to gauge the range of loads that Orion should test at. Increasing this parameter results in Orion using heavier IO loads. Default is the number of LUNs in <testname>.lun.​size_small Size of small IOs in KB. Default is 8.​size_large Size of large IOs in KB. Default is 1024.​typeType of large IOs (rand, seq). Default is rand.rand - Randomly distributed large IOs.seq - Sequential streams of large IOs.​num_streamIO Number of concurrent IOs per stream. This parameter is only used if -type is seq. Default is 4.​simulate Orion tests on a virtual LUN formed by combining thespecified LUNs in one of these ways. This parameter is typically only used if -type is seq. Default is concat.concat - A serial concatenation of the LUNs. Each sequential stream issues IOs to only one LUN. raid0 - A RAID-0 mapping across the LUNs. Each sequential stream issues IOs across all LUNs, using RAID-0 striping.​writePercentage of IOs that are writes (SEE WARNING ABOVE).Default is 0.​cache_size Size in MBs of the array's cache.Unless this option is set to 0, Orion issues a numberof unmeasured, random IOs before each large sequentialdata point. These IOs fill up the storage array's cache(if any) with random data so that IOs from onedata point do not result in cache hits for the nextdata point. Read tests are preceded with junk readsand write tests are preceded with junk writes. Ifspecified, this 'cache warming' is performed untilcache_size MBs of IO have been read or written.Default behavior is to issue 2 minutes of unmeasured random IOs before each data point.​duration Duration of each data point in seconds. Default is 60.​num_small Number of outstanding small IOs. This parameter controls the small IO load. Only used if -matrix is point, col, or max. No default.​num_large This parameter controls the large IO load.For -type rand, number of outstanding large IOs.For -type seq, number of sequential IO streams. Only used if -matrix is point, row, or max. No default.​matrixAn Orion test consists of multiple data point tests. These data point tests can be represented as a two-dimensional matrix. Each column in the matrix represents data point tests with the same small IO load, but varying large IO loads. Each row represents data point tests with the same large IO load, but varying small IO loads. An Orion test can be for a single point, a single row, a single column, or the whole matrix, depending on the matrix option setting below. Default is basic.basic - Test small IOs only, then large IOs only.detailed - Test entire matrix.point - Test with num_small small IOs, num_large large IOs.col- Test a varying large IO load with num_small small IOs.row- Test a varying small IO load with num_large large IOs.max- Test varying loads up to the num_small and num_large limits.​hugenotneeded Tells Orion to allocate regular pages when hugepages are not available. With out this option, on systemsthat support hugepages, Orion errors out when it cannotallocate sufficient huge pages. -hugenotneeded is not set bydefault.​verbosePrints tracing information to standard output if set.Not set by default.datainput Name of the input file for data to be written.​is_lgwrif set to true will run test using VIP level QOS, assuming the underlying transport supports it.​​Examples:For a preliminary set of data-run simple For a basic set of data-run normal To evaluate storage for an OLTP database-run oltp To evaluate storage for a data warehouse-run dss To generate combinations of 32KB and 1MB reads to random locations-run advanced -size_small 32 -size_large 1024 -type rand -matrix detailedTo generate multiple sequential 1MB write streams, simulating RAID0 striping-run advanced -simulate RAID0 -write 100 -type seq-matrix col -num_small 0​

配置lun测试文件

#测试磁盘/dev/mapper/test1,/dev/mapper/test2,/dev/mapper/test3[root@rac1 tmp]# cat tlun.lun /dev/mapper/test1/dev/mapper/test2/dev/mapper/test3

开始进行IO性能测试

#使用root用户export ORACLE_HOME=/u01/app/oracle/product/11.2.0/db_1export PATH=$ORACLE_HOME/bin:$PATH:/usr/local/binexport LD_LIBRARY_PATH=$ORACLE_HOME/lib:$ORACLE_HOME/rdbms/lib:/lib:/usr/libexport CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib:$ORACLE_HOME/network/jlib#测试参数#模拟数据库随机小事务8K，读写占比7:3#每个节点开启2个压测，共四个压测/u01/app/oracle/product/11.2.0/db_1/bin/orion -run advanced -testname tlun -num_disks 3 -size_small 8 -size_large 8 -type rand -write 30 -hugenotneeded​[root@rac1 tmp]# /u01/app/oracle/product/11.2.0/db_1/bin/orion -run advanced -testname tlun -num_disks 3 -size_small 8 -size_large 8 -type rand -write 30 -hugenotneededORION: ORacle IO Numbers -- Version 11.2.0.4.0tlun_0904_1717Calibration will take approximately 23 minutes.Using a large value for -cache_size may take longer.​​Maximum Large MBPS=74.07 @ Small=0 and Large=6​Maximum Small IOPS=9583 @ Small=6 and Large=0Small Read Latency: avg=623 us, min=85 us, max=10795 us, std dev=257 us @ Small=6 and Large=0Small Write Latency: avg=627 us, min=94 us, max=10431 us, std dev=243 us @ Small=6 and Large=0​Minimum Small Latency=208 usecs @ Small=1 and Large=0Small Read Latency: avg=232 us, min=78 us, max=6918 us, std dev=138 us @ Small=1 and Large=0Small Write Latency: avg=151 us, min=88 us, max=4942 us, std dev=109 us @ Small=1 and Large=0Small Read / Write Latency Histogram @ Small=1 and Large=0 Latency: # of IOs (read) # of IOs (write) 0 - 1 us: 002 - 4 us: 004 - 8 us: 008 - 16 us: 0016 - 32 us: 0032 - 64 us: 0064 - 128 us: 429014128 - 256 us: 9845138491256 - 512 us: 571904334512 - 1024 us: 24135491024 - 2048 us: 5221732048 - 4096 us: 126374096 - 8192 us: 1538192 - 16384 us: 0016384 - 32768 us: 0032768 - 65536 us: 0065536 - 131072 us: 00131072 - 262144 us: 00262144 - 524288 us: 00524288 - 1048576 us: 001048576 - 2097152 us: 002097152 - 4194304 us: 004194304 - 8388608 us: 008388608 - 16777216 us: 0016777216 - 33554432 us: 0033554432 - 67108864 us: 0067108864 - 134217728 us: 00134217728 - 268435456 us: 00---测试结果Maximum Large MBPS=74.07Maximum Small IOPS=9583

会在当前目录下生成测试的mbps,iops,latency的数据

[root@rac1 tmp]# ls -lrt tlun*-rw-r--r-- 1 root root 54 Sep 4 17:13 tlun.lun-rw-r--r-- 1 root root 20240 Sep 4 17:38 tlun_0904_1717_trace.txt-rw-r--r-- 1 root root 658 Sep 4 17:38 tlun_0904_1717_mbps.csv-rw-r--r-- 1 root root 758 Sep 4 17:38 tlun_0904_1717_lat.csv-rw-r--r-- 1 root root 734 Sep 4 17:38 tlun_0904_1717_iops.csv-rw-r--r-- 1 root root 20587 Sep 4 17:38 tlun_0904_1717_hist.txt-rw-r--r-- 1 root root 2151 Sep 4 17:38 tlun_0904_1717_summary.txt

总结:测试的结果为最大IOPS:9583,最大吞吐量74Mbps/s

IO性能压测步骤(DBMS_RESOURCE_MANAGER.CALIBRATE_IO)

DBMS_RESOURCE_MANAGER.CALIBRATE_IO为Oracle 11g数据库中的PL/SQL存储过程，可以用于对当前数据库实例的I/O性能进行测试，使用简单

CALIBRATE_IO参数：

进行IO性能压测

set timing onset time onSET SERVEROUTPUT ONDECLARElat INTEGER;iops INTEGER;mbps INTEGER;BEGINDBMS_RESOURCE_MANAGER.CALIBRATE_IO (num_physical_disks => 2,max_iops=> iops,max_mbps=> mbps,actual_latency => lat);DBMS_OUTPUT.PUT_LINE ('max_iops = ' || iops);DBMS_OUTPUT.PUT_LINE ('latency = ' || lat);DBMS_OUTPUT.PUT_LINE ('max_mbps = ' || mbps);end;/

查看IO性能压测结果

SELECT TO_CHAR(start_time, 'DD-MON-YYY HH24:MI:SS') AS start_time,TO_CHAR(end_time, 'DD-MON-YYY HH24:MI:SS') AS end_time,max_iops,max_mbps,max_pmbps,latency,num_physical_disks AS disksFROM dba_rsrc_io_calibrate;18:03:16 SQL> 18:03:16 SQL> 18:03:16 2 18:03:16 3 18:03:16 4 18:03:16 5 18:03:16 6 18:03:16 7 18:03:16 8 ​START_TIME END_TIMEMAX_IOPS MAX_MBPS MAX_PMBPS LATENCYDISKS---------------------- ---------------------- ---------- ---------- ---------- ---------- ----------04-SEP-022 17:53:24 04-SEP-022 18:02:5513070 190 112

总结:测试的结果为最大IOPS:13070,最大吞吐量1200Mbps/s

IO性能压测步骤(FIO)

fio软件安装

tar xvf fio-2.17.tar.gzcd fio-3.13./configure make make install

确认安装成功

[root@rac1 fio-2.17]# fio --helpfio-2.17fio [options] [job options] <job file(s)>--debug=options Enable debug logging. May be one/more of:process,file,io,mem,blktrace,verify,random,parse,diskutil,job,mutex,profile,time,net,rate,compress,steadystate,helperthread--parse-only Parse options only, don't start any IO--output Write output to file--bandwidth-log Generate aggregate bandwidth logs--minimal Minimal (terse) output--output-format=type Output format (terse,json,json+,normal)--terse-version=type Set terse version output format (default 3, or 2 or 4)--version Print version info and exit--help Print this page--cpuclock-test Perform test/validation of CPU clock--crctest=type Test speed of checksum functions--cmdhelp=cmd Print command help, "all" for all of them--enghelp=engine Print ioengine help, or list available ioengines--enghelp=engine,cmd Print help for an ioengine cmd--showcmd Turn a job file into command line options--eta=when When ETA estimate should be printedMay be "always", "never" or "auto"--eta-newline=time Force a new line for every 'time' period passed--status-interval=t Force full status dump every 't' period passed--readonly Turn on safety read-only checks, preventing writes--section=name Only run specified section in job file, multiple sections can be specified--alloc-size=kb Set smalloc pool to this size in kb (def 16384)--warnings-fatal Fio parser warnings are fatal--max-jobs=nr Maximum number of threads/processes to support--server=args Start a backend fio server--daemonize=pidfile Background fio server, write pid to file--client=hostname Talk to remote backend(s) fio server at hostname--remote-config=file Tell fio server to load this local job file--idle-prof=option Report cpu idleness on a system or percpu basis(option=system,percpu) or run unit workcalibration only (option=calibrate)--inflate-log=log Inflate and output compressed log--trigger-file=file Execute trigger cmd when file exists--trigger-timeout=t Execute trigger af this time--trigger=cmd Set this command as local trigger--trigger-remote=cmd Set this command as remote trigger--aux-path=path Use this path for fio state generated files​Fio was written by Jens Axboe <jens.axboe@>Jens Axboe <jaxboe@>Jens Axboe <axboe@>

使用参数说明

filename=/dev/mapper/test1 测试文件名称direct=1 绕过操作系统的buffer，即进行O_DIRECT操作iodepth IO队列深度，在文件之上的保持的IO单元,默认对于每个文件来说是1，可以设置一个更大的值来提供并发度，iodepth大于1不会影响同步IO引擎(除非verify_async这个选项被设置)​thread io默认会使用fork()创建job，如果这个选项设置的话，fio将使用pthread_create来创建线程rw=randwrite 测试随机写的I/Orw=randrw 测试随机写和读的I/Obs=8k 单次io的块文件大小为8k，仿照数据库的块大小size=1g 本次的测试文件大小为1g，以每次8k的io进行测试。numjobs=4 本次的测试线程为4.runtime=300 测试时间为300秒，如果不写测试时间，则将1g文件按每8k写完结束ioengine=psync io引擎使用pync方式rwmixwrite=30 在混合读写的模式下，写占30%rwmixread=70 在混合读写的模式下，读占70%group_reporting 关于显示结果的，汇总每个进程的信息​lockmem=1g 只使用1g内存进行测试​zero_buffers 用0初始化系统buffer​nrfiles=8 每个进程生成文件的数量

IO性能压测

---数据库场景主要是测试随机读写---混合随机读写测试[root@rac1 fio-2.17]# fio -filename=/dev/mapper/test1 -direct=1 -iodepth 1 -thread -rw=randrw -rwmixread=70 -ioengine=psync -bs=8k -size=1G -numjobs=4 -runtime=60 -group_reporting -name=iotest -ioscheduler=deadlineiotest: (g=0): rw=randrw, bs=8192B-8192B,8192B-8192B,8192B-8192B, ioengine=psync, iodepth=1...fio-2.17Starting 4 threadsfio: os or kernel doesn't support IO scheduler switchingfio: os or kernel doesn't support IO scheduler switchingfio: os or kernel doesn't support IO scheduler switchingfio: os or kernel doesn't support IO scheduler switching#最大读写吞吐量/IOPSJobs: 4 (f=4): [m(4)][100.0%][r=58.5MiB/s,w=25.4MiB/s][r=7486,w=3249 IOPS][eta 00m:00s]iotest: (groupid=0, jobs=4): err= 0: pid=97958: Sun Sep 4 19:05:36 read: IOPS=6897, BW=53.1MiB/s (56.6MB/s)(2867MiB/53205msec)clat (usec): min=75, max=16751, avg=391.74, stdev=335.46lat (usec): min=75, max=16751, avg=392.36, stdev=335.54clat percentiles (usec):| 1.00th=[ 80], 5.00th=[ 92], 10.00th=[ 108], 20.00th=[ 147],| 30.00th=[ 205], 40.00th=[ 266], 50.00th=[ 322], 60.00th=[ 382],| 70.00th=[ 454], 80.00th=[ 548], 90.00th=[ 724], 95.00th=[ 924],| 99.00th=[ 1656], 99.50th=[ 2096], 99.90th=[ 3312], 99.95th=[ 3760],| 99.99th=[ 5984]write: IOPS=2956, BW=23.2MiB/s (24.3MB/s)(1229MiB/53205msec)clat (usec): min=82, max=16158, avg=412.63, stdev=334.53lat (usec): min=83, max=16158, avg=414.12, stdev=334.91clat percentiles (usec):| 1.00th=[ 89], 5.00th=[ 96], 10.00th=[ 110], 20.00th=[ 149],| 30.00th=[ 219], 40.00th=[ 290], 50.00th=[ 354], 60.00th=[ 418],| 70.00th=[ 494], 80.00th=[ 588], 90.00th=[ 756], 95.00th=[ 940],| 99.00th=[ 1592], 99.50th=[ 2064], 99.90th=[ 3312], 99.95th=[ 3824],| 99.99th=[ 6304]lat (usec) : 100=6.81%, 250=29.70%, 500=37.52%, 750=16.68%, 1000=5.18%lat (msec) : 2=3.53%, 4=0.53%, 10=0.04%, 20=0.01%cpu: usr=1.31%, sys=19.35%, ctx=744673, majf=0, minf=9IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%issued rwt: total=366973,157315,0, short=0,0,0, dropped=0,0,0latency : target=0, window=0, percentile=100.00%, depth=1​Run status group 0 (all jobs):READ: bw=53.1MiB/s (56.6MB/s), 53.1MiB/s-53.1MiB/s (56.6MB/s-56.6MB/s), io=2867MiB (3006MB), run=53205-53205msecWRITE: bw=23.2MiB/s (24.3MB/s), 23.2MiB/s-23.2MiB/s (24.3MB/s-24.3MB/s), io=1229MiB (1289MB), run=53205-53205msec​---混合顺序读写测试：[root@rac1 fio-2.17]# fio -filename=/dev/mapper/test1 -direct=1 -iodepth 1 -thread -rw=rw -rwmixread=70 -ioengine=psync -bs=8k -size=1G -numjobs=4 -runtime=60 -group_reporting -name=iotest -ioscheduler=deadlineiotest: (g=0): rw=rw, bs=8192B-8192B,8192B-8192B,8192B-8192B, ioengine=psync, iodepth=1...fio-2.17Starting 4 threadsfio: os or kernel doesn't support IO scheduler switchingfio: os or kernel doesn't support IO scheduler switchingfio: os or kernel doesn't support IO scheduler switchingfio: os or kernel doesn't support IO scheduler switchingJobs: 4 (f=4): [M(4)][100.0%][r=48.5MiB/s,w=20.6MiB/s][r=6205,w=2627 IOPS][eta 00m:00s]iotest: (groupid=0, jobs=4): err= 0: pid=99479: Sun Sep 4 19:09:41 read: IOPS=6943, BW=54.3MiB/s (56.9MB/s)(2867MiB/52851msec)clat (usec): min=73, max=13036, avg=392.24, stdev=346.79lat (usec): min=73, max=13036, avg=392.79, stdev=347.15clat percentiles (usec):| 1.00th=[ 78], 5.00th=[ 87], 10.00th=[ 104], 20.00th=[ 147],| 30.00th=[ 199], 40.00th=[ 262], 50.00th=[ 318], 60.00th=[ 378],| 70.00th=[ 450], 80.00th=[ 548], 90.00th=[ 724], 95.00th=[ 948],| 99.00th=[ 1752], 99.50th=[ 2192], 99.90th=[ 3408], 99.95th=[ 4128],| 99.99th=[ 5984]write: IOPS=2976, BW=23.3MiB/s (24.4MB/s)(1229MiB/52851msec)clat (usec): min=81, max=12001, avg=413.64, stdev=360.46lat (usec): min=81, max=12001, avg=414.78, stdev=360.97clat percentiles (usec):| 1.00th=[ 86], 5.00th=[ 91], 10.00th=[ 103], 20.00th=[ 143],| 30.00th=[ 207], 40.00th=[ 278], 50.00th=[ 350], 60.00th=[ 410],| 70.00th=[ 486], 80.00th=[ 580], 90.00th=[ 756], 95.00th=[ 980],| 99.00th=[ 1784], 99.50th=[ 2256], 99.90th=[ 3472], 99.95th=[ 4192],| 99.99th=[ 6496]lat (usec) : 100=8.62%, 250=28.16%, 500=37.90%, 750=15.82%, 1000=5.00%lat (msec) : 2=3.81%, 4=0.63%, 10=0.06%, 20=0.01%cpu: usr=0.80%, sys=19.45%, ctx=750369, majf=0, minf=9IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%issued rwt: total=366973,157315,0, short=0,0,0, dropped=0,0,0latency : target=0, window=0, percentile=100.00%, depth=1​Run status group 0 (all jobs):READ: bw=54.3MiB/s (56.9MB/s), 54.3MiB/s-54.3MiB/s (56.9MB/s-56.9MB/s), io=2867MiB (3006MB), run=52851-52851msecWRITE: bw=23.3MiB/s (24.4MB/s), 23.3MiB/s-23.3MiB/s (24.4MB/s-24.4MB/s), io=1229MiB (1289MB), run=52851-52851msec​---其他场景---随机读测试：fio -filename=/dev/mapper/test1 -direct=1 -iodepth 1 -thread -rw=randread -ioengine=psync -bs=8k -size=1G -numjobs=4 -runtime=60 -group_reporting -name=iotest ​---随机写测试：fio -filename=/dev/mapper/test1 -direct=1 -iodepth 1 -thread -rw=randwrite -ioengine=psync -bs=8k -size=1G -numjobs=4 -runtime=60 -group_reporting -name=iotest​---顺序读测试：fio -filename=/dev/mapper/test1 -direct=1 -iodepth 1 -thread -rw=read -ioengine=psync -bs=1M -size=1G -numjobs=4 -runtime=60 -group_reporting -name=iotest​---顺序写测试：fio -filename=/dev/mapper/test1 -direct=1 -iodepth 1 -thread -rw=write -ioengine=psync -bs=1M -size=1G -numjobs=4 -runtime=60 -group_reporting -name=iotest

总结:

以上内容为Oracle RAC集群网络，IO性能常用的测试工具以及测试方法，大家可以根据自身的环境以及使用场景进行选择。