Projects / fio

fio

fio is an I/O tool meant to be used both for benchmark and stress/hardware verification. It has support for 19 different types of I/O engines (sync, mmap, libaio, posixaio, SG v3, splice, null, network, syslet, guasi, solarisaio, and more), I/O priorities (for newer Linux kernels), rate I/O, forked or threaded jobs, and much more. It can work on block devices as well as files. fio accepts job descriptions in a simple-to-understand text format. Several example job files are included. fio displays all sorts of I/O performance information, including complete IO latencies and percentiles. Fio is in wide use in many places, for both benchmarking, QA, and verification purposes. It supports Linux, FreeBSD, NetBSD, OpenBSD, OS X, OpenSolaris, AIX, HP-UX, Android, and Windows.

Tags
Licenses
Operating Systems
Implementation

RSS Recent releases

  •  01 Apr 2014 01:26

    Release Notes: Adds support for explicit file appending. More accurate implementation of number_ios. Increases the job description field length. More accurate data compression settings. Windows build improvements. A SPARC64 SIGBUS bug has been fixed. Various little bugfixes and improvements.

    •  07 Mar 2014 00:57

      Release Notes: Full support for IPv6 for the net I/O engine. Windows threading fixes. Support for ranges in the 'startdelay' option. Adds rbd (rados block device) engine support. Various fixes for blktrace playback on Linux. Adds OpenBSD support. An option added to manually seed random generators. Various client/server fixes. A fix for a crash on some file locking options. Adds support for fast software-based xxhash, and support for testing checksum speeds. A fix for not always honoring the 'openfiles' setting. Support for setting policy on how to distribute CPUs set for jobs. Support for data retention testing with verify.

      •  19 Nov 2013 04:01

        Release Notes: Adds JSON output for client/server. A fix for Windows build breakage in the previous release. A fix for ioengine specific options. Adds support for O_ATOMIC atomic writes on Linux. A failsafe check for realloc() failure during logging. A fix for a Solaris thread problem. Various man page and documentation updates. A fix for a crash on PowerPC without the mfspr instruction.

        •  07 Aug 2013 20:04

          Release Notes: This release adds basic support for an ACT (aerospike) profile, fixes atomic write support, adds various parsing fixes, makes a big revamp of the included graphing support, makes the IOPS log includes bandwidth, adds support for specific randomness settings for both reads and writes, adds support for random vs. sequential blocksize settings, fixes a potential verification failure, and adds multicast support for the net ioengine.

          •  15 May 2013 21:56

            Release Notes: Integration of the gfio GTK frontend. Better Android integration. The initial file fill honors data compression settings. An option for controlling randomness of I/O (percentage_random) instead of being fully random or fully sequential. Fixes a bug in uneven pattern sizes for verification. Various other important fixes.

            RSS Recent comments

            07 Oct 2013 13:56 diatta

            I have written a script using fio the benchmark filesystems that I would like to share:

            fio.pl

            #!/usr/bin/perl
            # Program name : my_fio_script_generator.pl
            # Author : Mamadou Lamine Diatta
            # Email: diatta at post dot harvard dot edu
            # Purpose : Generate job files used by Linux command fio.
            # ==========================================================================
            # Here few example how to use fio at the command line without a job file.
            #
            #Storage subsystem:
            # IBM EXP300 - 14 Disk Fiber Expansion, 17G - 15K RPMS
            # Host: AMD Barcelona, 2 socket, 8G RAM
            # HBA: QLogic Corp. ISP2312-based 2Gb Fibre Channel to PCI-X HBA (rev 02)
            #
            #Benchmark[1]:
            # fio --name=guestrun --filename=/dev/mapper/volumes-fibre \
            # --rw=randwrite --bs=16k --ioengine=libaio --direct=1 \
            # --norandommap --runtime=120 --time_based --numjobs=1 \
            # --group_reporting --thread --size=25g --write_lat_log \
            # --write_bw_log --iodepth=74
            #
            #Qemu parameters:
            # -m 1024 \
            # -drive file=/images/npt2-guest-virtio.qcow2,if=ide,boot=on,snapshot=off \
            # -drive file=/dev/mapper/volumes-fibre,if=virtio,cache=(on|off) \
            # -drive file=/dev/mapper/volumes-npt2--dom1,if=virtio,cache=off
            # -net nic,macaddr=00:FF:FF:00:00:01,model=rtl8139 -net tap -vnc :123 \
            # -monitor stdio
            #
            # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            # Few functions here.
            # ***************************************************************************
            sub usage {
            print<
            Where: is a full path access read and write directory
            NOTE: If the directory doesn't exist, it will be created if possible
            i.e: $0 /asm01/fio/io1
            --------------------------------------------------------------------

            END
            }
            # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            sub generate_filename {
            my $range = 36;
            my $minimum = 0;
            @ALPHA=('q','w','e','r','0','t','y','1','u','i','2','o','p','3','a','s','4','d','f','5','g','h','j','6','k','l','7','z','x','8','c','v','9','b','n','m');
            $filenamesize=10;
            $line=1;
            $filename="";
            while(1){
            my $random_number = int(rand($range)) + $minimum;
            $filename=$filename . $ALPHA[$random_number];
            if ($line == $filenamesize) {
            return $filename;
            }
            $line++;
            }
            }
            # ***************************************************************************
            sub generate_seconds {
            my $range = 60;
            my $minimum = 1;
            my $seconds = int(rand($range)) + $minimum;
            return $seconds;
            }
            # ***************************************************************************
            sub generate_percent {
            my $range = 100;
            my $minimum = 1;
            my $percent = int(rand($range)) + $minimum;
            return $percent;
            }
            # ***************************************************************************
            sub generate_numjobs {
            my $range = 10;
            my $minimum = 0;
            my $numjobs = int(rand($range)) + $minimum;
            return $numjobs;
            }
            # ***************************************************************************
            sub generate_runtime {
            my $range = 3600;
            # Run between 1s or 1 hour = 60*60 seconds
            my $minimum = 0;
            my $runtime = int(rand($range)) + $minimum;
            return $runtime;
            }
            # ***************************************************************************
            sub generate_nrfiles {
            my $range = 5000;
            my $minimum = 1;
            my $ioload = int(rand($range)) + $minimum;
            return $ioload;
            }
            # ***************************************************************************
            sub generate_filesize {
            my $rangesize = 9;
            my $rangebyte = 3;
            my $minimum = 0;
            @BYTE=('k','m','g');
            @SIZEK=('1024','2048','4092','4096','5120','6144','7168','8192','9216');
            @SIZEM=('100','200','300','400','500','600','700','800','900');
            @SIZEG=('1','2','3','4','5','6','7','8','9');
            $filesize="";

            my $random_number = int(rand($rangebyte)) + $minimum;
            my $byte=$BYTE[$random_number];
            if ( $byte eq "k" ){
            my $random_number = int(rand($rangesize)) + $minimum;
            $filesize=$SIZEK[$random_number] . $byte;
            }
            elsif ( $byte eq "m" ){
            my $random_number = int(rand($rangesize)) + $minimum;
            $filesize=$SIZEM[$random_number] . $byte;
            }
            else {
            my $random_number = int(rand($rangesize)) + $minimum;
            $filesize=$SIZEG[$random_number] . $byte;
            }
            return $filesize;
            }
            # ***************************************************************************
            sub generate_write_read_options {
            my $rangerw = 4;
            my $minimum = 0;
            @RW=('read','randread','write','randwrite');
            my $random_number = int(rand($rangerw)) + $minimum;
            $rw=$RW[$random_number];
            return $rw;
            }
            # ***************************************************************************
            # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            # Main function
            # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            #print "#ARGV=$#ARGV\n";
            if ( $#ARGV != 0) {
            &usage;
            exit 1;
            }
            my $workingdir=$ARGV[0];
            if ( ! -d "$workingdir" ) {
            mkdir ($workingdir, 0755) ||
            die "Can't make directory: $workingdir: $!\n";
            }
            else {
            print "Directory: $workingdir already exist!\n";
            }
            while(1) {
            $FIO="/usr/local/bin/fio";
            $RM="/bin/rm -fr";
            $CAT="/bin/cat";
            my $scriptdir="/tmp";

            my $fio_script= &generate_filename . ".fio";
            open(F ,">$scriptdir/$fio_script") ||
            die "Can't open file : $scriptdir/$fio_script : $!\n";

            my $rw= &generate_write_read_options;
            my $filename= &generate_filename;
            my $filesize= &generate_filesize;
            my $rwmixread= &generate_percent;
            my $rwmixwrite= 100 - $rwmixread;
            # Note: rwmixread and rwmixwrite are related and must
            # add up to 100%.

            my $nrfiles= &generate_nrfiles;
            my $runtime= &generate_runtime;
            my $numjobs= &generate_numjobs;
            print F "; $rw $filesize with rwmixread=$rwmixread and rwmixwrite=$rwmixwrite\n";
            print F "[${rw}-${rwmixread}-${rwmixwrite}]\n";
            print F "rw=$rw\n";
            print F "directory=$workingdir\n";
            print F "filename=$filename\n";
            print F "size=$filesize\n";
            print F "rwmixread=$rwmixread\n";
            print F "rwmixwrite=$rwmixwrite\n";
            print F "nrfiles=$nrfiles\n";
            print F "runtime=$runtime\n";
            print F "description=${rw}_${filename}_${filesize}_${rwmixread}_${rwmixwrite}_${nrfiles}_${runtime}\n";
            print F "unlink=1\n";
            # If set unlink files when it's done.
            print F "numjobs=$numjobs\n";
            print F "write_bw_log\n";
            # To write bandwidth log.
            print F "write_lat_log\n";
            # To write latency log.
            print F "; ----------------------------------------------------\n\n\n";
            close(F);
            print "---------------------- Running $FIO $scriptdir/$fio_script\n";
            open(R, "$scriptdir/$fio_script") ||
            die "Can't open $scriptdir/$fio_script : $!\n";
            while(){
            chop $_;
            print "$_\n";
            }
            print "\n";
            print "\n";
            close(R);
            `cat $scriptdir/$fio_script > ${fio_script}.log`;
            `$FIO $scriptdir/$fio_script >> ${fio_script}.log 2>&1`;
            `$RM $scriptdir/$fio_script`;
            sleep 10;
            }
            # ***********************************************************************************
            # #######################################################

            30 Nov 2012 18:10 bslade

            Getting 'stat64 already defined' type errors when trying to "make" fio on RedHat/Centos 6.3 (linux kernel 2.6.18-308). Nothing in the www.spinics.net/lists/fio mailing list archives about it. No response to my signup request to the fio@vger.kernel.org mailing list, so I'm posting this problem here.

            Has anybody else seen a problem like this?

            > make
            DEP .depend
            CC gettime.o
            : Assembler messages:
            :354: Error: symbol `fstatat64' is already defined
            :388: Error: symbol `fstat64' is already defined
            :418: Error: symbol `lstat64' is already defined
            :448: Error: symbol `stat64' is already defined
            :1704: Error: symbol `pread64' is already defined
            make: *** [gettime.o] Error 1

            25 Oct 2012 16:44 dschulz

            Id appreciate it if anyone here could let me know how to find that information.

            25 Oct 2012 16:43 dschulz

            When was the switch made to version 3 from version 2 of the headers for --minimal output and was something other than the order of the FIO Version/Jobname/Groupid/Error changed?

            moving between 1.5.8 and 2.0.10 it went from:
            fio 1.50
            2 job 0 0

            to

            3 fio-2.0.10 job 0 0

            10 Oct 2012 17:30 KeithLocke Thumbs up

            I have a couple of test requirements that I would appreciate guidance on how to perform with fio.

            1. 80/20 mix of random/sequential IO.
            2. 80% Random, 10% '00', & 10% 'FF' mixture of data patterns

            Screenshot

            Project Spotlight

            TurnKey Trac Appliance

            A Trac appliance that is easy to use and lightweight.

            Screenshot

            Project Spotlight

            libcsdbg

            A C++ exception stack tracer.