Although there no way around the initial send time without reworking the
underlying send code, however, if you are sending identical data to
multiple hosts, you could do the following:
* one 'zfs send' per filesystem piped through pigz (gzip) or lz4c, save
the file to local storage
* md5sum on generated file
* As the files become available, send some sort of 'trigger' to your
recv hosts to rsync, verify, and recv that dataset in the background. (I
use a PHP script on each host with lighttpd that lets the sending side
tell the recv side to copy this file from a read-only NFS mount, copy it
via SSH or even netcat to local storage - whatever the user has configured.)
I have iSCSI servers storing Windows and other desktop images, with
updates coming from a single master server. I may have 5 or more servers
recving the same data at once, and when storing a file before
transferring it over the network the data is stored in ARC and you can
achieve transfer speeds in the multi-hundred MB/s range. Using this
method of replication I can reduce transfer time from hours to just 30
minutes or 1 hour.. it's significant.
Post by firstname.lastname@example.org
There was another tests that I ran. When I turned the cache on for
creating a FS and it's snapshot, the SEND performance was excellent.
This was because the blocks were already in the ARC. But when I
restarted the system, the cache is empty, as expected, and you loose
the performance again.
I believe the solution you guys at Delphix is working on is great, but
it does not address the root cause. That is the SEND uses a single
tree walk and then writes to a single sector of the disk. We probably
need a parallel tree walker that uses range locks (already available
in ZFS), that gets the blocks in a parallel way and writes them to
disk in a parallel manner too.
To unsubscribe from this group and stop receiving emails from it, send
To unsubscribe from this group and stop receiving emails from it, send an email to email@example.com.