On Fri, Jul 11, 2014 at 04:02:09PM -0400, Andrew Z wrote:
> >
> > ... synchronizing a flashing drive (target) with my hard drive (source) ...
> >
> > Problem: it is slow -- takes three hours.  To help the
> > speed issue, I upgraded from USB 2 to USB 3.  Backup went
> > from 3 hr-15 min to 3 hr-5 min.  It is almost faster
> > to wipe the stick and rewrite it.
> >


The main question is this: what is the actual write speed of your USB flash media? How about the re-write speed? (not the same, obviously - as it requires as erase step).

I ask because I use USB flash media as boot and linux system disks on embedded machines (VME SBCs)
and I have looked at different USB flash media. Most of them are very slow, actually it is very hard
to find "fast" USB flash media.

The common media you get for $10 at Staples is "read 30M/s, write 10M/s" (regardless of USB2 or USB3 interface). This is probably consistent with the speeds that you see.

With some work, you can find media that writes at 20-30M/s, as measured by timing "dd", but drops
severely when you time "rsync" (must be inefficient at writing small files).

So when you select a brand USB flash drive for your workload, as you run "rsync", watch
the output of "vmstat 1" (the "bo" column is Mbytes/sec written to disk) and
the output of "iostat -x 1" - you will see %util pegged at 100% and "svctm" (in msec) running
in 1-10-20 seconds for "slow media", a little bit smaller for betyter media. For HDDs and SSDs,
the "svctm" is in low milliseconds. "svctm" is the request service time - time from sending
a request to the drive and getting the reply from the drive that the request is finished.

Some USB drives advertize high write speeds (not "up to" but actual "will write at" promises),
you can try those ($$$), but you will probably find that the speed of "rsync" does not reach
the promised rates because of inefficiency of flashing small files.

P.S. Another problem with USB flash drives - all brands except for 1 or 2 do not survive
being used as linux system disks - they brick themselves within days or weeks. I notice
that they tend to run quite hot, so I suspect they simply overheat and die. Actually, 
I did not find a single USB3 flash drive that survives use as linux system disk yet.
By luck I have enough Patriot RageXT 8GB and 16GB USB2 flash media, these seem to last.


-- 
Konstantin Olchanski
Data Acquisition Systems: The Bytes Must Flow!
Email: olchansk-at-triumf-dot-ca
Snail mail: 4004 Wesbrook Mall, TRIUMF, Vancouver, B.C., V6T 2A3, Canada