On 07/11/2014 06:12 PM, Konstantin Olchanski wrote:
> On Fri, Jul 11, 2014 at 04:02:09PM -0400, Andrew Z wrote:
>>>
>>> ... synchronizing a flashing drive (target) with my hard drive (source) ...
>>>
>>> Problem: it is slow -- takes three hours.  To help the
>>> speed issue, I upgraded from USB 2 to USB 3.  Backup went
>>> from 3 hr-15 min to 3 hr-5 min.  It is almost faster
>>> to wipe the stick and rewrite it.
>>>
>
>
> The main question is this: what is the actual write speed of your USB flash media? How about the re-write speed?
  (not the same, obviously - as it requires as erase step).

Kanguru SS3:  Read 105 MB/sec;  78 MB/sec

Don't have the re-write speed.

>
> I ask because I use USB flash media as boot and linux system disks on embedded machines (VME SBCs)
> and I have looked at different USB flash media. Most of them are very slow, actually it is very hard
> to find "fast" USB flash media.

The cheap ones crawl.  If you spend a little:

    El-Cheap-O:
        Read:  3 MB/sec
        Write: 2 MB/sec

    Kanguru SS3:
        Read:  105 MB/sec
        Write: 78 MB/sec

    Kanguru Flash Blu 30:
        Read:  145 MB/sec
        Write: 8GB: 25 MB/sec
               16,32,64GB: 45 MB/sec

    Kingston Data Travler:
        Read:  70 MB/sec
        Write: 30 MB/sec

    Kingston Data Travler Hyper-X 3.0:
        Read:  225 MB/sec
        Write: 135 MB/sec

For comparison:  IntelSSD 530 Series  SATA 3 flash drives:
     Sequential Read:   540 MB/s
     Sequential Write:  490 MB/s

> The common media you get for $10 at Staples is "read 30M/s, write 10M/s" (regardless of USB2 or USB3 interface).

Worse than that!

>  This is probably consistent with the speeds that you see.


I went from 20 MB/sec to 78 MB/sec.  Took 10 minutes off of
three hours.

>
> With some work, you can find media that writes at 20-30M/s, as measured by timing "dd", but drops
> severely when you time "rsync" (must be inefficient at writing small files).
>
> So when you select a brand USB flash drive for your workload, as you run "rsync", watch
> the output of "vmstat 1" (the "bo" column is Mbytes/sec written to disk) and
> the output of "iostat -x 1" - you will see %util pegged at 100% and "svctm" (in msec) running
> in 1-10-20 seconds for "slow media", a little bit smaller for betyter media. For HDDs and SSDs,
> the "svctm" is in low milliseconds. "svctm" is the request service time - time from sending
> a request to the drive and getting the reply from the drive that the request is finished.
>
> Some USB drives advertize high write speeds (not "up to" but actual "will write at" promises),
> you can try those ($$$), but you will probably find that the speed of "rsync" does not reach
> the promised rates because of inefficiency of flashing small files.

sda is the source hard drive; sdc is the target flash drive

$ iostat -x 1

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s 
avgrq-sz avgqu-sz   await  svctm  %util

sda               0.00     0.00   28.00    0.00  7168.00     0.00 
256.00     0.08    2.79   1.39   3.90

sdc               0.00     0.00    0.00  280.00     0.00  7406.00 
26.45     1.34    4.74   3.25  90.90


Couldn't find "svctm".

Does the above tell you anything?


>
> P.S. Another problem with USB flash drives - all brands except for 1 or 2 do not survive
> being used as linux system disks - they brick themselves within days or weeks. I notice
> that they tend to run quite hot, so I suspect they simply overheat and die. Actually,
> I did not find a single USB3 flash drive that survives use as linux system disk yet.
> By luck I have enough Patriot RageXT 8GB and 16GB USB2 flash media, these seem to last.

There is a lot of trash out there.  I refuse to sell Buffalo,
as  have had almost 100% returns on them.


-- 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Computers are like air conditioners.
They malfunction when you open windows
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~