SCIENTIFIC-LINUX-USERS Archives

July 2014

SCIENTIFIC-LINUX-USERS@LISTSERV.FNAL.GOV

Options: Use Monospaced Font
Show Text Part by Default
Show All Mail Headers

Message: [<< First] [< Prev] [Next >] [Last >>]
Topic: [<< First] [< Prev] [Next >] [Last >>]
Author: [<< First] [< Prev] [Next >] [Last >>]

Print Reply
Subject:
From:
Paul Robert Marino <[log in to unmask]>
Reply To:
Paul Robert Marino <[log in to unmask]>
Date:
Tue, 15 Jul 2014 13:44:58 -0400
Content-Type:
text/plain
Parts/Attachments:
text/plain (93 lines)
I have a question about your test did you unmount the stick between
runs of rsync. if not you may have already had all of the information
about the filesystem cached in memory instead of having to search the
FAT table for information. this could have a huge effect on the speed
of an update.


also I think it also is in the reformatting the file system may have helped.
I did a little testing on the st_block vs st_size discussion yesterday
what I found was interesting.

Older versions of Linux (2.4 kernel, yes I have a few non-networked
vintage boxes running for specific tasks like operating older robots
which require special kernel modules) displayed the behavior I
mentioned about the file size in stat but newer versions of Linux did
not.
I also know for a fact MS Windows acts this way because I ran into it
last year on Windows 2008R2. I hit it when a tool I wrote to do a
quick check to see if certain tape restores were working properly I
found it worked perfectly on Linux but in MS Windows It had problems
because it would return different byte sizes in st_size based on the
filesystem the file was restored too, but if i copied the file
afterword to a Linux box I always get the right size.

So at this point its just a theory but its making me wonder if there
is a difference in the formatting at some low level which doesn't
break compatiblity but effects the value returned by st_size.
Microsoft has a history of do strange things to filesystems in the
name of speeding up writes that in the end hurts them in other ways,
just consider how often MS Windows boxes need to be defragmented as an
example.

further more I have a question about your test did you unmount the
stick between runs of rsync. if not you may have already had all of
the information about the filesystem cached in memory instead of
having to search the FAT table for information. this could have a huge
effect on the speed of an update.

In any case when I get time I will investigate it more thoroughly.


On Tue, Jul 15, 2014 at 2:40 AM, Kay Diederichs
<[log in to unmask]> wrote:
> Am 15.07.14 07:08, schrieb SCIENTIFIC-LINUX-USERS automatic digest system:
>>>
>>> I wiped the stick (set all the charges to zero) and started over.
>>> I am using
>>>
>>> rsync -rv --delete --modify-window=1 --times --inplace \
>>>     $MyCDsSource/Linux $MyCDsTarget/.; sync; sync
>>>
>>> I will have results in a few hours.
>>>
>>> Then I am going to do it again with no changes to
>>> see change happens.
>>>
>>>
>>
>>
>> Uh, guys.  Something is weird.
>>
>> I zero'ed out the stick
>>    dd bs=4096  if=/dev/zero of=/sdc
>>
>> and reformatted to one primary partition of type
>> "c" (W95 FAT32).
>>
>> Ran a full sync with
>> rsync -rv --delete --modify-window=1 --times --inplace \
>>      $MyCDsSource/Linux $MyCDsTarget/.; sync; sync
>> 9 hrs - 9 min
>>
>> They I reran the sync.  Less than one second.
>> Did it again to make sure.  Again, less than one
>> second again.
>>
>> Moved two files and added one.  All changes were found.
>> Again, less than a second.
>>
>> Wow.  Can this really be correct?
>>
>> -T
>
> yes, this is the expected behaviour.
>
> Most of the discussion of this thread was not to the point of the
> problem. The only real problem was that you did not use the -t option
> (as in "rsync -rtv"  or "rsync -r --times"). You could use a USB2 stick
> and it would still be very fast.
>
> Kay
>

ATOM RSS1 RSS2