On 2012/02/06 13:37, Chris Schanzle wrote:
> On 02/06/2012 04:02 PM, jdow wrote:
>> (On a heavily loaded system, just when are you going to find 12 gigabytes
>> of fully contiguous storage?)
>
> Probably lots of places on the below 1.0 TB Dell R910 box: :-) [no, not heavily
> loaded at the moment, so your point is still valid, but don't forget times are
> a-changin'!]
>
> # numactl --hardware
> available: 4 nodes (0-3)
> node 0 size: 258511 MB
> node 0 free: 255268 MB
> node 1 size: 258560 MB
> node 1 free: 258467 MB
> node 2 size: 258560 MB
> node 2 free: 258356 MB
> node 3 size: 256540 MB
> node 3 free: 256450 MB
> node distances:
> node 0 1 2 3
> 0: 10 20 20 20
> 1: 20 10 20 20
> 2: 20 20 10 20
> 3: 20 20 20 10

I don't forget. My partner works on machines that dwarf that Dell. (He writes
emulator code for UniSys.) {^_-}

At the time I worked on HardFrame I had a 16 meg machine. So 100k was a small
chunk of memory and usually easy to find, until the machine had been really
active for a week or so.

Fragmentation Happens. And it's simply another major factor on slowdowns.
Each high level transaction involves that transaction time, disk rotation
time, command queuing time, low level transaction time, low level command
queuing, rotational latency, data transfer time, and possible copying time.
(I worked in a zero copy environment, which can be noticeably faster.)

In a nice world the low level latency is a one time per transaction thing.
It needs to be considered because sometimes machines really are heavily
loaded and your values above aren't anywhere to be found.

Once the transaction overhead equals the actual data transfer time you're
not going to go faster than twice your speed no how no way. So REALLY large
buffers are little or no improvement and in case of memory fragmentation
can start costing you time.

Note that some systems have "fairness" built in. That means really large
transfers are automatically broken into smaller chunks to avoid blocking
the machine for extended DMA transfers. I've seen this in driver software,
disk firmware, and even bus level DMA firmware. (That latter was an Amiga
feature in its last days.)

Really, I don't see much need for actual low level transfer sizes to exceed
2gig even with SSD type devices - YET. We're an order of magnitude or more
away from the conditions that will change that overly broad statement.

{^_^}