hi sven,

> i try at the moment to figure out why our QDR Infiniband is not
> as fast as it should be. So I realized that we use 2k mtu size, and
> tried to change this to 4k. But don't now how.
how fast are you expecting to get? (and what are you measuring)
>
> enviroment:
> mellanox connectX-3 cards in interlagos machine with sl6.3 64 bit
> (RHEL/SL Ofed stack)
interlagos as in AMD RD890 IO bridge that only has max pcie payload of 
128byte and thus can only reach 75% of a pcie-2 link (which will result 
in approx 75% of the QDR link)
(google pcie payload plxtech, theres a pdf soemwhere with a self 
explaining plot)

very nice of amd not to have pcie-3 chips ...


stijn
> mellanox sx6036 switch (4k mtu enabled)
>
> in the mellanox docu i found something about a module parameter
> "set_4k_mtu" but this don't work with SL6.3 (unknown parameter)
>
> this also don't work, because there is no "mlx4_port1_mtu"
> echo 4096 > /sys/class/infiniband/mlx4_0/device/mlx4_port1_mtu
>
> is it possible that the sl mellanox module don't support 4k mtu?
>
> best regards
>
> sven
>
>