On 02/12/2017 11:49 PM, Konstantin Olchanski wrote:
> On Sun, Feb 12, 2017 at 04:52:59PM -0800, ToddAndMargo wrote:
>> I am asking as several motherboards I have looked at only
>> have one M.2 slot and raid would need two.
>>
>> I may be misunderstanding here, but I think that if
>> you need two drives, you need a PCIe carrier to
>> put them into instead of the M.2 slot.
>>
>> Maybe I misunderstand for the M.2 slot works.
>>
>
> The best I can tell, on-mobo socket M.2 M-key can be configured
> as either SATA (stealing the connection to one of the sata connectors)
> or as x1, x2 or x4 PCIe. On the mobos that I have this is controlled
> by a BIOS setting (SATA vs PCIe).
>
> In other words, in SATA mode, the M.2 socket is a funny looking SATA
> port (combined data and power), while in PCIe mode, it is just a funny looking
> PCIe x4 slot.
>
> The "x4" part means the connector has provisions for 4 PCIe "lanes" (in and out signal "lvds pairs").
>
> Same as with normal PCIe slots, not all PCIe lanes are necessary connected
> to anything - quite common are x16 slots wired for 8 lanes, x8 slots wired for 4 lanes
> and x4 slots wired for 1 lane. (sometimes you can see how only half of the connector
> has metal pins).
>
> The Intel CPU and chipset ("north bridge") have only so many PCIe lanes available.
> For each motherboard their routing to external devices (network interfaces,
> additional SATA interfaces, etc) and to the PCIe slots (and M.2 slots) can
> be documented or not. Some mobos allow flexible routing, i.e. "x16 and x16
> to 2 PCIe slots, nothing to other slots" or "x8 to every slots", etc.
>
> The actual connectivity can be see in the very small print of "lspci -vvvv" (requires
> root permission) - shown is the device capability (number of PCIe lanes implemented
> on the device) and actual connection (how many PCIe lanes are active).
>
> All this seems to be very poorly understood:
>
> I was recently amused to open a few decomissioned professionaly built servers
> that had x8 PCIe RAID cards installed in "wired for x4" PCIe slots,
> while slots "wired for x8" were empty. Neither the people who build the machines noticed
> this not the people who used them for 10 years ever noticed that the RAID cards
> were in the wrong slots.
>
> So I wonder how many people buy super expensive fancy PCIe x4 SSDs for ultimate performance
> only to have them run at x1 speed because the mobo is not setup or wired correctly.
>
> The bottom line, with your x16 quad M.2 PCIe riser card, you better check that all 4 M.2 SSDs
> are actually connected at x4 speed. (each SSD should show up in "lspci").
>
> Some mobos have better description of PCIe routing than others - read the mobo manual
> very carefully. (read *before* buying, to avoid nasty surprises, like the USB-connected
> network on the RaspberryPi machines).
>
>

Thank you for the education.  Now I understand!

-- 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Serious error.
All shortcuts have disappeared.
Screen. Mind. Both are blank.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~