On Sun, Feb 12, 2017 at 04:52:59PM -0800, ToddAndMargo wrote:
>
> I am asking as several motherboards I have looked at only
> have one M.2 slot and raid would need two.
> 
> I may be misunderstanding here, but I think that if
> you need two drives, you need a PCIe carrier to
> put them into instead of the M.2 slot.
> 
> Maybe I misunderstand for the M.2 slot works.
> 


The best I can tell, on-mobo socket M.2 M-key can be configured
as either SATA (stealing the connection to one of the sata connectors)
or as x1, x2 or x4 PCIe. On the mobos that I have this is controlled
by a BIOS setting (SATA vs PCIe).

In other words, in SATA mode, the M.2 socket is a funny looking SATA
port (combined data and power), while in PCIe mode, it is just a funny looking
PCIe x4 slot.

The "x4" part means the connector has provisions for 4 PCIe "lanes" (in and out signal "lvds pairs").

Same as with normal PCIe slots, not all PCIe lanes are necessary connected
to anything - quite common are x16 slots wired for 8 lanes, x8 slots wired for 4 lanes
and x4 slots wired for 1 lane. (sometimes you can see how only half of the connector
has metal pins).

The Intel CPU and chipset ("north bridge") have only so many PCIe lanes available.
For each motherboard their routing to external devices (network interfaces,
additional SATA interfaces, etc) and to the PCIe slots (and M.2 slots) can
be documented or not. Some mobos allow flexible routing, i.e. "x16 and x16
to 2 PCIe slots, nothing to other slots" or "x8 to every slots", etc.

The actual connectivity can be see in the very small print of "lspci -vvvv" (requires
root permission) - shown is the device capability (number of PCIe lanes implemented
on the device) and actual connection (how many PCIe lanes are active).

All this seems to be very poorly understood:

I was recently amused to open a few decomissioned professionaly built servers
that had x8 PCIe RAID cards installed in "wired for x4" PCIe slots,
while slots "wired for x8" were empty. Neither the people who build the machines noticed
this not the people who used them for 10 years ever noticed that the RAID cards
were in the wrong slots.

So I wonder how many people buy super expensive fancy PCIe x4 SSDs for ultimate performance
only to have them run at x1 speed because the mobo is not setup or wired correctly.

The bottom line, with your x16 quad M.2 PCIe riser card, you better check that all 4 M.2 SSDs
are actually connected at x4 speed. (each SSD should show up in "lspci").

Some mobos have better description of PCIe routing than others - read the mobo manual
very carefully. (read *before* buying, to avoid nasty surprises, like the USB-connected
network on the RaspberryPi machines).


-- 
Konstantin Olchanski
Data Acquisition Systems: The Bytes Must Flow!
Email: olchansk-at-triumf-dot-ca
Snail mail: 4004 Wesbrook Mall, TRIUMF, Vancouver, B.C., V6T 2A3, Canada