Wiki . Report Bug · Request Feature

IOMemory-VSL4

This is an "unsupported" updated, and cleaned up version of the original driver source for newer generation FusionIO cards. It comes with no warranty, it may cause DATA LOSS or CORRUPTION.

IOMemory-VSL(3)

For the iomemory-vsl driver please go to the iomemory-vsl repo.

How to Identify your Fusion-io Card (including VSL version)

Please refer to the Fusion-io and OEM Part Numbers Matrix.

Background

Driver support for FusionIO cards has been lagging behind kernel releases, effectively making these cards an expensive paperweight when running a distribution like Ubuntu / Arch / Fedora / ProxMox which all supply newer kernels than supported.

Current version

The current driver version is derived from iomemory-vsl-4.3.7.1205, and has gone through rigorous rewriting and cleaning of redundant, unused, and old code. This driver is aimed to only support Linux kernels from 5.0 and upwards. The v4.20.0 branch is provided for convencience and is not actively maintained.

Releases

We've abandoned the notion of releases, Generally main should be checked out. main is completely backwards compatible for all 5. The latest working tested kernel is 5.16.14.

Historically releases were tagged, and were be checked out by their tag. The release tags follow Linux Kernel versions. E.g. v5.12.1 (Boop Noodle) will work on all 5.x kernels that are 5.12 and lower, but is not guaranteed to work on 5.13. v4.20.2 - Big Ole Nope Rope supports most kernels that pre-date 5.0. Again this way of following kernel releases has been abandoned. Please follow main.

Important Note for newer Linux Kernels

Starting with Linux kernel 5.4.0, significant changes to the kernel were made that require additional boot time kernel flags for this driver to work. These affect AMD CPUs starting with 5.4.0, and Intel CPUs after about kernel 5.8.0.

Add the following to your /etc/default/grub by looking for GRUB_CMDLINE_LINUX_DEFAULT="" and adding additional parameters inside the quotes. For AMD systems:

amd_iommu=on iommu=pt

For Intel system:

iommu=pt

Example: GRUB_CMDLINE_LINUX_DEFAULT="quiet iommu=pt"

Supported Hardware

Here's a not so exhaustive list of iomemory cards. I have only tested the 3.2TB card, and was able to crossflash back to OEM. The rest below all seem to be SX350s or PX600s. While they should all work, we don't have any PX or SX300 cards to test with.

Important notes!!!

At this moment the driver has been tested over time with different kernel version. Tests are run on an LVM volume with an ext4 filesystem. Workload testing is done with VM's, Containers, FIO and normal desktop usage.

Building

There are several ways to build and package the module.

Note! For many systems, the best option is to use DKMS, using the DKMS instructions below. If you prefer to build the module directly, or to create a dpkg or rpm package, you can proceed with these options:

From Source

git clone https://github.com/snuf/iomemory-vsl4
cd iomemory-vsl4/
git checkout <release-tag>
make module
cd root/usr/src/iomemory-vsl4-4.3.7
sudo insmod iomemory-vsl4.ko

.deb Ubuntu / Debian

git clone https://github.com/snuf/iomemory-vsl4
cd iomemory-vsl4/
git checkout <release-tag>
make dpkg

.rpm CentOS / RHEL

git clone https://github.com/snuf/iomemory-vsl4
cd iomemory-vsl4/
git checkout <release-tag>
make rpm

Installation

Installation can be done with created packages, DKMS or other options described in the original README.

DKMS

Dynamic Kernel Module Support automates away the requirement of having to repackage the kernel module with every kernel and headers update that takes place on the system.

Try building from main first as it works with most modern kernels up to about 5.14:

git clone https://github.com/snuf/iomemory-vsl4
cd iomemory-vsl4/
make dkms

If you know you need to build a specific branch based on a specific recommendation, use:

git clone https://github.com/snuf/iomemory-vsl4
cd iomemory-vsl4/
git checkout <release-tag>
make dkms

Utils

With fio-utils installed you should see the following kind of...,:

.deb https://www.dropbox.com/s/d18dokktt0g2aau/fio-util_4.3.7.1205-1.0_amd64.deb
.rpm https://www.dropbox.com/s/ba2vcxhmf9p2pb2/fio-util-4.3.7.1205-1.el7.x86_64.rpm

snuf@scipio:~/Documents/iodrive4/fio-util-4/usr/bin$ sudo ./fio-status -a

Note: 2 unmanaged ioMemory devices found requiring a v3.x driver.
   Install the driver package to see device information.
   Note that only one driver package can be installed at a time.

Found 1 VSL driver package:
   4.3.6 build 1173 Driver: not loaded

Found 1 ioMemory device in this system

Adapter: ioMono  (driver 4.3.6)
	ioMemory SX300-1300, Product Number:F13-004-1300-CS-0001, SN:1446G1531, FIO SN:1446G1531
	ioMemory Adapter Controller, PN:PA006002103
	Product UUID:d3e0a40c-7e14-5264-842f-316d3f353492
	PCIe Power limit threshold: Disabled
	PCIe slot available power: 0.24W
	PCIe negotiated link: 8 lanes at 5.0 Gt/sec each, 4000.00 MBytes/sec total
	Connected ioMemory modules:
	  08:00.0:	Product Number:F13-004-1300-CS-0001, SN:1446G1531

08:00.0	ioMemory Adapter Controller, Product Number:F13-004-1300-CS-0001, SN:1446G1531
	ioMemory Adapter Controller, PN:PA006002103
	Microcode Versions: App:0.0.44.0
	PCI:08:00.0
	Vendor:1aed, Device:3002, Sub vendor:1aed, Sub device:3002
	Firmware v8.9.5, rev 20160412 Public
	UEFI Option ROM v4.2.5, rev 1266 Disabled
	PCIe slot available power: 0.24W
	PCIe negotiated link: 8 lanes at 5.0 Gt/sec each, 4000.00 MBytes/sec total
	Internal temperature: 63.49 degC, max 79.24 degC
	Internal voltage: avg 1.01V, max 1.02V
	Aux voltage: avg 1.80V, max 1.83V

How to Get Help

• Open an issue in this Github repo
• Join our Discord server at https://discord.gg/VxDvVy3EdS

Acknowledgements

The support and maintenance of this driver is made possible by the people that actively contribute or contributed to its code base or by supporting the project in other ways.

Obviously all the regulars on the Discord channel, notably @bplein, @AcquaCow and @Tear.

Oh yes and the folks that were at Fusion IO for creating this product that was way ahead of its time and delivering the integration shim sources with the driver core.

Resources

Tools and resources often used to figure out what changed, and why things are not working as they are supposed to.

Other notes

Installing fio-util, fio-common, fio-preinstall and fio-sysvinit is recommended.