DLPrimitives
This project aims to provide cross platform OpenCL tools for deep learning and inference.
Today, most of deep learning training is done on NVidia GPUs using closed source CUDA and CUDNN libraries. It is either challenging or virtually impossible to use AMD or Intel GPUs. For example: AMD provides ROCm platform, but there is no support of RDNA platforms yet (more than a year since a release), there is no support of APUs and no support of any operating systems other than Linux.
Goals
- Create an open source, cross platform deep learning primitives library similar to cuDNN or MIOpen that supports multiple GPU architectures.
- Create an inference library with minimal dependencies for efficient inference on any modern GPU, similar to TensorRT or MIGraphX.
- Create minimalistic deep-learning framework as POC of capabilities and performance.
- Integrate to existing large scale deep learning projects like PyTorch, TF, MXNet such that vendor independent open-source OpenCL API will be first class citizen for deep learning.
Please note this is only work in progress - first and preliminary stages.
Initial Framework Integration
Integration with existing frameworks:
-
Pytorch, (almost) out-of-tree OpenCL backend project:
-
Caffe-OpenCL, performance improvements by using dlprimitives:
Integration With ONNX
ONNX Model loading and inference tested on following imagenet networks:
- Pytorch, opsets 9, 11, 13:
alexnet,vgg16,resnet18,resnext50_32x4d,wide_resnet50_2,efficientnet_b0,efficientnet_b4,regnet_y_400mf,squeezenet1_0,mobilenet_v2,densenet121 - MXNet:
vgg11_bn,alexnet,mobilenetv2_0.25,mobilenet0.25,densenet121,resnet18_v1,squeezenet1.0 - Tensorflow, limited initial support, channel first:
resnet50,densenet121
Documentation
Is published under http://dlprimitives.org/docs/
Features Matrix
| Operator | Features | Comment |
|---|---|---|
| Softmax | Softmax, LogSoftmax | |
| NLLLoss | ||
| MSELoss | ||
| SoftmaxWithLoss | ||
| Elementwise | ax+by, max(ax,by), ax*y, broadcasting | |
| Concat | ||
| Slice | ||
| Pooling2D | max, average | |
| GlobalPooling | max, average | 2D only |
| GlobalAvgPool2d | ||
| InnerProduct | ||
| BatchNorm | ||
| Reshape | ||
| Squeeze | ||
| Flatten | ||
| Threshold | ||
| Hardtanh | ||
| Abs | ||
| Parameter | ֹUtility | |
| Reduction | Sum, Mean, Sum Squares, L1 | |
| Convolution2D | GEMM, Winograd, Depthwise Separable | |
| TransposedConvolution2D | GEMM, Winograd, Depthwise Separable | |
| Activation | relu, sigmoid, tanh, relu6 |
Solvers: SGD, Adam
Tested GPUs
| Device | Vendor | Notes |
|---|---|---|
| RX 6600XT | AMD | ROCr |
| RX 560 | AMD | 16cu model, ROCm, PAL, Clover |
| HD 530 | Intel | i5-6600, NEO driver |
| GTX 960 | NVidia | |
| GTX 1080 | NVidia | |
| RTX 2060S | NVidia | |
| MaliG52 MC2 | ARM | performance not optimised yet |
| M1 Max | Apple | 32-core model |
Devices Tested on Windows: AMD RX 560, NVidia GTX 960.
Devices Tested on macOS: Apple M1 Max.
Other features
- Network object for inference
- ONNX to DLPrimitives model converter