All Projects → Sergio0694 →

Sergio0694 /

Licence: gpl-3.0
A TensorFlow-inspired neural network library built from scratch in C# 7.3 for .NET Standard 2.0, with GPU support through cuDNN

Programming Languages

926 projects

Projects that are alternatives of or similar to

Image classifier
CNN image classifier implemented in Keras Notebook 🖼️.
Stars: ✭ 139 (-64.54%)
Mutual labels:  ai, convolutional-neural-networks, cnn
Pytorch Image Classification
Tutorials on how to implement a few key architectures for image classification using PyTorch and TorchVision.
Stars: ✭ 272 (-30.61%)
Mutual labels:  convolutional-neural-networks, cnn
Go Cyber
Your 🔵 Superintelligence
Stars: ✭ 270 (-31.12%)
Mutual labels:  ai, cuda
Deep Learning Computer Vision Algorithms for Real-World Use
Stars: ✭ 326 (-16.84%)
Mutual labels:  ai, convolutional-neural-networks
A brand logo detection system using tensorflow object detection API.
Stars: ✭ 388 (-1.02%)
Mutual labels:  convolutional-neural-networks, cnn
Brain Simulator is a platform for visual prototyping of artificial intelligence architectures.
Stars: ✭ 262 (-33.16%)
Mutual labels:  ai, cuda
Komputation is a neural network framework for the Java Virtual Machine written in Kotlin and CUDA C.
Stars: ✭ 295 (-24.74%)
Mutual labels:  convolutional-neural-networks, cuda
An extension to the Krypton Toolkit suite of controls for .NET framework 4.7
Stars: ✭ 51 (-86.99%)
Mutual labels:  visual-studio, net-framework
Personality Detection
Implementation of a hierarchical CNN based model to detect Big Five personality traits
Stars: ✭ 338 (-13.78%)
Mutual labels:  convolutional-neural-networks, cnn
High-performance Bayesian Data Analysis on the GPU in Clojure
Stars: ✭ 342 (-12.76%)
Mutual labels:  gpu-acceleration, cuda
This repository contains Ipython notebooks and datasets for the data analytics youtube tutorials on The Semicolon.
Stars: ✭ 345 (-11.99%)
Mutual labels:  convolutional-neural-networks, gradient-descent
Experimental parallel compression algorithm
Stars: ✭ 21 (-94.64%)
Mutual labels:  cuda, gpu-acceleration
CUDA bindings for Ruby
Stars: ✭ 57 (-85.46%)
Mutual labels:  cuda, gpu-acceleration
Pytorch Saltnet
Kaggle | 9th place single model solution for TGS Salt Identification Challenge
Stars: ✭ 270 (-31.12%)
Mutual labels:  convolutional-neural-networks, cnn
An update to Component factory's krypton toolkit to support .NET Framework 4.6.2 - 4.8.1 to .NET Core/.NET
Stars: ✭ 194 (-50.51%)
Mutual labels:  visual-studio, net-framework
Pytorch Srgan
A modern PyTorch implementation of SRGAN
Stars: ✭ 289 (-26.28%)
Mutual labels:  convolutional-neural-networks, cnn
A light-weight, power efficient, and general purpose convolutional neural network
Stars: ✭ 377 (-3.83%)
Mutual labels:  convolutional-neural-networks, cnn
Interpretable ML package 🔍 for concise, transparent, and accurate predictive modeling (sklearn-compatible).
Stars: ✭ 194 (-50.51%)
Mutual labels:  ai, supervised-learning
Gordon cnn
A small convolution neural network deep learning framework implemented in c++.
Stars: ✭ 241 (-38.52%)
Mutual labels:  ai, cnn
Ios 10 Sampler
Code examples for new APIs of iOS 10.
Stars: ✭ 3,341 (+752.3%)
Mutual labels:  convolutional-neural-networks, cnn

Get it from NuGet ScuSharp STACK

NuGet NuGet AppVeyor AppVeyor tests Twitter Follow

What is it?

NeuralNetwork.NET is a .NET Standard 2.0 library that implements sequential and computation graph neural networks with customizable layers, built from scratch with C#. It provides simple APIs designed for quick prototyping to define and train models using stochastic gradient descent, as well as methods to save/load a network model and its metadata and more. The library also exposes CUDA-accelerated layers with more advanced features that leverage the GPU and the cuDNN toolkit to greatly increase the performances when training or using a neural network.

DISCLAIMER: this library is provided as is, and it's no longer being actively maintained. NeuralNetwork.NET was developed during a university course and it's not meant to be a replacement for other well known machine learning frameworks. If you're looking for a machine learning library for .NET to use in production, I recommend trying out ML.NET or alternatively TensorFlow.NET.

Table of Contents

Installing from NuGet

To install NeuralNetwork.NET, run the following command in the Package Manager Console

Install-Package NeuralNetwork.NET

More details available here.

Quick start

The NeuralNetwork.NET library exposes easy to use classes and methods to create a new neural network, prepare the datasets to use and train the network. These APIs are designed for rapid prototyping, and this section provides an overview of the required steps to get started.

Supervised learning

The first step is to create a custom network structure. Here is an example with a sequential network (a stack of layers):

INeuralNetwork network = NetworkManager.NewSequential(TensorInfo.Image<Alpha8>(28, 28),
    NetworkLayers.Convolutional((5, 5), 20, ActivationType.Identity),
    NetworkLayers.Convolutional((3, 3), 40, ActivationType.Identity),
    NetworkLayers.FullyConnected(125, ActivationType.LeakyReLU),
    NetworkLayers.FullyConnected(64, ActivationType.LeakyReLU),

The next step is to prepare the datasets to use, through the APIs in the DatasetLoader class:

// A training dataset with a batch size of 100
IEnumerable<(float[] x, float[] u)> data = ... // Your own dataset parsing routine
ITrainingDataset dataset = DatasetLoader.Training(data, 100);

// An optional test dataset with a callback to monitor the progress
ITestDataset test = DatasetLoader.Test(..., p =>
    Console.WriteLine($"Epoch {p.Iteration}, cost: {p.Cost}, accuracy: {p.Accuracy}"); // Progress report

Training a neural network is pretty straightforward - just use the methods in the NetworkManager class:

// Train the network using Adadelta and 0.5 dropout probability
TrainingSessionResult result = NetworkManager.TrainNetwork(
    network,                                // The network instance to train
    dataset,                                // The ITrainingDataset instance   
    TrainingAlgorithms.AdaDelta(),          // The training algorithm to use
    60,                                     // The expected number of training epochs to run
    0.5f,                                   // Dropout probability
    p => ...,                               // Optional training epoch progress callback
    null,                                   // Optional callback to monitor the training dataset accuracy
    null,                                   // Optional validation dataset
    test,                                   // Test dataset
    token);                                 // Cancellation token for the training

Note: the NetworkManager methods are also available as asynchronous APIs.

GPU acceleration

When running on a supported framework (.NET Framework, Xamarin or Mono), it is possible to use a different implementation of the available layers that leverages the cuDNN toolkit and parallelizes most of the work on the available CUDA-enabled GPU. To do that, just use the layers from the CuDnnNetworkLayers class when creating a network.

Some of the cuDNN-powered layers support additional options than the default layers. Here's an example:

// A cuDNN convolutional layer, with custom mode, padding and stride
LayerFactory convolutional = CuDnnNetworkLayers.Convolutional(
    ConvolutionInfo.New(ConvolutionMode.CrossCorrelation, 3, 3, 2, 2),
    (7, 7), 20, ActivationType.ReLU);
// An inception module, from the design of the GoogLeNet network
LayerFactory inception = CuDnnNetworkLayers.Inception(InceptionInfo.New(
    10,     // 1x1 convolution kernels
    20, 10, // 1x1 + 3x3 convolution pipeline kernels
    20, 10, // 1x1 + 5x5 convolution pipeline kernels
    PoolingMode.AverageExcludingPadding, 10)); // Pooling mode and 1x1 convolution kernels

These LayerFactory instances can be used to create a new network just like in the CPU example.

NOTE: in order to use this feature, the CUDA and cuDNN toolkits must be installed on the current system, a CUDA-enabled nVidia GeForce/Quadro GPU must be available and the Alea NuGet package must be installed in the application using the NeuralNetwork.NET library as well. Additional info are available here.

Computation graphs

Some complex network structures, like residual networks or inception modules , cannot be expressed as a simple sequential network structure: this is where computation graph networks come into play. Instead of forwarding the inputs through a linear stack of layers, a computation graph has a specific spatial structure that allows different nodes to be connected together. For example, it is possible to channel data through different parallel pipelines that are merged later on in the graph, or to have auxiliary classifiers that contribute to the gradient backpropagation during the training phase.

Computation graph networks are created using the NetworkManager.NewGraph API, here's an example:

INeuralNetwork network = NetworkManager.NewGraph(TensorInfo.Image<Rgb24>(32,32), root =>
    var conv1 = root.Layer(CuDnnNetworkLayers.Convolutional((5, 5), 20, ActivationType.Identity));
    var pool1 = conv1.Layer(CuDnnNetworkLayers.Pooling(ActivationType.ReLU));

    var conv2 = pool1.Pipeline(
        CuDnnNetworkLayers.Convolutional((1, 1), 20, ActivationType.ReLU),
        CuDnnNetworkLayers.Convolutional(ConvolutionInfo.Same(), (5, 5), 40, ActivationType.ReLU),
        CuDnnNetworkLayers.Convolutional((1, 1), 20, ActivationType.ReLU));
    var sum = conv2 + pool1;

    var fc1 = sum.Layer(CuDnnNetworkLayers.FullyConnected(250, ActivationType.LeCunTanh));
    var fc2 = fc1.Layer(CuDnnNetworkLayers.FullyConnected(125, ActivationType.LeCunTanh));
    _ = fc2.Layer(CuDnnNetworkLayers.Softmax(10));

Library settings

NeuralNetwork.NET provides various shared settings that are available through the NetworkSettings class. This class acts as a container to quickly check and modify any setting at any time, and these settings will influence the behavior of any existing INeuralNetwork instance and the library in general.

For example, it is possible to customize the criteria used by the networks to check their performance during training

NetworkSettings.AccuracyTester = AccuracyTesters.Argmax();       // The default mode (mutually-exclusive classes)

// Other testers are available too
NetworkSettings.AccuracyTester = AccuracyTesters.Threshold();    // Useful for overlapping classes
NetworkSettings.AccuracyTester = AccuracyTesters.Distance(0.2f); // Distance between results and expected outputs

When using CUDA-powered networks, sometimes the GPU in use might not be able to process the whole test or validation datasets in a single pass, which is the default behavior (these datasets are not divided into batches). To avoid memory issues, it is possible to modify this behavior:

NetworkSettings.MaximumBatchSize = 400;   // This will apply to any test or validation dataset

Serialization and deserialization

The INeuralNetwork interface exposes a Save method that can be used to serialize any network at any given time. In order to get a new network instance from a saved file or stream, just use the NetworkLoader.TryLoad method.

As multiple layer types have different implementations across the available libraries, you can specify the layer providers to use when loading a saved network. For example, here's how to load a network using the cuDNN layers, when possible:

FileInfo file = new FileInfo(@"C:\...\MySavedNetwork.nnet");
INeuralNetwork network = NetworkLoader.TryLoad(file, ExecutionModePreference.Cuda);

Note: the ExecutionModePreference option indicates the desired type of layers to deserialize whenever possible. For example, using ExecutionModePreference.Cpu, the loaded network will only have CPU-powered layers, if supported.

There's also an additional SaveMetadataAsJson method to export the metadata of an INeuralNetwork instance.

Built-in datasets

The NeuralNetworkNET.Datasets namespace includes static classes to quickly load a popular dataset and get an IDataset instance ready to use with a new neural network. As an example, here's how to get the MNIST dataset:

ITrainingDataset trainingData = await Mnist.GetTrainingDatasetAsync(400); // Batches of 400 samples
ITestDataset testData = await Mnist.GetTestDatasetAsync(p => ... /* Optional callback */);

Each API in this namespace also supports an optional CancellationToken to stop the dataset loading, as the source data is downloaded from the internet and can take some time to be available, depending on the dataset being used.


The NeuralNetwork.NET library requires .NET Standard 2.0 support, so it is available for applications targeting:

  • .NET Framework >= 4.6.1
  • .NET Core >= 2.0
  • UWP (from SDK 10.0.16299)
  • Mono >= 5.4
  • Xamarin.iOS 10.14, Xamarin.Mac 3.8, Xamarin.Android 8.0

In addition to the frameworks above, you need an IDE with C# 7.3 support to compile the library on your PC.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].