public GenTensor<T> GetSubtensor(params int[] indecies)

Allows to get a subtensor with SHARED data (so that any changes to intial tensor or the subtensor will be reflected in both).

For example, Subtensor of a matrix is a vector (row).

Works for O(1)

public void SetSubtensor(GenTensor<T, TWrapper> sub, params int[] indecies);

Allows to set a subtensor by forwarding all elements from sub to this. Override ConstantsAndFunctions<T, TWrapper>.Forward to enable it.

Works for O(V)

public void Transpose(int axis1, int axis2);
public void TransposeMatrix();

Swaps axis1 and axis2 in this. TransposeMatrix swaps the last two axes.

Works for O(1)

public static GenTensor<T, TWrapper> Concat(GenTensor<T, TWrapper> a, GenTensor<T, TWrapper> b);

Conatenates two tensors by their first axis. For example, concatenation of two tensors of shape [4 x 5 x 6] and [7 x 5 x 6] is a tensor of shape [11 x 5 x 6].

Works for O(N)

public static GenTensor<T, TWrapper> Stack(params GenTensor<T, TWrapper>[] elements);

Unites all same-shape elements into one tensor with 1 dimension more. For example, if t1, t2, and t3 are of shape [2 x 5], Stack(t1, t2, t3) will return a tensor of shape [3 x 2 x 5]

Works for O(V)

public GenTensor<T, TWrapper> Slice(int leftIncluding, int rightExcluding);

Slices this into another tensor with data-sharing. Syntax and use is similar to python's numpy:

v = myTensor[2:3]

is the same as

var v = myTensor.Slice(2, 3);

Works for O(N)

public byte[] Serialize();
public static GenTensor<T, TWrapper> Deserialize(byte[] data);

Non-static Serialize() serializes given tensor into a byte array. It will call TWrapper.Serialize(T a). Use this to store your tensor somewhere in file or database.

Static Deserialize() provides the opposite to Serialize operation. It takes a byte array as an argument and returns the same tensor as it was before serialization. As it calls TWrapper.Deserialize(byte[] data), it should be of the exact same type as it was before serialization.

public static T VectorDotProduct(GenTensor<T, TWrapper> a, GenTensor<T, TWrapper> b);
public static GenTensor<T, TWrapper> TensorVectorDotProduct(GenTensor<T, TWrapper> a, GenTensor<T, TWrapper> b);

Counts dot product of two same-shaped vectors. For example, you have v1 = {2, 3, 4}, v2 = {5, 6, 7}, then VectorDotProduct(v1, v2) = 2 * 5 + 3 * 6 + 4 * 7 = 56.

Works for O(V)

public static GenTensor<T, TWrapper> VectorCrossProduct(GenTensor<T, TWrapper> a, GenTensor<T, TWrapper> b);
public static GenTensor<T, TWrapper> TensorVectorCrossProduct(GenTensor<T, TWrapper> a, GenTensor<T, TWrapper> b);

Counts cross product of two same-shaped vectors. The resulting vector is such one that is perdendicular to all of the arguments.

Works for O(V)

public void RowMultiply(int rowId, T coef);  
public void RowAdd(int dstRowId, int srcRowId, T coef);
public void RowSubtract(int dstRowId, int srcRowId, T coef);
public void RowSwap(int row1Id, int row2Id)

Performs elementary row operations on a matrix. this must be a matrix (otherwise, an exception is thrown). The operations modify the matrix (without making copies).

Works for O(N)

public GenTensor<T, TWrapper> RowEchelonFormSimple();
public GenTensor<T, TWrapper> RowEchelonFormSafeDivision();
public GenTensor<T, TWrapper> RowEchelonFormLeadingOnesSimple();
public GenTensor<T, TWrapper> RowEchelonFormLeadingOnesSafeDivision();
public GenTensor<T, TWrapper> ReducedRowEchelonFormSimple();
public GenTensor<T, TWrapper> ReducedRowEchelonFormSafeDivision();

Finds the requested form of a matrix. Due to different definitions of row echelon forms, we provide both forms: the one with leading ones and the one with arbitrary leading elements. There is also reduced row echelon form.

They are implemented via Gaussian elimination. Every of these three methods has two variants: Simple and SafeDivision. In SafeDivision it will postpone the actual division of numbers up to the end of the algorithm, while Simple will perform division immediately. SafeDivision provides a higher precision, but it requires a high upper/lower bound of a number, because while computing even simple matrices, it might exceed billions.

Works for O(N^3)

public static GenTensor<T, TWrapper> MatrixMultiply(GenTensor<T, TWrapper> a, GenTensor<T, TWrapper> b, Threading threading = Threading.Single);
public static GenTensor<T, TWrapper> TensorMatrixMultiply(GenTensor<T, TWrapper> a, GenTensor<T, TWrapper> b, Threading threading = Threading.Single);

Performs matrix multiplication operation of two matrices. One's height should be the same as Another's width.

MatrixMultiply for Threading.Multi performs parallel computations over the first axis, TensorMatrixMultiplyParallel performs parallel computations over matrices.

Works for O(N^3)

public T DeterminantLaplace();
public T DeterminantGaussianSafeDivision();
public T DeterminantGaussianSimple();

Finds determinant of a square matrix. DeterminantLaplace is the simplest and true way to find determinant, but it is as slow as O(N!). Guassian elimination works for O(N^3) but might cause precision loss when dividing. If your type does not lose precision when being divided, use DeterminantGaussianSimple. Otherwise, for example, for int, use DeterminantGaussianSafeDivision.

Works for O(N!), O(N^3)

public void InvertMatrix();
public void TensorMatrixInvert();

Inverts A to B such that A * B = I where I is identity matrix.

Works for O(N^4)

public GenTensor<T, TWrapper> Adjoint();

Returns an adjugate matrix.

Works for O(N^4)

public static GenTensor<T, TWrapper> MatrixDivide(GenTensor<T, TWrapper> a, GenTensor<T, TWrapper> b);
public static GenTensor<T, TWrapper> TensorMatrixDivide(GenTensor<T, TWrapper> a, GenTensor<T, TWrapper> b)

Of A, B returns such C that A == C * B.

Works for O(N^4)

public static GenTensor<T, TWrapper> MatrixPower(GenTensor<T, TWrapper> m, int power);
public static GenTensor<T, TWrapper> TensorMatrixPower(GenTensor<T, TWrapper> m, int power);

Finds the power of a matrix.

Works for O(log(N) * N^3)

public (GenTensor<T, TWrapper> p, GenTensor<T, TWrapper> l, GenTensor<T, TWrapper> u) PluDecomposition()

Find PLU decomposition: matrices P, L, U such that for original matrix A: PA = LU.

Works for O(n^3)

public (GenTensor<T, TWrapper>, GenTensor<T, TWrapper>) LuDecomposition()   

Find LU decomposition: matrices L, U such that for original matrix A: A = LU.

Works for O(n^3)

public static GenTensor<T, TWrapper> PiecewiseAdd(GenTensor<T, TWrapper> a, GenTensor<T, TWrapper> b, Threading threading = Threading.Single);
public static GenTensor<T, TWrapper> PiecewiseSubtract(GenTensor<T, TWrapper> a, GenTensor<T, TWrapper> b, Threading threading = Threading.Single);
public static GenTensor<T, TWrapper> PiecewiseMultiply(GenTensor<T, TWrapper> a, GenTensor<T, TWrapper> b, Threading threading = Threading.Single);
public static GenTensor<T, TWrapper> PiecewiseDivide(GenTensor<T, TWrapper> a, GenTensor<T, TWrapper> b, Threading threading = Threading.Single);

Returns a tensor of an operation being applied to every matching pair so that Add is. Those with Parallel in its name are ran on multiple cores (via Parallel.For).

result[i, j, k...] = a[i, j, k...] + b[i, j, k...]

Works for O(V)

public static GenTensor<T, TWrapper> PiecewiseAdd(GenTensor<T, TWrapper> a, T b, Threading threading = Threading.Single);
public static GenTensor<T, TWrapper> PiecewiseSubtract(GenTensor<T, TWrapper> a, T b, Threading threading = Threading.Single);
public static GenTensor<T, TWrapper> PiecewiseSubtract(T a, GenTensor<T, TWrapper> b, Threading threading = Threading.Single);
public static GenTensor<T, TWrapper> PiecewiseMultiply(GenTensor<T, TWrapper> a, T b, Threading threading = Threading.Single);
public static GenTensor<T, TWrapper> PiecewiseDivide(GenTensor<T, TWrapper> a, T b, Threading threading = Threading.Single);
public static GenTensor<T, TWrapper> PiecewiseDivide(T a, GenTensor<T, TWrapper> b, Threading threading = Threading.Single);

Performs an operation on each of tensor's element and forwards them to the result You can set threading: Threading.Auto to let GT to determine whether it is better to use multi-threading or keep working at one core, or threading: Threading.Multi if you need multi-threading.

Works for O(V)

Towel is not a competitor to GT, those are completely different libraries. But since it is the only library that provides matrices for arbitrary type (not only numeric), we have to compare GT to it. We take Towel as of 08.08.2020 from its official repository. Yet some features are not presented in Towel, such as multi-threading.

Multithreading is a useful tool if you want to make computations faster. We do not support GPU computations and never will because our aim to keep GenericTensor supporting custom type, while GPU only works with fixed types like int, float, and a few others.

However, even on CPU it is sometimes better to keep single-core computations. So here we find out when it is better to keep single and where it is better to switch to multi-core. Here we provide graphs for multiplication of matrices and piecewise product for tensors of different sizes in those two modes (Threading.Single and Threading.Multi). Y-axis shows number of microseconds spent on one operation.

Matrix multiplication

Piecewise product