All Categories → No Category → precision

Top 8 precision open source projects

Arbitrary-precision fixed-point decimal numbers in go

✭ 3,588

go money decimals bignumber precision

Work with large numbers on the client side with high precision.

✭ 41

typescript javascript money operations arithmetic digits decimal bigdecimal digit precision

A Decimal type that plays nicely with literals and Decodable ✨

✭ 18

swift json literals decimal precision double codable jsondecoder

DoubleFloats.jl

math with more good bits

✭ 102

julia performance math accuracy floating-point precision extended-precision doubledouble

mixed-precision-pytorch

Training with FP16 weights in PyTorch

✭ 72

python computer-vision deep-learning overflow pytorch artificial-intelligence artificial-neural-networks gradient weight precision nueral-networks model-weights mixed-precision gradients-underflow precision-training

A tool for debugging and assessing floating point precision and reproducibility.

✭ 51

c python shell fortran C++M4 tool llvm floating-point precision numerical-analysis montecarlo-arithmetic mpfr mixed-precision

Magento2. Rounding Price to Prettier Value for Multi-Currency Stores.

✭ 60

PHP magento-2 price magento2 magento-extension magento2-extension precision round magento2-module currency-format magento-2-extension magento2-extension-free price-format magento2-free-extension magento-2-extension-free magento2-free-modules magento-2-module

cublasHgemm-P100

Code for testing the native float16 matrix multiplication performance on Tesla P100 and V100 GPU based on cublasHgemm

✭ 35

Cuda C++Makefile shell gpu cublas precision gemm half-precision float16 p100 v100

1-8 of 8 precision projects