All Projects → szymonmaszke → Torchfunc

szymonmaszke / Torchfunc

Licence: mit
PyTorch functions and utilities to make your life easier

Programming Languages

python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to Torchfunc

Ramda Adjunct
Ramda Adjunct is the most popular and most comprehensive set of functional utilities for use with Ramda, providing a variety of useful, well tested functions with excellent documentation.
Stars: ✭ 550 (+210.73%)
Mutual labels:  utilities, utils, extensions
Ramda Extension
🤘Utility library for functional JavaScript. With ❤️ to Ramda.
Stars: ✭ 139 (-21.47%)
Mutual labels:  utilities, utils, extensions
Lighthouse Monitor
Investigate performance over your whole company with lighthouse
Stars: ✭ 136 (-23.16%)
Mutual labels:  performance, performance-analysis
Fgprof
🚀 fgprof is a sampling Go profiler that allows you to analyze On-CPU as well as Off-CPU (e.g. I/O) time together.
Stars: ✭ 1,944 (+998.31%)
Mutual labels:  performance, performance-analysis
Deli
Stars: ✭ 148 (-16.38%)
Mutual labels:  performance, performance-analysis
Hotspot
The Linux perf GUI for performance analysis.
Stars: ✭ 2,415 (+1264.41%)
Mutual labels:  performance, performance-analysis
Pg stat kcache
Gather statistics about physical disk access and CPU consumption done by backends.
Stars: ✭ 106 (-40.11%)
Mutual labels:  performance, performance-analysis
Nemetric
前端性能指标的监控,采集以及上报。用于测量第一个dom生成的时间(FP/FCP/LCP)、用户最早可操作时间(fid|tti)和组件的生命周期性能,,网络状况以及资源大小等等。向监控后台报告实际用户测量值。
Stars: ✭ 145 (-18.08%)
Mutual labels:  performance, performance-analysis
Lighthouse Batch
Run Lighthouse analysis over multiple sites in a single command
Stars: ✭ 83 (-53.11%)
Mutual labels:  performance, performance-analysis
Fe Performance Journey
🚵 a Journey of Performance Optimizing in Frontend 🚀
Stars: ✭ 169 (-4.52%)
Mutual labels:  performance, performance-analysis
Heapinspector For Ios
Find memory issues & leaks in your iOS app without instruments
Stars: ✭ 1,819 (+927.68%)
Mutual labels:  performance, performance-analysis
Swissarmylib
Collection of helpful utilities we use in our Unity projects.
Stars: ✭ 154 (-12.99%)
Mutual labels:  utilities, performance
Tracy
C++ frame profiler
Stars: ✭ 3,115 (+1659.89%)
Mutual labels:  performance, performance-analysis
Go Perfbook
Thoughts on Go performance optimization
Stars: ✭ 9,597 (+5322.03%)
Mutual labels:  performance, performance-analysis
Crossplatformdisktest
Windows, macOS and Android storage (HDD, SSD, RAM) speed testing/performance benchmarking app
Stars: ✭ 123 (-30.51%)
Mutual labels:  performance, performance-analysis
Junitperf
⛵️Junit performance rely on junit5 and jdk8+.(java 性能测试框架)
Stars: ✭ 86 (-51.41%)
Mutual labels:  performance, performance-analysis
Speedracer
Collect performance metrics for your library/application.
Stars: ✭ 1,868 (+955.37%)
Mutual labels:  performance, performance-analysis
Jmeter Elasticsearch Backend Listener
JMeter plugin that lets you send sample results to an ElasticSearch engine to enable live monitoring of load tests.
Stars: ✭ 72 (-59.32%)
Mutual labels:  performance, performance-analysis
Schematics Utilities
🛠️ Useful exported utilities for working with Schematics
Stars: ✭ 73 (-58.76%)
Mutual labels:  utilities, utils
Pine Utils
Code Snippets + Tricks & Tips to help Pine Script developers
Stars: ✭ 149 (-15.82%)
Mutual labels:  tips, utils
  • Improve and analyse performance of your neural network (e.g. Tensor Cores compatibility)
  • Record/analyse internal state of torch.nn.Module as data passes through it
  • Do the above based on external conditions (using single Callable to specify it)
  • Day-to-day neural network related duties (model size, seeding, time measurements etc.)
  • Get information about your host operating system, torch.nn.Module device, CUDA capabilities etc.
Version Docs Tests Coverage Style PyPI Python PyTorch Docker Roadmap
Version Documentation Tests Coverage codebeat PyPI Python PyTorch Docker Roadmap

💡 Examples

Check documentation here: https://szymonmaszke.github.io/torchfunc

1. Getting performance tips

  • Get instant performance tips about your module. All problems described by comments will be shown by torchfunc.performance.tips:
class Model(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.convolution = torch.nn.Sequential(
            torch.nn.Conv2d(1, 32, 3),
            torch.nn.ReLU(inplace=True),  # Inplace may harm kernel fusion
            torch.nn.Conv2d(32, 128, 3, groups=32),  # Depthwise is slower in PyTorch
            torch.nn.ReLU(inplace=True),  # Same as before
            torch.nn.Conv2d(128, 250, 3),  # Wrong output size for TensorCores
        )

        self.classifier = torch.nn.Sequential(
            torch.nn.Linear(250, 64),  # Wrong input size for TensorCores
            torch.nn.ReLU(),  # Fine, no info about this layer
            torch.nn.Linear(64, 10),  # Wrong output size for TensorCores
        )

    def forward(self, inputs):
        convolved = torch.nn.AdaptiveAvgPool2d(1)(self.convolution(inputs)).flatten()
        return self.classifier(convolved)

# All you have to do
print(torchfunc.performance.tips(Model()))

2. Seeding, weight freezing and others

  • Seed globaly (including numpy and cuda), freeze weights, check inference time and model size:
# Inb4 MNIST, you can use any module with those functions
model = torch.nn.Linear(784, 10)
torchfunc.seed(0)
frozen = torchfunc.module.freeze(model, bias=False)

with torchfunc.Timer() as timer:
  frozen(torch.randn(32, 784)
  print(timer.checkpoint()) # Time since the beginning
  frozen(torch.randn(128, 784)
  print(timer.checkpoint()) # Since last checkpoint

print(f"Overall time {timer}; Model size: {torchfunc.sizeof(frozen)}")

3. Record torch.nn.Module internal state

  • Record and sum per-layer activation statistics as data passes through network:
# Still MNIST but any module can be put in it's place
model = torch.nn.Sequential(
    torch.nn.Linear(784, 100),
    torch.nn.ReLU(),
    torch.nn.Linear(100, 50),
    torch.nn.ReLU(),
    torch.nn.Linear(50, 10),
)
# Recorder which sums all inputs to layers
recorder = torchfunc.hooks.recorders.ForwardPre(reduction=lambda x, y: x+y)
# Record only for torch.nn.Linear
recorder.children(model, types=(torch.nn.Linear,))
# Train your network normally (or pass data through it)
...
# Activations of all neurons of first layer!
print(recorder[1]) # You can also post-process this data easily with apply

For other examples (and how to use condition), see documentation

🔧 Installation

🐍 pip

Latest release:

pip install --user torchfunc

Nightly:

pip install --user torchfunc-nightly

🐋 Docker

CPU standalone and various versions of GPU enabled images are available at dockerhub.

For CPU quickstart, issue:

docker pull szymonmaszke/torchfunc:18.04

Nightly builds are also available, just prefix tag with nightly_. If you are going for GPU image make sure you have nvidia/docker installed and it's runtime set.

❓ Contributing

If you find any issue or you think some functionality may be useful to others and fits this library, please open new Issue or create Pull Request.

To get an overview of things one can do to help this project, see Roadmap.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].