PyTorch 1.7 now obtainable with new APIs, profiling, and benchmarking instruments
The most recent model of the open-source machine studying library PyTorch is now obtainable. PyTorch 1.7 introduces new APIs, help for CUDA 11, updates to profiling and efficiency for RPC, TorchScript, and Stack tracers.
New entrance finish APIs embody torch.fft, which is a module for implementing FFT-related capabilities; C++ help for nn.transformer module abstraction from the C++ frontend; and torch.set_deterministic, which may direct operators to pick out deterministic algorithms when obtainable. These new APIs are all at present obtainable in beta.
Efficiency updates embody the addition of stack traces to the profiler, which permits customers to see not solely the operator identify within the profiler output desk, but in addition the place the operator is within the code.
As well as, TorchElastic is now a steady characteristic. TorchElastic gives a strict superset of the torch.distributed.launch CLI. It consists of added options for fault-tolerance and elasticity.
Different distributed coaching and RPC options embody beta help for uneven dataset inputs in DDP, improved async error and timeout dealing with in NCCL, the addition of TorchScript rpc_remote and rpc_sync, a distributed optimizer with TorchScript help, and extra.
Extra data on PyTorch 1.7 is offered right here.