The Incredible PyTorch - Ritchie Ng

This is a curated list of tutorials, projects, libraries, videos, papers, books and anything related to the incredible PyTorch. Feel free to make a pull request to contribute to this list.

Table Of Contents

• Table Of Contents
  • Tutorials
  • Large Language Models (LLMs)
  • Agentic AI
  • Guardrails and AI Safety
  • Tabular Data
  • Visualization
  • Explainability
  • Object Detection
  • Long-Tailed / Out-of-Distribution Recognition
  • Activation Functions
  • Energy-Based Learning
  • Missing Data
  • Architecture Search
  • Continual Learning
  • Optimization
  • Quantization
  • Quantum Machine Learning
  • Neural Network Compression
  • Facial, Action and Pose Recognition
  • Super resolution
  • Synthetesizing Views
  • Voice
  • Medical
  • 3D Segmentation, Classification and Regression
  • Video Recognition
  • Recurrent Neural Networks (RNNs)
  • Convolutional Neural Networks (CNNs)
  • Segmentation
  • Geometric Deep Learning: Graph \& Irregular Structures
  • Sorting
  • Ordinary Differential Equations Networks
  • Multi-task Learning
  • GANs, VAEs, and AEs
  • Unsupervised Learning
  • Adversarial Attacks
  • Style Transfer
  • Image Captioning
  • Transformers
  • Similarity Networks and Functions
  • Reasoning
  • General NLP
  • Question and Answering
  • Speech Generation and Recognition
  • Document and Text Classification
  • Text Generation
  • Text to Image
  • Translation
  • Sentiment Analysis
  • Deep Reinforcement Learning
  • Deep Bayesian Learning and Probabilistic Programmming
  • Spiking Neural Networks
  • Anomaly Detection
  • Regression Types
  • Time Series
  • Synthetic Datasets
  • Neural Network General Improvements
  • DNN Applications in Chemistry and Physics
  • New Thinking on General Neural Network Architecture
  • Linear Algebra
  • API Abstraction
  • Low Level Utilities
  • PyTorch Utilities
  • PyTorch Video Tutorials
  • Community
  • To be Classified
  • Links to This Repository
  • Contributions

  • [New Special Dedicated List to AI Agents

    The Incredible AI Agents](#new-special-dedicated-list-to-ai-agents–the-incredible-ai-agents)

Tutorials

• Official PyTorch Tutorials
• Official PyTorch Examples
• Dive Into Deep Learning with PyTorch
• How to Read Pytorch
• Minicourse in Deep Learning with PyTorch (Multi-language)
• Practical Deep Learning with PyTorch
• Deep Learning Models
• C++ Implementation of PyTorch Tutorial
• Simple Examples to Introduce PyTorch
• Mini Tutorials in PyTorch
• Deep Learning for NLP
• Deep Learning Tutorial for Researchers
• Fully Convolutional Networks implemented with PyTorch
• Simple PyTorch Tutorials Zero to ALL
• DeepNLP-models-Pytorch
• MILA PyTorch Welcome Tutorials
• Effective PyTorch, Optimizing Runtime with TorchScript and Numerical Stability Optimization
• Practical PyTorch
• PyTorch Project Template
• Semantic Search with PyTorch

Large Language Models (LLMs)

• LLM Tutorials
  • Build a Large Language Model (From Scratch)
  • Hugginface LLM Training Book, a collection of methodologies to help with successful training of large language models
• General
  • Starcoder 2, family of code generation models
  • GPT Fast, fast and hackable pytorch native transformer inference
  • Mixtral Offloading, run Mixtral-8x7B models in Colab or consumer desktops
  • Llama
  • Llama Recipes
  • TinyLlama
  • Mosaic Pretrained Transformers (MPT)
  • VLLM, high-throughput and memory-efficient inference and serving engine for LLMs
  • Dolly
  • Vicuna
  • Mistral 7B
  • BigDL LLM, library for running LLM (large language model) on Intel XPU (from Laptop to GPU to Cloud) using INT4 with very low latency1 (for any PyTorch model)
  • Simple LLM Finetuner
  • Petals, run LLMs at home, BitTorrent-style, fine-tuning and inference up to 10x faster than offloading
  • Gemma, Google’s family of lightweight, state-of-the-art open models
  • Qwen, Alibaba Cloud’s large language model
  • CodeT5, code-aware encoder-decoder model for code understanding and generation
  • OpenLLaMA, permissively licensed open source reproduction of Meta AI’s LLaMA
  • RedPajama, leading open-source models with package to reproduce LLaMA training dataset
  • MosaicML LLM Foundry, codebase for training, finetuning, and deploying LLMs
• Japanese
  • Japanese Llama
  • Japanese GPT Neox and Open Calm
• Chinese
  • Chinese Llamma-2 7B
  • Chinese Vicuna
• Retrieval Augmented Generation (RAG)
  • LlamaIndex, data framework for your LLM application
• Embeddings
  • ChromaDB, open-source embedding database
• Applications
  • Langchain, building applications with LLMs through composability
  • LangSmith, platform for building production-grade LLM applications
  • LiteLLM, call all LLM APIs using the OpenAI format
  • OpenAI Python, official Python library for the OpenAI API
  • Guidance, library for controlling large language models
• Finetuning
  • Huggingface PEFT, State-of-the-art Parameter-Efficient Fine-Tuning
  • Unsloth, finetune LLMs 2-5x faster with 80% less memory
  • LoRA, Low-Rank Adaptation of Large Language Models
  • QLoRA, efficient finetuning of quantized LLMs
  • Axolotl, tool designed to streamline the fine-tuning of various AI models
  • LLaMA-Factory, unified efficient fine-tuning of 100+ LLMs
• Training
  • Higgsfield, Fault-tolerant, highly scalable GPU orchestration, and a machine learning framework designed for training models with billions to trillions of parameters
  • DeepSpeed, deep learning optimization library
  • FairScale, PyTorch extensions for high performance and large scale training
  • Accelerate, simple way to train and use PyTorch models with multi-GPU, TPU, mixed-precision
  • ColossalAI, unified deep learning system for large-scale model training and inference
• Quantization
  • AutoGPTQ, easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm
  • BitsAndBytes, accessible large language models via k-bit quantization
  • GPTQ-for-LLaMa, 4 bits quantization of LLaMA using GPTQ
  • Optimum, acceleration of 🤗 Transformers and 🤗 Diffusers

Agentic AI

• Multi-Agent Systems
  • LangGraph, library for building stateful, multi-actor applications with LLMs
  • AutoGen, library that enables the creation of applications using multiple agents that can converse with each other
  • CrewAI, framework for orchestrating role-playing, autonomous AI agents
  • MetaGPT, multi-agent framework for software company simulation
  • AgentScope, user-friendly multi-agent platform
  • Swarm, educational framework for building and deploying multi-agent systems
• Autonomous Agents
  • AutoGPT, autonomous GPT-4 experiment to make GPT-4 fully autonomous
  • BabyAGI, example of an AI-powered task management system
  • LangChain Agents, building agents with LangChain
  • ReAct: Reasoning and Acting with Language Models
  • Voyager, open-ended embodied agent with large language models
• Agent Orchestration and Frameworks
  • Semantic Kernel, lightweight SDK for integrating AI services
  • OpenAI Function Calling, tools for function calling with OpenAI models
  • LlamaIndex Agents, data agents with LlamaIndex
  • Haystack Agents, building search and QA agents
  • DSPy, framework for algorithmically optimizing LM prompts and weights
• Planning and Reasoning
  • Tree of Thoughts, deliberate problem solving with large language models
  • ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models
  • Plan-and-Solve Prompting
• Memory and Learning
  • MemGPT, creating LLM agents with long-term memory
  • Zep, fast, scalable building blocks for production LLM apps

Guardrails and AI Safety

• Content Filtering and Moderation
  • Guardrails AI, framework for building reliable AI applications
  • NeMo Guardrails, toolkit for building trustworthy, safe and secure LLM applications
  • OpenAI Moderation API Tools
  • Detoxify, toxic comment classification using transformer models
  • Perspective API PyTorch Implementation, toxicity detection
• Prompt Injection Defense
  • Prompt Injection Detector, detecting prompt injection attacks
  • LLM Guard, security toolkit for LLM interactions
  • Garak, LLM vulnerability scanner
• Bias Detection and Mitigation
  • FairLearn, toolkit for assessing and improving fairness
  • AIF360, comprehensive set of fairness metrics and bias mitigation algorithms
  • What-If Tool, tool for analyzing and understanding ML models
• Privacy and Security
  • Opacus, library for training PyTorch models with differential privacy
  • PySyft, secure and private Deep Learning framework
  • CrypTen, framework for Privacy Preserving Machine Learning
  • Adversarial Robustness Toolbox, library for adversarial attacks and defenses
• Model Interpretability and Explainability
  • LIME, explaining the predictions of machine learning classifiers
  • SHAP, unified approach to explain the output of machine learning models
  • InterpretML, interpret and understand machine learning models
  • Alibi, algorithms for explaining machine learning models
• Safety Evaluation and Testing
  • AI Safety Gym, environments and tools for AI safety research
  • Anthropic’s Constitutional AI implementations
  • HarmBench, standardized evaluation framework for automated red teaming

Tabular Data

• PyTorch Frame: A Modular Framework for Multi-Modal Tabular Learning
• Pytorch Tabular,standard framework for modelling Deep Learning Models for tabular data
• Tab Transformer
• PyTorch-TabNet: Attentive Interpretable Tabular Learning
• carefree-learn: A minimal Automatic Machine Learning (AutoML) solution for tabular datasets based on PyTorch

Visualization

• Loss Visualization
• Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
• SmoothGrad: removing noise by adding noise
• DeepDream: dream-like hallucinogenic visuals
• FlashTorch: Visualization toolkit for neural networks in PyTorch
• Lucent: Lucid adapted for PyTorch
• DreamCreator: Training GoogleNet models for DeepDream with custom datasets made simple
• CNN Feature Map Visualisation

Explainability

• Neural-Backed Decision Trees
• Efficient Covariance Estimation from Temporal Data
• Hierarchical interpretations for neural network predictions
• Shap, a unified approach to explain the output of any machine learning model
• VIsualizing PyTorch saved .pth deep learning models with netron
• Distilling a Neural Network Into a Soft Decision Tree
• Captum, A unified model interpretability library for PyTorch

Object Detection

• MMDetection Object Detection Toolbox
• Mask R-CNN Benchmark: Faster R-CNN and Mask R-CNN in PyTorch 1.0
• YOLO-World
• YOLOS
• YOLOF
• YOLOX
• YOLOv12: Attention-Centric Real-Time Object Detectors
• YOLOv11
• YOLOv10
• YOLOv9
• YOLOv8
• Yolov7
• YOLOv6
• Yolov5
• Yolov4
• YOLOv3
• YOLOv2: Real-Time Object Detection
• SSD: Single Shot MultiBox Detector
• Detectron models for Object Detection
• Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks
• Whale Detector
• Catalyst.Detection

Long-Tailed / Out-of-Distribution Recognition

• Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization
• Invariant Risk Minimization
• Training Confidence-Calibrated Classifier for Detecting Out-of-Distribution Samples
• Deep Anomaly Detection with Outlier Exposure
• Large-Scale Long-Tailed Recognition in an Open World
• Principled Detection of Out-of-Distribution Examples in Neural Networks
• Learning Confidence for Out-of-Distribution Detection in Neural Networks
• PyTorch Imbalanced Class Sampler

Activation Functions

• Rational Activations - Learnable Rational Activation Functions
• FreeGrad, PyTorch library for custom backward passes, straight-through estimators and gradient transforms.

Energy-Based Learning

• EBGAN, Energy-Based GANs
• Maximum Entropy Generators for Energy-based Models

Missing Data

• BRITS: Bidirectional Recurrent Imputation for Time Series

Architecture Search

• EfficientNetV2
• DenseNAS
• DARTS: Differentiable Architecture Search
• Efficient Neural Architecture Search (ENAS)
• EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

Continual Learning

• Renate, Automatic Retraining of Neural Networks

Optimization

• AccSGD, AdaBound, AdaMod, DiffGrad, Lamb, NovoGrad, RAdam, SGDW, Yogi and more
• Lookahead Optimizer: k steps forward, 1 step back
• RAdam, On the Variance of the Adaptive Learning Rate and Beyond
• Over9000, Comparison of RAdam, Lookahead, Novograd, and combinations
• AdaBound, Train As Fast as Adam As Good as SGD
• Riemannian Adaptive Optimization Methods
• L-BFGS
• OptNet: Differentiable Optimization as a Layer in Neural Networks
• Learning to learn by gradient descent by gradient descent
• Surrogate Gradient Learning in Spiking Neural Networks
• TorchOpt: An Efficient Library for Differentiable Optimization

Quantization

• Additive Power-of-Two Quantization: An Efficient Non-uniform Discretization For Neural Networks

Quantum Machine Learning

• Tor10, generic tensor-network library for quantum simulation in PyTorch
• PennyLane, cross-platform Python library for quantum machine learning with PyTorch interface

Neural Network Compression

• Bayesian Compression for Deep Learning
• Neural Network Distiller by Intel AI Lab: a Python package for neural network compression research
• Learning Sparse Neural Networks through L0 regularization
• Energy-constrained Compression for Deep Neural Networks via Weighted Sparse Projection and Layer Input Masking
• EigenDamage: Structured Pruning in the Kronecker-Factored Eigenbasis
• Pruning Convolutional Neural Networks for Resource Efficient Inference
• Pruning neural networks: is it time to nip it in the bud? (showing reduced networks work better)

Facial, Action and Pose Recognition

• Facenet: Pretrained Pytorch face detection and recognition models
• DGC-Net: Dense Geometric Correspondence Network
• High performance facial recognition library on PyTorch
• FaceBoxes, a CPU real-time face detector with high accuracy
• How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)
• Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognition
• PyTorch Realtime Multi-Person Pose Estimation
• SphereFace: Deep Hypersphere Embedding for Face Recognition
• GANimation: Anatomically-aware Facial Animation from a Single Image
• Shufflenet V2 by Face++ with better results than paper
• Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach
• Unsupervised Learning of Depth and Ego-Motion from Video
• FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
• FlowNet: Learning Optical Flow with Convolutional Networks
• Optical Flow Estimation using a Spatial Pyramid Network
• OpenFace in PyTorch
• Deep Face Recognition in PyTorch

Super resolution

• Enhanced Deep Residual Networks for Single Image Super-Resolution
• Superresolution using an efficient sub-pixel convolutional neural network
• Perceptual Losses for Real-Time Style Transfer and Super-Resolution

Synthetesizing Views

• NeRF, Neural Radian Fields, Synthesizing Novels Views of Complex Scenes

Voice

• Google AI VoiceFilter: Targeted Voice Separatation by Speaker-Conditioned Spectrogram Masking

Medical

• Medical Zoo, 3D multi-modal medical image segmentation library in PyTorch
• U-Net for FLAIR Abnormality Segmentation in Brain MRI
• Genomic Classification via ULMFiT
• Deep Neural Networks Improve Radiologists’ Performance in Breast Cancer Screening
• Delira, lightweight framework for medical imaging prototyping
• V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation
• Medical Torch, medical imaging framework for PyTorch
• TorchXRayVision - A library for chest X-ray datasets and models. Including pre-trainined models.

3D Segmentation, Classification and Regression

• Kaolin, Library for Accelerating 3D Deep Learning Research
• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
• 3D segmentation with MONAI and Catalyst

Video Recognition

• Dancing to Music
• Devil Is in the Edges: Learning Semantic Boundaries from Noisy Annotations
• Deep Video Analytics
• PredRNN: Recurrent Neural Networks for Predictive Learning using Spatiotemporal LSTMs

Recurrent Neural Networks (RNNs)

• SRU: training RNNs as fast as CNNs
• Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks
• Averaged Stochastic Gradient Descent with Weight Dropped LSTM
• Training RNNs as Fast as CNNs
• Quasi-Recurrent Neural Network (QRNN)
• ReSeg: A Recurrent Neural Network-based Model for Semantic Segmentation
• A Recurrent Latent Variable Model for Sequential Data (VRNN)
• Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks
• Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling
• Attentive Recurrent Comparators
• Collection of Sequence to Sequence Models with PyTorch
  1. Vanilla Sequence to Sequence models
  2. Attention based Sequence to Sequence models
  3. Faster attention mechanisms using dot products between the final encoder and decoder hidden states

Convolutional Neural Networks (CNNs)

• LegoNet: Efficient Convolutional Neural Networks with Lego Filters
• MeshCNN, a convolutional neural network designed specifically for triangular meshes
• Octave Convolution
• PyTorch Image Models, ResNet/ResNeXT, DPN, MobileNet-V3/V2/V1, MNASNet, Single-Path NAS, FBNet
• Deep Neural Networks with Box Convolutions
• Invertible Residual Networks
• Stochastic Downsampling for Cost-Adjustable Inference and Improved Regularization in Convolutional Networks
• Faster Faster R-CNN Implementation
  • Faster R-CNN Another Implementation
• Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer
• Wide ResNet model in PyTorch -DiracNets: Training Very Deep Neural Networks Without Skip-Connections
• An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition
• Efficient Densenet
• Video Frame Interpolation via Adaptive Separable Convolution
• Learning local feature descriptors with triplets and shallow convolutional neural networks
• Densely Connected Convolutional Networks
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
• Deep Residual Learning for Image Recognition
• Training Wide ResNets for CIFAR-10 and CIFAR-100 in PyTorch
• Deformable Convolutional Network
• Convolutional Neural Fabrics
• Deformable Convolutional Networks in PyTorch
• Dilated ResNet combination with Dilated Convolutions
• Striving for Simplicity: The All Convolutional Net
• Convolutional LSTM Network
• Big collection of pretrained classification models
• PyTorch Image Classification with Kaggle Dogs vs Cats Dataset
• CIFAR-10 on Pytorch with VGG, ResNet and DenseNet
• Base pretrained models and datasets in pytorch (MNIST, SVHN, CIFAR10, CIFAR100, STL10, AlexNet, VGG16, VGG19, ResNet, Inception, SqueezeNet)
• NVIDIA/unsupervised-video-interpolation

Segmentation

• Detectron2 by FAIR
• Pixel-wise Segmentation on VOC2012 Dataset using PyTorch
• Pywick - High-level batteries-included neural network training library for Pytorch
• Improving Semantic Segmentation via Video Propagation and Label Relaxation
• Super-BPD: Super Boundary-to-Pixel Direction for Fast Image Segmentation
• Catalyst.Segmentation
• Segmentation models with pretrained backbones

Geometric Deep Learning: Graph & Irregular Structures

• PyTorch Geometric, Deep Learning Extension
• PyTorch Geometric Temporal: A Temporal Extension Library for PyTorch Geometric
• PyTorch Geometric Signed Directed: A Signed & Directed Extension Library for PyTorch Geometric
• ChemicalX: A PyTorch Based Deep Learning Library for Drug Pair Scoring
• Self-Attention Graph Pooling
• Position-aware Graph Neural Networks
• Signed Graph Convolutional Neural Network
• Graph U-Nets
• Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks
• MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing
• Semi-Supervised Graph Classification: A Hierarchical Graph Perspective
• PyTorch BigGraph by FAIR for Generating Embeddings From Large-scale Graph Data
• Capsule Graph Neural Network
• Splitter: Learning Node Representations that Capture Multiple Social Contexts
• A Higher-Order Graph Convolutional Layer
• Predict then Propagate: Graph Neural Networks meet Personalized PageRank
• Lorentz Embeddings: Learn Continuous Hierarchies in Hyperbolic Space
• Graph Wavelet Neural Network
• Watch Your Step: Learning Node Embeddings via Graph Attention
• Signed Graph Convolutional Network
• Graph Classification Using Structural Attention
• SimGNN: A Neural Network Approach to Fast Graph Similarity Computation
• SINE: Scalable Incomplete Network Embedding
• HypER: Hypernetwork Knowledge Graph Embeddings
• TuckER: Tensor Factorization for Knowledge Graph Completion
• PyKEEN: A Python library for learning and evaluating knowledge graph embeddings
• Pathfinder Discovery Networks for Neural Message Passing
• SSSNET: Semi-Supervised Signed Network Clustering
• MagNet: A Neural Network for Directed Graphs
• PyTorch Geopooling: Geospatial Pooling Modules for Neural Networks in PyTorch

Sorting

• Stochastic Optimization of Sorting Networks via Continuous Relaxations

Ordinary Differential Equations Networks

• Latent ODEs for Irregularly-Sampled Time Series
• GRU-ODE-Bayes: continuous modelling of sporadically-observed time series

Multi-task Learning

• Hierarchical Multi-Task Learning Model
• Task-based End-to-end Model Learning
• torchMTL: A lightweight module for Multi-Task Learning in pytorch

GANs, VAEs, and AEs

• BigGAN: Large Scale GAN Training for High Fidelity Natural Image Synthesis
• High Fidelity Performance Metrics for Generative Models in PyTorch
• Mimicry, PyTorch Library for Reproducibility of GAN Research
• Clean Readable CycleGAN
• StarGAN
• Block Neural Autoregressive Flow
• High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
• A Style-Based Generator Architecture for Generative Adversarial Networks
• GANDissect, PyTorch Tool for Visualizing Neurons in GANs
• Learning deep representations by mutual information estimation and maximization
• Variational Laplace Autoencoders
• VeGANS, library for easily training GANs
• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• Conditional GAN
• Wasserstein GAN
• Adversarial Generator-Encoder Network
• Image-to-Image Translation with Conditional Adversarial Networks
• Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
• On the Effects of Batch and Weight Normalization in Generative Adversarial Networks
• Improved Training of Wasserstein GANs
• Collection of Generative Models with PyTorch
  • Generative Adversarial Nets (GAN)
    1. Vanilla GAN
    2. Conditional GAN
    3. InfoGAN
    4. Wasserstein GAN
    5. Mode Regularized GAN
  • Variational Autoencoder (VAE)
    1. Vanilla VAE
    2. Conditional VAE
    3. Denoising VAE
    4. Adversarial Autoencoder
    5. Adversarial Variational Bayes
• Improved Training of Wasserstein GANs
• CycleGAN and Semi-Supervised GAN
• Improving Variational Auto-Encoders using Householder Flow and using convex combination linear Inverse Autoregressive Flow
• PyTorch GAN Collection
• Generative Adversarial Networks, focusing on anime face drawing
• Simple Generative Adversarial Networks
• Adversarial Auto-encoders
• torchgan: Framework for modelling Generative Adversarial Networks in Pytorch
• Evaluating Lossy Compression Rates of Deep Generative Models
• Catalyst.GAN
  1. Vanilla GAN
  2. Conditional GAN
  3. Wasserstein GAN
  4. Improved Training of Wasserstein GANs

Unsupervised Learning

• Unsupervised Embedding Learning via Invariant and Spreading Instance Feature
• AND: Anchor Neighbourhood Discovery

Adversarial Attacks

• Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images
• Explaining and Harnessing Adversarial Examples
• AdverTorch - A Toolbox for Adversarial Robustness Research

Style Transfer

• Pystiche: Framework for Neural Style Transfer
• Detecting Adversarial Examples via Neural Fingerprinting
• A Neural Algorithm of Artistic Style
• Multi-style Generative Network for Real-time Transfer
• DeOldify, Coloring Old Images
• Neural Style Transfer
• Fast Neural Style Transfer
• Draw like Bob Ross

Image Captioning

• CLIP (Contrastive Language-Image Pre-Training)
• Neuraltalk 2, Image Captioning Model, in PyTorch
• Generate captions from an image with PyTorch
• DenseCap: Fully Convolutional Localization Networks for Dense Captioning

Transformers

• Attention is all you need
• Spatial Transformer Networks

Similarity Networks and Functions

• Conditional Similarity Networks

Reasoning

• Inferring and Executing Programs for Visual Reasoning

General NLP

• nanoGPT, fastest repository for training/finetuning medium-sized GPTs
• minGPT, Re-implementation of GPT to be small, clean, interpretable and educational
• Espresso, Module Neural Automatic Speech Recognition Toolkit
• Label-aware Document Representation via Hybrid Attention for Extreme Multi-Label Text Classification
• XLNet
• Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading
• Cross-lingual Language Model Pretraining
• Libre Office Translate via PyTorch NMT
• BERT
• VSE++: Improved Visual-Semantic Embeddings
• A Structured Self-Attentive Sentence Embedding
• Neural Sequence labeling model
• Skip-Thought Vectors
• Complete Suite for Training Seq2Seq Models in PyTorch
• MUSE: Multilingual Unsupervised and Supervised Embeddings
• TorchMoji: PyTorch Implementation of DeepMoji to under Language used to Express Emotions

Question and Answering

• Visual Question Answering in Pytorch
• Reading Wikipedia to Answer Open-Domain Questions
• Deal or No Deal? End-to-End Learning for Negotiation Dialogues
• Interpretable Counting for Visual Question Answering
• Open Source Chatbot with PyTorch

Speech Generation and Recognition

• PyTorch-Kaldi Speech Recognition Toolkit
• WaveGlow: A Flow-based Generative Network for Speech Synthesis
• OpenNMT
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• WeNet: Production First and Production Ready End-to-End Speech Recognition Toolkit

Document and Text Classification

• Hierarchical Attention Network for Document Classification
• Hierarchical Attention Networks for Document Classification
• CNN Based Text Classification

Text Generation

• Pytorch Poetry Generation

Text to Image

• Stable Diffusion
• Dall-E 2
• Dall-E

Translation

• Open-source (MIT) Neural Machine Translation (NMT) System

Sentiment Analysis

• Recurrent Neural Networks for Sentiment Analysis (Aspect-Based) on SemEval 2014
• Seq2Seq Intent Parsing
• Finetuning BERT for Sentiment Analysis

Deep Reinforcement Learning

• Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels
• Exploration by Random Network Distillation
• EGG: Emergence of lanGuage in Games, quickly implement multi-agent games with discrete channel communication
• Temporal Difference VAE
• High-performance Atari A3C Agent in 180 Lines PyTorch
• Learning when to communicate at scale in multiagent cooperative and competitive tasks
• Actor-Attention-Critic for Multi-Agent Reinforcement Learning
• PPO in PyTorch C++
• Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback
• Asynchronous Methods for Deep Reinforcement Learning
• Continuous Deep Q-Learning with Model-based Acceleration
• Asynchronous Methods for Deep Reinforcement Learning for Atari 2600
• Trust Region Policy Optimization
• Neural Combinatorial Optimization with Reinforcement Learning
• Noisy Networks for Exploration
• Distributed Proximal Policy Optimization
• Reinforcement learning models in ViZDoom environment with PyTorch
• Reinforcement learning models using Gym and Pytorch
• SLM-Lab: Modular Deep Reinforcement Learning framework in PyTorch
• Catalyst.RL

Deep Bayesian Learning and Probabilistic Programmming

• BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning
• Subspace Inference for Bayesian Deep Learning
• Bayesian Deep Learning with Variational Inference Package
• Probabilistic Programming and Statistical Inference in PyTorch
• Bayesian CNN with Variational Inferece in PyTorch

Spiking Neural Networks

• Norse, Library for Deep Learning with Spiking Neural Networks

Anomaly Detection

• Detection of Accounting Anomalies using Deep Autoencoder Neural Networks

Regression Types

• Quantile Regression DQN

Time Series

• Dual Self-Attention Network for Multivariate Time Series Forecasting
• DILATE: DIstortion Loss with shApe and tImE
• Variational Recurrent Autoencoder for Timeseries Clustering
• Spatio-Temporal Neural Networks for Space-Time Series Modeling and Relations Discovery
• Flow Forecast: A deep learning for time series forecasting framework built in PyTorch

Synthetic Datasets

• Meta-Sim: Learning to Generate Synthetic Datasets

Neural Network General Improvements

• The Artificial Dendrite Network Library for PyTorch
• In-Place Activated BatchNorm for Memory-Optimized Training of DNNs
• Train longer, generalize better: closing the generalization gap in large batch training of neural networks
• FreezeOut: Accelerate Training by Progressively Freezing Layers
• Binary Stochastic Neurons
• Compact Bilinear Pooling
• Mixed Precision Training in PyTorch

DNN Applications in Chemistry and Physics

• Wave Physics as an Analog Recurrent Neural Network
• Neural Message Passing for Quantum Chemistry
• Automatic chemical design using a data-driven continuous representation of molecules
• Deep Learning for Physical Processes: Integrating Prior Scientific Knowledge
• Differentiable Molecular Simulation for Learning and Control

New Thinking on General Neural Network Architecture

• Complement Objective Training
• Decoupled Neural Interfaces using Synthetic Gradients

Linear Algebra

• Eigenvectors from Eigenvalues

API Abstraction

• Torch Layers, Shape inference for PyTorch, SOTA Layers
• Hummingbird, run trained scikit-learn models on GPU with PyTorch

Low Level Utilities

• TorchSharp, .NET API with access to underlying library powering PyTorch

PyTorch Utilities

• Functorch: prototype of JAX-like composable Function transformers for PyTorch
• Poutyne: Simplified Framework for Training Neural Networks
• PyTorch Metric Learning
• Kornia: an Open Source Differentiable Computer Vision Library for PyTorch
• BackPACK to easily Extract Variance, Diagonal of Gauss-Newton, and KFAC
• PyHessian for Computing Hessian Eigenvalues, trace of matrix, and ESD
• Hessian in PyTorch
• Differentiable Convex Layers
• Albumentations: Fast Image Augmentation Library
• Higher, obtain higher order gradients over losses spanning training loops
• Neural Pipeline, Training Pipeline for PyTorch
• Layer-by-layer PyTorch Model Profiler for Checking Model Time Consumption
• Sparse Distributions
• Diffdist, Adds Support for Differentiable Communication allowing distributed model parallelism
• HessianFlow, Library for Hessian Based Algorithms
• Texar, PyTorch Toolkit for Text Generation
• PyTorch FLOPs counter
• PyTorch Inference on C++ in Windows
• EuclidesDB, Multi-Model Machine Learning Feature Database
• Data Augmentation and Sampling for Pytorch
• PyText, deep learning based NLP modelling framework officially maintained by FAIR
• Torchstat for Statistics on PyTorch Models
• Load Audio files directly into PyTorch Tensors
• Weight Initializations
• Spatial transformer implemented in PyTorch
• PyTorch AWS AMI, run PyTorch with GPU support in less than 5 minutes
• Use tensorboard with PyTorch
• Simple Fit Module in PyTorch, similar to Keras
• torchbearer: A model fitting library for PyTorch
• PyTorch to Keras model converter
• Gluon to PyTorch model converter with code generation
• Catalyst: High-level utils for PyTorch DL & RL research
• PyTorch Lightning: Scalable and lightweight deep learning research framework
• Determined: Scalable deep learning platform with PyTorch support
• PyTorch-Ignite: High-level library to help with training and evaluating neural networks in PyTorch flexibly and transparently
• torchvision: A package consisting of popular datasets, model architectures, and common image transformations for computer vision.
• Poutyne: A Keras-like framework for PyTorch and handles much of the boilerplating code needed to train neural networks.
• torchensemble: Scikit-Learn like ensemble methods in PyTorch
• TorchFix - a linter for PyTorch-using code with autofix support
• pytorch360convert - Differentiable image conversions between 360° equirectangular images, cubemaps, and perspective projections

PyTorch Video Tutorials

• PyTorch Zero to All Lectures
• PyTorch For Deep Learning Full Course
• PyTorch Lightning 101 with Alfredo Canziani and William Falcon
• Practical Deep Learning with PyTorch

Community

• PyTorch Discussion Forum
• StackOverflow PyTorch Tags
• Catalyst.Slack

To be Classified

• Perturbative Neural Networks
• Accurate Neural Network Potential
• Scaling the Scattering Transform: Deep Hybrid Networks
• CortexNet: a Generic Network Family for Robust Visual Temporal Representations
• Oriented Response Networks
• Associative Compression Networks
• Clarinet
• Continuous Wavelet Transforms
• mixup: Beyond Empirical Risk Minimization
• Network In Network
• Highway Networks
• Hybrid computing using a neural network with dynamic external memory
• Value Iteration Networks
• Differentiable Neural Computer
• A Neural Representation of Sketch Drawings
• Understanding Deep Image Representations by Inverting Them
• NIMA: Neural Image Assessment
• NASNet-A-Mobile. Ported weights
• Graphics code generating model using Processing

Links to This Repository

• Github Repository
• Website

Contributions

Do feel free to contribute!

You can raise an issue or submit a pull request, whichever is more convenient for you. The guideline is simple: just follow the format of the previous bullet point or create a new section if it’s a new category.

New Special Dedicated List to AI Agents | The Incredible AI Agents

Feel free to visit The Incredible AI Agents, a curated list of resources on building, evaluating, deploying, monitoring AI Agents. Feel free to view, star, share and/or contribute!