8 Advanced parallelization - Deep Learning with JAX

Using easy-to-revise parallelism with xmap() · Compiling and automatically partitioning functions with pjit() · Using tensor sharding to achieve parallelization with XLA · Running code in multi-host configurations

Efficiently Scale LLM Training Across a Large GPU Cluster with

Applying sequence and parallel graph splits on a data-parallel

Using Cloud TPU Multislice to scale AI workloads

Why You Should (or Shouldn't) be Using Google's JAX in 2023

Lecture 2: Development Infrastructure & Tooling - The Full Stack

Self-directed online machine learning for topology optimization

Training Deep Networks with Data Parallelism in Jax

GitHub - che-shr-cat/JAX-in-Action: Notebooks for the JAX in

Why You Should (or Shouldn't) be Using Google's JAX in 2023

Learn JAX in 2023: Part 2 - grad, jit, vmap, and pmap

Vectorize and Parallelize RL Environments with JAX: Q-learning at

Model Parallelism

Grigory Sapunov on LinkedIn: Deep Learning with JAX

Dive into Deep Learning — Dive into Deep Learning 1.0.3 documentation