# Quantile-balancing training code

> **Warning:** This is research code optimized only for small-scale high-
> sparsity experiments. It will not be efficient for large-scale training.

Dependencies: Python 3.13, NumPy 2.3.2, and PyTorch 2.9.1.

```bash
python3 -m pip install -r requirements.txt
```

It is also possible to use different backends - FA3 and SonicMoE can be
used with newer package versions.

Use `reproduce.py` to reproduce the post's experiments.