A collection of terms and definitions used throughout the documentation.
| Term | Tags | Full Notes |
|---|---|---|
| Sparse/Dense Reward | Training | Details |
| Fine-Tuning | Fine-tuning | Details |
| Pre-Training | Pre-training | Details |
| Reinforcement Learning | Reinforcement learning, Training | Details |
| Dropout | Optimization, Training | Details |
| Transformer | Model Architecture, Training | Details |
| LoRA | Optimization, Training, Fine-tuning, Parameter-efficient training | Details |
| Model Collapse | Training, Failure modes, Optimization | Details |
| Feed-Forward Network | Neural Networks, Architecture, Basic Concepts | Details |
| Residual Stream | Transformers, Architecture, Deep Learning | Details |
| RAG | Natural Language Processing, Information Retrieval, Text Generation | Details |
| Poly-semanticity | Interpretability, Neural Networks, Semantics, Model Analysis | Details |