FineTuneLab

Unified LoRA/QLoRA Fine-Tuning Framework

FineTuneLab combines Unsloth's efficient LoRA training pipeline with woct0rdho's GGUF-over-LoRA training methods, enabling fine-tuning from both safetensor models and GGUF quantized files on a single consumer GPU.

Why

Modern fine-tuning workflows force a choice:

• Safetensors + Unsloth: Fast training, but large base models (13B+ params) require 24+ GB VRAM.
• GGUF (llama.cpp format): Tiny disk footprints and VRAM savings via deferred dequantization, but no first-class LoRA training support.

FineTuneLab unifies both into a single framework. You pick the model source (.safetensors or .gguf), and the framework handles the rest.

Architecture

FineTuneLab
├── Src/
│   ├── data/          Dataset loading & chat templates
│   ├── models/        Model loaders (safetensors + GGUF) & LoRA merger
│   ├── training/      LoRA/QLoRA training orchestration
│   ├── export/        GGUF export & format conversion
│   ├── config.py      Dataclass-based config schemas
│   └── cli.py         Command-line entry point
├── Examples/          [git-ignored] Reference repos for study
├── docs/              Research notes
└── tests/             Unit tests

Dual Loading Paths

Model Source          Training Path                       Inference Path
=================     ==================================   ====================
*.safetensors  ---    Unsloth FastModel                     Unsloth FastModel
                     (frozen + LoRA trainable params)      (merged weights)

*.gguf         ---    GGUF Dequant-JIT loader               GGUF native kernels
                     (deferred dequant on-demand,           (llama.cpp backends)
                      LoRA patches linear layers)

Both paths produce an nn.Module wrapping a frozen base model with trainable LoRA adapters injected into the target modules.

Key Integrations

Component	Source	Role
LoRA training	unsloth.FastModel	Efficient 2-stage optimizer, XPos, etc.
GGUF dequant	woct0rdho/transformers-qwen3-moe-fused	On-demand GPU dequant kernels
GGUF LoRA patches	woct0rdho/transformers-qwen3-moe-fused	Custom quantized linear + PEFT injection
Quantization alg	PrismML/Bonsai-demo	1-bit (Q1_0_g128) storage format spec

See docs/research_notes.md for detailed technical notes.

Installation

# Install dependencies (Python 3.10+)
pip install -e ".[dev]"

# Or with full GPU support
pip install -e ".[dev,gguf]"

Quick Start

Train LoRA from a GGUF model (16 GB VRAM)

finetune-lab train \
  --model ./models/Qwen3-8B-Q4_K_M.gguf \
  --dataset ./data/my_dataset.json \
  --output ./output/qwen3-lora/ \
  --lora-rank 16 \
  --batch-size 1 \
  --lr 1e-4 \
  --epochs 3

Train LoRA from safetensors

finetune-lab train \
  --model ./models/Qwen3-8B.safetensors \
  --dataset ./data/my_dataset.csv \
  --output ./output/qwen3-lora/ \
  --lora-rank 16 \
  --batch-size 2

Train from a pandas DataFrame

from finetune_lab import FineTuneLab

trainer = FineTuneLab.from_dataframe(
    df,
    model_path="model.gguf",
    lora_rank=16,
    target_modules=["q_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
)
trainer.train(epochs=3, learning_rate=1e-4)

Dataset Formats

FineTuneLab supports pandas DataFrames and any format convertible to one:

• CSV / TSV files
• JSON / JSONL files
• Parquet files
• HuggingFace datasets objects
• In-memory pandas.DataFrame

Each dataset must have columns that can be mapped to text (pre-tokenized) or messages (chat format with role/content pairs).

Hardware Notes

Tested on RTX 4070 Super (16 GB) + Intel i5-12400F. GGUF models at Q3_K_M or smaller quantizations can run full LoRA training within 16 GB VRAM. Safetensor models should use Unsloth's QLoRA mode or bnb 4-bit for similar memory efficiency.

License

Apache 2.0