QLoRA Fine-Tuning for Offline Structured Generation

Constrained Transformer Adaptation

Fine-tuned a 4-bit Qwen2.5-1.5B model with MLX QLoRA on synthetic structured-task examples. The base weights remained frozen while low-rank adapters updated 0.171% of parameters. A fixed benchmark, unchanged external smoke set, resource profiler, deterministic validator, and request-level trace made both gains and failure modes visible.

★7/8 Held-Out Tasks
Start a project like this

Challenge

A structured-generation workflow needed to run locally without depending on a network connection or a large inference service. The model had to fit a constrained compute budget, emit machine-readable output, and remain subordinate to deterministic safety and schema validation.

Solution

Fine-tuned a 4-bit Qwen2.5-1.5B model with MLX QLoRA on synthetic structured-task examples. The base weights remained frozen while low-rank adapters updated 0.171% of parameters. A fixed benchmark, unchanged external smoke set, resource profiler, deterministic validator, and request-level trace made both gains and failure modes visible.

Impact Metrics

7/8
Held-Out Tasks
passed
100%
JSON Validity
held-out
0.171%
Trainable Weights
of model
849.5
Model Footprint
MiB

Results

  • • Improved the fixed held-out benchmark from 0/8 to 7/8 passing scenarios
  • • Maintained 100% valid JSON on the held-out benchmark
  • • Measured 1.18-second mean local inference and 1.78 GB peak memory
  • • Preserved the weaker 1/6 unchanged external-template result as a documented generalization limitation
  • • Added 12 smoke scenarios, 25 automated tests, and terminal plus JSONL request tracing

Business Impact

Demonstrates the full constrained-model loop: choose a small base model, adapt it with parameter-efficient training, measure latency and memory, evaluate task behavior, expose failure modes, and keep deterministic controls outside the model.

Architecture

Constrained Transformer AdaptationSynthetictask examples4-bit BaseQwen 1.5BQLoRA0.171% trainedBenchmarkheld-out + smokeValidatorschema + policyOfflinestructured outputQuality7/8 held-out • 1/6 externalResources1.18s • 1.78 GB peakControl Planerules remain authoritativeLocal MLX path • fixed evaluation • request tracing • no network dependency

Portfolio research project using synthetic scenarios and local compute. It is not a production edge deployment, and the unchanged external-template result is included to make the remaining generalization gap explicit.

Adapt, Measure, and Constrain

A proven approach combining statistical rigor, automation, and AWS best practices.

1

Define the Contract

Fix the structured output schema, scenario set, validator rules, and resource measurements before judging the tuned model.

2

Adapt Efficiently

Use 4-bit QLoRA so the frozen base model remains compact while a small set of low-rank adapter weights learns the task format.

3

Benchmark Behavior

Compare the base and tuned checkpoints on held-out scenarios, JSON validity, unchanged external templates, latency, memory, and artifact size.

4

Keep Rules Authoritative

Treat generated output as a proposal. Deterministic parsing and policy validation decide whether the system may accept it.

Scale & Scope

12
Scenarios
smoke cases - fixed structured-task coverage
25
Validation
automated tests - validation and observability checks
Offline
Execution
local path - no cloud inference dependency

Technology Stack

  • Qwen2.5-1.5B
  • MLX and MLX-LM
  • 4-bit QLoRA / LoRA adapters
  • Python evaluation harness
  • Deterministic JSON and policy validation
  • JSONL observability
  • Local browser workbench

Need a similar solution?

Let's replicate this success within your organization with a tailored engagement plan.

Start a ConversationView Our Process