<< All versions
Skill v1.0.0
currentAutomated scan100/100mkurman/zorai/peft
──Details
PublishedJuly 29, 2026 at 12:05 PM
Content Hashsha256:c3637dc1d783e926...
Git SHAd0acbfaf3d62
──Files
Files (1 file, 1.7 KB)
SKILL.md1.7 KBactive
SKILL.md · 53 lines · 1.7 KB
version: "1.0.0" name: peft description: "Parameter-Efficient Fine-Tuning (PEFT) library. LoRA, QLoRA, AdaLoRA, IA3, Prefix Tuning, P-Tuning, Prompt Tuning. Fine-tune large models with minimal memory overhead. Hugging Face ecosystem integration." tags: [peft, lora, qlora, fine-tuning, llm, huggingface, parameter-efficient, zorai]
Overview
PEFT (Parameter-Efficient Fine-Tuning) adapts large pretrained models by training only a small subset of parameters. Supports LoRA, QLoRA, AdaLoRA, IA3, Prefix Tuning, P-Tuning, and Prompt Tuning. Reduces GPU memory by 4-16x compared to full fine-tuning.
Installation
bash
uv pip install peft
LoRA
python
from transformers import AutoModelForCausalLMfrom peft import LoraConfig, get_peft_model, TaskTypemodel = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")peft_config = LoraConfig(r=16, lora_alpha=32,target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],task_type=TaskType.CAUSAL_LM,)model = get_peft_model(model, peft_config)model.print_trainable_parameters()
Save & Merge
python
model.save_pretrained("adapter")from peft import PeftModelbase = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")merged = PeftModel.from_pretrained(base, "adapter").merge_and_unload()
QLoRA
python
from transformers import BitsAndBytesConfigmodel = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct",quantization_config=BitsAndBytesConfig(load_in_4bit=True), device_map="auto")model = get_peft_model(model, peft_config)