<\/p>\n
<\/p>\n
<\/p>\n
Fine-tuning the Gemma 4 E2B (Effective 2B) model is highly accessible, allowing for local training on consumer hardware with as little as <\/span>8GB VRAM<\/b><\/span>. Gemma 4 E2B is a multimodal model (text, image, and audio) designed for efficiency using Per-Layer Embeddings (PLE).<\/span><\/p>\n <\/p>\n The most recommended approach for fine-tuning Gemma 4 E2B is using <\/span>Unsloth<\/b><\/span>, which offers ~1.5x faster training with ~60% less VRAM compared to traditional methodologies.<\/span><\/p>\n <\/p>\n Key Information for Gemma 4 E2B Fine-Tuning<\/b><\/span><\/p>\n Model ID:<\/b><\/span> google\/gemma-4-E2B-it or unsloth\/gemma-4-E2B-it-unsloth-bnb-4bit<\/span><\/p>\n<\/li>\n Requirements:<\/b><\/span> 8GB VRAM (minimum for QLoRA\/4-bit).<\/span><\/p>\n<\/li>\n Frameworks:<\/b><\/span> Unsloth, Hugging Face Transformers, TRL (Transformer Reinforcement Learning).<\/span><\/p>\n<\/li>\n Technique:<\/b><\/span> QLoRA (Quantized LoRA) is recommended for efficient local tuning.<\/span><\/p>\n<\/li>\n Context Window:<\/b><\/span> Supports up to 256K tokens.<\/span><\/p>\n<\/li>\n <\/p>\n<\/li>\n<\/ul>\n Fine-Tuning Steps (using Unsloth)<\/b><\/span><\/p>\n Unsloth provides optimized notebooks for training.<\/span><\/p>\n Environment Setup:<\/b><\/span> Install Unsloth and necessary dependencies.<\/span><\/p>\n<\/li>\n<\/ol>\n python<\/span><\/p>\n pip install “unsloth[colab-new] @ git+https:\/\/github.com”<\/span><\/p>\n pip install –no-deps “xformers<0.0.27” “trl<0.9.0” peft accelerate bitsandbytes<\/span><\/p>\n Load Model and Tokenizer:<\/b><\/span> Use 4-bit quantization to save memory.<\/span><\/p>\n<\/li>\n<\/ol>\n python<\/span><\/p>\n from unsloth import FastLanguageModel<\/span><\/p>\n import torch<\/span><\/p>\n model, tokenizer = FastLanguageModel.from_pretrained(<\/span><\/p>\n model_name = “unsloth\/gemma-4-E2B-it-unsloth-bnb-4bit”,<\/span><\/p>\n max_seq_length = 2048,<\/span><\/p>\n dtype = None,<\/span><\/p>\n load_in_4bit = True,<\/span><\/p>\n )<\/span><\/p>\n Apply LoRA Adapters:<\/b><\/span> Apply Parameter-Efficient Fine-Tuning (PEFT) to adapt only a fraction of the parameters.<\/span><\/p>\n<\/li>\n<\/ol>\n python<\/span><\/p>\n model = FastLanguageModel.get_peft_model(<\/span><\/p>\n model,<\/span><\/p>\n r = 16, <\/span># Rank<\/i><\/span><\/p>\n target_modules = [“q_proj”, “k_proj”, “v_proj”, “o_proj”,<\/span><\/p>\n “gate_proj”, “up_proj”, “down_proj”,],<\/span><\/p>\n lora_alpha = 16,<\/span><\/p>\n lora_dropout = 0,<\/span><\/p>\n bias = “none”,<\/span><\/p>\n use_gradient_checkpointing = “unsloth”,<\/span><\/p>\n )<\/span><\/p>\n Data Preparation:<\/b><\/span> Format your dataset (e.g., chat templates for instruction tuning).<\/span><\/p>\n<\/li>\n Train the Model:<\/b><\/span> Use the SFTTrainer (Supervised Fine-tuning Trainer) from TRL.<\/span><\/p>\n<\/li>\n Save the Model:<\/b><\/span> Save the trained LoRA adapters or merge them into a 16-bit model for inference.<\/span><\/p>\n<\/li>\n<\/ol>\n <\/p>\n Best Practices and Considerations<\/b><\/span><\/p>\n Unsloth Bug Fixes:<\/b><\/span> Ensure you use the latest Unsloth version, as they fixed issues with gradient accumulation (preventing losses from exploding) and inference bugs in Gemma 4.<\/span><\/p>\n<\/li>\n Multimodal Inputs:<\/b><\/span> To train with image or audio data, use AutoModelForMultimodalLM rather than AutoModelForCausalLM.<\/span><\/p>\n<\/li>\n Memory Management:<\/b><\/span> If you encounter Out of Memory (OOM) errors, reduce per_device_train_batch_size to 1 and increase gradient_accumulation_steps.<\/span><\/p>\n<\/li>\n Data Quality:<\/b><\/span> Use specialized datasets (e.g., function calling\/tool use) for specialized tasks.<\/span><\/p>\n<\/li>\n<\/ul>\n <\/p>\n gemma4 e2b \ud30c\uc778\ud130\ub2dd<\/span><\/p>\n Gemma 4 E2B(\uc2e4\uc9c8\uc801 2B) \ubaa8\ub378\uc758 \ubbf8\uc138 \uc870\uc815\uc740 \ub9e4\uc6b0 \uc811\uadfc\uc131\uc774 \ub192\uc544, 8GB VRAM\ub9cc\uc73c\ub85c\ub3c4 \uc18c\ube44\uc790\uc6a9 \ud558\ub4dc\uc6e8\uc5b4\uc5d0\uc11c \ub85c\uceec \uad50\uc721\uc744 \ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. Gemma 4 E2B\ub294 Per-Layer Embeddings(PLE)\ub97c \ud6a8\uc728\uc801\uc73c\ub85c \uc0ac\uc6a9\ud558\uc5ec \uc124\uacc4\ub41c \uba40\ud2f0\ubaa8\ub2ec \ubaa8\ub378(\ud14d\uc2a4\ud2b8, \uc774\ubbf8\uc9c0, \uc624\ub514\uc624)\uc785\ub2c8\ub2e4.<\/span><\/p>\n <\/p>\n Gemma 4 E2B\ub97c \ubbf8\uc138 \uc870\uc815\ud558\ub294 \ub370 \uac00\uc7a5 \ucd94\ucc9c\ub418\ub294 \ubc29\ubc95\uc740 <\/span>Unsloth<\/b><\/span>\ub97c \uc0ac\uc6a9\ud558\ub294 \uac83\uc73c\ub85c, \uc804\ud1b5\uc801\uc778 \ubc29\ubc95\ub860\uc5d0 \ube44\ud574 \uc57d 1.5\ubc30 \ube60\ub978 \ud559\uc2b5\uacfc \uc57d 60% \uc801\uc740 VRAM\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4.<\/span><\/p>\n <\/p>\n Gemma 4 E2B \ubbf8\uc138 \uc870\uc815\uc5d0 \ub300\ud55c \uc8fc\uc694 \uc815\ubcf4<\/b><\/span><\/p>\n 1. <\/span>\ubaa8\ub378 ID:<\/b><\/span> google\/gemma-4-E2B-it \ub610\ub294 unsloth\/gemma-4-E2B-it-unsloth-bnb-4bit<\/span><\/p>\n 2. <\/span>\uc694\uad6c \uc0ac\uc591:<\/b><\/span> 8GB VRAM(QLoRA\/4\ube44\ud2b8 \ucd5c\uc18c \uc0ac\uc591).<\/span><\/p>\n 3. <\/span>\ud504\ub808\uc784\uc6cc\ud06c:<\/b><\/span> \uc5b8\uc2ac\ub85c\uc2a4, \ud3ec\uc639 \ud398\uc774\uc2a4 \ud2b8\ub79c\uc2a4\ud3ec\uba38, TRL(\ud2b8\ub79c\uc2a4\ud3ec\uba38 \uac15\ud654 \ud559\uc2b5).<\/span><\/p>\n 4. <\/span>\uae30\ubc95:<\/b><\/span> \ud6a8\uc728\uc801\uc778 \ub85c\uceec \ud29c\ub2dd\uc744 \uc704\ud574 QLoRA(\uc591\uc790\ud654 LoRA)\ub97c \uad8c\uc7a5\ud569\ub2c8\ub2e4.<\/span><\/p>\n 5. <\/span>\ucee8\ud14d\uc2a4\ud2b8 \uc708\ub3c4\uc6b0:<\/b><\/span> \ucd5c\ub300 256K \ud1a0\ud070\uc744 \uc9c0\uc6d0\ud569\ub2c8\ub2e4.<\/span><\/p>\n 6.<\/span><\/p>\n \ubbf8\uc138 \uc870\uc815 \ub2e8\uacc4 (Unsloth \uc0ac\uc6a9)<\/b><\/span><\/p>\n Unsloth\ub294 \ud6c8\ub828\uc744 \uc704\ud55c \ucd5c\uc801\ud654\ub41c \ub178\ud2b8\ubd81\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4.<\/span><\/p>\n 1. <\/span>\ud658\uacbd \uc124\uc815:<\/b><\/span> Unsloth \ubc0f \ud544\uc694\ud55c \uc758\uc874\uc131 \uc124\uce58.<\/span><\/p>\n \ud30c\uc774\uc36c<\/span><\/p>\n PIP install “unsloth[colab-new] @ git+https:\/\/github.com”<\/span><\/p>\n PIP \uc124\uce58 –no-deps “xformers<0.0.27” “trl<0.9.0” peft accelerate bitsandbytes<\/span><\/p>\n 2. <\/span>\ub85c\ub4dc \ubaa8\ub378 \ubc0f \ud1a0\ud070\ub77c\uc774\uc800:<\/b><\/span> \uba54\ubaa8\ub9ac\ub97c \uc808\uc57d\ud558\uae30 \uc704\ud574 4\ube44\ud2b8 \uc591\uc790\ud654\ub97c \uc0ac\uc6a9\ud558\uc138\uc694.<\/span><\/p>\n \ud30c\uc774\uc36c<\/span><\/p>\n unsloth\uc5d0\uc11c \uac00\uc838\uc624\ub294 FastLanguageModel\uc5d0\uc11c<\/span><\/p>\n \uc218\uc785 \ud1a0\uce58<\/span><\/p>\n model, tokenizer = FastLanguageModel.from_pretrained(<\/span><\/p>\n model_name = “unsloth\/gemma-4-E2B-it-unsloth-bnb-4bit”,<\/span><\/p>\n max_seq_length = 2048,<\/span><\/p>\n dtype = \uc5c6\uc74c,<\/span><\/p>\n load_in_4bit = \ucc38,<\/span><\/p>\n )<\/span><\/p>\n 3. <\/span>LoRA \uc5b4\ub311\ud130 \uc801\uc6a9:<\/b><\/span> \ud30c\ub77c\ubbf8\ud130 \ud6a8\uc728\uc801 \ubbf8\uc138 \uc870\uc815(PEFT)\uc744 \uc801\uc6a9\ud558\uc5ec \uc77c\ubd80 \ub9e4\uac1c\ubcc0\uc218\ub9cc \uc801\uc751\uc2dc\ud0a4\uc138\uc694.<\/span><\/p>\n \ud30c\uc774\uc36c<\/span><\/p>\n \ubaa8\ub378 = FastLanguageModel.get_peft_model(<\/span><\/p>\n \ubaa8\ub378,<\/span><\/p>\n r = 16, <\/span># \ub7ad\ud06c<\/i><\/span><\/p>\n target_modules = [“q_proj”, “k_proj”, “v_proj”, “o_proj”,<\/span><\/p>\n “gate_proj”, “up_proj”, “down_proj”,],<\/span><\/p>\n lora_alpha = 16,<\/span><\/p>\n lora_dropout = 0,<\/span><\/p>\n \ud3b8\ud5a5 = “\uc5c6\uc74c”,<\/span><\/p>\n use_gradient_checkpointing = “\ub098\ud0dc \ud574\uc81c”,<\/span><\/p>\n\n
\n
\n
\n
\n
\n