{"id":11183,"date":"2025-03-30T23:25:14","date_gmt":"2025-03-30T14:25:14","guid":{"rendered":"https:\/\/www.stechstar.com\/user\/wordpress\/?p=11183"},"modified":"2026-08-20T03:42:15","modified_gmt":"2026-08-19T18:42:15","slug":"%ec%9d%b8%ea%b3%b5%ec%a7%80%eb%8a%a5-%ea%b8%b0%ec%88%a0-pytorch-pytorch-llm-training-pipeline-pytorch%eb%a1%9c-chatgpt%eb%a7%8c%eb%93%a4%ea%b8%b0-%ea%b8%b0%eb%b3%b8%ec%98%88%ec%a0%9c","status":"publish","type":"post","link":"https:\/\/www.stechstar.com\/user\/wordpress\/%ec%9d%b8%ea%b3%b5%ec%a7%80%eb%8a%a5-%ea%b8%b0%ec%88%a0-pytorch-pytorch-llm-training-pipeline-pytorch%eb%a1%9c-chatgpt%eb%a7%8c%eb%93%a4%ea%b8%b0-%ea%b8%b0%eb%b3%b8%ec%98%88%ec%a0%9c\/","title":{"rendered":"[\uc778\uacf5\uc9c0\ub2a5 \uae30\uc220]\u00a0 [Pytorch] Pytorch &#038; LLM Training Pipeline (pytorch\ub85c chatgpt\ub9cc\ub4e4\uae30, \uae30\ubcf8\uc608\uc81c)"},"content":{"rendered":"<h1>[\uc778\uacf5\uc9c0\ub2a5 \uae30\uc220]&nbsp; [Pytorch] Pytorch &amp; LLM Training Pipeline (pytorch\ub85c chatgpt\ub9cc\ub4e4\uae30, \uae30\ubcf8\uc608\uc81c)<\/h1>\n<h1 id=\"pytorch\">Pytorch<\/h1>\n<h2 id=\"pytorych\">Pytorych<\/h2>\n<h3 id=\"what\">What<\/h3>\n<p>Pytorch\ub294 Opensource Machine Learning(ML) Framework\ub85c, Python \uc5b8\uc5b4\uc640 Torch Library\ub97c \uae30\ubc18\uc73c\ub85c \ud55c\ub2e4. Torch\ub294 Opensource ML Library\ub85c, Deep learning\uc744 \uc774\uc6a9\ud558\uae30 \uc704\ud574\uc11c Lua Scripting\uc73c\ub85c \uc791\uc131\ub418\uc5c8\ub2e4. \ud604\uc7ac, DL \uc5f0\uad6c\ub97c \uc704\ud574\uc11c \uac00\uc7a5 \uc120\ud638\ub418\ub294 \ud50c\ub7ab\ud3fc\uc774\ub2e4.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.stechstar.com\/user\/zbxe\/files\/attach\/images\/92879\/952\/111\/658edd7a8d59ee53e587929c18a08df8.png\" width=\"768\" height=\"491\" data-autoattach=\"success\" data-evernote-lazy-src=\".\/files\/attach\/images\/92879\/952\/111\/658edd7a8d59ee53e587929c18a08df8.png\" data-cke-saved-src=\".\/files\/attach\/images\/92879\/952\/111\/658edd7a8d59ee53e587929c18a08df8.png\"><\/p>\n<h3 id=\"how-to-use\">How to use?<\/h3>\n<h4 id=\"tools\">Tools<\/h4>\n<p>Pytorch\ub294 \ud074\ub77c\uc6b0\ub4dc\uc640 \ub85c\uceec, \ub450 \uac00\uc9c0 \ud658\uacbd\uc5d0\uc11c \ud65c\uc6a9\ud560 \uc218 \uc788\ub2e4.<\/p>\n<ul>\n<li>In the cloud: Pytorch\ub97c \uc811\uadfc\ud558\uae30 \uac00\uc7a5 \uc26c\uc6b4 \ubc29\ubc95\uc73c\ub85c, Microsoft\ub97c \ud1b5\ud574\uc11c \uc811\uadfc\ud558\uac70\ub098 Google Colab\uc5d0\uc11c \uc2e4\ud589\ud574 \ubcfc \uc218 \uc788\ub2e4. \ub9c1\ud06c\uc5d0\ub9cc \uc811\uadfc\ud55c\ub2e4\uba74 Microsoft\uc640 Colab\uc5d0 \uc791\uc131\ub41c Notebook\uc5d0\uc11c \ucf54\ub4dc\ub97c \uc2e4\ud589\ud560 \uc218 \uc788\ub2e4.<\/li>\n<li>Locally: \ub85c\uceec\uc5d0\uc11c\ub294 PyTorch and TorchVision \ub4f1\uc744 \uc6b0\uc120 \uc124\uce58\ud574\uc57c\ud55c\ub2e4. \uc124\uce58 \uac00\uc774\ub4dc(installation instructions)\ub97c \ub530\ub77c\uc11c \uc124\uce58\ud560 \uc218 \uc788\ub2e4. \ub2e4\uc591\ud55c \ubc84\uc804\uc774 \ud648\ud398\uc774\uc9c0\uc5d0 \uac8c\uc7ac\ub418\uc5b4\uc788\ub2e4. Notebook\uc744 \ud65c\uc6a9\ud558\uac70\ub098 IDE()\uc5d0\uc11c \ucf54\ub4dc\ub97c \uc791\uc131\ud558\uc5ec \uc2e4\ud589\ud558\uba74 \ub41c\ub2e4.<\/li>\n<\/ul>\n<h2 id=\"pipieline\">Pipieline<\/h2>\n<h3 id=\"01-prerequisites\">01 Prerequisites<\/h3>\n<p>Pytorch\ub294 python \ud504\ub85c\uadf8\ub798\ubc0d\uc744 \ud1b5\ud574,\ub525\ub7ec\ub2dd\uacfc \ud2b8\ub79c\uc2a4\ud3ec\uba38 \uac1c\ub150\uc744 \uad6c\ud604\ud574\uc57c\ud55c\ub2e4.<\/p>\n<h3 id=\"02-library-installation\">02 Library installation<\/h3>\n<p><code>trl<\/code>&nbsp;: Transformer language odel\uc744 Reinforcement Leanring\uc73c\ub85c \ud559\uc2b5\ud558\ub294\ub370 \uc0ac\uc6a9\ud55c\ub2e4.<br \/>\n<code>peft<\/code>&nbsp;: Prameter-effiecient Fine-tuning(PEFT)\ubc29\ubc95\uc744 \ud1b5\ud574\uc11c \ud6a8\uc728\uc801\uc73c\ub85c pre-treined model\uc744 \uc870\uc815\ud55c\ub2e4.<br \/>\n<code>torrch<\/code>&nbsp;: Opensource ML library<br \/>\n<code>datasets<\/code>&nbsp;: ML datasets\uc744 \ub85c\ub529(loading) \ub610\ub294 \ub2e4\uc6b4\ub85c\ub529(downloading)\ud558\uae30 \uc704\ud574 \ud558\uc6a9\ud55c\ub2e4.<br \/>\n<code>Trasformers<\/code>&nbsp;: Hugging Face\uac00 \uac1c\ubc1c\ud55c \ub77c\uc774\ube0c\ub7ec\ub9ac\ub85c, \ubd84\ub958\/\uc694\uc57d\/\ubc88\uc5ed \ub4f1\uacfc \uac19\uc740 \ub2e4\uc591\ud55c \ud14d\uc2a4\ud2b8\ub97c \uc704\ud55c task\ub97c \uc218\ud589\ud558\uae30 \uc704\ud574 \uc218\ucc9c \uac1c\uc758 PLM(pre-trained model)\ubaa8\ub378\uc744 \uc81c\uacf5\ud55c\ub2e4.<\/p>\n<h4 id=\"02-1-pip-\ub97c-\ud1b5\ud574\uc11c-pythong-package-\ub2e4\uc6b4\ub85c\ub4dc\ud55c\ub2e4\">02-1 pip \ub97c \ud1b5\ud574\uc11c pythong package \ub2e4\uc6b4\ub85c\ub4dc\ud55c\ub2e4.<\/h4>\n<pre class=\"language-python\"><code class=\"language-python\"><span class=\"token operator\">%<\/span><span class=\"token operator\">%<\/span>bash\npip <span class=\"token operator\">-<\/span>q install trl\npip <span class=\"token operator\">-<\/span>q install peft\npip <span class=\"token operator\">-<\/span>q install torch\npip <span class=\"token operator\">-<\/span>q install datasets\npip <span class=\"token operator\">-<\/span>q install transformers<\/code><\/pre>\n<h4 id=\"02-1-import-\ud1b5\ud574\uc11c-pythong-package\ub97c-\uac00\uc838\uc628\ub2e4\">02-1 import \ud1b5\ud574\uc11c pythong package\ub97c \uac00\uc838\uc628\ub2e4<\/h4>\n<pre class=\"language-python\"><code class=\"language-python\"><span class=\"token keyword\">import<\/span> torch\n<span class=\"token keyword\">from<\/span> trl <span class=\"token keyword\">import<\/span> SFTTrainer\n<span class=\"token keyword\">from<\/span> datasets <span class=\"token keyword\">import<\/span> load_dataset\n<span class=\"token keyword\">from<\/span> peft <span class=\"token keyword\">import<\/span> LoraConfig<span class=\"token punctuation\">,<\/span> get_peft_model<span class=\"token punctuation\">,<\/span> prepare_model_for_int8_training\n<span class=\"token keyword\">from<\/span> transformers <span class=\"token keyword\">import<\/span> AutoModelForCausalLM<span class=\"token punctuation\">,<\/span> AutoTokenizer<span class=\"token punctuation\">,<\/span> TrainingArguments<\/code><\/pre>\n<h3 id=\"03-data-loading-and-preparation\">03 Data Loading and Preparation<\/h3>\n<h4 id=\"03-01-data-loading\">03-01 Data loading<\/h4>\n<p>\ud5c8\uae45\ud398\uc774\uc2a4\uc758 \uacf5\uac1c \ub370\uc774\ud130\uc14b\uc744 \ub85c\ub4dc\ud55c\ub2e4.<br \/>\n<code>load_dataset<\/code>&nbsp;: \ub370\uc774\ud130 \ub85c\ub4dc<br \/>\n<code>dataset_name<\/code>&nbsp;: \ub85c\ub4dc\ud558\uae30 \uc704\ud55c \ub370\uc774\ud130\uc14b \uc774\ub984<br \/>\n<code>split<\/code>&nbsp;parameter\ub97c &#8220;<code>train<\/code>&#8220;\ub85c \uc124\uc815\ud558\uc5ec \ud559\uc2b5(traing)\ub370\uc774\ud130\ub85c\ub9cc \uc124\uc815\ub41c \ub370\uc774\ud130 \ub85c\ub4dc<\/p>\n<pre class=\"language-python\"><code class=\"language-python\">dataset_name<span class=\"token operator\">=<\/span><span class=\"token string\">\"tatsu-lab\/alpaca\"<\/span>\ntrain_dataset <span class=\"token operator\">=<\/span> load_dataset<span class=\"token punctuation\">(<\/span>dataset_name<span class=\"token punctuation\">,<\/span> split<span class=\"token operator\">=<\/span><span class=\"token string\">\"train\"<\/span><span class=\"token punctuation\">)<\/span>\n<span class=\"token keyword\">print<\/span><span class=\"token punctuation\">(<\/span>train_dataset<span class=\"token punctuation\">)<\/span><\/code><\/pre>\n<h4 id=\"03-02-converting-the-dictionary-into-a-pandas-dataframe\">03-02 Converting the dictionary into a pandas dataframe<\/h4>\n<p>\ub85c\ub4dc\ud55c \ub370\uc774\ud130\ub294 dictionary \ud0c0\uc785\uc73c\ub85c, pandas\uc758 dataframe\uc73c\ub85c \ubcc0\ud658\ud55c\ub2e4.<\/p>\n<p><code>.to_pandas()<\/code>&nbsp;: train_dataset\uc744 pandas dataframe\uc73c\ub85c \ubcc0\ud658<\/p>\n<pre class=\"language-python\"><code class=\"language-python\">pandas_format <span class=\"token operator\">=<\/span> train_dataset<span class=\"token punctuation\">.<\/span>to_pandas<span class=\"token punctuation\">(<\/span><span class=\"token punctuation\">)<\/span>\ndisplay<span class=\"token punctuation\">(<\/span>pandas_format<span class=\"token punctuation\">.<\/span>head<span class=\"token punctuation\">(<\/span><span class=\"token punctuation\">)<\/span><span class=\"token punctuation\">)<\/span><\/code><\/pre>\n<h3 id=\"04-model-training\">04 Model Training<\/h3>\n<ul>\n<li>Pre-trained Model : Pretrained \ubaa8\ub378<\/li>\n<li>Tokenizer : \ub370\uc774\ud130\uc758 \ud1a0\ud070\ud654(tokenization)\ub97c \uc704\ud574 \uc0ac\uc6a9\ud558\uba70, PLM\uacfc tokenizer\ub97c \uac00\uc838\uc634<\/li>\n<li><code>AutoModelForCausalLM<\/code>&nbsp;:\n<ul>\n<li><code>from_pretrained<\/code>&nbsp;\uba54\uc11c\ub4dc\ub85c PLM\uc744 \uac00\uc838\uc624\uae30 \uc704\ud574 \uc0ac\uc6a9<\/li>\n<li><code>pretrained_model_name<\/code>\uc740 PLM \ubaa8\ub378\uc744 \uc9c0\uc815\ud558\uba70,&nbsp;<code>data type<\/code>\uc740 \ubaa8\ub378\uc758 tensor\ub97c<code>torch.bfloat16<\/code>\ub85c \uc124\uc815<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>*<code>AutoTokenizer<\/code>:<\/p>\n<ul>\n<li><code>from_pretrained<\/code>&nbsp;: \uac00 PLM\uc5d0 \ub9de\ub294 tokenizer\ub97c \uac00\uc838\uc624\uae30 \uc704\ud574 \uc0ac\uc6a9\ud558\ub294 \uba54\uc11c\ub4dc<\/li>\n<li><code>pretrained_model_name<\/code>\uacfc&nbsp;<code>trust_remote_code<\/code>\ub97c true\ub85c \uc124\uc815<\/li>\n<\/ul>\n<pre class=\"language-python\"><code class=\"language-python\">pretrained_model_name <span class=\"token operator\">=<\/span> <span class=\"token string\">\"Salesforce\/xgen-7b-8k-base\"<\/span>\nmodel <span class=\"token operator\">=<\/span> AutoModelForCausalLM<span class=\"token punctuation\">.<\/span>from_pretrained<span class=\"token punctuation\">(<\/span>pretrained_model_name<span class=\"token punctuation\">,<\/span> torch_dtype<span class=\"token operator\">=<\/span>torch<span class=\"token punctuation\">.<\/span>bfloat16<span class=\"token punctuation\">)<\/span>\ntokenizer <span class=\"token operator\">=<\/span> AutoTokenizer<span class=\"token punctuation\">.<\/span>from_pretrained<span class=\"token punctuation\">(<\/span>pretrained_model_name<span class=\"token punctuation\">,<\/span> trust_remote_code<span class=\"token operator\">=<\/span><span class=\"token boolean\">True<\/span><span class=\"token punctuation\">)<\/span><\/code><\/pre>\n<h3 id=\"05-training-configuration\">05 Training configuration<\/h3>\n<p>Training\uc744 \uc704\ud574&nbsp;<code>training arguments(\uc778\uc218)<\/code>\uc640&nbsp;<code>training configurations(\uad6c\uc131)<\/code>\uc774 \ud544\uc694\ud558\ub2e4. \uc774\ub294&nbsp;<code>TrainingArguments<\/code>\ub85c \uad6c\uc131\ud558\uba70,&nbsp;<code>LoraConfig model<\/code>\uacfc&nbsp;<code>SFTTrainer model<\/code>\uc758 \uc778\uc2a4\ud134\uc2a4\uc774\ub2e4.<\/p>\n<h3 id=\"06-trainingarguments\">06 TrainingArguments<\/h3>\n<p><code>output_dir<\/code>&nbsp;\ud559\uc2b5 \ubaa8\ub378 \uc800\uc7a5 \uc704\uce58<br \/>\n<code>per_device_train_batch_size<\/code>&nbsp;\ud55c \ud559\uc2b5\uc5d0 \uc0ac\uc6a9\ud558\uae30 \uc704\ud55c \ud559\uc2b5 \uc0d8\ud50c\uc758 \uc0ac\uc774\uc988<br \/>\n<code>optim<\/code>&nbsp;\uc635\ud2f0\ub9c8\uc774\uc800<br \/>\n<code>logging_steps<\/code>&nbsp;\ub85c\uae45\uc744 \uc704\ud55c \ube48\ub3c4\ub85c, n\ubc88\uc758 \uc2a4\ud15d\ub9c8\ub2e4 \uae30\ub85d<br \/>\n<code>learning_rate<\/code>&nbsp;\uc635\ud2f0\ub9c8\uc774\uc800\uc758 learning rate \uc124\uc815<br \/>\n<code>warmup_ratio<\/code><br \/>\n<code>lr_scheduler_type<\/code><br \/>\n<code>num_train_epochs<\/code><br \/>\n<code>save_strategy<\/code><\/p>\n<pre class=\"language-python\"><code class=\"language-python\">model_training_args <span class=\"token operator\">=<\/span> TrainingArguments<span class=\"token punctuation\">(<\/span>\n       output_dir<span class=\"token operator\">=<\/span><span class=\"token string\">\"xgen-7b-8k-base-fine-tuned\"<\/span><span class=\"token punctuation\">,<\/span>\n       per_device_train_batch_size<span class=\"token operator\">=<\/span><span class=\"token number\">4<\/span><span class=\"token punctuation\">,<\/span>\n       optim<span class=\"token operator\">=<\/span><span class=\"token string\">\"adamw_torch\"<\/span><span class=\"token punctuation\">,<\/span>\n       logging_steps<span class=\"token operator\">=<\/span><span class=\"token number\">80<\/span><span class=\"token punctuation\">,<\/span>\n       learning_rate<span class=\"token operator\">=<\/span><span class=\"token number\">2e<\/span><span class=\"token operator\">-<\/span><span class=\"token number\">4<\/span><span class=\"token punctuation\">,<\/span>\n       warmup_ratio<span class=\"token operator\">=<\/span><span class=\"token number\">0.1<\/span><span class=\"token punctuation\">,<\/span>\n       lr_scheduler_type<span class=\"token operator\">=<\/span><span class=\"token string\">\"linear\"<\/span><span class=\"token punctuation\">,<\/span>\n       num_train_epochs<span class=\"token operator\">=<\/span><span class=\"token number\">1<\/span><span class=\"token punctuation\">,<\/span>\n       save_strategy<span class=\"token operator\">=<\/span><span class=\"token string\">\"epoch\"<\/span>\n   <span class=\"token punctuation\">)<\/span><\/code><\/pre>\n<h3 id=\"07-loraconfig\">07 LoRAConfig<\/h3>\n<p>LoRa(Low-Rank Adaptation)\uc740 Hugging Face\uac00 \uac1c\ubc1c\ud55c PEFT\uc774\ub2e4. peft \ub77c\uc774\ube0c\ub7ec\ub97c \ud1b5\ud574\uc11c \uc0ac\uc6a9\ud560 \uc218 \uc788\ub2e4.<br \/>\n\ud604\uc7ac\uc758 \ud30c\uc774\ud504\ub77c\uc778\uc5d0\uc11c\ub294 \ud558\uc704 \ubcc0\ud658 \ud589\ub82c\uc744 16\uc73c\ub85c \uc124\uc815\ud55c\ub2e4. LoRa\uc758 \ub9e4\uac1c\ubcc0\uc218\uc758 \uc2a4\ucf00\uc77c\ub9c1 \uacc4\uc218\ub294 32\ub85c \uc11c\ub801\ud55c\ub2e4. \ub4dc\ub86d\uc544\uc6c3 \ube44\uc728\uc740 0.05\ub85c \uc124\uc815\ub418\uc5c8\ub2e4.<br \/>\n<code>CASULA_LM<\/code>\uc740&nbsp;<code>\uc778\uacfc\uc801 \uc5b8\uc5b4 \ubaa8\ub378(causual lanauge model)<\/code>\uc758 \uc18d\uc131\uc744 \ucd08\uae30\ud654\ud55c\ub2e4.<\/p>\n<h3 id=\"08-sfttrainer\">08 SFTTrainer<\/h3>\n<p>training data, tokenizer, \ubaa8\ub378\uc758 \ucd94\uac00 \uc815\ubcf4\ub97c \uc774\uc6a9\ud558\uc5ec \ud559\uc2b5\ud55c\ub2e4.<\/p>\n<p>\uc544\ub798\ub294 \ub85c\ub4dc\ud55c \ud14d\uc2a4\ud2b8 \uce7c\ub7fc\uc758 \uae38\uc774\ub97c \uceec\ub7fc\uc744 \ucd94\uac00\ud558\uc5ec \uc800\uc7a5\ud558\ub294 \uac83\uc774\ub2e4. \uc774 \ub370\uc774\ud130\uc14b\uc740 0~1000 \ud06c\uae30\uc758 \ud14d\uc2a4\ud2b8 \ubd84\ud3ec\uac00 \ud06c\ub2e4. (\ubd84\ud3ec\ub97c \ubcf4\uc5ec\uc8fc\ub294 \ud45c\ub294 \uc0dd\ub7b5)<br \/>\n1024\ubcf4\ub2e4 \ud070 \uac12\uc758 \ube44\uc728\uc744 \uad6c\ud55c\ub2e4.<\/p>\n<p><code>pandas_foramt<\/code>\uc740 \ud2b9\uc815 \uce7c\ub7fc\uc758 \ud06c\uae30\uac00 1024\ubcf4\ub2e4 \ud070\uac83\uc744 \ucc3e\uc544&nbsp;<code>mask<\/code>\ub85c \uc800\uc7a5<br \/>\n<code>percentage<\/code>\uc5d0 mask\uac00 &#8216;texst_length&#8217;\uceec\ub7fc\uc5d0\uc11c \ucc28\uc9c0\ud558\ub294 \ube44\uc728\uc744 \uad6c\ud568<\/p>\n<pre class=\"language-python\"><code class=\"language-python\">pandas_format<span class=\"token punctuation\">[<\/span><span class=\"token string\">'text_length'<\/span><span class=\"token punctuation\">]<\/span> <span class=\"token operator\">=<\/span> pandas_format<span class=\"token punctuation\">[<\/span><span class=\"token string\">'text'<\/span><span class=\"token punctuation\">]<\/span><span class=\"token punctuation\">.<\/span><span class=\"token builtin\">apply<\/span><span class=\"token punctuation\">(<\/span><span class=\"token builtin\">len<\/span><span class=\"token punctuation\">)<\/span>\n\nmask <span class=\"token operator\">=<\/span> pandas_format<span class=\"token punctuation\">[<\/span><span class=\"token string\">'text_length'<\/span><span class=\"token punctuation\">]<\/span> <span class=\"token operator\">&gt;<\/span> <span class=\"token number\">1024<\/span>\npercentage <span class=\"token operator\">=<\/span> <span class=\"token punctuation\">(<\/span>mask<span class=\"token punctuation\">.<\/span><span class=\"token builtin\">sum<\/span><span class=\"token punctuation\">(<\/span><span class=\"token punctuation\">)<\/span> <span class=\"token operator\">\/<\/span> pandas_format<span class=\"token punctuation\">[<\/span><span class=\"token string\">'text_length'<\/span><span class=\"token punctuation\">]<\/span><span class=\"token punctuation\">.<\/span>count<span class=\"token punctuation\">(<\/span><span class=\"token punctuation\">)<\/span><span class=\"token punctuation\">)<\/span> <span class=\"token operator\">*<\/span> <span class=\"token number\">100<\/span>\n\n<span class=\"token keyword\">print<\/span><span class=\"token punctuation\">(<\/span><span class=\"token string-interpolation\"><span class=\"token string\">f\"The percentage of text documents with a length greater than 1024 is: <\/span><span class=\"token interpolation\"><span class=\"token punctuation\">{<\/span>percentage<span class=\"token punctuation\">}<\/span><\/span><span class=\"token string\">%\"<\/span><\/span><span class=\"token punctuation\">)<\/span><\/code><\/pre>\n<ul>\n<li>\uc544\ub798 \ucf54\ub4dc\ub294&nbsp;<code>SFTTrainer<\/code>&nbsp;\uc758 \uc778\uc2a4\ud134\uc2a4\ub97c \ub9cc\ub4e4\uace0 \uc788\ub2e4.<br \/>\n<code>model=model<\/code>&nbsp;PLM\uc744 SFTTrainer\uc5d0 \uc804\ub2ec<br \/>\n<code>train_dataset=train_dataset<\/code>&nbsp;train dataset\uc744 SFTTrainer\uc5d0 \uc804\ub2ec<br \/>\n<code>dataset_text_field=\"text\"<\/code>&nbsp;\ud14d\uc2a4\ud2b8 \ub370\uc774\ud130\ub97c \ud3ec\ud568\ud558\ub294 dataset \ud544\ub4dc \uc9c0\uc815<br \/>\n<code>max_seq_length=1024<\/code>&nbsp;\ubaa8\ub378 \ub825\uc758 \ucd5c\ub300 \uc2dc\ud000\uc2a4 \uae38\uc774\ub97c \uc124\uc815<br \/>\n<code>tokenizer=tokenizer<\/code>&nbsp;tokenizer\ub97c SFTTrainer\uc5d0 \uc804\ub2ec<br \/>\n<code>args=model_training_args<\/code>&nbsp;\ubaa8\ub378 \ud2b8\ub808\uc774\ub2dd\uc744 \uc704\ud55c \uc778\uc218\ub97c SFTTrainer\uc5d0 \uc804\ub2ec<br \/>\n<code>packing=True<\/code>\ud6a8\uc728\uc801\uc778 \ud6c8\ub828\uc744 \uc704\ud55c \uc2dc\ud000\uc2a4 \ud328\ud0b9 \uc124\uc815<br \/>\n<code>peft_config=lora_peft_config<\/code>&nbsp;PEFT\uc5d0 loraConfig \uc124\uc815<\/li>\n<\/ul>\n<pre class=\"language-python\"><code class=\"language-python\">SFT_trainer <span class=\"token operator\">=<\/span> SFTTrainer<span class=\"token punctuation\">(<\/span>\n       model<span class=\"token operator\">=<\/span>model<span class=\"token punctuation\">,<\/span>\n       train_dataset<span class=\"token operator\">=<\/span>train_dataset<span class=\"token punctuation\">,<\/span>\n       dataset_text_field<span class=\"token operator\">=<\/span><span class=\"token string\">\"text\"<\/span><span class=\"token punctuation\">,<\/span>\n       max_seq_length<span class=\"token operator\">=<\/span><span class=\"token number\">1024<\/span><span class=\"token punctuation\">,<\/span>\n       tokenizer<span class=\"token operator\">=<\/span>tokenizer<span class=\"token punctuation\">,<\/span>\n       args<span class=\"token operator\">=<\/span>model_training_args<span class=\"token punctuation\">,<\/span>\n       packing<span class=\"token operator\">=<\/span><span class=\"token boolean\">True<\/span><span class=\"token punctuation\">,<\/span>\n       peft_config<span class=\"token operator\">=<\/span>lora_peft_config<span class=\"token punctuation\">,<\/span>\n   <span class=\"token punctuation\">)<\/span><\/code><\/pre>\n<h3 id=\"09-training-execution\">09 Training execution<\/h3>\n<p><code>tokenizer.pad_token =tokenizer.eos_token<\/code>&nbsp;padding token\uc744 eos_token\uc73c\ub85c \uc124\uc815(eos, end of sequence)<br \/>\n<code>model.resize_token_embeddings(len(tokenizer))<\/code>&nbsp;\ubaa8\ub378\uc758 \ud1a0\ud070 \uc784\ubca0\ub529 \uacc4\uce35\uc744 tokenizer \uae38\uc774\ub85c \uc124\uc815<br \/>\n<code>model=prepare_model_for_int8_training(model))<\/code>&nbsp;\uc591\uc790\ud654\ucc98\ub7fc, INT8 \uc815\ubc00\ub3c4\ub85c \ud2b8\ub808\uc774\ub2dd\ud560 \uc218 \uc788\ub3c4\ub85d \uc900\ube44<br \/>\n<code>model = get_peft_model (model, lora_peft_config)<\/code>&nbsp;\uc8fc\uc5b4\uc9c4 \ubaa8\ub378\uc744 PEFT \uad6c\uc131\uc5d0 \ub530\ub530 \uc870\uc815<br \/>\n<code>training_args = model_training_args<\/code>&nbsp;training_args\uc5d0 \ubbf8\ub9ac \uc815\uc758\ub41c training \uc778\uc218\ub97c \ud560\ub2f9<br \/>\n<code>trainer =SFT_Trainer<\/code>&nbsp;SFTTrainer \uc778\uc2a4\ud134\uc2a4\ub97c trainer\uc5d0 \uc800\uc7a5<br \/>\n<code>trainer.train()<\/code>&nbsp;(\uc704\uc758) \uc124\uc815\uc5d0 \ub530\ub77c \ubaa8\ub378\uc744 \ud559\uc2b5<\/p>\n<pre class=\"language-python\"><code class=\"language-python\">tokenizer<span class=\"token punctuation\">.<\/span>pad_token <span class=\"token operator\">=<\/span> tokenizer<span class=\"token punctuation\">.<\/span>eos_token\nmodel<span class=\"token punctuation\">.<\/span>resize_token_embeddings<span class=\"token punctuation\">(<\/span><span class=\"token builtin\">len<\/span><span class=\"token punctuation\">(<\/span>tokenizer<span class=\"token punctuation\">)<\/span><span class=\"token punctuation\">)<\/span>\nmodel <span class=\"token operator\">=<\/span> prepare_model_for_int8_training<span class=\"token punctuation\">(<\/span>model<span class=\"token punctuation\">)<\/span>\nmodel <span class=\"token operator\">=<\/span> get_peft_model<span class=\"token punctuation\">(<\/span>model<span class=\"token punctuation\">,<\/span> lora_peft_config<span class=\"token punctuation\">)<\/span>\ntraining_args <span class=\"token operator\">=<\/span> model_training_args\ntrainer <span class=\"token operator\">=<\/span> SFT_trainer\ntrainer<span class=\"token punctuation\">.<\/span>train<span class=\"token punctuation\">(<\/span><span class=\"token punctuation\">)<\/span><\/code><\/pre>\n<p><strong>Reference<\/strong><br \/>\n[1]&nbsp;<a href=\"https:\/\/pytorch.org\/tutorials\/beginner\/basics\/intro.html\" data-cke-saved-href=\"https:\/\/pytorch.org\/tutorials\/beginner\/basics\/intro.html\">https:\/\/pytorch.org\/tutorials\/beginner\/basics\/intro.html<\/a><br \/>\n[2]&nbsp;<a href=\"https:\/\/www.datacamp.com\/tutorial\/how-to-train-a-llm-with-pytorch\" data-cke-saved-href=\"https:\/\/www.datacamp.com\/tutorial\/how-to-train-a-llm-with-pytorch\">https:\/\/www.datacamp.com\/tutorial\/how-to-train-a-llm-with-pytorch<\/a><br \/>\n[3]&nbsp;<a href=\"https:\/\/github.com\/keitazoumana\/Medium-Articles-Notebooks\/blob\/main\/Train_your_LLM.ipynb\" data-cke-saved-href=\"https:\/\/github.com\/keitazoumana\/Medium-Articles-Notebooks\/blob\/main\/Train_your_LLM.ipynb\">https:\/\/github.com\/keitazoumana\/Medium-Articles-Notebooks\/blob\/main\/Train_your_LLM.ipynb<\/a><\/p>\n<p>[\ucd9c\ucc98]&nbsp;https:\/\/velog.io\/@jasmine_s2\/Pytorch-Pytorch-LLM-Training-Pipeline<\/p>\n<div class=\"pvc_clear\"><\/div>\n<p id=\"pvc_stats_11183\" class=\"pvc_stats all  \" data-element-id=\"11183\" style=\"\"><i class=\"pvc-stats-icon medium\" aria-hidden=\"true\"><svg aria-hidden=\"true\" focusable=\"false\" data-prefix=\"far\" data-icon=\"chart-bar\" role=\"img\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" viewBox=\"0 0 512 512\" class=\"svg-inline--fa fa-chart-bar fa-w-16 fa-2x\"><path fill=\"currentColor\" d=\"M396.8 352h22.4c6.4 0 12.8-6.4 12.8-12.8V108.8c0-6.4-6.4-12.8-12.8-12.8h-22.4c-6.4 0-12.8 6.4-12.8 12.8v230.4c0 6.4 6.4 12.8 12.8 12.8zm-192 0h22.4c6.4 0 12.8-6.4 12.8-12.8V140.8c0-6.4-6.4-12.8-12.8-12.8h-22.4c-6.4 0-12.8 6.4-12.8 12.8v198.4c0 6.4 6.4 12.8 12.8 12.8zm96 0h22.4c6.4 0 12.8-6.4 12.8-12.8V204.8c0-6.4-6.4-12.8-12.8-12.8h-22.4c-6.4 0-12.8 6.4-12.8 12.8v134.4c0 6.4 6.4 12.8 12.8 12.8zM496 400H48V80c0-8.84-7.16-16-16-16H16C7.16 64 0 71.16 0 80v336c0 17.67 14.33 32 32 32h464c8.84 0 16-7.16 16-16v-16c0-8.84-7.16-16-16-16zm-387.2-48h22.4c6.4 0 12.8-6.4 12.8-12.8v-70.4c0-6.4-6.4-12.8-12.8-12.8h-22.4c-6.4 0-12.8 6.4-12.8 12.8v70.4c0 6.4 6.4 12.8 12.8 12.8z\" class=\"\"><\/path><\/svg><\/i> <img loading=\"lazy\" decoding=\"async\" width=\"16\" height=\"16\" alt=\"Loading\" src=\"https:\/\/www.stechstar.com\/user\/wordpress\/wp-content\/plugins\/page-views-count\/ajax-loader-2x.gif\" border=0 \/><\/p>\n<div class=\"pvc_clear\"><\/div>\n","protected":false},"excerpt":{"rendered":"<p>[\uc778\uacf5\uc9c0\ub2a5 \uae30\uc220]&nbsp; [Pytorch] Pytorch &amp; LLM Training Pipeline (pytorch\ub85c chatgpt\ub9cc\ub4e4\uae30, \uae30\ubcf8\uc608\uc81c) Pytorch Pytorych What Pytorch\ub294 Opensource Machine Learning(ML) Framework\ub85c, Python \uc5b8\uc5b4\uc640 Torch Library\ub97c \uae30\ubc18\uc73c\ub85c \ud55c\ub2e4. Torch\ub294 Opensource ML Library\ub85c, Deep learning\uc744 \uc774\uc6a9\ud558\uae30 \uc704\ud574\uc11c Lua Scripting\uc73c\ub85c \uc791\uc131\ub418\uc5c8\ub2e4. \ud604\uc7ac, DL \uc5f0\uad6c\ub97c \uc704\ud574\uc11c \uac00\uc7a5 \uc120\ud638\ub418\ub294 \ud50c\ub7ab\ud3fc\uc774\ub2e4. How to use? Tools Pytorch\ub294 \ud074\ub77c\uc6b0\ub4dc\uc640 \ub85c\uceec, \ub450 \uac00\uc9c0 \ud658\uacbd\uc5d0\uc11c \ud65c\uc6a9\ud560 \uc218 [&hellip;]<\/p>\n<div class=\"pvc_clear\"><\/div>\n<p id=\"pvc_stats_11183\" class=\"pvc_stats all  \" data-element-id=\"11183\" style=\"\"><i class=\"pvc-stats-icon medium\" aria-hidden=\"true\"><svg aria-hidden=\"true\" focusable=\"false\" data-prefix=\"far\" data-icon=\"chart-bar\" role=\"img\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" viewBox=\"0 0 512 512\" class=\"svg-inline--fa fa-chart-bar fa-w-16 fa-2x\"><path fill=\"currentColor\" d=\"M396.8 352h22.4c6.4 0 12.8-6.4 12.8-12.8V108.8c0-6.4-6.4-12.8-12.8-12.8h-22.4c-6.4 0-12.8 6.4-12.8 12.8v230.4c0 6.4 6.4 12.8 12.8 12.8zm-192 0h22.4c6.4 0 12.8-6.4 12.8-12.8V140.8c0-6.4-6.4-12.8-12.8-12.8h-22.4c-6.4 0-12.8 6.4-12.8 12.8v198.4c0 6.4 6.4 12.8 12.8 12.8zm96 0h22.4c6.4 0 12.8-6.4 12.8-12.8V204.8c0-6.4-6.4-12.8-12.8-12.8h-22.4c-6.4 0-12.8 6.4-12.8 12.8v134.4c0 6.4 6.4 12.8 12.8 12.8zM496 400H48V80c0-8.84-7.16-16-16-16H16C7.16 64 0 71.16 0 80v336c0 17.67 14.33 32 32 32h464c8.84 0 16-7.16 16-16v-16c0-8.84-7.16-16-16-16zm-387.2-48h22.4c6.4 0 12.8-6.4 12.8-12.8v-70.4c0-6.4-6.4-12.8-12.8-12.8h-22.4c-6.4 0-12.8 6.4-12.8 12.8v70.4c0 6.4 6.4 12.8 12.8 12.8z\" class=\"\"><\/path><\/svg><\/i> <img loading=\"lazy\" decoding=\"async\" width=\"16\" height=\"16\" alt=\"Loading\" src=\"https:\/\/www.stechstar.com\/user\/wordpress\/wp-content\/plugins\/page-views-count\/ajax-loader-2x.gif\" border=0 \/><\/p>\n<div class=\"pvc_clear\"><\/div>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"ngg_post_thumbnail":0,"footnotes":""},"categories":[15,25,21,16,20],"tags":[],"class_list":["post-11183","post","type-post","status-publish","format-standard","hentry","category-stechstar-com-","category-25","category-21","category-16","category-20"],"a3_pvc":{"activated":true,"total_views":21,"today_views":1},"_links":{"self":[{"href":"https:\/\/www.stechstar.com\/user\/wordpress\/wp-json\/wp\/v2\/posts\/11183","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.stechstar.com\/user\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.stechstar.com\/user\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.stechstar.com\/user\/wordpress\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.stechstar.com\/user\/wordpress\/wp-json\/wp\/v2\/comments?post=11183"}],"version-history":[{"count":3,"href":"https:\/\/www.stechstar.com\/user\/wordpress\/wp-json\/wp\/v2\/posts\/11183\/revisions"}],"predecessor-version":[{"id":11945,"href":"https:\/\/www.stechstar.com\/user\/wordpress\/wp-json\/wp\/v2\/posts\/11183\/revisions\/11945"}],"wp:attachment":[{"href":"https:\/\/www.stechstar.com\/user\/wordpress\/wp-json\/wp\/v2\/media?parent=11183"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.stechstar.com\/user\/wordpress\/wp-json\/wp\/v2\/categories?post=11183"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.stechstar.com\/user\/wordpress\/wp-json\/wp\/v2\/tags?post=11183"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}