From 79e166e7b7cb61cd4837c4ce657db41847258fd9 Mon Sep 17 00:00:00 2001 From: Ren Xuancheng Date: Tue, 12 Mar 2024 15:50:23 +0800 Subject: [PATCH] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 2b132d1..dd5c64d 100644 --- a/README.md +++ b/README.md @@ -620,7 +620,7 @@ We also measure the inference speed and GPU memory usage with different settings ### Usage Now we provide the official training script, `finetune.py`, for users to finetune the pretrained model for downstream applications in a simple fashion. Additionally, we provide shell scripts to launch finetuning with no worries. This script supports the training with [DeepSpeed](https://github.com/microsoft/DeepSpeed) and [FSDP](https://engineering.fb.com/2021/07/15/open-source/fsdp/). The shell scripts that we provide use DeepSpeed (Note: this may have conflicts with the latest version of pydantic and you should use make sure `pydantic<2.0`) and Peft. You can install them by: ```bash -pip install peft deepspeed +pip install peft<0.8.0 deepspeed ``` To prepare your training data, you need to put all the samples into a list and save it to a json file. Each sample is a dictionary consisting of an id and a list for conversation. Below is a simple example list with 1 sample: