diff --git a/README.md b/README.md index 6fc75e3..a3e85f1 100644 --- a/README.md +++ b/README.md @@ -76,24 +76,7 @@ Now you can start with ModelScope or Transformers. #### 🤗 Transformers -To use Qwen-7B for the inference, all you need to do is to input a few lines of codes as demonstrated below: - -```python -from transformers import AutoModelForCausalLM, AutoTokenizer -from transformers.generation import GenerationConfig - -tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen-7B", trust_remote_code=True) -model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen-7B", device_map="auto", trust_remote_code=True).eval() -model.generation_config = GenerationConfig.from_pretrained("Qwen/Qwen-7B", trust_remote_code=True) # 可指定不同的生成长度、top_p等相关超参 - -inputs = tokenizer('蒙古国的首都是乌兰巴托(Ulaanbaatar)\n冰岛的首都是雷克雅未克(Reykjavik)\n埃塞俄比亚的首都是', return_tensors='pt') -inputs = inputs.to('cuda:0') -pred = model.generate(**inputs) -print(tokenizer.decode(pred.cpu()[0], skip_special_tokens=True)) -# 蒙古国的首都是乌兰巴托(Ulaanbaatar)\n冰岛的首都是雷克雅未克(Reykjavik)\n埃塞俄比亚的首都是亚的斯亚贝巴(Addis Ababa)... -``` - -Running Qwen-7B-Chat is also simple. We provide you with an example of IPython to show how to interactive with the model. +To use Qwen-7B-chat for the inference, all you need to do is to input a few lines of codes as demonstrated below: ```python >>> from transformers import AutoModelForCausalLM, AutoTokenizer @@ -127,6 +110,27 @@ Running Qwen-7B-Chat is also simple. We provide you with an example of IPython t 《奋斗创业:一个年轻人的成功之路》 ``` + +Running Qwen-7B is also simple. +
+ Running Qwen-7B + +```python +from transformers import AutoModelForCausalLM, AutoTokenizer +from transformers.generation import GenerationConfig + +tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen-7B", trust_remote_code=True) +model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen-7B", device_map="auto", trust_remote_code=True).eval() +model.generation_config = GenerationConfig.from_pretrained("Qwen/Qwen-7B", trust_remote_code=True) # 可指定不同的生成长度、top_p等相关超参 + +inputs = tokenizer('蒙古国的首都是乌兰巴托(Ulaanbaatar)\n冰岛的首都是雷克雅未克(Reykjavik)\n埃塞俄比亚的首都是', return_tensors='pt') +inputs = inputs.to('cuda:0') +pred = model.generate(**inputs) +print(tokenizer.decode(pred.cpu()[0], skip_special_tokens=True)) +# 蒙古国的首都是乌兰巴托(Ulaanbaatar)\n冰岛的首都是雷克雅未克(Reykjavik)\n埃塞俄比亚的首都是亚的斯亚贝巴(Addis Ababa)... +``` +
+ #### 🤖 ModelScope ModelScope is an opensource platform for Model-as-a-Service (MaaS), which provides flexible and cost-effective model service to AI developers. Similarly, you can run the models with ModelScope as shown below: