ptuning训练后无法加载预测数据[BUG/Help] <title>

[natureLanguageQing]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

训练好的ptuning模型，无法用cli_demo加载预测。发生最大token溢出情况。
tokenizer = AutoTokenizer.from_pretrained('ptuning/outputs/adgen-chatglm-6b-pt-8-dev/checkpoint-6000', trust_remote_code=True) model = AutoModel.from_pretrained('ptuning/outputs/adgen-chatglm-6b-pt-8-dev/checkpoint-6000', trust_remote_code=True).half().cuda() model = model.eval()

Expected Behavior

预测时发生错误

Traceback (most recent call last): File "cli_demo_trained.py", line 57, in <module> main() File "cli_demo_trained.py", line 42, in main for response, history in model.stream_chat(tokenizer, query, history=history): File "/usr/local/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 43, in generator_context response = gen.send(None) File "/root/.cache/huggingface/modules/transformers_modules/checkpoint-6000/modeling_chatglm.py", line 1281, in stream_chat for outputs in self.stream_generate(**inputs, **gen_kwargs): File "/usr/local/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 43, in generator_context response = gen.send(None) File "/root/.cache/huggingface/modules/transformers_modules/checkpoint-6000/modeling_chatglm.py", line 1356, in stream_generate model_inputs = self.prepare_inputs_for_generation(input_ids, **model_kwargs) File "/root/.cache/huggingface/modules/transformers_modules/checkpoint-6000/modeling_chatglm.py", line 1091, in prepare_inputs_for_generation mask_positions = [seq.index(mask_token) for seq in seqs] File "/root/.cache/huggingface/modules/transformers_modules/checkpoint-6000/modeling_chatglm.py", line 1091, in <listcomp> mask_positions = [seq.index(mask_token) for seq in seqs] ValueError: 130000 is not in list

Steps To Reproduce

在ptuning下运行train.sh生成模型。
通过cli_demo进行终端交互加载。
输入预测问题时返回错误。

Environment

- OS:
- Python:
- Transformers:
- PyTorch:
- CUDA Support (`python -c "import torch; print(torch.cuda.is_available())"`) :

Anything else?

No response

[duzx16]

之前的tokenizer config保存的时候有点问题，你可以先把tokenizer换成THUDM/chatglm-6b

[HkkSimple]

                                                                                             │

│ /mnt/data/rz/programe/glm/venv/lib/python3.8/site-packages/transformers/generation/utils.py:1406 │
│ in generate │
│ │
│ 1403 │ │ │ │ ) │
│ 1404 │ │ │ │
│ 1405 │ │ │ # 11. run greedy search │
│ ❱ 1406 │ │ │ return self.greedy_search( │
│ 1407 │ │ │ │ input_ids, │
│ 1408 │ │ │ │ logits_processor=logits_processor, │
│ 1409 │ │ │ │ stopping_criteria=stopping_criteria, │
│ │
│ /mnt/data/rz/programe/glm/venv/lib/python3.8/site-packages/transformers/generation/utils.py:2198 │
│ in greedy_search │
│ │
│ 2195 │ │ │ │ │ break │
│ 2196 │ │ │ │
│ 2197 │ │ │ # prepare model inputs │
│ ❱ 2198 │ │ │ model_inputs = self.prepare_inputs_for_generation(input_ids, **model_kwargs) │
│ 2199 │ │ │ │
│ 2200 │ │ │ # forward pass to get next token │
│ 2201 │ │ │ outputs = self( │
│ │
│ /mnt/data/rz/programe/glm/ChatGLM-Tuning/modeling_chatglm.py:979 in │
│ prepare_inputs_for_generation │
│ │
│ 976 │ │ mask_token = MASK if MASK in input_ids else gMASK │
│ 977 │ │ use_gmask = False if MASK in input_ids else gMASK │
│ 978 │ │ seq = input_ids[0].tolist() │
│ ❱ 979 │ │ mask_position = seq.index(mask_token) │
│ 980 │ │ │
│ 981 │ │ if mask_token not in seq: │
│ 982 │ │ │ raise ValueError("You have to add either [MASK] or [gMASK] in your input") │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ValueError: 150001 is not in list
我也遇到同样的问题，昨天还能跑的，今天有个bin文件更新后，就跑不起来了

[duzx16]

Duplicate of #432