This checkpoint is a states tuning file from RWKV-6-7B. Please download the base model from https://huggingface.co/BlinkDL/rwkv-6-world/tree/main . It will give a judgement about the type of the sentence. Usage:

  • update the latest rwkv package: pip install --upgrade rwkv
  • Download the base model and the states file. You may download the states from the epoch_2 directory.
  • Following the codes:
    • Loading the model and states
from rwkv.model import RWKV
from rwkv.utils import PIPELINE, PIPELINE_ARGS
import torch

# download models: https://huggingface.co/BlinkDL
model = RWKV(model='/media/yueyulin/KINGSTON/models/rwkv6/RWKV-x060-World-7B-v2.1-20240507-ctx4096.pth', strategy='cuda fp16')
print(model.args)
pipeline = PIPELINE(model, "rwkv_vocab_v20230424") # 20B_tokenizer.json is in https://github.com/BlinkDL/ChatRWKV
# use pipeline = PIPELINE(model, "rwkv_vocab_v20230424") for rwkv "world" models
states_file = '/media/yueyulin/data_4t/models/states_tuning/custom_trainer/epoch_2/RWKV-x060-World-7B-v2.1-20240507-ctx4096.pth.pth'
states = torch.load(states_file)
states_value = []
device = 'cuda'
n_head = model.args.n_head
head_size = model.args.n_embd//model.args.n_head
for i in range(model.args.n_layer):
    key = f'blocks.{i}.att.time_state'
    value = states[key]
    prev_x = torch.zeros(model.args.n_embd,device=device,dtype=torch.float16)
    prev_states = value.clone().detach().to(device=device,dtype=torch.float16).transpose(1,2)
    prev_ffn = torch.zeros(model.args.n_embd,device=device,dtype=torch.float16)
    states_value.append(prev_x)
    states_value.append(prev_states)
    states_value.append(prev_ffn)

The typed schema is :

schema = ['事件', '自然科学', '建筑结构', '地理地区', '组织', '医学', '天文对象', '人造物件', '运输', '作品', '生物', '人物']

Try the following case:

cat_char = '🐱'
bot_char = '🤖'
instruction ='你是一个图谱实体知识结构化专家。请从input中抽取出符合schema定义的实体实例和其属性,不存在的属性不输出,属性存在多值就返回列表。请按照JSON字符串的格式回答。'
schema = schemas['人物']
input_text = "个人简介姓名:拉塞·维比  所属球队:布伦特福德  国籍:丹麦、法国、荷兰、法属圭亚那  出生日期:1987-02-22  身高:181cm  体重:73kg  场上位置:前锋  球衣号码:21  丹麦射手拉塞-维比,获得了2014赛季瑞超联赛金靴"
input_text = {'input': input_text, 'schema': schema}
input_text = json.dumps(input_text).decode('UTF-8')
ctx = f'{cat_char}:{instruction}\n{input_text}\n{bot_char}:'
print(ctx)

def my_print(s):
    print(s, end='', flush=True)

# For alpha_frequency and alpha_presence, see "Frequency and presence penalties":
# https://platform.openai.com/docs/api-reference/parameter-details

args = PIPELINE_ARGS(temperature = 1.0, top_p = 0, top_k = 0, # top_k = 0 then ignore
                     alpha_frequency = 0.25,
                     alpha_presence = 0.25,
                     alpha_decay = 0.996, # gradually decay the penalty
                     token_ban = [0], # ban the generation of some tokens
                     token_stop = [0,1], # stop generation whenever you see any token here
                     chunk_len = 256) # split input into chunks to save VRAM (shorter -> slower)

pipeline.generate(ctx, token_count=200, args=args, callback=my_print,state=states_value)
print('\n')

The output should look like:

{"result":人物}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The model has no library tag.