import torch
from transformers import AutoModelForTokenClassification, AutoTokenizer, AutoConfig
tokenizer = AutoTokenizer.from_pretrained("fiveflow/roberta-base-spacing")
roberta = AutoModelForTokenClassification.from_pretrained("fiveflow/roberta-base-spacing")
org_text = "탄소중립과ESG경영에대한사회적요구확대".replace(" ", "") # 공백제거
label = ["UNK", "PAD", "O", "B", "I", "E", "S"]
# char 단위로 토큰화
token_list = [tokenizer.cls_token_id]
for char in org_text:
token_list.append(tokenizer.encode(char)[1])
token_list.append(tokenizer.eos_token_id)
tkd = torch.tensor(token_list).unsqueeze(0)
output = roberta(tkd).logits
_, pred_idx = torch.max(output, dim=2)
tags = [label[idx] for idx in pred_idx.squeeze()][1:-1]
pred_sent = ""for char_idx, spc_idx inenumerate(pred_idx.squeeze()[1:-1]):
# "E" tag 단위로 띄어쓰기if label[spc_idx] == "E": pred_sent += org_text[char_idx] + " "else: pred_sent += org_text[char_idx]
print(pred_sent.strip())
# '탄소중립과 ESG 경영에 대한 사회적 요구 확대'
@misc{park2021klue,
title={KLUE: Korean Language Understanding Evaluation},
author={Sungjoon Park and Jihyung Moon and Sungdong Kim and Won Ik Cho and Jiyoon Han and Jangwon Park and Chisung Song and Junseong Kim and Yongsook Song and Taehwan Oh and Joohong Lee and Juhyun Oh and Sungwon Lyu and Younghoon Jeong and Inkwon Lee and Sangwoo Seo and Dongjun Lee and Hyunwoo Kim and Myeonghwa Lee and Seongbo Jang and Seungwon Do and Sunkyoung Kim and Kyungtae Lim and Jongwon Lee and Kyumin Park and Jamin Shin and Seonghyun Kim and Lucy Park and Alice Oh and Jungwoo Ha and Kyunghyun Cho},
year={2021},
eprint={2105.09680},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
Runs of fiveflow roberta-base-spacing on huggingface.co
102
Total runs
0
24-hour runs
-1
3-day runs
-12
7-day runs
-13
30-day runs
More Information About roberta-base-spacing huggingface.co Model
roberta-base-spacing huggingface.co
roberta-base-spacing huggingface.co is an AI model on huggingface.co that provides roberta-base-spacing's model effect (), which can be used instantly with this fiveflow roberta-base-spacing model. huggingface.co supports a free trial of the roberta-base-spacing model, and also provides paid use of the roberta-base-spacing. Support call roberta-base-spacing model through api, including Node.js, Python, http.
roberta-base-spacing huggingface.co is an online trial and call api platform, which integrates roberta-base-spacing's modeling effects, including api services, and provides a free online trial of roberta-base-spacing, you can try roberta-base-spacing online for free by clicking the link below.
fiveflow roberta-base-spacing online free url in huggingface.co:
roberta-base-spacing is an open source model from GitHub that offers a free installation service, and any user can find roberta-base-spacing on GitHub to install. At the same time, huggingface.co provides the effect of roberta-base-spacing install, users can directly use roberta-base-spacing installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
roberta-base-spacing install url in huggingface.co: