This Seq2Seq transformer model is designed to translate text, compose headings, generate questions and summarizations. Its size is only 80 million parameters, which allows you to work with good speed on equipment with low computing power.
Supported languages: Russian and English.
Maximum sequence length: 256 tokens.
For work, files in the main directory are sufficient.
q8 is a quantized model of ctranslate2 in INT8. Use it if you want to save disk space and RAM.
OpenNMT is a averaged OpenNMT model. It can be used in cases where the capabilities of the ctranslate are not enough.
Example of use
You can check the features on the demo site:
https://tolmacher.wia.su/
The items "write an article" and "rewrite" an article are implemented in tandem with the Queen 05B model, the rest are performed entirely by the Requestor model.
To work on your computer
-
Install the Python packages
pip install ctranslate2 OpenNMT-py==2.* sentencepiece
-
Download model in some dirrectory
-
Use with the Python program
import ctranslate2
import sentencepiece as spm
def requestor0(block, prompt="Summarize", ender="0o0"):
block+=ender
tokens = sp.encode(block, out_type=str)
input_text = "<s><###System:>" + prompt + ": "
input_tokens = sp.encode(input_text, out_type=str)
input_tokens.extend(tokens)
results = translator.translate_batch([input_tokens])
output_tokens = results[0].hypotheses[0]
output_text = sp.decode(output_tokens)
output_text=output_text.replace("<s><###Requestor:>", "").replace(" ⁇ ", "").strip()
output_text=output_text.split("0o0")[0]
return output_text
sp = spm.SentencePieceProcessor(model_file='140/sentencepiece.model') #140 - the directory to put the model files in
translator = ctranslate2.Translator('140/', device="cuda")
text="""The first-ever published research on Tinshemet Cave reveals that Neanderthals and Homo sapiens in the mid-Middle Paleolithic Levant not only coexisted but actively interacted, sharing technology, lifestyles, and burial customs. These interactions fostered cultural exchange, social complexity, and behavioral innovations, such as formal burial practices and the symbolic use of ocher for decoration."""
text=text.replace("\n","")
rtext=requestor0(text,"Summarize")
print(rtext+"\n")
The prompts on which the model was trained:
"Translate to english"; "Translate to russian"; "Generate question", "Make a title" "What is the topic of this text?", "Summarize", "Summarize briefly", "Make a step-by-step plan for this text" and russian analogs for this commnads. However, some combinations that were not present in the training sample also work. For example, "Translate briefly" can give a translation about twice as compact as the original text.
