PKU-Alignment / beaver-7b-v2.0-cost

huggingface.co
Total runs: 23
24-hour runs: -1
7-day runs: 2
30-day runs: 5
Model's Last Updated: April 21 2024
reinforcement-learning

Introduction of beaver-7b-v2.0-cost

Model Details of beaver-7b-v2.0-cost

🦫 Beaver's Cost Model

Model Details

The Beaver cost model is a preference model trained using the PKU-SafeRLHF dataset. It can play a role in the safe RLHF algorithm, helping the Beaver model become more safe and harmless.

  • Developed by: the PKU-Alignment Team.
  • Model Type: An auto-regressive language model based on the transformer architecture.
  • License: Non-commercial license.
  • Fine-tuned from model: LLaMA , Alpaca .
Model Sources
How to Use the Cost Model
import torch
from transformers import AutoTokenizer
from safe_rlhf.models import AutoModelForScore

model = AutoModelForScore.from_pretrained('PKU-Alignment/beaver-7b-v2.0-cost', torch_dtype=torch.bfloat16, device_map='auto')
tokenizer = AutoTokenizer.from_pretrained('PKU-Alignment/beaver-7b-v2.0-cost')

input = 'BEGINNING OF CONVERSATION: USER: hello ASSISTANT:Hello! How can I help you today?'

input_ids = tokenizer(input, return_tensors='pt')
output = model(**input_ids)
print(output)

# ScoreModelOutput(
#     scores=tensor([[[ 1.2031],
#          [ 2.0469],
#          [ 2.1875],
#          [ 2.0938],
#          [ 2.9219],
#          [ 2.2656],
#          [ 3.1250],
#          [ 2.4219],
#          [ 3.6406],
#          [ 2.4062],
#          [ 0.7383],
#          [ 0.6719],
#          [-0.4414],
#          [-1.2734],
#          [-1.6562],
#          [ 0.3340],
#          [ 0.2432],
#          [-0.6914],
#          [-1.0938],
#          [-1.9453],
#          [-3.0469],
#          [-2.7812],
#          [-2.2188],
#          [-1.6250],
#          [-1.5000],
#          [-1.9922],
#          [-2.6562],
#          [-9.4375]]], grad_fn=<ToCopyBackward0>),
#     end_scores=tensor([[-9.4375]], grad_fn=<ToCopyBackward0>),
#     last_hidden_state=tensor([[[ 7.4219e-02,  3.6865e-02, -2.4414e-01,  ..., -5.7129e-02,
#            1.1963e-01,  2.7734e-01],
#          [-7.0703e-01,  1.0234e+00,  9.8145e-02,  ...,  2.6719e+00,
#            8.2422e-01,  4.7119e-02],
#          [-1.5332e-01,  1.0938e+00, -5.0000e-01,  ..., -1.6699e-01,
#           -6.0156e-01,  5.3516e-01],
#          ...,
#          [-1.0469e+00,  3.5858e-03, -1.1094e+00,  ..., -1.1094e+00,
#            9.2578e-01,  1.3750e+00],
#          [ 3.1445e-01, -9.7266e-01, -1.8984e+00,  ..., -9.4141e-01,
#            2.0703e-01,  9.4531e-01],
#          [ 5.5625e+00, -1.8672e+00, -1.3359e+00,  ...,  8.0078e-01,
#           -1.8906e+00, -1.3516e+00]]], dtype=torch.bfloat16,
#        grad_fn=<ToCopyBackward0>),
#     end_last_hidden_state=tensor([[ 5.5625, -1.8672, -1.3359,  ...,  0.8008, -1.8906, -1.3516]],
#        dtype=torch.bfloat16, grad_fn=<ToCopyBackward0>),
#     end_index=tensor([27])
# )

Runs of PKU-Alignment beaver-7b-v2.0-cost on huggingface.co

23
Total runs
-1
24-hour runs
1
3-day runs
2
7-day runs
5
30-day runs

More Information About beaver-7b-v2.0-cost huggingface.co Model

beaver-7b-v2.0-cost huggingface.co

beaver-7b-v2.0-cost huggingface.co is an AI model on huggingface.co that provides beaver-7b-v2.0-cost's model effect (), which can be used instantly with this PKU-Alignment beaver-7b-v2.0-cost model. huggingface.co supports a free trial of the beaver-7b-v2.0-cost model, and also provides paid use of the beaver-7b-v2.0-cost. Support call beaver-7b-v2.0-cost model through api, including Node.js, Python, http.

PKU-Alignment beaver-7b-v2.0-cost online free

beaver-7b-v2.0-cost huggingface.co is an online trial and call api platform, which integrates beaver-7b-v2.0-cost's modeling effects, including api services, and provides a free online trial of beaver-7b-v2.0-cost, you can try beaver-7b-v2.0-cost online for free by clicking the link below.

PKU-Alignment beaver-7b-v2.0-cost online free url in huggingface.co:

https://huggingface.co/PKU-Alignment/beaver-7b-v2.0-cost

beaver-7b-v2.0-cost install

beaver-7b-v2.0-cost is an open source model from GitHub that offers a free installation service, and any user can find beaver-7b-v2.0-cost on GitHub to install. At the same time, huggingface.co provides the effect of beaver-7b-v2.0-cost install, users can directly use beaver-7b-v2.0-cost installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

beaver-7b-v2.0-cost install url in huggingface.co:

https://huggingface.co/PKU-Alignment/beaver-7b-v2.0-cost

Url of beaver-7b-v2.0-cost

beaver-7b-v2.0-cost huggingface.co Url

Provider of beaver-7b-v2.0-cost huggingface.co

PKU-Alignment
ORGANIZATIONS

Other API from PKU-Alignment