Skywork / Skywork-13B-Base-Intermediate

👨‍💻 Github • 🤗 Hugging Face • 🤖 ModelScope • 💬 WeChat • 📜 Tech Report

介绍（Introduction）

本项目提供Skywork-13B-Base第一阶段预训练（约0～3T）过程中产生的中间存档。In this repository we provide intermediate checkpoints of our Skywork-13B-Base during Stage-1 pretraining (0~3T), including:

• Skywork-13B-Base-0.5T
• Skywork-13B-Base-1T
• Skywork-13B-Base-1.5T
• Skywork-13B-Base-2T
• Skywork-13B-Base-2.5T
• Skywork-13B-Base-3T

如果您希望了解更多的信息，如训练方案，评估方法，请参考我们的 技术报告 ， Skymath 论文， SkyworkMM 论文。 If you are interested in more training and evaluation details, please refer to our technical report , Skymath paper and SkyworkMM paper.

声明和协议（Declaration and License Agreement）

声明（Declaration）

我们在此声明，不要利用Skywork模型进行任何危害国家社会安全或违法的活动。另外，我们也要求使用者不要将 Skywork 模型用于未经适当安全审查和备案的互联网服务。我们希望所有的使用者都能遵守这个原则，确保科技的发展能在规范和合法的环境下进行。

我们已经尽我们所能，来确保模型训练过程中使用的数据的合规性。然而，尽管我们已经做出了巨大的努力，但由于模型和数据的复杂性，仍有可能存在一些无法预见的问题。因此，如果由于使用skywork开源模型而导致的任何问题，包括但不限于数据安全问题、公共舆论风险，或模型被误导、滥用、传播或不当利用所带来的任何风险和问题，我们将不承担任何责任。

We hereby declare that the Skywork model should not be used for any activities that pose a threat to national or societal security or engage in unlawful actions. Additionally, we request users not to deploy the Skywork model for internet services without appropriate security reviews and records. We hope that all users will adhere to this principle to ensure that technological advancements occur in a regulated and lawful environment.

We have done our utmost to ensure the compliance of the data used during the model's training process. However, despite our extensive efforts, due to the complexity of the model and data, there may still be unpredictable risks and issues. Therefore, if any problems arise as a result of using the Skywork open-source model, including but not limited to data security issues, public opinion risks, or any risks and problems arising from the model being misled, abused, disseminated, or improperly utilized, we will not assume any responsibility.

协议（License Agreement）

社区使用Skywork模型需要遵循 《Skywork 模型社区许可协议》 。Skywork模型支持商业用途，如果您计划将Skywork模型或其衍生品用于商业目的，无需再次申请， 但请您仔细阅读 《Skywork 模型社区许可协议》 并严格遵守相关条款。

The community usage of Skywork model requires Skywork Community License . The Skywork model supports commercial use. If you plan to use the Skywork model or its derivatives for commercial purposes, you must abide by terms and conditions within Skywork Community License .

引用和联系我们（Contact Us and Citation）

如果您觉得我们的工作对您有帮助，欢迎引用我们的论文~

If you find our work helpful, please feel free to cite our paper~

@misc{wei2023skywork,
      title={Skywork: A More Open Bilingual Foundation Model}, 
      author={Tianwen Wei and Liang Zhao and Lichang Zhang and Bo Zhu and Lijie Wang and Haihua Yang and Biye Li and Cheng Cheng and Weiwei Lü and Rui Hu and Chenxia Li and Liu Yang and Xilin Luo and Xuejie Wu and Lunan Liu and Wenjun Cheng and Peng Cheng and Jianhao Zhang and Xiaoyu Zhang and Lei Lin and Xiaokun Wang and Yutuan Ma and Chuanhai Dong and Yanqi Sun and Yifu Chen and Yongyi Peng and Xiaojuan Liang and Shuicheng Yan and Han Fang and Yahui Zhou},
      year={2023},
      eprint={2310.19341},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

@article{skyworkmath,
  title={SkyMath: Technical Report},
  author={Liu Yang, Haihua Yang, Wenjun Cheng, Lei Lin, Chenxia Li, Yifu Chen, Lunan Liu, Jianfei Pan, Tianwen Wei, Biye Li, Liang Zhao, Lijie Wang, Bo Zhu, Guoliang Li, Xuejie Wu, Xilin Luo, Rui Hu},
  journal={arXiv preprint arXiv: 2310.16713},
  url={https://arxiv.org/abs/2310.16713},
  year={2023}
}

@article{Skywork_Multi-Modal_Group_Empirical_Study_Towards_2023,
    author = {Skywork Multi-Modal Group},
    month = sep,
    title = {{Empirical Study Towards Building An Effective Multi-Modal Large Language Model}},
    year = {2023}
}