Commit Graph

2 Commits

Author SHA1 Message Date
Leymore
e810974068
[Fix] Fix when missing both pad and eos token (#287)
* fix when missing both pad and eos token

* update pad_token_id impl
2023-08-31 16:53:39 +08:00
gowithme
57fcfc975a
[Feature] Support intern lanuage model (#51)
* support internLM

* support internLM

* simplify intern model files

* update storage_manager

* support internLM

* Modify the file organization structure

* support internLM

* support internLM

* support internLM

* support internLM

* change some details
2023-07-27 18:49:36 +08:00