Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support for Octpus LLM #493

Open
avcode-exe opened this issue May 19, 2024 · 4 comments
Open

Support for Octpus LLM #493

avcode-exe opened this issue May 19, 2024 · 4 comments
Labels
currently fixing Am fixing now! feature request Feature request pending on roadmap

Comments

@avcode-exe
Copy link

Hi guys!, It will be nice to add support to Octopus LLMs or are they any alternative? The MMLU score of Octopus v4 is 74.8% under 5-shot, very impressive for such a small model!
Octopus is based on Phi3 architecture, but it receive an error.
图片
图片
@danielhanchen
Copy link
Contributor

Automatic model support is in the works!!

@danielhanchen danielhanchen added currently fixing Am fixing now! feature request Feature request pending on roadmap labels May 19, 2024
@gjyotin305
Copy link

hi @danielhanchen can I take this up ?

@danielhanchen
Copy link
Contributor

Wait this is a Phi-3 model but 128K - this requires first mistral-fying it, then adding the special RoPE methodology. @gjyotin305 Yes you're more than welcome to thanks!

@gjyotin305
Copy link

Ohk will look into it @danielhanchen

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
currently fixing Am fixing now! feature request Feature request pending on roadmap
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@danielhanchen @avcode-exe @gjyotin305