Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: extra zero being added to Context Length and Max Tokens #4288

Open
edwardochoaphd opened this issue May 9, 2024 · 1 comment
Open
Labels
bug Something isn't working

Comments

@edwardochoaphd
Copy link

edwardochoaphd commented May 9, 2024

What is the issue?

first reported to OpenWebUI at: open-webui/open-webui#2141

then, checked by a OpenWebUI dev who says may be an issue on ollama-end...
image

Set context length then read logfile. these are 2 logs examples showing my context length had an extra zero added

time=2024-05-08T20:12:24.868Z level=WARN source=memory.go:17 msg="requested context length is greater than model max context length" requested=81920 model=65536
time=2024-05-08T05:24:12.168Z level=WARN source=memory.go:17 msg="requested context length is greater than model max context length" requested=20480 model=8192

also
appears there may be a 2nd (related?) issue/bug with Max Tokens
when set to 2048... in a conversation, getting an extra zero getting added here too?... see below two examples of 20480 which should have been 2048
...
time=2024-05-09T14:33:27.120Z level=WARN source=server.go:77 msg="requested context length is greater than the model's training context window size" requested=20480 "training size"=4096
...
.................................................................................................
llama_new_context_with_model: n_ctx = 20480

OS

Linux

GPU

Nvidia

CPU

No response

Ollama version

0.1.34

@edwardochoaphd edwardochoaphd added the bug Something isn't working label May 9, 2024
@edwardochoaphd
Copy link
Author

using OpenWebUI to workaround the bug, in case helpful for others - see addt'l comments in: open-webui/open-webui#2147 (reply in thread)
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant