You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Since n_embd_gqa is set to n_embd_v_gqa this assert only works for models where n_embd_k_gqa == n_embd_v_gqa, that is when n_embd_head_k == n_embd_head_v - and it fails when n_embd_head_k != n_embd_head_v. Is this intentional behavior?
The text was updated successfully, but these errors were encountered:
During my work on DeepSeek-V2 I noticed a failing assert:
llama.cpp/llama.cpp
Line 4923 in 9afdffe
Since
n_embd_gqa
is set ton_embd_v_gqa
this assert only works for models wheren_embd_k_gqa == n_embd_v_gqa
, that is whenn_embd_head_k == n_embd_head_v
- and it fails whenn_embd_head_k != n_embd_head_v
. Is this intentional behavior?The text was updated successfully, but these errors were encountered: