You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The actual past_key_values are stored on servers and can't be viewed from clients at the moment. In theory, it's possible to extend the server to send them when necessary (e.g., when a user reads fields of the RemotePastKeyValues object). Alternatively, it may be possible to extend the transformer block code to perform operations you're interested in on servers directly, without sending extra data to the clients.
Hi there.
I'm trying to combine chat.petals.dev with streaming-llm (https://github.com/mit-han-lab/streaming-llm).
However, when streaming-llm needs the output.past_key_values, I found that petals returns [RemotePastKeyValues].
And the annotation says 'A mock class representing the fact that
past_key_values
do exist but are stored on remote servers.'.SO, I wonder if there is any way to get the real value of 'past_key_values'.
like this img :
looking for your reply :)
The text was updated successfully, but these errors were encountered: