GenAI: Fixes multiple calls to settings and health #87623
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
torkelo
requested review from
oscarkilhed,
dprokop,
ivanortegaalba and
nmarrs
and removed request for
a team
torkelo
added
no-backport
|
@SandersAaronD do you think we should clear out the cached promise at a set interval? As far as I am aware the health check is just that status of configuration from the LLM plugin in Grafana which in theory should only change if the LLM plugin config is modified / disabled and a page refresh will cause health to be checked again In local testing seems fine Screen.Recording.2024-05-10.at.11.29.01.AM.mov |
nmarrs
approved these changes
May 10, 2024
|
|
||
| const enabled = await isLLMPluginEnabled(); | ||
|
|
||
| expect(enabled).toBe(false); | ||
| expect(enabled).toBe(true); | ||
| }); |
There was a problem hiding this comment.
These changes are just switching the order of tests to first check false case then true given newly added cache
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Keep seeing 1 call to llm app settings and 1 health for every input that uses the generative AI features. I find these duplicated calls pretty annoying.
This caches the is enabled check by using a shared promise. Also avoids future checks if the check successed we assume it will succeed in the future as well (we could have a timeout and clear the cached promise after an hour if we think the the health check is likely to fail later)