mirror of
https://gitlab.com/buildroot.org/buildroot.git
synced 2026-08-01 21:23:51 -09:00
f163d20002
Since llama.cpp update in Buildroot commit [1], the test_aichat can
fail for several reasons:
The loop checking for the llama-server availability can fail if curl
succeed, but the returned json data is not formatted as expected.
This can happen if the server is ready but the model is not completely
loaded. In that case, the server returns:
{"error":{"message":"Loading model","type":"unavailable_error","code":503}}
This commit ignore Python KeyError exceptions while doing the
server test, to avoid failing if this message is received.
Also, this new llama-server version introduced a prompt caching, which
uses too much memory. This commit completely disable this prompt
caching by adding "--cache-ram 0" in the llama-server options.
[1] 05c36d5d87
Signed-off-by: Julien Olivain <ju.o@free.fr>