[Bug]: "error":"model 'llama2' not found, try pulling it first"

[wwjCMP]

What happened?

A bug happened!

I have added a custom model for ollama, but it still prompts the following error.

"error":"model 'llama2' not found, try pulling it first"

Relevant log output

backend-core  |     |     return [self._process_emb_response(prompt) for prompt in iter_]
                    backend-core  |     |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                    backend-core  |     |   File "/usr/local/lib/python3.11/site-packages/langchain_community/embeddings/ollama.py", line 199, in <listcomp>
                    backend-core  |     |     return [self._process_emb_response(prompt) for prompt in iter_]
                    backend-core  |     |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                    backend-core  |     |   File "/usr/local/lib/python3.11/site-packages/langchain_community/embeddings/ollama.py", line 173, in _process_emb_response
                    backend-core  |     |     raise ValueError(
                    backend-core  |     | ValueError: Error raised by inference API HTTP code: 404, {"error":"model 'llama2' not found, try pulling it first"}
                    backend-core  |     +------------------------------------
                    backend-core  | INFO:     172.28.0.1:35210 - "GET /user/credits HTTP/1.1" 200 OK
                    backend-core  | INFO:     127.0.0.1:45266 - "GET /healthz HTTP/1.1" 200 OK

Twitter / LinkedIn details

No response

[dosubot]

To resolve the "model 'llama2' not found" error, follow these steps:

1. Confirm Ollama is installed on your system. If not, install it following the official documentation.
2. Execute ollama run llama2 in your terminal to ensure Ollama is running with the Llama2 model.
3. In the Supabase dashboard (http://localhost:54323), navigate to the models table and add a new row for the Llama2 model with the necessary details.
4. Update the user_settings table to include "ollama/llama2" in the models field for your account, ensuring the Llama2 model is selectable and integrated into Quivr.

These steps are based on the latest Quivr documentation, ensuring the Llama2 model is correctly recognized by the system.

Details

Sources

• docs/developers/contribution/llm/ollama.mdx
• docs/configuring/new-model.mdx

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[wwjCMP]

All the steps you mentioned, I have completed, but it still prompts an error.

[dosubot]

It seems like the "model 'llama2' not found" error persists despite following the initial troubleshooting steps. This issue typically indicates that the inference API cannot locate the 'llama2' model, which could be due to several reasons. Here's a more focused approach to diagnose and resolve this:

1. Verify Model Name: Ensure the model name "ollama/llama2" is correctly specified in all configurations, including the Supabase dashboard where you manage the models and user_settings tables.

2. Check Ollama Service Logs: Look into the Ollama service logs to confirm that the 'llama2' model is indeed loaded. If not, consider restarting the Ollama service.

3. Confirm Endpoint URL: Double-check the endpoint URL used by the inference API to ensure it correctly points to the 'llama2' model's location.

4. Review Model Compatibility: Make sure the 'llama2' model is compatible with your current version of Quivr by checking the model compatibility in the Quivr codebase.

If after these steps the issue remains, please share more detailed information about the steps you're taking when encountering this error, including any relevant code snippets or additional log outputs. This will help in pinpointing the exact cause of the problem.

Details

Sources

• docs/developers/contribution/llm/ollama.mdx
• docs/configuring/new-model.mdx
• backend/modules/brain/service/utils/init.py
• backend/modules/brain/rags/quivr_rag.py

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[wwjCMP]

I don't want to use llama2, I want to use a custom model.

[dosubot]

To integrate your custom model within Quivr and avoid the 'llama2' model not found error, especially since you're not intending to use 'llama2', ensure you've correctly added your custom model to the Supabase models table and updated the user_settings table accordingly. Here's a concise guide:

1. Supabase Dashboard: Access your dashboard, navigate to the models table, and add a new row for your custom model with all necessary details.
2. Update user_settings: In the user_settings table, add your custom model to the models field. Ensure the model's name is correctly entered, matching exactly what Ollama expects, including case sensitivity.
3. Correct Model Identifier: Double-check the model identifier in all configurations to ensure it matches the expected format and spelling.

If you've done all the above and still face issues, it might be due to an invalid Brain ID, model compatibility, or configuration problems. Consulting the Quivr Discord community or Ollama's support resources could provide further assistance.

This approach focuses on ensuring your custom model is correctly recognized and integrated, addressing the specific error you're encountering.

Details

Sources

• docs/developers/contribution/llm/ollama.mdx
• docs/configuring/new-model.mdx
• backend/modules/brain/rags/quivr_rag.py

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[wwjCMP]

I am certain that I have filled in correctly.