-
Notifications
You must be signed in to change notification settings - Fork 5.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix embedding by adding fixes from llama.cpp upstream #4399
base: main
Are you sure you want to change the base?
Conversation
I can confirm this PR indeed fixes very obvious issues I had when doing embedding and queries with |
Desperately waiting for this fix to be integrated into next version. Having to stay on 0.1.31 is kind of a pain ... |
@multiduplikator are you on a Mac? If so, it's very easy to download, compile and run this branch on your computer.
|
@fredrik-smedberg Not on a Mac. I still have time to wait ... would like to avoid getting into custom building. But thanks for the input. Will come in handy when the time comes :) |
GJ @deadbeef84 🚀🚀 |
@jmorganca please merge this! Thanks 😁😁 |
@jmorganca Hi, I can confirm this is important fix for any application running in prod. |
Embedding appears broken since v0.1.32
See #3777 #4207 for details.
This PR applies fixes based on ggerganov/llama.cpp@1b67731#diff-87355a1a297a9f0fdc86af5e2a59cae153290f58d68822cd10c30fee4f7f7076.
I've tested it and embedding vectors looks correct after applying this patch.