Skip to content

(WIP) chatbot for Matrix using Meta's LLaMA model

Notifications You must be signed in to change notification settings

spv420/LLaMA_chatbot

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

LLaMA_chatbot

WARNING!!!

THIS CODE IS FUCKING AWFUL. DO NOT EXPECT IT TO WORK OUT OF THE BOX. I'M GOING TO ATTEMPT TO IMPROVE IT OVER THE NEXT WHILE, BUT FOR NOW, IT'S SHIT.

IF YOU CAN GET IT TO WORK, YOU DESERVE A COOKIE. CHEERS.

"general install guide"

  • install text-generation-webui from oogabooga
  • download the LLaMA weights, whatever is the largest that'll fit on your GPU(s)
  • run text-generation-webui for that model, --listen
  • make a venv python -m venv venv
  • install requirements venv/bin/pip install -r requirements.txt
  • set env MATRIX_{SERVER,USER,PASSWORD,ROOM} to applicable vals, ex: https://synapse.example.com, @AzureDiamond:example.com, hunter2, fancy room ID that's like \![A-Za-z]*:example\.com) respectively
  • run client.py venv/bin/python client.py
  • enjoy