-
Notifications
You must be signed in to change notification settings - Fork 3.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature]: Implement and test speculative editing #1854
Comments
@neubig I'm interested in this, can take a look at this, this weekend. |
@neubig Ok, so I think the easiest approach would be to implement this with huggingface's assistant model feature. I found some references to this with whisper, and another hf blog here. So would a valid implementation be to use a hf model, with an assistant model, and then update the
|
Hmm, I'm a bit confused. I think the main difference that we would need to have is that we would need to generate edits in the diff format like that cited on the blog: @@ ... @@
function binarySearch(arr, x) {
- let low = 0, high = arr.length - 1;
- while (low <= high) {
- let mid = Math.floor((low + high) / 2);
- if (arr[mid] === x) {
- return mid;
- }
- low += 1;
- }
- return -1;
+ let low = 0, high = arr.length - 1;
+ while (low <= high) {
+ let mid = Math.floor((low + high) / 2);
+ if (arr[mid] === x) {
+ return mid;
+ } else if (arr[mid] < x) {
+ low = mid + 1;
+ } else {
+ high = mid - 1;
+ }
+ }
+ return -1;
} I'm not sure how this has something to do with the assistant feature on hugging face. Could you elaborate more? |
Ok, that helps. I was looking at the implementation portion, where they talk about speculative decoding in the blog post.
And the assistant model is how speculative decoding works with huggingface. Ok, so now I get that you are talking about generating edits in a diff format. I'll look into this. |
What problem or use case are you trying to solve?
File editing is not perfect with our current method using SWE-Agent style actions.
Do you have thoughts on the technical implementation?
Cursor has recently published work on speculative editing, that we could possibly incorporate into our agent workflows: https://cursor.sh/blog/instant-apply
We would want to implement a prototype and benchmark it against SWE-Bench.
This would be a good issue for new contributors who have significant experience with LLMs and agents!
The text was updated successfully, but these errors were encountered: