Getting your thoughts

[olimorris]

I had an idea a month ago for improving my dumb workflow of pasting code from Neovim into ChatGPT and back again. I'd tried chatgpt.nvim but interacting through the popup and having to remember 15+ keymaps did not feel intuitive.

Searching GitHub I came across the awesome work that SteveArc had done for his personal OpenAI.nvim plugin. How he was using native Neovim buffers to interact with OpenAI and also stream the output back into Neovim was genius. One of the thing's I love about Steve's plugins is how he builds them around native Neovim features.

There were a few ways I wanted to build on his work to further improve my workflow:

• Allow OpenAI to write code (and refactor) directly into the current buffer
• To provide some assistance for LSP warnings
• To provide inline advice for code I've visually selected
• To automatically paste visually selected text into a chat buffer
• To be able to save chat conversations and restore them later

So after a week and a half of riffing, this is where I've ended up...

I'm hesitant to release it the wider community owing the amount of interest anything with OpenAI, LLMs, ChatGPT and Neovim gets. I'm not sure I'll have the willingness to respond to the huge influx of enhancements or bugs.

But, I wanted to share it with you to get your feedback and solicit some potential ideas to make it even better.

CC: @stevearc @mrjones2014

[olimorris]

@mmirus

[mmirus]

Thanks for inviting me to check this out! I've never really used AI besides just Copilot in neovim, so this should be interesting. 😁

Just an FYI in case it's relevant for anyone else down the line: I guess Lazy was fetching plugins from git using HTTLS, so since this is a private repo I got an auth error until I added the url line to the plugin spec:

{
  url = "[email protected]:olimorris/codecompanion.nvim.git",
  dependencies = { "stevearc/dressing.nvim" },
  cmd = { "CodeCompanionChat", "CodeCompanionActions" },
  config = true,
}

I think you can also configure which method to use for Lazy as a whole with git.url_format. Haven't tried it.

[mmirus]

Unfortunately, I'm having trouble trying this out--getting errors when I try.

I eliminated one error by disabling mkdx, which I saw referenced in the error.

Have you tested on nvim 0.9.5, or only nightly? Could be a version compat issue.

If it works for you on 0.9.5, it's likely something in my config conflicting.

---

Here are examples of a couple of things I'm still hitting:

Chat

1. Open a file
2. Run :CodeCompanionChat
3. Enter insert mode and type prompt (e.g., "Explain Lua in 5 words")
4. Hit Esc to exit insert mode. Error appears:

Error detected while processing InsertLeave Autocommands for "<buffer=7>":                                                   
Error executing lua callback: /usr/share/nvim/runtime/lua/vim/treesitter/languagetree.lua:264: bad argument #3 to 'set_included_ranges' (treesitter_node expected, got number)                                                                            
stack traceback:                                                                                                             
        [C]: in function 'set_included_ranges'                                                                               
        /usr/share/nvim/runtime/lua/vim/treesitter/languagetree.lua:264: in function 'parse'                                 
        ...y/codecompanion.nvim/lua/codecompanion/strategy/chat.lua:33: in function 'parse_settings'                         
        ...y/codecompanion.nvim/lua/codecompanion/strategy/chat.lua:210: in function <...y/codecompanion.nvim/lua/codecompanion/strategy/chat.lua:209> 

7. Try to write buffer (I assume this is how you send the request to OpenAI). An error appears, and the buffer isn't written:

Error detected while processing BufWriteCmd Autocommands for "<buffer=7>":                                                   
Error executing lua callback: /usr/share/nvim/runtime/lua/vim/treesitter/languagetree.lua:264: bad argument #3 to 'set_included_ranges' (treesitter_node expected, got number)                                                                            
stack traceback:                                                                                                             
        [C]: in function 'set_included_ranges'                                                                               
        /usr/share/nvim/runtime/lua/vim/treesitter/languagetree.lua:264: in function 'parse'                                 
        ...y/codecompanion.nvim/lua/codecompanion/strategy/chat.lua:33: in function 'parse_settings'                         
        ...y/codecompanion.nvim/lua/codecompanion/strategy/chat.lua:83: in function 'parse_messages_buffer'                  
        ...y/codecompanion.nvim/lua/codecompanion/strategy/chat.lua:288: in function 'submit'                                
        ...y/codecompanion.nvim/lua/codecompanion/strategy/chat.lua:203: in function <...y/codecompanion.nvim/lua/codecompanion/strategy/chat.lua:198>

8. Try to write without autocommands (:noa w). Another error, and the buffer is not written:

E676: No matching autocommands for buftype=acwrite buffer

Actions

1. Open a file
2. Run :CodeCompanionActions
3. Select Code author. Error:

Error executing vim.schedule lua callback: ...lazy/codecompanion.nvim/lua/codecompanion/utils/util.lua:8: bad argument #1 to 'pairs' (table expected, got nil)                                                                                            
stack traceback:                                                                                                             
        [C]: in function 'pairs'                                                                                             
        ...lazy/codecompanion.nvim/lua/codecompanion/utils/util.lua:8: in function 'contains'                                
        ...codecompanion.nvim/lua/codecompanion/strategy/author.lua:114: in function 'on_choice'                             
        ...are/nvim/lazy/dressing.nvim/lua/dressing/select/init.lua:78: in function ''                                       
        vim/_editor.lua: in function <vim/_editor.lua:0>

[mmirus]

That fixed it. Thanks!

Another thing I'm seeing is that the actions which require a selection (such as code advisor) only appear when you invoke the actions menu from a mapping, not if you select code and then run the command (i.e., :'<,'>CodeCompanionActions)

I'll play around with the plugin some more and share any other thoughts.

[olimorris]

Great spot. I'd never even considered that possibility! And thanks for playing around with it. Could you see it being a part of your regular workflow?

[mmirus]

Potentially--I'm kind of arbitrary with what I do or don't define mappings for. It probably depends on just how frequently I do the thing. I do think it's something users would expect to work both ways, though.

[mmirus]

If you mean could I see the feature/plugin as a whole being part of my regular workflow (rather than the method of invoking the actions for a selection 😅): definitely possible; copilot did. I'm new to using AI in my work beyond that, so still just learning what it has to offer. Without an in-editor interface like this I probably wouldn't have bothered trying it out, but this makes it accessible enough to be worth trying!

[olimorris]

Haha! Yes using it was what I meant, sorry. Been a long day. It's been great to use the plugin to develop the plugin. I had played around with Co-Pilot chat in VS Code and it was amazing. I'm not sure if that will ever come to Neovim so part of the motivation for this plugin was trying to emulate the workflow VS Code users can have.

[stevearc]

Played around with it briefly. Here's some raw feedback:

• no autocomplete on the 'model' parameter in the yaml metadata. Hard for users to know what all the options are without that
• something I really wanted to add to my plugin but never got around to:
  • smoother input streaming. Right now it streams in the chunks we get back from the API, but you could smooth that out and add a bit of interpolation to mimic what the web UI looks like
  • cancel ongoing requests with
  • stop tailing the output if I move my cursor around in the window (e.g. if I want to select/yank some portion of it before it's finished)
• when I use the action "code author" and provide a prompt, I just get a markdown code block with "Error - I don't know" as the contents

[olimorris]

Appreciate your feedback, thank you.

I've implemented the cancelling of requests (use q in the chat window when it's streaming) and fixed the autocomplete (just need to return a function). Also fixed the author strategy not showing errors clearly.

The tailing of the output is a good suggestion too. A real quality of life improvement so will add that soon and also look at streaming like ChatGPT...that would be epic!

[mmirus]

Here's some initial feedback on the code advisor.

After submitting the code advisor prompt, I got the following error:

Error executing vim.schedule lua callback: ...odecompanion.nvim/lua/codecompanion/strategy/advisor.lua:84: attempt to index local 'data' (a nil value)                                                                               
stack traceback:                                                                                                                                                                                                                     
        ...odecompanion.nvim/lua/codecompanion/strategy/advisor.lua:84: in function 'cb'                                                                                                                                             
        ...vim/lazy/codecompanion.nvim/lua/codecompanion/client.lua:74: in function ''                                                                                                                                               
        vim/_editor.lua: in function <vim/_editor.lua:0>

which was followed by a notification:

API Error: The model `gpt-4-1106-preview` does not exist or you do not have access to it. Learn more: https://help.openai.com/en/articles/7102672-how-can-i-access-gpt-4.

after upgrading to a paid account to get access to that model, I no longer get the error.

Certainly not a big deal, but it would be a smoother experience to just get the notification and not the error.

---

Beyond that, is there a way to indicate when a request is processing? The API has been taking a loooooong time to return a result to my prompts, and in the meanwhile it looked like nothing was happening. That's actually what I thought the first time, and then was a little surprised to see the window pop up when it finally did.

That said, this is pretty neat so far! I could actually see myself using these features in my work. Will need to experiment more.

[olimorris]

The advisor requests do feel like they take ages. I think it's a perception issue because we don't "stream" the response into the buffer. In the chat buffer we see the response immediately but that too can take aaaages to fully complete.

Regarding indication of processing, I've documented the user events that you can hook into here. I actually use it in my statusline plugin to display an icon.

One nice feature of the advisor is that if you press c in the popup/split, it takes the full conversation over to a chat buffer. I was finding that the explanation from OpenAI led me to want to ask more questions so saved me copying the full context into a new chat.

[mmirus]

Thanks for the pointers! I'll see if I can hook in there to send notifications or something. And being able to pivot from the advisor to a chat seems like a nice workflow. 💪

[mmirus]

BTW, the docs for the plugin are getting really good. Nice job on those.

[mmirus]

Simple example for a status notification:

local notify_record
vim.api.nvim_create_autocmd({ "User" }, {
  pattern = "CodeCompanion",
  group = vim.api.nvim_create_augroup("CodeCompanionHooks", {}),
  callback = function(request)
    local running = request.data.status == "started"
    local message = running and "Processing..." or "Complete!"
    local timeout = running == false and 5000 or false

    notify_record = vim.notify(message, vim.log.levels.INFO, {
      title = "CodeCompanion",
      timeout = timeout,
      replace = notify_record,
      hide_from_history = true,
    })
  end,
})

[mmirus]

@olimorris holy smokes, this blew my mind. It occurred to me that since we can have a terminal in a buffer in neovim, we could potentially use this plugin there too! I entered the command in the shell, went to normal mode, highlighted the line, and ran the code advisor, asking what the command does.

The only awkward part was that the advisor prompt opened in normal mode, so I had to enter insert. Minor issue.

I wonder what other non-obvious things we could do like this? Ask it to write a tricky shell command for you?

[olimorris]

Haha wow! Now that would be some functionality...

[olimorris]

@mmirus the Code Author should now write better terminal commands if you open the Actions prompt from a terminal buffer 👍🏼

[mmirus]

Nice!

Took two tries to get a command that worked, but that's not CodeCompanion's fault.

[olimorris]

...and let me say they are some hideous commands to have to figure out yourself 😆. The context for the terminal prompt might need some tweaking:

"I want you to act as an expert in writing terminal commands that will work for my current shell "
.. os.getenv("SHELL") .. ". I will ask you specific questions and I want you to return the raw command only 
(no codeblocks and explanations). If you can't respond with a command, please explain why but ensure the 
first word in your response is '[Error]' so I can parse it"

[mmirus]

Yeah, even needing to prompt twice was way easier than having to figure it all out myself. Of course, there can be a tradeoff in terms of learning, but that's probably worth it in many cases, or can be mitigated by taking some time to understand the answer.

That looks like a pretty smart prompt already! Clever putting the shell in there and the bit about [Error].

[mrjones2014]

Heh, I guess I was over-eager 😅

Biggest thing for me is I'll never pay for OpenAI because it's (in my opinion) overpriced and not a very ethical company, so I'd love to see alternative models supported, perhaps via LocalAI to simplify integration; LocalAI provides a drop-in replacement for OpenAI's REST API using alternative open source models.

[mmirus]

Was looking around for anything that integrates Copilot chat into neovim, and stumbled across these AI-related projects. Sharing here in case any of them are useful references you haven't already checked out.

https://github.com/gsuuon/model.nvim
https://github.com/Bryley/neoai.nvim
https://github.com/dense-analysis/neural
https://github.com/gptlang/CopilotChat.nvim

[mmirus]

Another new entry: https://github.com/z0rzi/ai-chat.nvim.

[olimorris]

Thanks again!

Do you use edgy.nvim? I'm working on adding a custom file type to the chat buffer, adding the option of making it appear in a popup so Edgy can intercept it and display it as a "pane" so to speak.

I've messed around with splits but it hasn't felt as intuitive as Edgy can make it feel.

[mmirus]

No, I've never tried edgy. Looking at it, it's one of those things that seems powerful but it's hard to conceptualize how I'd use it in practice.

If nothing else, a custom ft would enable any user to more easily customize the display even outside of edgy (e.g., disable line numbers in the split).

[olimorris]

I've made quite a few changes in the last week btw. Text now streams in the chat buffer much like it does with ChatGPT. Also, you can toggle the chat buffers to be hidden to allow streaming to continue in the background. Expect I will have busted your config though!

[olimorris]

I've also added in a Recipes guide that may be of interest...