Discussion on the future of lspconfig and neovim's LSP ecosystem

[mjlbach]

Hi all,

Kinda long-winded and subject to change, be warned. lspconfig has reached what I would consider a stable state. I want to talk about what I foresee as the future of lspconfig and the next phase of the neovim lsp ecosystem.

TLDR

• lspconfig will be renamed server-configurations
• everything but the individual server configurations and documentation will be removed from lspconfig and replaced with comparable functionality in core.
• root_dirs will be moved out to a generic project-workspace-detection repository
• the “core” of lspconfig will be moved into a new project module in core

Intro

First off, I want to say I think with neovim 0.6.1, we have finally hit what I would call stable. Most things are now bug-free (36 issues across lspconfig and core!), and while there are still a few known features missing (semantic tokens, workspace watchers), I think the built-in client is solid and fast.

That being said, I think now is the time to start thinking about what comes next and solicit feedback.

lspconfig will be subsumed by features in neovim core

lspconfig has served it’s purpose well as a stop-gap for configuring the built-in client. It offers many things “close” to project level configuration, but I believe the configuration paradigms it encourages are fundamentally broken by the growing trend of using multiple language servers.

lspconfig users treat the lspconfig config object as a form of project management, using on_attach to set up keybindings, autocommands, and even buffer options. This becomes problematic when you have two language servers fighting with the same on_attach. Many users want to conditionally dispatch certain functions to one server or the other, they want a single unified set of settings for a given project’s buffers.

Basically lspconfig will be merged into core, and the remainder will be split across two separate repos

Projects (neovim/neovim)

I’m proposing a Project as the atomic unit for server management and configuration. A project configuration would be something like this: project.lua · GitHub. The main idea is to unify the on_attach to operate for the collection of servers (and other features) over the project directory. The project code would live in neovim croe.

neovim/server-configurations

lspconfig as it stands would be renamed to server-configurations. The only thing that would remain is the server configurations stored under lspconfig/server_configurations. These would also be stripped of root_dir. The main purpose of this repo would be to provide the “sane” defaults": settings, initializationOptions, and the cli args for launching the servers.

neovim/workspace-patterns

It’s never made much sense to me that root_dir is tied so directly to language servers. Really, root_dir is about detecting project roots which is based on the language packaging conventions. As projects are “bigger” than language servers, these would be decoupled from lspconfig and moved into their own repo.

Won’t this make things more complicated?

I think this makes things more verbose but less complicated. I see many anti-patterns (disabling capabilities to make null-ls the default formatter, wrapping lspconfig to allow imperative installation of language servers) that this pattern solves.

For those that want an extremely simple experience (vscode, coc, whatever) this is mostly about backend restructuring, I have a plan for you

Creation of server specific plugins

By moving all of the lspconfig infrastructure into core, I hope to spawn a new wave of language-server specific plugins. Right now, lspconfig offers some support for things off protocol. I’ve tried to discourage this in favor of server specific plugins, but it’s hard. null-ls ended up having to copy/paste the entire lspconfig infra. By moving the concept of a Project to core, plugins can reuse this by either providing utility functions that users can add to their own projects, or even providing wholesale project templates that combine multiple language servers, formatters, and opinionated settings into one, which brings me to…

The need for an opinionated automagical meta-plugin

I see many complaints about “lspconfig being hard to set up”. Here is what I understand people to want:

a single installation line `use ‘nvim-lsp/my_automagical_plugin’ enables:

• autocompletion
• snippets
• automatic installation of servers when you open a filetype
• automatic launching of a preferred language server when you open a filetype
• project templates (think react, vue, basically anything webdev which invites multiple language servers
• off-protocol support for all language servers
• support for launching the language server
• sane default keybindings

Snippets and autocompletion may some day make their way into neovim core, the rest will likely not.

I personally would not use or develop this meta-plugin, but I’m hoping my efforts to expose/build the requisite APIs in the built-in client, and the upcoming projects module (when it’s ready, etc.) will make building this meta-plugin easier.

I think there is a lot of duplicate effort amongst nvim-lsp-installer, navigator.lua, lunarvim, other frameworks, etc. that could be pooled into this effort. The biggest blocker, IMO, right now is the lack of dependencies in most package managers, and the fact that most package managers do not expose an imperative API a package can use to dynamically fetch additional dependencies at runtime.

Ask questions, get involved

Anyways, that’s kinda messy but wanted to loop people in. None of this is set in stone, I just wanted to provide some future “direction” for people interested in contributing to the ecosystem.

[wbthomason]

Thanks for the update! We’ve discussed the “imperative API” for adding package dependencies a bit before, but: what, to your mind, would this need to support to be useful? In particular, what’s the argument for this happening at runtime as opposed to install time (e.g. via something like a requirements.lua file)?

[mjlbach]

We’ve discussed the “imperative API” for adding package dependencies a bit before, but: what, to your mind, would this need to support to be useful?

I think this meta-plugin could totally be done today (or soon) with the equivalent of a requirements.lua. I think plugins should be able to declare their own dependencies to avoid the current situation of requiring users to list the dependencies explicitly in their package manager. Your suggestion would be a great improvement over status quo.

In particular, what’s the argument for this happening at runtime as opposed to install time (e.g. via something like a requirements.lua file)?

The main advantage of dynamic, runtime dependencies comes for optional features. If this meta-plugin wanted to say, support nvim-lint or null-ls. With the requirements.lua approach users would have to have both null-ls and nvim-lint installed should the meta-plugin want to support both, or have nvim-jdtls installed even if they never write java. I think dynamic dependencies (based on some key in setup) would be good for allowing the plugin to offer plugin integrations without surfacing the complexity of installing the plugin manually, or installing all plugins regardless if the user uses that particular integration.

[wbthomason]

Fair enough - I agree re: not wanting to have everything installed if it’s not needed. When I wrote my initial response, I was thinking that it would be challenging to figure out how to trigger a dynamic install. That’s definitely not true - packer.install already allows you to specify a subset of plugins to install (as long as they’re added to the managed set), so this “imperative API” is technically already possible (albeit more hacking on the dynamism of Lua than an actual API). The challenge is probably in persistently adding plugins to the managed set, to prevent dynamically installed dependencies from being marked for removal on the next clean/sync/etc. This seems to point to packer needing to adopt something on disk recording the full set of statically specified + dynamically added plugins, and using this as the set of plugins to update. Should be feasible - I think the bigger obstacle for the broader ecosystem is finding a way to specify dynamic requirements in a plugin-manager-independent way (rather than having to test for packer and use the packer-specific method, or plug and use the plug-specific method, etc.).

[mjlbach]

I think the bigger obstacle for the broader ecosystem is finding a way to specify dynamic requirements in a plugin-manager-independent way (rather than having to test for packer and use the packer -specific method, or plug and use the plug -specific method, etc.).

I’m personally ok with treating packer as a defacto standard I think it’s not unreasonable to expect/ask users to have a specific package manager installed if they are planning on delegating a large portion of their setup to some automagical plugin.

As you know there have been many attempts to come up with standard packaging guidelines for neovim that haven’t really materialized. I think usecases like this are a good opportunity to dogfood some of these ideas and see what works. We can chat on matrix but with some examples I’d probably be able to make an MVP of the “automagic” plugin that I could pass off to someone (definitely not something I want to maintain though).

[MohdAfnan]

Is there any plan to add vscode’s marketplace like stuff?

[mjlbach]

Is there any plan to add vscode’s marketplace like stuff?

No, I don’t see value in this (or what it has to do with LSP).

[ratmice]

@mjlbach

neovim/workspace-patterns

It’s never made much sense to me that root_dir is tied so directly to language servers. Really, root_dir is about detecting project roots which is based on the language packaging conventions. As projects are “bigger” than language servers, these would be decoupled from lspconfig and moved into their own repo.

Bit of an older thread, so I hope it is alright to bump it as I’ve been getting my feet wet in nvim/lua and trying to figure out how to change my configuration a bit based on root_dir, and there are a couple of things that I don’t quite understand.

• The root_dir checks seem to be predicated on having a file open, if you open a directory e.g. vim foo/ or vim . it does not seem to initiate launch until you open a file in that directory.

• I would like to use a plugin like nvim-tree opening it conditionally on being in a workspace with a root_dir.

The first one is perhaps best solved by checking directories and filetypes in neovim autostart stuff?
or I need some special loop over lsp servers in my init for launch on directories.

configuring nvim tree is a bit awkward If I want some specific nvim-tree.setup({open_on_setup = true}) within on_attach, is really not the right place for it especially when multiple lsp servers attach. I think being able to check if a project-workspace is detected outside of lsp, So I could set {open_on_setup = project-workspace-detection.something_that_resembles_a_workspace() == true} would be seem more appropriate.

I would be remiss not to ask if there is a place or a branch where this work is going on, or if we’re just in a collecting thoughts state of development?

[mjlbach]

I’m not really understanding your specific usecase, but I sketched out my ideas in the unfinished repo here GitHub - mjlbach/projects.nvim: NOT FOR PUBLIC USE

[ratmice]

I should have probably specified a bit more clearly nvim-tree (a file manager/directory browser), has a
{ diagnostics = { enable = true, } } where my lsp emits diagnostics for files which aren’t opened

These will highlight files with diagnostics colors, and show diagnostic signs in the directory browser.
So I’m trying to set it up so that this is opened on startup but only if i’m in a project workspace.

[mjlbach]

I think you’ll need to consult with the plugin resources and whatever its support forum is, that’s not what this topic is about.

[ratmice]

I believe there is nothing it can do either it really falls into the want to use project-workspace-detection outside of lsp category afaict anyhow, thanks for linking to your branch I will have a look through it.

[mjlbach]

I see, right now the only option is to use a project detection plugin, many of which double did and check for active lsp clients and read off the URI from workspaceFolders and convert that to a sanitized filepath. I’m basically proposing making the project level concept front and center so both LSP and whatever plugin (or your hook to use nvim-tree) can build on that more directly instead of poking through the client list.