Introduction
A standard plugin metadata format could be very helpful in helping people sort, search, curate, and explore the Vim and Neovim plugin ecosystem in this time of amazing growth.
It’s interesting that, after many many years of Vim plugins, every attempt at creating a metadata and/or package format has more or less failed to catch on.
I am hoping to compile a list of past attempts at plugin metadata specifications, in order to learn from past ideas. I am also hoping to gather feedback on whether Vim/Neovim plugins should have a standard metadata specification, and, if so, where and how that specification should be maintained/enforced.
I am also interested in prior art from other ecosystems. What works well, and what doesn’t? It might be interesting to look at other IDEs and text editors with “plugin-oriented” designs (JetBrains, Sublime), the other “hackable” text editor Emacs, programming languages with package metadata systems (Python, NodeJS, etc), and even system-level package managers that have a high rate of community contribution (AUR, Brew).
There is also a question of interoperability with Vim, as we continue to diverge from Vim on things like remote plugins, Lua, etc. Should interoperability be a goal? What are the implications of deciding yes or no?
Or maybe we just don’t need any of this, and you can all ignore this post 
The situation
With the (relatively recent!) arrival of the built-in package system (:packadd and friends), we no longer need to worry about a stable file format for the most part. Great! Plugins have more or less standardized around the Vim runtimepath file layout, because they can be copied and pasted verbatim into pack/*/opt or pack/*/start. Some non-standard elements remain, for things like compiling artifacts required for remote plugins, but the existing ad-hoc solutions work well enough.
However, the community has not successfully developed a metadata format for packages. Moreover, the community has largely fixated itself on centrally-hosted Git repositories as the primary distribution channel, specifically Github repositories.
The benefit is that it is trivially easy for a hobbyist to distribute their plugin; all they need to do is publish the source code to a Git repository somewhere, most likely Github. This is a good outcome!
Github in particular has a lot of features that do the job of a package format and distribution system. It is a low-friction, high-convenience way to get software out into the world, and Vim plugin managers have embraced it as the default way to get software out into the world.
However, it’s fascinating to see a perfectly good distribution channel, Vim.org, almost entirely cast aside in favor of something as chaotic as “head-only”/“rolling” Git repos. Have we lost something in the process? I would argue that we have, particularly as the Neovim plugin ecosystem continues to expand very quickly, and a typical power user could have 50, 75, even 100+ plugins in their configuration.
Moreover, the current state of affairs makes it extremely difficult to do certain basic tasks and perform certain queries, such as:
- Find plugins that have been updated in the last 6 months
- Find plugins that can be used with both Vim and Neovim
- Find remote plugins written in Rust
- Find plugins that define colorschemes
- Find plugins that define new filetypes
Lists like awesome-neovim and Neovimcraft are all the more impressive because of how scattered and disorganized the plugin ecosystem is. Building a curated list of plugins is challenging, and so is building a tidy user interface for browsing and interacting with them. Doing all of that without a metadata standard is an even bigger challenge, and it’s one that I think the community should be interested in mitigating.
Past attempts
Vim.org scripts
The Vim.org scripts collection is still alive and well. But I suspect that a significant fraction of its use comes through its Github mirror, and it’s certainly not where the cool kids go to upload Neovim plugins written in Lua and Rust.
I think that using opaque numerical “script IDs” instead of human-readable names/slugs was and remains its single biggest design flaw. It has pretty much everything one might want in a distribution system, but it’s impossible to tell what a plugin is, based on its URL.
There are/were other good reasons for people to switch to Github from Vim.org (e.g. nice Markdown rendering in readme files, Github stars, Git itself). But I genuinely think it might have maintained more its relevance if it didn’t have the opaque script IDs.
The craptacular Google custom search probably didn’t help. Those always sucked, even back when Google wasn’t considered evil.
What would a successful metadata standard look like?
I propose that any plugin metadata standard should have the following desirable attributes, in no particular order.
It needs a “killer app”
Show, don’t tell! People won’t start using a thing unless they see a benefit from using the thing. If a site like Neovimcraft adopted the specification and used it to improve the site, people would be eager to add it to their projects.
People aren’t lazy, but they need to be convinced.
Compatibility with existing standards
Vim-plug was not the first of its kind, but its tremendous popularity has helped solidify some standards around how people refer to and work with plugins.
For example, plugins are more or less universally identified with the author-name/plugin-name pattern. There’s no reason this shouldn’t be a universal identifier plugin across platforms. Name collisions would be very, very unlikely.
If you wanted to guard against name collisions, you could convert these to proper URIs by prepending schemes like github:/gh:, gitlab:/gl:, sourcehut:/srht:, et al. Omitting the scheme would of course imply github:. An easy upgrade path from informal current standards to something a little bit tidier with a little bit more longevity is a great feature, in my opinion.
(Note: These URI schemes seem like an easy win, and could be implemented in Your Favorite Plugin Manager right now, without regard to anything else I wrote in this post.)
Some kind of dependency tracking
Lack of sensible dependency tracking and management I think is a nontrivial impediment to building sophisticated and interesting software on top of Neovim. With an unambiguous naming specification, it should be pretty easy to list your dependencies, and it will be easy for plugin managers to resolve those dependencies.
Easy to get started
Metadata should be simple: easy to explain, easy to understand, easy to learn, easy to remember. It should be so easy to add metadata to your plugin that it should be embarrassing to skip it.
Low friction leads to moderate adoption, moderate adoption leads to social pressure on the holdouts. This is how network effects accumulate.
Smooth progression from “nothing” to “something”
The set of minimal required fields should be tiny. Maybe it should be nonexistent; clients could infer the metadata from a Git repository. Name is easy, author is easy on most (all?) major platforms, version and release dates are derived from tag and commit history, etc.
Wait, what?? Zero required fields? What’s the point of specifying anything at all? Because you can add metadata, eventually, whenever you get around to it, or when your plugin gets popular on Reddit and you want it to be taken more seriously, or when someone kindly does it for you in a PR.
As an additional benefit, having a specification means that people who build tooling (e.g. package managers like Vim-plug and Packer) have a set of common standard behavior to agree on.
Maybe the total set of fields would be as simple as:
- A name; fall back to Git repo name
- An author; fall back to Github author or equivalent if available
- Where to get the code, if it isn’t already obvious
- A version, if one can’t be inferred from a Git tag
- A homepage, if it isn’t the same as “where to get the code” (most plugins don’t have separate homepages)
- A summary description
- Dependencies
- Tags. I think people should tag their stuff. See below.
Metadata fields should not be tightly constrained
Don’t force people to think too hard. Don’t force people to make hard decisions. Naming things is hard. Don’t force people to do hard things. Simple should also be easy; we should work hard to make sure that writing your package metadata isn’t hard (!!).
Individual fields should not be tightly constrained. Version numbers just need to go up over time, they don’t need to follow Semver or Calver or Whateverver. Version numbering isn’t being used for high-precision reproducibility here. I just want to know if there’s been a stable release in the last 3 years.
The only exception is tags. People love to make a giant fucking mess out of tags, so don’t let them do it. Tags should be ASCII lowercase, /^[a-z0-9_-]+$/. Use a / to separate hierarchical levels. Examples of useful tags: colorscheme, implemented-in/rust, programming-language/lua, file-browser, frontend/gui, operators. That said, even with all these restrictions, tagging systems devolve into unusable chaos without strict central control (e.g. Stackoverflow). So maybe we just don’t have tags in the metadata; leave this up to the Neovimcrafts over the world?
Easy-to-use file format
The file format itself should easy to type without machine or IDE assistance, and it should also be easy to parse and work with programmatically.
TOML seems like a good candidate now that it has reached version 1.0, but there doesn’t appear to be a maintained TOML parser in Lua.
Maybe the configuration format could be (a subset of) Lua itself.
I’m not sure if the Neovim community is (or ever will be) ready for S-expressions. Although I’m sure the Fennel/Aniseed fans among us would be overjoyed to see it.
Notes
-
I am not here to bemoan the lack of a traditional package distribution channel, the dependency on M$oft-owned Github, etc. Vim is not unique in its dependency on Git and specifically Github; very serious programming languages like Go and Crystal have adopted similar conventions.
-
Another interesting consequence of the transition to Github is the growing epidemic of plugins that don’t use Vim help files for documentation. Perhaps we as a community should invest in better help file authoring tools, such as a Markdown-to-Vim-help converter.
