Maintaining the Neovim project
==============================

Notes on maintaining the Neovim project.

General guidelines
------------------

* Decide by cost-benefit
* Write down what was decided
* Constraints are good
* Use automation to solve problems
* Never break the API... but sometimes break the UI

Issue triage
------------

In practice we haven't found a way to forecast more precisely than "next" and
"after next". So there are usually one or two (at most) planned milestones:

* Next bugfix-release (1.0.x)
* Next feature-release (1.x.0)

The forecasting problem might be solved with an explicit priority system (like
Vim's todo.txt). Meanwhile the Neovim priority system is defined by:

* PRs nearing completion.
* Issue labels. E.g. the `has:plan` label increases the ticket's priority merely
  for having a plan written down: it is _closer to completion_ than tickets
  without a plan.
* Comment activity or new information.

Anything that isn't in the next milestone, and doesn't have a finished PR—is
just not something you care very much about, by construction. Post-release you
can review open issues, but chances are your next milestone is already getting
full... :)

Release policy
--------------

Release "often", but not "early".

The (unreleased) `master` branch is the "early" channel; it should not be
released if it's not stable. High-risk changes may be merged to `master` if
the next release is not imminent.

For maintenance releases, create a `release-x.y` branch. If the current release
has a major bug:

1. Fix the bug on `master`.
2. Cherry-pick the fix to `release-x.y`.
3. Cut a release from `release-x.y`.
    * Run `./scripts/release.sh` (requires [git cliff](https://github.com/orhun/git-cliff))
    * The [CI job](https://github.com/neovim/neovim/blob/3d45706478cd030c3ee05b4f336164bb96138095/.github/workflows/release.yml#L11-L13)
      will update the release assets and [force-push to the "stable" tag](https://github.com/neovim/neovim/blob/cdd87222c86c5b2274a13d36f23de0637462e317/.github/workflows/release.yml#L229).

### Release automation

Neovim automation includes a [backport bot](https://github.com/korthout/backport-action).
Trigger the action by labeling a PR with `ci:backport release-x.y`. See `.github/workflows/backport.yml`.

Deprecating and removing features
---------------------------------

Neovim inherits many features and design decisions from Vim, not all of which
align with the goals of this project. It is sometimes desired or necessary to
remove existing features, or refactor parts of the code that would change
user's workflow. In these cases, a deprecation policy is needed to properly
inform users of the change.

When a (non-experimental) feature is slated to be removed it should:

1. Be _soft_ deprecated in the _next_ release
    - Use of the deprecated feature will still work.
    - This means deprecating via documentation and annotation (`@deprecated`).
    - Include a note in `deprecated.txt`.
    - For Lua features, use `vim.deprecate()`. The specified version is the
      current minor version + 2. For example, if the current version is
      `v0.10.0-dev-1957+gd676746c33` then use `0.12`.
    - For Vimscript features, use `v:lua.vim.deprecate()`. Use the same version
      as described for Lua features.
    - `vim.deprecate(…, 'x.y.z')` where major version `x` is greater than the
      current Nvim major version, is always treated as _soft_ deprecation.
2. Be _hard_ deprecated in a release following the release in which it was soft deprecated.
    - Use of the deprecated feature will still work but should issue a warning.
    - Features implemented in C will need bespoke implementations to communicate
      to users that the feature is deprecated.
3. Be removed in a release following the release in which it was hard deprecated
    - Usually this will be the next release, but it may be a later release if
      a longer deprecation cycle is desired
    - If possible, keep the feature as a stub (e.g. function API) and issue an
      error when it is accessed.

Example:

                    Deprecation                            Removal
                         ┆                 ┆                 ┆
                         ┆      Soft       ┆      Hard       ┆
                         ┆   Deprecation   ┆   Deprecation   ┆
                         ┆     Period      ┆     Period      ┆
             ────────────────────────────────────────────────────────────
    Version:            0.10              0.11              0.12
             ────────────────────────────────────────────────────────────
             Old code         Old code          Old code
                                 +                 +
                              New code          New code         New code

Feature removals which may benefit from community input or further discussion
should also have a tracking issue (which should be linked to in the release
notes).

Exceptions to this policy may be made (for experimental subsystems or when
there is broad consensus among maintainers). The rationale for the exception
should be stated explicitly and publicly.

Dependencies
------------

### Third-party dependencies

(Note: keep this in sync with the "external" section of `LICENSE.txt`).
(TODO: declare deps in `sbom.json`, CycloneDX SBOM format).

"Bundled" dependencies can be updated by bumping their versions in `cmake.deps/deps.txt`.
Some can be auto-bumped by `scripts/bump_deps.lua`.

* [LuaJIT](https://github.com/LuaJIT/LuaJIT)
* [Lua](https://www.lua.org/download.html)
* [libuv](https://github.com/libuv/libuv)
    * Requires a PR to update the [Zig package](https://github.com/allyourcodebase/libuv) first. If the script fails, run `zig fetch --save git+https://github.com/allyourcodebase/libuv.git#<refname>` manually.
* [Luv](https://github.com/luvit/luv)
    * When bumping, also sync
      - [our bundled meta file](https://github.com/neovim/neovim/blob/master/runtime/lua/uv/_meta.lua) with [the upstream meta file](https://github.com/luvit/luv/blob/master/docs/meta.lua);
      - [our bundled documentation](https://github.com/neovim/neovim/blob/master/runtime/doc/luvref.txt) with [the upstream documentation](https://github.com/luvit/luv/blob/master/docs/docs.md).
* [gettext](https://ftp.gnu.org/pub/gnu/gettext/)
* [libiconv](https://ftp.gnu.org/pub/gnu/libiconv)
* [lua-compat](https://github.com/keplerproject/lua-compat-5.3)
* [tree-sitter](https://github.com/tree-sitter/tree-sitter)
* [treesitter parsers](https://github.com/neovim/neovim/blob/7e97c773e3ba78fcddbb2a0b9b0d572c8210c83e/cmake.deps/deps.txt#L47-L62)
* (Deprecated) [unibilium](https://github.com/neovim/unibilium)
    * The original project [was abandoned](https://github.com/neovim/neovim/issues/10302), so the [neovim/unibilium](https://github.com/neovim/unibilium) fork is considered "upstream" and is maintained on the `master` branch.
    * **Note:** unibilium is NOT required. See [BUILD.md](./BUILD.md#build-without-unibilium) to build Nvim without unibilium.

### Vendored dependencies

"Vendored" (inlined) dependencies are part of the repo tree and we must update
the sources manually:

* `src/mpack/`: [libmpack](https://github.com/libmpack/libmpack)
    * send improvements upstream!
* `src/mpack/lmpack.c`: [libmpack-lua](https://github.com/libmpack/libmpack-lua)
    * send improvements upstream!
* `src/xdiff/`: [xdiff](https://github.com/git/git/tree/master/xdiff)
* `src/cjson/`: [lua-cjson](https://github.com/openresty/lua-cjson)
* `src/klib/`: [Klib](https://github.com/attractivechaos/klib)
* `runtime/lua/vim/inspect.lua`: [inspect.lua](https://github.com/kikito/inspect.lua)
* `src/nvim/tui/terminfo_defs.h`: terminfo definitions
    * Run `scripts/update_terminfo.sh` to update these definitions.
* `runtime/lua/vim/lsp/_meta/protocol.lua`: LSP specification
    * Run `src/gen/gen_lsp.lua` to update.
* `runtime/lua/vim/_meta/lpeg.lua`: LPeg definitions.
    * Refer to [`LuaCATS/lpeg`](https://github.com/LuaCATS/lpeg) for updates.
    * Update the git SHA revision from which the documentation was taken.
* `runtime/lua/vim/re.lua`: LPeg regex module.
    * Vendored from LPeg. Needs to be updated when LPeg is updated.
* `runtime/lua/vim/_meta/re.lua`: docs for LPeg regex module.
    * Needs to be updated when LPeg is updated.
* `src/bit.c`: only for PUC lua: port of `require'bit'` from luajit https://bitop.luajit.org/
* `runtime/lua/coxpcall.lua`: coxpcall (only needed for PUC lua, builtin to luajit)

Operational dependencies
--------------------------

* GitHub users:
    * https://github.com/marvim
    * https://github.com/nvim-winget
* Org secrets/tokens:
    * `CODECOV_TOKEN`
    * `BACKPORT_KEY`
* Org/repo variables:
    * `BACKPORT_APP`
* Domain names (held in https://namecheap.com):
    * neovim.org
    * neovim.io
    * packspec.org
    * pkgjson.org
* The above domains are registered and managed in https://cloudflare.com


Refactoring
-----------

### Frozen legacy modules

Refactoring Vim structurally and aesthetically is an important goal of Neovim.
But there are some modules that should not be changed significantly, because
they are maintained by Vim, at present. Until someone takes "ownership" of
these modules, the cost of any significant changes (including style or
structural changes that re-arrange the code) to these modules outweighs the
benefit. The modules are:

- `regexp.c`
- `indent_c.c`

Automation (CI)
---------------

### Backup

Discussions from issues and PRs are backed up here:
https://github.com/neovim/neovim-backup

### Development guidelines

* CI and automation jobs are primarily driven by GitHub Actions.
* Avoid macOS if an Ubuntu or a Windows runner can be used instead. This is
  because macOS runners have [tighter restrictions on the number of concurrent
  jobs](https://docs.github.com/en/actions/learn-github-actions/usage-limits-billing-and-administration#usage-limits).
* Runner versions:
    * For special-purpose jobs where the runner version doesn't really matter,
      prefer `-latest` tags so we don't need to manually bump the versions. An
      example of a special-purpose workflow is `labeler_pr.yml`.
    * For our testing job `test.yml`, prefer to use the latest version
      explicitly. Avoid using the `-latest` tags here as it makes it difficult
      to determine from an unrelated PR if a failure is due to the PR itself or
      due to GitHub bumping the `-latest` tag without our knowledge. There's
      also a high risk that automatically bumping the CI versions will fail due
      to manual work being required from experience.
    * For our release job, which is `release.yml`, prefer to use the oldest
      stable (i.e. non-deprecated) versions available. The reason is that we're
      trying to produce images that work in the broadest number of environments,
      and therefore want to use older releases.

### Special labels

Some github labels are used to trigger certain jobs:

* `ci:backport release-x.y` - backport to branch `release-x.y`
* `ci:s390x` - enable s390x CI
* `ci:skip-news` - skip news.yml workflows
* `ci:windows-asan` - test windows with ASAN enabled
* `needs:response` - close PR after a certain amount of time if author doesn't
  respond

Vim patches
-----------

Multiple Vim versions are tracked in `version.c`, so that `has('patch-x.y.z')`
works even if `v:version` is "behind". Whenever we "complete" merging all
patches from a Vim `v:version`, the steps to bump `v:version` are:

1. Remove the fully-merged version from `vim-patch.sh` by adjusting the regexp.
   For example, to remove version "8.1":
   ```diff
   diff --git a/scripts/vim-patch.sh b/scripts/vim-patch.sh
   index d64f6b6..1d3dcdf 100755
   --- a/scripts/vim-patch.sh
   +++ b/scripts/vim-patch.sh
   @@ -577,7 +577,7 @@ list_vimpatch_tokens() {
    # Left-pad the patch number of "vim-patch:xxx" for stable sort + dedupe.
    # Filter reverted Vim tokens.
    list_vimpatch_numbers() {
   -  local patch_pat='(8\.[12]|9\.[0-9])\.[0-9]{1,4}'
   +  local patch_pat='(8\.[2]|9\.[0-9])\.[0-9]{1,4}'
      diff "${NVIM_SOURCE_DIR}/scripts/vimpatch_token_reverts.txt" <(
        git -C "${NVIM_SOURCE_DIR}" log --format="%s%n%b" -E --grep="^[* ]*vim-patch:${patch_pat}" |
        grep -oE "^[* ]*vim-patch:${patch_pat}" |
   ```
2. Run `nvim -l scripts/vimpatch.lua` to regenerate `version.c`. Or wait for the
   `vim_patches.yml` CI job to do it.


See also
--------

* https://github.com/neovim/neovim/issues/862
* https://github.com/git/git/blob/master/Documentation/howto/maintain-git.adoc
