Neovim display bug/artefacts

I am trying to set up neovim on a remote machine (accessed by ssh) and while doing so I noticed visual artefacts, especially across windows when using visual mode line selection. For example, with a visual line select on the right window, bleeds into the left window.

visual bug with selection

After hitting <C+l>

Some things that may be relevant:

$ echo $TERM
xterm
$ cat /etc/*-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.6 LTS"
NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.6 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

Any ideas how I could fix this / debug this? This does not seem to happen in vim