Vim.diagnostic.open_float window won't stay open

I have followed this guide: UI Customization · neovim/nvim-lspconfig Wiki · GitHub

But the window closes itself after like a second. Even if I manually run :lua vim.diagnostic.open_float(nil, opts) it does the same thing. I also have a keymapping to manually trigger the float, same issue. The float window closes itself after like a second.

CleanShot-2022-06-23-at-08.27.21

If I go into command mode, it triggers the popup window :thinking:

CleanShot-2022-06-23-at-08.32.19

Going into insert mode temporarily fixes it. Until I move my cursor again.