How to make cursor color work in tmux? (iterm2, windows terminal)

Neovim
1

According to the FAQ, I set the cursor like this

set termguicolors
hi Cursor guifg=#17afff guibg=#17afff
au VimEnter,VimResume * set guicursor=n-v-c:block,i-ci-ve:ver25,r-cr:hor20,o:hor50
  \,a:blinkwait700-blinkoff400-blinkon250-Cursor/lCursor
  \,sm:block-blinkwait175-blinkoff150-blinkon175
hi CursorReset guifg=#c7c7c7 guibg=#c7c7c7
au VimLeave,VimSuspend * set guicursor=a:block-blinkwait700-blinkoff400-blinkon550-CursorReset/lCursorReset

The cursor color works fine in iTerm2, becomes blue (17afff) when entering neovim and resets to white (c7c7c7) when exiting, but the color does not show up if Neovim is inside tmux.

EDIT: to my surprise, the cursor color when entering neovim works in Windows Terminal + tmux. The issue for windows terminal + tmux is the cursor color does not reset to white when exiting neovim. So this is a terminal related issue?

Some key settings in tmux conf, I even have the line in the FAQ for cursor shape in tmux, but that didn’t help, and even without the line the cursor shape is still fine. Tmux is launched with -2 option to force true color.

set -g default-terminal "tmux-256color"
#set -as terminal-features ",xterm-256color:RGB"
#set -ga terminal-overrides ",*256col*:RGB"
set -as terminal-features ",*:RGB"
set -ga terminal-overrides ",*:RGB"
set -ga terminal-overrides '*:Ss=\E[%p1%d q:Se=\E[ q'