How can I enable the line numbers for the manpager?

I want to enable the line numbers by default when using the MANPAGER='nvim +Man!' environment variable. How can I do that?

export MANPAGER=‘nvim +Man! “+set number”’