I’d extend that myself to say: ctrl+n/p and/or ctrl+j/k should work in lieu of up/down arrow everywhere arrows currently work.
One issue with ctrl-u and ctrl-n, though, might be that they are by default taken by the browser. ctrl+j/k (vim-like) for up/down should be OK. Roam uses those.