Open settings json file in windows terminal (ctrl+shift+,), find "actions" json array, add two commands (don't forget about preceding coma):
From now on pressing Alt+Up will go one level up (like in Windows Explorer) and pressing Alt+Down will bring powershell hint for switching the directory:
The trick here is to use \u0000 which is a terminal equivalent of pressing Ctrl+Space. It is applicable only to powershell, since there is no directory browsing feature in a regular cmd.