That’s what full screen does. It hides the title bar.
Even when you are in full screen mode, you still have access to the commands, “Navigate back”, and “Navigate forward”, which you can assign to a hotkey.
By default on MacOS, it is Cmd-Alt-left and Cmd-Alt-right arrow keys. On Windows, I assume it is Ctrl-Alt-left and Ctrl-Alt-right.
And you can still close the window by using whatever app closing hotkey you have in your OS. (Cmd-Q in Mac, maybe alt-F4 in Windows.)