Back and forward buttons (and hotkeys) that are specific to a particular side panel would be very helpful.
That way we get a back/next navigation just like each tab in the browser has its own independent history.
The current navigate back and forward behaviour is useful in specific situations, but doesn’t make it possible to go back in a specific panel (and have the risk that you ‘override’ the other panels).