OK, so outgoing links, not backlinks. There is no native way to hide certain file types.
For the file explorer, there is a css snippet - in a rush, I couldn’t find out how to adapt this snippet for the outgoing links pane but it should basically be possible, I guess. There is also a feature request for hiding certain file types in the file explorer.