Sorry, my fault - it is ALT (not SHIFT) + Click, as WhiteNoise explained here. This explains the strange behavior observed above - thus no bug (and I was pretty certain I hadn’t observed such a behavior before - now I know why… 
).
So, just to be sure, after selecting any folder or file (or several of them), you can both use the DELETE key and right click in order to delete it.