The default theme doesn’t use icons in the file explorer. Does the theme you’re using support the Style Settings community plugin? If so, it might have a setting for that (I’ve seen at least one theme that lets you turn them on/off).
In the Community themes selection menu, the screenshots of some themes will say they are customizable with the Style Settings plugin. I do not know of a list or any other method of knowing absent installing and verifying each one. To list a few:
Yin and Yang does support the Style Settings plugin. Unfortunately I don’t see a setting to remove those icons. The nearest thing I found is Minimalize Elements > File Explorer where you can Remove Custom Icons (I saw no change) and Show Folder Icons.
You can use a CSS snippet to hide the icons, but I don’t know the specific elements to select. You could also check the theme’s Github and its issues page for info, or try a different theme.