No, it will all be handled by Obsidian. You include them when uploading just like any other Markdown file.
This video might help https://youtu.be/_9mOVTztawA?t=223, or you can search for “Obsidian Publish” on YouTube to see if any video helps.
I think it’s likely going to be really straightforward once you try it, because many of the sites contain images and media (for example https://publish.obsidian.md/anthony-agbay/Retina), and we haven’t received reports or questions about “how do I do that?”
In general I think it should be easier than you think, because using Obsidian Publish requires no knowledge of HTML of FTP. You don’t even need to own a domain! The custom domain feature is new, and the setup process is entirely independent of the uploading process.