Enable Internet Archive / Wayback Machine to crawl Publish websites

Feature requests
,
1

Use case or problem

It appears the Internet Archive cannot make snapshots of Obsidian Publish sites. Only 404 pages appear in the Wayback Machine.

I have confirmed that my Publish site allows web crawlers and is discoverable. The issue appears to be unique to the Internet Archive because alternative archive sites (like archive.today) can create snapshots without issue.

Proposed solution

Make it possible for the Internet Archive to properly crawl and create snapshots of Publish pages.

Current workaround (optional)

I have not been able to come up with a workaround.

Related feature requests (optional)

n/a

2 Likes
2

Just to clarify, we do not block Internet Archive. We think that their crawler isn’t not able to run and then save Obsidian Publish pages (which are not static HTML).