#1

If you update a file quite frequently it seems Obsidian Publish doesn’t reflect the change. Also it seems to struggle with H2s. I uploaded content and it showed everything + H2s in the Table of Contents… just not the content. So still quite buggy.

1 Like
#2

+1 – I’ve updated my index page, waited over an hour and the changes have not been published.

#3

Sorry about that! Does clear browser cache work for you guys?

@Licat Could you take a look at this cache issue?

#4

Clearing cache worked for me; great short term fix - thanks. But in the long term, it would be great if you guys could make it work without clearing cache.

#5

Hi @scott1, definitely. The initial decision to introduce cache was to make the page load faster, and reduce unnecessary loads. But if it gets in the way, we should definitely do something about it.

#6

I’ll copy over my response from Discord:

Caching is done for a few reasons:

  • First having a cached version makes the page load noticeably faster; it often makes a difference between 500ms vs 200ms from where I live. In areas further from our server (located in the west cost US), this could take well over a second if cache isn’t available.
  • Second is bandwidth, which usually a concern for when using mobile data. Obviously the more things cached the less bandwidth cost to re-download metadata of the site and pages.
  • And third this also normalizes server resource usage so one site having lots of visitors won’t affect others thanks to intermediate caches by our CDN.

Given that the vast majority of the time, sites aren’t being actively changed, but passively read, it doesn’t make much sense to have cache disabled.

We indicate this clearly once you finish the publishing step in side the app:

image
image696×314 21.8 KB

#7

Unfortunately, the way caching works is by telling the browser: “here’s this file, assume it’s unchanged for the next hour and don’t bother checking back until after that time has passed”. The whole point is to avoid the round-trip(s) of loading the data so the page can load much faster. The only way to defy that is by clearing or disabling the cache manually. We don’t have a way to tell the browser that the underlying file has changed if the browser doesn’t check back with us during that time period.

As such, we’ll probably close this issue for now. Hopefully the workaround to disable cache/hard refresh is sufficient for most users.

1 Like
#8

Not necessarily so because you’re using Cloudflare. If you use a low browser cache on Cloudflare (say 5 min) and a high edge cache (say 1 day), then the asset is cached on the Cloudflare POP for a day, and thus saving bandwidth and being served at a higher speed, without having to use a higher browser cache at the same time.

For one of my websites on Cloudflare I set cache-control to public, max-age=0, must-revalidate, which essentially tells the browser “don’t cache this HTML page, and always check back with the server”.

Yet my site is very fast (35 ms HTML page download) because I’ve cached the pages on Cloudflare’s POPs for 1 week. (When I update the website, I purge the Cloudflare cache.)

Obsidian could use a similar model by offloading the assets to Cloudflare’s POPs, and still maintaining a low browser cache so that updates can be published quickly.

Another benefit of using a low browser cache (and a high Cloudflare edge cache) is that Cloudflare automatically evicts assets from its cache that aren’t requested frequently. If the browser cache is for instance one day for an asset, then there are a lot less requests for that asset in Cloudflare’s POP and so it seems less popular (and with a higher cache of being evicted from cache earlier than the edge cache expires).

I’m not sure about this one. If you use the Last-Modified HTTP header, the browser will (should) check back with the server to see if the asset has changed since its last visit.

If it didn’t change, the server should return the 304 - Not modified status code, which prevents the browser from having to download the asset again. If the asset did change, the server should send the regular HTTP response with the updated asset, which the browser then receives.

You can also use the ETag HTTP header to have the browser check in with the server.

You’re right though that with the Cache-Control HTTP header there’s no additional check in with the server, so with that approach there’s indeed no way to tell the browser that the local cache asset may (or may not) have changed.

More about caching (not saying you need this, but for interested readers):

1 Like
#9

@JkNML Thank you for such a detailed reply!

I believe Cloudflare has a minimal Browser TTL of 30 min as seen from the page rule. Cloudflare will override the browser TTL regardless of what your server actually sends, and the default was something like 10 days (if this rule isn’t set).

image
image777×279 5.53 KB

Without page rules, cloudflare only caches URLs that ends in certain extensions (https://support.cloudflare.com/hc/en-us/articles/200172516#h_a01982d4-d5b6-4744-bb9b-a71da62c160a). So the only way for us to get caching to work is by adding a page rule.

Can you give an example of how you’re able to configure such a thing?

Yes we have Last-Modified as well as ETag implemented. My quote refers to the fact that our Cache-Control is public, max-age=3600, which means the browser won’t revalidate.

Thanks for the list. Seems like I’ve already read these resources a dozen times during implementation already :joy:

#10

Ah that is true. I don’t use page rules myself so I don’t knew about this 30 minute limitation.

I use a Cloudflare Worker to set the Cache-Control header on the assets. That makes it possible to set any caching rule possible, including the ‘store on edge up to a month’ and set the browser cache to zero.

Even though this works great, for handling geographical latency caching isn’t that great I think, because cache MISS response on Cloudflare will still have a high latency response.

For instance, the https://publish.obsidian.md/app.js asset for me just now had a cache MISS and a 1.29 seconds load time (I’m in Europe).

The primary reason I use a Cloudflare Worker script is to handle requests in such a manner that visitors to my website from different continents and countries are directed to the Azure blob storage that’s most nearby.

Yes but could you use in your environment Last-Modified/Etag for the HTML without Cache-Control directives? That should make browsers download the new page right after changes are published to Obsidian Publish.

#11

That is really smart! I’d love to give that a try. Any resources you recommend for setting up something like this?

That’s something I thought of doing too, basically cache-busting from the client using the initial (always uncached) HTML to check for resource updates.

#12

The Workers doc are of course a good place to start.

When a Worker receives a HTTP request (like for a HTML page or .css file), it will need to get those resources from your origin server with the fetch() function.

Since that is a request, you can configure the fetch() operation with several of the properties listed here, including cacheTtl which is the time (in seconds) that Cloudflare caches the assets in the data centre where the request happened.

You can then modify the response you get from the origin by setting the Cache-Control header to, for example, 5 minutes.

Here is a quick demo with the following code:

addEventListener('fetch', event => {
  event.respondWith(handleRequest(event.request))
})

/**
 * Fetch and log a given request object
 * @param {Request} request
 */
async function handleRequest(request) {
  // Make request and store on Cloudflare POP for one month
  const response = await fetch(request, { cf: {
    cacheTtl: 2592000
  }});

  // Adjust headers from origin to return to browser
  const newHeaders = new Headers(response.headers);

  // Set browser caching to 5 minutes
  newHeaders.set("Cache-Control", "public, immutable, max-age=300");
  
  // Return updated response 
  return new Response(response.body, {
    headers: newHeaders,
    method: request.method,
    status: response.status
  });
}

Note that this approach does depend on purging the cache given the long caching period that Cloudflare uses on its CDN.

(With the Cloudflare API you can purge 30 URLs at a time, in case one person’s published Obsidian needs to be removed.)

#13

Looks like I can actually make my life a lot easier with this, and I can think of several other things that could improve the way things work right now.

Thanks again.

1 Like