Solving a micro.blog and Cloudflare connectivity issue

I’m a big believer in the philosophy of ‘Post (on) Own Site, Syndicate Elsewhere’, or POSSE, as it has come to be known. My website is the canonical place where I post anything I want to get out into the world. The syndication takes place via micro.blog — it reads the main RSS feed and then does an excellent job of posting the content to my Bluesky, Mastodon and micro.blog accounts. The implementation is very elegant; it got me into the concept of posting small ‘snippets’ or status updates without titles to this blog, which get intelligently posted in full (if small) or in a truncated form to those other sites. I post to my site and it then gets relayed to all of the places I want the content to go to.

I use a free Cloudflare account to accelerate how my site loads for anyone who visits it. Cloudflare serves cached pages and images — i.e., content that other visitors have already requested — from its large content delivery network all over the globe.

A couple of years ago, I started to experience a problem where micro.blog was reporting errors as it tried to read the feed. My suspicion is that when micro.blog polled my website for the feed, Cloudflare was responding with a ‘prove you are human’ challenge or some kind of ‘wait while we verify you’ screen, which isn’t the same as RSS content and therefore caused a failure. What made this difficult to understand was that when I asked for the RSS feed myself, or used another tool such as Feedbin to read the data, everything worked fine. The problem was very specific to wherever micro.blog was sending the request from.

This became a pain in the butt that I’ve lived with for months. Every time I was ready to publish something to my blog, I had to:

1. Log into Cloudflare (which is very aggressive in logging you out after a short period)
2. Disable Cloudflare for the DNS A record for my website, changing it from ‘proxied’ to ‘DNS only’
3. Publish my post
4. Go to micro.blog and force a read of my feed
5. Check the logs to see if the post had been syndicated
6. Go back to Cloudflare and re-enable proxying for the website’s A record, so that anyone visiting my site would be able to load the pages with good performance
   

This routine made me think twice before posting content, particularly short updates, as the effort required was so great.

I’d contacted the micro.blog team a few times and also looked around the web to see if anyone had a solution. I saw that fellow blogger Scott Yoshinaga had been experiencing the same issue. He had decided to stop worrying about it, knowing that his posts probably wouldn’t be syndicated via micro.blog.

This week, with some help from ChatGPT, I came up with a solution that seems to be working. It’s a bit convoluted, but it does the job. In short: I created a feeds.andrewdoran.uk subdomain that bypasses Cloudflare, then used a WordPress plugin to rewrite only media URLs in my feeds to use that subdomain. Here’s how I did it.

Step 1: Create a non-proxied feeds subdomain

Cloudflare has all of the DNS records for my site. I added an additional A record called ‘feeds’, with the same IP address as the main record for andrewdoran.uk. This gives me a new subdomain of feeds.andrewdoran.uk. This A record in Cloudflare is set to ‘DNS only’, i.e. it is not proxied through Cloudflare. When a request comes in to an address at this subdomain, Cloudflare just passes it along to the origin server.

Step 2: Map the subdomain in cPanel

I then had to log into my website’s cPanel, an administrative tool provided by my host, and create a new domain called feeds.andrewdoran.uk. This also has the same details as the main domain entry, with the same ‘document root’. This makes feeds.andrewdoran.uk a mirror/alias of andrewdoran.uk.

Step 3: Enable SSL/TLS for the feeds subdomain

The next step was to ensure that SSL/TLS worked for this new subdomain. In cPanel I went to the SSL/TLS Status administration page and clicked ‘Run AutoSSL’. This applied an SSL/TLS certificate for feeds.andrewdoran.uk.

I updated the feed in micro.blog and could see that it was working — it was downloading the feed with no errors. However, there was one more step.

Step 4: Rewrite media URLs in the feed with a WordPress plugin

When micro.blog reads a feed entry, it will post it to the requested services in a well-thought-out way. This includes images from the post, if any are available. However, the RSS feed entries referred to those images at the main domain, which meant that they were unreachable by the micro.blog service. I needed to find a way to ensure that the images referenced in the feed used the URLs at the mirrored/aliased DNS only subdomain. This can be done by ‘rewriting’ the URLs at the time they are published. But I didn’t want to rewrite all of the URLs published in the RSS feed; sometimes a post contains links to other posts that I have previously written, and I want those links to be to the versions of the posts at the main domain, not the subdomain. The solution to this is a WordPress plugin:

<?php
/**
 * Plugin Name: AD Feed Image Domain Rewrite
 * Description: In feeds only, rewrite media URLs (uploads) from andrewdoran.uk to feeds.andrewdoran.uk so external consumers bypass Cloudflare.
 * Author: Andrew Doran
 * Version: 1.1
 */

function ad_feed_image_domain_swap( $content ) {
    // Only touch RSS/Atom feeds, never normal page views.
    if ( ! is_feed() ) {
        return $content;
    }

    // Get the base URL for media uploads, e.g.
    // https://andrewdoran.uk/blog/wp-content/uploads
    $uploads = wp_upload_dir();

    if ( empty( $uploads['baseurl'] ) ) {
        // If for some reason WordPress can't tell us, bail out safely.
        return $content;
    }

    $origin_base = $uploads['baseurl'];

    // Build the equivalent base URL on feeds.andrewdoran.uk,
    // keeping the path (/blog/wp-content/uploads) the same.
    $feeds_base = str_replace(
        'https://andrewdoran.uk',
        'https://feeds.andrewdoran.uk',
        $origin_base
    );

    // If there's no media URL in this content, don't do any string work.
    if ( strpos( $content, $origin_base ) === false ) {
        return $content;
    }

    // Replace only the uploads base URL.
    // This affects img/src, srcset, <picture> sources, etc. that point
    // into /wp-content/uploads, but leaves normal links to posts alone.
    return str_replace( $origin_base, $feeds_base, $content );
}

add_filter( 'the_content',      'ad_feed_image_domain_swap', 9999 );
add_filter( 'the_excerpt_rss',  'ad_feed_image_domain_swap', 9999 );
add_filter( 'the_content_feed', 'ad_feed_image_domain_swap', 9999 );

This ensures that images in the feeds are referred to at feeds.andrewdoran.uk, whilst regular on-site links continue to point at andrewdoran.uk.

I used cPanel to add this as a PHP file in my WordPress plugins directory. After installing the file and activating it via WordPress, it now checks content at the point of serving it. If the content is an RSS feed, it rewrites the URLs of images to start with feeds.andrewdoran.uk instead of andrewdoran.uk, whilst leaving all other URLs alone.

This feels like a lot of work, with a slightly complex setup. The biggest downside is that my website is directly reachable at the new subdomain without any of the protection that Cloudflare provides from distributed denial of service (DDoS) attacks and bots. But given that I host a tiny blog in a little corner of the web, I’m not a big target, so I’m not worried about this too much. The tradeoff works for me. I can already feel the relief of not having to go through all of the various steps to publish on my blog just to ensure that the posts turn up elsewhere.