Human Web CollectiveBrowser.Pub and Anubis
submitted by
julianHey @js@podcastindex.social, I enabled Anubis for my site, but now requests from Browser.pub don’t work :cry:
Is there something I can write a rule against, perhaps a header or user agent string that is unique to browser.pub?
@julian hmm, I believe it tries to make each AP call from the browser first, then falls back to a cloudflare worker if that fails.I just checked the cf worker and found this gem in it, so that might work for you too
@js@podcastindex.social thanks! 🤘
@js@podcastindex.social Any chance you can update that worker to add in something related to BrowserPub (like the main browserpub url)?
Deno/1.45.2is vague (and also would change, over time :smile: )It’s the
commentpart of the User-Agent header@julian not sure why you'd want to put your public AP stuff behind something like Anubis - a thing in general that I think is pretty anti-web to begin with
you're losing out on all sorts of cool integrations that you don't even know about
@js@podcastindex.social AI crawlers are hammering the site fairy regularly, and my options are to allocate additional resources or put something in front, otherwise my site is inaccessible for hours/days at a time.
Cool integrations are great until they hamper usability of the site 🫤 if the crawlers behaved a bit more like search engine crawlers I’d have no problem.
@js@podcastindex.social Although… now that you mention it, Anubis does have the ability to filter by header values, so I will update the bot policy file to allow the standard ActivityPub accept headers.
@js@podcastindex.social done and tested as working. Looking for the AP Accept headers now. Thanks for the suggestion :smile:
@js @julian PLEASE tell me that Threads is not looking for that exact User-Agent string 🙏🏼🙏🏼🙏🏼🙏🏼🙏🏼