RSS feeds with auth won't show media protected by auth (regardless of af_http_proxy)

Tiny Tiny RSS Support
1
  • [x] I’m not using docker on my primary instance, but my issue can be reproduced on the aforementioned docker setup and/or official demo.

Issue

TinyTiny RSS fetches RSS feeds behind authentication, but even with the af_proxy_http plugin set to proxy all images, authentication is not used for fetching media attachments, breaking embedded media.

Test case

Steps

  1. On the official demo instance, subscribe to a new RSS feed
  2. Subscribe to the feed
  3. Navigate to the Fox / @hourlyFox feed
  4. Try to view the photos attached to posts

Note: The demo instance does not have af_proxy_http installed. Testing with a local (non-Docker) instance with af_http_proxy installed and set to proxy all requests doesn’t address the issue.

Expected

TinyTiny RSS either provides a way to proxy media attachments for authenticated RSS feeds (via a new plugin or a setting in af_http_proxy to proxy authenticated media attachments from the same domain), or provides a way to forward/embed authentication requests from the server to the browser/mobile client (e.g. rewriting media attachments served from the same domain with https://<username>:<password>@example.com/…).

I’m not sure how difficult this would be - it’s fine to say this is a known issue/no fix will be provided.

Actual

Images fail to load, requiring viewing the original article (including launching the browser app on Android).

Version details

  • Tiny Tiny RSS version (including git commit id):
    • Official demo: Tiny Tiny RSS v22.04-b17b4a4
    • Local install: Tiny Tiny RSS v22.06-59b0ae8af (Unsupported)
  • Platform (i.e. Linux distro, Docker, PHP, PostgreSQL, etc) versions:
    • Ubuntu 20.04
    • Direct/legacy git install
    • PHP: 7.4.3
    • PostgreSQL: 12.11 (Ubuntu 12.11-0ubuntu0.20.04.1)

Context

The public NixNet Nitter instance is behind username nitter and password nitter due to misinformed automated abuse reports (Nitter is a Twitter proxy, not a host):

I’m able to use the Authentication tab in Tiny Tiny RSS to access the RSS feed, but this does not help with accessing media attachments.

It is possible to get Firefox to fetch the media by rewriting the URL.

Feed-provided URL: http://nitter.nixnet.services/pic/media%2FFV2u6CSWYAE83ch.jpg
Rewritten URL: https://<username>:<password>@nitter.nixnet.services/pic/media%2FFV2u6CSWYAE83ch.jpg

However, there doesn’t seem to be a way to get TT-RSS to apply this automatically.

2

af_proxy_http works on URLs, i don’t think its aware of the originating feed.

you can try enabling caching for this feed in settings, this way any sort of further proxying/resizing happens on your local copy.

3

Ah, good point! And I entirely overlooked the Cache media option; that seems like the ideal way to handle this.

Unfortunately, it appears that cache_media_url(…) will never pass the feed’s $auth_login/$auth_pass to UrlHelper::fetch(…). Favicon fetching (lower priority) doesn’t pass auth along, either.

Optional, suggested change

cache_media_url(…) could check if the requested $url belongs to the same domain as the RSS feed itself ($site_url?), and if so, forward on the feed’s authentication username/password.

It’s also possible to unconditionally apply the username/password, but that seems like it risks leaking authentication credentials. Limiting it to the same domain (maybe including subdomains?) seems reasonable.

In the process of debugging this, it appears that update_favicon(…) fetching won’t use authentication either.

Feed debugging logs

[18:47:53/2819] site_url: http://nitter.nixnet.services/hourlyFox
[18:47:53/2819] feed_title: Fox / @hourlyFox
[18:47:53/2819] favicon: needs check: 1 is custom:  avg color: 
[18:47:53/2819] favicon: trying to update favicon...
[18:47:54/2819] favicon: fetching http://nitter.nixnet.services/favicon.ico failed
[…favicon fetch failed…]
[18:47:54/2819] =================================================================================================================================
[18:47:54/2819] guid 2,http://nitter.nixnet.services/hourlyFox/status/1540031947027742721#m (hash: {"ver":2,"uid":2,"hash":"SHA1:a5f6919df10cf5dcecd4e72e7800825e431c63e6"} compat: SHA1:8770802fc990f87c39f217fa32742dd746276dbd)
[…]
[18:47:54/2819] cache_media: checking http://nitter.nixnet.services/pic/media%2FFV9K5OeWIAQJ2WO.jpg
[18:47:54/2819] cache_media: downloading: http://nitter.nixnet.services/pic/media%2FFV9K5OeWIAQJ2WO.jpg to 856639b89680df2adb98f210fe922fc8b813d17b
[18:47:54/2819] cache_media: failed with 401: HTTP Code: 401 
[…cache_media fetch failed…]
4

i’d rather not do anything unpredictable with credentials, password protected feeds are a niche anyway, just upload the favicon manually.

5

Understood!

The favicon I can easily deal with, the media inside the feeds not so much. However, I agree that this is an uncommon problem. If I find the time, I may look at how difficult it’d be to add another setting to Options (e.g. Use authentication for caching media as a sub-setting of Cache media), or trying a custom plugin to intercept media fetches for this domain.

6

tbh now that you made me think about it, i’m not sure if requesting anything but the actual feed URL
with stored credentials is a good idea.

i’m not going to bother checking how this works now, but i don’t remember implementing any domain-based controls, so its possible that those are leaked already to any media domains included by the feed. not that limiting this to an actual domain would be any better, a higher-level domain for example could belong to a different organization entirely.

we could keep previous behavior behind a global config tunable something like TTRSS_I_DONT_CARE_ABOUT_MY_FEED_PASSWORDS or w/e although i’d rather not.