As a webmaster, I see Tiny Tiny RSS in my logs and see it does not use data compression (accept-encoding header). As all modern HTTP clients support this, we decided to block those.
Why is the software not using data compression? It is obvious that this saves a lot of data to be transferred, which is good for publishers, users, and data centers.
Addition: I’m not a user of TT-RSS, but see it a lot showing up in our log (multiple IP addresses). That is why I took the time to register here and submit the issue.
Yes, and no. You don’t have to disable compression, because the compression method itself isn’t a vulnerability in itself. Only with the right conditions (TLS, compression, user input, sensitive information) it can be an issue. But that is something the server owner or application owner should consider. All common browsers use compression/Accept-Encoding as well (gzip, brotli, deflate, sometimes zstd).
Addition: older version has no problem using data encoding: 2024-04-20T22:20:04+00:00 200 x.x.x.x "GET /feed/ HTTP/2.0" 36267 "-" "Tiny Tiny RSS/23.09-f489f620 (https://tt-rss.org/)" TLSv1.3/TLS_AES_256_GCM_SHA384 0.000 .
So maybe something in the code changed last 6-7 months?
It was the switch to Guzzle. Previously CURLOPT_ENCODING was set to "", which resulted in all options supported by the local curl lib being passed via Accept-Encoding. In contrast, Guzzle (for whatever reason) doesn’t send Accept-Encoding by default-- but could be made to do so by setting the aforementioned curl option, doing something like 'decode_content' => 'gzip' (or whatever’s supported), setting a default Accept-Encoding: gzip (or whatever’s supported), etc.
I’m here in good faith, with the intention of contribution an observation I made after reviewing our log files. As said, I’m not using TT RSS myself, but took the time and effort to create an account here and report an issue.
Regarding the “we decided to block those”. Not sure where I implied that we are important or why that would make me an internet weirdo. We simply implemented something on our web server to save data traffic. We also blocked HTTP/1.0 requests. How does that make us important in any way? These measures were taking to block old clients that usually don’t add much value (to us). It is also part of our effort to provide a more sustainable website, by reducing our footprint. With the change we discovered a few interesting things, including blocking some legitimate clients (including RSS readers, Nextcloud). Instead of reversing the change, we decided to report the issues to the respective software tools, so they can consider activating data encoding and improving things. Most of them already took an action, which shows me that they also believe in the importance of data compression.
The blog post that @wn_name shared, is actually written by me. And seeing the other link that the underlying HTTP client was switched to Guzzle recently, does perfectly explain why older version do the compression and new ones not. That is actually great! I guess it is just a matter of waiting till the next release, if that one uses the recent changes to Guzzle. That particular change to Guzzle was initiated by an involved developer that made the change after a similar reported issue to the other project (Selfoss).
While typing this message it shows me some community guidelines (be kind, does the reply improve the conversation? Criticize ideas, not people). If you want people to contribute, then I guess calling me some self-important internet weirdo is not in line?
blocking people to save some pittance of egress traffic definitely puts you into weirdo category in my book.
you casually mentioning it in your post i’m personally parsing as a misguided attempt to strongarm us into implementing changes for your unspecified website.
i don’t like weirdos and i don’t like being strongarmed.
that’s likely some stock discourse copywrite nobody cared enough to edit out.
I explained already why I report issues and simply hope for a better web. Not sure where strong arming and other things come from, but you are free to interpreter things like you want. That, and calling people names, is a sign to me. I consider this thing closed, and marked the helpful comment by @wn_name as the solution. I wish you well.