1

Commit b14f6d58b4 broke some feeds.

Example: [03:12:30/17858] fetching [https://habr.com/rss/hubs/all/] (force_refetch: )... - Pastebin.com
However it works when I increase CURLOPT_BUFFERSIZE - [03:16:57/18197] fetching [https://habr.com/rss/hubs/all/] (force_refetch: )... - Pastebin.com

2

What did you adjust CURLOPT_BUFFERSIZE to? I am seeing the same errors (unable to fetch: 28 Operation timed out after 15001 milliseconds) on a few feeds after updating to b14f6d5 this evening. The failing feeds work in CURL and https://fakecake.org/myfeedsucks/.

3

512 works just fine for me. This is a random value to just confirm the issue.

4

it works fine here with your feed url (Хабр / Все / Тематические публикации) on debian 9

which platform are you using? also, file a PR i guess

5

Ubuntu 16.04.4 LTS
PHP 7.0.30-0ubuntu0.16.04.1
curl 7.47.0

Sorry, “PR” is … -?

6

strange, should be similar enough to debian. maybe it’s network-related somehow.

pull request with your change

e: if you don’t know how to do that don’t bother, i’ll update this later today

7

I’m on Ubuntu 16.04.4 LTS. I am seeing a couple of the feeds that have been constantly failing occasionally complete a refresh, a couple being Commits for tt-rss.git and Cats.

8

Ah, I see…
https://git.tt-rss.org/fox/tt-rss/pulls/62

9

btw i wonder if it would be a good idea to increase this further, to something like 8192 bytes. we don’t really need the granularity and it would probably improve performance.

10

To be honest, I have no idea.
256 works just fine for me, it fixes the issue.

11

Just did a few tests…
It’s possible to fix the issue by increasing the timeout (used 30 sec instead of 15).

CURLOPT_PROGRESSFUNCTION is too slow, I guess.

So, yes, it’s a good idea to increase the size of the buffer even further.

12

whoa. what kind of hardware are we talking about here?

i would prefer to keep the timeout as-is, increasing the buffer size seems like a better idea. maybe something like 16K.

13

VPS at DigitalOcean

configuration: boot=normal cpus=1 family=DigitalOcean_Droplet uuid=8072F01B-E808-4D84-B47D-67139CA0AB52
 *-cpu
      description: CPU
      product: Intel(R) Xeon(R) CPU E5-2630L v2 @ 2.40GHz
      bus info: cpu@0
      slot: CPU 1
14

looks like writefunction is not affected by buffer size btw:

not sure about progress function

anyway, i’ve increased the buffer to 16384 bytes.

looks like progress function is still called quite a lot, even with a larger buffer size:

[07:44:48/28886] [curl progressfunction] 0 16777216
[07:44:48/28886] [curl progressfunction] 0 16777216
[07:44:48/28886] [curl progressfunction] 0 16777216
[07:44:48/28886] [curl progressfunction] 0 16777216
[07:44:48/28886] [curl progressfunction] 0 16777216
[07:44:49/28886] [curl progressfunction] 0 16777216
[07:44:49/28886] [curl progressfunction] 0 16777216
[07:44:49/28886] [curl progressfunction] 0 16777216
[07:44:49/28886] [curl progressfunction] 0 16777216
[07:44:49/28886] [curl progressfunction] 0 16777216
[07:44:49/28886] [curl progressfunction] 0 16777216
[07:44:49/28886] [curl progressfunction] 0 16777216
[07:44:49/28886] [curl progressfunction] 0 16777216
[07:44:49/28886] [curl progressfunction] 0 16777216
[07:44:49/28886] [curl progressfunction] 0 16777216
[07:44:49/28886] [curl progressfunction] 0 16777216
[07:44:49/28886] [curl progressfunction] 15853 16777216
[07:44:49/28886] [curl progressfunction] 15853 16777216
[07:44:49/28886] [curl progressfunction] 15853 16777216
[07:44:49/28886] [curl progressfunction] 15853 16777216
[07:44:49/28886] [curl progressfunction] 15853 16777216
[07:44:49/28886] [curl progressfunction] 15853 16777216
[07:44:49/28886] [curl progressfunction] 15853 16777216
[07:44:49/28886] [curl progressfunction] 15853 16777216
[07:44:49/28886] [curl progressfunction] 15853 16777216
[07:44:49/28886] [curl progressfunction] 15853 16777216
[07:44:49/28886] [curl progressfunction] 32229 16777216
[07:44:49/28886] [curl progressfunction] 32229 16777216
[07:44:49/28886] [curl progressfunction] 37188 16777216
[07:44:49/28886] [curl progressfunction] 37188 16777216
[07:44:49/28886] [curl progressfunction] 37188 16777216
[07:44:49/28886] [curl progressfunction] 37188 16777216

etc etc

15

I had been testing increased timeouts since it seemed to be a timeout issue based on the error messages I saw in the GUI. I removed all that from and added MAX_DOWNLOAD_FILE_SIZE = 0 to config.php per your recommendation and will monitor it today.

16

You can try to turn off curl. Define NO_CURL in config.php.

17

you can also define MAX_DOWNLOAD_FILE_SIZE as 0 in config.php which should disable max size checking and progress function

also: try increasing block size, instead of timeouts.

18

I removed any extra declarations from config.php and performed a git update yesterday morning. Everything’s been working fine.