1

When the updater container is gracefully stopped (by docker or kubernetes), it should remove the lock/update_daemon.lock file, so that another instance of this container can continue the updates.

It’s currently not always the case. I suppose it happens when you stop the container during an update.

This happens to me under kubernetes (if the corresponding pod is gracefully killed by k8s during an update, for any good reason), but the same issue probably affects the updater when run by docker-compose (I did not check).

I see there is already code to handle that in update_daemon2.php. I suppose this could be fixed by handling process signals in updater.sh, but I’m not 100% sure.

2

unless i’m missing something here lockfile simply being there shouldn’t prevent updates, unless it is flock()ed. if it’s not, updater assumes it crashed or something and goes on updating.

3

I just reproduced the problem I try to solve (on k8s):

I manually stop the updater container while it is running.
It leaves several lock/update_daemon-*.lock files.

After that, the next updater container fails with this error message:

error: Can’t create lockfile. Maybe another daemon is already running.

So it restarts in loop, and does not update feeds any more, until I manually remove the lock files.

1 Like
4

https://dev.tt-rss.org/tt-rss/tt-rss/src/branch/master/include/functions.php#L367

maybe you’re missing flock() or it doesn’t work correctly.

e: i’ll remove this function_exists() check altogether and make it always fail if flock() is not present.

5

My data is on a remote NFS server (mount with “nolock” option on the client side). This might be a reason?

6

yes, this might not work as you expect.

require PHP to have support for flock() · 4d825fa6a6 - tt-rss - Tiny Tiny RSS flock() is mandatory now anyway.

NFS

i must comment that stuff like this is why you don’t put any random thing into k8s.

anything that needs persistence - unless it’s S3 or something that lives outside of your cluster - doesn’t really belong.

i mean, i’m not going to tell you how to run your clusters, but you’re doing it wrong. tt-rss is definitely not ‘cloud native’. :slight_smile:

7

Thanks for your help on the investigation.

I slightly disagree on the vision of k8s.
It’s true that tt-rss is not cloud-native, and that it needs persistence.
It’s also true that k8s has been initially designed for stateless workloads.

However, k8s can bring many advantages over docker-compose (at least ability to spread pods among several nodes), and works fine with stateful workloads and/or non-cloud-native apps, as long as you keep only one replica for each deployment, and don’t expect to have all the nicest k8s advantages (like zero-downtime if you stop a pod or a node, ability to scale to handle many requests etc)

I currently self-host many things on a home k8s cluster (actually, it’s k3s, but it does not matter), with all data (PVs) on an NFS server. Wordpress, Nextcloud, Kanboard, Matomo, Piwigo (to cite only the ones that use a similar language/architecture): none of them are cloud-native, they are clearly stateful, and most of them even need to put the PHP files on NFS.
Nextcloud, for example, provides a Helm chart.

You even provided some manifests to do that (thanks for that).

However, my problem indeed probably comes from NFS. In this case, it’s highly probable that I would have the same problem with docker-compose+NFS, but I did not check.

(I wanted to post a new message to give ideas I have to make a graceful shutdown remove the lock files, but I don’t have the right to post to 2 successive messages here, yet)

8

around here (speaking as a senior infrastructure admin) first thing we ask from developer teams when they want to migrate their stuff into k8s is whether it has any persistence. if they do, we tell them to come back after they remove it.

you can of course run whatever you want, however you want, in your homelab cluster, but you’re getting used to objective antipatterns which are not used anywhere in the real world.

and if you come anywhere near any semi-competent devops team suggesting to use NFS of all things (to store PHP files on, oh god…) you’ll get laughed out of the building.

exactly. that was me, a year ago, playing around with my homelab cluster while having zero understanding of how those things are actually used. a lot has changed since then. many things i had to unlearn.

p.s. even way back then i wouldn’t ever consider using NFS for PVs unless this was an actual disposable lab environment.