Hey folks,
Recently, I’ve upgraded my Tiny Tiny RSS setup from the (fairly old) Debian 11 package to the latest git version on Debian 12. This also meant a jump from PHP 7.4 to PHP 8.2.
Unfortunately, after the migration, all (really, all, not just a subset) posts got duplicated. I searched and searched for solutions unsuccessfully, and also read about how this just happens sometimes and one way to deal with it is to just start over. Well, starting over was a non-starter for me, so I dug in my heels and solved my problem.
I’d like to share what I discovered in the hopes that it might help other TT-RSS users with similar upgrade situations.
For context, what one needs to understand is that when TT-RSS inserts a new post into its database, it generates a unique identifier that takes into account a number of things including (but not limited to) the title, URL, post content, etc. When a feed is updated, it calculates identifiers for all posts comes across, and if that identifier doesn’t already exist in the database, it considers that post to be new and adds it.
Code change
One of the first things I discovered is that there has been a code change in classes/rssutils.php that caused identifiers to change. I reverted the change on my local instance and this helped fix one of the ways the identifier was changing in comparison with the previous version of TT-RSS.
Plugins
Secondly, I didn’t notice it at first but identifiers seem to also depend on plugins. In the new version I deployed, some plugins were moved out of the core and are now distributed via external repositories (link missing, can’t add it). Re-installing plugins which were previosuly enabled in my old setup also helped with the identifiers issue.
JSON
Lastly, even when the identifiers were identical, TT-RSS kept on duplicating entries. This time I had a look on the database side of things. Sure enough, in the new installation, entries in the guid column of the ttrss_entries table had a subtle, yet significant difference: the old installation had JSON bits like "uid":"<number>" whereas the new version created the same entries but with different quoting: "uid":<number>. The most likely culprit is difference in the PHP json module between 7.4 and 8.2.
The fix for this was to run a database query to convert the preexisting entries to the new format:
update ttrss_entries set guid = REGEXP_REPLACE(guid, '"uid":"([0-9]+)"', '"uid":\\1');
Conclusion
After these three adjustments, the upgrade was finally a success and the updater stopped duplicating all the existing posts. Such a massive relief! So if you’re upgrading from a very old TT-RSS and are facing a similar problem, try the above and let us know what you find out!
