PHP Fatal error - open-telemetry - SpanContextValidator.php

Kolo · October 22, 2023, 8:34am

[x] I’m using docker compose setup, with modifications (modified .yml files, third party plugins/themes, etc.) - if so, describe your modifications in your post. Before reporting, see if your issue can be reproduced on the unmodified setup.
→ just added labels to exclude from Watchtower

After patching and rebooting my Docker host yesterday, my tt-rss containers app and updater will no longer start. Both containers report the same PHP Fatal error every view seconds:

updater’s loop:

db:5432 - accepting connections
[07:18:32/9] Installing shutdown handlers
[07:18:32/9] Spawned child process with PID 11 for task 0.
[07:18:32/9] Spawned child process with PID 13 for task 1.
PHP Fatal error:  Uncaught Error: Call to undefined function OpenTelemetry\API\Trace\ctype_xdigit() in /var/www/html/tt-rss/vendor/open-telemetry/api/Trace/SpanContextValidator.php:25
Stack trace:
#0 /var/www/html/tt-rss/vendor/open-telemetry/api/Trace/SpanContext.php(35): OpenTelemetry\API\Trace\SpanContextValidator::isValidTraceId()
#1 /var/www/html/tt-rss/vendor/open-telemetry/api/Trace/SpanContext.php(110): OpenTelemetry\API\Trace\SpanContext->__construct()
#2 /var/www/html/tt-rss/vendor/open-telemetry/api/Trace/SpanContext.php(122): OpenTelemetry\API\Trace\SpanContext::create()
#3 /var/www/html/tt-rss/vendor/open-telemetry/api/Trace/Propagation/TraceContextPropagator.php(101): OpenTelemetry\API\Trace\SpanContext::getInvalid()
#4 /var/www/html/tt-rss/vendor/open-telemetry/api/Trace/Propagation/TraceContextPropagator.php(89): OpenTelemetry\API\Trace\Propagation\TraceContextPropagator::extractImpl()
#5 /var/www/html/tt-rss/classes/tracer.php(52): OpenTelemetry\API\Trace\Propagation\TraceContextPropagator->extract()
#6 /var/www/html/tt-rss/classes/tracer.php(96): Tracer->__construct()
#7 /var/www/html/tt-rss/classes/tracer.php(91): Tracer::get_instance()
#8 /var/www/html/tt-rss/classes/pluginhost.php(448): Tracer::start()
#9 /var/www/html/tt-rss/include/functions.php(437): PluginHost->load()
#10 /var/www/html/tt-rss/update.php(78): init_plugins()
#11 {main}
  thrown in /var/www/html/tt-rss/vendor/open-telemetry/api/Trace/SpanContextValidator.php on line 25PHP Fatal error:  Uncaught Error: Call to undefined function OpenTelemetry\API\Trace\ctype_xdigit() in /var/www/html/tt-rss/vendor/open-telemetry/api/Trace/SpanContextValidator.php:25
Stack trace:
#0 /var/www/html/tt-rss/vendor/open-telemetry/api/Trace/SpanContext.php(35): OpenTelemetry\API\Trace\SpanContextValidator::isValidTraceId()
#1 /var/www/html/tt-rss/vendor/open-telemetry/api/Trace/SpanContext.php(110): OpenTelemetry\API\Trace\SpanContext->__construct()
#2 /var/www/html/tt-rss/vendor/open-telemetry/api/Trace/SpanContext.php(122): OpenTelemetry\API\Trace\SpanContext::create()
#3 /var/www/html/tt-rss/vendor/open-telemetry/api/Trace/Propagation/TraceContextPropagator.php(101): OpenTelemetry\API\Trace\SpanContext::getInvalid()
#4 /var/www/html/tt-rss/vendor/open-telemetry/api/Trace/Propagation/TraceContextPropagator.php(89): OpenTelemetry\API\Trace\Propagation\TraceContextPropagator::extractImpl()
#5 /var/www/html/tt-rss/classes/tracer.php(52): OpenTelemetry\API\Trace\Propagation\TraceContextPropagator->extract()
#6 /var/www/html/tt-rss/classes/tracer.php(96): Tracer->__construct()
#7 /var/www/html/tt-rss/classes/tracer.php(91): Tracer::get_instance()
#8 /var/www/html/tt-rss/classes/pluginhost.php(448): Tracer::start()
#9 /var/www/html/tt-rss/include/functions.php(437): PluginHost->load()
#10 /var/www/html/tt-rss/update.php(78): init_plugins()
#11 {main}
  thrown in /var/www/html/tt-rss/vendor/open-telemetry/api/Trace/SpanContextValidator.php on line 25

[07:18:33/9] Child process with PID 11 reaped.
[07:18:33/9] Child process with PID 13 seems active but lockfile is unlocked.
[07:18:33/9] Received SIGCHLD, 0 active tasks left.
[07:18:33/9] Received SIGCHLD, 0 active tasks left.
[07:19:32/9] 0 active tasks, next spawn at 60 sec.

app’s loop:

db:5432 - accepting connections
updating tt-rss source in /var/www/html/tt-rss from https://git.tt-rss.org/fox/tt-rss.git...
From https://git.tt-rss.org/fox/tt-rss
 * branch                master     -> FETCH_HEAD
Already up to date.
updating plugins.local/nginx_xaccel...
From https://git.tt-rss.org/fox/ttrss-nginx-xaccel
 * branch            master     -> FETCH_HEAD
Already up to date.
NOTICE:  extension "pg_trgm" already exists, skipping
PHP Fatal error:  Uncaught Error: Call to undefined function OpenTelemetry\API\Trace\ctype_xdigit() in /var/www/html/tt-rss/vendor/open-telemetry/api/Trace/SpanContextValidator.php:25
Stack trace:
#0 /var/www/html/tt-rss/vendor/open-telemetry/api/Trace/SpanContext.php(35): OpenTelemetry\API\Trace\SpanContextValidator::isValidTraceId()
#1 /var/www/html/tt-rss/vendor/open-telemetry/api/Trace/SpanContext.php(110): OpenTelemetry\API\Trace\SpanContext->__construct()
#2 /var/www/html/tt-rss/vendor/open-telemetry/api/Trace/SpanContext.php(122): OpenTelemetry\API\Trace\SpanContext::create()
#3 /var/www/html/tt-rss/vendor/open-telemetry/api/Trace/Propagation/TraceContextPropagator.php(101): OpenTelemetry\API\Trace\SpanContext::getInvalid()
#4 /var/www/html/tt-rss/vendor/open-telemetry/api/Trace/Propagation/TraceContextPropagator.php(89): OpenTelemetry\API\Trace\Propagation\TraceContextPropagator::extractImpl()
#5 /var/www/html/tt-rss/classes/tracer.php(52): OpenTelemetry\API\Trace\Propagation\TraceContextPropagator->extract()
#6 /var/www/html/tt-rss/classes/tracer.php(96): Tracer->__construct()
#7 /var/www/html/tt-rss/classes/tracer.php(91): Tracer::get_instance()
#8 /var/www/html/tt-rss/classes/pluginhost.php(448): Tracer::start()
#9 /var/www/html/tt-rss/include/functions.php(437): PluginHost->load()
#10 /var/www/html/tt-rss/update.php(78): init_plugins()
#11 {main}
  thrown in /var/www/html/tt-rss/vendor/open-telemetry/api/Trace/SpanContextValidator.php on line 25

Steps to reproduce the problem: start the container, wait a view seconds

Tiny Tiny RSS version (including git commit id):
→ not sure how to get the version number; web interface is no longer accessible and I have not seen the version number in the log files
Platform (i.e. Linux distro, Docker, PHP, PostgreSQL, etc) versions:
→ Docker via the provided docker-compose; host system is Debian 12

Is there something missing in my setup that causes SpanContext.php to crash? A missing config variable or something?

fox · October 22, 2023, 8:52am

going by the “updating tt-rss source” log entry, you’re seemingly not using prebuilt docker hub images, i suggest using those instead.

https://tt-rss.org/wiki/InstallationNotes here’s an uptodate installation guide.

r35p3ct · October 22, 2023, 9:38am

Your images won’t run on RaspberryPi!

TTRSS on raspberry rip now?

fox · October 22, 2023, 9:43am

please don’t hijack a thread with random questions, instead just read the faq entry in its entirety.

great first post, enjoy your probation.

Kolo · October 22, 2023, 11:42am

Yes, you were totally right. I think I was using an older version of the docket-compose template - probably coming from here?

Old config file

# [...]
  app:
    build:
      context:
        ./app
    restart: unless-stopped
# [...]

New version:

# [...]
  app:
    image: cthulhoo/ttrss-fpm-pgsql-static:latest
    build:
      dockerfile: .docker/app/Dockerfile
      context: https://git.tt-rss.org/fox/tt-rss.git
    restart: unless-stopped
# [...]

After changing all lines to the new version, it’s working again. Thank you

Because the new version now uses pre-built images: Do you recommend regularly check for new versions and update the image/rebuild the container (e.g. using Watchtower), or does the updater container take care of this?

My old setup was still using Postgres 12. In your link, you described how to restore a database using zcat /backups/ttrss-backup-yyyymmdd.sql.gz | psql -h db -U $TTRSS_DB_USER $TTRSS_DB_NAME.
Is this the same process for upgrading to Postgres 15 or should I do a clean reinstall of the hole setup instead?

fox · October 22, 2023, 12:15pm

it’s less about compose itself and more about what’s in the app/Dockerfile and associated scripts. that was a stupid idea on my part, not using prebuilt images in the beginning. i didn’t factor in that scripts and Dockerfile would change over time but people won’t update them and their setup would eventually break.

watchtower has problems with multi-container compose images, i had it break and do strange things with dependencies. just saying. that’s when i had it set to update, notifications are fine, i think.

you don’t need to build your own images, unless you’re on ARM. just docker compose pull off docker hub when you feel like it.

full dump and restore should work. alternatively you can keep using PG12. i haven’t updated my own instance yet

e:

that’s for updating feeds.

Kolo · October 22, 2023, 2:10pm

Understood, thank you very much