docker compose up starts unhealthy datapusher

[jjohnston-aims]

CKAN version: 2.10.4

Docker version 26.0.0, build 2ae903e

Describe the bug

docker compose up starts unhealthy containers.

Steps to reproduce

• git clone git@github.com:ckan/ckan-docker.git
• cd ckan-docker
• mv .env.example .env
• docker compose up

Expected behavior

I would expect all containers to start healthy with the default configuration.

Additional details

❯ docker compose up
WARN[0000] /home/jaredjohnston/Workspace/ckan-docker/docker-compose.yml: `version` is obsolete
[+] Running 11/11
 ✔ Network ckan-docker_dbnet     Created                                                                                                                                          0.1s
 ✔ Network ckan-docker_solrnet   Created                                                                                                                                          0.1s
 ✔ Network ckan-docker_redisnet  Created                                                                                                                                          0.1s
 ✔ Network ckan-docker_ckannet   Created                                                                                                                                          0.2s
 ✔ Network ckan-docker_webnet    Created                                                                                                                                          0.1s
 ✔ Container db                  Created                                                                                                                                          0.2s
 ✔ Container solr                Created                                                                                                                                          0.2s
 ✔ Container redis               Created                                                                                                                                          0.2s
 ✔ Container datapusher          Created                                                                                                                                          0.2s
 ✔ Container ckan                Created                                                                                                                                          0.1s
 ✔ Container nginx               Created                                                                                                                                          0.1s
Attaching to ckan, datapusher, db, nginx, redis, solr
db          |
db          | PostgreSQL Database directory appears to contain a database; Skipping initialization
db          |
db          | 2024-04-09 03:18:53.422 UTC [1] LOG:  starting PostgreSQL 12.18 on x86_64-pc-linux-musl, compiled by gcc (Alpine 13.2.1_git20231014) 13.2.1 20231014, 64-bit
db          | 2024-04-09 03:18:53.422 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
db          | 2024-04-09 03:18:53.422 UTC [1] LOG:  listening on IPv6 address "::", port 5432
db          | 2024-04-09 03:18:53.432 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db          | 2024-04-09 03:18:53.481 UTC [22] LOG:  database system was shut down at 2024-04-09 02:54:30 UTC
db          | 2024-04-09 03:18:53.490 UTC [1] LOG:  database system is ready to accept connections
solr        | Executing /opt/solr/docker/scripts/solr-precreate ckan /opt/solr/server/solr/configsets/ckan
solr        | Executing /opt/solr/docker/scripts/precreate-core ckan /opt/solr/server/solr/configsets/ckan
solr        | Core ckan already exists
solr        | Starting Solr
redis       | 1:C 09 Apr 2024 03:18:53.504 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis       | 1:C 09 Apr 2024 03:18:53.504 # Redis version=6.2.14, bits=64, commit=00000000, modified=0, pid=1, just started
redis       | 1:C 09 Apr 2024 03:18:53.504 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis       | 1:M 09 Apr 2024 03:18:53.504 * monotonic clock: POSIX clock_gettime
redis       | 1:M 09 Apr 2024 03:18:53.505 * Running mode=standalone, port=6379.
redis       | 1:M 09 Apr 2024 03:18:53.505 # Server initialized
redis       | 1:M 09 Apr 2024 03:18:53.505 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis       | 1:M 09 Apr 2024 03:18:53.505 * Ready to accept connections
solr        | Java 17 detected. Enabled workaround for SOLR-16463
solr        | [0.001s][warning][pagesize] UseLargePages disabled, no large pages configured and available on the system.
datapusher  | [uWSGI] getting INI configuration from /srv/app/datapusher-uwsgi.ini
datapusher  | *** Starting uWSGI 2.0.19.1 (64bit) on [Tue Apr  9 03:18:53 2024] ***
datapusher  | compiled with version: 10.3.1 20210424 on 05 April 2021 18:18:03
datapusher  | os: Linux-6.5.0-26-generic #26~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue Mar 12 10:22:43 UTC 2
datapusher  | nodename: eba76fc826d0
datapusher  | machine: x86_64
datapusher  | clock source: unix
datapusher  | pcre jit disabled
datapusher  | detected number of CPU cores: 12
datapusher  | current working directory: /srv/app
datapusher  | writing pidfile to /tmp/datapusher-uwsgi.pid
datapusher  | detected binary path: /usr/sbin/uwsgi
datapusher  | your memory page size is 4096 bytes
datapusher  |  *** WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers ***
datapusher  | detected max file descriptor number: 1048576
datapusher  | lock engine: pthread robust mutexes
datapusher  | thunder lock: disabled (you can enable it with --thunder-lock)
datapusher  | uWSGI http bound on 0.0.0.0:8800 fd 3
datapusher  | uwsgi socket 0 bound to UNIX address /tmp/uwsgi.sock fd 6
datapusher  | Python version: 3.9.16 (main, Dec 10 2022, 13:47:19)  [GCC 10.3.1 20210424]
datapusher  | Python main interpreter initialized at 0x738ce9a22c00
datapusher  | python threads support enabled
datapusher  | your server socket listen backlog is limited to 100 connections
datapusher  | your mercy for graceful operations on workers is 60 seconds
datapusher  | mapped 203184 bytes (198 KB) for 1 cores
datapusher  | *** Operational MODE: single process ***
solr        | CompileCommand: exclude com/github/benmanes/caffeine/cache/BoundedLocalCache.put bool exclude = true
solr        | WARNING: A command line option has enabled the Security Manager
solr        | WARNING: The Security Manager is deprecated and will be removed in a future release
datapusher  | /usr/lib/python3.9/site-packages/tzlocal/unix.py:183: UserWarning: Can not find any timezone configuration, defaulting to UTC.
datapusher  |   warnings.warn("Can not find any timezone configuration, defaulting to UTC.")
datapusher  | WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0x738ce9a22c00 pid: 1 (default app)
datapusher  | *** uWSGI is running in multiple interpreter mode ***
datapusher  | spawned uWSGI master process (pid: 1)
datapusher  | spawned uWSGI worker 1 (pid: 9, cores: 1)
datapusher  | spawned uWSGI http 1 (pid: 10)
solr        | 2024-04-09 03:18:54.482 INFO  (main) [] o.e.j.s.Server jetty-10.0.13; built: 2022-12-07T20:13:20.134Z; git: 1c2636ea05c0ca8de1ffd6ca7f3a98ac084c766d; jvm 17.0.7+7
solr        | 2024-04-09 03:18:54.703 WARN  (main) [] o.e.j.u.DeprecationWarning Using @Deprecated Class org.eclipse.jetty.servlet.listener.ELContextCleaner
solr        | 2024-04-09 03:18:54.724 INFO  (main) [] o.a.s.s.CoreContainerProvider Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
solr        | 2024-04-09 03:18:54.728 INFO  (main) [] o.a.s.s.CoreContainerProvider  ___      _       Welcome to Apache Solr™ version 9.2.1
solr        | 2024-04-09 03:18:54.728 INFO  (main) [] o.a.s.s.CoreContainerProvider / __| ___| |_ _   Starting in standalone mode on port 8983
solr        | 2024-04-09 03:18:54.728 INFO  (main) [] o.a.s.s.CoreContainerProvider \__ \/ _ \ | '_|  Install dir: /opt/solr-9.2.1
solr        | 2024-04-09 03:18:54.728 INFO  (main) [] o.a.s.s.CoreContainerProvider |___/\___/_|_|    Start time: 2024-04-09T03:18:54.728667921Z
solr        | 2024-04-09 03:18:54.731 INFO  (main) [] o.a.s.s.CoreContainerProvider Solr started with "-XX:+CrashOnOutOfMemoryError" that will crash on any OutOfMemoryError exception. The cause of the OOME will be logged in the crash file at the following path: /var/solr/logs/jvm_crash_8.log
solr        | 2024-04-09 03:18:54.733 INFO  (main) [] o.a.s.s.CoreContainerProvider Solr Home: /var/solr/data (source: system property: solr.solr.home)
solr        | 2024-04-09 03:18:54.735 INFO  (main) [] o.a.s.c.SolrXmlConfig solr.xml not found in SOLR_HOME, using built-in default
solr        | 2024-04-09 03:18:54.735 INFO  (main) [] o.a.s.c.SolrXmlConfig Loading solr.xml from /opt/solr-9.2.1/server/solr/solr.xml
solr        | 2024-04-09 03:18:54.796 INFO  (main) [] o.a.s.c.SolrResourceLoader Added 1 libs to classloader, from paths: [/opt/solr-9.2.1/lib]
solr        | 2024-04-09 03:18:55.521 WARN  (main) [] o.a.s.u.StartupLoggingUtils Jetty request logging enabled. Will retain logs for last 3 days. See chapter "Configuring Logging" in reference guide for how to configure.
solr        | 2024-04-09 03:18:55.522 WARN  (main) [] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
solr        | 2024-04-09 03:18:55.648 INFO  (main) [] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /var/solr/data
solr        | 2024-04-09 03:18:55.648 INFO  (main) [] o.a.s.c.CorePropertiesLocator Cores are: [ckan]
solr        | 2024-04-09 03:18:55.670 INFO  (coreLoadExecutor-10-thread-1) [ x:ckan] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.0
solr        | 2024-04-09 03:18:55.724 INFO  (coreLoadExecutor-10-thread-1) [ x:ckan] o.a.s.s.IndexSchema Schema name=ckan-2.10
solr        | 2024-04-09 03:18:55.793 INFO  (coreLoadExecutor-10-thread-1) [ x:ckan] o.a.s.s.IndexSchema Loaded schema ckan-2.10/1.6 with uniqueid field index_id
solr        | 2024-04-09 03:18:55.798 INFO  (coreLoadExecutor-10-thread-1) [ x:ckan] o.a.s.c.CoreContainer Creating SolrCore 'ckan' using configuration from instancedir /var/solr/data/ckan, trusted=true
solr        | 2024-04-09 03:18:55.827 INFO  (coreLoadExecutor-10-thread-1) [ x:ckan] o.a.s.c.SolrCore Opening new SolrCore at [/var/solr/data/ckan], dataDir=[/var/solr/data/ckan/data/]
solr        | 2024-04-09 03:18:55.963 INFO  (main) [] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer
solr        | 2024-04-09 03:18:56.048 INFO  (main) [] o.e.j.s.h.ContextHandler Started o.e.j.w.WebAppContext@3f093abe{solr-jetty-context.xml,/solr,file:///opt/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE}{/opt/solr-9.2.1/server/solr-webapp/webapp}
solr        | 2024-04-09 03:18:56.056 INFO  (main) [] o.e.j.s.RequestLogWriter Opened /var/solr/logs/2024_04_09.request.log
solr        | 2024-04-09 03:18:56.059 INFO  (main) [] o.e.j.s.AbstractConnector Started ServerConnector@45c8d09f{HTTP/1.1, (http/1.1, h2c)}{0.0.0.0:8983}
solr        | 2024-04-09 03:18:56.060 INFO  (main) [] o.e.j.s.Server Started Server@38be305c{STARTING}[10.0.13,sto=0] @2510ms
solr        | 2024-04-09 03:18:56.238 INFO  (coreLoadExecutor-10-thread-1) [ x:ckan] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer
solr        | 2024-04-09 03:18:56.248 INFO  (coreLoadExecutor-10-thread-1) [ x:ckan] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
solr        | 2024-04-09 03:18:56.248 INFO  (coreLoadExecutor-10-thread-1) [ x:ckan] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
solr        | 2024-04-09 03:18:56.256 INFO  (coreLoadExecutor-10-thread-1) [ x:ckan] o.a.s.u.CommitTracker Hard AutoCommit: if uncommitted for 15000ms;
solr        | 2024-04-09 03:18:56.257 INFO  (coreLoadExecutor-10-thread-1) [ x:ckan] o.a.s.u.CommitTracker Soft AutoCommit: disabled
solr        | 2024-04-09 03:18:56.318 INFO  (coreLoadExecutor-10-thread-1) [ x:ckan] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /var/solr/data/ckan/conf
solr        | 2024-04-09 03:18:56.330 INFO  (coreLoadExecutor-10-thread-1) [ x:ckan] o.a.s.s.DirectSolrSpellChecker init: {name=default, field=_text_, classname=solr.DirectSolrSpellChecker, distanceMeasure=internal, maxEdits=2, minPrefix=1, maxInspections=5, minQueryLength=4, accuracy=0.5, maxQueryFrequency=0.01}
solr        | 2024-04-09 03:18:56.337 INFO  (coreLoadExecutor-10-thread-1) [ x:ckan] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
solr        | 2024-04-09 03:18:56.340 INFO  (coreLoadExecutor-10-thread-1) [ x:ckan] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1795825584140451840
solr        | 2024-04-09 03:18:56.343 INFO  (searcherExecutor-12-thread-1-processing-ckan) [ x:ckan] o.a.s.c.QuerySenderListener QuerySenderListener done.
solr        | 2024-04-09 03:18:56.343 INFO  (searcherExecutor-12-thread-1-processing-ckan) [ x:ckan] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
solr        | 2024-04-09 03:18:56.349 INFO  (searcherExecutor-12-thread-1-processing-ckan) [ x:ckan] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms
ckan        | Setting a temporary value for ckan.datapusher.api_token
ckan        | Setting beaker.session.secret in ini file
solr        | 2024-04-09 03:19:30.842 INFO  (qtp1663888181-20) [ x:ckan] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&rows=1&wt=json} hits=0 status=0 QTime=26
solr        | 2024-04-09 03:19:30.857 INFO  (qtp1663888181-21) [ x:ckan] o.a.s.c.S.Request webapp=/solr path=/schema params={wt=schema.xml} status=0 QTime=4
solr        | 2024-04-09 03:19:31.394 INFO  (qtp1663888181-22) [ x:ckan] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&rows=1&wt=json} hits=0 status=0 QTime=2
solr        | 2024-04-09 03:19:31.399 INFO  (qtp1663888181-23) [ x:ckan] o.a.s.c.S.Request webapp=/solr path=/schema params={wt=schema.xml} status=0 QTime=0
db          | 2024-04-09 03:19:32.865 UTC [43] ERROR:  relation "migrate_version" does not exist at character 21
db          | 2024-04-09 03:19:32.865 UTC [43] STATEMENT:  select version from migrate_version
ckan        | 2024-04-09 03:19:34,773 INFO  [ckan.cli] Using configuration file /srv/app/ckan.ini
ckan        | 2024-04-09 03:19:34,773 INFO  [ckan.config.environment] Loading static files from public
solr        | 2024-04-09 03:19:35.048 INFO  (qtp1663888181-18) [ x:ckan] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&rows=1&wt=json} hits=0 status=0 QTime=0
solr        | 2024-04-09 03:19:35.054 INFO  (qtp1663888181-19) [ x:ckan] o.a.s.c.S.Request webapp=/solr path=/schema params={wt=schema.xml} status=0 QTime=0
ckan        | 2024-04-09 03:19:35,062 INFO  [ckan.config.environment] Loading templates from /srv/app/src/ckan/ckan/templates
ckan        | 2024-04-09 03:19:35,430 WARNI [ckanext.reclineview.plugin] The Recline-based views are deprecated andwill be removed in future versions
solr        | 2024-04-09 03:19:35.449 INFO  (qtp1663888181-25) [ x:ckan] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&rows=1&wt=json} hits=0 status=0 QTime=0
solr        | 2024-04-09 03:19:35.454 INFO  (qtp1663888181-27) [ x:ckan] o.a.s.c.S.Request webapp=/solr path=/schema params={wt=schema.xml} status=0 QTime=0
ckan        | 2024-04-09 03:19:35,459 INFO  [ckan.config.environment] Loading templates from /srv/app/src/ckan/ckan/templates
ckan        | 2024-04-09 03:19:35,744 WARNI [ckan.config.middleware.flask_app] Extensions are excluded from CSRF protection! We allow extensions to run without CSRF protection but it will be forced future releases. Read the documentation for more information on how to add CSRF protection to your extension.
db          | 2024-04-09 03:19:35.923 UTC [49] WARNING:  no privileges could be revoked for "public"
db          | 2024-04-09 03:19:35.923 UTC [49] WARNING:  no privileges could be revoked for "public"
db          | 2024-04-09 03:19:35.923 UTC [49] WARNING:  no privileges were granted for "public"
db          | 2024-04-09 03:19:35.923 UTC [49] WARNING:  no privileges were granted for "public"
db          | 2024-04-09 03:19:35.923 UTC [49] WARNING:  no privileges were granted for "public"
db          | 2024-04-09 03:19:35.923 UTC [49] WARNING:  no privileges were granted for "public"
db          | 2024-04-09 03:19:35.923 UTC [49] WARNING:  no privileges were granted for "public"
solr        | 2024-04-09 03:19:35.941 INFO  (qtp1663888181-20) [ x:ckan] o.a.s.c.S.Request webapp=/solr path=/schema/name params={wt=json} status=0 QTime=3
ckan        | 2024-04-09 03:19:36,684 INFO  [ckan.cli] Using configuration file /srv/app/ckan.ini
ckan        | 2024-04-09 03:19:36,684 INFO  [ckan.config.environment] Loading static files from public
solr        | 2024-04-09 03:19:36.942 INFO  (qtp1663888181-21) [ x:ckan] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&rows=1&wt=json} hits=0 status=0 QTime=0
solr        | 2024-04-09 03:19:36.946 INFO  (qtp1663888181-22) [ x:ckan] o.a.s.c.S.Request webapp=/solr path=/schema params={wt=schema.xml} status=0 QTime=0
ckan        | 2024-04-09 03:19:36,954 INFO  [ckan.config.environment] Loading templates from /srv/app/src/ckan/ckan/templates
ckan        | 2024-04-09 03:19:37,320 WARNI [ckanext.reclineview.plugin] The Recline-based views are deprecated andwill be removed in future versions
solr        | 2024-04-09 03:19:37.340 INFO  (qtp1663888181-23) [ x:ckan] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&rows=1&wt=json} hits=0 status=0 QTime=0
solr        | 2024-04-09 03:19:37.343 INFO  (qtp1663888181-25) [ x:ckan] o.a.s.c.S.Request webapp=/solr path=/schema params={wt=schema.xml} status=0 QTime=0
ckan        | 2024-04-09 03:19:37,350 INFO  [ckan.config.environment] Loading templates from /srv/app/src/ckan/ckan/templates
ckan        | 2024-04-09 03:19:37,611 WARNI [ckan.config.middleware.flask_app] Extensions are excluded from CSRF protection! We allow extensions to run without CSRF protection but it will be forced future releases. Read the documentation for more information on how to add CSRF protection to your extension.
db          | 2024-04-09 03:19:37.779 UTC [61] LOG:  unexpected EOF on client connection with an open transaction
ckan        | [prerun] Initializing or upgrading db - start
ckan        | [prerun] Initializing or upgrading db - end
ckan        | [prerun] Setting the following plugins in /srv/app/ckan.ini:
ckan        | envvars image_view text_view recline_view datastore datapusher
ckan        | [prerun] Plugins set.
ckan        | [prerun] Initializing datastore db - start
ckan        | WARNING:  no privileges could be revoked for "public"
ckan        |
ckan        | WARNING:  no privileges could be revoked for "public"
ckan        |
ckan        | WARNING:  no privileges were granted for "public"
ckan        |
ckan        | WARNING:  no privileges were granted for "public"
ckan        |
ckan        | WARNING:  no privileges were granted for "public"
ckan        |
ckan        | WARNING:  no privileges were granted for "public"
ckan        |
ckan        | WARNING:  no privileges were granted for "public"
ckan        |
ckan        | [prerun] Initializing datastore db - end
ckan        | b''
ckan        | [prerun] Succesfully connected to solr and CKAN schema loaded
ckan        | [prerun] Sysadmin user exists, skipping creation
ckan        | /srv/app/start_ckan.sh: Running init file /docker-entrypoint.d/01_setup_datapusher.sh
ckan        | Set up ckan.datapusher.api_token in the CKAN config file
ckan        | 2024-04-09 03:19:38,520 INFO  [ckan.cli] Using configuration file /srv/app/ckan.ini
ckan        | 2024-04-09 03:19:38,520 INFO  [ckan.config.environment] Loading static files from public
solr        | 2024-04-09 03:19:38.793 INFO  (qtp1663888181-27) [ x:ckan] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&rows=1&wt=json} hits=0 status=0 QTime=0
solr        | 2024-04-09 03:19:38.797 INFO  (qtp1663888181-26) [ x:ckan] o.a.s.c.S.Request webapp=/solr path=/schema params={wt=schema.xml} status=0 QTime=0
ckan        | 2024-04-09 03:19:38,802 INFO  [ckan.config.environment] Loading templates from /srv/app/src/ckan/ckan/templates
ckan        | 2024-04-09 03:19:39,185 WARNI [ckanext.reclineview.plugin] The Recline-based views are deprecated andwill be removed in future versions
solr        | 2024-04-09 03:19:39.205 INFO  (qtp1663888181-21) [ x:ckan] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&rows=1&wt=json} hits=0 status=0 QTime=0
solr        | 2024-04-09 03:19:39.208 INFO  (qtp1663888181-22) [ x:ckan] o.a.s.c.S.Request webapp=/solr path=/schema params={wt=schema.xml} status=0 QTime=0
ckan        | 2024-04-09 03:19:39,213 INFO  [ckan.config.environment] Loading templates from /srv/app/src/ckan/ckan/templates
ckan        | 2024-04-09 03:19:39,429 WARNI [ckan.config.middleware.flask_app] Extensions are excluded from CSRF protection! We allow extensions to run without CSRF protection but it will be forced future releases. Read the documentation for more information on how to add CSRF protection to your extension.
ckan        | /srv/app/start_ckan.sh: Ignoring /docker-entrypoint.d/README.md (not an sh or py file)
ckan        | *** Starting uWSGI 2.0.21 (64bit) on [Tue Apr  9 03:19:40 2024] ***
ckan        | compiled with version: 12.2.1 20220924 on 09 November 2022 12:47:28
ckan        | os: Linux-6.5.0-26-generic #26~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue Mar 12 10:22:43 UTC 2
ckan        | nodename: 3f035d0c94df
ckan        | machine: x86_64
ckan        | clock source: unix
ckan        | pcre jit disabled
ckan        | detected number of CPU cores: 12
ckan        | current working directory: /srv/app
ckan        | detected binary path: /usr/sbin/uwsgi
ckan        | uWSGI running as root, you can use --uid/--gid/--chroot options
ckan        | setgid() to 92
ckan        | setuid() to 92
ckan        | your memory page size is 4096 bytes
ckan        |  *** WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers ***
ckan        | detected max file descriptor number: 1048576
ckan        | lock engine: pthread robust mutexes
ckan        | thunder lock: disabled (you can enable it with --thunder-lock)
ckan        | uWSGI http bound on 0.0.0.0:5000 fd 3
ckan        | uwsgi socket 0 bound to UNIX address /tmp/uwsgi.sock fd 6
ckan        | Python version: 3.10.13 (main, Aug 26 2023, 11:33:35) [GCC 12.2.1 20220924]
ckan        | Python main interpreter initialized at 0x7feab7b78b60
ckan        | python threads support enabled
ckan        | your server socket listen backlog is limited to 100 connections
ckan        | your mercy for graceful operations on workers is 60 seconds
ckan        | mapped 304776 bytes (297 KB) for 2 cores
ckan        | *** Operational MODE: preforking ***
ckan        | *** uWSGI is running in multiple interpreter mode ***
ckan        | spawned uWSGI master process (pid: 27)
ckan        | spawned uWSGI worker 1 (pid: 28, cores: 1)
ckan        | spawned uWSGI worker 2 (pid: 29, cores: 1)
ckan        | spawned uWSGI http 1 (pid: 30)
ckan        | 2024-04-09 03:19:40,711 CRIT Supervisor is running as root.  Privileges were not dropped because no user is specified in the config file.  If you intend to run as root, you can set user=root in the config file to avoid this message.
ckan        | 2024-04-09 03:19:40,711 WARN No file matches via include "/etc/supervisord.d/*.conf"
ckan        | 2024-04-09 03:19:40,714 INFO RPC interface 'supervisor' initialized
ckan        | 2024-04-09 03:19:40,714 CRIT Server 'unix_http_server' running without any HTTP authentication checking
ckan        | 2024-04-09 03:19:40,715 INFO supervisord started with pid 26
ckan        | 2024-04-09 03:19:41,330 INFO  [ckan.config.environment] Loading static files from public
ckan        | 2024-04-09 03:19:41,340 INFO  [ckan.config.environment] Loading static files from public
solr        | 2024-04-09 03:19:41.614 INFO  (qtp1663888181-23) [ x:ckan] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&rows=1&wt=json} hits=0 status=0 QTime=0
solr        | 2024-04-09 03:19:41.618 INFO  (qtp1663888181-25) [ x:ckan] o.a.s.c.S.Request webapp=/solr path=/schema params={wt=schema.xml} status=0 QTime=0
ckan        | 2024-04-09 03:19:41,623 INFO  [ckan.config.environment] Loading templates from /srv/app/src/ckan/ckan/templates
solr        | 2024-04-09 03:19:41.625 INFO  (qtp1663888181-27) [ x:ckan] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&rows=1&wt=json} hits=0 status=0 QTime=0
solr        | 2024-04-09 03:19:41.629 INFO  (qtp1663888181-26) [ x:ckan] o.a.s.c.S.Request webapp=/solr path=/schema params={wt=schema.xml} status=0 QTime=0
ckan        | 2024-04-09 03:19:41,635 INFO  [ckan.config.environment] Loading templates from /srv/app/src/ckan/ckan/templates
ckan        | 2024-04-09 03:19:42,061 WARNI [ckanext.reclineview.plugin] The Recline-based views are deprecated andwill be removed in future versions
ckan        | 2024-04-09 03:19:42,077 WARNI [ckanext.reclineview.plugin] The Recline-based views are deprecated andwill be removed in future versions
solr        | 2024-04-09 03:19:42.130 INFO  (qtp1663888181-21) [ x:ckan] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&rows=1&wt=json} hits=0 status=0 QTime=0
solr        | 2024-04-09 03:19:42.134 INFO  (qtp1663888181-22) [ x:ckan] o.a.s.c.S.Request webapp=/solr path=/schema params={wt=schema.xml} status=0 QTime=0
ckan        | 2024-04-09 03:19:42,139 INFO  [ckan.config.environment] Loading templates from /srv/app/src/ckan/ckan/templates
solr        | 2024-04-09 03:19:42.158 INFO  (qtp1663888181-23) [ x:ckan] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&rows=1&wt=json} hits=0 status=0 QTime=0
solr        | 2024-04-09 03:19:42.162 INFO  (qtp1663888181-25) [ x:ckan] o.a.s.c.S.Request webapp=/solr path=/schema params={wt=schema.xml} status=0 QTime=0
ckan        | 2024-04-09 03:19:42,167 INFO  [ckan.config.environment] Loading templates from /srv/app/src/ckan/ckan/templates
ckan        | 2024-04-09 03:19:42,408 WARNI [ckan.config.middleware.flask_app] Extensions are excluded from CSRF protection! We allow extensions to run without CSRF protection but it will be forced future releases. Read the documentation for more information on how to add CSRF protection to your extension.
ckan        | 2024-04-09 03:19:42,408 WARNI [ckan.config.middleware.flask_app] Extensions are excluded from CSRF protection! We allow extensions to run without CSRF protection but it will be forced future releases. Read the documentation for more information on how to add CSRF protection to your extension.
ckan        | WSGI app 0 (mountpoint='') ready in 2 seconds on interpreter 0x7feab7b78b60 pid: 29 (default app)
ckan        | mountpoint  already configured. skip.
ckan        | WSGI app 0 (mountpoint='') ready in 2 seconds on interpreter 0x7feab7b78b60 pid: 28 (default app)
ckan        | mountpoint  already configured. skip.

I would expect this following to start healthily also:

❯ docker compose up datapusher
WARN[0000] /home/jaredjohnston/Workspace/ckan-docker/docker-compose.yml: `version` is obsolete
[+] Running 6/6
 ✔ Network ckan-docker_redisnet  Created                                                                                                                                          0.1s
 ✔ Network ckan-docker_ckannet   Created                                                                                                                                          0.2s
 ✔ Network ckan-docker_webnet    Created                                                                                                                                          0.1s
 ✔ Network ckan-docker_dbnet     Created                                                                                                                                          0.1s
 ✔ Network ckan-docker_solrnet   Created                                                                                                                                          0.1s
 ✔ Container datapusher          Created                                                                                                                                          0.1s
Attaching to datapusher
datapusher  | [uWSGI] getting INI configuration from /srv/app/datapusher-uwsgi.ini
datapusher  | *** Starting uWSGI 2.0.19.1 (64bit) on [Tue Apr  9 03:28:27 2024] ***
datapusher  | compiled with version: 10.3.1 20210424 on 05 April 2021 18:18:03
datapusher  | os: Linux-6.5.0-26-generic #26~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue Mar 12 10:22:43 UTC 2
datapusher  | nodename: 01f02483e10d
datapusher  | machine: x86_64
datapusher  | clock source: unix
datapusher  | pcre jit disabled
datapusher  | detected number of CPU cores: 12
datapusher  | current working directory: /srv/app
datapusher  | writing pidfile to /tmp/datapusher-uwsgi.pid
datapusher  | detected binary path: /usr/sbin/uwsgi
datapusher  | your memory page size is 4096 bytes
datapusher  |  *** WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers ***
datapusher  | detected max file descriptor number: 1048576
datapusher  | lock engine: pthread robust mutexes
datapusher  | thunder lock: disabled (you can enable it with --thunder-lock)
datapusher  | uWSGI http bound on 0.0.0.0:8800 fd 3
datapusher  | uwsgi socket 0 bound to UNIX address /tmp/uwsgi.sock fd 6
datapusher  | Python version: 3.9.16 (main, Dec 10 2022, 13:47:19)  [GCC 10.3.1 20210424]
datapusher  | Python main interpreter initialized at 0x7c25a465ac00
datapusher  | python threads support enabled
datapusher  | your server socket listen backlog is limited to 100 connections
datapusher  | your mercy for graceful operations on workers is 60 seconds
datapusher  | mapped 203184 bytes (198 KB) for 1 cores
datapusher  | *** Operational MODE: single process ***
datapusher  | /usr/lib/python3.9/site-packages/tzlocal/unix.py:183: UserWarning: Can not find any timezone configuration, defaulting to UTC.
datapusher  |   warnings.warn("Can not find any timezone configuration, defaulting to UTC.")
datapusher  | WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0x7c25a465ac00 pid: 1 (default app)
datapusher  | *** uWSGI is running in multiple interpreter mode ***
datapusher  | spawned uWSGI master process (pid: 1)
datapusher  | spawned uWSGI worker 1 (pid: 9, cores: 1)
datapusher  | spawned uWSGI http 1 (pid: 10)

❯ docker ps
CONTAINER ID   IMAGE                              COMMAND                  CREATED         STATUS                     PORTS      NAMES
01f02483e10d   ckan/ckan-base-datapusher:0.0.20   "sh -c 'uwsgi --plug…"   4 minutes ago   Up 4 minutes (unhealthy)   8800/tcp   datapusher

[kowh-ai]

Iv'e not been successful in recreating this issue...I get the timezone warning message also...

[jazzabeanie]

The same commands work on another computer with docker version 24.0.7, build afdd53b running inside wsl2. I will leave the issue open for now so I can add details of the solution once I figure it out.

[EricSoroos]

We've seen some stability issues based on the latest docker updates and docker-compose, especially with datapusher. In our case, it's looking a bit like a container startup ordering issue.

[blixt70]

Don´t know if it is relevant here but I also had problems with getting ckan and datapusher to report as "healthy" and it seems that inside the containers they don´t think they are "localhost".
373434b4128b:/srv/app# wget http://localhost:5000/
Connecting to localhost:5000 ([::1]:5000)
wget: can't connect to remote host: Connection refused
Changing to to 0.0.0.0:5000 works:
373434b4128b:/srv/app# wget http://0.0.0.0:5000/
Connecting to 0.0.0.0:5000 (0.0.0.0:5000)
saving to 'index.html'
index.html 100%

Changing healthcheck in docker-compose.yml made datapusher and ckan report as healthy for me:
for datapusher:
healthcheck:
test: ["CMD", "wget", "-qO", "/dev/null", http://0.0.0.0:8800/]
and for ckan:
healthcheck:
test: ["CMD", "wget", "-qO", "/dev/null", http://0.0.0.0:5000/]

[kowh-ai]

OK, I'll investigate on a couple of different host machines when I get back to early Berlin next week