ckan-docker & harvest vs xloader approach to workers #8066
Replies: 2 comments
-
FYI @kowh-ai |
Beta Was this translation helpful? Give feedback.
-
Yes this is all very interesting, I think there are advantages and disadvantages for both running everything (UI + extensions) in the one container or running UI and extensions (that run independently to the UI web server) in separate containers. Probably the biggest problem with multiple containers is “more containers = more maintenance” eg: when the CKAN code base changes. You are right, it would be cleaner / simpler to have separate containers for each extension. Perhaps a “worker container” is a better solution to both these alternatives See: ckan/ckan-docker#118 There is a decent chunk of work scheduled for the next 6 months to replace DataPusher with XLoader and then further down the track replacing XLoader with DataPusher+ |
Beta Was this translation helpful? Give feedback.
-
I am curious about best practice for installing ckanext-harvest within ckan-docker. I ask because I see a divergence between how harvest is mentioned in the ckan-docker Dockerfile and the XLoader Wiki. Both of these extensions require running separate workers.
The source install of XLoader uses supervisor, but the recommendation is a separate container for docker. This approach makes sense to me as the logs would be more accessible and it is easier to manage with only a single item running in a container. A wrinkle for harvester is that it may require a harvester provided by another extension. This could be accommodated by running other containers utilizing the the same image (of ckan, harvester and all the desired extensions), just with different commands to start the workers instead. So I am wondering what is the best approach?
Beta Was this translation helpful? Give feedback.
All reactions