Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Single Dockerfile using multi-stages for production and dev image #751

Open
wants to merge 5 commits into
base: 3.x
Choose a base branch
from
Open

Single Dockerfile using multi-stages for production and dev image #751

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
8fe46ef
Define baseimage with arguments required in multiple stages
AlMaVizca Nov 7, 2023
ee9f49b
Avoid repeating common path
AlMaVizca Dec 14, 2023
a248e17
Refactor composer dependencies to create the production image
AlMaVizca Apr 26, 2024
682fdbb
Relocate php unit configuration
AlMaVizca Dec 14, 2023
8320482
Merge development image into a single Dockerfile
AlMaVizca Dec 14, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/deliver.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ jobs:
# farmOS Composer project 3.x branch is always used.
- name: Build and save farmOS dev Docker image
run: |
docker build --build-arg FARMOS_REPO=https://github.com/${FARMOS_REPO} --build-arg FARMOS_VERSION=${FARMOS_VERSION} -t farmos/farmos:3.x-dev docker/dev
docker build --build-arg FARMOS_REPO=https://github.com/${FARMOS_REPO} --build-arg FARMOS_VERSION=${FARMOS_VERSION} -t farmos/farmos:3.x-dev --target dev docker
docker save farmos/farmos:3.x-dev > /tmp/farmos-dev.tar
- name: Cache farmOS dev Docker image
uses: actions/cache@v3
Expand Down
132 changes: 102 additions & 30 deletions docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,14 @@
# Use the official Drupal 10 image to build GEOS PHP extension.
FROM drupal:10.1 as php-dependencies
# Use the official Drupal 10 as base image.
FROM drupal:10.1 as baseimage

# Define common paths.
ENV FARMOS_PATH=/var/farmOS
ENV DRUPAL_PATH=/opt/drupal

##
# Build PHP extensions, GEOS and bcmath.
FROM baseimage as php-dependencies

# Build and install the GEOS PHP extension.
# See https://git.osgeo.org/gitea/geos/php-geos
ARG PHP_GEOS_VERSION=e77d5a16abbf89a59d947d1fe49381a944762c9d
ADD https://github.com/libgeos/php-geos/archive/${PHP_GEOS_VERSION}.tar.gz /opt/php-geos.tar.gz
Expand All @@ -17,54 +24,119 @@ RUN apt-get update && apt-get install -y libgeos-dev \
# Install the BCMath PHP extension.
RUN docker-php-ext-install bcmath

# Inherit from the official Drupal 10 image.
FROM drupal:10.1
# Setup dependencies and sources for composer installations.
FROM baseimage as composer-file

# Set the farmOS and composer project repository URLs and versions.
ARG FARMOS_REPO=https://github.com/farmOS/farmOS.git
ARG FARMOS_VERSION=3.x
ARG PROJECT_REPO=https://github.com/farmOS/composer-project.git
ARG PROJECT_VERSION=3.x
ARG PROJECT_REPO=https://github.com/farmOS/composer-project/raw/${PROJECT_VERSION}/composer.json

# Set the COMPOSER_MEMORY_LIMIT environment variable to unlimited.
ENV COMPOSER_MEMORY_LIMIT=-1
# Allow root to install plugins.
ENV COMPOSER_ALLOW_SUPERUSER=1
ENV COMPOSER_NO_INTERACTION=1

# Install apt dependencies.
RUN apt-get update && apt-get install -y --no-install-recommends\
# Install git and unzip (needed by Composer).
git unzip

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

git and unzip are currently installed and available only in stages related to building farmOS. In my opinion that they should also be available at least in the final dev version of the image but probably also in the final prod image since they are required to install additional modules using composer which is a workflow available in current image versions, so removing it here would be a breaking change.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, when installing them in the stage that will be included in the final image, it would be good to add --no-install-recommends option to the apt-get install command to skip installing additional packages that are not really needed and only take up space in the image. It is a good practice to use this option in all instances of apt-get in docker images.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

git and unzip are currently installed and available only in stages related to building farmOS. In my opinion that they should also be available at least in the final dev version of the image but probably also in the final prod image since they are required to install additional modules using composer which is a workflow available in current image versions, so removing it here would be a breaking change.

Can you, please, share a module that requires some of those packages?
I've tried with a default installation, also adding modules that are displayed during setup, and later on /setup/modules and they were installed without errors.

I understand that those packages might be useful for development image. Yet, for production, I think that adding any module without creating a new image for distribution, or without custom shared volumes, might create a scenario where if the container is replaced, it will lose functionality.
Since there is no documentation about shared volumes to persist modules information, I think, it is not a case to resolve in this PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, when installing them in the stage that will be included in the final image, it would be good to add --no-install-recommends option to the apt-get install command to skip installing additional packages that are not really needed and only take up space in the image. It is a good practice to use this option in all instances of apt-get in docker images.

I've tried to minimize the number of changes that has to do with packages installations, and propose a different structural organization.
I took the time to evaluate which packages are installed by recommendations, and only git and unzip are adding extra packages. Would it be enough to add them only to the dev image? Is it worth it?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking of community modules, not included in the farmOS itself. For example https://github.com/symbioquine/farmOS_asset_link or https://www.drupal.org/project/farm_map_custom_layers.

On the Building farmOS with Composer documentation page, there is described approach for adding additional community modules to the composer.json and building docker image based on official production one with these customizations. Currently, in instructions for building a custom image when it comes to installing composer dependencies only composer install is provided, without git and unzip in the prod image these instructions will no longer work and will need to be updated to also include installation of git and unzip. So not including git and uzip in production image may break some existing setups when the new farmOS base image version will be pulled.

I would include git and unzip in both dev and prod image to don't break existing documented functionality.

Copy link
Contributor Author

@AlMaVizca AlMaVizca Dec 24, 2023 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, although, I think that it would be more appropriate to add the information on the documentation, so the image is kept as slim as possible, It's the right thing to keep it working as it was.
Thanks for the explanation.

# Add the build-farmOS.sh script.
COPY build-farmOS.sh /usr/local/bin/

# Setup composer file for non interactive installation.
WORKDIR ${FARMOS_PATH}
RUN build-farmOS.sh

##
# Create layer with farmOS sources.
FROM composer-file as farmos-sources

# Install sources.
RUN composer install --no-dev

# Set the version in farm.info.yml.
RUN sed -i "s|version: 3.x|version: ${FARMOS_VERSION}|g" ${FARMOS_PATH}/web/profiles/farm/farm.info.yml

##
# Create layer with farmOS dev sources.
FROM farmos-sources as farmos-dev-sources

# Install sources.
RUN composer update

##
# Dependencies layer.
FROM baseimage as farmos-baseimage

# Set Apache ServerName directive globally to suppress AH00558 message.
RUN echo "ServerName localhost" >> /etc/apache2/apache2.conf

# Install and enable geos.
# Enable PHP dependencies.
COPY --from=php-dependencies /usr/local/lib/php/extensions/ /usr/local/lib/php/extensions/
RUN docker-php-ext-enable geos bcmath

# Add custom PHP configurations.
COPY conf.d/ /usr/local/etc/php/conf.d

# Install apt dependencies.
RUN apt-get update && apt-get install -y \
# Install git and unzip (needed by Composer).
git unzip \
# Install apt dependencies and clean up.
RUN apt-get update && apt-get install -y --no-install-recommends\
# Install postgresql-client so Drush can connect to the database.
postgresql-client \
# Install libgeos-c1v5 so geos php extension can use libgeos_c.so.1.
libgeos-c1v5 \
&& rm -rf /var/lib/apt/lists/* \
&& apt-get clean
# Install git and unzip (needed by Composer).
git unzip \
&& rm -rf /var/lib/apt/lists/* \
&& apt-get clean \
# Clean up ${DRUPAL_PATH}.
&& rm -r ${DRUPAL_PATH} \
&& mkdir ${DRUPAL_PATH}

# Set the COMPOSER_MEMORY_LIMIT environment variable to unlimited.
ENV COMPOSER_MEMORY_LIMIT=-1
# Set the entrypoint.
COPY --chown=www-data docker-entrypoint.sh /usr/local/bin/

# Set COMPOSER_ALLOW_SUPERUSER=1 to allow plugins to run as root/super user.
ENV COMPOSER_ALLOW_SUPERUSER=1
# Set the working directory, entrypoint and command.
WORKDIR ${DRUPAL_PATH}
ENTRYPOINT ["docker-entrypoint.sh"]
CMD ["apache2-foreground"]

# Add the build-farmOS.sh script.
COPY build-farmOS.sh /usr/local/bin/
RUN chmod a+x /usr/local/bin/build-farmOS.sh
##
# Development image.
FROM farmos-baseimage as dev

# Build the farmOS codebase in /var/farmoS with the --no-dev flag.
# Change the ownership of the sites directory and copy the farmOS codebase into /opt/drupal.
RUN mkdir /var/farmOS \
&& /usr/local/bin/build-farmOS.sh --no-dev \
&& chown -R www-data:www-data /var/farmOS/web/sites \
&& rm -r /opt/drupal && cp -rp /var/farmOS /opt/drupal
# Change the user/group IDs of www-data inside the image to match the ID of the
# developer's user on the host machine. This allows Composer to create files
# owned by www-data inside the container, while keeping those files editable by
# the developer outside of the container.
# This defaults to 1000, based on the assumption that the developer is running
# as UID 1000 on the host machine. It can be overridden at image build time with:
# --build-arg WWW_DATA_ID=$(id -u)
ARG WWW_DATA_ID=1000
RUN usermod -u ${WWW_DATA_ID} www-data && groupmod -g ${WWW_DATA_ID} www-data

# Set the entrypoint.
COPY docker-entrypoint.sh /usr/local/bin/
ENTRYPOINT ["docker-entrypoint.sh"]
CMD ["apache2-foreground"]
# Install and configure XDebug.
RUN yes | pecl install xdebug \
&& docker-php-ext-enable xdebug

# Add opcache revalidation frequency configuration.
COPY dev/conf.d/ /usr/local/etc/php/conf.d

# Add Configurations for PHP CodeSniffer, PHPStan
COPY --chown=www-data:www-data ./dev/files/ ${FARMOS_PATH}

# Add farmOS dev sources.
COPY --from=farmos-dev-sources --chown=www-data:www-data ${FARMOS_PATH} ${FARMOS_PATH}

# Configure PHPUnit.
RUN ${FARMOS_PATH}/phpunit.sh

##
# Final image.
FROM farmos-baseimage

# Add farmOS sources.
COPY --from=farmos-sources --chown=www-data:www-data ${FARMOS_PATH} ${DRUPAL_PATH}
35 changes: 7 additions & 28 deletions docker/build-farmOS.sh
100644 → 100755
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,26 +2,18 @@
set -e

###
# This script will build the farmOS codebase in /var/farmOS.
# This script will build the farmOS codebase in ${FARMOS_PATH},
# by default it is /var/farmOS.
###

# If /var/farmOS is not empty, bail.
if [ "$(ls -A /var/farmOS/)" ]; then
# If ${FARMOS_PATH} is not empty, bail.
if [ "$(ls -A ${FARMOS_PATH})" ]; then
echo "The ${FARMOS_PATH} is not empty, terminate."
exit 1
fi

# Make /var/farmOS the working directory.
cd /var/farmOS

# Generate an empty Composer project project and checkout a specific version.
git clone ${PROJECT_REPO} project
mv project/.git ./.git
rm -rf project
git checkout ${PROJECT_VERSION}
git reset --hard

# Create a temporary Composer cache directory.
export COMPOSER_HOME="$(mktemp -d)"
# Fetch composer template
curl -L ${PROJECT_REPO} -o composer.json

# If FARMOS_VERSION is a valid semantic versioning string, we assume that it is
# a tagged version.
Expand Down Expand Up @@ -61,16 +53,3 @@ allowedPlugins=(
for plugin in ${allowedPlugins[@]}; do
composer config --no-plugins allow-plugins.$plugin true
done

# Run composer install with optional arguments passed into this script.
if [ $# -eq 0 ]; then
composer install
else
composer install "$*"
fi

# Set the version in farm.info.yml.
sed -i "s|version: 3.x|version: ${FARMOS_VERSION}|g" /var/farmOS/web/profiles/farm/farm.info.yml

# Remove the Composer cache directory.
rm -rf "$COMPOSER_HOME"
63 changes: 0 additions & 63 deletions docker/dev/Dockerfile
View file
Open in desktop

This file was deleted.

19 changes: 19 additions & 0 deletions docker/dev/files/phpunit.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
#!/usr/bin/env sh

cp -p ${FARMOS_PATH}/web/core/phpunit.xml.dist ${FARMOS_PATH}/phpunit.xml
sed -i 's|bootstrap="tests/bootstrap.php"|bootstrap="web/core/tests/bootstrap.php"|g' ${FARMOS_PATH}/phpunit.xml
sed -i '/failOnWarning="true"/a \ failOnIncomplete="true"' ${FARMOS_PATH}/phpunit.xml
sed -i '/failOnWarning="true"/a \ failOnSkipped="true"' ${FARMOS_PATH}/phpunit.xml
sed -i 's|name="SIMPLETEST_BASE_URL" value=""|name="SIMPLETEST_BASE_URL" value="http://www"|g' ${FARMOS_PATH}/phpunit.xml
sed -i 's|name="SIMPLETEST_DB" value=""|name="SIMPLETEST_DB" value="pgsql://farm:farm@db/farm"|g' ${FARMOS_PATH}/phpunit.xml
sed -i 's|name="BROWSERTEST_OUTPUT_DIRECTORY" value=""|name="BROWSERTEST_OUTPUT_DIRECTORY" value="/var/www/html/sites/simpletest/browser_output"|g' ${FARMOS_PATH}/phpunit.xml
sed -i 's|name="MINK_DRIVER_ARGS_WEBDRIVER" value='\'''\''|name="MINK_DRIVER_ARGS_WEBDRIVER" value='\''["chrome", { "chromeOptions": { "w3c": false, "args": ["--disable-gpu","--headless", "--no-sandbox"] } }, "http://chrome:4444/wd/hub"]'\''|g' ${FARMOS_PATH}/phpunit.xml
sed -i 's|\./|\./web/core/|g' ${FARMOS_PATH}/phpunit.xml
sed -i 's|\.\./web/core/|\./web/|g' ${FARMOS_PATH}/phpunit.xml
sed -i 's| </php>| <env name="SYMFONY_DEPRECATIONS_HELPER" value="disabled"/>'"\n"' </php>|g' ${FARMOS_PATH}/phpunit.xml

# Create output directory for phpunit tests and permissions for testing user.
mkdir -p ${FARMOS_PATH}/web/sites/simpletest/browser_output
chown -R www-data:www-data ${FARMOS_PATH}/web/sites/simpletest

rm ${FARMOS_PATH}/phpunit.sh
8 changes: 4 additions & 4 deletions docker/docker-entrypoint.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,15 @@ set -e
###

# If the Drupal directory is empty, populate it from pre-built files.
if [ -d /opt/drupal ] && ! [ "$(ls -A /opt/drupal/)" ]; then
if [ -d ${DRUPAL_PATH} ] && ! [ "$(ls -A ${DRUPAL_PATH}/)" ]; then
echo "farmOS codebase not detected. Copying from pre-built files in the Docker image."
cp -rp /var/farmOS/. /opt/drupal
cp -rp ${FARMOS_PATH}/. ${DRUPAL_PATH}
fi

# If the sites directory is empty, populate it from pre-built files.
if [ -d /opt/drupal/web/sites ] && ! [ "$(ls -A /opt/drupal/web/sites/)" ]; then
if [ -d ${DRUPAL_PATH}/web/sites ] && ! [ "$(ls -A ${DRUPAL_PATH}/web/sites/)" ]; then
echo "farmOS sites directory not detected. Copying from pre-built files in the Docker image."
cp -rp /var/farmOS/web/sites/. /opt/drupal/web/sites
cp -rp ${FARMOS_PATH}/web/sites/. ${DRUPAL_PATH}/web/sites
fi

if [ -n "$FARMOS_FS_READY_SENTINEL_FILENAME" ]; then
Expand Down
9 changes: 8 additions & 1 deletion docs/development/environment/docker.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,9 @@ Available arguments and their default values are described below:
check out.
- Default: `3.x`

The `3.x-dev` image also provides the following:
## Development image

The `3.x-dev` image also provides the following build arguments:

- `WWW_DATA_ID` - The ID to use for the `www-data` user and group inside the
image. Setting this to the ID of the developer's user on the host machine
Expand All @@ -26,3 +28,8 @@ The `3.x-dev` image also provides the following:
container. If your user ID is not `1000`, build the image with:
`--build-arg WWW_DATA_ID=$(id -u)`
- Default: `1000`

To build the development image, you will have to define the target dev,
for example:

`docker build --build-arg WWW_DATA_ID=$(id -u) -t farmos/farmos:3.x-dev --target dev docker`