Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use ubuntu/debian instead of alpine in Docker image #3504

Open
2 of 5 tasks
curquiza opened this issue Feb 16, 2023 · 11 comments · May be fixed by #3516
Open
2 of 5 tasks

Use ubuntu/debian instead of alpine in Docker image #3504

curquiza opened this issue Feb 16, 2023 · 11 comments · May be fixed by #3516
Assignees
Labels
enhancement New feature or improvement impacts cloud This issue involves changes for the Meilisearch's cloud team tooling Not directly project related, like Docker, Homebrew...

Comments

@curquiza
Copy link
Member

curquiza commented Feb 16, 2023

Discussed with @dureuill

Currently, in our Dockerfile, we use the alpine image as a base.

We spotted multi potential issues we could have in the future, and here is why we would rather use the ubuntu/debian image:

  • Meilisearch is a program doing a lot of system calls and then highly depends on the platform where it runs: we don't run any integration tests with Alpine, and even if it's a Linux distribution, we definitely better know the Ubuntu and Debian usages than Alpine
  • a lot of users use our docker image, especially @meilisearch/cloud-team. Perfectly knowing the edge cases of the docker use case is more and more mandatory. Engine team would be more confident with a debian or ubuntu image, since it's what we test every day.
  • Alpine uses the musl libc implementation, and of course, Meilisearch uses libc a lot -> this musl version of the libc could bring latency (not verified yet) compared to glibc but also edge cases we cannot identify now.

For all these reasons, we would rather use an ubuntu or debian image to build our own Meilisearch image at top of it

TODO

  • Test with Debian or Ubuntu instead of Alpine
    • Is the size of the image still correct? Yes, 256Mb (alpine) vs 354Mb (debian). Plus, some improvement regarding the binary size will be done for v1.1
    • Give a prototype to the cloud team, and wait for their approval -> is it working as previously? They told us everything is ok
    • Use our new benchmarks to measure the performance impact. If we see a big impact, we would consider apply the change.
  • Implement the changes if the performance impact is significant.

Impacted team

@meilisearch/cloud-team

@curquiza curquiza added enhancement New feature or improvement tooling Not directly project related, like Docker, Homebrew... impacts cloud This issue involves changes for the Meilisearch's cloud team labels Feb 16, 2023
@curquiza curquiza added this to the v1.1.0 milestone Feb 16, 2023
@curquiza curquiza self-assigned this Feb 16, 2023
@curquiza curquiza changed the title Use ubuntu instead of alpine in Docker image Use ubuntu/debian instead of alpine in Docker image Feb 16, 2023
@curquiza curquiza linked a pull request Feb 20, 2023 that will close this issue
@curquiza
Copy link
Member Author

Some update: we will not use ubuntu because there is not official rust docker image with ubuntu

@curquiza
Copy link
Member Author

curquiza commented Feb 21, 2023

The current prototype you can use to test the new image is

docker run -it --rm -p 7700:7700 -v $(pwd)/meili_data:/meili_data getmeili/meilisearch:prototype-debian-docker-image-0

Please let me know here if anyone has any issue with it 🙏

⚠️ Prototypes are not recommended for production

@mmachatschek
Copy link

mmachatschek commented Feb 22, 2023

@curquiza I support this change, but it may happen that users are extending the docker image in their own dockerfile. So this may lead to a breaking change.

e.g.

FROM getmeili/meilisearch:1.0.0

RUN ...do some stuff \
         # this will break the user CI pipeline if alpine is switched to debian
         && apk run some alpine specific command

A workaround would be to build for debian and alpine and tag the images accordingly

# docker tag
getmeili/meilisearch:1.0.0 === getmeili/meilisearch:1.0.0-alpine
# docker tag
getmeili/meilisearch:1.0.0-debian <= new debian image

@curquiza
Copy link
Member Author

curquiza commented Feb 22, 2023

Hum you are right that's true, it can be breaking for the users...
And I'm not really happy with having two dockerfile + tagging every Meilisearch image twice

Thanks @mmachatschek for making us realize this 🙏

@curquiza
Copy link
Member Author

@dureuill what do you think?

@dureuill
Copy link
Contributor

We're moving away from alpine because of its use of musl as libc, which is thought to potentially affect performance, and could cause issues specific to this libc implementation.

This change is made to be consistent with what we're testing day to day. With regards to these motivations I don't think it makes sense to make this change if we're going to keep supporting both docker variants.

As a result I think the solutions that make sense currently are either:

  1. commit to make musl supported instead of glibc. Don't provide a debian image, or
  2. make the breaking change by switching to the debian image and stop providing the musl image, with a note in the changelogs of v1.x about the breakage.

To help us decide it would be good to have some measure of how many people and CIs would be impacted by that change. Is extending the Meilisearch docker image common? What are the typical use-cases?

@curquiza
Copy link
Member Author

Is extending the Meilisearch docker image common?

I guess it can concern a lot of people since it's quite common in the Docker world to create an image based on another one. It's all the "power" of Docker. But I don't know how many people are concerned for Meilisearch specifically. I know the cloud team is doing it for example and would have to change their Dockerfile because of this change

@curquiza
Copy link
Member Author

However, in Semver, changing a dependency is not considered as breaking, I don't know if the rule is exactly related to our situation
We added a rule about this in our versioning policy

@mmachatschek
Copy link

mmachatschek commented Feb 22, 2023

@dureuill

to be clear here, I totally support this change.

We were also running alpine images with a node application that uses prisma (it's written in rust under the hood). Alpine slowed down our application a lot and switching to the debian based images gave us a lot of performance back. there are some GH issues and blog posts that confirm this.

These articles (+ internal performance tests) made me switch to the debian image because the main reason (small image size) was irrelevant as we use debian-slim that has roughly the same image size as the alpine image version.

Is extending the Meilisearch docker image common? What are the typical use-cases?

when the meilisearch batch feature was opt in, we created a meilisearch dockerfile that just changed the command and added the cli flag to activate it. But there was no alpine specific code involved. This could have been solved via a docker-compose command config too (without needing to add a dockerfile)

Those results look like most people don't use any alpine specifc commands in their dockerfile
https://github.com/search?l=Dockerfile&q=getmeili%2Fmeilisearch&type=Code

These were the only usages that I could find that apply some alpine specific commands (and where the dockerfile uses the latest tag)

  1. make the breaking change by switching to the debian image and stop providing the musl image, with a note in the changelogs of v1.x about the breakage.

preferred +1

@dureuill
Copy link
Contributor

@mmachatschek thank you so much this is a great answer ❤️

I thought we linked articles on musl performance somewhere but couldn't find them back after a cursory search, so thank you for contributing some of them.

These were the only usages that I could find that apply some alpine specific commands (and where the dockerfile uses the latest tag)

Thank you for the search! Since this appears to be a manageable number of cases, I guess we could even send advance notice to these repositories before making the change.

@curquiza
Copy link
Member Author

curquiza commented Feb 23, 2023

Thanks a lot @mmachatschek for your help here

I talked with @dureuill and because of the change it involves for the users, we would rather not release it before measuring it: we want to ensure this change has a real impact on performance to deserve a breaking change on one of our tools. Even if the breaking change is not in the search engine directly, the docker image is still a widely used tool.
I've updated the issue description.

Because of the lack of time before v1.1, we will not be able to apply this change for release v1.1, but more for release v1.2 or v1.3

I ping @meilisearch/cloud-team about the change.

@curquiza curquiza removed this from the v1.1.0 milestone Feb 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or improvement impacts cloud This issue involves changes for the Meilisearch's cloud team tooling Not directly project related, like Docker, Homebrew...
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants