Disk writes to media folder failing after upgrade from 2.021.45 to 2.022.16

[Yann-J]

Hello!

We recently tried the upgrade described in the title, and while the application was running fine, all disk writes to our media folder started failing, basically causing all the submissions to fail when they include an attachment, as well as all new form creations to fail.

Our media folder is mounted as an Azure File (the only storage provider in our Kubernetes cluster that supports ReadWriteMany mode, so we can have multiple pods mounting the same drive). This storage provider has very basic permissions management, in that all files have the same ownership (root) and permissions (0777).

The error reported by the stack trace indicate an issue when kobocat seems to attempt to update the permissions for the file being written:

  File "/opt/venv/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1317, in <listcomp>
    [self.prepare_value(field, self.pre_save_val(field, obj)) for field in fields]
  File "/opt/venv/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1317, in <listcomp>
    [self.prepare_value(field, self.pre_save_val(field, obj)) for field in fields]
  File "/opt/venv/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1268, in pre_save_val
    return field.pre_save(obj, add=True)
  File "/opt/venv/lib/python3.8/site-packages/django/db/models/fields/files.py", line 288, in pre_save
    file.save(file.name, file.file, save=False)
  File "/opt/venv/lib/python3.8/site-packages/django/db/models/fields/files.py", line 87, in save
    self.name = self.storage.save(name, content, max_length=self.field.max_length)
  File "/opt/venv/lib/python3.8/site-packages/django/core/files/storage.py", line 52, in save
    return self._save(name, content)
  File "/opt/venv/lib/python3.8/site-packages/django/core/files/storage.py", line 290, in _save
    os.chmod(full_path, self.file_permissions_mode)
PermissionError: [Errno 1] Operation not permitted: '/srv/src/kobocat/media/kobo/attachments/4aef14ddfba543f6b0fae779a0ed41c2/d1776c8c-8101-4137-af83-dea7874c906f/rapid_tests-17_30_57_Ysuccrn.png'

Now, I know this configuration is not really supported since Kobocat expects the media directory to be owned by the kobo user (1000:1000 in the docker images). We're working on fixing this and changing ownership to the kobo user (chown does not work on this storage provider - the command succeeds [when run as root], but it has no effect, so we don't have an obvious solution yet).

However, I'm curious as to what could be causing this issue, since:

• Even if the files are not owned by the kobo user, they have RW permissions to all users, so there shouldn't be any permission issues
• This issue is clearly introduced by some new logic betweem these 2 releases, and 2.021.45 works perfectly fine on this setup...

Is there any workaround you could suggest to allow kobocat to run with a media directory on which is has permissions, but not ownership? I'm not sure why it seems to be trying to change permissions on the files, can this be deactivated?

Many thanks!

[noliveleger]

Hello @Yann-J,
Your setup is pretty different from what we daily use.
You can have a look at our beta branches (KPI and KoBoCAT). It has some support for Azure storage.
You can find the environment variables needed to be configured here in Azure section.

[Yann-J]

Thank you, this is helpful! Using an Azure blob storage is indeed certainly a much more scalable approach than mounting volumes.

Is there a publicly available docker image build for the beta branch or do we have to build our own?

Also, how can we get an idea of when the beta branch makes it into a release?

[noliveleger]

Is there a publicly available docker image build for the beta branch or do we have to build our own?

You have to build your own.

Also, how can we get an idea of when the beta branch makes it into a release?

Unfortunately, no ETA but I would say soon (without any guarantee ;-) ).