Test failures on s390x

[sergiodj]

Hello,

This is a followup of #420. Some tests are still failing on s390x:

[----------] 2 tests from CryConfigCompatibilityTest
[ RUN      ] CryConfigCompatibilityTest.v0_8_1_with_aes_256_gcm
[2022-09-02 06:17:36.994] [Log] [error] Error deserializing outer configuration: Deserialization failed - size overflow
unknown file: Failure
C++ exception with description "Tried to get right side of an either which is left." thrown in the test body.
[  FAILED  ] CryConfigCompatibilityTest.v0_8_1_with_aes_256_gcm (1 ms)
[ RUN      ] CryConfigCompatibilityTest.v0_8_1_with_serpent_128_cfb
[2022-09-02 06:17:36.995] [Log] [error] Error deserializing outer configuration: Deserialization failed - size overflow
unknown file: Failure
C++ exception with description "Tried to get right side of an either which is left." thrown in the test body.
[  FAILED  ] CryConfigCompatibilityTest.v0_8_1_with_serpent_128_cfb (1 ms)
[----------] 2 tests from CryConfigCompatibilityTest (3 ms total)

https://ci.debian.net/data/autopkgtest/unstable/s390x/c/cryfs/25603505/log.gz

This is also affecting Ubuntu, and is preventing the new version of cryfs from migrating to the release pocket there.

Thanks.

[sergiodj]

@davesteele may be interested in this as well.

[davesteele]

This is blocking package migration in Debian.

[smessmer]

Is this a big endian architecture?

[davesteele]

Google says yes.

[smessmer]

hm I looked through that code path and didn't find any obvious endianness errors, unless it's a bug in the crypto++ library we use. Where does debian run those tests? Do you have access to those machines? Running a git bisect could be useful to figure out which commit introduced the bug.

[davesteele]

I modified the makefile to add -DDISABLE_ASM=ON to the s390x build (re #420). The failure persists. Debian is not flagging the problem at the moment, because it is not a regression from the first time it was tested. That could change.

The git bisect on the hardware has not proved viable.

[davesteele]

Trying to find it via code inspection:

$ git log --oneline  0.10.2..0.11.1 -- src/cryfs/impl/config/ src/cryfs/config
b28b8b6e Show old config values at mount time, otherwise they'd just always show the current version
a245ac1f Add XChaCha20-Poly1305 cipher and make it the default
3938942a - cryfs-stats tool is guaranteed to be readonly and not modify the file system - Now shows a better error message when failing to load the config file and distinguishes between 'wrong password' and 'config file not found' - The cryfs-stats tool only reads and never writes the cryfs.config file
f6ef18b0 Fix gcc build
50341b76 Merge branch 'release/0.10' into develop
c5febd03 Merge branch 'release/0.10' into develop
040f0636 Merge branch 'release/0.10' into develop
58fd7c39 Merge from release/0.10
b2f9e5cc Move cryfs code into impl/ subfolder
6fff8c80 Merge branch 'develop' into feature/library_intermediate
86ef1012 Merge branch 'develop' into feature/remove_cryptopp_compat
3ad6e484 fix clang-tidy
227058a8 Merge branch 'develop' into feature/library_intermediate
1118e9d5 Fix clang-tidy
a34095ff Merge branch 'release/0.10' into develop
b526c3fd Merge branch 'develop' into feature/library_intermediate
4f704377 Remove legacy code making CryFS compatible with older Crypto++ versions. We need a newer Crypto++ versions now anyhow because we use its scrypt implementation.
0db851f7 Merge branch 'release/0.10' into develop
99688e51 Use either<> to indicate errors in config file loading

Recalling that the error message log for the failure includes:

[ RUN      ] CryConfigCompatibilityTest.v0_8_1_with_aes_256_gcm
[2022-12-01 08:36:38.259] [Log] [error] Error deserializing outer configuration: Deserialization failed - size overflow
unknown file: Failure
C++ exception with description "Tried to get right side of an either which is left." thrown in the test body.
[  FAILED  ] CryConfigCompatibilityTest.v0_8_1_with_aes_256_gcm (0 ms)
[ RUN      ] CryConfigCompatibilityTest.v0_8_1_with_serpent_128_cfb
[2022-12-01 08:36:38.259] [Log] [error] Error deserializing outer configuration: Deserialization failed - size overflow
unknown file: Failure
C++ exception with description "Tried to get right side of an either which is left." thrown in the test body.
[  FAILED  ] CryConfigCompatibilityTest.v0_8_1_with_serpent_128_cfb (1 ms)
[----------] 2 tests from CryConfigCompatibilityTest (1 ms total)

Candidates from a cursory inspection:

• 99688e51
• 3938942a

See also git diff 0.10.2..0.11.1 -- src/cryfs/impl/config/ src/cryfs/config

[davesteele]

This line looks suspect, given the error:

C++ exception with description "Tried to get right side of an either which is left." thrown in the test body.