[BUG] Hyperscan not supported for ARM64 == Error logs for bad hs database?

[DerLinkman]

Prerequisites

• Put an X between the brackets on this line if you have done all of the following:
  • Read about bug reporting in general: https://rspamd.com/doc/faq.html#how-to-report-bugs-found-in-rspamd
  • Enabled relevant debugging logs: https://rspamd.com/doc/faq.html#how-to-debug-some-module-in-rspamd
  • Checked the FAQs about Core files in case of fatal crash: https://rspamd.com/doc/faq.html#how-to-figure-out-why-rspamd-process-crashed
  • Tried ASAN package and obtained the ASAN report (if possible): https://rspamd.com/doc/faq.html#asan-builds
  • Checked that your issue isn't already filed: https://github.com/issues?utf8=%E2%9C%93&q=is%3Aissue+user%3Arspamd
  • Checked that there is not already an experimental package or master branch

Describe the bug
As mailcow is currently evolving for ARM64 as well we of course want to implement Rspamd for ARM64 as it is officially supported now.

Rspamd itself is starting normally and it seem to run fine until it is loading the Hyperscan files. For each and every .hs File which has been freshly compiled a few seconds ago Rspamd argues: "bad hs database".

Could it be that this issue is caused by the incompatibility of Hyperscan on ARM64 Devices?

Steps to Reproduce (within mailcow, configs are the same as on x86)

Do on ARM64 Server

1. Clone mailcow Repo
2. Checkout arm64-cow Branch
3. Setup mailcow with generate_config.sh
4. Startup mailcow with docker compose up -d
5. Observe logs (rspamd-mailcow)

Expected behavior
On first boot Rspamd will start compile the Regex files with Hyperscan (works, seems at least).
After a short while Rspamd moans:

mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//b639fb40fa9aff96f99153827657595d850b9c386892a63abb8e59ba90cc5570.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//90b29c525932f495bd73e4e4564678b3dee3cae036c59337e25ec759f0d1d26c.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//937f2f25c7c3264d25d59ae805484ab45d63d4fc63225a77c2caf8db5f92f66f.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//8a08d133fb2fb2b200d6fd1a1ac31a6754a336abc48e259972b5071ef4b7de3e.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//4cb1e41666056df0b9bff463256027b1632db6528f4f9846140944826cf9191f.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//e627a564d29a4c19b5dfd0da4815169d196b5b3b71e92fd28491f5f25bd15e92.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//e2d495cb169a1e5b7766dc7940d140c14081bec675915e1482a687b26eca659a.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//51a7c1bd6443cca3c8c5d0381e6a98f4c520c4993896959ac2dba8138d5ff866.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//c6b8dde1c87f1c2739366bc8d531624b3c13203d83cd7c8a5dd811cab5da784f.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//8ffa1c8581755f6e3bcc0540c81d3db7a499274be89a6072ff7ca82cb50c3939.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//30337d31a5bab88189d68d230e129b3599be234aa72d2ce88ff64288080d9b5b.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//b37a14eb0753a94f724d85b1f3eeedef701921abeecb41523e7203cc033bd0fc.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//1f3461cde3f69b64c049d7294c364fe9c66eb1c5339b503a5688c53900bd9fa7.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//2285cbc8b550bbe23af8ad5afdb0a7550da5b51a79316bbd6f06c6ff6586f590.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//4605bee0e64e78d06ab7b07464931e36ae729821b6819369c7ea51b1d25dc9c6.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//7837b4fdb442981f90ecff4a87740703dae61d345edbe31f9b64ceccf54111d9.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//af31ae57dc9aba28b96468cbd13de536d43667fa4cf54bf0651696b79284af54.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//327b2778d1e25ab160c98cbcd6d414b97f7457611d64090566431b945b6fe159.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//1ca5b57924257003d13141db523474997a85c768a7fe01c09ad3cf5a523c6d6e.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//3e705c4872513f9dbab26150bbccc6b68b9f2fb6b8684a499942b874631bea08.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//63cc96daad1f8b77687820e097a90da2b7373e188f146106ade3cd04f01a62b2.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//d0bc9bc603ee2c626e8af3cfb956508a67697f640506ef6db52645af6cdbf2b6.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//e94dceecdb25306c8e86a5b7f04c43e7ef8c7a440fb6217718db260bde657204.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//5e8c4d4566577c878048357a61c8d1895bf8cdb6b2941f4dd06c2c5926d2ced7.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//3ee9b4c4591c1d9353ab84ab62bf150453ac1d3c128b73b332a5d09e21b2df8b.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//616d9eea426c68bc1857f43b46ed885741b040c452b118db8382ff45fb83e5ec.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//bfcebcae6551d61e3fc03f324eeceef4319ca749ee18809791b4dd8003d0f645.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//ce82ac85a0c4f48dae612c5445065e8203fed938401dc3d0e19b70d441ed06cd.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//365a1d0758771159d90c40d24343aef3dc2091aa0c0c6ad0ab0e0835933f54a2.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//58701d6068d860aeff7d0a0d7b89b5b76912229a3590e904bde66ae2e0d5a5bd.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//d30f9b7232aa37752b41f49bf03ed61a384d8a05c483ccb181e52920f378329e.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//47fd77d5ad38a19d64ad5cb3457664d02624f8742dfceb441ad2d810525b1b03.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//1d3ba7e3ffab40345e576837cc869b7b81bcdc06ac1e4b3969a64e175b1f4a74.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//030ad6e18e6437ea229216d38a98989be8c947a3069dc6ea35ea1dab23d77d3f.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//922fe5308e444bbc4a5b3a5fcf224e882848e2db9f878a557c2db5bdfc91a8d3.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//65971896830c8aa1f75fd7e946bf5cb47815bc1cf95d44fd926d6c0eaec2129e.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//bff3f996bfce141c49551c494ee7e6fd6e7353cec1789192d01a5f41ec0ecd19.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//9868c80b8bf89c7f245d571330efb245ac727bb9283af081513b450ad2410b59.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//25f0dffaa9e00daa269996697bdaf190e12a677b0f17e971fafdfead59a4d1cb.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//e32a0af4e0ab52fd8f212bc7b4991f1b3ec5db59bb299215810addeb5b3fb4a2.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//a7af6aa709d4d4140eaf15457e80afd2126baf934fdfc6f0d3715c6e606b626b.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//1db7b754c05eb9978fa150ec7111b542ae0648b8079bc843c8c5ad9c3bf2635c.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//184db27df3f6475850bcb6e0da86ce1b5c0ac62792d6ab626e9678c1a85e7b88.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//7d50fd03725c90655980bfb0bbb2be6e4e3efa06e33d72fea7c9b404598b9355.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//89e653e5cd23d8d54c6ef3146e18cf4aaff422e8c03dd4cdca193c70baa64ab6.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//4a798f9fa466bba776fedc307b8170fbc596243906b7eedf7c1ed26cc0cbc46c.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//d10d50cebd5dade2dda86964f0a97fb82855e3ca3b12278391b3bbe143678558.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//ade144f5b7b3319071295269424a499f8ca66f60d7e82748bfce0c01d78ea46e.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//a3efd5f26ef1fd42b06b3177a25b79a11cb98de8220cb39cfe563079b7a307aa.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//ad90f739ad604ff25f014e1ebecaefbfc7bc61d2fd3bd6dcf9e47d6b3be0cf4f.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//e42ce3f5c623e3a10263af1b90dadeb5d1ea03ddc76295c1639d9dc21f7d268e.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//fe75297ae444cecac48d69deed9da91c66194c14673e54ea8cd46cdea2197af9.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//a5d0aaf81d1a6e8d4960bbc48c349fe7502a182a4676f03f8b0e03143f9f3a51.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//dc28acb31092193b04ec8c5b3b74c905403db23a31c54e52e1630c96167d9bb9.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//a37be7ff7bb18bce9f2c28e5bae2f5d9db6f46f4c2ccef70647d7371e9a98914.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//f238a1d673bfdd578bae7e1cc675da92805205334caab73091b9cf191e36c86f.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//37e1680c6c6eea62581ab2bb4d45d9dd83e1a964beb724741a9064c3f8408b7f.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//720388b3efb4f7ab4f77df89dcc0bc6285007f87b33567b63a5541330e4de223.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//9229a654f0ced64ae14b9b769363a4f455adc1e32192b0971d536b2c5cb45ece.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//e951f097c63cb18ac7926deb7fa3fd487448a3f345bcab37334cc7c196f14dba.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//d3f9209070b1b69283cdb8afd376e463856a0969a1b25fcceaa6cbc20272e953.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//4f2dbf8e252e5b57a2b1efe053e38f12d08f3af816664df4742e42b7b7b765e2.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//103c565313a0148208f1d051e056535cbb59a65293dd4137eff7140e66c6244c.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//20118d1a15231f8f14a054495e07e755b86178122a3902da5c2549593d9195e0.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//a45d8b768dabb2389e8787e00f02642ce8cb964b9b5691c7b5651f7cecdb69eb.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//fe339d1ff91cd68d88e0d02547d48d976d5fc466d38402f65f1bc528077961d2.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//2de525844493f6c1b013db4293533f876fcb2a231bed9022de00837878f2b9e2.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//fa200d5fa262af6276fefdd36a0dbab92098dacd5569e185523fa2c0544f6bb4.hs
mailcowdockerized-rspamd-mailcow-1  | 2023-05-23 16:26:30 #47(controller) <c917e5>; re_cache; rspamd_re_cache_load_hyperscan: bad hs database in /var/lib/rspamd//27dc856a60f0c102159d78eabe2dbf6cea01878ab698e8e5b843f11b4416b91c.hs

Versions

rspamd --version
Rspamd daemon version 3.5

CPU: Ampere® Altra® (Hetzner Cloud Server)

Additional Information

The settings within mailcow did not changed the Dockerfile (to build Rspamd) neither. As Rspamd is available for ARM64 on Debian 11 as well it should be fine in my opinion.

[BarsMonster]

My understanding is that HyperScan will never work on non-x86 platform. Requests to merge ARM64 support were declined by hyperscan maintainer (as it's spearheaded by Intel): intel/hyperscan#197 (comment) intel/hyperscan#212

Potential solutions could be:

1. Use Vectorscan for non-x86 platform instead, or for all platforms. Vectorscan does not support x86_32 though (only x86_64), not sure how popular it is. Optimistically, Vectorscan can be plug-and-play replacement. In the future this will also enable rspamd on other emerging platforms, like RISC-V.
2. Disable hyperscan in ARM64 (or non-x86) build scripts. Lower performance is expected, but might not be the end of the world.

[vstakhov]

Rspamd official packages for arm64 are built with vectorscan. But the log files above are quite strange, as they should have more information about the fail reason...

[vstakhov]

I'd suggest to enable debug for hyperscan module.

[DerLinkman]

I've got some logs after enabling the Debug logging:
rspamd_hs-debug.txt

That .hs Files have been freshly compiled right after i've started the rspamd container for a fresh initialisation.

[vstakhov]

I have occasionally reproduced this issue locally. I'll try to fix it asap.

[DerLinkman]

Thank you so much!

[vstakhov]

It seems that Rspamd do not delete those files for some reason.

[DerLinkman]

Maybe coexistent with the issue on ARM64 @vstakhov ?

mailcow/mailcow-dockerized#5232

[DerLinkman]

Hi again!

I've recently tested the latest Repo Build of Rspamd 3.6 (Build Date is 04th July 2023) but i can still see Rspamd moaning about the bad HS Databases inside the /var/lib/rspamd/folder (see logs attached).

It seems that your fixes did not achieved the desired effect i'm affraid :(
log.txt

(It's still on ARM64 though)

[vstakhov]

Ok, I have debugged this issue and it seems to be clearly a vectorscan issue. It exports HS_PATCH as 0 and defines it to 8 internally. It leads to this misbehaviour. This issue is occasionally fixed in 5.4.9.

[DerLinkman]

Hey @vstakhov!

Thank you for debugging! Looking forward to see it working soon™️

Very appreciate it!

[DerLinkman]

Hello again!

Thank you so much. I could test the latest build on your APT Repo (build date: 31-Jul-2023 17:35) and it works now as expected on ARM64.

Just curios to ask: Did you have a ETA (or release window) for Rspamd 3.6?

[DerLinkman]

Thank you for Releasing 3.6 i think this issue can be closed then as it has been solved within the previously dev builds already.