Compilation failed in ArchLinux #3189

mortymacs · 2022-05-18T21:23:19Z

In ArchLinux I got this error recently, while it worked in the past:

OS: ArchLinux
GCC: 12.1.0
CMake: 3.23.1

This is how I compile it (based on sample Dockerfile):

../bootstrap \
    --prefix=/usr/local \
    --enable-verbose \
    --enable-serialization \
    --enable-tools
make -j "$(nproc)"

Logs:

-- Starting TileDB superbuild.
-- Found Zlib, adding imported target: /usr/lib/libz.so
-- Adding Magic as an external project
-- Found CapnProto lib: capnp
-- Found CapnProto lib: kj
-- Found CapnProto lib: capnp-json
-- Found OpenSSL: /usr/lib/libssl.so -- OpenSSL crypto: /usr/lib/libcrypto.so -- root: /home/mort/tiledb/src/TileDB-2.9.0/build/externals/install;/usr/local/opt/openssl
-- Found CURL: '/usr/lib/libcurl.so' (found version "")
-- searching for catch in /home/mort/tiledb/src/TileDB-2.9.0/build/externals/src
-- Found clang-tidy: /usr/bin/clang-tidy
-- Found clang-format: /usr/bin/clang-format
-- clang hunt, found /usr/bin/clang-format
-- Install prefix is /home/mort/tiledb/pkg/tiledb/usr/local
-- Configuring done
-- Generating done
-- Build files have been written to: /home/mort/tiledb/src/TileDB-2.9.0/build
bootstrap success. Run "make" to build, "make check" to test, or "make -C tiledb install" to install.
[ 50%] Built target ep_magic
[ 56%] Performing build step for 'tiledb'
Consolidate compiler generated dependencies of target baseline
[  2%] Generating ../../../tiledb/sm/misc/magic_mgc_gzipped.bin
[  5%] Built target time
[  5%] Built target stringx
[  5%] Built target math
[  7%] Built target try_assert
[  7%] Built target constants
[ 10%] Built target datum
[ 10%] Building CXX object tiledb/common/CMakeFiles/baseline.dir/heap_memory.cc.o
[ 10%] Built target test_assert
[ 10%] Built target gen_mgc_unarch
Consolidate compiler generated dependencies of target TILEDB_CORE_OBJECTS
[ 10%] Building CXX object tiledb/CMakeFiles/TILEDB_CORE_OBJECTS.dir/common/heap_memory.cc.o
[ 13%] Building CXX object tiledb/CMakeFiles/TILEDB_CORE_OBJECTS.dir/sm/c_api/tiledb.cc.o
[ 13%] Building CXX object tiledb/CMakeFiles/TILEDB_CORE_OBJECTS.dir/sm/cache/buffer_lru_cache.cc.o
[ 13%] Building CXX object tiledb/CMakeFiles/TILEDB_CORE_OBJECTS.dir/sm/compressors/dd_compressor.cc.o
[ 13%] Building CXX object tiledb/CMakeFiles/TILEDB_CORE_OBJECTS.dir/sm/compressors/bzip_compressor.cc.o
[ 15%] Building CXX object tiledb/CMakeFiles/TILEDB_CORE_OBJECTS.dir/sm/compressors/dict_compressor.cc.o
[ 15%] Building CXX object tiledb/CMakeFiles/TILEDB_CORE_OBJECTS.dir/sm/compressors/gzip_compressor.cc.o
/home/mort/tiledb/src/TileDB-2.9.0/tiledb/common/heap_memory.cc: In function ‘void* tiledb::common::tiledb_realloc(void*, std::size_t, const std::string&)’:
/home/mort/tiledb/src/TileDB-2.9.0/tiledb/common/heap_memory.cc:93:31: error: pointer ‘p’ may be used after ‘void* realloc(void*, size_t)’ [-Werror=use-after-free]
   93 |   heap_profiler.record_dealloc(p);
      |   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
/home/mort/tiledb/src/TileDB-2.9.0/tiledb/common/heap_memory.cc:88:39: note: call to ‘void* realloc(void*, size_t)’ here
   88 |   void* const p_realloc = std::realloc(p, size);
      |                           ~~~~~~~~~~~~^~~~~~~~~
/home/mort/tiledb/src/TileDB-2.9.0/tiledb/common/heap_memory.cc: In function ‘void tiledb::common::tiledb_free(void*)’:
/home/mort/tiledb/src/TileDB-2.9.0/tiledb/common/heap_memory.cc:108:31: error: pointer ‘p’ used after ‘void free(void*)’ [-Werror=use-after-free]
  108 |   heap_profiler.record_dealloc(p);
      |   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
/home/mort/tiledb/src/TileDB-2.9.0/tiledb/common/heap_memory.cc:107:7: note: call to ‘void free(void*)’ here
  107 |   free(p);
      |   ~~~~^~~
cc1plus: all warnings being treated as errors
make[5]: *** [tiledb/common/CMakeFiles/baseline.dir/build.make:118: tiledb/common/CMakeFiles/baseline.dir/heap_memory.cc.o] Error 1
make[4]: *** [CMakeFiles/Makefile2:1113: tiledb/common/CMakeFiles/baseline.dir/all] Error 2
make[4]: *** Waiting for unfinished jobs....
[ 15%] Building CXX object tiledb/CMakeFiles/TILEDB_CORE_OBJECTS.dir/sm/compressors/lz4_compressor.cc.o
/home/mort/tiledb/src/TileDB-2.9.0/tiledb/common/heap_memory.cc: In function ‘void* tiledb::common::tiledb_realloc(void*, std::size_t, const std::string&)’:
/home/mort/tiledb/src/TileDB-2.9.0/tiledb/common/heap_memory.cc:93:31: error: pointer ‘p’ may be used after ‘void* realloc(void*, size_t)’ [-Werror=use-after-free]
   93 |   heap_profiler.record_dealloc(p);
      |   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
/home/mort/tiledb/src/TileDB-2.9.0/tiledb/common/heap_memory.cc:88:39: note: call to ‘void* realloc(void*, size_t)’ here
   88 |   void* const p_realloc = std::realloc(p, size);
      |                           ~~~~~~~~~~~~^~~~~~~~~
/home/mort/tiledb/src/TileDB-2.9.0/tiledb/common/heap_memory.cc: In function ‘void tiledb::common::tiledb_free(void*)’:
/home/mort/tiledb/src/TileDB-2.9.0/tiledb/common/heap_memory.cc:108:31: error: pointer ‘p’ used after ‘void free(void*)’ [-Werror=use-after-free]
  108 |   heap_profiler.record_dealloc(p);
      |   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
/home/mort/tiledb/src/TileDB-2.9.0/tiledb/common/heap_memory.cc:107:7: note: call to ‘void free(void*)’ here
  107 |   free(p);
      |   ~~~~^~~
cc1plus: all warnings being treated as errors
make[5]: *** [tiledb/CMakeFiles/TILEDB_CORE_OBJECTS.dir/build.make:76: tiledb/CMakeFiles/TILEDB_CORE_OBJECTS.dir/common/heap_memory.cc.o] Error 1
make[5]: *** Waiting for unfinished jobs....
make[4]: *** [CMakeFiles/Makefile2:983: tiledb/CMakeFiles/TILEDB_CORE_OBJECTS.dir/all] Error 2
make[3]: *** [Makefile:146: all] Error 2
make[2]: *** [CMakeFiles/tiledb.dir/build.make:86: tiledb-prefix/src/tiledb-stamp/tiledb-build] Error 2
make[1]: *** [CMakeFiles/Makefile2:117: CMakeFiles/tiledb.dir/all] Error 2
make: *** [Makefile:91: all] Error 2

The text was updated successfully, but these errors were encountered:

eddelbuettel · 2022-05-18T22:18:52Z

Thanks for the bug report.

The most recent g++ and clang++ versions emit more warnings, and the build defaults to treating warnings as errrors, creating an impasse. The build will work with g++-11, for g++-12 you can add the --disable-werror flag to the bootstrap invocation. Please let us know if the build still fails with warnings 'as just warnings'.

eddelbuettel · 2022-12-27T18:38:51Z

FWIW this now also happens on e.g. Ubuntu 22.10 with its default compiler.

mortymacs · 2022-12-27T22:41:46Z

This is what I applied in the ArchLinux AUR package https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=tiledb#n25 as you suggested.

eddelbuettel · 2022-12-27T22:55:06Z

Oh yes, that works, I use it too with the newer compiler -- but we really should change the code to not have to rely on 'no warnings as errors' which is otherwise nice to have. I have a quick peek again at the code but it's not obvious how to rearrange it to not upset the compiler here.

mortymacs · 2023-01-13T22:12:15Z

Hello :)

Did you change something in the new version? because the building process has been failed:
Could not determine library location for WebP::webp

The complete log:

-- Found OpenSSL: /usr/lib/libssl.so -- OpenSSL crypto: /usr/lib/libcrypto.so -- root: /home/mort/Workspaces/aur.archlinux.org/tiledb/src/TileDB-2.13.2/build/externals/install;/usr/local/opt/openssl
-- Found CURL: '/usr/lib/libcurl.so' (found version "")
-- Adding Catch as an external project
CMake Warning (dev) at /usr/share/cmake/Modules/ExternalProject.cmake:3075 (message):
  The DOWNLOAD_EXTRACT_TIMESTAMP option was not given and policy CMP0135 is
  not set.  The policy's OLD behavior will be used.  When using a URL
  download, the timestamps of extracted files should preferably be that of
  the time of extraction, otherwise code that depends on the extracted
  contents might not be rebuilt if the URL changes.  The OLD behavior
  preserves the timestamps from the archive instead, but this is usually not
  what you want.  Update your project to the NEW behavior or specify the
  DOWNLOAD_EXTRACT_TIMESTAMP option with a value of true to avoid this
  robustness issue.
Call Stack (most recent call first):
  /usr/share/cmake/Modules/ExternalProject.cmake:4185 (_ep_add_download_command)
  cmake/Modules/FindCatch_EP.cmake:71 (ExternalProject_Add)
  cmake/TileDB-Superbuild.cmake:133 (include)
  CMakeLists.txt:189 (include)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Found clang-tidy: /usr/bin/clang-tidy
-- Found clang-format: /usr/bin/clang-format
-- clang hunt, found /usr/bin/clang-format
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
-- Install prefix is /home/mort/Workspaces/aur.archlinux.org/tiledb/pkg/tiledb/usr/local
-- Configuring done
-- Generating done
-- Build files have been written to: /home/mort/Workspaces/aur.archlinux.org/tiledb/src/TileDB-2.13.2/build
bootstrap success. Run "make" to build, "make check" to test, or "make -C tiledb install" to install.
[ 46%] Built target ep_webp
[ 50%] Built target ep_magic
[ 75%] Built target ep_catch
[ 78%] Performing configure step for 'tiledb'
-- Starting TileDB regular build.
-- Performing Test HAVE_AVX2
-- Performing Test HAVE_AVX2 - Success
-- +++++++ in experimental
-- Found Zlib, adding imported target: /usr/lib/libz.so
CMake Error at cmake/Modules/TileDBCommon.cmake:91 (message):
  Could not determine library location for WebP::webp
Call Stack (most recent call first):
  cmake/Modules/FindWebp_EP.cmake:84 (install_target_libs)
  tiledb/sm/compressors/CMakeLists.txt:73 (find_package)


-- Configuring incomplete, errors occurred!
See also "/home/mort/Workspaces/aur.archlinux.org/tiledb/src/TileDB-2.13.2/build/tiledb/CMakeFiles/CMakeOutput.log".
make[2]: *** [CMakeFiles/tiledb.dir/build.make:95: tiledb-prefix/src/tiledb-stamp/tiledb-configure] Error 1
make[1]: *** [CMakeFiles/Makefile2:175: CMakeFiles/tiledb.dir/all] Error 2
make: *** [Makefile:91: all] Error 2

Here is the build system:

ls /usr/include/webp/                                                                                                                                   4 5s
Octal Permissions Size User Date Modified Name
0644  .rw-r--r--   24k root 13 Jan 11:47  decode.h
0644  .rw-r--r--   16k root 13 Jan 11:47  demux.h
0644  .rw-r--r--   28k root 13 Jan 11:47  encode.h
0644  .rw-r--r--   23k root 13 Jan 11:47  mux.h
0644  .rw-r--r--  3.2k root 13 Jan 11:47  mux_types.h
0755  drwxr-xr-x     - root 13 Jan 21:11  sharpyuv
0644  .rw-r--r--  2.2k root 13 Jan 11:47  types.h

ls /usr/share/WebP/cmake
Octal Permissions Size User Date Modified Name
0644  .rw-r--r--  1.3k root 13 Jan 11:47  WebPConfig.cmake
0644  .rw-r--r--  2.0k root 13 Jan 11:47  WebPConfigVersion.cmake
0644  .rw-r--r--  2.7k root 13 Jan 11:47  WebPTargets-none.cmake
0644  .rw-r--r--  5.2k root 13 Jan 11:47  WebPTargets.cmake

eddelbuettel · 2023-01-13T22:49:19Z

The build system was recently updated to use an external webp if present; this works for me on Ubuntu. Now, did you 'restart' here with a clean build/ dir, or is this by chance one from an earlier build?

mortymacs · 2023-01-13T23:07:40Z

I did, the same error. Is it libwebp? What package did you install in Ubuntu as a dependency?

eddelbuettel · 2023-01-13T23:13:00Z

For me it is these:

edd@rob:~$ dpkg -l| grep libwebp | cut -c-80
ii  libwebp-dev:amd64                              1.2.2-2                      
ii  libwebp7:amd64                                 1.2.2-2                      
ii  libwebpdemux2:amd64                            1.2.2-2                      
ii  libwebpmux3:amd64                              1.2.2-2                      
edd@rob:~$

But I actually muddied the waters. The change to use a system webp is still in a branch, mainline (also on my box) still builds it from (even though it probably does not have to). So I still have

edd@rob:~/git/tiledb/build-dev(dev)$ ls -l externals/install/lib/libwebp*
-rw-r--r-- 1 edd edd 3409358 Dec 12 11:36 externals/install/lib/libwebp.a
-rw-r--r-- 1 edd edd 1770728 Dec 12 11:36 externals/install/lib/libwebpdecoder.a
-rw-r--r-- 1 edd edd   81394 Dec 12 11:36 externals/install/lib/libwebpdemux.a
-rw-r--r-- 1 edd edd  218512 Dec 12 11:36 externals/install/lib/libwebpmux.a
edd@rob:~/git/tiledb/build-dev(dev)$

eddelbuettel · 2023-01-13T23:13:42Z

If this is a road-block you can also try -DTILEDB_WEBP=0 in the cmake call.

mortymacs · 2023-01-13T23:19:15Z

I'm trying to update the AUR package. What's your idea? should we ignore it?

davisp · 2023-01-13T23:27:36Z

@mortymacs @eddelbuettel First off, I'm pretty sure this is a bug in how we're building WebP without really trying to look for system installed versions (As @eddelbuettel mentioned, I have an in progress branch at [1] for this). A quick test you can do is to comment out the lines at [2].

@mortymacs That doesn't look like the entire output of the bootstrap step. Yours starts with -- Found OpenSSL: ... where there should also be compiler detection before that. And locally the important lines are just before that where we spit out whether webp was found. If you can paste the earlier lines that might make things more clear in what's going on. For the moment I can only guess that CMake is getting confused by the system installed WebPConfig.cmake after it decided that it needs to build webp.

[1] #3783
[2] https://github.com/TileDB-Inc/TileDB/blob/dev/cmake/Modules/FindWebp_EP.cmake#L81-L88

eddelbuettel · 2023-01-13T23:37:54Z

I'm trying to update the AUR package. What's your idea? should we ignore it?

I happen to have "inherited" (in the sense of I didn't start the process knowing how demanding our build can be once you turn enough features on) the Debian package (as I have been a Debian Developer / Contributor for many moons now) and if you promise to not tell anyone 😁 that is exactly what I did for 2.13.* -- turn off webp. Life too short and all that. We already must patch (or override) capnp and catch to have a 'clean room' build that doesn't run away to github so I (for now) suppress webp. I look forward to re-enabling it once the branch is merged.

mortymacs · 2023-01-14T19:33:23Z

Pkg-config shows I have the library installed:

$ pkg-config --libs libwebp --cflags
-I/usr/include/webp -lwebp -lsharpyuv

eddelbuettel · 2023-01-14T19:42:44Z

For what it is worth on Ubuntu it just shows

$ pkg-config libwebp --cflags 

$ pkg-config libwebp --libs
-lwebp
$

mortymacs · 2023-01-14T20:23:46Z

So, I have it as well. It's weird why it doesn't work here 🤔
But, as you suggested, I'm going to disable it just for now, but as soon as you can find a solution or fix it, please let me know to update the AUR package.
Thanks :)

eddelbuettel mentioned this issue May 19, 2022

failing to build tiledb 2.9.0 heap_memory.cc #3191

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Compilation failed in ArchLinux #3189

Compilation failed in ArchLinux #3189

mortymacs commented May 18, 2022

eddelbuettel commented May 18, 2022

eddelbuettel commented Dec 27, 2022

mortymacs commented Dec 27, 2022

eddelbuettel commented Dec 27, 2022

mortymacs commented Jan 13, 2023 •

edited

eddelbuettel commented Jan 13, 2023

mortymacs commented Jan 13, 2023

eddelbuettel commented Jan 13, 2023

eddelbuettel commented Jan 13, 2023

mortymacs commented Jan 13, 2023

davisp commented Jan 13, 2023

eddelbuettel commented Jan 13, 2023

mortymacs commented Jan 14, 2023

eddelbuettel commented Jan 14, 2023

mortymacs commented Jan 14, 2023

Compilation failed in ArchLinux #3189

Compilation failed in ArchLinux #3189

Comments

mortymacs commented May 18, 2022

eddelbuettel commented May 18, 2022

eddelbuettel commented Dec 27, 2022

mortymacs commented Dec 27, 2022

eddelbuettel commented Dec 27, 2022

mortymacs commented Jan 13, 2023 • edited

eddelbuettel commented Jan 13, 2023

mortymacs commented Jan 13, 2023

eddelbuettel commented Jan 13, 2023

eddelbuettel commented Jan 13, 2023

mortymacs commented Jan 13, 2023

davisp commented Jan 13, 2023

eddelbuettel commented Jan 13, 2023

mortymacs commented Jan 14, 2023

eddelbuettel commented Jan 14, 2023

mortymacs commented Jan 14, 2023

mortymacs commented Jan 13, 2023 •

edited