OPTEE build failure due to buildroot's incompatibility with glibc 2.33

[xiangmy]

Dear OPTEE group,

The glibc 2.33 removed _STAT_VER and the upstream buildroot has reflected this change (https://git.busybox.net/buildroot/commit/?id=f45925a951318e9e53bead80b363e004301adc6f). So, I'm wondering whether OPTEE wants to update the buildroot to 2021.02.

PS: I've tested the build process with qemu_v8.xml on my branch: 89c5402. I also attach the log of my failed build.

BTW, the edk2 is also incompatible with python 3.9. For instance, python 3.9 removed tostring() and fromstring, and this will cause errors while building (https://bugs.python.org/issue38916).

Thanks,
Mingyuan

[jforissier]

Hi @xiangmy,

Have you worked on upgrading Buildroot and EDK2 already? Or have you just met these errors? If you have patches to propose please do and I will be glad to review them. Otherwise I can work on the upgrade.

[xiangmy]

Hi @jforissier ,

I can clean up my changes and push them.

Thanks,
Mingyuan

[jforissier]

@xiangmy that would be very helpful, thanks!

[xiangmy]

Hi @jforissier ,

Sorry for the late update. Upgrading the EDK2 is quite painful. I tried various of commits/tags, but they all failed at some warnings with cc1: all warnings being treated as errors, then the make process failed. (Here is an example of a failed log: log1.txt) However, I manage to compile the whole system with only the following patches applied, which fix the python 3.9 compatibility:

• 5d8648345cd9ad729837118c731063c59edea192
• 43bec9ea3d56f3662ede78023baa2a791b66acac
• 20b292d0cdf7dce58d824fdf9ab1613c2a1ad2ec

BTW, the QEMU was also broken with binutils 2.35.50 (https://bugs.launchpad.net/ubuntu/+source/binutils/+bug/1907789), but I fixed this by using the latest QEMU. The problem is that QEMU has switched to meson after v5.2, which will break the commands for invoking the QEMU. (https://wiki.qemu.org/ChangeLog/5.2#Build_Information). To fix this, we should change $(QEMU_PATH)/aarch64-softmmu/qemu-system-aarch64 to $(QEMU_PATH)/build/aarch64-softmmu/qemu-system-aarch64 as well as other similar places.

However, the major problem is how I should upgrade EDK2. Do you have any advice?

In summary, I manage to upgrade the QEMU and Buildroot but need your adice in updagraing EDK2.

Thanks,
Mingyuan

[jforissier]

Hi @xiangmy,

The latest EDK2 stable tag works for me, with one additional change in build/qemu_v8.mk:

.repo/manifests $ git diff
diff --git a/qemu_v8.xml b/qemu_v8.xml
index 219ec3b..165771c 100644
--- a/qemu_v8.xml
+++ b/qemu_v8.xml
@@ -21,7 +21,7 @@

         <!-- Misc gits -->
         <project path="buildroot"            name="buildroot/buildroot.git"               revision="refs/tags/2020.08" clone-depth="1" />
-        <project path="edk2"                 name="tianocore/edk2.git"                    revision="refs/tags/edk2-stable201905" sync-s="true" />
+        <project path="edk2"                 name="tianocore/edk2.git"                    revision="refs/tags/edk2-stable202102" sync-s="true" />
         <project path="mbedtls"              name="ARMmbed/mbedtls.git"                   revision="refs/tags/mbedtls-2.16.0" clone-depth="1" />
         <project path="qemu"                 name="qemu/qemu.git"                         revision="refs/tags/v5.1.0" clone-depth="1" />
         <project path="trusted-firmware-a"   name="TF-A/trusted-firmware-a.git"           revision="refs/tags/v2.3" clone-depth="1" remote="tfo" />

build $ git diff
diff --git a/qemu_v8.mk b/qemu_v8.mk
index dfc52d4..6934949 100644
--- a/qemu_v8.mk
+++ b/qemu_v8.mk
@@ -29,7 +29,7 @@ DEBUG ?= 1
 TF_A_PATH              ?= $(ROOT)/trusted-firmware-a
 BINARIES_PATH          ?= $(ROOT)/out/bin
 EDK2_PATH              ?= $(ROOT)/edk2
-EDK2_TOOLCHAIN         ?= GCC49
+EDK2_TOOLCHAIN         ?= GCC5
 EDK2_ARCH              ?= AARCH64
 ifeq ($(DEBUG),1)
 EDK2_BUILD             ?= DEBUG

[xiangmy]

Hi @jforissier ,

Thanks for your help! Would you mind helping me review #180 and OP-TEE/build#464?

Thanks,
Mingyuan

[jbech-linaro]

I'm not sure if you noticed issues also with the toolchain. I got a new computer last week and did a setup from scratch. I get the impression that Arm have updated the toolchain package to comply with the expected toolchain string (i.e. in include vendor). Since the two of you are sorting out various build errors due to recent changes, you'll probably need this patch as well at some point. I.e., what I'm saying is that it could be worth re-try doing a full build from scratch, including make toolchains. The patch only fixes arm32, but the same issue exist on arm64 as well.

[jforissier]

@jbech-linaro isn't this addressed by https://github.com/OP-TEE/build/blob/3.12.0/toolchain.mk#L27?

[jbech-linaro]

@jbech-linaro isn't this addressed by https://github.com/OP-TEE/build/blob/3.12.0/toolchain.mk#L27?

Hmm, it certainly looks like it does. It looks like I must have made some mistake on my end in that case. Checking the commit date when you added this, tells me that it's pretty old (>6 months), which makes me even more confused. I'll re-run it to cross check.

[jbech-linaro]

I'll re-run it to cross check.

Symlinks are there after a re-run, so ... sorry for noise (and thanks pointing to this).

[jforissier]

@jbech-linaro no worries ;-)

[bitwave]

Is there any progress on this issue? I have the same problem building under Arch.

[xiangmy]

Hi @bitwave ,

I think the fix was already merged (check #180). I tested the P/R under Arch so I suppose it should work for you. What are the errors you got?

Thanks,
Mingyuan

[bitwave]

I'm on the 3.12.0 branch and make buildroot fails with an _STAT_VER not declared error when building host-fakeroot.

[xiangmy]

Hi @bitwave

It seems 3.12.0 doesn't contain this fix. You can try master/3.13.0.

Thanks,
Mingyuan

[bitwave]

I tried 3.13.0, now getting this:

make -C BrotliCompress
make[3]: Entering directory '/home/bitwave/dev/qemu-optee/edk2/BaseTools/Source/C/BrotliCompress'
gcc  -c -I ./brotli/c/include -I .. -I ../Include/Common -I ../Include/ -I ../Include/IndustryStandard -I ../Common/ -I .. -I . -I ../Include/X64/ -MD -fshort-wchar -fno-strict-aliasing -fwrapv -fno-delete-null-pointer-checks -Wall -Werror -Wno-deprecated-declarations -Wno-stringop-truncation -Wno-restrict -Wno-unused-result -nostdlib -g -O2  brotli/c/dec/decode.c -o brotli/c/dec/decode.o
brotli/c/dec/decode.c:2033:41: error: argument 2 of type 'const uint8_t *' {aka 'const unsigned char *'} declared as a pointer [-Werror=vla-parameter]
 2033 |     size_t encoded_size, const uint8_t* encoded_buffer, size_t* decoded_size,
      |                          ~~~~~~~~~~~~~~~^~~~~~~~~~~~~~
In file included from brotli/c/dec/decode.c:7:
./brotli/c/include/brotli/decode.h:204:19: note: previously declared as a variable length array 'const uint8_t[*decoded_size]' {aka 'const unsigned char[*decoded_size]'}
  204 |     const uint8_t encoded_buffer[BROTLI_ARRAY_PARAM(encoded_size)],
      |     ~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
brotli/c/dec/decode.c:2034:14: error: argument 4 of type 'uint8_t *' {aka 'unsigned char *'} declared as a pointer [-Werror=vla-parameter]
 2034 |     uint8_t* decoded_buffer) {
      |     ~~~~~~~~~^~~~~~~~~~~~~~
In file included from brotli/c/dec/decode.c:7:
./brotli/c/include/brotli/decode.h:206:13: note: previously declared as a variable length array 'uint8_t[encoded_size]' {aka 'unsigned char[encoded_size]'}
  206 |     uint8_t decoded_buffer[BROTLI_ARRAY_PARAM(*decoded_size)]);
      |     ~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
make[3]: *** [../Makefiles/footer.makefile:21: brotli/c/dec/decode.o] Error 1
make[3]: Leaving directory '/home/bitwave/dev/qemu-optee/edk2/BaseTools/Source/C/BrotliCompress'
make[2]: *** [GNUmakefile:76: BrotliCompress] Error 2
make[2]: Leaving directory '/home/bitwave/dev/qemu-optee/edk2/BaseTools/Source/C'
make[1]: *** [GNUmakefile:19: Source/C] Error 2
make[1]: Leaving directory '/home/bitwave/dev/qemu-optee/edk2/BaseTools'
make: *** [common.mk:350: edk2-common] Error 2