How to gather up object files by type

[Ed-CMI]

What is your question?

Hi I have a question regarding object file used during linking.
There was a linking error caused by too many libraries being opened when trying to link a list of library files. One option to correct this issue is to gather up all the .o file by component type and link them into a single library file. In conan, there is cpp_info.objects that is used to gather object files for linking. The one problem with this attribute is that I don’t have any way of determining the component type. I need to gather all objects files that pertain to a specific set of components and link all those object files into a single library. What is the best way to do this in conan?

I found self.cpp_info.components["ssl-objs"].objects in the conan documentation but not sure this should be used for this type is scenario.

I also look at set_property but not sure if this is also the correct functionality to use.

I have used self.cpp_info.resdirs to gather up a2l file based off a particular a2l file format. Should this be used to gather up .o files based off component type?

Have you read the CONTRIBUTING guide?

• I've read the CONTRIBUTING guide

[memsharded]

Hi @Ed-CMI

Thanks for your question.

by component type

What do you mean by component type? Components don't have a type defined, the only type defined is the package_type, at the package level.

[Ed-CMI]

Our program have application , bsw , gtm, and other type components. I have create a CompClass variable for all recipe files that define the component types as SWC, HW_SPECIFIC_INTERFACE, TOOLS, GTM, BUILD_ENVIRONMENT, BSW_PROJECT, etc. I want to gather up these objects files by this CompClass type so I can create one library file used during linking. Our programs have over 950 different components and during linking windows can't open this many libraries. The following error occurs during linking.

LD Octane_Gold/target/Octane_Gold.elftc F004: cannot open "C:\Users\bq824.conan2\p\b\rxdd1df9688d2e14e3\p\SWC\RxDD1.a" (Too many open files)

[memsharded]

I am still not sure what are those components. In Conan, one package can contain multiple components if it defines in its package_info() something like:

def package_info(self):
     self.cpp_info.components["component1"].libs = ["libcomp1"]
     self.cpp_info.components["component2"].libs = ["libcomp2"]
     ...

So one package can contain multiple components, but this doesn't seem connected to what you are explaining above, which seems more related to packages, not components.

[Ed-CMI]

I don't want one package with multiple components. I need to gather up the object files contained in multiple component and combine then into a single library based off a certain set of components. For ex, I need to gather up all object file contain in components that have a CompClass of SWC and create a single library out of all these object files. Is there a way we can have a short meeting to discuss to help clarify

[memsharded]

I need to gather up the object files contained in multiple component

Then you mean:

"I need to gather up the object files contained in multiple packages"?

It is very confusing, as said, components have a very specific meaning in Conan, it is important to understand what you mean.
Does it also mean that you have a certain conanfile.py recipe, that has multiple self.requires() to some dependencies, and you want to collect all the object files from all dependencies packages?

[Ed-CMI]

No, This has nothing to do with the requires field. In our builds there is Component A, Component B, Component C, and Component D. Each of these components has one or more object files. All of these components are considered to be SWC component type. Components E, Components, F, Component G will have a different component type called BSW. I need to be able to gather up the object files from all the component types SWC and create a single library file out of them . I also need to gather up all the object files from component type BSW and create a different library file from them. This is need to prevent the linking issue LD Octane_Gold/target/Octane_Gold.elftc F004: cannot open "C:\Users\bq824.conan2\p\b\rxdd1df9688d2e14e3\p\SWC\RxDD1.a" (Too many open files)

[memsharded]

No, This has nothing to do with the requires field. In our builds there is Component A, Component B, Component C, and Component D. Each of these components has one or more object files. All of these components are considered to be SWC component type. Components E, Components, F, Component G will have a different component type called BSW. I need to be able to gather up the object files from all the component types SWC and create a single library file out of them .

But what is Component A? I don't know what that means. If this is a Conan question, we need to formulate the question in terms of Conan things: Conan packages, Conan dependencies, Conan recipes, etc. The only component that exists in Conan is the self.cpp_info.components inside a Conan package, but you are telling me that this is not what you want. To illustrate it should be something like this:

• I have a package recipe for mypkg_a, and in its conanfile.py it contains a self.requires("mypkg_b/1.0") and self.requires("mypkg_c/1.0")
• The mypkg_b/1.0 package recipe contains 3 components, defined as self.cpp_info.components["compb1"].libs = ... in its package_info() method, and the mypkg_c/1.0 package recipe defines 2 components as self.cpp_info.components["compc1"]...
• Both mypkg_b and mypkg_c components (or not using components, whatever they are doing), contain a self.cpp_info.objects = [...] definition
• etc.

Otherwise, I am afraid that I honestly don't understand what you mean, I am a bit lost, sorry.

[Ed-CMI]

Each component is a single recipe file. For ex, Component A is a recipe file that consist of a single components source files that are compiled into object files.

This is why I request a meeting to help explain

[memsharded]

Ok, understood, so we are talking about Conan packages, each package containing some objects.

The next thing to understand would be how do you encode the type of that package, that SWC, BSW, etc? It is encoded as an option? It is encoded in the package name itself, like mypkg_bsw/1.0? What determines that type?

[Ed-CMI]

I have created an option called CompClass. The default value for the option is set in each recipe file based off the type.

For ex in recipe file for BSW_TEMPLATES
options = {
"CompClass": ['SWC', 'INFINEON_MCAL', 'VECTOR_SIP', 'SUPPLIER', 'HW_SPECIFIC_INTERFACES', 'MODEL_TARGET', 'GTM', 'BSW_PROJECT', 'TOOL', 'TESTSTUBS', 'BSW_TEMPLATES', 'IMPORT', 'INTEGRATION', 'BUILD_ENVIRONMENT'],

default_options = {
"CompClass": "BSW_TEMPLATES",

[Ed-CMI]

HI
Were you able to review my last response

[memsharded]

Then, the way to collect objects would be something like:

def generate(self):
    all_objects = []
    for dep in self.dependencies.host.values():
        if dep.options.CompClass == "BSW":
            all_objects.extend(os.path.join(dep.package_folder, obj) for obj in dep.cpp_info.objects)

    ....

Something like that (the code is not tested), will collect all the abs paths for all objects of all dependencies that match the BSW CompClass option.

[Ed-CMI]

Thanks. Will give it a try.

[Ed-CMI]

Hi

I was able to get the code working with a few modifications, but I am running into the following error and have a question. The error below is being generated because our header only recipe files do not define the CompClass option in the recipe file because it is not needed. Is there a way to tell if the CompClass attribute exist for a particular recipe file before checking the value of the CompClass to prevent this error?

while calling 'GenGNUProjectMk', line 173
if dep.options.CompClass == "SWC":
ConanException: option 'CompClass' doesn't exist
Possible options are []

[memsharded]

Then, instead of using self.options.MyOption you might want to protect it with a try-except or use self.options.get_safe("MyOption"): https://docs.conan.io/2/reference/conanfile/attributes.html#options

[Ed-CMI]

The get_safe work perfectly. This item can be closed. Thx for the help.

[memsharded]

Happy to help!