[bug] Conan 2.X generates invalid transitive dependency info when building project with editable packages

[Artalus]

Describe the bug

(not sure if this is actually a bug, might be my misunderstanding of the dependency model, but things worked just fine in 1.62)
_{(also this could reeeallly benefit from a better title...)}

We have a monorepo with quite a bunch (40+) libraries and executables under the same roof. We use Conan to work with them like this, essentially:

$ for P in project1, project2, ... : conan editable add ./$P

# in ./BigProject/conanfile.py:
name = "BigProject"
version = "editable"
self.requires("AnotherProject/editable")

# in ./BigProject/CMakeLists.txt:
add_subdirectory(../AnotherProject)
find_package(Eigen3 REQUIRED)
target_link_libraries(BigProject Eigen3::Eigen3 AnotherProject ...)

$ cd BigProject/build/
$ conan install ..
$ cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=`pwd`/Conan/conan_toolchain.cmake
$ cmake --build .

All projects use the same path for generators directory; this is very likely suboptimal, but allows conan install to generate aaalll the package-Config.cmakes under one roof, and so the add_subdirectory(local) + find_package(thirdparty) approach works seamlessly. In Conan 1, that is.

With Conan 2 (tested with 2.10.2 in work repo and 2.11 in example repo) things get worse. With the dependency chain like BigProject -> Subproject -> Wrappers -> 3rdparty I now get cases where I can easily build Wrappers and Subproject on their own, but building the exact same targets as part of the BigProject fails, because the include directories for targets in 3rdparty from Conan are not set.

---

How to reproduce it

I made a repro repo here for convenience - but the issue can be reproduced without any CMake, only conanfiles are needed:

Conanfiles...

• Start with a library that provides some very common functionality and maybe wrappers around widely used libraries (like rttr in this case):

# Wrappers/conanfile.py
from conan import ConanFile
class C(ConanFile):
    name = "Wrappers"
    def requirements(self):
        self.requires("rttr/0.9.6", transitive_headers=True)
    python_requires = "pyreq/editable"
    python_requires_extend = "pyreq.PkgBase"

• Then there is a library that implements some standalone things using those wrappers:

# Subproject/conanfile.py
from conan import ConanFile
class C(ConanFile):
    name = "Subproject"
    requires = ("Wrappers/editable",)
    python_requires = "pyreq/editable"
    python_requires_extend = "pyreq.PkgBase"

• Finally a more-or-less final product utilizing those things:

# BigProject/conanfile.py
from conan import ConanFile
class C(ConanFile):
    name = "BigProject"
    requires = ("Subproject/editable",)
    python_requires = "pyreq/editable"
    python_requires_extend = "pyreq.PkgBase"

• And lastly the deduplication of Conanfile boilerplate:

# pyreq.py
import os
from conan import ConanFile, conan_version
class PkgBase(object):
    version = "editable"
    generators = ("CMakeToolchain", "CMakeDeps")
    settings = ("os", "compiler", "build_type", "arch")
    options = {
        "shared": [False, True],
    }
    default_options = {
        "shared": False,
    }

    def layout(self):
        self.folders.build = os.getcwd()
        self.folders.generators = f"{self.folders.build}/Conan/"
        self.folders.source = "."

    def init(self):
        # Conan does not inherit options by default, so docs and Slack suggest this instead
        base = self.python_requires["pyreq"].module.PkgBase
        if conan_version.major == 1:
            self.options = {**base.options, **self.options}
            self.default_options = {**base.default_options, **self.default_options}
        else:
            self.options.update(base.options, base.default_options)

class C(ConanFile):
    name = "pyreq"
    version = "editable"

It is crucial that the packages have a shared option, I could not reproduce the issue outside of our monorepo without it. I assume this relates somehow to how Conan deduces package types (shared/static/headeronly/executable) based on the options prescence or something.
Here I do for x in pyreq Wrappers Subproject BigProject ; do conan editable add ./$x ; done to initialize the "workspace", and then install the dependencies for BigProject:

mkdir BigProject/build
cd BigProject/build
conan install ..

No need to bother with CMake and sources here, as already after conan install I can see the problem. The generated BigProject/build/Conan/rttr-release-x86_64-data.cmake file has this:

$ grep 'set(rttr_' Conan/rttr-release-x86_64-data.cmake

set(rttr_PACKAGE_FOLDER_RELEASE "/tmp/c/.conan2/p/b/rttrb050f2053cedf/p")
set(rttr_INCLUDE_DIRS_RELEASE )
...
set(rttr_LIB_DIRS_RELEASE "${rttr_PACKAGE_FOLDER_RELEASE}/lib")
...
set(rttr_RTTR_Core_Lib_INCLUDE_DIRS_RELEASE )
set(rttr_RTTR_Core_Lib_LIB_DIRS_RELEASE "${rttr_PACKAGE_FOLDER_RELEASE}/lib")

, so the rttr target created by Conan for BigProject is missing the include directories.
If instead of Parsing I repeat the same commands for either Subproject or Wrappers, the -data.cmake file is generated correctly:

$ grep 'set(rttr_' Conan/rttr-release-x86_64-data.cmake

set(rttr_PACKAGE_FOLDER_RELEASE "/tmp/cc/.conan2/p/b/rttr36cf223c461c8/p")
set(rttr_INCLUDE_DIRS_RELEASE "${rttr_PACKAGE_FOLDER_RELEASE}/include")
...
set(rttr_LIB_DIRS_RELEASE "${rttr_PACKAGE_FOLDER_RELEASE}/lib")
...
set(rttr_RTTR_Core_Lib_INCLUDE_DIRS_RELEASE "${rttr_PACKAGE_FOLDER_RELEASE}/include")
set(rttr_RTTR_Core_Lib_LIB_DIRS_RELEASE "${rttr_PACKAGE_FOLDER_RELEASE}/lib")

---

We probably do a few things weirdly/incorrectly here; I, for starters, am kinda sus about all the packages using the same self.folders.build = os.getcwd() - and therefore overwriting files of each other. I gladly accept any hints on how to improve this setup, but I have been stuck in the groundhog day of "migrate to Conan 2.0" for about a year now, and for now would very much prefer things to Just Work™ without any huge refactorings and breaking compabitility with Conan-1.

[memsharded]

Hi @Artalus

Thanks for your report.

I am having a look to your code, the project doesn't seem usable in Conan 2, because it contains package names in uppercase, which are not allowed in Conan 2 and will fail

Without trying yet, it seems the main issue is that layout() is not defining any self.cpp.source.includedirs directory. Note the default cmake_layout() already defines it internally (self.cpp.source.includedirs = ["include"]), but as you are not using it, you need to add something like that explicitly.

Actually, it will need more information, that is the first part. So self.cpp.build.libdirs will most likely be necessary.

[Artalus]

I am having a look to your code, the project doesn't seem usable in Conan 2, because it contains package names in uppercase, which are not allowed in Conan 2 and will fail

It works just fine for me locally right now with 2.11.0, and what few projects in our monorepo currently work with 2.10.2, do not seem to complain about names being UpperCase either. 🤔
Is this something that is supposed to break soon across whole Conan? I thought only the uppercase names in CCI got deprecated a long time ago...

it seems the main issue is that layout() is not defining any self.cpp.source.includedirs directory

That would make sense with the headers provided by our packages themselves - but the problem is with headers coming from rttr, which does have it in the recipe.
Still, I tried adding cmake_layout(self) call right at the start of def layout(self):, but nothing changed.

[memsharded]

It works just fine for me locally right now with 2.11.0, and what few projects in our monorepo currently work with 2.10.2, do not seem to complain about names being UpperCase either.

It works only in editable mode, because packages are not created. But the first conan export or conan create done for those packages it will raise an error.

Is this something that is supposed to break soon across whole Conan? I thought only the uppercase names in CCI got deprecated a long time ago...

No, it was this way from the very early 2.0 alpha and beta releases, even discussed in the Conan tribe IIRC, and it has been raising the error since the first 2.0 alpha versions like 4 years ago 😅. It solves some of the long standing issues of case-insensitive filesystems and also reduce possible typo-squatting attacks risks.

That would make sense with the headers provided by our packages themselves - but the problem is with headers coming from rttr, which does have it in the recipe.

Ok, yes, I see what you mean know. As the rttr is not in editable mode, it shouldnt be the issue, let me check.

[memsharded]

The dependency chain is:

3BigProject->2Subproject->1Wrappers->rttr

Only 1Wrappers propagate the headers of rttr down the graph. But 2Subproject is not propagating the headers of 1Wrappers, which is correct, as the 2Subproject.hpp contains only #pragma once, and do not include any public header of 1Wrappers and that implies that rttr doesn't need to be propagated either.

So this wouldn't be a bug, but totally expected. With the current dependency definition, the rttr headers shouldn't be visible by the 3BigProject package.

[Artalus]

It works only in editable mode, because packages are not created. But the first conan export or conan create done for those packages it will raise an error.

Ah, indeed! It did fail that way, and I now vaguely remember seeing similar message when first trying Conan 2 about a year ago.
Is the "status quo" supposed to stay this way, or is this essentially an oversight from the dev team, and it might break with any new release when some of the case-sensitive functions gets called inside the editable call stack? As in, should we focus on remaining our 40+ projects right away, or are free to keep the legacy until we decide to use conan create instead of using editable?

... With the current dependency definition, the rttr headers shouldn't be visible by the 3BigProject package.

Fully agree that they shouldn't be added to the compiler input while compiling 3BigProject sources themselves. And if that was the need, then the 3BigProject recipe would be responsible to do self.requires('rttr').
But in our setup, 3BigProject rebuilds 1Wrappers from scratch in order to use it - all as part of a single build tree in CMake. So the set of -I<path>s for compiling 1Wrappers is different based on whether it is being built as 1) standalone project or direct dependency, or as 2) transitive dependency for another project.

Is there anything to be done to support this case, aside from throwing transitive_headers=True, transitive_libs=True left and right?

As mentioned before, I anticipate there might be better ways at project organization than we currently use. I skimmed over the new tutorial but did not immediately see anything useful for our case.
I also noticed there is "Declaring the layout when we have multiple subprojects", but it does not look like an exact solution either...

[memsharded]

Ah, indeed! It did fail that way, and I now vaguely remember seeing similar message when first trying Conan 2 about a year ago.
Is the "status quo" supposed to stay this way, or is this essentially an oversight from the dev team, and it might break with any new release when some of the case-sensitive functions gets called inside the editable call stack? As in, should we focus on remaining our 40+ projects right away, or are free to keep the legacy until we decide to use conan create instead of using editable?

It is not an oversight, but mostly avoiding extra costly checks, trying to keep good performance. It means that editable packages can have whatever name, but that is extremely unusual to have packages only in editable mode that are never created.

It is not a big deal, the moment you start creating the packages, it will error and you will need to rename all the packages. As everything will be local, nothing really published to Conan remotes or anything like that, it is not that you will break anyone depending on those packages with uppercase, because the packages were never uploaded.

Fully agree that they shouldn't be added to the compiler input while compiling 3BigProject sources themselves. And if that was the need, then the 3BigProject recipe would be responsible to do self.requires('rttr').

They shouldn't be visible at all, as that promotes bad practices, allows developers to accidentally break encapsulation, etc. If the headers of a dependency are not propagated through the public headers of a package, they are an internal implementation detail that should never be accidentally exposed to consumers. In this case, the 1Wrappers package is an internal implementation detail of 2Subproject, so 3Bigproject shouldn't be allowed to accidentally #include headers from 1Wrappers or attr at all. If the 3Bigproject wants to do that, it has to declare its own direct self.requires to the thing it wants to #include in the C++ code.

But in our setup, 3BigProject rebuilds 1Wrappers from scratch in order to use it - all as part of a single build tree in CMake. So the set of -Is for compiling 1Wrappers is different based on whether it is being built as 1) standalone project or direct dependency, or as 2) transitive dependency for another project.

You might have some meta-project that contains the logic, adds the necessary include folders, etc, but that doesn't belong to Conan (at the moment, we are working on the workspace feature, see https://docs.conan.io/2/incubating.html). What Conan knows about is the find_package() to find other packages, but find_package() doesn't work within the same build because it brings CMake imported targets. The find_package() shouldn't propagate the headers if they aren't transitively propagated through the public headers, because if it does that, then incurs in the issues above (breaking encapsulation, etc).

Is there anything to be done to support this case, aside from throwing transitive_headers=True, transitive_libs=True left and right?

Indeed, transitive_libs=True should be quite rarely used, and if the headers are not really exposed, then transitive_headers=True shouldn't be used. Still, for Conan, the 3BigProject Conan package should never have visibility over the 1Wrappers or attr headers, as they are an internal implementation detail. So the logic to be able to do that sounds that it belongs to the "meta-project" that builds 1Wrappers as part of a single build tree. In other words, the CMakeLists.txt that implements the single build tree cannot be exactly the same as the code that builds from packages via find_package(). (This is kind of a limitation of CMake, we have been trying to achieve this in different ways, but failed so far, we keep exploring alternatives for the Workspaces feature, but it is a challenge). So maybe having some conditionals in the CMakeLists.txt that differentiate the scenario, and for the single build tree scenario, it would do the necessary include_directories(), target_include_directories(), or whatever other CMake way of finding 1Wrappers files to do their build, and this code will not execute when 3BigProject is being built as a regular independent package.

[Artalus]

They shouldn't be visible at all, as that promotes bad practices, allows developers to accidentally break encapsulation ... 1Wrappers package is an internal implementation detail of 2Subproject, so 3Bigproject shouldn't be allowed to accidentally #include headers from 1Wrappers

I agree! And all this is being addressed on CMake side with proper PRIVATE & PUBLIC dependencies between the targets.

But the key difference between Conan-1 and Conan-2 behavior (and the main issue for me) is that Conan-2 now imposes additional limitations above the CMake model. It is not only preventing the accidental #include, it straight up prevents the discovery of said include directories altogether. Even when they are righteously needed to build exactly the thing that needs them.
So to speak, where Conan-1 fully offloaded this decision making to CMake part of the project, just providing the means to handle package paths - Conan-2 now has its own opinion on what package paths should be handled at all.

( I just tested the above setup with pip install conan<2 - and the rttr-release-x86_64-data.cmake files generated between build folders are identical completely to the same sha1sum)

---

The meta-project approach sounds awfully vague and "theoretical".
The Workspaces docs clearly state this feature is for testing only, it cannot be used in production.
I wouldn't be making a drama out of this, but it is one more friction point in the long list of friction points where things Just Worked™ in Conan-1 - yet with migration to Conan-2 are now broken with no seemingly clear way to repair.

I remember once reading about a project where folks were ExternalProject_Add()ing the very same CMakeLists project to build parts of it twice or thrice with different configurations - I guess I will try out something along these lines...

[memsharded]

I agree! And all this is being addressed on CMake side with proper PRIVATE & PUBLIC dependencies between the targets.

But Conan packages do not necessarily use CMake. So whatever PRIVATE/PUBLIC definition a package contains in its CMakeLists.txt is not automatically propagated downstream, because that is an internal implementation detail of the build system. The recipe needs to define the propagation behavior so it is universal for other consumers and build systems.

But the key difference between Conan-1 and Conan-2 behavior (and the main issue for me) is that Conan-2 now imposes additional limitations above the CMake model. It is not only preventing the accidental #include, it straight up prevents the discovery of said include directories altogether. Even when they are righteously needed to build exactly the thing that needs them.

Not sure what you mean. The Conan model is actually more flexible/expressive than the CMake model, as it has larger control over the headers and libs propagation. Only very recently CMake added things like LINK_ONLY that gets closer to the Conan model (consider Conan 2 initial alpha versions were 4 years ago)

So to speak, where Conan-1 fully offloaded this decision making to CMake part of the project, just providing the means to handle package paths - Conan-2 now has its own opinion on what package paths should be handled at all.

Because not everything is CMake, there are many users not using CMake or not using CMake only. And what seemed to work in Conan 1 was doing many times the wrong thing. For example it was massively overlinking by default, because it had the wrong propagation model of libraries.
The headers visibility model was also causing issues to many users at scale, so this was a common pain reported by users, that the default propagation of headers of any dependency in the dependency graph was just making the headers visible by the consumers, no matter what level of encapsulation were there, like you could have an executable depending on a shared library, that depends on another shared library, that depends on a static library that depends on another static library, all of them as internal implementation details, and yet the final static library headers were exposed, visible and usable to the final executable developers. This was plainly bad SW engineering practice.

I understand that it could be a bit more convenient in Conan 1, but Conan 2 is by far way more correct in the behaviors and SW engineering practices. The tradeoff is that when a package is transitively exposing its dependencies headers, has to define transitive_headers=True, otherwise it will be considered an internal implementation detail and not exposed.

The meta-project approach sounds awfully vague and "theoretical".

This is not theoretical. If your 3BigProject CMakeLists.txt is actually also building source code from 1Wrappers, then it is not a package CMakeLists.txt anymore. It is already a "meta-project" CMakeLists.txt that builds different subprojects (packages).

What we will try to do in the Workspaces feature is to provide tools or helpers or automation around such concept. There are already many users using Conan 2 with editables, each Conan editable package will have its own CMakeLists.txt for the specific package, and they have a root CMakeLists.txt that gathers all of them together, typically with some logic as the one I described above, that allow them to do different things when having a single build tree or when building them as different packages. This is not theoretical, we have seen this successfully in production.

Yes, the Workspaces feature is very new, and given its scope and complexity, it will mean there will be some instability. But without users feedback it is impossible to stabilize it, so this is why we created the new "incubating" section. But this is a chicken and egg problem, if nobody gives it a try, experiment with it and give feedback, it will be impossible to stabilize it, we are just trying our best to get it out there and be useful for users.

[Artalus]

The Conan model is actually more flexible/expressive than the CMake model, as it has larger control over the headers and libs propagation.

Then how do I utilize this control to solve my current situation?

If your 3BigProject CMakeLists.txt is actually also building source code from 1Wrappers, then it is not a package CMakeLists.txt anymore.

It never was, is not intended to be right now, and might never be at all. Neither of those 3 CMakeLists are a package to be conan uploaded to a remote. If anything, the whole setup is more akin to Rust, where the root Cargo.toml would have workspace.members = ["wrappers", "bigproject"], and the subproject's bigproject/Cargo.toml would have dependencies.wrappers = { path = "../wrappers" }.

It is already a "meta-project" CMakeLists.txt that builds different subprojects (packages).

Do you imply that we might try to remove all add_subdirectory() from the <whatever>/CMakeLists.txts, make their find_package() focus only on their own 3rdparty dependencies, and then provide some sort of a root CMakeLists that would add_subdirectory() different parts of monorepo as needed?

But this is a chicken and egg problem, if nobody gives it a try, experiment with it and give feedback, it will be impossible to stabilize it

And it gets kinda harder to give it a try, if one accounts for the past trauma of Conan-2 migration processes. I literally am unsure if I should touch things marked "experimental" in the docs, because I vividly remember how even things not marked as such got butchered with the update! 🙃 Still, understandable. I will check Workspaces with the example from this issue, and it goes well, add it to the list of things to experiment on monorepo.

Speaking of experiments... I guess could also try to split the folders.generators, so the the .cmake glue files for each conanfile.py go to their own directory. This should at least give Wrappers its own "proper" rttr-data.cmake with include directories not overwritten to nothingness by BigProject. Then I will have to somehow befriend different CMAKE_PREFIX_PATHs from multiple conan_toolchain.cmakes from different generate() calls - so that a find_package() inside Wrappers does not get config files generated by BigProject....
Do you have any advice on that by chance, or maybe know if it would break things even further?

[memsharded]

It never was, is not intended to be right now, and might never be at all. Neither of those 3 CMakeLists are a package to be conan uploaded to a remote. If anything, the whole setup is more akin to Rust, where the root Cargo.toml would have workspace.members = ["wrappers", "bigproject"], and the subproject's bigproject/Cargo.toml would have dependencies.wrappers = { path = "../wrappers" }

I think that is the issue. The CMakeDeps generator generates xxx-config.cmake files for the consumers of Conan packages, that can be used via find_package(). But the 3BigProject is not using find_package() to find its dependencies, but rather directly building the other subprojects.

If 3bigProject and 2SubProject are not creating Conan packages yet, maybe it would be enough to just limit the conan install to the 1Wrappers? If you do that, it will generate a rttr-config.cmake in the 1Wrappers build that it will contain the correct includedirs defined, and the propagation from there will be pure CMake as you want.

It is the challenge of both trying to create independent packages and have a single build-tree which CMake doesn't have a solution, so it is necessary to do something like:

if(SINGLE_BUILD_TREE)
    add_subdirectory(mydep)
else()
    find_project(mydep ...)
endif()
...
# If the imported target name is not the same as the project library name
if(SINGLE_BUILD_TREE)
    target_link_libraries(mytarget ... mydep)
else()
    target_link_libraries(mytarget ... mydep::mydep)
endif()

And this is more or less what users doing this are doing.

So if you are not creating Conan packages for those yet, maybe it is possible to simplify it? Not sure what is the editable use case there, the editable was always intended, also in Conan 1, as a mechanism to edit Conan packages simultaneously, but on the premise that they are basically Conan packages that are created at some point. If they are not Conan packages created ever, maybe just forget about that part?

Speaking of experiments... I guess could also try to split the folders.generators, so the the .cmake glue files for each conanfile.py go to their own directory. This should at least give Wrappers its own "proper" rttr-data.cmake with include directories not overwritten to nothingness by BigProject.

yes, this is the idea I was suggesting above. The 1Wrappers already have its own space, its own layout, etc. It will get a different rttr-config.cmake file than the consumer 3BigProject is getting, because they have different scope.

So the thing to remove would be the:

def layout(self):
    self.folders.build = os.getcwd()

and let the 1Wrappers create its own rttr-config.cmake file in its own location that it will find. But 3BigProject shouldn't be the one creating the file, it might even drop CMakeDeps completely, because the idea is that it uses its dependencies directly from CMake, not via Conan. So the CMakeDeps for it is unnecessary, and it is the thing that is actually harming here, because due to the self.folders.build = os.getcwd() it is overwriting and destroying the previous right rttr-config.cmake file that 1Wrappers would have been generated in the first place.

As an extra hint, if you maintain the editables, and you want the 1Wrappers to correctly call its generate() method automatically (and other potential editable packages), then the conan install ... --build=editable would be the way to go, as this triggers the call of the editables dependencies generate() method.

[Artalus]

But the 3BigProject is not using find_package() to find its dependencies, but rather directly building the other subprojects. ... it might even drop CMakeDeps completely, because the idea is that it uses its dependencies directly from CMake, not via Conan

Unfortunately this was just an oversimplification for the example needs 😅
For the reproduction case from the starting post, only a 3rdparty dependency inside Wrappers mattered; but in reality there are maybe 1, at max 2 packages in the monorepo that wouldn't depend upon something external. And even these would still self.requires("catch2") for the unit tests.

It is the challenge of both trying to create independent packages and have a single build-tree which CMake doesn't have a solution, so it is necessary to do something like: if(SINGLE_BUILD_TREE)

TBH I am unsure if we ever need to switch between add_subdirectory()/find_package(). The project is essentially an SDK consisting of libraries and some tools. The only place I can think of, that would find_package(Subproject) or find_package(BigProject), - is a small set of wrappers for the ROS ecosystem, consuming the fully cmake --installed SDK - and those are outside of the current buildsystem "scope".
It might make sense to turn at least BigProject-alikes into proper Conan packages at some point in future, but I feel like this would be a huge endeavor on its own.

The main use case to keep the packages independent to some degree, is so that a dev could focus first on development of a Subproject without immediately running into issues on consuming it in BigProject. Plus it is less moutful to cd Subproject/build ; cmake --build . ; ctest rather than cd BigProject/build ; cmake --build . --target Subproject ; ctest -R 'TestSubproject*' .
This might sound as an antipattern, "but you should worry about your changes breaking the consuming projects!" - but it's kinda just the workflow the company is accustomed to; focus first on the part, then on the whole.

Not sure what is the editable use case there, the editable was always intended, also in Conan 1, as a mechanism to edit Conan packages simultaneously, but on the premise that they are basically Conan packages that are created at some point. If they are not Conan packages created ever, maybe just forget about that part?

I must admit that I myself do not fully grasp all "why"s behind the current setup, as it was designed long before me. My understanding - editable is used so you can conan editable add all projects (essentially just once, at workspace initialization, and then once in a blue moon when someone adds a new project) - and then a single conan install . inside any project directory X would automatically gather all the deps for said X, including deps for its (editable) dependencies inside the monorepo.
We do have def build() and rudimentary def package() + def package_info() methods defined in our conanfiles - so I assume at some point the goal was to consolidate conan install && cmake && cmake --build under a single conan build or conan create command. But I doubt these were ever fully fleshed out to be used.

If there is another popular workflow to do this without editables - I will gladly research it, as editables have been a bit of a pain in our CI with shared caches (another antipattern, I know)

then the conan install ... --build=editable would be the way to go, as this triggers the call of the editables dependencies generate() method

Not sure I follow. How is this different from the default conan install .. behavior? This is what I get with the example here:

rttr/0.9.6: Already installed! (1 of 3)
Wrappers/editable: Rewriting files of editable package 'Wrappers' at '/tmp/c/3BigProject/build/Conan'
Wrappers/editable: Writing generators to /tmp/c/3BigProject/build/Conan
...
Subproject/editable: Rewriting files of editable package 'Subproject' at '/tmp/c/3BigProject/build/Conan'
Subproject/editable: Writing generators to /tmp/c/3BigProject/build/Conan
...
======== Finalizing install (deploy, generators) ========
conanfile.py (BigProject/editable): Writing generators to /tmp/c/3BigProject/build/Conan

, so the generate() seems to be already working fine 🤔