Re: [boost] Fwd: Installing CMake configuration files
Steven Watanabe wrote:
Just a couple comments: ... * I'm not fond of the ROOT parameter for boost-install.boost-install. * Dependencies can be referenced with /boost/$(deps)//[stage|install] * You can find the full path to the boost-install.jam with [ path.root [ modules.binding $(__name__) ] [ path.pwd ] ] This avoids hard-coding the location of boost_install relative to the superproject.
Actually, [ modules.binding $(__name__) ] seems to return an absolute path, so path.root/path.pwd may not be needed here? boostcpp.jam does BOOST_ROOT = [ modules.binding $(__name__) ] ; BOOST_ROOT = $(BOOST_ROOT:D) ; which kind of implies that it relies on getting the absolute path.
* That just leaves the stage directory, which is tied to the project root, and should perhaps be calculated in Jamroot instead of in boost_install.
The stage directory needs to be retrieved from boostcpp, but boostcpp just sets it to "stage", which only works from top-level. It should probably use $(BOOST_ROOT)/stage, and then other modules can peek for it. There are other things that are needlessly replicated - install-header-subdir, install-default-prefix. All of these need to be peekable from a canonical location.
* This seems to handle --layout=versioned, but I'm not clear on how well it handles --layout=system where we expect to use a single compiled library for all configurations.
It worked when I tried it. Note that you can have two libraries with --layout=system, not just one; libfoo.a and foo.so.
* Are the error messages that you get from CMake when there isn't a configuration that matches the request intelligible?
Not very. There's Boost_DEBUG, which tells you why something was skipped, but ideally, it needs to detect the case when no variant matched, and _then_ tell you why things were skipped without your having to set Boost_DEBUG beforehand. I haven't gotten that far yet; usability can be improved once we have it working.
It looks like the target will exist but will silently do nothing?
Yes, you get a link error.
What if something goes wrong and there are multiple matches?
I haven't encountered this so far :-) although it's possible in principle because at the moment I ignore some "non salient" properties.
* If generate-cmake-config takes the library as a source, then it's possible to derive both the library dependencies and the exact library name from the library target. The dependencies can be found at the virtual target level in generate-cmake-config, and the exact file name will appear as the source in generate-cmake-config-. Doing it this way also has the side effect that the CMake configuration won't be installed if the library fails to build, which may be a good thing.
These may take me a while to figure out. :-)
AMDG On 10/19/2018 04:52 PM, Peter Dimov via Boost wrote:
Steven Watanabe wrote:
Just a couple comments: ... * I'm not fond of the ROOT parameter for boost-install.boost-install. * Dependencies can be referenced with /boost/$(deps)//[stage|install] * You can find the full path to the boost-install.jam with [ path.root [ modules.binding $(__name__) ] [ path.pwd ] ] This avoids hard-coding the location of boost_install relative to the superproject.
Actually, [ modules.binding $(__name__) ] seems to return an absolute path, so path.root/path.pwd may not be needed here? boostcpp.jam does
I looked it up, and you are correct: modules.binding always returns an absolute path, provided that the module is loaded with `import` and not the lower level `modules.load`.
BOOST_ROOT = [ modules.binding $(__name__) ] ; BOOST_ROOT = $(BOOST_ROOT:D) ;
which kind of implies that it relies on getting the absolute path.
<snip> * If generate-cmake-config takes the library as a source, then it's possible to derive both the library dependencies and the exact library name from the library target. The dependencies can be found at the virtual target level in generate-cmake-config, and the exact file name will appear as the source in generate-cmake-config-. Doing it this way also has the side effect that the CMake configuration won't be installed if the library fails to build, which may be a good thing.
These may take me a while to figure out. :-)
Getting the library name this way isn't too bad. Just pass it through all the generate/action logic and then in `generate-cmake-variant-` say: local fname = $(sources:BS) ; The logic for choosing static/shared/import lib is really annoying actually. The test you have ([ os.name ] = NT) is definitely wrong, as it should be based on the target-os. Also cygwin needs import libs (IIRC, cygwin can link directly to a dll, but there are a few corner cases that don't work right.). # Process virtual targets containing a LIB # target. IMPORT_LIBs take priority if both a # SHARED_LIB and an IMPORT_LIB are present. # Note: There might be other random files like *.pdb, # which are ignored. Call this in generate-cmake-variant # to filter the sources to pass to the action. local rule .choose-lib-target ( sources * ) { local result ; for local t in $(sources) { if [ type.is-derived [ $(t).type ] IMPORT_LIB ] { return $(t) ; } else if [ type.is-derived [ $(t).type ] LIB ] { result = $(t) ; } } return $(result) ; } Automating the dependencies is harder, but I would also give it lower priority (since you've already scripted it, it's less of a problem when it gets out of sync). I think that to make it work, you'll need to put it into *-variant.cmake file, since it would need to be extracted from a specific library build. For static libraries, you can just look at the <library> property, since the dependencies will be propagated upwards as usage-requirements. In Christ, Steven Watanabe
Steven Watanabe wrote:
Getting the library name this way isn't too bad.
It may even be called trivial, as I already receive it in $(sources) in generate-cmake-variant and can just take $(src).name. However, I have to pick the right source for that, as I want the import library on Windows (but only on Windows) and for a DLL I get three sources from MSVC (.dll, .lib, .pdb) two for Cygwin g++ (.dll.a, .dll), and their order doesn't even match.
Just pass it through all the generate/action logic and then in `generate-cmake-variant-` say: local fname = $(sources:BS) ;
Yeah, I haven't been able to figure this part out. How do I pass the $(sources) from generate-cmake-variant to generate-cmake-variant-'s sources? I was going to use a free feature for the fname.
The logic for choosing static/shared/import lib is really annoying actually. The test you have ([ os.name ] = NT) is definitely wrong, as it should be based on the target-os. Also cygwin needs import libs (IIRC, cygwin can link directly to a dll, but there are a few corner cases that don't work right.).
[ os.name = NT ] may be wrong but it does handle Cygwin the right way. :-)
# Process virtual targets containing a LIB # target. IMPORT_LIBs take priority if both a # SHARED_LIB and an IMPORT_LIB are present. # Note: There might be other random files like *.pdb, # which are ignored. Call this in generate-cmake-variant # to filter the sources to pass to the action. local rule .choose-lib-target ( sources * ) { local result ; for local t in $(sources) { if [ type.is-derived [ $(t).type ] IMPORT_LIB ] { return $(t) ; } else if [ type.is-derived [ $(t).type ] LIB ] { result = $(t) ; } } return $(result) ; }
Ah, you've answered my question from above. Thanks. :-)
Automating the dependencies is harder,
That's what I thought. :-)
AMDG On 10/19/2018 06:29 PM, Peter Dimov via Boost wrote:
Steven Watanabe wrote: <snip> [ os.name = NT ] may be wrong but it does handle Cygwin the right way. :-)
That only works because you're using a windows build of b2. If you use cygwin to build b2 itself, then it will fail.
Automating the dependencies is harder,
That's what I thought.
I'll take a stab at it over the weekend. In Christ, Steven Watanabe
AMDG On 10/19/2018 06:29 PM, Peter Dimov via Boost wrote:
Steven Watanabe wrote:
Automating the dependencies is harder,
That's what I thought. :-)
It turns out to be easier than I originally anticipated. Before we go there, however, I'd like to discuss whether it's actually the right thing to do. In particular, I don't want to go down a path that will get in the way of modular installation for headers. # Ideally it should look something like this: link-directory headers : ../include : requirements <location>$(BOOST_ROOT) : usage-requirements <include>$(BOOST_ROOT) ; alias boost_any : headers /boost/config//boost_config ; boost-install boost_any ; A few issues that are going to come up: - The stage target needs a hard dependency on the headers. The default behavior for link-directory is to let the #include scanner trace the headers to stage. - The install target will need to introspect the headers target to find the files to copy. - For the generated cmake files, distinguishing between the headers of the current library (which should be represented directly) and the headers of some other library (which should be represented as a dependency) is a bit of a pain. Comparing the project of a headers target with the project of the current install target will work I think. - The dependencies found will probably end up being the transitive closure of all dependencies unless we take extra measures to reduce them. This isn't technically wrong, but may result in excessively long lists. - We have to assume that the library targets are defined in the same project that has the stage/install targets. For instance, putting the actual library targets in a subproject and making an alias in build/Jamfile will break badly. This is probably the biggest issue I have, as I think that implicit behavior based on targets being in the same project is unintuitive and potentially surprising. In Christ, Steven Watanabe
Steven Watanabe wrote:
Before we go there, however, I'd like to discuss whether it's actually the right thing to do. In particular, I don't want to go down a path that will get in the way of modular installation for headers.
# Ideally it should look something like this: link-directory headers : ../include : requirements <location>$(BOOST_ROOT) : usage-requirements <include>$(BOOST_ROOT) ; alias boost_any : headers /boost/config//boost_config ; boost-install boost_any ;
I'm not sure we want that. A modular header-only library should have usage-requirements <include>../include, not <include>$(BOOST-ROOT) (and consequently should not link headers when used as a dependency.) There are two cases: the library using the header-only library is "modularity-aware" and has enumerated its header-only dependencies as <library>/boost/any, or isn't and hasn't. If it is and has, it doesn't need the links in boost/. If it isn't and hasn't, `any` having <include>$(BOOST-ROOT) in its usage-requirements doesn't matter as it won't be seen. In this case, the dependency on /boost/headers that we inject will both link the headers and give the <include>$(BOOST-ROOT). (I was thinking that /boost/headers can become a regular alias to libs/headers/build, whose implicit `stage` will then link the headers. But it will work the same either way.) In case it isn't clear, the above assumes incremental transition to the modular layout, where some libraries are modular, others aren't yet. So we have four types of libraries: 1. Old-school header-only libraries, without build/Jamfile. These all form the metalibrary /boost/headers. When staged, /boost/headers links everything into boost/, as before. When installed, it copies all headers that aren't otherwise covered below, into the install location. 2. Newfangled header-only libraries, with build/Jamfile. When staged, these do nothing. When installed, they install their headers. 3. Old-school buildable libraries. These have a dependency on /boost/headers. 4. Newfangled buildable libraries. These enumerate all of their dependencies in their Jamfile, including type 2 header-only ones. If they use no type 1 libraries, they don't need a dependency on /boost/headers, and don't need the header links.
AMDG On 10/20/2018 12:19 PM, Peter Dimov via Boost wrote:
Steven Watanabe wrote:
Before we go there, however, I'd like to discuss whether it's actually the right thing to do. In particular, I don't want to go down a path that will get in the way of modular installation for headers.
# Ideally it should look something like this: link-directory headers : ../include : requirements <location>$(BOOST_ROOT) : usage-requirements <include>$(BOOST_ROOT) ; alias boost_any : headers /boost/config//boost_config ; boost-install boost_any ;
I'm not sure we want that. A modular header-only library should have usage-requirements <include>../include, not <include>$(BOOST-ROOT) (and consequently should not link headers when used as a dependency.)
<snip> 3. Old-school buildable libraries. These have a dependency on /boost/headers.
4. Newfangled buildable libraries. These enumerate all of their dependencies in their Jamfile, including type 2 header-only ones. If they use no type 1 libraries, they don't need a dependency on /boost/headers, and don't need the header links.
I mostly agree with you, except for one issue: This is severely complicated by the fact that the dependency on headers is currently implicit. How are we going to distinguish (3) from (4)? If we're making incremental updates, then we can't make any changes at all to the old-school libraries, so we can't rely on them telling us that they need /boost/headers. If we leave the dependency on /boost/headers alone, then type (4) libraries will still pick it up and we'll lose any ability to test that the dependencies listed are sufficient. In Christ, Steven Watanabe
3. Old-school buildable libraries. These have a dependency on /boost/headers.
4. Newfangled buildable libraries. These enumerate all of their dependencies in their Jamfile, including type 2 header-only ones. If they use no type 1 libraries, they don't need a dependency on /boost/headers, and don't need the header links.
I mostly agree with you, except for one issue: This is severely complicated by the fact that the dependency on
Steven Watanabe wrote: headers is currently implicit. How are we going to distinguish (3) from (4)? If we're making incremental updates, then we can't make any changes at all to the old-school libraries, so we can't rely on them telling us that they need /boost/headers. There aren't that many type 3 libraries, and they already have build/Jamfiles, so making the dependency explicit for type 3 is feasible. We can just add it to them. Testing might be a problem. test/Jamfile for header-only type 1 libraries needs the implicit headers too. Still doable though.
AMDG On 10/20/2018 01:17 PM, Peter Dimov via Boost wrote:
Steven Watanabe wrote:
3. Old-school buildable libraries. These have a dependency on > /boost/headers.
4. Newfangled buildable libraries. These enumerate all of their > dependencies in their Jamfile, including type 2 header-only ones. If > they use no type 1 libraries, they don't need a dependency on > /boost/headers, and don't need the header links.
I mostly agree with you, except for one issue: This is severely complicated by the fact that the dependency on headers is currently implicit. How are we going to distinguish (3) from (4)? If we're making incremental updates, then we can't make any changes at all to the old-school libraries, so we can't rely on them telling us that they need /boost/headers.
There aren't that many type 3 libraries, and they already have build/Jamfiles, so making the dependency explicit for type 3 is feasible. We can just add it to them.
Testing might be a problem. test/Jamfile for header-only type 1 libraries needs the implicit headers too. Still doable though.
It's easy enough to detect by checking for the presence of build/Jamfile. I don't see any way to poke the dependency in externally, though. If we're going to have to modify everything anyway, we might as well give up on making the changes incremental. In Christ, Steven Watanabe
On 10/20/18 3:40 PM, Steven Watanabe via Boost wrote:
It's easy enough to detect by checking for the presence of build/Jamfile. I don't see any way to poke the dependency in externally, though. If we're going to have to modify everything anyway, we might as well give up on making the changes incremental.
Not sure whether I understand all the details of what you are doing, but it sounds like the change to remove references to the "root" could be implemented incrementally, if you follow the dependency ordering. That is, if you start with the low-level Boost libraries that don't have any (Boost) dependencies (let's call the "core" for the moment), then do the next round of changes by addressing all those that only depend on "core", etc., you could incrementally eliminate the /hoost/headers requirement from all Boost libs. Stefan -- ...ich hab' noch einen Koffer in Berlin...
Steven Watanabe wrote:
If we're going to have to modify everything anyway, we might as well give up on making the changes incremental.
I don't consider all modifications equally disqualifying. Adding project : requirements <library>/boost/headers ; to all test Jamfiles is indeed a global change, but it's mechanistic and does not require any further maintenance, so it's not going to be a burden for the maintainer of the library. Adding a build/Jamfile with the dependencies, on the other hand, both requires getting the dependencies correct and keeping them correct, so it does impose an additional responsibility on the maintainer. (It also may have packaging implications downstream... and if we don't magically solve cycles, those are going to be a problem too.) The idea of being "incremental" is to allow those maintainers who would like to accept the additional responsibility to be able to do so, and to allow those who would rather not, to, well, not. This is not invalidated by our adding a line to their Jamfiles - once, without them needing to do anything about it.
AMDG On 10/20/2018 02:00 PM, Peter Dimov via Boost wrote:
<snip> The idea of being "incremental" is to allow those maintainers who would like to accept the additional responsibility to be able to do so, and to allow those who would rather not, to, well, not. This is not invalidated by our adding a line to their Jamfiles - once, without them needing to do anything about it.
That seems a bit worrisome to me, as I really don't like the idea of having a mixed-state sytem in the long term. It'll be really easy to end up with targets that use both the global /boost/headers and the library specific include paths. I can live with that as a temporary measure, but it's still ugly. In Christ, Steven Watanabe
Steven Watanabe wrote:
That seems a bit worrisome to me, as I really don't like the idea of having a mixed-state sytem in the long term. It'll be really easy to end up with targets that use both the global /boost/headers and the library specific include paths. I can live with that as a temporary measure, but it's still ugly.
This is unavoidable - in the interim or otherwise - unless we either somehow solve, or outright eliminate, cyclic dependencies. Until then, libraries that form a cycle have to remain /boost//headers.
AMDG On 10/20/2018 03:52 PM, Peter Dimov via Boost wrote:
Steven Watanabe wrote:
That seems a bit worrisome to me, as I really don't like the idea of having a mixed-state sytem in the long term. It'll be really easy to end up with targets that use both the global /boost/headers and the library specific include paths. I can live with that as a temporary measure, but it's still ugly.
This is unavoidable - in the interim or otherwise - unless we either somehow solve, or outright eliminate, cyclic dependencies. Until then, libraries that form a cycle have to remain /boost//headers.
It's not unsolvable and I think we definitely need to have a game plan for solving it before beginning this particular project. In Boost.Build, it may be as simple as saying: alias targets may be cyclic provided that the build-requests are consistent for each alias-target in the cycle. i.e. you can't have a/release -> b/release -> a/debug. This is a pain to implement, but it's definitely possible, and I'm reasonably confident that it won't result in any weird paradoxes. In Christ, Steven Watanabe
AMDG On 10/20/2018 12:19 PM, Peter Dimov via Boost wrote:
Steven Watanabe wrote:
Before we go there, however, I'd like to discuss whether it's actually the right thing to do. In particular, I don't want to go down a path that will get in the way of modular installation for headers.
# Ideally it should look something like this: link-directory headers : ../include : requirements <location>$(BOOST_ROOT) : usage-requirements <include>$(BOOST_ROOT) ; alias boost_any : headers /boost/config//boost_config ; boost-install boost_any ;
I'm not sure we want that. A modular header-only library should have usage-requirements <include>../include, not <include>$(BOOST-ROOT) (and consequently should not link headers when used as a dependency.)
I'd like to back up a step to my original point, which is: how is this going to interact with automatic dependencies for stage and install. Under this scheme the most obvious way to handle a header only library is like this: alias boost_any : /boost/config//boost_config # + other deps : usage-requirements <include>../include ; boost-install boost_any ; This isn't going to work as is, because the bare <include> is not enough to identify something that needs to be installed and tracked as a dependency in the cmake output vs. some external include path. To make it work we need to give an extra path to boost-install like this: boost-install boost_any : ../include ; Now, boost-install can keep around a global map tracking which include paths correspond to which libraries, and we can use that to sort out the header dependencies. Note: boost-install could simply assume that the headers are in ../include, but I strongly prefer to limit implicit behavior like that. Okay. I think I've convinced myself that we can derive the dependencies for libraries without painting ourselves into a corner, which is what I was worried about. In Christ, Steven Watanabe
Steven Watanabe wrote: ...
* I'm not fond of the ROOT parameter for boost-install.boost-install. ... * If generate-cmake-config takes the library as a source, then it's possible to derive [...] the exact library name from the library target.
I implemented these suggestions of yours and this resulted in significantly improved code and as a side effect fixed a bug with the handling of a relative --stagedir. Thanks.
participants (3)
-
Peter Dimov -
Stefan Seefeld -
Steven Watanabe