clang-win, (Thin)LTO this time
Starting new thread. Substantial progress has been made towards the goal of compiling Boost with Clang/LLVM (clang-cl.exe, targeting MSVC). The latest PR in this respect by Peter has achieved this. The next step is to be also able to use lld (lld-link in case of clang-cl.exe targeting MSVC) to generate the Boost-libs (static/dynamic). The main interest, IMO, is to be able to use LTO (Thin or Full), which is not possible without the use of LLD. A secondary benefit appears to be faster link times (info from the lld manual). In order to build static ThinLTO-enabled libs, it suffices to pass <compileflags>-flto=thin in the project jam file. The archiving, though, needs to be done with lld-link.exe, passing /lib as the first option (this way it will act as lib.exe, while being able to handle the llvm-byte-code [lib.exe finds the so generated .obj-files to be corrupt]). At consumption time of the libs, lld-link needs to be used for linking. Hacking clang-win.jam:156 to: toolset.flags clang-win.archive .LD $(cond) : "\"C:\\Program Files\\LLVM\\bin\\lld-link.exe\"" /lib ; achieves this goal. I would like to ask (probably Peter) to have a look into whether this can be achieved "the proper way", i.e. to programmatically set that particular line to the required value for this use-case. degski -- *“If something cannot go on forever, it will stop" - Herbert Stein*
degski wrote:
Hacking clang-win.jam:156 to:
toolset.flags clang-win.archive .LD $(cond) : "\"C:\\Program Files\\LLVM\\bin\\lld-link.exe\"" /lib ;
achieves this goal.
This shouldn't be necessary with -fuse-ld=lld... but you need to give the option in the actual command rather than as a compile flag: using clang-win : : "C:\\Program Files\\LLVM\\bin\\clang-cl.exe" "-fuse-ld=lld" : ... ; It should then autodetect the above "lld-link.exe /lib" as the archiver. Or, you could set <archiver> manually: using clang-win : : : <archiver>"\"C:\\Program Files\\LLVM\\bin\\lld-link.exe\" /lib" ... ;
On Wed, 31 Oct 2018 at 16:32, Peter Dimov via Boost
This shouldn't be necessary with -fuse-ld=lld... but you need to give the option in the actual command rather than as a compile flag:
using clang-win : : "C:\\Program Files\\LLVM\\bin\\clang-cl.exe" "-fuse-ld=lld" : ... ;
It should then autodetect the above "lld-link.exe /lib" as the archiver.
Or, you could set <archiver> manually:
using clang-win : : : <archiver>"\"C:\\Program Files\\LLVM\\bin\\lld-link.exe\" /lib" ... ;
Thank you, I am going to try [all of] that. That bjam-syntax is pretty mind-boggling. It sounds like the whole thing is already cracked [which is great!]. Thanks, Peter, thanks again! degski -- *“If something cannot go on forever, it will stop" - Herbert Stein*
On Wed, 31 Oct 2018 at 16:32, Peter Dimov via Boost
using clang-win : : "C:\\Program Files\\LLVM\\bin\\clang-cl.exe" "-fuse-ld=lld" : ... ;
I have: import option ; using clang-win : : "C:\\Program Files\\LLVM\\bin\\clang-cl.exe" "-fuse-ld=lld" : ... ; output: C:/boost/tools/build/src/tools\clang-win.jam:74: in clang-win.init *** argument error * rule path.native ( path ) * called with: ( C:\Program Files\LLVM\bin\clang-cl.exe -fuse-ld=lld ) * extra argument -fuse-ld=lld So, this syntax does not seem to be accepted by b2. Or, you could set <archiver> manually:
using clang-win : : : <archiver>"\"C:\\Program Files\\LLVM\\bin\\lld-link.exe\" /lib" ... ;
This works, up to a point, it does create the libs, but some of the feature detection .exe's fail to compile: clang-win.link c:\boost-build\boost\bin.v2\standalone\ac\clng-win-8.0.0\dbg\adrs-mdl-64\instr-set-hswl\lnk-sttc\rntm-lnk-sttc\thrd-mlt\lzma.exe "C:\Program Files\LLVM\bin\clang-cl.exe" -m64 /link /incremental:no /DEBUG /subsystem:console /out:"c:\boost-build\boost\bin.v2\standalone\ac\clng-win-8.0.0\dbg\adrs-mdl-64\instr-set-hswl\lnk-sttc\rntm-lnk-sttc\thrd-mlt\lzma.exe" -flto=thin @"c:\boost-build\boost\bin.v2\standalone\ac\clng-win-8.0.0\dbg\adrs-mdl-64\instr-set-hswl\lnk-sttc\rntm-lnk-sttc\thrd-mlt\lzma.exe.rsp" if %ERRORLEVEL% NEQ 0 EXIT %ERRORLEVEL% LINK : warning LNK4044: unrecognized option '/flto=thin'; ignored c:\boost-build\boost\bin.v2\standalone\ac\clng-win-8.0.0\dbg\adrs-mdl-64\instr-set-hswl\lnk-sttc\rntm-lnk-sttc\thrd-mlt\main-lzma.obj : fatal error LNK1107: invalid or corrupt file: cannot read at 0x9E4 clang-cl: error: linker command failed with exit code 1107 (use -v to see invocation) In view of the format of the linker warning, I must conclude that link.exe is called. In my case, they would have failed anyway, but it's obviously not correct. What does work is: "C:\Program Files\LLVM\bin\clang-cl.exe" -fuse-ld=lld -flto=thin -m64 /link /incremental:no /DEBUG /subsystem:console /out:"c:\boost-build\boost\bin.v2\standalone\ac\clng-win-8.0.0\dbg\adrs-mdl-64\instr-set-hswl\lnk-sttc\rntm-lnk-sttc\thrd-mlt\lzma.exe" @"c:\boost-build\boost\bin.v2\standalone\ac\clng-win-8.0.0\dbg\adrs-mdl-64\instr-set-hswl\lnk-sttc\rntm-lnk-sttc\thrd-mlt\lzma.exe.rsp" But that is obviously not how things are passed around. So it seems we do need the first option [ using clang-win : : "C:\\Program Files\\LLVM\\bin\\clang-cl.exe" "-fuse-ld=lld" : ... ; ] in order for all things to work, unfortunately that is not (yet) accepted by b2. We are getting there, but not fully, yet. degski -- *“If something cannot go on forever, it will stop" - Herbert Stein*
On Thu, 1 Nov 2018 at 12:27, degski
On Wed, 31 Oct 2018 at 16:32, Peter Dimov via Boost
wrote: using clang-win : : "C:\\Program Files\\LLVM\\bin\\clang-cl.exe" "-fuse-ld=lld" : ... ;
I have:
import option ; using clang-win : : "C:\\Program Files\\LLVM\\bin\\clang-cl.exe" "-fuse-ld=lld" : ... ;
output:
C:/boost/tools/build/src/tools\clang-win.jam:74: in clang-win.init *** argument error * rule path.native ( path ) * called with: ( C:\Program Files\LLVM\bin\clang-cl.exe -fuse-ld=lld ) * extra argument -fuse-ld=lld
So, this syntax does not seem to be accepted by b2.
As it is, for me, easier to write some C++-code than to try and figure out bjam-code, I wrote a wrapper https://github.com/degski/clang-cl around clang-cl.exe [yes, it's not pretty, but on the other hand, gives me full control, I can parse the arguments and decide whatever] to solve the above problem. For now I can see 1 regression, I'm going to look into that. degski -- *“If something cannot go on forever, it will stop" - Herbert Stein*
I was going to look into this but got sidetracked, sorry.
Try the current develop.
-----Original Message-----
From: degski via Boost
Sent: Monday, November 05, 2018 17:09
To: boost
Cc: degski ; Peter Dimov
Subject: Re: [boost] clang-win, (Thin)LTO this time
On Thu, 1 Nov 2018 at 12:27, degski
On Wed, 31 Oct 2018 at 16:32, Peter Dimov via Boost
wrote: using clang-win : : "C:\\Program Files\\LLVM\\bin\\clang-cl.exe" "-fuse-ld=lld" : ... ;
I have:
import option ; using clang-win : : "C:\\Program Files\\LLVM\\bin\\clang-cl.exe" "-fuse-ld=lld" : ... ;
output:
C:/boost/tools/build/src/tools\clang-win.jam:74: in clang-win.init *** argument error * rule path.native ( path ) * called with: ( C:\Program Files\LLVM\bin\clang-cl.exe -fuse-ld=lld ) * extra argument -fuse-ld=lld
So, this syntax does not seem to be accepted by b2.
As it is, for me, easier to write some C++-code than to try and figure out bjam-code, I wrote a wrapper https://github.com/degski/clang-cl around clang-cl.exe [yes, it's not pretty, but on the other hand, gives me full control, I can parse the arguments and decide whatever] to solve the above problem. For now I can see 1 regression, I'm going to look into that. degski -- *“If something cannot go on forever, it will stop" - Herbert Stein* _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On Mon, 5 Nov 2018 at 18:46, Peter Dimov via Boost
I was going to look into this but got sidetracked, sorry.
No worries. Try the current develop.
I did, and I've use this project-config: import option ; using clang-win : : "C:\\Program Files\\LLVM\\bin\\clang-cl.exe" -fuse-ld=lld : <compatibility>vc14 <compileflags>-fmsc-version=1915 <compileflags>-fms-compatibility <compileflags>-fno-delayed-template-parsing <compileflags>-flto=thin <cxxflags>"/std:c++17" <cxxflags>"/Zc:forScope" <cxxflags>"/arch:AVX2" <cxxflags>"/Zc:wchar_t" <cxxflags>"/EHsc" <cxxflags>"/Zc:inline" <cxxflags>"/Gd" <cxxflags>"/diagnostics:classic" <cxxflags>-Wno-unknown-argument <cxxflags>-Wno-unknown-pragmas <cxxflags>-Wno-macro-redefined <cxxflags>-Wno-unused-variable <cxxflags>-Wno-mismatched-tags <cxxflags>-Wno-deprecated-declarations <cxxflags>-D_UNICODE <cxxflags>-DUNICODE <cxxflags>-DBOOST_USE_WINDOWS_H <cxxflags>-DBOOST_NO_ANSI_APIS <cxxflags>-DBOOST_USE_WINAPI_VERSION=0x1000 <cxxflags>-DBOOST_USE_WINDOWS_H=1 <cxxflags>-DNOMINMAX <cxxflags>-DWIN32_LEAN_AND_MEAN <cxxflags>-D_CRT_SECURE_NO_WARNINGS ; Note that -fuse-ld=lld in the above is not quoted, but it comes out like [as an example]: "C:\Program Files\LLVM\bin\clang-cl.exe" "-fuse-ld=lld" -m64 @"c:\boost-build\boost\bin.v2\libs\log\config\atomic-int32\clng-win-8.0.0\dbg\adrs-mdl-64\pch-off\atomic_int32.obj.rsp" So, -fuse-ld=lld is quoted in the command. It appears [looks like] clang-cl.exe gobbles the quotes, though, i.e. it seems to work. There are some options that are passed, that are linux, like template_depth, which is set to 255 [while the default is 1024]. These are ignored, and in the case of template_depth, that seems better. Then I notice this: clang-win.link c:\boost-build\boost\bin.v2\libs\stacktrace\build\clng-win-8.0.0\dbg\adrs-mdl-64\WinDbg_exe.exe "C:\Program Files\LLVM\bin\clang-cl.exe" "-fuse-ld=lld" -m64 /link /incremental:no /DEBUG /subsystem:console /out:"c:\boost-build\boost\bin.v2\libs\stacktrace\build\clng-win-8.0.0\dbg\adrs-mdl-64\WinDbg_exe.exe" @"c:\boost-build\boost\bin.v2\libs\stacktrace\build\clng-win-8.0.0\dbg\adrs-mdl-64\WinDbg_exe.exe.rsp" if %ERRORLEVEL% NEQ 0 EXIT %ERRORLEVEL% lld-link: error: undefined symbol: __imp_CoInitializeEx So it looks like ole32.lib is not linked in, or that lld-link cannot link it [for whatever reason]. There's probably more small stuff, but in general, it looks pretty good. I also tried to link to one of the thus compiled static libs (boost_random), that works. It would be cool and helpful if the output [of the build-process] be reduced. I'm on an SSD, my guess is that the bottle-neck in the build process is the console output [most of the time]. I'm also looking at an 100'000 line log file, mostly "copy x to y" or "mklink x", it's hard to spot problems. Thanks, Peter, we're getting oh so close. degski -- *“If something cannot go on forever, it will stop" - Herbert Stein*
On Tue, 6 Nov 2018 at 14:17, degski
I did, and I've use this project-config:
import option ; using clang-win : : "C:\\Program Files\\LLVM\\bin\\clang-cl.exe" -fuse-ld=lld : <compatibility>vc14 <compileflags>-fmsc-version=1915 <compileflags>-fms-compatibility <compileflags>-fno-delayed-template-parsing <compileflags>-flto=thin <cxxflags>"/std:c++17"
Somebody asked about this, I believe Robert, so yes, compiles fine, even with C++17 [serialization and wserialization]. <cxxflags>"/Zc:forScope"
<cxxflags>"/arch:AVX2" <cxxflags>"/Zc:wchar_t" <cxxflags>"/EHsc"
By default, /EHs is used and passed in it seems [so the above duplicates this option, differently]. /EHs is not the default [/EHsc is] and in the IDE, it [/EHs] is marked "Yes with Extern C functions". I don't know what exactly this entails, but "Yes with Extern C functions" does not exactly strike me as the thing I want [but who knows, maybe I do]. <cxxflags>"/Zc:inline"
<cxxflags>"/Gd" <cxxflags>"/diagnostics:classic" <cxxflags>-Wno-unknown-argument <cxxflags>-Wno-unknown-pragmas <cxxflags>-Wno-macro-redefined <cxxflags>-Wno-unused-variable <cxxflags>-Wno-mismatched-tags <cxxflags>-Wno-deprecated-declarations <cxxflags>-D_UNICODE <cxxflags>-DUNICODE <cxxflags>-DBOOST_USE_WINDOWS_H <cxxflags>-DBOOST_NO_ANSI_APIS <cxxflags>-DBOOST_USE_WINAPI_VERSION=0x1000 <cxxflags>-DBOOST_USE_WINDOWS_H=1 <cxxflags>-DNOMINMAX <cxxflags>-DWIN32_LEAN_AND_MEAN <cxxflags>-D_CRT_SECURE_NO_WARNINGS ;
degski -- *“If something cannot go on forever, it will stop" - Herbert Stein*
On 11/6/2018 6:29 AM, degski via Boost wrote:
On Tue, 6 Nov 2018 at 14:17, degski
wrote: I did, and I've use this project-config:
import option ; using clang-win : : "C:\\Program Files\\LLVM\\bin\\clang-cl.exe" -fuse-ld=lld : <compatibility>vc14 <compileflags>-fmsc-version=1915 <compileflags>-fms-compatibility <compileflags>-fno-delayed-template-parsing <compileflags>-flto=thin <cxxflags>"/std:c++17"
Somebody asked about this, I believe Robert, so yes, compiles fine, even with C++17 [serialization and wserialization].
I do not recall if I ask. However, I have to escape the colon character with a backslash. I discover that by accident a few builds ago with b2, building all Boost with the cl.exe, approximately Visual Studio 15.6.4. Have you observed any messages stating how an unknown option is ignored? I purposely redirect all b2 output to a text file with the redirect operator. The tail end of the b2 invocation is similar to:
buildVSx64Fri5_6.log 2>&1
I highly recommend using the "2>&1". All error and normal messages will be in the one text file. Reviewing the file should isolate results faster. --Robert
<cxxflags>"/Zc:forScope"
<cxxflags>"/arch:AVX2" <cxxflags>"/Zc:wchar_t" <cxxflags>"/EHsc"
By default, /EHs is used and passed in it seems [so the above duplicates this option, differently]. /EHs is not the default [/EHsc is] and in the IDE, it [/EHs] is marked "Yes with Extern C functions". I don't know what exactly this entails, but "Yes with Extern C functions" does not exactly strike me as the thing I want [but who knows, maybe I do].
<cxxflags>"/Zc:inline"
<cxxflags>"/Gd" <cxxflags>"/diagnostics:classic" <cxxflags>-Wno-unknown-argument <cxxflags>-Wno-unknown-pragmas <cxxflags>-Wno-macro-redefined <cxxflags>-Wno-unused-variable <cxxflags>-Wno-mismatched-tags <cxxflags>-Wno-deprecated-declarations <cxxflags>-D_UNICODE <cxxflags>-DUNICODE <cxxflags>-DBOOST_USE_WINDOWS_H <cxxflags>-DBOOST_NO_ANSI_APIS <cxxflags>-DBOOST_USE_WINAPI_VERSION=0x1000 <cxxflags>-DBOOST_USE_WINDOWS_H=1 <cxxflags>-DNOMINMAX <cxxflags>-DWIN32_LEAN_AND_MEAN <cxxflags>-D_CRT_SECURE_NO_WARNINGS ;
degski
On Tue, 6 Nov 2018 at 17:45, Robert via Boost
I do not recall if I ask.
I meant Robert Ramey, my bad. The question was more like does anyone build [successfully] Boost with other than C++03? However, I have to escape the colon character
with a backslash. I discover that by accident a few builds ago with b2, building all Boost with the cl.exe, approximately Visual Studio 15.6.4. Have you observed any messages stating how an unknown option is ignored? I purposely redirect all b2 output to a text file with the redirect operator. The tail end of the b2 invocation is similar to:
buildVSx64Fri5_6.log 2>&1
I highly recommend using the "2>&1". All error and normal messages will be in the one text file. Reviewing the file should isolate results faster.
b2 has an option: -oC:\tmp\clang_log_debug.log for the command-line. How to get only the error-messages? degski -- *“If something cannot go on forever, it will stop" - Herbert Stein*
On Tue, 6 Nov 2018 at 14:17, degski
So, -fuse-ld=lld is quoted in the command. It appears [looks like] clang-cl.exe gobbles the quotes, though, i.e. it seems to work.
A simple test [slow brain I guess, alzheimers' kicking in] confirms the above: Y:\>clang-cl --version clang version 8.0.0 (trunk) Target: x86_64-pc-windows-msvc Thread model: posix InstalledDir: C:\Program Files\LLVM\bin Y:\>clang-cl "--version" clang version 8.0.0 (trunk) Target: x86_64-pc-windows-msvc Thread model: posix InstalledDir: C:\Program Files\LLVM\bin Y:\>clang-cl "--versio" clang-cl: warning: unknown argument ignored in clang-cl '--versio' (did you mean '--version'?) [-Wunknown-argument] clang-cl: error: no input files Y:\> degski -- *“If something cannot go on forever, it will stop" - Herbert Stein*
degski wrote:
So, -fuse-ld=lld is quoted in the command. It appears [looks like] clang-cl.exe gobbles the quotes, though, i.e. it seems to work.
A simple test [slow brain I guess, alzheimers' kicking in] confirms the above:
The first level of quoting is b2 quoting: using clang-win : : "C:\\Program Files\\LLVM\\clang-cl.exe" ; Here the quotes and the backslashes are b2 quoting; the value itself ends up 'C:\Program Files\LLVM\clang-cl.exe' (without the single quotes of course.) If you don't quote, as in using clang-win : : C:/Program Files/LLVM/clang-cl.exe ; b2 will split this on the space character and the value will be a list of `C:/Program` and `Files/LLVM/clang-cl.exe`. With using clang-win : : "C:\\Program Files\\LLVM\\clang-cl.exe" -fuse-ld=lld ; the result is a list of `C:\Program Files\LLVM\clang-cl.exe` and `-fuse-ld=lld`. The second level of quoting, which clang-win.jam applies to the above list manually with compiler = "\"$(compiler)\"" ; is for the Windows command line. The result is a command line of the form "C:\Program Files\LLVM\clang-cl.exe" "-fuse-ld=lld" <args> It's not clang-cl.exe that strips the quotes here, it's cmd.exe. Without the quotes, it will give an error that `C:\Program` isn't found. The quotes around `-fuse-ld=lld` are unnecessary in this case, as this argument has no spaces, but they don't hurt. They would be needed for something like (hypothetically) "-fsome-path=C:\Program Files\somepath".
On Wed, 7 Nov 2018 at 16:01, Peter Dimov via Boost
The first level of quoting is b2 quoting:
using clang-win : : "C:\\Program Files\\LLVM\\clang-cl.exe" ;
Here the quotes and the backslashes are b2 quoting; the value itself ends up 'C:\Program Files\LLVM\clang-cl.exe' (without the single quotes of course.)
If you don't quote, as in
using clang-win : : C:/Program Files/LLVM/clang-cl.exe ;
b2 will split this on the space character and the value will be a list of `C:/Program` and `Files/LLVM/clang-cl.exe`.
With
using clang-win : : "C:\\Program Files\\LLVM\\clang-cl.exe" -fuse-ld=lld ;
the result is a list of `C:\Program Files\LLVM\clang-cl.exe` and `-fuse-ld=lld`.
The second level of quoting, which clang-win.jam applies to the above list manually with
compiler = "\"$(compiler)\"" ;
is for the Windows command line. The result is a command line of the form
"C:\Program Files\LLVM\clang-cl.exe" "-fuse-ld=lld" <args>
It's not clang-cl.exe that strips the quotes here, it's cmd.exe. Without the quotes, it will give an error that `C:\Program` isn't found.
The quotes around `-fuse-ld=lld` are unnecessary in this case, as this argument has no spaces, but they don't hurt. They would be needed for something like (hypothetically) "-fsome-path=C:\Program Files\somepath".
Thanks for the clarification. degski -- *“If something cannot go on forever, it will stop" - Herbert Stein*
[degski]
Substantial progress has been made towards the goal of compiling Boost with Clang/LLVM (clang-cl.exe, targeting MSVC).
If you encounter any issues with MSVC's STL here, please let me know. Our test suites (internal and libc++) are clean with MSVC's STL + Clang/LLVM with maximum conformance switches (-fno-ms-compatibility -fno-delayed-template-parsing) but we don't have Boost coverage on top of that. This includes being warning-clean (we test with clang-cl /W4 but will consider cleaning up warnings beyond that). Thanks, STL
participants (4)
-
degski
-
Peter Dimov
-
Robert
-
Stephan T. Lavavej