Re: [boost] [EXTERNAL] boost no longer builds mpi libraries when using intel mpi

20 Jul 2017


      On 20/07/2017 10:34, Alain Miniussi via Boost wrote:
...
On 20/07/2017 03:53, Belcourt, Kenneth via Boost wrote:
...
On Jul 19, 2017, at 4:00 PM, Edward Diener via Boost 
mailto:boost@lists.boost.org> wrote:
On 7/19/2017 5:40 PM, Alain Miniussi via Boost wrote:
On 19/07/2017 22:57, Edward Diener via Boost wrote:
On 7/19/2017 3:04 PM, Alain Miniussi via Boost wrote:
On 19/07/2017 16:26, Franck Houssen via Boost wrote:
Hello,
Just to get your attention to this : 
https://github.com/easybuilders/easybuild-easyconfigs/issues/1065
It hasn't build with that syntax in years (edit, just saw it's from 
2017), Intel's MPI does not support the -show* options family (and 
don't plan to).
There is a specific piece of doc for that in:
http://www.boost.org/doc/libs/1_64_0/doc/html/mpi/getting_started.html#mpi.c...
I have no clue where the "using mpi"... is it called a directive ?.. 
is implemented, and if I did, I don't know how to detect we're using 
Intel's Implementation with bjam, nor how to pass the retrieved 
options, nor to what.
The 'using mpi' is a Boost Build rule which sets up a Boost Build 
toolset for use, as explained at 
http://www.boost.org/build/doc/html/bbv2/reference/tools.html.
Yes, what I don't know is where/how that rule is implemented. Nor how 
to implement one like mpi.
I remember greping mpi in the build system a few year ago, but could 
not find a way to touch it (it failed with no meaningful message).
Look at build/src/tools/mpi.jam. The 'init' rule corresponds to the 
'using mpi ... ;' line(s).
Hi Alain,
I think I’ve sent this before, but here’s the user-config.jam I use 
to build and run Boost.MPI tests with Intel 17 and Intel MPI 5.1.
using mpi
   : /projects/linux_rh6/SDK/mpi/intel/5.1/bin64/mpicxx
   ;
using intel
   ;
cd libs/mpi/test
../../../b2 -d+2 toolset=intel
Forgot to mention my toolset was intel-linux, so that could be an issue.
...
...
Does this work for you?  Previously you’d mentioned IMPI not working 
with Boost.Build, can you remind me what that failure involved?
Sure.
I try to take the story from the beginning so that all the involved 
people can get what I see as the whole picture. Sorry for the boring, 
well known bits.
The MPI people thought that the easiest way to compile MPI application 
was through compiler wrapper like mpicc that would call the underlying 
compiler with the right options. Those options include, but are not 
limited to[*],  the -I<incl path> -L<lib path> -l<libname> option.
Sometime (most of the time IMO) this is not good: we want to run the 
compiler directly with right options (after all, we're just want to 
use a library at that point). So the question is how to retrieve them.
Some/most MPI implementations provides wrappers options to retrieve 
those. OpenMPI based implementation usually provide 
--showme:(compile|link) option, so adding the output of $(mpicc 
--showme:link) to the link flags will do the job for example. Mpich 
based implementation usually provides -compile_info -link_info (which 
work slightly differently).
Most tools like bjam/cmake/autotool/<other I guess> tries to find the 
compile and link flags through those options. They try -show:compile, 
see if it succeed, and if it does not, try --compile_info.
Intel is based on mpich, but both --compile_info/--link_info/-show* 
return the same output (a mix of link and compile information). Even 
more annoying, the -show* option, even when it fails, returns a 
success status on many versions[**], meaning the build tool does not 
detect that the option is not supported (FindMPI.cmake does some magic 
to detect the Intel problem on recent versions) and does not know it 
should try something else.
As a result, with most currently deployed Intel's MPI, you need to 
explicitly set the include and lib path and libraries.
And of course, some MPI implementations do not have wrappers. Real 
world sucks.
Alain
[*]: meaning it's not so obvious that the bjam solution of providing 
includes path, lib search and libs is equivalent. We could be missing 
something, obviously not critical. But that's another story.
[**]: just saw that the very last one does not, which is a good thing, 
maybe there is hope for humanity and reporting issues to support had 
some effect after all. But the behavior is still problematic, both 
link and compilation queries returns the same options.
...
Noel
_______________________________________________
Unsubscribe & other changes: 
http://lists.boost.org/mailman/listinfo.cgi/boost
_______________________________________________
Unsubscribe & other changes: 
http://lists.boost.org/mailman/listinfo.cgi/boost