I can't speak for the boost community, but I guess another big question mark is probably long term support and response to security vulnerabilities. I wouldn't be surprised, if people are reluctant to base the security of their communication on a lesser known library, when they can't be confident that bugs and vulnerabilities are getting fixed quickly.
I'd fear the same. Common usage of Boost is: Install specific version and stick to it until absolutely required to upgrade. --> This longevity is not suitable for a crypto library which need to adapt to new threats quickly. I'd hence suggest to make this a standalone version with CI tested against various boost versions (at least min. required, latest release and (opt) master) and use proper semantic versioning as well as good integration with build systems (I'd suggest CMake and/or Meson) However I can imagine that some Boost libraries may use that, especially if it integrates well enough