On Sunday, August 23, 2020, Mateusz Loskot via Boost wrote:

On Sun, 23 Aug 2020 at 01:05, Zach Laine via Boost wrote:

...

I've made all the recommended changes to Boost.Text that came up in the review. Glen, the previous review manager, recommended a mini review of Boost.Text once I'd made those changes.

The Mini-Review is usually conducted by the same review manager [1]. Would it be possible?

Zach already asked me, but I am unavailable at the moment. :-) Glen

Zach Laine wrote:

On Sun, Aug 23, 2020 at 11:08 AM Phil Endecott via Boost wrote:

...

Could you please explain what you've done about the copyright issues?

Sure. I've reimplemented the code that originally came from ICU, and ...

...

As far as I can tell, you still depend on the Unicode data files that have a Boost-incompatible licence. You previously included this Unicode copyright text in the documentation but that page has now been removed, if I'm looking in the right place.

... removed the ICU copyright from these files. They are the output of a code generation tool, and so are not copyrightable individually (like the output of lex and yacc).

For the benefit of everyone else let me describe what Zach does: 1. There are some files at unicode.org that have a Boost-incompatible licence. 2. Zach has some Python scripts at https://github.com/tzlaine/text/tree/master/scripts 3. The scripts download the files from unicode.org, convert them into C++ source files, and prefix the result "(C) Zach Laine Boost License". 4. These generated files are checked in at https://github.com/tzlaine/text/tree/master/include/boost/text/data The intention is not that end-users of Boost.Text will run the scripts, but rather that the generated files will be included in the Boost source distribution. Zach thinks this is OK because "they are the output of a code generation tool, and so are not copyrightable individually (like the output of lex and yacc)". I think that's completely wrong. I believe it's a well-established principle of software copyright law that the output of a tool - whether that is g++, bison, or rot13 - is a derived work of the input to that tool. You cannot (without permission) take example.y that's (C) Megacorp, run bison on it, and claim that the resulting example.tab.c is now (C) Someone Else. This worries me. We really, really don't want to be shipping code that has copyright violations! Glen, in your review result announcement you said you were confident that Zach would be able to resolve the copyright problems. What did you have in mind when you wrote that? Regards, Phil.

Peter Dimov via Boost said: (by the date of Mon, 24 Aug 2020 17:10:48 +0300)

Either way, if we go with the strict interpretation and decide that

{0x0028, 0x0029, bidi_bracket_type::open}, {0x0029, 0x0028, bidi_bracket_type::close}, {0x005B, 0x005D, bidi_bracket_type::open}, {0x005D, 0x005B, bidi_bracket_type::close}, {0x007B, 0x007D, bidi_bracket_type::open}, {0x007D, 0x007B, bidi_bracket_type::close},

is a derived work of a Unicode data file, I see no way of ever having a Unicode library in Boost.

There are debian packages available which support UTF-8 encoding, conversion etc. Debian has rather strict license requirements. These packages wouldn't be possible if the code numbers of UTF code glyphs weren't publicly accessible. Also there are many online UTF-8 databases. -- # Janek Kozicki http://janek.kozicki.pl/

Peter Dimov wrote:

Phil Endecott wrote: ...

...

4. These generated files are checked in at https://github.com/tzlaine/text/tree/master/include/boost/text/data

https://github.com/tzlaine/text/tree/master/include/boost/text/detail surely?

Err yes, both I think. It's doesn't matter much though.

if we go with the strict interpretation and decide that

{0x0028, 0x0029, bidi_bracket_type::open}, {0x0029, 0x0028, bidi_bracket_type::close}, {0x005B, 0x005D, bidi_bracket_type::open}, {0x005D, 0x005B, bidi_bracket_type::close}, {0x007B, 0x007D, bidi_bracket_type::open}, {0x007D, 0x007B, bidi_bracket_type::close},

is a derived work of a Unicode data file, I see no way of ever having a Unicode library in Boost.

I see various possible ways forward though none is particularly appealing: 1. A run-time dependency on libicu, getting libicu to do all the work, as Boost.Locale does. 2. A run-time dependency on libicu that generates these tables by interrogating libicu at start-up, and then using Zach's code. 3. Getting users to download the Unicode data files and run the scripts at Boost compile time. 4. Making an exception to the licensing policy. 5. Removing this from Boost.Text, keeping just the UTFn conversion code and anything else that doesn't depend on the problematic data files (as I advocated in my review). Regards, Phil.

On Wed, 26 Aug 2020 at 09:59, Christian Mazakas via Boost wrote:

...

5. Removing this from Boost.Text, keeping just the UTFn conversion code and anything else that doesn't depend on the problematic data files (as I advocated in my review).

I think this would be to the ultimate detriment of the library. Anything Boost.Text can do for the user is a boon in this regard and only increases the quality of the lib.

Agree

...

1. A run-time dependency on libicu, getting libicu to do all the work, as Boost.Locale does.

I think this would be the best path forward. Obtaining a dynamic version of libicu is relatively straight-forward on all systems and would dodge the licensing issues.

I disagree as a user. From a user perspective a libicu dependency does not "dodge" anything. The user then definitely has to comply with the libicu license. Also, for many uses of the words "all systems" (that want to process unicode text) obtaining and using a dynamic version of libicu is anything but "reasonably straight-forward". Having only a licence issue to resolve is simpler than having a library dependency that itself creates a dependency on the same license. Note I am not referring to any "copyleft" issues that might arise (and might or might not be avoided through dynamic linking - its hard to dynamically link a header dependency) as there is no copyleft involved in this license. The preceding is about licence terms - but there is first an issue of what rights the copyright holder actually has over the work / whether the work is actually copyrightable in the first place. Some would claim that the icu data files are simply a list of facts and not creative. However, as ICU "made up" (defined) some of those facts it perhaps can't be said not to be creative. In the absence of explicit permission from ICU to place a different licence on the generated files it may be best to ensure that these files do identify their ICU dependency/derivation, and reference the relevant licence. Any/all of Boost.Text that does not depend on these files should be usable in a "NO_ICU" build that does not include/compile any of the generated files. And the ICU licence should be included with and referenced from the generated files. What a user chooses to do is up to them. Boost has definitely complied. And the user issue is JUST how (and, dare I say, if) to comply with the ICU license terms - which at worst requires distributing the ICU license file somehow. This isn't a huge hardship.

Gesendet: Donnerstag, 27. August 2020 um 07:15 Uhr Von: "Rainer Deyke via Boost"

On 26.08.20 01:58, Christian Mazakas via Boost wrote:

...

...

1. A run-time dependency on libicu, getting libicu to do all the work, as Boost.Locale does.

I think this would be the best path forward. Obtaining a dynamic version of libicu is relatively straight-forward on all systems and would dodge the licensing issues.

The only reason I'm interested in Boost.Text is that it will help me get rid of my dependency on ICU, with its broken build system and bloat. A version of Boost.Text that relies on ICU has no value to me, and I would not use it.

For me it wouldn't become useless, but I'd certainly say it would be the least appealing solution for me and make it significantly less likely that we will use this library in one of our projects. Also, how would such a design impact the interface and performance of this library? For example this would make it impossible to ever use parts of this lib in a constexpr context correct?. Considering that it seems unclear if such measure are necessary to begin with, I'd implore the boost organization to first seek legal advice and/or contact whoever holds the copyright for libicu to establish that this is really the only viable option before taking "the easy way out" and delivering a technically inferior library just out of uncertainty. What I wonder: If the same data could - in theory - be scraped from a published ISO standards document, then can that data really be subject to the libicu license requirements? I'm also not positive, that if you put a copyrighted document through a language processor and the output retains none of the structure of the original document, that this counts as "derived work" in the sense of copyright law. Extreme case: If I run a novel through a tool like, wc, I don't believe the output of that tool would stell be under the copyright of the owner. Of course IANAL so my opinion doesn't mean anything, but this is why this should be reviewd by someone who is actually a lawyer. Best Mike

-- Rainer Deyke (rainerd@eldwood.com)

Gesendet: Freitag, 28. August 2020 um 13:23 Uhr Von: "Janek Kozicki via Boost"

Mike via Boost said: (by the date of Fri, 28 Aug 2020 12:47:55 +0200)

...

Extreme case: If I run a novel through a tool like, wc, I don't believe the output of that tool would stell be under the copyright of the owner.

that would be an attempt to copyright a natural number. That's not possible.

the output isn't a natural number, but 3 natural numbers corresponding to the line, word and bytecount of that document (at least on my system).

The actual information that UTF *has* is which natural number corresponds to which glyph. Ask about this.

Whom should I ask what?

-- # Janek Kozicki http://janek.kozicki.pl/

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On 2020-08-28 19:29, Janek Kozicki via Boost wrote:

Mike via Boost said: (by the date of Fri, 28 Aug 2020 15:12:50 +0200)

...

...

The actual information that UTF *has* is which natural number corresponds to which glyph. Ask about this.

Whom

My apologies, clearly it was another person who wanted to ask (or already asked) the boost lawyers and the UTF consortium lawyers.

...

should I ask what?

If this "number" ↔ "glyph" correspondence can be used by boost on the terms of boost license.

The license applies not to the abstract coding of glyphs or characters but to the data files published by Unicode Inc. that describe this. It is these files Zach translates into bits of Boost.Text code, which is then claimed to be not covered by Unicode Inc. license. The important difference is that (a) Zach is using a material document that is licenseable (as opposed to abstract facts or numbers, which are not) as a source and (b) the result of translation preserves the original information in some meaningful amount (as opposed to e.g. applying a hash function or `wc`, which does not).

On 2020-08-24 16:29, Phil Endecott via Boost wrote:

Zach Laine wrote:

...

On Sun, Aug 23, 2020 at 11:08 AM Phil Endecott via Boost wrote:

...

Could you please explain what you've done about the copyright issues?

Sure.  I've reimplemented the code that originally came from ICU, and ...

...

As far as I can tell, you still depend on the Unicode data files that have a Boost-incompatible licence.  You previously included this Unicode copyright text in the documentation but that page has now been removed, if I'm looking in the right place.

... removed the ICU copyright from these files. They are the output of a code generation tool, and so are not copyrightable individually (like the output of lex and yacc).

For the benefit of everyone else let me describe what Zach does:

1. There are some files at unicode.org that have a Boost-incompatible licence.

I believe, these are the terms of use published by Unicode Inc.: http://www.unicode.org/copyright.html https://www.unicode.org/license.html I'll cite the relevant paragraph from the latter document: Permission is hereby granted, free of charge, to any person obtaining a copy of the Unicode data files and any associated documentation (the "Data Files") or Unicode software and any associated documentation (the "Software") to deal in the Data Files or Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, and/or sell copies of the Data Files or Software, and to permit persons to whom the Data Files or Software are furnished to do so, provided that either (a) this copyright and permission notice appear with all copies of the Data Files or Software, or (b) this copyright and permission notice appear in associated Documentation. I'm not a lawyer, but is doesn't seem to contradict our license requirements: https://www.boost.org/development/requirements.html#License In particular: - It grants permission to copy, use and distribute free of charge and commercially. - It requires the license to appear in the docs or the source code of the data (which, I believe, is in textual form). - It does *not* require the license to appear in the compiled binaries, which may contain the data in binary form. Am I missing some aspect where the license is incompatible with Boost requirements?

2. Zach has some Python scripts at https://github.com/tzlaine/text/tree/master/scripts

3. The scripts download the files from unicode.org, convert them into C++ source files, and prefix the result "(C) Zach Laine Boost License".

4. These generated files are checked in at https://github.com/tzlaine/text/tree/master/include/boost/text/data The intention is not that end-users of Boost.Text will run the scripts, but rather that the generated files will be included in the Boost source distribution.

Zach thinks this is OK because "they are the output of a code generation tool, and so are not copyrightable individually (like the output of lex and yacc)".

I think that's completely wrong. I believe it's a well-established principle of software copyright law that the output of a tool - whether that is g++, bison, or rot13 - is a derived work of the input to that tool.  You cannot (without permission) take example.y that's (C) Megacorp, run bison on it, and claim that the resulting example.tab.c is now (C) Someone Else.

Changing the copyright indeed does not look good. However, generally, the terms of use of the source and compiled/translated forms can be different, depending on the license.

This worries me.  We really, really don't want to be shipping code that has copyright violations!

Agreed to this. I think, Boost needs to consult with a lawyer on this matter. Personally, I wouldn't like if Boost stopped being entirely covered by the BSL. As I said earlier, this is an important property for Boost to be acceptable in many environments. However, if the Unicode Inc. terms of use don't contradict the BSL and Boost License requirements (as determined by a lawyer) and there really is no other way around it, I'd accept it as an exception for the sake of a greater good of improving Unicode support in C++. Should it end up this way, the licensing terms must be explained clearly in Boost.Text docs, with proper copyright attribution. With a note that license compatibility has been verified and confirmed by a lawyer.

On 2020-08-24 20:22, Peter Dimov via Boost wrote:

Andrey Semashev wrote:

...

- It does *not* require the license to appear in the compiled binaries, which may contain the data in binary form.

It does. Without explicit permission (which the BSL contains for this very reason), binaries are derived works as much as script-processed files are.

I thought "copies" means copies of the source code, doesn't it? There is also a latch that the license can appear only in the docs, not the data itself, whether in source or binary form. I guess, that's what the layer has to clarify. But if it indeed means binary form as well then, alas, no Unicode in Boost.

On 2020-08-24 21:24, Peter Dimov via Boost wrote:

Andrey Semashev wrote:

...

But if it indeed means binary form as well then, alas, no Unicode in Boost.

It does, but I'm pretty sure the Unicode consortium isn't going to go after us for this specific use (using the data files to generate code that is necessary to implement the functionality specified in the standard.)

It doesn't matter how likely it is. This isn't just about us, but also about our users. One other option would be to ask Unicode Inc. for a permission to use Unicode data in binary form without the requirement to present the license.

On 2020-08-24 22:40, Zach Laine via Boost wrote:

On Mon, Aug 24, 2020 at 2:31 PM Andrey Semashev via Boost wrote:

...

On 2020-08-24 21:24, Peter Dimov via Boost wrote:

...

Andrey Semashev wrote:

...

But if it indeed means binary form as well then, alas, no Unicode in Boost.

It does, but I'm pretty sure the Unicode consortium isn't going to go after us for this specific use (using the data files to generate code that is necessary to implement the functionality specified in the standard.)

It doesn't matter how likely it is. This isn't just about us, but also about our users.

One other option would be to ask Unicode Inc. for a permission to use Unicode data in binary form without the requirement to present the license.

Why is this suddenly a problem if Boost.Spirit has been doing it for years?

Because apparently noone knew they were doing this. It may well become a problem of Boost.Spirit now. Personally, I'm still not sure there is a license incompatibility, until a lawyer comments.

Did anyone try to contact the unicode consortium about this? I'm not part of that community, but maybe they are ok and they can officially say so? Or maybe they are not and we can avoid guessing what they are going to do. El lun., 24 ago. 2020 a las 17:22, Peter Dimov via Boost (< boost@lists.boost.org>) escribió:

Andrey Semashev wrote:

...

Personally, I'm still not sure there is a license incompatibility, until a lawyer comments.

Lawyers can't tell you whether the Unicode consortium will be OK with it; they can't even tell you with any certainty whether a court will rule that this is a violation. It's a sufficiently gray area that it's potentially problematic, but not clearly problematic.

My opinion is that this use is fine in practice. (Using headers bearing the Unicode consortium copyright directly was not.)

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

-----Original Message----- From: Boost On Behalf Of Peter Dimov via Boost Sent: 24 August 2020 22:22 To: boost@lists.boost.org Cc: Peter Dimov Subject: Re: [boost] Boost.Text Unicode licence issues

Andrey Semashev wrote:

...

Personally, I'm still not sure there is a license incompatibility, until a lawyer comments.

Lawyers can't tell you whether the Unicode consortium will be OK with it; they can't even tell you with any certainty whether a court will rule that this is a violation. It's a sufficiently gray area

that it's

potentially problematic, but not clearly problematic.

My opinion is that this use is fine in practice. (Using headers bearing the Unicode consortium copyright directly was not.)

I agree with this. It is clearly a grey area, but Boost has a defensible position (and the likelihood of being called to defend it is sub-normal near zero). Confirmation from Unicode would be perfect, but I doubt if we will get it. My 2 p. Paul

The data files that the Unicode Consortium provides are based on the Unicode specifications, yes? Could we not simply reproduce the data tables from the specification? I realize this is potentially a bunch of work. -- Bryce Adelstein Lelbach aka wash US Programming Language Standards (PL22) Chair ISO C++ Library Evolution Chair CppCon and C++Now Program Chair CUDA Core C++ Libraries (Thrust, CUB, libcu++) Lead @ NVIDIA --

On Mon, Aug 24, 2020 at 2:47 PM Ville Voutilainen wrote:

On Mon, 24 Aug 2020 at 22:40, Zach Laine via Boost wrote:

...

...

It doesn't matter how likely it is. This isn't just about us, but also about our users.

One other option would be to ask Unicode Inc. for a permission to use Unicode data in binary form without the requirement to present the license.

Why is this suddenly a problem if Boost.Spirit has been doing it for years?

IANAL and all that, but I don't think that argument works very well. In the case of a hypothetical copyright violation, I have never heard anything suggesting that the length of a period of violation somehow renders the copyright moot, nor is there any suggestion that not coming after a violation does so either.

Sure, but that was not the point of my comment. My my point is that if we are concerned that Spirit's use of the Unicode Character Database would cause difficulty for end-user license approval, or potential lawsuits from Unicode (which I think is approximately 0% likely), it probably would have come up already in the last 10 years or so. Zach

On 2020-08-24 22:17, Ville Voutilainen via Boost wrote:

Where does boost go for legal advice?

The Software Freedom Conservancy: https://lists.boost.org/Archives/boost/2015/12/226951.php Although we would need confirmation from the Boost Steering Committee that we are still a member.

Software Conservancy used to have a general counsel on staff, but unfortunately he left a few years ago and I don't believe they have general counsel anymore (he's still on the board of directors, though). I'm not sure if we have legal representation for the Boost Foundation, and more specifically the right type of legal representation (e.g. someone with ample experience in software intellectual property law). Software Conservancy or the LLVM Foundation may be able to point us in the direction of law firms / lawyers specializing in this area. If we really believe we need a lawyer for this, it will probably cost money. On Mon, Aug 24, 2020 at 2:04 PM Glen Fernandes via Boost wrote:

On Mon, Aug 24, 2020 at 5:01 PM Bjorn Reese wrote:

...

The Software Freedom Conservancy:

Although we would need confirmation from the Boost Steering Committee that we are still a member.

https://sites.google.com/a/boost.org/steering/boost-foundation

"After being a project of Software Freedom Conservancy for over a decade, Boost has spun-off into it owns 501(c)(3) non-profit corporation named Boost Foundation. We are forever grateful to Software Freedom Conservancy for their support and guidance to achieve this milestone.

Boost Foundation Board of Directors will now assume the role that was filled by the Boost Steering Committee."

-- Bryce Adelstein Lelbach aka wash US Programming Language Standards (PL22) Chair ISO C++ Library Evolution Chair CppCon and C++Now Program Chair CUDA Core C++ Libraries (Thrust, CUB, libcu++) Lead @ NVIDIA --

On 2020-09-01 14:50, Paul A Bristow via Boost wrote:

I sense it is unlikely that we would get a definitive legal opinion, even after paying money, for a license detail that is clearly ill-defined.

Are there any reasons why Unicode would wish to complain to (or worse sue) Boost or its users? I can't conceive of any, especially when Unicode have not done so already to Boost when it has been in use for a decade.

Have the most stringent reading of the requirements been imposed on any other user of Unicode? Not to my knowledge.

Zach's work provides a defensible position and allows us to plausibly claim compliance.

I feel we are worrying about a non-issue.

Licensing clarity is important for our downstream users. It doesn't matter if Unicode will sue anyone or not. If they are actually fine with the license not being distributed with binaries, then let them confirm this in written form. If they don't confirm then that would be seen as a violation, which we don't want to have in Boost, regardless of the consequences.

-----Original Message----- From: Boost On Behalf Of Peter Dimov via Boost Sent: 1 September 2020 14:17 To: boost@lists.boost.org Cc: Peter Dimov Subject: Re: [boost] Boost.Text Unicode licence issues

Paul A Bristow wrote:

...

I sense it is unlikely that we would get a definitive legal opinion, even after paying money, for a license detail that is clearly ill-defined.

I agree. The proper role of our legal representative - if we had one - in this case would not be to provide us with legal advice, but to contact the legal representative of the Unicode Consortium, explain

situation (Boost does not allow libraries that impose an attribution requirement for binaries, which on its face precludes us ever having a Unicode library), ask them to maybe consider dropping that requirement from their license, failing that, ask them for an explicit permission for Boost

the libraries to use

their data files without such a license requirement, failing that, ask them for a clear and an official statement that they do stand by this license requirement.

Ok with this - but, since we do not have a legal representative, can someone else write 'officially' from Boost? One of the steering group? Paul

On Tue, 1 Sep 2020, 12:13 pm Bryce Adelstein Lelbach aka wash via Boost, < boost@lists.boost.org> wrote:

If we really believe we need a lawyer for this, it will probably cost money.

Boost only needs legal advice if Boost wants to avoid passing the license buck onto users of Boost.Text. And the user feedback here is that such avoidance is not essential and much preferable to having to depend on the icu library. The only barrier to passing through the license (and it only applies to the generated files, nothing else) is Boost policy. Boost doesn't need legal advice to grant/allow an exception to that policy only for code generated from other people's copyright "data only" source, only as explicitly granted by Boost, presumably only granted after Boost have attempted to get the upstream copyright holder(s) to allow at least this specific use under BSL...