[test] unit test command-line handling
Hello, this is a question concerning the Boost.Test framework. I'd like to pass command-line arguments to my tests, and I'm unsure of how to proceed. I have found https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/adv_scen..., which shows how to access the command-line programmatically, but that's declared obsolete, and it's also lacking some important info, such as how I should handle the argv vector. Are there any examples that demonstrate this use case ? Specifically: * What (non-deprecated) API can I use to access command-line arguments during test initialization ? * How does my own handling of command-line arguments interact with Boost.Test's own command-line argument handling ? Thanks, Stefan -- ...ich hab' noch einen Koffer in Berlin...
On 12.09.18 00:21, Stefan Seefeld via Boost wrote:
Hello,
this is a question concerning the Boost.Test framework.
I'd like to pass command-line arguments to my tests, and I'm unsure of how to proceed. I have found https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/adv_scen..., which shows how to access the command-line programmatically, but that's declared obsolete, and it's also lacking some important info, such as how I should handle the argv vector. Are there any examples that demonstrate this use case ? Specifically:
* What (non-deprecated) API can I use to access command-line arguments during test initialization ?
By "test initialization", do you mean before the first test starts? The argc/argv are available to the master test suite: https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/tests_or... In 1.68, it is also possible to use this for data driven test cases, using the make_delayed (if you need a test generator that is dependent on your argc/argv). https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/change_l...
* How does my own handling of command-line arguments interact with Boost.Test's own command-line argument handling ?
I am surprised I did not create a doc entry about this. The way it works is indicated in the change logs of boost.test: https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/change_l... Anything after -- is passed to the master test suite argc/argv and is available to your test module. I will update the doc on the two points above to make it clearer. Raffi
Thanks,
Stefan
On 09/12/18 14:28, Raffi Enficiaud wrote:
On 12.09.18 00:21, Stefan Seefeld via Boost wrote:
* What (non-deprecated) API can I use to access command-line arguments during test initialization ?
By "test initialization", do you mean before the first test starts?
Yes.
The argc/argv are available to the master test suite:
https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/tests_or...
I see. That code could use a bit of an explanation. You present a function that takes "argc" and "argv" as input, but rather than using those variables themselves, you access the argument vector via "framework::master_test_suite().argc" etc.. I find that a bit...em... counter-intuitive. What's the rationale for that ? Is the function argument the full argument vector, including the ones already consumed by Boost.Test itself ?
In 1.68, it is also possible to use this for data driven test cases, using the make_delayed (if you need a test generator that is dependent on your argc/argv).
https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/change_l...
* How does my own handling of command-line arguments interact with Boost.Test's own command-line argument handling ?
I am surprised I did not create a doc entry about this. The way it works is indicated in the change logs of boost.test:
https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/change_l...
Anything after -- is passed to the master test suite argc/argv and is available to your test module.
That makes sense.
I will update the doc on the two points above to make it clearer.
Thank you. Please consider adding a top-level section on passing command-line arguments, perhaps under "Runtime parameters". I believe that's a common enough use-case that it might be worthwhile for this not to be hidden in a (seemingly unrelated) section about the master test suite. Thanks for your help ! Stefan -- ...ich hab' noch einen Koffer in Berlin...
On 09/12/18 18:50, Stefan Seefeld wrote:
On 09/12/18 14:28, Raffi Enficiaud wrote:
The argc/argv are available to the master test suite:
https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/tests_or...
I see. That code could use a bit of an explanation. You present a function that takes "argc" and "argv" as input, but rather than using those variables themselves, you access the argument vector via "framework::master_test_suite().argc" etc.. I find that a bit...em... counter-intuitive. What's the rationale for that ? Is the function argument the full argument vector, including the ones already consumed by Boost.Test itself ?
...and what is the appropriate way to signal an error, such as a missing or unsupported command-line argument ? (All the examples in the docs I could find return '0', so another related question is: what is the meaning of the return value ?) Thanks, Stefan -- ...ich hab' noch einen Koffer in Berlin...
On 13.09.18 03:21, Stefan Seefeld via Boost wrote:
On 09/12/18 18:50, Stefan Seefeld wrote:
On 09/12/18 14:28, Raffi Enficiaud wrote:
The argc/argv are available to the master test suite:
https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/tests_or...
I see. That code could use a bit of an explanation. You present a function that takes "argc" and "argv" as input, but rather than using those variables themselves, you access the argument vector via "framework::master_test_suite().argc" etc.. I find that a bit...em... counter-intuitive. What's the rationale for that ? Is the function argument the full argument vector, including the ones already consumed by Boost.Test itself ?
...and what is the appropriate way to signal an error, such as a missing or unsupported command-line argument ? (All the examples in the docs I could find return '0', so another related question is: what is the meaning of the return value ?)
There is this exception you may throw: https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost/unit_test/fra... Again, this is not so well documented here: https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/adv_scen... Raffi
...and what is the appropriate way to signal an error, such as a missing or unsupported command-line argument ? (All the examples in the docs I could find return '0', so another related question is: what is the meaning of the return value ?)
There is this exception you may throw:
https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost/unit_test/fra...
Again, this is not so well documented here:
https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/adv_scen...
I assume throwing setup_error works only with the obsolete init_func? How would it be with the 'new' way described at the caution note on bottom of the last URL. Thanks, Olaf
On 13.09.18 00:50, Stefan Seefeld via Boost wrote:
On 09/12/18 14:28, Raffi Enficiaud wrote:
On 12.09.18 00:21, Stefan Seefeld via Boost wrote:
* What (non-deprecated) API can I use to access command-line arguments during test initialization ?
By "test initialization", do you mean before the first test starts?
Yes.
The argc/argv are available to the master test suite:
https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/tests_or...
I see. That code could use a bit of an explanation. You present a function that takes "argc" and "argv" as input, but rather than using those variables themselves, you access the argument vector via "framework::master_test_suite().argc" etc.. I find that a bit...em... counter-intuitive. What's the rationale for that ? Is the function argument the full argument vector, including the ones already consumed by Boost.Test itself ?
Apart from the historical side of this design, this has two benefits IMO: 1- the argc/argv passed to the rest of the test module at runtime may have been altered, either by the test module itself (dropping some params in a global fixture), or by the boost.test framework. The argc/argv seen by the test module are the ones that appear after the -- 2- the master test suite is a singleton, so you do not need to pass around argc/argv
In 1.68, it is also possible to use this for data driven test cases, using the make_delayed (if you need a test generator that is dependent on your argc/argv).
https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/change_l...
* How does my own handling of command-line arguments interact with Boost.Test's own command-line argument handling ?
I am surprised I did not create a doc entry about this. The way it works is indicated in the change logs of boost.test:
https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/change_l...
Anything after -- is passed to the master test suite argc/argv and is available to your test module.
That makes sense.
I will update the doc on the two points above to make it clearer.
Thank you. Please consider adding a top-level section on passing command-line arguments, perhaps under "Runtime parameters". I believe that's a common enough use-case that it might be worthwhile for this not to be hidden in a (seemingly unrelated) section about the master test suite.
Yes, will do, I regularly have questions about this part.
Thanks for your help !
You're very welcome.
On 2018-09-13 02:53 PM, Raffi Enficiaud via Boost wrote:
On 13.09.18 00:50, Stefan Seefeld via Boost wrote:
On 09/12/18 14:28, Raffi Enficiaud wrote:
On 12.09.18 00:21, Stefan Seefeld via Boost wrote:
* What (non-deprecated) API can I use to access command-line arguments during test initialization ?
By "test initialization", do you mean before the first test starts?
Yes.
The argc/argv are available to the master test suite:
https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/tests_or...
I see. That code could use a bit of an explanation. You present a function that takes "argc" and "argv" as input, but rather than using those variables themselves, you access the argument vector via "framework::master_test_suite().argc" etc.. I find that a bit...em... counter-intuitive. What's the rationale for that ? Is the function argument the full argument vector, including the ones already consumed by Boost.Test itself ?
Apart from the historical side of this design, this has two benefits IMO: 1- the argc/argv passed to the rest of the test module at runtime may have been altered, either by the test module itself (dropping some params in a global fixture), or by the boost.test framework. The argc/argv seen by the test module are the ones that appear after the -- 2- the master test suite is a singleton, so you do not need to pass around argc/argv
And what about the return value ? What does it mean to return 0, or something else ? Thanks, Stefan -- ...ich hab' noch einen Koffer in Berlin...
On 14.09.18 20:12, Stefan Seefeld via Boost wrote:
On 2018-09-13 02:53 PM, Raffi Enficiaud via Boost wrote:
On 13.09.18 00:50, Stefan Seefeld via Boost wrote:
On 09/12/18 14:28, Raffi Enficiaud wrote:
On 12.09.18 00:21, Stefan Seefeld via Boost wrote:
* What (non-deprecated) API can I use to access command-line arguments during test initialization ?
By "test initialization", do you mean before the first test starts?
Yes.
The argc/argv are available to the master test suite:
https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/tests_or...
I see. That code could use a bit of an explanation. You present a function that takes "argc" and "argv" as input, but rather than using those variables themselves, you access the argument vector via "framework::master_test_suite().argc" etc.. I find that a bit...em... counter-intuitive. What's the rationale for that ? Is the function argument the full argument vector, including the ones already consumed by Boost.Test itself ?
Apart from the historical side of this design, this has two benefits IMO: 1- the argc/argv passed to the rest of the test module at runtime may have been altered, either by the test module itself (dropping some params in a global fixture), or by the boost.test framework. The argc/argv seen by the test module are the ones that appear after the -- 2- the master test suite is a singleton, so you do not need to pass around argc/argv
And what about the return value ? What does it mean to return 0, or something else ?
I updated the documentation with a new section about runtime parameters (in master), with a few examples. It should appear in the 1.70 release or in the next cycle of doc update for the beta. Raffi
participants (3)
-
Olaf Peter -
Raffi Enficiaud -
Stefan Seefeld