Klemens Morgenstern wrote:

The documentation can be found here: https://spacetime.mireo.com/async-mqtt5/ For the record, this link seems to take me to a page that's asking me to make an account and register for a demo of SpaceTime.

I can't seem to actually access the docs. - Christian

On Fri, Oct 18, 2024 at 12:38 AM Christian Mazakas via Boost wrote:

Klemens Morgenstern wrote:

...

The documentation can be found here: https://spacetime.mireo.com/async-mqtt5/ For the record, this link seems to take me to a page that's asking me to make an account and register for a demo of SpaceTime.

I can't seem to actually access the docs.

It looks like their webserver is not ideally configured. I think you'll need to include the / at the end or add /index.html ( https://spacetime.mireo.com/async-mqtt5/index.html ) Does that work?

- Christian

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Vinnie, thanks for reviewing the library! Here are the answers to your questions. These should probably not go into official documents but rather into separate doc. Motivation We needed a robust, automotive-grade MQTT client for our commercial projects. Most of these projects require a highly stable, bug-free MQTT implementation, as our systems are typically deployed in automotive environments where any software malfunction can lead to costly repairs. Another motivation for developing an industrial-grade MQTT library was our collaboration with automotive partners who produce vehicle tracking devices. Over the years, we've encountered numerous reliability issues with these devices, most of which were related to custom implementations of network data transmission. MQTT, originally designed in 1999 to address exactly these problems, seemed like the perfect solution to help these companies resolve at least the network-related issues. Since our projects are C++-based, we were looking for an MQTT implementation that wasn’t tied to any specific commercial cloud infrastructure. For instance, Amazon AWS offers its own MQTT client, but it's deeply integrated into the AWS ecosystem and isn't practical for use outside of it. We analyzed the two most popular open-source C-based clients, Eclipse Mosquitto and Paho. While these libraries are mature and stable, they don’t integrate well with the asynchronous model of ASIO, which we wanted to use in our projects. We also spent considerable time trying to integrate Mr. Kondo's mqtt_cpp library https://github.com/redboltz/mqtt_cpp, which is an earlier version of his MQTT client. Although it internally uses ASIO, it exposes its functionality through simple, non-awaitable callbacks. While we were able to wrap it with an ASIO-compliant asynchronous interface, extensive testing revealed surprising bugs within the library itself. Additionally, the library's internal structure was highly complex, making it extremely difficult to understand and maintain. Given that MQTT is relatively straightforward to implement, we decided to write our own MQTT client that would be fully ASIO-compliant and rock-solid. Implementation The Async.MQTT5 library was deliberately designed with an extremely simple public interface. Notably, it automatically handles all network errors and performs reconnections when the network link goes down. From our experience, one of the most complex and error-prone aspects of working with MQTT is managing the protocol's behavior during client reconnections to the broker. The three libraries we mentioned earlier leave connection error handling to the user, which inevitably complicates the user’s code and increases the likelihood of errors. By eliminating this problematic aspect, the library becomes straightforward and easy to use, as long as the user follows fundamental Asio guidelines. That said, the Async.MQTT5 library sees about 200 (non-unique) daily clones, with only 17 issues reported so far since its initial release. We take this as a positive sign of the library’s stability. Async.MQTT5 has been in use for about a year, since the fall of 2023. We specifically measured the performance in terms of PUBLISH message throughput. We hit the broker's limit (where the broker could no longer handle the volume of messages) while publishing 2KB payloads from a Raspberry Pi 4 on a 1GBit/s local network. Apart from that, we did not perform other benchmarks. HiveMQ is probably the most known user. Rimac is also quite well known. Pixie AS is another one. I believe it should be possible to get review of the library from those and other companies, although it would need some time.

On 20.10.2024., at 18:15, Vinnie Falco via Boost wrote:

On Tue, Oct 15, 2024 at 11:16 PM Klemens Morgenstern via Boost < boost@lists.boost.org> wrote:

...

The library has been developed by mireo, a GPS/GIS software development company, and their CTO Ivica Siladić submitted it.

Ivica, thank you for submitting this library.

In my opinion, the most important quality of a submission is its usage in production systems or in other projects. I prefer libraries to demonstrate utility by gaining popularity or usage well ahead of the review.

Can you please explain the history of this library? Example questions:

* What was the motivation for writing async-mqtt? * Did you first try using other already-existing libraries and if so, what was the experience?. If not, then why not? * How long has async-mqtt been in production? * Are there any benchmarks or performance metrics? * What are notable experiences with the production usage? * What other companies or projects are using async-mqtt? * Can we reach out to the other users and get their feedback (or better yet, a review)?

This is important, and I believe this information should be in the documentation of the library and permanently available. Furthermore, if there are other mqtt libraries which use Asio (or even in C++) I would like to see a comparison of differences between async-mqtt and the best 1 or 2 competitors. For example, have a look at this information from Beast:

https://www.boost.org/doc/libs/1_86_0/libs/beast/doc/html/beast/design_choic... https://www.boost.org/doc/libs/1_86_0/libs/beast/doc/html/beast/design_choic... https://www.boost.org/doc/libs/1_86_0/libs/beast/doc/html/beast/design_choic...

I think I should mention that there's another mqtt library that has

...

been proposed for boost, but has yet to find a review manager: https://github.com/redboltz/async_mqtt

The documentation for mireo/async_mqtt should include a section which compares itself to redboltz/async_mqtt. This is useful whether or not the Mireo library is accepted, as users who are looking for an MQTT library will be more well-informed by this data.

As a side note, the author of redboltz/async_mqtt should also do all of the above, if not already done.

Thanks

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Thanks, Ivica Siladic

Klemens, I have reviewed the documentation for the MQTT5 library and here are my comments - hope they are useful! Overall there is a lot of technical documentation, a decent range of examples, and a lot of linking to internal and relevant external topics - all of which is good. I agree with comments above that the "Introduction" is weak and could use some work. The first paragraphs of an introduction should critically answer the "why" question - "why should I be interested in this library?". The "what" question should be answered, as it briefly is, but is a lot less important than the why question. Another interpretation of the why question is "what does this do for me?" and a detailed answer is good and seems to be addressed in some of the email responses above. The API reference is good and fairly comprehensive - though I might ask if there is enough information on the function calls. In particular I would like to know what error messages/exceptions can be thrown by which functions. In other words a table of all - or the most common - errors that might be the response to a function call would be good information to add to the page on each function - which at the moment is a tad thin on content. Secondly, a link to an example containing use of that function would be useful - should an example exist. When reading the Introduction I came up with the following nits, though the comments apply throughout the documentation. Nits The following are minor grammatical/spelling issues: 1. remove the word "very" from the following sentence as it adds nothing: *The aim of Async.MQTT5 is to provide a very simple asynchronous C++ interface for application developers.* 2. Add a sentence after this one, describing what the "*compatible with completion tokens*" implies - what does this enable (otherwise what goes through the reader's mind can be "so what?"): *The Client's asynchronous functions are compatible with all completion tokens supported by Boost.Asio.* 3. Consider changing the "Features" section to a table, with two columns with headings "Feature" and "Description". The first column is the feature (such as *Support for QoS 0, QoS 1, and QoS 2*) the second is a description including links. This will force the authors to add descriptions to the features which currently do not have one (3 at present). The focus of the description should be to answer the question "why is this useful?" rather than what it is. 4. Similarly to the use of the word "very" described above, the word "simple" adds pretty much nothing to the understanding of a sentence - consider removing "simple" from: *The following example illustrates a simple scenario of configuring a Client and publishing a "Hello World!" Application Message with QoS 0.* [consider searching all the docs for "simple", "very" and removing them - just makes the text sharper and easier to read] [great to have this example, and a link to more examples] 5. Consider moving the "*When to use*" section up to the top as a subheading under "*Introduction*" 6. Consistently use "*an MQTT*" and not "*a MQTT*". This is because M is pronounced "em" (so it starts with a vowel in the mind of the reader). Mostly you have used "an" but not consistently as here: *Your application uses Boost.Asio and requires integrating a MQTT Client.* [again, search the docs for "a MQTT" and update to "an" appropriately] 7. Consider changing the "*Requirements*" heading to "*Getting Started*" and move it above the "*Examples*" section - as it contains essential information to get anything going. Also add this info from Git if is current: Using the library https://github.com/mireo/async-mqtt5/#using-the-library 1. Download Boost https://www.boost.org/users/download/, and add it to your include path. 2. If you use SSL, download OpenSSL https://www.openssl.org/, link the library and add it to your include path. 3. Add Async.MQTT5's include folder to your include path. 8. Remove the word "Only" - all the necessary meaning is provided by the word "If": *Only if you require an SSL connection by using boost::asio::ssl::stream.* 9. Perhaps add a link to the Boost license if that is the license that is pertinent, or a link to the correct license if not. 10. Consider consistently using Title Capitalization in all headings throughout the docs - does look better than the current Sentence Caps, so: *Using the library*, becomes *Using the Library* 11. Perhaps comment on whether the library is "modularized" and fits neatly into the Boost super project. 12. Is the only dependency Boost.asio? If there are other dependencies that are not sub-dependencies of Boost.asio, perhaps list them. 13. Consider commenting on performance - are there other similar libraries out there you have compared perf with? All I have for now. For the record I am the Technical Writer for C++ Alliance. best - Peter Turcan On Sun, Oct 20, 2024 at 11:12 AM Vinnie Falco via Boost < boost@lists.boost.org> wrote:

On Sun, Oct 20, 2024 at 10:53 AM Ivica Siladic wrote:

...

Here are the answers to your questions. These should probably not go into official documents but rather into separate doc.

Thank you so much.

This information is very relevant and I think it should be in the official documentation (that is, the library HTML documentation which is part of every release). Assuming mqtt5 is accepted, I prefer people who evaluate Boost to see that it has a best-of-class MQTT client, as IoT is an area where C++ has considerable strengths.

Exposition which reveals the mind of the authors and their design principles greatly enhances the perception of the library's value. Knowing it is used in production environments increases confidence that the library will be actively improved and maintained.

Authors who propose libraries in the future who are reading this take note: these things make the submission look better :)

Thanks

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On Wed, 23 Oct 2024 at 12:11, Klemens Morgenstern via Boost < boost@lists.boost.org> wrote:

On Wed, Oct 16, 2024 at 2:16 PM Klemens Morgenstern wrote:

...

the formal boost review of the async-mqtt5 library strats today, the 16th of October and will last until and including the 25th of this month.

With permission by the Review Wizards, the review period is extended until Monday, the 28th.

On behalf of the Review Wizards, I'm happy to confirm that. I'd like to encourage everyone to contribute a review: - the Authors of the library will appreciate your feedback. - the Review Manager work will be easier to complete Best regards, -- Mateusz Loskot

Thank you sincerely, Vinnie, for your detailed review and the considerable effort you’ve put into understanding how MQTT works! We're, of course, really happy that you believe the library should be accepted into Boost.

On 23.10.2024., at 22:51, Vinnie Falco via Boost wrote:

On Tue, Oct 15, 2024 at 11:16 PM Klemens Morgenstern via Boost < boost@lists.boost.org> wrote:

...

Please explicitly state that you either *accept* or *reject* the inclusion of this library into boost.

What follows is my review of the async-mqtt5 library by Ivica Siladic (through Mireo).

I believe this library should be ACCEPTED.

I spent about 20 hours looking at the documentation, the source code, Internet resources related to the MQTT protocol, and talking with the author. I was not able to compile the library and examples, for reasons related to the dependency on OpenSSL.

In my opinion this submission represents the gold standard of library proposals. It has been deployed by multiple companies in commercial settings for over a year. It has received considerable field testing and bug fixes. While not perfect, the API clearly works and delivers value for users. In short, async-mqtt5 has proven itself outside of Boost.

The library could use improvement in the documentation, the example code, the APIs, and the implementation that would bring it up to an outstanding level of quality. I struggled quite a lot with deciding whether or not some of these improvements should be conditions of acceptance and ultimately I have concluded that Boost would be better with this library in it as the library is currently implemented, rather than not.

Writing out a list of conditions is laborious and I aborted two attempts at doing so. Furthermore, the nature of some of the improvements are quite complicated and require experimentation, measurement, and iteration. It doesn't make sense to hold out for a "perfect" submission.

Furthermore a good MQTT client implementation is necessarily complex; the interaction of reading and writing combined with quality of service and server behavior is tricky to get right. Optimizing such an implementation requires making a lot of correct decisions regarding tradeoffs. I have rough ideas for improvements, and only the author can explore them in a way that preserves the unique features that MQTT clients need.

ANSWERS

Q: Does this library bring real benefit to C++ developers for real world use-case? A: It obviously does. MQTT is the industry-wide peer to peer communication solution for the Internet of Things ("IoT"). This library has already benefited its users for over a year.

Q: Do you have an application for this library? A: I personally do not have a use for this library. However I will note that the use of async-mqtt5 would be a natural fit for a reimplementation of Beast's websocket chat server.

Q: Does the API match with current best practices? A: I personally am unfamiliar with MQTT and IoT best practices. Yet I am confident that it either matches or will match best practices, as the pressures from its commercial usage create the necessary incentives to do so. I have some concerns that the

Q: Is the documentation helpful and clear? A: The documentation needs a lot of work.

Q: Did you try to use it? What problems or surprises did you encounter? A: I wanted to build it locally but cmake and vcpkg conspired against me. My willingness to offer acceptance despite the inability to build the thing comes from the existence of actual users prior to submission.

Q: What is your evaluation of the implementation? A: The implementation is good, not great. More importantly, the author is skilled with Asio and he appears to have his wits about him so I would bet that the support and improvement for this library should it become part of Boost, will be very good.

Q: Do you have experience with asio? A: I am an expert with Asio, and I have over 30 years of experience writing concurrent network programs.

Q: Did you work with MQTT in the past? A: I did not work with MQTT in the past, but I have heard about it. Eight years ago Michael Caisse presented a talk at CppCon 2016 on it, yet the promised code was never published.

NOTES

The implementation looks very complicated, just like the MQTT protocol. I have confidence that it mostly does the right thing, since it is used in production. There are a lot of memory allocations and a lot of intermediate structures, and I wonder if this could be improved.

The implementation queues both incoming and outgoing packets, and these queues appear to be unbounded. My experience tells me this is a problem, as the lack of backpressure subverts TCP's application-level flow control. Yet I do not have evidence that this is a problem in practice. A lot of time spent in my review is struggling over whether to demand improvements here or to just let it be.

I do not know how MQTT is used in the IoT, and a refactoring of this library's APIs to handle backpressure would be an enormous undertaking. I have a lot of ideas about how to reduce the allocations, and make backpressure available to the application but these need experimentation and design work to fit into the protocol. I trust that the author will explore these in his own time.

The async_run interface might be unnecessary. A typical program using mqtt_client will always have an outstanding call to async_receive, and periodically call async_publish. In a fashion similar to beast websocket, it is possible to piggyback the low level reading and writing on top of the async_receive and async_publish calls. If there are no outstanding calls to those functions, then the implementation will effectively stop communicating. And this might be a good thing for better support of application level TCP flow control.

The library dedicates a lot of ceremonies to allocators, and I appreciate the effort. However there are no motivating use-cases and no measurements. Given that there are so many calls to allocate, in the current state I would not advertise the allocator support so much. A refactoring of the implementation to cause fewer calls to allocate and deallocate may make allocators relevant to performance. Disclaimer, I have also not done any measuring.

The async_receive API only returns one message at a time, which is probably inefficient. A better technique would be to read as much data from the socket into a large buffer as possible in a single I/O, extract all complete messages into a range, and return that as a batch to the caller. I am of course glossing over many important details such as QoS, the need for ACK packets in response, and so on, and this idea will need to be explored further and measured to determine if it is practical. Is the necessary? I don't know.

Header files should be included in the order of most specific to most generic. That is, library specific (async-mqtt5) headers first, then dependencies (asio, boost), and then standard library headers. This helpfully increases the chances of getting a compiler error when a header is missing.

None of the examples work when you run them, because the broker "<your-mqtt-broker>" does not exist. I'm not sure what will happen if you run them as-is, but I would guess that since the library does infinite reconnection attempts with no logging, it will appear to hang.

Publishing, receiving, and properties are core aspects of MQTT yet these words do not appear anywhere in the table of contents. Nor do the subjects appear by other names.

Using operator[] instead of get<> for properties is unusual, as this creates something which looks like an array where each index holds a different typed object. I'm not going to get too worked up about it, since according to the author properties are an advanced usage. The implementation is in sore need of comments especially in the more complex areas where it is not clear which executor or which allocator is being used.

Examples need to have command line arguments for brokers and credentials. Otherwise, default to HIveMQ's public demo broker here https://www.mqtt-dashboard.com/. Running the example with no arguments could show a "usage" string which shows the HiveMQ broker as choice for broker

Table of Contents needs to have "Reading Messages", "Publishing Messages", and "Using Properties" as list items.

The documentation needs a section which explains the MQTT protocol at a high level. How it is used, the client and broker roles, and links to more supporting documentation and information. This will help users who come across on async-mqtt5 that do not have any knowledge of the protocol, to determine if the library is a fit for their needs. One common opinion is that explaining MQTT in the documentation is out of scope. Is isn't strictly wrong for an introduction to the protocol to be delegated to outside sources, yet it is better for Boost if this is included.

More clear documentation regarding properties. How commonly used they are, why they are used, and especially to point out how the indexes to operator[] result in objects of varying types depending on the index.

More information in the documentation is strictly better than less. Things like design notes, rationale for trade offs, references to where the library is used, benchmarks, and so on all have value and including this distinguishes the library from its competitors.

As pointed out in other reviews, the beast dependency should be optional and the TlsContext specification should be better, or the feature of TLS should be refactored.

CONCLUSION

Overall this is a good, working library which has considerable room to become great. Boost is better off with it than without it.

Thanks to the author for investing the time to prepare the library for review.

Regards

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Thanks. Ivica Siladic

Thanks for the suggestions and encouraging me to clarify. On 28/10/2024 6:37 am, Marcelo Zimbres Silva wrote:

Hi,

On Sun, 27 Oct 2024 at 09:53, Darryl Green via Boost mailto:boost@lists.boost.org> wrote:

This is another face of the "it needs logging" comments from other reviewers. If the user can be informed what the library is doing, then the user can simply log this, or, potentially, react in other ways.

I would like to mention that depending on how it is implemented, logging will also solve the "notification" problem. In Boost.Redis, for example, the async_run function gets a lightweight log object with the following interface

   class logger {    public:       void on_resolve(...);       void on_connect(...);       void on_ssl_handshake(...);       void on_connection_lost(...);       void on_write(...);       void on_read(...);       void on_run(...);       void on_hello(...);       void on_check_health(...);       ...    };

This is primarily meant for logging but can also be used to communicate events to the upper layer. I must admit though that I don't find this approach scalable, there are too many places that need logging and you don't want to extend the class with new functions every time. Also, there does not seem to be any use for this except logging, so why not use an interface with trace(), info(), warning(), critical() etc. instead.

I simply meant that its somewhat like logging. Im not sure if on_<everything> is appropriate but being able to react to connectivity changes seems pretty fundamental.

...

...

- Does the API match with current best practices?

If we took Beast as an example of best practice, probably not.

I would like to understand how you come to this conclusion, because it is low-level Beast requires users to perform steps manually: resolve, connect, handshake, write, read etc. Also, if anything more advanced like http pipelining is needed you can expect to write a lot more code, at least compared to popular http libs written in go, node-js etc. The library being reviewed on the other hand hides all those steps plus the multiplexing of multiple async_publishes behind a single call async_run function, this is extremely convenient IMO.

Sometimes you just want to do a "it works or it doesn't" operation in, of all things, an MQTT client. That said, since posting my review I saw in other discussion that there is the option to use total cancellation at the async publish level. I'd only seen examples and docs for some destructive cancellation approaches. This alone might be enough to address some of the problems while using only a high level interface.

...

As an interface for implementing latency sensitive real time distributed systems, probably not.

What part of this library exactly is this referring to? I agree it has room for improving latency, for example, the async_receive signature reads ... IMO a better solution would be to allow users direct access to the buffer so he can avoid copies, this is how Boost.Redis and IIRC Boost.Mysql works.

I wasn't referring to the performance of the library code, I was as in all my comments referring to the performance / service offered by a system that includes an application using the library code and that is trying to deliver a particular type of service. Consider a client that has a large, active and complex enough collection of topics to publish that the performance features we are talking about here are of benefit. In some cases not only is low latency in the sense of ability to utilise the large capacity channel important, but so is constraining latency when the rate of publication attempts is actually higher than the capacity of the channel. When and what to "leak" becomes important. Consider an intermittently or "weakly" connected but very active client publishing thousands of topics, that normally get queued and delivered to some remote server (it may or may not be a "broker" in the conventional sense). If even with those features achieving high efficiency / throughput the rate of publication is such that the latency from the publisher to the the server grows due to channel capacity issues (slow channels and in the extreme case, channels that are intermittently unavailable) then for some groups of topics it is likely appropriate and even necessary to drop old/stale "queued" publications in favour of making sure the server remains up to date (at the loss of some visibility / temporal resolution regarding what are now "historical" values/changes). Note that it is likely there are other topic groups for which delivering every publication is important. MQTT and the library claim to support multiplexing topics and messages with different QoS. One common usage/expectation is that QoS 0 messages will be delivered "fast" but that the eventual receiver of them will not see every message (as one could expect from calling this at_most_once). Meanwhile a message that must be delivered exactly once.. is slow(er).. but not expected (or is it) to get "stuck" indefinitely behind a (growing?) set of QoS 0 messages. Id like to see how one can use the library in a system with this type of performance requirement. Id also like to se how it can be used with QoS 1 and a constrained window used specifically to stop the "channel" (TCP stack and general buffer bloat) blowing out latency - its not in practice safe to assume latency sensitive but relatively message loss insensitive traffic should be sent with "QoS 0" in MQTT terms. It is important that if one does intend to drop messages (which with a TCP transport needs to happen prior to the network stack) that this can be done "fairly". A high level library that can do all this needs a tunable "QoS" in the general sense (not just the selection of one of 3 values for an MQTT parameter). A low level library can legitimately claim its out of scope. This library has high level features but appears to lack some tunability (but that could be user ignorance on my part) and also access to low level features to allow the user to take responsibility for these issues. That needs addressing. Ive also seen some informed discussion around whether a library should be accepted exactly so that the author is encouraged to modify and improve it. I am all for encouraging authors to produce high quality, useful libraries and my "reject at this point" is using the "reject" word because that is the only "non accept" word on offer unless one can formulate a detailed list of conditions for "conditional acceptance" - which are as Vinnie already mentioned is hard for a library like this. Regards Darryl.