Re: [Boost-users] Boost SSL handshake(http_server_sync_ssl) : still reachable memory keeps increasing
Hi, I tried the async program as suggested and I still see one of the stacks where the "still reachable" memory keeps increasing with duration. I have attached the relevant stack and the program. Sorry I had to resend this email multiple times due to size limitations. Hopefully this will go through. On Fri, 4 Feb 2022 at 12:28, Sandeep Bhardwaj <sandybharblore@gmail.com> wrote:
Thanks a lot. I will give it a try.
On Thu, 3 Feb 2022 at 23:15, Vinnie Falco <vinnie.falco@gmail.com> wrote:
On Thu, Feb 3, 2022 at 9:41 AM Sandeep Bhardwaj via Boost-users <boost-users@lists.boost.org> wrote:
http_server_sync_ssl.cpp
Oh, right. Synchronous APIs have no way to time out. So if the remote host does not close gracefully (i.e. just slams the connection shut) then you will be left with a connection object which either has no way to be destroyed, or has to wait what could be a very long time (up to 2 hours) for the operating system to time out the synchronous read.
Please try the asynchronous example, http_server_async_ssl.cpp and determine if the problem persists.
Thanks
On Thu, 3 Mar 2022 at 18:25, Sandeep Bhardwaj via Boost-users < boost-users@lists.boost.org> wrote:
Hi,
I tried the async program as suggested and I still see one of the stacks where the "still reachable" memory keeps increasing with duration. I have attached the relevant stack and the program. Sorry I had to resend this email multiple times due to size limitations. Hopefully this will go through.
It's entirely possible that OpenSSL caches memory, which would be beyond the responsibility of Beast or Asio. I don't see anything controversial in your program. I see this on stack overflow, but no answers: https://stackoverflow.com/questions/56355690/valgrind-reports-memory-leak-re...
On Fri, 4 Feb 2022 at 12:28, Sandeep Bhardwaj <sandybharblore@gmail.com> wrote:
Thanks a lot. I will give it a try.
On Thu, 3 Feb 2022 at 23:15, Vinnie Falco <vinnie.falco@gmail.com> wrote:
On Thu, Feb 3, 2022 at 9:41 AM Sandeep Bhardwaj via Boost-users <boost-users@lists.boost.org> wrote:
http_server_sync_ssl.cpp
Oh, right. Synchronous APIs have no way to time out. So if the remote host does not close gracefully (i.e. just slams the connection shut) then you will be left with a connection object which either has no way to be destroyed, or has to wait what could be a very long time (up to 2 hours) for the operating system to time out the synchronous read.
Please try the asynchronous example, http_server_async_ssl.cpp and determine if the problem persists.
Thanks
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
I have added some code in on_shutdown to get the session object and free it explicitly. The same stack is not observed in the valgrind report anymore. Is this fine? any comments? void on_shutdown(beast::error_code ec) { if(ec) return fail(ec, "shutdown"); * std::cout<<"Shutting down"<<std::endl; SSL *ssl = stream_.native_handle(); SSL_SESSION *session_object = SSL_get_session(ssl); SSL_SESSION_free(session_object);* // At this point the connection is closed gracefully } On Fri, 4 Mar 2022 at 01:53, Richard Hodges via Boost-users < boost-users@lists.boost.org> wrote:
On Thu, 3 Mar 2022 at 18:25, Sandeep Bhardwaj via Boost-users < boost-users@lists.boost.org> wrote:
Hi,
I tried the async program as suggested and I still see one of the stacks where the "still reachable" memory keeps increasing with duration. I have attached the relevant stack and the program. Sorry I had to resend this email multiple times due to size limitations. Hopefully this will go through.
It's entirely possible that OpenSSL caches memory, which would be beyond the responsibility of Beast or Asio.
I don't see anything controversial in your program.
I see this on stack overflow, but no answers:
https://stackoverflow.com/questions/56355690/valgrind-reports-memory-leak-re...
On Fri, 4 Feb 2022 at 12:28, Sandeep Bhardwaj <sandybharblore@gmail.com> wrote:
Thanks a lot. I will give it a try.
On Thu, 3 Feb 2022 at 23:15, Vinnie Falco <vinnie.falco@gmail.com> wrote:
On Thu, Feb 3, 2022 at 9:41 AM Sandeep Bhardwaj via Boost-users <boost-users@lists.boost.org> wrote:
http_server_sync_ssl.cpp
Oh, right. Synchronous APIs have no way to time out. So if the remote host does not close gracefully (i.e. just slams the connection shut) then you will be left with a connection object which either has no way to be destroyed, or has to wait what could be a very long time (up to 2 hours) for the operating system to time out the synchronous read.
Please try the asynchronous example, http_server_async_ssl.cpp and determine if the problem persists.
Thanks
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
My bad. I probably spoke too soon. It is resulting in invalid write. On Fri, 4 Mar 2022 at 11:21, Sandeep Bhardwaj <sandybharblore@gmail.com> wrote:
I have added some code in on_shutdown to get the session object and free it explicitly. The same stack is not observed in the valgrind report anymore. Is this fine? any comments?
void on_shutdown(beast::error_code ec) { if(ec) return fail(ec, "shutdown");
* std::cout<<"Shutting down"<<std::endl; SSL *ssl = stream_.native_handle(); SSL_SESSION *session_object = SSL_get_session(ssl); SSL_SESSION_free(session_object);* // At this point the connection is closed gracefully }
On Fri, 4 Mar 2022 at 01:53, Richard Hodges via Boost-users < boost-users@lists.boost.org> wrote:
On Thu, 3 Mar 2022 at 18:25, Sandeep Bhardwaj via Boost-users < boost-users@lists.boost.org> wrote:
Hi,
I tried the async program as suggested and I still see one of the stacks where the "still reachable" memory keeps increasing with duration. I have attached the relevant stack and the program. Sorry I had to resend this email multiple times due to size limitations. Hopefully this will go through.
It's entirely possible that OpenSSL caches memory, which would be beyond the responsibility of Beast or Asio.
I don't see anything controversial in your program.
I see this on stack overflow, but no answers:
https://stackoverflow.com/questions/56355690/valgrind-reports-memory-leak-re...
On Fri, 4 Feb 2022 at 12:28, Sandeep Bhardwaj <sandybharblore@gmail.com> wrote:
Thanks a lot. I will give it a try.
On Thu, 3 Feb 2022 at 23:15, Vinnie Falco <vinnie.falco@gmail.com> wrote:
On Thu, Feb 3, 2022 at 9:41 AM Sandeep Bhardwaj via Boost-users <boost-users@lists.boost.org> wrote:
http_server_sync_ssl.cpp
Oh, right. Synchronous APIs have no way to time out. So if the remote host does not close gracefully (i.e. just slams the connection shut) then you will be left with a connection object which either has no way to be destroyed, or has to wait what could be a very long time (up to 2 hours) for the operating system to time out the synchronous read.
Please try the asynchronous example, http_server_async_ssl.cpp and determine if the problem persists.
Thanks
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
Yes this is too early. I would expect these functions ti be called in the destructor of your ssl stream object Can you confirm that this is actually destroyed? Maybe put a breakpoint in the destructor of asio::ssl::stream ? On Fri, 4 Mar 2022 at 08:34, Sandeep Bhardwaj via Boost-users < boost-users@lists.boost.org> wrote:
My bad. I probably spoke too soon. It is resulting in invalid write.
On Fri, 4 Mar 2022 at 11:21, Sandeep Bhardwaj <sandybharblore@gmail.com> wrote:
I have added some code in on_shutdown to get the session object and free it explicitly. The same stack is not observed in the valgrind report anymore. Is this fine? any comments?
void on_shutdown(beast::error_code ec) { if(ec) return fail(ec, "shutdown");
* std::cout<<"Shutting down"<<std::endl; SSL *ssl = stream_.native_handle(); SSL_SESSION *session_object = SSL_get_session(ssl); SSL_SESSION_free(session_object);* // At this point the connection is closed gracefully }
On Fri, 4 Mar 2022 at 01:53, Richard Hodges via Boost-users < boost-users@lists.boost.org> wrote:
On Thu, 3 Mar 2022 at 18:25, Sandeep Bhardwaj via Boost-users < boost-users@lists.boost.org> wrote:
Hi,
I tried the async program as suggested and I still see one of the stacks where the "still reachable" memory keeps increasing with duration. I have attached the relevant stack and the program. Sorry I had to resend this email multiple times due to size limitations. Hopefully this will go through.
It's entirely possible that OpenSSL caches memory, which would be beyond the responsibility of Beast or Asio.
I don't see anything controversial in your program.
I see this on stack overflow, but no answers:
https://stackoverflow.com/questions/56355690/valgrind-reports-memory-leak-re...
On Fri, 4 Feb 2022 at 12:28, Sandeep Bhardwaj <sandybharblore@gmail.com> wrote:
Thanks a lot. I will give it a try.
On Thu, 3 Feb 2022 at 23:15, Vinnie Falco <vinnie.falco@gmail.com> wrote:
On Thu, Feb 3, 2022 at 9:41 AM Sandeep Bhardwaj via Boost-users <boost-users@lists.boost.org> wrote: > http_server_sync_ssl.cpp
Oh, right. Synchronous APIs have no way to time out. So if the remote host does not close gracefully (i.e. just slams the connection shut) then you will be left with a connection object which either has no way to be destroyed, or has to wait what could be a very long time (up to 2 hours) for the operating system to time out the synchronous read.
Please try the asynchronous example, http_server_async_ssl.cpp and determine if the problem persists.
Thanks
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
-- Richard Hodges hodges.r@gmail.com office: +44 2032 898 513 home: +376 861 195 mobile: +376 380 212
Need to check that. Just to be a little more clear, my requirements for the http server are pretty basic and if it can serve just one request at a time, it would serve my requirement. With this requirement, if io_context object can be stopped and post that if we clear the ssl related stuff and then start io_context again to serve the next request, would it help in clearing the memory being held? Any thoughts? On Fri, 4 Mar 2022 at 13:29, Richard Hodges via Boost-users < boost-users@lists.boost.org> wrote:
Yes this is too early. I would expect these functions ti be called in the destructor of your ssl stream object Can you confirm that this is actually destroyed? Maybe put a breakpoint in the destructor of asio::ssl::stream ?
On Fri, 4 Mar 2022 at 08:34, Sandeep Bhardwaj via Boost-users < boost-users@lists.boost.org> wrote:
My bad. I probably spoke too soon. It is resulting in invalid write.
On Fri, 4 Mar 2022 at 11:21, Sandeep Bhardwaj <sandybharblore@gmail.com> wrote:
I have added some code in on_shutdown to get the session object and free it explicitly. The same stack is not observed in the valgrind report anymore. Is this fine? any comments?
void on_shutdown(beast::error_code ec) { if(ec) return fail(ec, "shutdown");
* std::cout<<"Shutting down"<<std::endl; SSL *ssl = stream_.native_handle(); SSL_SESSION *session_object = SSL_get_session(ssl); SSL_SESSION_free(session_object);* // At this point the connection is closed gracefully }
On Fri, 4 Mar 2022 at 01:53, Richard Hodges via Boost-users < boost-users@lists.boost.org> wrote:
On Thu, 3 Mar 2022 at 18:25, Sandeep Bhardwaj via Boost-users < boost-users@lists.boost.org> wrote:
Hi,
I tried the async program as suggested and I still see one of the stacks where the "still reachable" memory keeps increasing with duration. I have attached the relevant stack and the program. Sorry I had to resend this email multiple times due to size limitations. Hopefully this will go through.
It's entirely possible that OpenSSL caches memory, which would be beyond the responsibility of Beast or Asio.
I don't see anything controversial in your program.
I see this on stack overflow, but no answers:
https://stackoverflow.com/questions/56355690/valgrind-reports-memory-leak-re...
On Fri, 4 Feb 2022 at 12:28, Sandeep Bhardwaj < sandybharblore@gmail.com> wrote:
Thanks a lot. I will give it a try.
On Thu, 3 Feb 2022 at 23:15, Vinnie Falco <vinnie.falco@gmail.com> wrote:
> On Thu, Feb 3, 2022 at 9:41 AM Sandeep Bhardwaj via Boost-users > <boost-users@lists.boost.org> wrote: > > http_server_sync_ssl.cpp > > Oh, right. Synchronous APIs have no way to time out. So if the remote > host does not close gracefully (i.e. just slams the connection shut) > then you will be left with a connection object which either has no > way > to be destroyed, or has to wait what could be a very long time (up to > 2 hours) for the operating system to time out the synchronous read. > > Please try the asynchronous example, http_server_async_ssl.cpp and > determine if the problem persists. > > Thanks > _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
-- Richard Hodges hodges.r@gmail.com office: +44 2032 898 513 home: +376 861 195 mobile: +376 380 212
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
On Fri, 4 Mar 2022 at 12:14, Sandeep Bhardwaj via Boost-users < boost-users@lists.boost.org> wrote:
Need to check that. Just to be a little more clear, my requirements for the http server are pretty basic and if it can serve just one request at a time, it would serve my requirement. With this requirement, if io_context object can be stopped and post that if we clear the ssl related stuff and then start io_context again to serve the next request, would it help in clearing the memory being held? Any thoughts?
The io_context is a separate, orthogonal concern to the ssl::context. The io_context represents your program's io loop. Normally there would be only one per program. The ssl::context represents a common set of parameters for negotiating SSL sessions and acts as a container for those sessions. Note that these SSL sessions are not necessarily associated with any particular TCP connection - they merely represent the state of an encrypted tunnel over some transport. You would normally have one ssl::context per domain in your program. For example, a server that accepted inbound connections from the internet but then made outbound connections to a specific set of servers guarded by private keys, might use two ssl::context objects - one for all inbound connections and one for all outbound. Destroying the ssl::context and recreating is possible but I don't think it gains you anything. It might be that the context simply keeps hold of allocated memory in anticipation of re-using it. In this case, allowing it to cache memory is harmless. If the memory use grows during the lifetime of the program, then this might be more serious. But I have never experienced this. Destroying and recreating the io_context serves no purpose whatsoever. It simply means that cached services will be destroyed and re-created each time you service a request.
On Fri, 4 Mar 2022 at 13:29, Richard Hodges via Boost-users < boost-users@lists.boost.org> wrote:
Yes this is too early. I would expect these functions ti be called in the destructor of your ssl stream object Can you confirm that this is actually destroyed? Maybe put a breakpoint in the destructor of asio::ssl::stream ?
On Fri, 4 Mar 2022 at 08:34, Sandeep Bhardwaj via Boost-users < boost-users@lists.boost.org> wrote:
My bad. I probably spoke too soon. It is resulting in invalid write.
On Fri, 4 Mar 2022 at 11:21, Sandeep Bhardwaj <sandybharblore@gmail.com> wrote:
I have added some code in on_shutdown to get the session object and free it explicitly. The same stack is not observed in the valgrind report anymore. Is this fine? any comments?
void on_shutdown(beast::error_code ec) { if(ec) return fail(ec, "shutdown");
* std::cout<<"Shutting down"<<std::endl; SSL *ssl = stream_.native_handle(); SSL_SESSION *session_object = SSL_get_session(ssl); SSL_SESSION_free(session_object);* // At this point the connection is closed gracefully }
On Fri, 4 Mar 2022 at 01:53, Richard Hodges via Boost-users < boost-users@lists.boost.org> wrote:
On Thu, 3 Mar 2022 at 18:25, Sandeep Bhardwaj via Boost-users < boost-users@lists.boost.org> wrote:
Hi,
I tried the async program as suggested and I still see one of the stacks where the "still reachable" memory keeps increasing with duration. I have attached the relevant stack and the program. Sorry I had to resend this email multiple times due to size limitations. Hopefully this will go through.
It's entirely possible that OpenSSL caches memory, which would be beyond the responsibility of Beast or Asio.
I don't see anything controversial in your program.
I see this on stack overflow, but no answers:
https://stackoverflow.com/questions/56355690/valgrind-reports-memory-leak-re...
On Fri, 4 Feb 2022 at 12:28, Sandeep Bhardwaj < sandybharblore@gmail.com> wrote:
> Thanks a lot. I will give it a try. > > On Thu, 3 Feb 2022 at 23:15, Vinnie Falco <vinnie.falco@gmail.com> > wrote: > >> On Thu, Feb 3, 2022 at 9:41 AM Sandeep Bhardwaj via Boost-users >> <boost-users@lists.boost.org> wrote: >> > http_server_sync_ssl.cpp >> >> Oh, right. Synchronous APIs have no way to time out. So if the >> remote >> host does not close gracefully (i.e. just slams the connection shut) >> then you will be left with a connection object which either has no >> way >> to be destroyed, or has to wait what could be a very long time (up >> to >> 2 hours) for the operating system to time out the synchronous read. >> >> Please try the asynchronous example, http_server_async_ssl.cpp and >> determine if the problem persists. >> >> Thanks >> > _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
-- Richard Hodges hodges.r@gmail.com office: +44 2032 898 513 home: +376 861 195 mobile: +376 380 212
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
participants (3)
-
Richard Hodges -
Sandeep Bhardwaj