Re: [Boost-users] Server application won't bind to ports on IP address other than my internal ones
I just tried commenting out the POST request handling code and the client_session stuff, and then tried to run it with my external IP, but the same error from bind() came up saying that the requested address is not valid in its context. I also tried using the wildcard IP 0.0.0.0 as the command line argument for the address when running the server and entering my external IP into the browser address bar, but I still couldn't visit the app in the browser (can't get to it either way). The problem is probably in the server code, and is because of Asio (probably). I need some advice on how best to confirm this and troubleshoot the problem. Thanks in advance.
What is more likely? A. Asio, a library written by one of the most talented people on the planet, maintained for 15 years and running in millions of servers around the world has a bug in it that only you have found? B. You have an incomplete understanding of how to deploy network servers behind a firewall. Take all the time you need. On Thu, 10 Jan 2019, 16:50 Osman Zakir via Boost-users < boost-users@lists.boost.org wrote:
I just tried commenting out the POST request handling code and the client_session stuff, and then tried to run it with my external IP, but the same error from bind() came up saying that the requested address is not valid in its context. I also tried using the wildcard IP 0.0.0.0 as the command line argument for the address when running the server and entering my external IP into the browser address bar, but I still couldn't visit the app in the browser (can't get to it either way). The problem is probably in the server code, and is because of Asio (probably). I need some advice on how best to confirm this and troubleshoot the problem. Thanks in advance. _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
It's definitely the second one. I don't even need all the time in the world to figure that out. But why is bind() telling me that my external IP address is invalid? How do I find out what happened? I'm no expert in network programming, so I need some help here.
On 10/01/2019 18:21, Osman Zakir via Boost-users wrote:
It's definitely the second one. I don't even need all the time in the world to figure that out. But why is bind() telling me that my external IP address is invalid? How do I find out what happened? I'm no expert in network programming, so I need some help here.
Perhaps you are behind a NAT router and are using port forwarding to route traffic to and from your deployment host. That host will have no network interfaces with the address that is your publicly visible IP address so you cannot bind to that address, but you do not need to, just bind to the address of a network interface that is connected to the router, the NAT translation and port forwarding will do the rest for you. You can also, more easily, bind to the wildcard address and let your local router work out how to route traffic. Regards Bill Somerville.
On your internal server in a cobsile, type: netstat -an | grep inet Note the IP addresses. These are the only addresses available on your server. You are not supplying one of these addresses to bind via the endpoint. Therefore, error. On Thu, 10 Jan 2019, 19:21 Osman Zakir via Boost-users < boost-users@lists.boost.org wrote:
It's definitely the second one. I don't even need all the time in the world to figure that out. But why is bind() telling me that my external IP address is invalid? How do I find out what happened? I'm no expert in network programming, so I need some help here.
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
Thanks for that, but then why is it not binding to a port when Heroku tries to bind it? Is Heroku also running behind a firewall on my computer?
________________________________
From: Boost-users
On 11/01/2019 07:35, Osman Zakir wrote:
Thanks for that, but then why is it not binding to a port when Heroku tries to bind it? Is Heroku also running behind a firewall on my computer?
Define what you mean when you say "Heroku". Because a quick Google says that's a cloud platform, which suggests it doesn't run on your computer at all. In which case you would have to run Richard's command on the cloud machine, not your own machine. Normally in the server code you should just bind to 0.0.0.0 always, and let the network configuration figure out which port to make externally accessible. It only really matters when you're trying to write a DMZ app that provides different content or security for different network routes on a multi-homed server, or similar discretionary cases. Read a good book on how networks work.
I wrote a Dockerfile and tried to deploy it to Heroku (they accept Dockerfiles), but it wouldn't bind to any port they tried to bind it to. My server application requires an IP address and a port number as two of the command line arguments required to run it (it actually requires more than two; they are: IP address, port, docroot, number of threads). I set up port forwarding on my router for port number 8443. Then I tried to use that port with my app. I tried passing in 0.0.0.0 as the IP address and entered my external IP address in the browser's address bar, but I saw an error page. On MS Edge, I get that "Hmmm...can't reach this page" error page. And when I click on "Details", I see an error message saying, "This website cannot be found." Error code is "INET_E_RESOURCE_NOT_FOUND". I also tried to run it by giving my external IP address for that command line argument, but of course I got that error from accelerator::bind() about it being invalid in its context. The server app didn't exit, but I did get that same error I mentioned when I tried to visit my external IP on port 8443 in my browser. I wrote the Dockerfile for a synchronous version of the app, and the current one I have is asynchronous (I'm running it with 2 threads because that's apparently all my processor can take since it has two cores). I made it asynchronous because I wanted to port it to WebAssembly, but Jinja2Cpp and Asio are giving me problems on that so I decided to not do it (Jinja2Cpp doesn't work well with Clang on Windows and Emscripten uses LLVM, and there's an Emscripten port for standalone Asio but it's an earlier version). I don't like how the asynchronous version needs me to use global variables, but there's nothing I can do about that here. Maybe I switch back to the synchronous version? I might.
It sounds as if the problem is with the edge configuration of your Heroku deployment. Is there a way to test this by deploying a different application? On Fri, 11 Jan 2019 at 17:15, Osman Zakir via Boost-users < boost-users@lists.boost.org> wrote:
I wrote a Dockerfile and tried to deploy it to Heroku (they accept Dockerfiles), but it wouldn't bind to any port they tried to bind it to. My server application requires an IP address and a port number as two of the command line arguments required to run it (it actually requires more than two; they are: IP address, port, docroot, number of threads).
I set up port forwarding on my router for port number 8443. Then I tried to use that port with my app. I tried passing in 0.0.0.0 as the IP address and entered my external IP address in the browser's address bar, but I saw an error page. On MS Edge, I get that "Hmmm...can't reach this page" error page. And when I click on "Details", I see an error message saying, "This website cannot be found." Error code is "INET_E_RESOURCE_NOT_FOUND". I also tried to run it by giving my external IP address for that command line argument, but of course I got that error from accelerator::bind() about it being invalid in its context. The server app didn't exit, but I did get that same error I mentioned when I tried to visit my external IP on port 8443 in my browser.
I wrote the Dockerfile for a synchronous version of the app, and the current one I have is asynchronous (I'm running it with 2 threads because that's apparently all my processor can take since it has two cores). I made it asynchronous because I wanted to port it to WebAssembly, but Jinja2Cpp and Asio are giving me problems on that so I decided to not do it (Jinja2Cpp doesn't work well with Clang on Windows and Emscripten uses LLVM, and there's an Emscripten port for standalone Asio but it's an earlier version). I don't like how the asynchronous version needs me to use global variables, but there's nothing I can do about that here. Maybe I switch back to the synchronous version? I might. _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
-- Richard Hodges hodges.r@gmail.com office: +442032898513 home: +376841522 mobile: +376380212 (this will be *expensive* outside Andorra!) skype: madmongo facebook: hodges.r
The error on Heroku and the MS Edge stuff I mentioned are separate things. Please don't mix them up. The error with Heroku when it can't get my server app to bind to any ports is a 404 Not Found error and in the browser it shows Heroku's own custom error page. When I said that about running the app and visiting the browser, I wasn't talking about Dockerfiles and Heroku, I was talking about just running the server app by itself on my computer. Right now that's what I have to get to work first. I'll try using a Dockerfile for it again after the port binding issue is fixed.
If you compile one of the example applications (without modifying them at all), can you connect then? If not, perhaps it's your own computer's firewall? If you're using windows you may need to relax the personal firewall security policy. If linux, then you may need to open a port on the network-facing interface (loopback interfaces are normally more permissive). But I'm guessing here. You need to get a known working program running and start from there. If the verbatim asio examples fail, the problem is your environment If they work, it's your program that's at fault. Of course it could easily be both - which is why we need to start with something we can trust. On Fri, 11 Jan 2019 at 18:33, Osman Zakir via Boost-users < boost-users@lists.boost.org> wrote:
The error on Heroku and the MS Edge stuff I mentioned are separate things. Please don't mix them up. The error with Heroku when it can't get my server app to bind to any ports is a 404 Not Found error and in the browser it shows Heroku's own custom error page.
When I said that about running the app and visiting the browser, I wasn't talking about Dockerfiles and Heroku, I was talking about just running the server app by itself on my computer. Right now that's what I have to get to work first. I'll try using a Dockerfile for it again after the port binding issue is fixed. _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
-- Richard Hodges hodges.r@gmail.com office: +442032898513 home: +376841522 mobile: +376380212 (this will be *expensive* outside Andorra!) skype: madmongo facebook: hodges.r
I'm going to try the Asio HTTP server example, then, but first I'll ask this: is the Jamfile.v2 file there for building the example? And if so, how do I run it?
________________________________
From: Boost-users
you need to either: read the docs on b2, or copy the source files into a project format of your choice. On Fri, 11 Jan 2019 at 21:43, Osman Zakir via Boost-users < boost-users@lists.boost.org> wrote:
I'm going to try the Asio HTTP server example, then, but first I'll ask this: is the Jamfile.v2 file there for building the example? And if so, how do I run it? ------------------------------ *From:* Boost-users
on behalf of Richard Hodges via Boost-users *Sent:* Friday, January 11, 2019 4:50 PM *To:* Thomas Quarendon via Boost-users *Cc:* Richard Hodges *Subject:* Re: [Boost-users] Server application won't bind to ports on IP address other than my internal ones If you compile one of the example applications (without modifying them at all), can you connect then?
If not, perhaps it's your own computer's firewall? If you're using windows you may need to relax the personal firewall security policy. If linux, then you may need to open a port on the network-facing interface (loopback interfaces are normally more permissive).
But I'm guessing here. You need to get a known working program running and start from there. If the verbatim asio examples fail, the problem is your environment If they work, it's your program that's at fault.
Of course it could easily be both - which is why we need to start with something we can trust.
On Fri, 11 Jan 2019 at 18:33, Osman Zakir via Boost-users < boost-users@lists.boost.org> wrote:
The error on Heroku and the MS Edge stuff I mentioned are separate things. Please don't mix them up. The error with Heroku when it can't get my server app to bind to any ports is a 404 Not Found error and in the browser it shows Heroku's own custom error page.
When I said that about running the app and visiting the browser, I wasn't talking about Dockerfiles and Heroku, I was talking about just running the server app by itself on my computer. Right now that's what I have to get to work first. I'll try using a Dockerfile for it again after the port binding issue is fixed. _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
-- Richard Hodges hodges.r@gmail.com office: +442032898513 home: +376841522 mobile: +376380212 (this will be *expensive* outside Andorra!) skype: madmongo facebook: hodges.r
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
-- Richard Hodges hodges.r@gmail.com office: +442032898513 home: +376841522 mobile: +376380212 (this will be *expensive* outside Andorra!) skype: madmongo facebook: hodges.r
From: Boost-users [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Osman Zakir via Boost-users
Sent: 11 January 2019 14:43
To: boost-users@lists.boost.org
Cc: Osman Zakir
Subject: Re: [Boost-users] Server application won't bind to ports on IP address other than my internal ones
I'm going to try the Asio HTTP server example, then, but first I'll ask this: is the Jamfile.v2 file there for building the example?
And if so, how do I run it?
Yes all examples should have a jamfile (but not all do and writing one is tricky).
from the jamfiles folder, say-boost\libs\asio\test
(but would be better to try a simpler library first to get your bearings)
or -boost\libs\asio\example\cpp03\iostreams
Ø b2 > my_logfile.txt
or add your special sauce
-a forces a completely rebuild (cleans first)
Ø b2 a toolset=gcc,msvc,clang debug release -your_other_options >my log_file.txt
NB no spaces between compilers. b2/bjam plumbs new depths in stupid syntax.
HTH
Paul
_____
From: Boost-users
I built the HTTP server example from Asio and tried to run it. Same problem. I set up port forwarding for port 8080, passed 0.0.0.0 as the address in command line arguments and tried to visit my external IP address in my browser (wrote an HTML page to act as a landing page). Can't reach the app, as with my own server application. So either it's both of the possibilities mentioned before, or just that firewall needs to configured properly. Except it already seems like the firewall is allowing both applications through. Maybe there's something in the configuration that I'm still missing, though.
________________________________
From: Boost-users
On 11/01/2019 21:42, Osman Zakir via Boost-users wrote:
I built the HTTP server example from Asio and tried to run it. Same problem. I set up port forwarding for port 8080, passed 0.0.0.0 as the address in command line arguments and tried to visit my external IP address in my browser (wrote an HTML page to act as a landing page). Can't reach the app, as with my own server application. So either it's both of the possibilities mentioned before, or just that firewall needs to configured properly. Except it already seems like the firewall is allowing both applications through. Maybe there's something in the configuration that I'm still missing, though. ------------------------------------------------------------------------
If you are behind a NAT router then I don't think you can access a service port at the external IP address the NAT is actually doing translation for. To test within the LAN behind the NAT router simply use the local host address, either the loopback if you are on the server host or via the server host LAN address. The port forwarding is only valid for incoming external traffic to the NAT router. This is one good reason why your server should bind to the wildcard address so that you can test it via any of it's host's addresses. Regards Bill Somerville.
I bound it to the wildcard address in the command line arguments when calling the .exe (I'm using Windows). And I already know the localhost address will work. I'm trying to see if the external IP address works. I just wrote a Dockerfile for this and am building it now. Would the same thing happen with a Docker container?
________________________________
From: Boost-users
If I configure a reverse proxy on Apache for my server app, will it need to be able to see my external IP address? And yeah, I'm using Windows, so I have an NAT and a firewall. Is there a way I can let networked apps like ipconfig see my external IP address, or is it really impossible? It'd be good if I could get it to work without having to turn off NAT.
There are still 4 failed targets and 5 skipped targets when trying to build Boost. There were some "__imp_" unresolved symbols when building Boost, before, too. I wonder if they're there because of zlib.dll. If so, how do I tell b2 where that DLL is? I say this because when I built Node.js from source before, there were unresolved symbols for __imp_read, __imp_write and __imp_open, and specifying the path to the folder with zlib.dll in Visual Studio solved that problem.
________________________________
From: Osman Zakir
On 12.01.2019 11:28, Osman Zakir via Boost-users wrote:
If I configure a reverse proxy on Apache for my server app, will it need to be able to see my external IP address? And yeah, I'm using Windows, so I have an NAT and a firewall. Is there a way I can let networked apps like ipconfig see my external IP address, or is it really impossible? It'd be good if I could get it to work without having to turn off NAT.
Sir, as you have been repeatedly told in this thread your problem has nothing to do with the boost code and everything with you network environment and your lack of understanding the fundamentals of networking. Please do hit the good books on networking basics or seek the (possibly non-free) advice from a network expert. Networking focused forums and groups may also help you, albeit not without understanding the networking basics. This is a boost users mailing list, primarily focused on problems with using boost libraries and many people here do an excellent (non-payed) job on giving us advice and offering a helping hand when we are in trouble. It is disrespectful to them to keep asking the same, or similar, questions over and over again although being already told that this has nothing to do with either with the library or with your code. Also, top posting is a sign of bad etiquette. Best regards, Leon Mlakar
There are still 4 failed targets and 5 skipped targets when trying to build Boost. There were some "__imp_" unresolved symbols when building Boost, before, too. I wonder if they're there because of zlib.dll. If so, how do I tell b2 where that DLL is? I say this because when I built Node.js from source before, there were unresolved symbols for __imp_read, __imp_write and __imp_open, and specifying the path to the folder with zlib.dll in Visual Studio solved that problem. ------------------------------------------------------------------------
I was also told that the problem could be either my network environment, something in my code, or both, and I was just giving info to help with figuring out which one it might be. And notice that I did also ask about a problem with building Boost, too.
________________________________
From: Boost-users
On 12.01.2019 12:30, Osman Zakir via Boost-users wrote:
I was also told that the problem could be either my network environment, something in my code, or both, and I was just giving info to help with figuring out which one it might be. And notice that I did also ask about a problem with building Boost, too.
I did not mention your zlib problem as there is another thread "Forcing boost to build zlib" active right know that I believe should answer your question. BTW, do you know what the term "top posting" means? Cheers, Leon
There are still 4 failed targets and 5 skipped targets when trying to build Boost. There were some "__imp_" unresolved symbols when building Boost, before, too. I wonder if they're there because of zlib.dll. If so, how do I tell b2 where that DLL is? I say this because when I built Node.js from source before, there were unresolved symbols for __imp_read, __imp_write and __imp_open, and specifying the path to the folder with zlib.dll in Visual Studio solved that problem. ------------------------------------------------------------------------
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
On 12.01.2019 12:30, Osman Zakir via Boost-users wrote:
I was also told that the problem could be either my network environment, something in my code, or both, and I was just giving info to help with figuring out which one it might be. And notice that I did also ask about a problem with building Boost, too.
But those were hints for you to figure it out where the problem could be, not for us. Cheers, Leon
On 12.01.2019 12:30, Osman Zakir via Boost-users wrote:
I was also told that the problem could be either my network environment, something in my code, or both, and I was just giving info to help with figuring out which one it might be. And notice that I did also ask about a problem with building Boost, too.
Now, to ultimately check where the problem is, here's a homework for you. Write the code using the plain sockets (WinSock2 on Windows) that binds the TCP socket to the address you wish to bind your acceptor to. If the bind doesn't fail, it is a problem with the boost library, and return here for more help. If the bind fails, it is a problem outside the boost library. In this case, first read the following, then read some network introduction books and then seek help on network forums. IP addresses you see from outside your local network (possibly from the Internet) are typically not the IP addresses that belong to your local computer but are NAT mapped by some router in your environment. You mentioned that you computer has loopback and a network interface with an IP address in range 192.168.x.x. That's typical home environment. The latter address is a private number belonging to your private network and is not visible from outside, hence some piece of equipment in your environment is mapping this private number into external IP address (usually assigned by ISPs). Generally, you cannot reach a computer behind the router that uses NAT from the outside using the externally visible IP address. Unless you configure the router properly. You cannot bind a socket to the IP address that is not an IP address of a network interface on your computer. When calling bind, unless having a very good reasons against it, one typically binds to INADDR_ANY (IP address 0.0.0.0) which binds a socket to all network interfaces on your local computer. Good luck , Leon
On 13/01/2019 06:22, Leon Mlakar wrote:
Generally, you cannot reach a computer behind the router that uses NAT from the outside using the externally visible IP address. Unless you configure the router properly.
Another key misunderstanding that sounds like has been happening: If you are on a computer "inside" the network, you cannot access anything that is also "inside" the network by using the external IP address. (It's not completely impossible, but routers are normally not configured to route traffic like that.) The external IP only exists outside of the network. If you want to test it, you must test it from outside of the network. When you're inside the network you have to use the internal IP instead.
What if I set up a reverse proxy, then?
On 14.01.2019 14:30, Osman Zakir via Boost-users wrote:
What if I set up a reverse proxy, then?
To use external IP address inside your network? Nope. Reverse (HTTP/HTTPS) proxy operates at the application layer (http://bpastudio.csudh.edu/fac/lpress/471/hout/netech/layers.htm) IP addresses and routing of IP traffic are the network layer questions. I think it's doable but requires quite sophisticated hacks. And even then it really would be just for fun as it has no practical value. But seriously, why not simply bind your server socket to INADDR_ANY? Why are throwing an external IP address into the mix/mess? Cheers, Leon
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
I want to make it available to the whole Internet. Not just my network. I'm considering a reverse proxy so that my app itself can stay behind a firewall while the reverse proxy talks to the Internet.
What's INADDR_ANY? What happens if I bind to it?
________________________________
From: Boost-users
I want to make it available to the whole Internet. Not just my network. I'm considering a reverse proxy so that my app itself can stay behind a firewall while the reverse proxy talks to the Internet.
What's INADDR_ANY? What happens if I bind to it?
Try googling it. /svante
On Mon, Jan 14, 2019 at 7:37 AM Svante Karlsson via Boost-users
Try googling it.
First I would like to extend my most sincere apologies, as it seems Osman infected the Boost mailing lists using the Beast github repository as a vector. I originally learned of Osman from the issues he opened against Beast, such as: https://github.com/boostorg/beast/issues/958 I helped him at first, but then I noticed a consistent pattern. Guidance is ignored, and more questions are simply asked. Questions which could have been answered on their own, and I could tell from the repeat questions that there was little to no effort invested in finding a solution. It seems like Osman's default strategy is "ask first." Which I suppose makes sense from a game-theory perspective, if you discount to zero the value of your reputation in the community. It costs almost nothing to ask, and the payout could in theory be significant (remote as the chances are). He has been doing this for over a year, on various social networks: https://www.google.com/search?q=osman+zakir+c%2B%2B Even more amusing is that he seems to be charging $3/hour for his "advice:" https://www.upwork.com/o/profiles/users/_~01d8016292466f5900/ Is he delegating his paid consulting work to the community of volunteers? Who knows. I try to be as helpful as possible but in this case it is quite clear that Osman is a "help vampire:" http://slash7.com/2006/12/22/vampires/ As such, I have taken the step to ban Osman from my personal GitHub account. I am making this known publicly because of the significant investments that are being made in answering his questions. I am not advising anyone on what to do, but the volunteers in the mailing lists who generously share their time to help others should at least be fully informed. Regards
I searched about it on Bing. How will it help me with making the app available to everyone on the Internet? Doesn't it only help bind to any address and port on the internal network?
On 14.01.2019 18:42, Osman Zakir via Boost-users wrote:
I searched about it on Bing. How will it help me with making the app available to everyone on the Internet? Doesn't it only help bind to any address and port on the internal network?
It doesn't. Since you can't bind to anything else but to your local network address. But it helps you to avoid failed bind due to invalid IP address. Now that we established that there is nothing wrong with the Boost library in question (and everything with your knowledge on the topic), next is to configure your router to let external requests into the local network. Hence hit the texts on networking. Here are two starting pointers: https://en.wikipedia.org/wiki/Port_forwarding https://en.wikipedia.org/wiki/Dynamic_DNS If you decide to skip the reading and go straight to asking people, please do it elsewhere. The topic is way out of scope of this mailing list. Good luck, Leon
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
@Vinnie Falcomailto:vinnie.falco@gmail.com My Upwork account is there so I can get work when I'm ready as a programmer. It's not for charging fees for what you're thinking of. And I guess my mistake was not actually posting back to give updates on the problem, but I did try on my end. Anyway, yeah, I'll go to that link and read now.
It seems like the problem is solved, at least for me. I can get to the app with my external IP address now when I use the port I forwarded. But it seems like other people still can't get to it. I turned off the server for now so it won't work even if it would've, but I'll give the IP address and port here anyway: http://39.48.207.154:8443/
________________________________
From: Osman Zakir
The problem is solved and now I can get to my server app using my external IP address on the port I forwarded on it, but it seems like other people still can't. I've turned off the server so it won't work now even if it would've, but I'll give the IP address and port needed to connect to it here anyway: 39.48.207.154:8443
participants (8)
-
Bill Somerville
-
Gavin Lambert
-
Leon Mlakar
-
Osman Zakir
-
Paul A. Bristow
-
Richard Hodges
-
Svante Karlsson
-
Vinnie Falco