This is a simple trick I use to debug connection issues with the mule app deployed to CloudHub.

First of all its important to understand that there are different ways to set up your application

1. Using shared load balancer (i.e. cloudhub.io domain) with HTTP protocol.

In this scenario, the client application makes a request to a shared load balancer URL using HTTP protocol. The SLB forwards the request to the upstream mule application listening on port 8081.

Note that in this scenario the SLB URL receives the request on port 80, and forwards it to mule application listening on port 8081.

2. Using shared load balancer (i.e. cloudhub.io domain) with HTTPS protocol.

In this scenario, the client application makes a request to a shared load balancer URL using HTTPS protocol. The SLB forwards the request to the upstream mule application listening on port 8082.

Note that in this scenario the SLB URL receives the request on port 443, and forwards it to mule application listening on port 8082.

3. Using dedicated load balancer (i.e. vanity domain) with HTTPS protocol.

In this scenario, the client application makes a request to a dedicated load balancer URL(exposed by a vanity domain) using HTTPS protocol. The DLB forwards the request to the upstream mule application listening on port 8091.

Note that in this scenario the DLB URL receives the request on port 443, and forwards it to mule application listening on port 8091.

4. Using dedicated load balancer (i.e. vanity domain) with HTTPS protocol.

In this scenario, the client application makes a request to a dedicated load balancer URL(exposed by a vanity domain) using HTTPS protocol. The DLB forwards the request to the upstream mule application listening on port 8092.

Note that in this scenario the DLB URL receives the request on port 443, and forwards it to mule application listening on port 8092.

5. Not using load balancers.

This is one of those scenarios where you are not using Anypoint Platform load balancers. You are either using third-party load balancers, or your applications are using non HTTP protocols like a simple socket, or MLLP. In this case, the client application makes a request directly to the application (mule-worker).

As seen above, to make your application endpoint available to the clients

• When using a shared load balancer, your application should be listening on an appropriate port (8081 or 8082)
• When using a dedicated load balancer, your application should be listening on an appropriate port (8091 or 8092), and the load balancer’s upstream HTTP protocol should be configured properly.
• The VPC firewall should be configured properly to allow the ingress traffic on the appropriate port.

If there is a misconfiguration, your application endpoint will not be accessible. Your clients will get errors like ‘Bad Gateway’, or ‘Timeout’. The easiest way to debug these types of errors is to use netcat to perform port scanning and identify open ports on your application.

For example, to check if your application hello-httpbin is listening on port 8081 you can use

  nc -w 1 -G 1 -z -v mule-worker-hello-httpbin.cloudhub.io 8081

  Connection to mule-worker-hello-httpbin.cloudhub.io port 8081 [tcp/sunproxyadmin] succeeded!

In this you can see, the application name is prefixed with mule-worker, to get the correct DNS name. For more details about this convention, please see the DNS records section in Cloudhub Networking Guide. The other options of netcat (or nc) are as follows

• -w: Timeout when a connection is idle for more than specified seconds.
• -G: TCP connection timeout in seconds.
• -z: Just scan for listening daemons, without sending any data to them.
• -v: Give more verbose output.

If you are not sure about the specific open port in your application, you can specify the range and identify all open ports using

  nc -w 1 -G 1 -z -v mule-worker-hello-httpbin.cloudhub.io 8081-8092
  Connection to mule-worker-hello-httpbin.cloudhub.io port 8081 [tcp/sunproxyadmin] succeeded!
  nc: connectx to mule-worker-hello-httpbin.cloudhub.io port 8082 (tcp) failed: Connection refused
  nc: connectx to mule-worker-hello-httpbin.cloudhub.io port 8083 (tcp) failed: Operation timed out
  nc: connectx to mule-worker-hello-httpbin.cloudhub.io port 8084 (tcp) failed: Operation timed out
  nc: connectx to mule-worker-hello-httpbin.cloudhub.io port 8085 (tcp) failed: Operation timed out
  nc: connectx to mule-worker-hello-httpbin.cloudhub.io port 8086 (tcp) failed: Operation timed out
  nc: connectx to mule-worker-hello-httpbin.cloudhub.io port 8087 (tcp) failed: Operation timed out
  nc: connectx to mule-worker-hello-httpbin.cloudhub.io port 8088 (tcp) failed: Operation timed out
  nc: connectx to mule-worker-hello-httpbin.cloudhub.io port 8089 (tcp) failed: Operation timed out
  nc: connectx to mule-worker-hello-httpbin.cloudhub.io port 8090 (tcp) failed: Operation timed out
  nc: connectx to mule-worker-hello-httpbin.cloudhub.io port 8091 (tcp) failed: Connection refused
  nc: connectx to mule-worker-hello-httpbin.cloudhub.io port 8092 (tcp) failed: Connection refused

In this, you can see that the application only has port 8081 open. Also interesting to note that the ports 8082, 8091, and 8092 are ‘refusing connection’, as they are allowed in the firewall but they are not exposed in the application. Whereas for the remaining ports we are getting an ‘Operation timed out’ error as they are blocked at the firewall.

The simple netcat/nc utility has saved me a lot of time. I hope it helps you as well.

The end of the year is the time to pause, look back, and think about how the year went by. I am very grateful and fortunate to had an opportunity to enjoy a reasonably good time in 2020.

On the personal front

• We visited our families in India in January. I enjoyed meeting my parents, brother, in-laws, and friends.
• I did not travel for work and got more time to spend with my wife.
• I started reading books. As a kid, I was an avid reader. The librarian in my town knew me by name. He would hold the books for me. With job and other things taking precedence over the years, I forgot about the books.
• I stopped procrastinating about writing this blog. I had some solid excuses like not having enough content to write, poor writing skills, not enough time with work-related travel, etc. I discovered that none of this matters if I can just sit down and write something.
• I re-started playing Clash of Clans. I enjoyed playing it a few years ago, but got addicted to it and burned out. During the pandemic, I wanted something other than movies to wind down. I downloaded my favorite game again and took it for a spin. I was surprised to find my old clan-mates still playing. Then I realized that I was thinking about this game all wrong. There is no end goal to it. The journey of playing is what makes it enjoyable.
• I was able to stick to my intermittent fasting routine throughout the year, even during my visit to India.
• And the best for the last. I was positively surprised by something magic. :)

On the professional front

• I got promoted to a senior position in the team, started mentoring the team members. By the end of the year, I transitioned to a more exciting role that I am looking forward to in the next year.
• I continued my learning by becoming a Salesforce Ranger, as well as became a Certified Administrator. I also completed the HL7 FHIR training and became proficient in FHIR.

What are my plans for the next year? I like to keep things simple, so I will continue to do the things I did this year already, but in a more structured way.