Most of you would already be aware of the "Connect using VMRC" option for a Virtual Machine through the vRealize Automation Self Service Console.
Its one of the most commonly used feature and i have never faced any challenges with it up until one of my customers complaint that they are getting a Black black screen while connecting with VMRC.
I immediately started thinking it towards a port problem.
VMRC through vRA uses TCP port 8444 from endpoint browser to VRA. So we tried a telnet and it worked...hence it wasn't a port issue.
So i ran a netstat command from the endpoint machine to vRA server while launching VMRC to see if the session state on 8444 port.
The session was getting to "Established" state but soon moving to "Close_Wait".
I switched my attention to the VMRC logs created in the %Temp% directory on the client machine.
The logs had it all, where we found SSL handshake error...
2017-03-15T12:18:56.850+05:30| mks| I125: SSL: EOF in violation of protocol
2017-03-15T12:18:56.850+05:30| mks| W115: SSL: connect failed (5)
2017-03-15T12:18:56.850+05:30| mks| W115: SOCKET 1 (608) Could not negotiate SSL
2017-03-15T12:18:56.850+05:30| mks| W115:
2017-03-15T12:18:56.850+05:30| mks| W115: SOCKET 1 (608) Expected thumbprint doesn't match actual thumbprint.
2017-03-15T12:18:56.850+05:30| mks| W115: Expected thumbprint is: xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx
2017-03-15T12:18:56.850+05:30| mks| W115+ Actual thumbprint is: yy:yy:yy:yy:yy:yy:yy:yy:yy:yy:yy:yy:yy:yy:yy:yy:yy:yy:yy:yy:yy
2017-03-15T12:18:56.850+05:30| mks| W115: SOCKET 1 (608) Cannot verify target host.
2017-03-15T12:18:56.850+05:30| mks| W115: VNC CLIENT: received socket error 13: Connection error: could not negotiate SSL
2017-03-15T12:18:56.850+05:30| mks| I125: VNC CLIENT: Destroying VNC Client socket.
2017-03-15T12:18:56.850+05:30| mks| I125: MKSRoleMain: Disconnected from server.
The machine we were using to access the VRA (VMRC) console was not a domain joined machine and didn't have the same root certificate, hence the SSL handshake was failing while creating a VMRC proxy connection through vRA.
We copied the VRA root certificate and the VRA self signed certificate in the Trusted Certificate Authority list on the Client machine and boom...VMRC started working like a charm.