Today's guest blog entry is from Shaun Hurley and an errant certificate. Enjoy
A Lesson in SSL Troubleshooting
It was an odd problem.
For some reason our trusted business partner (BP) was unable to send an SSL certificate when they made a call to our IIS web-server. The response from the webserver: 403.12 - Client certificate has been denied access by the server certificate mapper. The connection was based on a client side certificate, verified against the public key on our end that they provided for us. When we tested it with our certificate from a remove location it was OK (should have been a red flag).
Our BP wanted to know what was in the path from their server to our server. Well, the server they were hitting was in our development lab, so: a border firewall, several routers, and another firewall to be able to access our lab. They wanted to know if one of our devices (firewall, in particular) was stripping out the SSL certificate as it was being sent across the network. Short answer: no.
Perhaps someone can correct me if I am wrong, but if 443 and 80 are both open, valid (sometimes invalid) traffic should be allowed to cross the firewall. Now there are other small
details: such as valid session states, etc. If there were a problem at the firewall level, we wouldn’t be able to hit the server. This shouldn’t have been called into question.
Besides, the error message indicates a problem with either the server or client certificate. They connected using openssl, it displayed a list of options for Certificate Authorities (CAs) and didn’t see the one there for the CA they are using for their certificate. So they sent it to us, claiming that
it was and old Verisign CA that is apparently not on our trusted server list (another flag).
So they sent it to us, we imported it, and now the CA was in the list but they were getting a different error. 401.13: client certificate has been revoked on the web server. Yay! Something new.
So we played around with the IIS settings, turned off all certificate checking to verify if it is a valid certificate (big no-no). And we finally got a new error message: 401.16 – client certificate
is ill-formed or is not trusted by the web server. Now we are getting somewhere.
We asked them about the validity of the certificate, but they claimed that it works with other clients, but my co-worker (thank god), a meticulous worker, forced them to compare every piece of their public-key with the private key they have generated. Things were matching up, until we found out that not only was this certificate NOT generated by Verisign, it had the wrong hashing algorithm. On top of that, we found out that the CA they sent us was NOT an old Verisign CA. Removing the CA, and changing the settings back, we had them generate a valid certificate.
All is well.
Lessons:
Always check the minor things. It never hurts to double check the validity of a certificate; no matter how
certain you are that it is OK. Small things like this can prevent a couple days of banging your head against
a wall trying to figure out the problem.
Shaun Hurley
Shamrockhoax.blogspot.com
Myspace.com/shamrockhoax
A facebook thing-a-ma-bob as well.
Something to Ask: If anybody know of a virus that will infect juke boxes that will remove the band OAR from the machine, I would be eternally grateful. It is time for their horrible brand of soft-mold, half-baked, [expletive deleted] music to go the [expletive deleted] away. I swear to god, if I hear that steaming turd of a song called “crazy game of poker” one more time, I will shoot somebody (Nickleback is next, I promise). 30 minute songs about some hippy’s lame acid trip/white boy tribute to jah should not be a song/should not exist on a jukebox. EOM.