TLS handshake error: failed to verify client's certificate

[simonsparks]

Hi, I've been trying to set up client authentication, as discussed previously in #106, and have managed to successfully authenticate a local self signed certificate via Firefox browser interaction.

However, my primary use case is to restrict access to calls from AWS API Gateway which may include it's own generated client certificates. In this case, when attempting to make a call via the gateway, Fabio logs the following error:

2016/06/07 12:56:01 http: TLS handshake error from 52.30.177.125:47122: tls: failed to verify client's certificate: x509: certificate signed by unknown authority (possibly because of "x509: invalid signature: parent certificate cannot sign this kind of certificate" while trying to verify candidate authority certificate "ApiGateway")

This looks similar to the problem described in this SO post where the client certificate had to be marked as IsCA.

Is there a way of achieving this in Fabio through configuration, if this is in fact the same problem?

[magiconair]

@simonsparks without trying this myself I'd say this looks like this problem. When I generate a test certificate I always have to set IsCA and add x509.KeyUsageCertSign to the KeyUsage for the verification to work. The question is whether fabio should "fix" the cert on the fly which may be valid for some and invalid for other certs. Or whether it should provide a tool to "fix" the cert. I can't just assume that all client auth certificates are self-signed certs for the AWS API Gateway.

[simonsparks]

@magiconair Yes, that's a fair comment. I agree it can't be assumed that all client certs would require this conversion.
Would such a tool be able to parse the pem content, modify the structure as appropriate, and serialize out to a modified pem file which could be used in Fabio? Not being too familiar with go programming and it's crypto package, I don't know if this is how it would work.

[magiconair]

Lets first see whether this works. I'll push a patch in a minute for testing. Are you comfortable building a fabio binary yourself or do you rely on a pre-built or docker image?

[simonsparks]

I'm building Amazon AMIs with Packer.
Fabio is currently installed from a script which runs: go get github.com/eBay/fabio

p.s. really appreciate your support!

[magiconair]

I can upload a binary to github and you could replace go get temporarily with wget. Would that work? I assume linux_amd64 ...

[magiconair]

or can you ssh into your AMI instance and replace the binary? Sorry, I'm a total AWS noob.

[simonsparks]

I can log in to the deployed servers and manually update fabio for the time being.

[magiconair]

BTW, I've just created https://fabiolb.slack.com/ . Might be a good use case to test this.

[simonsparks]

That would be good. I'd be happy to test.

[magiconair]

I am a bit torn about the slack channel since it splits the discussions across multiple mediums. Now others don't see the discussion we had there. OTOH, chat provides a mechanism for others to participate. I've added a Gitter (https://gitter.im/eBay/fabio) link to the README. Someone has used that before but I forgot about it Lets see whether that picks up. So far I'm OK with responding to issues but I'm curious what others think.

[magiconair]

@simonsparks I've pushed an updated patch which introduces a new config option aws.apigw.cert.cn which you now have to set to ApiGateway to enable the workaround. Could you test this and let me know whether it works like the hard-coded version? Then I can merge this to master

[simonsparks]

@magiconair I can confirm that these latest changes, with the configurable option, do work for our scenario.

[magiconair]

@simonsparks I've merged the change to master. I agree that the new config name aws.apigw.cert.cn does not fully express what this change does but so far I've failed to come up with a better name. I'll revisit this when moving to 1.2 or when someone else comes up with another use case. At least this way people can use it who have the same problem you have. I need to update the docs anyway.

[simonsparks]

That sounds reasonable @magiconair. Thanks again for your support with this issue.

[raben2]

Hi,
I was trying to fetch the update because i have the same issue even though i do not run AWS. I issue client certs based on a self signed root cert (no CA)

Jul 04 08:24:22 Web-01 fabio[27551]: 2016/07/04 08:24:22 http: TLS handshake error from xxxxxx: tls: failed to verify client's certificate: x509: certificate signed by unknown authority (possibly because of "x509: cannot verify signature: insecure algorithm MD5-RSA" while trying to verify candidate authority certificate

when i try to update fabio to 1.1.5 i run into this error:

stderr": "# github.com/ebay/fabio\n/usr/local/go/packages/src/github.com/ebay/fabio/listen.go:101: l.AWSApiGWCertCN undefined (type config.Listen has no field or method AWSApiGWCertCN)"

[magiconair]

@raben2 this is on master but I did not create a new release yet. Please run go get -u github.com/eBay/fabio to build the master version. I plan to release a 1.1.6 with this change in the next couple of days and then close of the 1.1 branch so that I can switch to 1.2

[raben2]

Thanks that helped.

[magiconair]

@raben2 @simonsparks BTW, I'm going to drop the parameter aws.apigw.cert.cn in the 1.2 release since it doesn't make sense there. Instead, I'm adding a caupgcn option to the certificate source which you can then set to ApiGateway. I've also come up with a better name for this option which is CA Upgrade CN. That should make it clearer what this is. Because of the changes I'm going to push a 1.2rc3 today and it would be cool if you could give this a quick spin to check whether that part works. 1.2rc3 is live as of today and if there are no issues then I'll merge this to master later this week. Would be cool if you can test.