From PHP, you can access the useful cURL Library (libcurl) to make requests to URLs using a variety of protocols such as HTTP, FTP, LDAP and even Gopher. (If you’ve spent time on the *nix command line, most environments also have the curl command available that uses the libcurl library)
In practice, however, the most commonly-used protocol tends to be HTTP, especially when using PHP for server-to-server communication. Typically this involves accessing another web server as part of a web service call, using some method such as XML-RPC or REST to query a resource. For example, Delicious offers a HTTP-based API to manipulate and read a user’s posts. However, when trying to access a HTTPS resource (such as the delicious API), there’s a little more configuration you have to do before you can get cURL working right in PHP.
The problem
If you simply try to access a HTTPS (SSL or TLS-protected resource) in PHP using cURL, you’re likely to run into some difficulty. Say you have the following code: (Error handling omitted for brevity)
// Initialize session and set URL.
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
// Set so curl_exec returns the result instead of outputting it.
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
// Get the response and close the channel.
$response = curl_exec($ch);
curl_close($ch);If $url points toward an HTTPS resource, you’re likely to encounter an error like the one below:
Failed: Error Number: 60. Reason: SSL certificate problem, verify that the CA cert is OK. Details:
error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failedThe problem is that cURL has not been configured to trust the server’s HTTPS certificate. The concepts of certificates and PKI revolves around the trust of Certificate Authorities (CAs), and by default, cURL is setup to not trust any CAs, thus it won’t trust any web server’s certificate. So why don’t you have problems visiting HTTPs sites through your web browser? As it happens, the browser developers were nice enough to include a list of default CAs to trust, covering most situations, so as long as the website operator purchased a certificate from one of these CAs.
The quick fix
There are two ways to solve this problem. Firstly, we can simply configure cURL to accept any server(peer) certificate. This isn’t optimal from a security point of view, but if you’re not passing sensitive information back and forth, this is probably alright. Simply add the following line before calling curl_exec():
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);This basically causes cURL to blindly accept any server certificate, without doing any verification as to which CA signed it, and whether or not that CA is trusted. If you’re at all concerned about the data you’re passing to or receiving from the server, you’ll want to enable this peer verification properly. Doing so is a bit more complicated.
The proper fix
The proper fix involves setting the CURLOPT_CAINFO parameter. This is used to point towards a CA certificate that cURL should trust. Thus, any server/peer certificates issued by this CA will also be trusted. In order to do this, we first need to get the CA certificate. In this example, I’ll be using the https://api.del.icio.us/ server as a reference.
First, you’ll need to visit the URL with your web browser in order to grab the CA certificate. Then, (in Firefox) open up the security details for the site by double-clicking on the padlock icon in the lower right corner:
Then click on “View Certificate”:
Bring up the “Details” tab of the cerficates page, and select the certificate at the top of the hierarchy. This is the CA certificate.
Then click “Export”, and save the CA certificate to your selected location, making sure to select the X.509 Certificate (PEM) as the save type/format.
Now we need to modify the cURL setup to use this CA certificate, with CURLOPT_CAINFO set to point to where we saved the CA certificate file to.
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
curl_setopt($ch, CURLOPT_CAINFO, getcwd() . "/CAcerts/BuiltinObjectToken-EquifaxSecureCA.crt");The other option I’ve included, CURLOPT_SSL_VERIFYHOST can be set to the following integer values:
- 0: Don’t check the common name (CN) attribute
- 1: Check that the common name attribute at least exists
- 2: Check that the common name exists and that it matches the host name of the server
If you have CURLOPT_SSL_VERIFYPEER set to false, then from a security perspective, it doesn’t really matter what you’ve set CURLOPT_SSL_VERIFYHOST to, since without peer certificate verification, the server could use any certificate, including a self-signed one that was guaranteed to have a CN that matched the server’s host name. So this setting is really only relevant if you’ve enabled certificate verification.
This ensures that not just any server certificate will be trusted by your cURL session. For example, if an attacker were to somehow redirect traffic from api.delicious.com to their own server, the cURL session here would not properly initialize, since the attacker would not have access to a server certificate (i.e. would not have the private key) trusted by the CA we added. These steps effectively export the trusted CA from the web browser to the cURL configuration.
More information
If you have the CA certificate, but it is not in the PEM format (i.e. it is in a binary or DER format that isn’t Base64-encoded), you’ll need to use something like OpenSSL to convert it to the PEM format. The exact command differs depending on whether you’re converting from PKCS12 or DER format.
There is a CURLOPT_CAPATH option that allows you to specify a directory that holds multiple CA certificates to trust. But it’s not as simple as dumping every single CA certificate in this directory. Instead, they CA certificates must be named properly, and the OpenSSL c_rehash utility can be used to properly setup this directory for use by cURL.
PHP is rather new to me, but I was comfortable with the interpretive style with my experiences in Classic ASP, and PHP’s C-like syntax. I figured I could whip something out in a few minutes (maybe an hour once I saw what we could do) that would retrieve my Delicious feed using their API.
I had this problem with Amazon S3 and your post helped me solved it.
Thanks!
thank you so much. i have searched for over 4 hours to solve this issue and you have described it wonderfully.
Solved it.
There were 2 PEM export options for me in Firefox (when working with harvestapp.com):
X.509 Certificate (PEM)
X.509 Certificate with chain (PEM)
After using the first, I was still getting the error:
SSL certificate problem, verify that the CA cert is OK. Details:
error:14090086:SSL routines:func(144):reason(134)
However, using the second (with chain) export option made it work fine.
Thank you for this post. Other tutorials on how to use PHP, cURL, and Delicious’ API failed to mention the CURLOPT_SSL_VERIFYPEER setting. My code works now
Thanks everyone for their input and comments!
HI ,really nice article. Keep on writing… its helps me to solve a issue. thank you very much..
hey ! thanks a LOT !!! I was writing a quick
and dirty script so that first idea hit to spot!
very timely and very well written
Its a very good solution, i am able to read a https page.
BUT when try to auto login( using post method) it fails, do not show any data. can you please help me out.
thanks.
Excellent post mate! Thanks a lot for your help
Thank you. This solution solve my problem.
Thanks, works for me
but over proxy I can´t access any https hosts.
function getPage($proxy, $url, $referer, $agent, $header, $timeout) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, $header);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_PROXY, $proxy);
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 0);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
curl_setopt($ch, CURLOPT_REFERER, $referer);
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_FRESH_CONNECT, 1);
$result['EXE'] = curl_exec($ch);
$result['INF'] = curl_getinfo($ch);
$result['ERR'] = curl_error($ch);
curl_close($ch);
return $result;
}
$proxyip=”66.178.105.245:8080″;
$result = getPage($proxyip,’https://twitter.com/signup?follow=octanefx&commit=Join+today!’,'http://www.google.com/’,'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8′,1,5);
print_r($result);
[...] by checking against the genuine CA certificate in PEM format. There are plenty of tutorials (e.g. here) on the internet to obtain the PEM certificate, so I won’t go into that here. All you need to [...]
Thank you so much for this. It was a huge help in solving an issue I was having.
Thank you for this; worked perfectly and finally allowed me to stop banging my head on the desk!
Cheers,
James.
Great article! Thank you!
Just a quick question – when the certificate on the server expires and the CA has to re-issue the certificate, would you have to go through the process of downloading the PEM file (as shown in the illustration above)?
Thanks so much again!
@RD
I haven’t encountered this situation yet, but I’ve got the expiration dates marked in my calendar…
Since the cert. will expire and the site will get a new one, then I’d expect the key it contains to be different – and hence your local copy will need updating if you’re using it to verify against with cURL / PHP.
Thanks Tom! Much appreciated. Have an excellent 2010.
@RD
As Tom indicated, you will have to re-download the certificate if it’s re-issued. Even if the public/private key used in the certificate stays the same, the other details (serial number, issue/expiry dates) will all be different, so in effect it’s a completely different certificate.
Thanks for this post, works like a charm
Saved me hours. 
Dear Peter,
I discover that my yahoomail could not open the message page, instead of taking me to my page after loging it will display done without diplay the page. what can i do? meanwhile other website were open correctly.
Thanks
Reagards
Moses