[SSL Observatory] CDN services

[Ondrej Mikle]

Hi,

I've created a dataset covering CDN services (to see how common the "citibank
effect" is). CDN service is defined as hostname serving certificates with
overlapping time periods (i.e. cert A is seen, cert B is seen, then A again;
mostly due to reverse NATs, fast-flux DNS, multiple IPs or misconfiguration).


Following CSV lists 11017 CDN hostnames and certificate issuers for their 26403
certs:
http://constructibleuniverse.net/CDN/CDN_hosts.csv

Format is:
host|db_id|issuer organization|issuer CN|first_seen|last_seen

Taking out only hosts that have certs issued by different issuers, we get:
- compared by issuer organization and CN strings - 4633 hosts:
  http://constructibleuniverse.net/CDN/CDN_hosts_filtered_by_org_cn.csv
- compared by issuer organization string only - 4022 hosts:
  http://constructibleuniverse.net/CDN/CDN_hosts_filtered_by_org.csv

Full certificate chains sent by the hosts (25 MB, format
db_id|server_cert|intermed_cert1|...) :
http://constructibleuniverse.net/CDN/CDN_cert_chains.csv.bz2


Few picks and oddities from the set:

- most CDNs tend to stick with one CA, examples of "large" exceptions: Facebook
(DigiCert, Verisign, Equifax), m.unionbank.com (Usertrust, Verisign)
- self-signed certs popping up along with CA-issued ones seem rather common,
sometimes it's just once, sometimes both coexist for long time (e.g.
accessanywhere.net, webaccess.gtbankuk.com)
- accessorycenter.brightstarcorp.com - one of certs it sends is revoked
- SSL inspection/MitM boxes sometimes show up before being configured (Blue
Coat, SonicWall, Watchguard Fireware)

Final notes:
- scanning was done daily between 2011-09-23 and 2011-11-04 on 1.5M+ hostnames
- four certs failed to parse (noted as "!!!parse error!!!" in issuer CN/O field)
- I filtered out around 800 hostnames hosted by fastdomain.com and hosts
pointing to 127.0.0.1 to unclutter the set (unfiltered set is at
http://constructibleuniverse.net/CDN/CDN_unfiltered.csv)


Ondrej