[HTTPS-Everywhere] HTTPS Everywhere vs Preloaded HSTS list

[Dave Warren]

On 2015-03-17 11:57, Jacob Hoffman-Andrews wrote:
> On 03/16/2015 11:37 PM, Dave Warren wrote:
>> I'm curious if anyone has ever looked at HTTPS Everywhere's database
>> and considered dropping sites that are in preloaded HSTS lists? -- I'm
>> assuming that part of the performance impact is linked to the number
>> of rules, and under this theory, it seems like reducing the number of
>> rules without reducing security would be a net win.
> I've definitely considered this, but I think it's not likely to be a big
> performance win. As I understand it, there are ~300 hostnames on the
> preloaded list (updated numbers welcome!), vs ~14.5k rulesets in HTTPS
> Everywhere, with many hostnames per ruleset.

I see 1743 "force-https" items from the current Chromium source, 1597 of 
those have the "include_subdomains" flag (which might indicate that they 
can eliminate multiple hostnames worth of tests in HTTPS Everywhere)

Some domains (e.g. facebook.com) have multiple entries to cover the fact 
that facebook.com itself uses force-https and pins to facebook, plus 
various subdomains are included with the "include_subdomains" flag on 
those subdomains, but this is an exception rather than the rule.

In practice, this means the 1743 "force-https" lines are roughly similar 
to HTTPS Everywhere's count of hostnames rather than rule sets. However, 
the vast majority are both "force-https" and "include_subdomains" and 
therefore might replace multiple tests in HTTPS Everywhere unless 
they're similarly wildcarded in HTTPS Everywhere.

I wonder how many of those overlap? I don't have a good way to make that 
comparison at this moment, although it probably wouldn't be especially 
difficult to do, especially taking the include_subdomains entries into 
account, as they might eliminate a large number of hostname hits and 
ultimately have a larger performance impact.

It's a shame Chrom(e|ium) doesn't have an API, and that the list is 
hardcoded rather than dynamically updated, meaning that potentially 
every browser build has a different list, such that from a HTTPS 
Everywhere perspective, rules would either need to remain until the 
browser update reaches some level of critical mass, or HTTPS Everywhere 
would need to add "Only prior to browser version 'x'" conditionals into 
the rulesets, which sounds messy and terrible. If there were at least an 
API that HTTPS Everywhere could use to read the browser preload list, 
HTTPS Everywhere could periodically scan it's lists and disregard the 
unneeded rules at the browser level.


-- 
Dave Warren
http://www.hireahit.com/
http://ca.linkedin.com/in/davejwarren