[HTTPS-Everywhere] HTTPS Everywhere ruleset checker alpha version

[Peter Eckersley]

I have started messing about with this script.  It's still firing quite a lot
of false alarms, only some of which I understand.  A lot of these look like
gnutls bugs, and it seems you've implemented an important test suite that the
gnutls people should look at ;).  Here are a few examples:

1) gnutls TLS errors on www.zotero.org:

2012-08-05 14:14:16,045 DEBUG =**= Start http://forums.zotero.org/ => https://forums.zotero.org/ **** [check_rules.py:86]
2012-08-05 14:14:16,045 DEBUG Fetching plain page http://forums.zotero.org/ [check_rules.py:87]
2012-08-05 14:14:16,046 DEBUG =**= Start http://zotero.org/ => https://www.zotero.org/ **** [check_rules.py:86]
2012-08-05 14:14:16,046 DEBUG =**= Start http://www.zotero.org/ => https://www.zotero.org/ **** [check_rules.py:86]
2012-08-05 14:14:16,047 DEBUG Fetching plain page http://zotero.org/ [check_rules.py:87]
2012-08-05 14:14:16,047 DEBUG Fetching plain page http://www.zotero.org/ [check_rules.py:87]
2012-08-05 14:14:16,584 DEBUG Following redirect http://zotero.org/ => http://www.zotero.org/ [/home/pde/eff/ssl/https-everywhere/https-everywhere-checker/http_client.py:176]
2012-08-05 14:14:16,730 DEBUG Fetching transformed page https://www.zotero.org/ [check_rules.py:89]
2012-08-05 14:14:16,858 DEBUG Fetching transformed page https://forums.zotero.org/ [check_rules.py:89]
2012-08-05 14:14:17,135 DEBUG Fetching transformed page https://www.zotero.org/ [check_rules.py:89]
2012-08-05 14:14:17,860 ERROR Failed to process http://www.zotero.org/: (56, 'GnuTLS recv error (-9): A TLS packet with unexpected length was received.'). Rulefile: Zotero.xml [check_rules.py:112]
Traceback (most recent call last):
  File "check_rules.py", line 90, in run
    transformedRcode, transformedPage = fetcherRewriting.fetchHtml(transformedUrl)
  File "/home/pde/eff/ssl/https-everywhere/https-everywhere-checker/http_client.py", line 161, in fetchHtml
    c.perform()
error: (56, 'GnuTLS recv error (-9): A TLS packet with unexpected length was received.')

Firefox seems happy to talk to https://www.zotero.org, though

2) gnutls cert validation snafu on www.wyndhamrentals.com (that chain looks
valid and isn't even transvalid, according to openssl)

2012-08-05 14:11:02,411 ERROR Failed to process http://wyndhamrentals.com/: (60, 'server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none'). Rulefile: Wyndham.xml [check_rules.py:112]
Traceback (most recent call last):
  File "check_rules.py", line 90, in run
    transformedRcode, transformedPage = fetcherRewriting.fetchHtml(transformedUrl)
  File "/home/pde/eff/ssl/https-everywhere/https-everywhere-checker/http_client.py", line 161, in fetchHtml
    c.perform()
error: (60, 'server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none')
2012-08-05 14:11:02,411 INFO Finished comparing http://wyndhamrentals.com/ -> https://www.wyndhamrentals.com/. Rulefile: Wyndham.xml. [check_rules.py:116]

3) We probably need to augment this script to recognise rules for sites with
transvalid certs, which are still about as "correct" as they can be until the
browsers fix the underlying problem of transvalidity.

On Fri, Jul 20, 2012 at 01:28:51PM -0700, Peter Eckersley wrote:
> This is awesome!  We should schedule a hacking day to get this properly
> integrated into the ruleset authorship and integration process.  There are
> various options, such as making these tests a build script option, having a
> robot somewhere that runs the tests on the master ruleset library and pushes
> results to a git repo, or perhaps even doing something with commit hooks.
> 
> On Fri, Jul 20, 2012 at 04:38:43PM +0200, Ondrej Mikle wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> > 
> > Hi,
> > 
> > I've finished an alpha version of automated checker/tester for HTTPS
> > Everywhere rules:
> > 
> > https://github.com/hiviah/https-everywhere-checker
> > 
> > While developing/testing, I've encountered bunch of rules having for
> > instance incorrectly written regex capture groups, domains existing no
> > more, rewrites causing loops and all kinds of certchain validation
> > errors (I didn't test all the rules, maybe 1000-1500). See README.md
> > in the above github repo for details.
> > 
> > For now the checker only fetches the HTML pages, but adding fetching
> > of js/css/images would be very easy (it's parsed with lxml's HTML
> > parser, thus the elements are easy to find with XPath).
> > 
> > 
> > Ondrej
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG v2.0.14 (GNU/Linux)
> > 
> > iQEcBAEBAgAGBQJQCW1wAAoJEFAA8mCNYrseZMkH/1Z30J+gErQzjrZc/MTLBbLj
> > lZ2QIoQObKX6w31QSQyPHsCK4lBEncQKPzZ2MWq+pAzrxOOG6HrX4Ti3KBzU08qA
> > xlUTkNdf+zSRwMLTSnFM0dr0gsaprHg026UqAg28URVy91sLycSon5noxJ1reiXN
> > 2c/meF5geHi5pnuBm6vrKVOJJfY5ceXG7C5Fmx6icZXHytpFyovjd4HTISWh4dQI
> > +GTpDiNg66uLEfK+pnxxtWTjlpnwjgNYPONC+vu8mAZ4TrxoU9U2zvl3bbRYXomL
> > skFgeBKxUiBS/ZqcsWXo4bieZCq/sUBnbbsEdt08yViHQIbm20AfGFd82XKyQcg=
> > =bImI
> > -----END PGP SIGNATURE-----
> > 
> > _______________________________________________
> > HTTPS-everywhere mailing list
> > HTTPS-everywhere at mail1.eff.org
> > https://mail1.eff.org/mailman/listinfo/https-everywhere
> 
> -- 
> Peter Eckersley                            pde at eff.org
> Technology Projects Director      Tel  +1 415 436 9333 x131
> Electronic Frontier Foundation    Fax  +1 415 436 9993

-- 
Peter Eckersley                            pde at eff.org
Technology Projects Director      Tel  +1 415 436 9333 x131
Electronic Frontier Foundation    Fax  +1 415 436 9993