[HTTPS-Everywhere] HTTPS Everywhere ruleset checker alpha version

[Ondrej Mikle]

On 08/05/2012 11:43 PM, Peter Eckersley wrote:
> I have started messing about with this script.  It's still firing quite a lot
> of false alarms, only some of which I understand.  A lot of these look like
> gnutls bugs, and it seems you've implemented an important test suite that the
> gnutls people should look at ;).  Here are a few examples:
> 
> 1) gnutls TLS errors on www.zotero.org:
> 
> 2012-08-05 14:14:16,045 DEBUG =**= Start http://forums.zotero.org/ => https://forums.zotero.org/ **** [check_rules.py:86]
> 2012-08-05 14:14:16,045 DEBUG Fetching plain page http://forums.zotero.org/ [check_rules.py:87]
> 2012-08-05 14:14:16,046 DEBUG =**= Start http://zotero.org/ => https://www.zotero.org/ **** [check_rules.py:86]
> 2012-08-05 14:14:16,046 DEBUG =**= Start http://www.zotero.org/ => https://www.zotero.org/ **** [check_rules.py:86]
> 2012-08-05 14:14:16,047 DEBUG Fetching plain page http://zotero.org/ [check_rules.py:87]
> 2012-08-05 14:14:16,047 DEBUG Fetching plain page http://www.zotero.org/ [check_rules.py:87]
> 2012-08-05 14:14:16,584 DEBUG Following redirect http://zotero.org/ => http://www.zotero.org/ [/home/pde/eff/ssl/https-everywhere/https-everywhere-checker/http_client.py:176]
> 2012-08-05 14:14:16,730 DEBUG Fetching transformed page https://www.zotero.org/ [check_rules.py:89]
> 2012-08-05 14:14:16,858 DEBUG Fetching transformed page https://forums.zotero.org/ [check_rules.py:89]
> 2012-08-05 14:14:17,135 DEBUG Fetching transformed page https://www.zotero.org/ [check_rules.py:89]
> 2012-08-05 14:14:17,860 ERROR Failed to process http://www.zotero.org/: (56, 'GnuTLS recv error (-9): A TLS packet with unexpected length was received.'). Rulefile: Zotero.xml [check_rules.py:112]
> Traceback (most recent call last):
>   File "check_rules.py", line 90, in run
>     transformedRcode, transformedPage = fetcherRewriting.fetchHtml(transformedUrl)
>   File "/home/pde/eff/ssl/https-everywhere/https-everywhere-checker/http_client.py", line 161, in fetchHtml
>     c.perform()
> error: (56, 'GnuTLS recv error (-9): A TLS packet with unexpected length was received.')

I can reproduce it with gnutls 2.8.5 and libcurl compiled with "--without-ssl
--with-gnutls".

Some of the "A TLS packet with unexpected length was received." errors are
caused by the fact that gnutls tries to use TLS 1.1. Server does not like it and
closes connection. Using "c.setopt(c.SSLVERSION, c.SSLVERSION_SSLv3)" makes it
work (but there's no way to specify TLSv1 minor version in libcurl/pycurl).

It's different with zotero.org though: I can still get the content of the page.
By looking at the stream in Wireshark I'd guess the problem is:

i) gnutls ends connection, sends encrypted alert, probably close_notify
ii) zotero.org should send the close_notify back, but doesn't (per section 7.2.1
of RFC 5246, same in RFC 2246 for TLS 1.0)
iii) zotero.org closes connection with FIN/ACK instead
iv) gnutls doesn't like it - can be reproduced on command line with:
  gnutls-cli -d 5 --priority "EXPORT:%UNSAFE_RENEGOTIATION" www.zotero.org

Pycurl seems to also not have properly implemented the gnutls callbacks for
thread use (probably pycurl-gnutls-threads combo is not used so much) - I get
following abort with multiple threads:

python: ath.c:193: _gcry_ath_mutex_lock: Assertion `*lock == ((ath_mutex_t) 0)'
failed.

Ondrej

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: OpenPGP digital signature
URL: <http://lists.eff.org/pipermail/https-everywhere/attachments/20120806/f34e58f1/attachment.sig>