[HTTPS-Everywhere] Verifying signatures in a FF extension?

Sun Jul 6 17:30:06 PDT 2014

A quick status report:
Recently I tested to see if my nsIDataSignatureVerifier test will
succeed (or at least not result in an undocumented error) with a
1024-bit RSA key.  The test failed the same way it did with a 2048-bit
RSA key.
I have been asking around repeatedly on #extdev and #jetpack on
irc.mozilla.org, but no one seems to really know much about
nsIDataSignatureVerifier.  I was recommended to try getting in touch
with the person who wrote the original unit test for the component, and
to create an issue on Bugzilla in the hopes that someone more familiar
with the component will know what's up.  I have done both of these
things, and the issue can be found at
https://bugzilla.mozilla.org/show_bug.cgi?id=1035018.

Cheers,
Zack

On 2014-07-04, 3:57 PM, Yan Zhu wrote:
> One idea is to look through the signing code from Uhura (command line
> signing utility for Mozilla extensions):
> http://www.softlights.net/download.html. This should make the correct
> signature format, since we use it to generate the signature field in
> update.rdf for HTTPS Everywhere.
>
> Actually, it looks like what you want is lines 148-187 in the Linux
> Uhura script.
>
> On 07/04/2014 03:19 PM, Red wrote:
>> I've been looking into the options available to the `openssl dgst` and
>> `openssl rsautl` commands, and can't find anything regarding
>> DER-encoding of signatures.  I have already found that keys can be
>> der-encoded, and that going on to base64-encoding those keys (obviously)
>> produces the same result as PEM-encoding the key, and stripping the
>> header and footer from the resulting file.  I've also been
>> triple-checking the nsIDataSignatureVerifier documentation page to make
>> sure I was doing things right
>> (https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Reference/Interface/nsIDataSignatureVerifier),
>> and confirmed that it seems what I've done so far is correct.  The data
>> to verify is, in this case, the hash of `update.json`, which I've
>> already tested is computed successfully, and then the signature is just
>> supposed to be the base64-encoding of the binary-encoded signature
>> produced by openssl, and that the public key is the base64-encoded
>> DER-encoding of the public key.  As far as I can tell, having
>> experimented with all kinds of different combinations of openssl
>> commands and such, I've produced the right data.
>> At the very least, the most recent process described in the spec
>> produces a signature that Openssl can verify.
>>
>> I have even searched Github for examples of nsIDataSignatureVerifier in
>> use.  Just about all of the results are from the unit tests of forks of
>> other projects.  The files `test_datasignatureverifier.js` and
>> `AddonUpdateChecker.jsm` show up a lot in
>> https://github.com/search?l=javascript&q=nsIDataSignatureVerifier&ref=cmdform&type=Code.
>> As far as I can tell, the way I hardcoded update.json, its signature, an
>> the public key corresponding to the private key I used to generate the
>> signature is fine.
>>
>> As Yan said, it seems that these signatures begin with 'MI', but the
>> ones I have been getting don't seem to contain that prefix.  I've even
>> tried running my test with a signature generated with a 1024-bit RSA key
>> (in case there was some kind of undocumented limitation to
>> nsIDataSignatureVerifier), but even then:
>> 1. The signature I generated doesn't begin with 'MI' and
>> 2. I still just get an exception in my test.
>>
>> At this point, all I can think is that either
>> 1. Openssl is giving me signatures that nsIDSV can't handle or are
>> non-standard somehow,
>> 2. My process for signing is incorrect, or
>> 3. nsIDSV just doesn't work in the context of an SDK-based extension
>> (the testing extension).
>>
>> I'm completely at a loss, though.
>>
>> Cheers,
>> Zack
>>
>> On 2014-07-04, 2:02 AM, Yan Zhu wrote:
>>> At a glance, it seems like your signature string is not properly
>>> encoded. The first bytes should be "MII" for a Base64 encoding of a
>>> DER-encoded signature.
>>>
>>> Unfortunately I don't think I can make the usual IRC meeting tonight
>>> because I'm in France - let's try to meet on Monday instead?
>>>
>>> -Yan
>>>
>>> On 07/03/2014 07:36 AM, Red wrote:
>>>> Here's all of the test output on pastebin: http://pastebin.com/mYyw3WNW
>>>> The exact exception message I am getting is:
>>>>
>>>> Message: [Exception... "Component returned failure code: 0x80004005
>>>> (NS_ERROR_FAILURE) [nsIDataSignatureVerifier.verifyData]"  nsresult:
>>>> "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame ::
>>>> resource://gre/modules/addons/XPIProvider.jsm ->
>>>> jar:file:///Users/redwire/Developer/Sources/Projects/https-everywhere/test_profile/extensions/jid1-we5BQOfc6skSMg@jetpack.xpi!/bootstrap.js
>>>> -> resource://gre/modules/commonjs/toolkit/loader.js ->
>>>> resource://jid1-we5bqofc6sksmg-at-jetpack/https-everywhere-tests/tests/test-rsupdate-verify.js
>>>> :: exports["test update JSON signature validity"] :: line 78"  data: no]
>>>>
>>>> On 2014-07-02, 10:05 PM, Yan Zhu wrote:
>>>>> On 07/02/2014 10:00 PM, Red wrote:
>>>>>> Hello everyone,
>>>>>>     I've been working on writing some tests[1] for signature
>>>>>> verification and hashing using the unit testing solution Yan built after
>>>>>> we discussed some new ideas during our meeting last week, but have had
>>>>>> no success in getting signature verification working.  I've been using
>>>>>> the nsIDataSignatureVerifier XPCOM component[2] to do this, using some
>>>>>> testing data and an RSA key pair[3] I generated just for testing. All of
>>>>>> my tests with hashing have worked perfectly well and are passing, but
>>>>>> for some reason my attempt at verifying the signature I created are
>>>>>> leading to an exception being thrown.  I followed the process I outlined
>>>>>> in the update.json spec to create my key pair, an example update.json
>>>>>> file, and to sign the hash of the contents of update.json[4].
>>>>> Could you post the exception that you're getting and the test output?
>>>>>
>>>>> -Yan
>>>>>
>>>>>>     I've asked on both the #extdev and #jetpack channels of
>>>>>> irc.mozilla.org about this, and have scoured Google, Duck Duck Go, MDN,
>>>>>> and Stack Overflow for an answer to the question of what could cause
>>>>>> this behavior (and have referenced some code I found on github to
>>>>>> confirm I had hardcoded my public key and signature correctly), and
>>>>>> haven't turned up anything that brings me anywhere near a solution.  So,
>>>>>> I come to you.   As people who have worked on HTTPS Everywhere, are any
>>>>>> of you familiar with the process for verifying signatures, and could you
>>>>>> perhaps review my test to see if I've done something wrong?  Are there
>>>>>> any alternative ways of verifying signatures (perhaps using an external
>>>>>> library) that might be more reliable?
>>>>>>
>>>>>> Thanks,
>>>>>> Zack
>>>>>>
>>>>>> Note1: my `ruleset_update_manifest.py` script doesn't append a newline
>>>>>> character to the end of the written `update.json` file, but I had added
>>>>>> one accidentally while playing with the content in vim. Rather than
>>>>>> hashing and signing the file again, I decided to simply append a newline
>>>>>> character to the hardcoded data in my test code.
>>>>>>
>>>>>> Note2: I've gone through a lot of permutations of, what I hope are,
>>>>>> reasonable modifications to my test code to try to resolve this issue,
>>>>>> so I highly recommend having a peek through the commit history on [1] to
>>>>>> get an idea of what I've been trying.
>>>>>>
>>>>>> [1]:
>>>>>> https://github.com/redwire/https-everywhere/blob/feature/tests/https-everywhere-tests/test/test-rsupdate-verify.js
>>>>>> [2]:
>>>>>> https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Reference/Interface/nsIDataSignatureVerifier
>>>>>> [3]:
>>>>>> https://github.com/redwire/https-everywhere/tree/rulesetUpdating/utils/testing/sign_verify
>>>>>> [4]:
>>>>>> https://github.com/redwire/https-everywhere/blob/rulesetUpdating/doc/updateJSONSpec.md#updatejson-and-updatejsonsig
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> HTTPS-Everywhere mailing list
>>>>>> HTTPS-Everywhere at lists.eff.org
>>>>>> https://lists.eff.org/mailman/listinfo/https-everywhere
>>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> HTTPS-Everywhere mailing list
>>>> HTTPS-Everywhere at lists.eff.org
>>>> https://lists.eff.org/mailman/listinfo/https-everywhere
>>>>
>>
>>
>>
>> _______________________________________________
>> HTTPS-Everywhere mailing list
>> HTTPS-Everywhere at lists.eff.org
>> https://lists.eff.org/mailman/listinfo/https-everywhere
>>
>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 341 bytes
Desc: OpenPGP digital signature
URL: <https://lists.eff.org/pipermail/https-everywhere/attachments/20140706/34e5b8bf/attachment.sig>