[HTTPS-Everywhere] Verifying signatures in a FF extension?

Fri Jul 4 18:57:37 PDT 2014

On 2014-07-04, 3:57 PM, Yan Zhu wrote:
> One idea is to look through the signing code from Uhura (command line
> signing utility for Mozilla extensions):
> http://www.softlights.net/download.html. This should make the correct
> signature format, since we use it to generate the signature field in
> update.rdf for HTTPS Everywhere.
>
> Actually, it looks like what you want is lines 148-187 in the Linux
> Uhura script.
I appreciate the suggestion!

I found that Uhura also uses `openssl dgst` to sign data, which is what
I have been using more recently.  The script also, however, explicitly
specifies the use of the "-binary" flag, which appears to be the default
behavior.  Just to be sure, I tried signing and then base64-encoding the
signature of the digest of update.json, but in both cases I ended up
with the same thing.

For reference, the signature I'm dealing with right now is

IS3xOEJZ3E5zsScccgqfmnESnobyHKwdi2o+27T3fTe7BgbynU/a5HdKgKOK2lNP
9MEeiijuPbLJ8XkbrYsxZ6ylg66fR4ZRRH8KOzny60UC/r9Pmhe8A85ciXnDapp/
Ryiaknk+nShRxD1QyqCXl6mP1ZyOG3oQoaIY2Ku7nNA=

which was created using a 1024-bit key I was experimenting with.
> On 07/04/2014 03:19 PM, Red wrote:
>> I've been looking into the options available to the `openssl dgst` and
>> `openssl rsautl` commands, and can't find anything regarding
>> DER-encoding of signatures.  I have already found that keys can be
>> der-encoded, and that going on to base64-encoding those keys (obviously)
>> produces the same result as PEM-encoding the key, and stripping the
>> header and footer from the resulting file.  I've also been
>> triple-checking the nsIDataSignatureVerifier documentation page to make
>> sure I was doing things right
>> (https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Reference/Interface/nsIDataSignatureVerifier),
>> and confirmed that it seems what I've done so far is correct.  The data
>> to verify is, in this case, the hash of `update.json`, which I've
>> already tested is computed successfully, and then the signature is just
>> supposed to be the base64-encoding of the binary-encoded signature
>> produced by openssl, and that the public key is the base64-encoded
>> DER-encoding of the public key.  As far as I can tell, having
>> experimented with all kinds of different combinations of openssl
>> commands and such, I've produced the right data.
>> At the very least, the most recent process described in the spec
>> produces a signature that Openssl can verify.
>>
>> I have even searched Github for examples of nsIDataSignatureVerifier in
>> use.  Just about all of the results are from the unit tests of forks of
>> other projects.  The files `test_datasignatureverifier.js` and
>> `AddonUpdateChecker.jsm` show up a lot in
>> https://github.com/search?l=javascript&q=nsIDataSignatureVerifier&ref=cmdform&type=Code.
>> As far as I can tell, the way I hardcoded update.json, its signature, an
>> the public key corresponding to the private key I used to generate the
>> signature is fine.
>>
>> As Yan said, it seems that these signatures begin with 'MI', but the
>> ones I have been getting don't seem to contain that prefix.  I've even
>> tried running my test with a signature generated with a 1024-bit RSA key
>> (in case there was some kind of undocumented limitation to
>> nsIDataSignatureVerifier), but even then:
>> 1. The signature I generated doesn't begin with 'MI' and
>> 2. I still just get an exception in my test.
>>
>> At this point, all I can think is that either
>> 1. Openssl is giving me signatures that nsIDSV can't handle or are
>> non-standard somehow,
>> 2. My process for signing is incorrect, or
>> 3. nsIDSV just doesn't work in the context of an SDK-based extension
>> (the testing extension).
>>
>> I'm completely at a loss, though.
>>
>> Cheers,
>> Zack
>>
>> On 2014-07-04, 2:02 AM, Yan Zhu wrote:
>>> At a glance, it seems like your signature string is not properly
>>> encoded. The first bytes should be "MII" for a Base64 encoding of a
>>> DER-encoded signature.
>>>
>>> Unfortunately I don't think I can make the usual IRC meeting tonight
>>> because I'm in France - let's try to meet on Monday instead?
>>>
>>> -Yan
>>>
>>> On 07/03/2014 07:36 AM, Red wrote:
>>>> Here's all of the test output on pastebin: http://pastebin.com/mYyw3WNW
>>>> The exact exception message I am getting is:
>>>>
>>>> Message: [Exception... "Component returned failure code: 0x80004005
>>>> (NS_ERROR_FAILURE) [nsIDataSignatureVerifier.verifyData]"  nsresult:
>>>> "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame ::
>>>> resource://gre/modules/addons/XPIProvider.jsm ->
>>>> jar:file:///Users/redwire/Developer/Sources/Projects/https-everywhere/test_profile/extensions/jid1-we5BQOfc6skSMg@jetpack.xpi!/bootstrap.js
>>>> -> resource://gre/modules/commonjs/toolkit/loader.js ->
>>>> resource://jid1-we5bqofc6sksmg-at-jetpack/https-everywhere-tests/tests/test-rsupdate-verify.js
>>>> :: exports["test update JSON signature validity"] :: line 78"  data: no]
>>>>
>>>> On 2014-07-02, 10:05 PM, Yan Zhu wrote:
>>>>> On 07/02/2014 10:00 PM, Red wrote:
>>>>>> Hello everyone,
>>>>>>     I've been working on writing some tests[1] for signature
>>>>>> verification and hashing using the unit testing solution Yan built after
>>>>>> we discussed some new ideas during our meeting last week, but have had
>>>>>> no success in getting signature verification working.  I've been using
>>>>>> the nsIDataSignatureVerifier XPCOM component[2] to do this, using some
>>>>>> testing data and an RSA key pair[3] I generated just for testing. All of
>>>>>> my tests with hashing have worked perfectly well and are passing, but
>>>>>> for some reason my attempt at verifying the signature I created are
>>>>>> leading to an exception being thrown.  I followed the process I outlined
>>>>>> in the update.json spec to create my key pair, an example update.json
>>>>>> file, and to sign the hash of the contents of update.json[4].
>>>>> Could you post the exception that you're getting and the test output?
>>>>>
>>>>> -Yan
>>>>>
>>>>>>     I've asked on both the #extdev and #jetpack channels of
>>>>>> irc.mozilla.org about this, and have scoured Google, Duck Duck Go, MDN,
>>>>>> and Stack Overflow for an answer to the question of what could cause
>>>>>> this behavior (and have referenced some code I found on github to
>>>>>> confirm I had hardcoded my public key and signature correctly), and
>>>>>> haven't turned up anything that brings me anywhere near a solution.  So,
>>>>>> I come to you.   As people who have worked on HTTPS Everywhere, are any
>>>>>> of you familiar with the process for verifying signatures, and could you
>>>>>> perhaps review my test to see if I've done something wrong?  Are there
>>>>>> any alternative ways of verifying signatures (perhaps using an external
>>>>>> library) that might be more reliable?
>>>>>>
>>>>>> Thanks,
>>>>>> Zack
>>>>>>
>>>>>> Note1: my `ruleset_update_manifest.py` script doesn't append a newline
>>>>>> character to the end of the written `update.json` file, but I had added
>>>>>> one accidentally while playing with the content in vim. Rather than
>>>>>> hashing and signing the file again, I decided to simply append a newline
>>>>>> character to the hardcoded data in my test code.
>>>>>>
>>>>>> Note2: I've gone through a lot of permutations of, what I hope are,
>>>>>> reasonable modifications to my test code to try to resolve this issue,
>>>>>> so I highly recommend having a peek through the commit history on [1] to
>>>>>> get an idea of what I've been trying.
>>>>>>
>>>>>> [1]:
>>>>>> https://github.com/redwire/https-everywhere/blob/feature/tests/https-everywhere-tests/test/test-rsupdate-verify.js
>>>>>> [2]:
>>>>>> https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Reference/Interface/nsIDataSignatureVerifier
>>>>>> [3]:
>>>>>> https://github.com/redwire/https-everywhere/tree/rulesetUpdating/utils/testing/sign_verify
>>>>>> [4]:
>>>>>> https://github.com/redwire/https-everywhere/blob/rulesetUpdating/doc/updateJSONSpec.md#updatejson-and-updatejsonsig
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> HTTPS-Everywhere mailing list
>>>>>> HTTPS-Everywhere at lists.eff.org
>>>>>> https://lists.eff.org/mailman/listinfo/https-everywhere
>>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> HTTPS-Everywhere mailing list
>>>> HTTPS-Everywhere at lists.eff.org
>>>> https://lists.eff.org/mailman/listinfo/https-everywhere
>>>>
>>
>>
>>
>> _______________________________________________
>> HTTPS-Everywhere mailing list
>> HTTPS-Everywhere at lists.eff.org
>> https://lists.eff.org/mailman/listinfo/https-everywhere
>>
>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 341 bytes
Desc: OpenPGP digital signature
URL: <https://lists.eff.org/pipermail/https-everywhere/attachments/20140704/520db971/attachment-0001.sig>