[HTTPS-Everywhere] Draft specification for file used to check for ruleset updates
Red
redwire at riseup.net
Thu Jun 19 10:10:38 PDT 2014
On 2014-06-18, 12:21 PM, Maxim Nazarenko wrote:
> Two more ideas:
>
> 1) Switch the design documentation to Git as opposed to Gist -- pull
> requests are great!
The only reason I didn't put it in a Git repo is because I can't really
think of a suitable place to put it. I'm not sure the document warrants
it's own repository and HTTPS-E doesn't have a docs directory. Think
it'd be worth making one in my fork?
> 2) Consider using SHA3 (or at least SHA2) instead of SHA1 --
> performance loss should be negligible. According to
> http://valerieaurora.org/hash.html , SHA1 is way into the orange zone :)
Great idea! I ended up using SHA1 because that's apparently what's used
for update.rdf, and part of the goal has been to keep the update.json
spec consistent with update.rdf. I suppose there's really no reason not
to use a more secure hash function here though, since we have control
over which algorithm nsICryptoHash uses and which algorithm the script
used to build update.json uses.
> Best regards,
> Maxim Nazarenko
>
>
> On 18 June 2014 22:43, Yan Zhu <yan at eff.org <mailto:yan at eff.org>> wrote:
>
> Nice work! I have some comments on the spec below (why does gist not
> allow you to post inline comments??)
>
> > However, fetching files using standard XMLHTTPRequests from
> within the
> extension is trivial to accomplish and standards-compliant.
>
> What does standards-compliant mean here? Maybe omit.
>
> > And finally, the public key to hardcode into the HTTPS-Everywhere
> extension to enable it to verify such signatures can be output to
> pubkey.pem via the command [...]
>
> This command should be modified to generate a base64-encoded
> signature,
> which is what nsIDataSignatureVerifier expects.
>
> Note that the header/footer of pubkey.pem need to be stripped before
> giving it as input to nsIDataSignatureVerifier.
>
> > If an attempt to fetch or verify an update fails, the extension
> should
> request update.json again every 5 + R minutes, where R is a random
> number between 0 and 5.
>
> The padding should be a random integer in milliseconds, so it's
> probably
> better to specify this time interval in milliseconds.
>
> > Every time the extension finds that the data provided by update.json
> to be inauthentic, either as a result of the hash of the database file
> not matching or the signature not verifying, the extension must send a
> POST request to a hardcoded url containing the data in the update.json
> file that it tried and failed to verify.
>
> Replace "hardcoded URL" with "hardcoded failure-reporting URL" for
> clarity. Add that the format for failure reports is yet to be
> determined.
>
> > This will, of course, only occur after the extension has determined
> that an update to the ruleset database is available, and verified the
> content of the update.json file to be authentic.
>
> Replace with "Replacing the local rulesets library will only occur
> after
> validating update.json (pseudocode below)."
>
> > The format for the date is " , ". For example, "08 June, 2014".
>
> Why not 08-06-2014 for easier date comparison in JS if we ever need to
> do it?
>
> > hash is a SHA1 (for now) hash of the database file's raw content.
>
> What encoding should we use for the hash? The default from
> https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Reference/Interface/nsICryptoHash#finish%28%29
> is base64, so I would lean towards that.
>
> Also, it's better to specify SHA1 somewhere in the update.json file in
> case anyone is reading it independently. This could either be an
> additional field, or we could use the format
> "sha1/5R0zeLx7EWRxqw6HRlgCRxNLHDo=" (<name of hash function>/<base
> 64-encoded string).
>
> > Pseudocode of update procedure [...]
>
> line 9: "update.version" should be "updateData.version"
>
> "inauthentic" should be initialized to true instead of false so
> that if
> isValidSignature throws an error or gets skipped, nothing is updated
> (generally safer).
>
> updateData.source should be validated to be a valid eff.org
> <http://eff.org> URL before
> fetching it.
>
> if any of the XHR fails (due to network disconnectivity, for
> instance),
> we should try again in 5 minutes plus padding.
>
>
>
> On 06/17/2014 06:18 PM, Red wrote:
> > I have recently been working on updating the ruleset updater spec to
> > reflect the changes that Yan and I discussed during our meeting last
> > Friday (notes at
> > https://gist.github.com/redwire/b62f03905a826e79947a#week-5), so I
> > wanted to inform everyone of the changes. If anyone would like
> to have
> > a look over the document, I'd be happy to receive any
> suggestions for
> > improvements.
> > https://gist.github.com/redwire/2e1d8377ea58e43edb40
> >
> > Another thing I want to report is that I have finished updating the
> > utility script I have also been working on to automate most of the
> > process of building the update.json file. One thing I have
> added into
> > it is a simple mechanism for creating and applying sanity checks to
> > input values. One such test is the "valid_eff_url" function I wrote
> > that attempts to match the provided source (directing the
> extension to
> > the location of the ruleset database file) to a valid eff.org
> <http://eff.org> URL
> > (ending with .sqlite). I wrote the regular expression here
> specifically
> > to be exactly the same as what would be used to repeat this
> sanity check
> > in the update mechanism itself. I have tested the regex in both
> Python
> > and Javascript and found it behaves the same in Javascript and
> Python,
> > but as Yan suggested, it seems obvious that several others
> should review
> > this regex to be sure. Here is a direct link to the beginning
> of the
> > function.
> >
> https://github.com/redwire/https-everywhere/blob/makeJSONManifest/utils/ruleset_update_manifest.py#L58
> >
> > Cheers,
> > Zack
> >
> >
> >
> > _______________________________________________
> > HTTPS-Everywhere mailing list
> > HTTPS-Everywhere at lists.eff.org
> <mailto:HTTPS-Everywhere at lists.eff.org>
> > https://lists.eff.org/mailman/listinfo/https-everywhere
> >
>
>
> --
> Yan Zhu <yan at eff.org <mailto:yan at eff.org>>, <yan at torproject.org
> <mailto:yan at torproject.org>>
> Staff Technologist
> Electronic Frontier Foundation https://www.eff.org
> 815 Eddy Street, San Francisco, CA 94109 +1 415 436 9333
> x134 <tel:%2B1%20415%20436%209333%20x134>
>
>
> _______________________________________________
> HTTPS-Everywhere mailing list
> HTTPS-Everywhere at lists.eff.org <mailto:HTTPS-Everywhere at lists.eff.org>
> https://lists.eff.org/mailman/listinfo/https-everywhere
>
>
>
>
> _______________________________________________
> HTTPS-Everywhere mailing list
> HTTPS-Everywhere at lists.eff.org
> https://lists.eff.org/mailman/listinfo/https-everywhere
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.eff.org/pipermail/https-everywhere/attachments/20140619/5247243f/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 341 bytes
Desc: OpenPGP digital signature
URL: <https://lists.eff.org/pipermail/https-everywhere/attachments/20140619/5247243f/attachment-0001.sig>
More information about the HTTPS-Everywhere
mailing list