[HTTPS-Everywhere] Draft specification for file used to check for ruleset updates

[Maxim Nazarenko]

Two more ideas:

1) Switch the design documentation to Git as opposed to Gist -- pull
requests are great!

2) Consider using SHA3 (or at least SHA2) instead of SHA1 -- performance
loss should be negligible. According to http://valerieaurora.org/hash.html
, SHA1 is way into the orange zone :)

Best regards,
Maxim Nazarenko


On 18 June 2014 22:43, Yan Zhu <yan at eff.org> wrote:

> Nice work! I have some comments on the spec below (why does gist not
> allow you to post inline comments??)
>
> > However, fetching files using standard XMLHTTPRequests from within the
> extension is trivial to accomplish and standards-compliant.
>
> What does standards-compliant mean here? Maybe omit.
>
> > And finally, the public key to hardcode into the HTTPS-Everywhere
> extension to enable it to verify such signatures can be output to
> pubkey.pem via the command [...]
>
> This command should be modified to generate a base64-encoded signature,
> which is what nsIDataSignatureVerifier expects.
>
> Note that the header/footer of pubkey.pem need to be stripped before
> giving it as input to nsIDataSignatureVerifier.
>
> > If an attempt to fetch or verify an update fails, the extension should
> request update.json again every 5 + R minutes, where R is a random
> number between 0 and 5.
>
> The padding should be a random integer in milliseconds, so it's probably
> better to specify this time interval in milliseconds.
>
> > Every time the extension finds that the data provided by update.json
> to be inauthentic, either as a result of the hash of the database file
> not matching or the signature not verifying, the extension must send a
> POST request to a hardcoded url containing the data in the update.json
> file that it tried and failed to verify.
>
> Replace "hardcoded URL" with "hardcoded failure-reporting URL" for
> clarity. Add that the format for failure reports is yet to be determined.
>
> > This will, of course, only occur after the extension has determined
> that an update to the ruleset database is available, and verified the
> content of the update.json file to be authentic.
>
> Replace with "Replacing the local rulesets library will only occur after
> validating update.json (pseudocode below)."
>
> > The format for the date is " , ". For example, "08 June, 2014".
>
> Why not 08-06-2014 for easier date comparison in JS if we ever need to
> do it?
>
> >  hash is a SHA1 (for now) hash of the database file's raw content.
>
> What encoding should we use for the hash? The default from
>
> https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Reference/Interface/nsICryptoHash#finish%28%29
> is base64, so I would lean towards that.
>
> Also, it's better to specify SHA1 somewhere in the update.json file in
> case anyone is reading it independently. This could either be an
> additional field, or we could use the format
> "sha1/5R0zeLx7EWRxqw6HRlgCRxNLHDo=" (<name of hash function>/<base
> 64-encoded string).
>
> > Pseudocode of update procedure [...]
>
> line 9: "update.version" should be "updateData.version"
>
> "inauthentic" should be initialized to true instead of false so that if
> isValidSignature throws an error or gets skipped, nothing is updated
> (generally safer).
>
> updateData.source should be validated to be a valid eff.org URL before
> fetching it.
>
> if any of the XHR  fails (due to network disconnectivity, for instance),
> we should try again in 5 minutes plus padding.
>
>
>
> On 06/17/2014 06:18 PM, Red wrote:
> > I have recently been working on updating the ruleset updater spec to
> > reflect the changes that Yan and I discussed during our meeting last
> > Friday (notes at
> > https://gist.github.com/redwire/b62f03905a826e79947a#week-5), so I
> > wanted to inform everyone of the changes.  If anyone would like to have
> > a look over the document, I'd be happy to receive any suggestions for
> > improvements.
> > https://gist.github.com/redwire/2e1d8377ea58e43edb40
> >
> > Another thing I want to report is that I have finished updating the
> > utility script I have also been working on to automate most of the
> > process of building the update.json file.  One thing I have added into
> > it is a simple mechanism for creating and applying sanity checks to
> > input values.  One such test is the "valid_eff_url" function I wrote
> > that attempts to match the provided source (directing the extension to
> > the location of the ruleset database file) to a valid eff.org URL
> > (ending with .sqlite).  I wrote the regular expression here specifically
> > to be exactly the same as what would be used to repeat this sanity check
> > in the update mechanism itself.  I have tested the regex in both Python
> > and Javascript and found it behaves the same in Javascript and Python,
> > but as Yan suggested, it seems obvious that several others should review
> > this regex to be sure.  Here is a direct link to the beginning of the
> > function.
> >
> https://github.com/redwire/https-everywhere/blob/makeJSONManifest/utils/ruleset_update_manifest.py#L58
> >
> > Cheers,
> > Zack
> >
> >
> >
> > _______________________________________________
> > HTTPS-Everywhere mailing list
> > HTTPS-Everywhere at lists.eff.org
> > https://lists.eff.org/mailman/listinfo/https-everywhere
> >
>
>
> --
> Yan Zhu  <yan at eff.org>, <yan at torproject.org>
> Staff Technologist
> Electronic Frontier Foundation                  https://www.eff.org
> 815 Eddy Street, San Francisco, CA  94109       +1 415 436 9333 x134
>
>
> _______________________________________________
> HTTPS-Everywhere mailing list
> HTTPS-Everywhere at lists.eff.org
> https://lists.eff.org/mailman/listinfo/https-everywhere
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.eff.org/pipermail/https-everywhere/attachments/20140618/b5c4796b/attachment.html>