[HTTPS-Everywhere] Maintainability changes to HTTPS Everywhere

[Ian Cordasco]

On Mon, Jan 26, 2015 at 4:41 PM, Jacob Hoffman-Andrews <jsha at eff.org> wrote:
> Hi all,
>
> HTTPS Everywhere is growing rapidly. We have more rulesets than ever
> (14k+), and more volunteers than ever (thank you)! The downside of our
> growth is that it's increasingly hard to provide a high-quality product.
> I'm planning, with your help, to add two new processes to improve our
> collective maintenance capability: better automated testing and a
> clearer branching strategy.
>
> Better automated testing
>
> Right now we have 3.2k rulesets in the stable branch and 14k in master.
> It's time to promote the master branch to stable. Among other things,
> master has e10s support, while stable does not. However, there are a
> large number of rules that completely break their target sites. See for
> instance:  https://github.com/EFForg/https-everywhere/issues/529
> https://github.com/EFForg/https-everywhere/issues/849
> https://github.com/EFForg/https-everywhere/pull/931
>
> Promoting master branch to stable will break many websites for many
> users, which is not okay. We have a set of ruleset tests from 2013
> (https://github.com/EFForg/https-everywhere/blob/master/src/chrome/content/ruleset-tests.js),
> but these are only designed to detect mixed content. Also, by default
> they run for all 14k rulesets, which means loading 24k URLs. And they
> run a very simple heuristic to figure out which URLs to load.
>
> We need to automate and improve the ruleset tests. We need to add a
> test-url syntax to our ruleset files so we can specify which URLs to
> load. Newly added rulesets must include test URLs, and we need to
> retroactively add test URLs to existing rulesets (at first with an
> automated process, then later with manual maintenance). And the ruleset
> tests need to produce output that is easily used to disable failing
> rulesets. I've broken this down into tasks here:
> https://github.com/EFForg/https-everywhere/labels/Ruleset%20Testing
>
> Clearer branching strategy
>
> Right now it's not clear when you should merge to 4.0 (stable) vs when
> you should merge to master. Often we merge ruleset fixes to master, then
> later decide that they are important enough to cherry-pick into 4.0.
> This cherry-picking makes it very challenging to merge 4.0 into master,
> because git cannot recognize that the cherry-pick commits are already
> merged.
>
> I propose a new branching strategy:
>
> - Code: All bug fixes must be merged to 4.0 first.
> - Code: All new features or refactorings must be merged to master.
> - Rulesets: Any change to a ruleset that exists in 4.0 (stable) must be
> merged to 4.0 first.
> - Rulesets: Any new ruleset, or change to a ruleset that does not yet
> exist in 4.0, must be merged to master, and will not be cherry-picked
> into 4.0, barring exceptional circumstances.
>
> Since GitHub automatically opens new pull requests against master, it
> will be easy to make mistakes. I propose to write a webhook, similar to
> Travis CI, that will check new pull requests to see if they are made
> against the right branch, and add an indicator if they are not
> (https://github.com/EFForg/https-everywhere/issues/982). It will then be
> the responsibility of the requestor to change the target branch, since
> repo owners can't change that.

I just wanted to point out that contributors can change the target
branch of PR (last I checked) either. They'll have to close their
existing PR and open a new one. If you want that branch to be
targetted by default, you should make it the default branch through
the GitHub UI. I can walk you through this over IRC if you're
unfamiliar with it.

> All of these changes are a significant amount of work. If you would like
> to pitch in and help bring us closer to a 5.0-stable release, please
> comment on the specific issue you'd like to take.
>
> Thanks again for all your help. HTTPS Everywhere has a great community.
>
> Jacob
>
> _______________________________________________
> HTTPS-Everywhere mailing list
> HTTPS-Everywhere at lists.eff.org
> https://lists.eff.org/mailman/listinfo/https-everywhere