[HTTPS-Everywhere] Implement build of specific branch in makexpi.sh
Peter Eckersley
pde at eff.org
Tue Oct 4 17:15:59 PDT 2011
It's nice to have this functionality back, though I had to make several
further changes
(https://gitweb.torproject.org/https-everywhere.git/commitdiff/431db1a7436dbef37d61426b103af9906d9bed4e).
One conceptual issue with the script as it stands is that it partially tries
to use the current revision's validation logic on the old target's codebase.
This can cause bugs, as was quickly demonstrated by trying to build
2.0development.1 and having that fail because Seth had subsequently added a
script to check that all the translations were valid.
There isn't any really nice solution to this problem. I hacked my way around
the specific problem with translations, but future changes to makexpi will
need similar hacks. Calling make <target name> inside the checkout/ directory
seems conceptually right, but that would currently recurse to infinite depth!
One possible hack is to do something along these lines:
checkout the target to build in checkout/
cd checkout
make
cd ..
assert (there is one xpi file in checkout/pkg/)
cp checkout/pkg/*.xpi pkg/https-everywhere-<target>.xpi
On Sat, Oct 01, 2011 at 11:07:01PM -0400, Alex Xu wrote:
> Pushed to GitHub. https://github.com/Hello71/HTTPS-Everywhere
>
> Attached new diff combining both patches.
>
> A side note: Fun how the patch size is larger than the actual script.
>
> On 11-10-01 02:42 PM, Peter Eckersley wrote:
> > For some reason the patch isn't applying, and "git apply" isn't giving me any
> > meaningful feedback. Can you apply this and push it to your remote?
> >
> > On Sat, Oct 01, 2011 at 01:32:49PM -0400, Alex Xu wrote:
> >> Increased portability and efficiency. Also removed "multi-line comment"
> >> on trivial-validate included for testing; forgot to remove in last diff.
> >>
> >> On 11-09-30 11:45 PM, Eitan Adler wrote:
> >>> On Fri, Sep 30, 2011 at 10:50 PM, Alex Xu <alex_y_xu at yahoo.ca> wrote:
> >>>> Also cleaned up inaccurate documentation in makexpi.sh.
> >>>>
> >>>> Did basic testing and it appears to work.. decently.
> >>>>
> >>>> Probably needs some code cleanup before merge, though.
> >>>
> >>> Some portability notes:
> >>>
> >>> + cp --recursive --archive .git $SUBDIR
> >>>
> >>> Please use cp -r -a
> >>>
> >>> + mkdir --parents $XPI_NAME
> >>>
> >>> Please use mkdir -p
> >>>
> >>> + printf >&2 "Total included rules: $(ls chrome/content/rules/*.xml
> >>> | wc -l)\n"
> >>>
> >>> Don't script on ls: use find instead.
> >>>
> >>> While here can you please fix the sed lines? it appears that they are
> >>> using some non-standard extensions (although this was not introduced
> >>> by your patch).
> >>>
> >>>
> >>>>
> >>>> _______________________________________________
> >>>> HTTPS-everywhere mailing list
> >>>> HTTPS-everywhere at mail1.eff.org
> >>>> https://mail1.eff.org/mailman/listinfo/https-everywhere
> >>>>
> >>>>
> >>>
> >>>
> >>>
> >
> >> diff --git a/makexpi.sh b/makexpi.sh
> >> index 8d87ab9..52cf9a6 100755
> >> --- a/makexpi.sh
> >> +++ b/makexpi.sh
> >> @@ -20,12 +20,11 @@ if [ -n "$1" ]; then
> >> BRANCH=$(git branch | head -n 1 | cut -d \ -f 2-)
> >> SUBDIR=checkout
> >> [ -d $SUBDIR ] || mkdir $SUBDIR
> >> - cp --recursive --archive .git $SUBDIR
> >> + cp -r -a .git $SUBDIR
> >> cd $SUBDIR
> >> git reset --hard "$1"
> >> fi
> >>
> >> -if false; then
> >> if ./trivial-validate src/chrome/content/rules >&2
> >> then
> >> echo Validation of included rulesets completed. >&2
> >> @@ -34,7 +33,6 @@ else
> >> echo ERROR: Validation of rulesets failed. >&2
> >> exit 1
> >> fi
> >> -fi
> >>
> >> if [ -n "$1" ]; then
> >> VERSION="$1"
> >> @@ -43,7 +41,7 @@ else
> >> fi
> >>
> >> XPI_NAME="pkg/$APP_NAME-$VERSION.xpi"
> >> -mkdir --parents $XPI_NAME
> >> +mkdir -p $XPI_NAME
> >> rmdir $XPI_NAME
> >>
> >> cd "src"
> >> @@ -53,9 +51,9 @@ echo "</rulesetlibrary>" >> chrome/content/rules/default.rulesets
> >> echo "Removing whitespaces and comments..."
> >> sed -i -e :a -re 's/<!--.*?-->//g;/<!--/N;//ba' chrome/content/rules/default.rulesets
> >> sed -i ':a;N;$!ba;s/\n//g' chrome/content/rules/default.rulesets
> >> -sed -i 's/>[[:blank:]]*</></g' chrome/content/rules/default.rulesets
> >> -sed -i 's/[[:blank:]]*to=/ to=/g' chrome/content/rules/default.rulesets
> >> -sed -i 's/[[:blank:]]*from=/ from=/g' chrome/content/rules/default.rulesets
> >> +sed -i 's/>[ ]*</></g' chrome/content/rules/default.rulesets
> >> +sed -i 's/[ ]*to=/ to=/g' chrome/content/rules/default.rulesets
> >> +sed -i 's/[ ]*from=/ from=/g' chrome/content/rules/default.rulesets
> >> sed -i 's/ \/>/\/>/g' chrome/content/rules/default.rulesets
> >> touch -r chrome/content/rules chrome/content/rules/default.rulesets
> >> rm -f "../$XPI_NAME"
> >> @@ -66,8 +64,8 @@ if [ "$ret" != 0 ]; then
> >> rm -f "../$XPI_NAME"
> >> exit "$?"
> >> else
> >> - printf >&2 "Total included rules: $(ls chrome/content/rules/*.xml | wc -l)\n"
> >> - printf >&2 "Rules disabled by default: $(grep -lr default_off chrome/content/rules | wc -l)\n"
> >> + printf >&2 "Total included rules: $(find -name "chrome/content/rules/*.xml" | wc -l)\n"
> >> + printf >&2 "Rules disabled by default: $(grep -lrc default_off chrome/content/rules)\n"
> >> printf >&2 "Created %s\n" "$XPI_NAME"
> >> if [ -n "$BRANCH" ]; then
> >> cd ../..
> >
> >> _______________________________________________
> >> HTTPS-everywhere mailing list
> >> HTTPS-everywhere at mail1.eff.org
> >> https://mail1.eff.org/mailman/listinfo/https-everywhere
> >
> >
> diff --git a/makexpi.sh b/makexpi.sh
> index bdb9779..52cf9a6 100755
> --- a/makexpi.sh
> +++ b/makexpi.sh
> @@ -4,11 +4,7 @@ APP_NAME=https-everywhere
> # builds a .xpi from the git repository, placing the .xpi in the root
> # of the repository.
>
> -# invoke with no arguments to pull a prerelease of the current
> -# development point.
> -
> -# invoke with the literal argument "uncommitted" to build from the
> -# current src directory.
> +# invoke with no arguments to build from the current src directory.
>
> # invoke with a tag name to build a specific branch or tag.
>
> @@ -18,13 +14,17 @@ APP_NAME=https-everywhere
> # or just:
> # ./makexpi.sh
>
> -# BUGS: if you have a branch or tagged named "uncommitted" then this
> -# is kind of ambiguous. Also, the validation of rule syntax is done
> -# against the rules in the current directory, not necessarily the
> -# committed rules in git.
> -
> cd "$(dirname $0)"
>
> +if [ -n "$1" ]; then
> + BRANCH=$(git branch | head -n 1 | cut -d \ -f 2-)
> + SUBDIR=checkout
> + [ -d $SUBDIR ] || mkdir $SUBDIR
> + cp -r -a .git $SUBDIR
> + cd $SUBDIR
> + git reset --hard "$1"
> +fi
> +
> if ./trivial-validate src/chrome/content/rules >&2
> then
> echo Validation of included rulesets completed. >&2
> @@ -34,24 +34,15 @@ else
> exit 1
> fi
>
> -if [ -n "$1" ] && [ "$1" != "uncommitted" ]; then
> +if [ -n "$1" ]; then
> VERSION="$1"
> - TARG="$1"
> else
> VERSION="$(grep em:version src/install.rdf | sed -e 's/[<>]/ /g' | cut -f3)~pre"
> - TARG=HEAD
> - if [ "$1" != "uncommitted" ] && [ -n "$(git status src -s)" ] ; then
> - printf >&2 "\
> -WARNING: There are uncommitted changes in your current repostitory.
> -WARNING: These changes will not be included in the generated .xpi
> -WARNING: Run 'git status' for information about the uncommitted changes.
> -WARNING: Or, use 'makexpi.sh uncommitted' to include them in the build.
> -"
> - fi
> fi
>
> XPI_NAME="pkg/$APP_NAME-$VERSION.xpi"
> -[ -d pkg ] || mkdir pkg
> +mkdir -p $XPI_NAME
> +rmdir $XPI_NAME
>
> cd "src"
> echo "<rulesetlibrary>" > chrome/content/rules/default.rulesets
> @@ -60,15 +51,12 @@ echo "</rulesetlibrary>" >> chrome/content/rules/default.rulesets
> echo "Removing whitespaces and comments..."
> sed -i -e :a -re 's/<!--.*?-->//g;/<!--/N;//ba' chrome/content/rules/default.rulesets
> sed -i ':a;N;$!ba;s/\n//g' chrome/content/rules/default.rulesets
> -sed -i 's/>[[:blank:]]*</></g' chrome/content/rules/default.rulesets
> -sed -i 's/[[:blank:]]*to=/ to=/g' chrome/content/rules/default.rulesets
> -sed -i 's/[[:blank:]]*from=/ from=/g' chrome/content/rules/default.rulesets
> +sed -i 's/>[ ]*</></g' chrome/content/rules/default.rulesets
> +sed -i 's/[ ]*to=/ to=/g' chrome/content/rules/default.rulesets
> +sed -i 's/[ ]*from=/ from=/g' chrome/content/rules/default.rulesets
> sed -i 's/ \/>/\/>/g' chrome/content/rules/default.rulesets
> touch -r chrome/content/rules chrome/content/rules/default.rulesets
> rm -f "../$XPI_NAME"
> -if [ -n "$CHANGES" ]; then
> - printf >&2 "WARNING: uncommitted changes were included:\n%s\n" "$CHANGES"
> -fi
> zip -q -X -9r "../$XPI_NAME" . "-x at ../.build_exclusions"
>
> ret="$?"
> @@ -76,7 +64,12 @@ if [ "$ret" != 0 ]; then
> rm -f "../$XPI_NAME"
> exit "$?"
> else
> - printf >&2 "Total included rules: $(ls chrome/content/rules/*.xml | wc -l)\n"
> - printf >&2 "Rules disabled by default: $(grep -lr default_off chrome/content/rules | wc -l)\n"
> + printf >&2 "Total included rules: $(find -name "chrome/content/rules/*.xml" | wc -l)\n"
> + printf >&2 "Rules disabled by default: $(grep -lrc default_off chrome/content/rules)\n"
> printf >&2 "Created %s\n" "$XPI_NAME"
> + if [ -n "$BRANCH" ]; then
> + cd ../..
> + cp $SUBDIR/$XPI_NAME pkg
> + rm -rf $SUBDIR
> + fi
> fi
> _______________________________________________
> HTTPS-everywhere mailing list
> HTTPS-everywhere at mail1.eff.org
> https://mail1.eff.org/mailman/listinfo/https-everywhere
--
Peter Eckersley pde at eff.org
Technology Projects Director Tel +1 415 436 9333 x131
Electronic Frontier Foundation Fax +1 415 436 9993
More information about the HTTPS-everywhere
mailing list