[HTTPS-Everywhere] Blekko search engine rewriting search results to point to HTTPS URLs

[Dan Kaminsky]

On Fri, Apr 20, 2012 at 3:23 PM, Peter Eckersley <pde at eff.org> wrote:

> On Fri, Apr 13, 2012 at 07:18:08PM -0700, Greg Lindahl wrote:
> > On Fri, Mar 30, 2012 at 01:47:19AM -0400, mezzanine at Safe-mail.net wrote:
> >
> > > Of possible interest, it appears that the Blekko search engine
> > > automatically rewrites some of its search results
> >
> > Indeed, we are. Most of our rewrites come out of the HTTPS Everywhere
> > rulesets, thanks! We think this is a way to improve privacy for a lot
> > more people than just those who might download a browser
> > extension. We've defaulted this feature on in our search engine for a
> > long time now.
> >
> > We have a button people can push to complain when we've broken the web
> > for them. We get a few complaints a day, mostly from countries which I
> > imagine are pretty enthusiastic at blocking encrypted access to
> > naughty websites (like facebook, youtube, etc.) I generally email such
> > people and they never reply back: probably most of them don't speak
> > English.
> >
> > I got one a few days ago from someone in the Dominican Republic, who
> > pushed the "this link is broken" button 3 times on a YouTube link, and
> > once added a comment in Spanish that clearly said that all YouTube
> > links were broken. The link worked for me. He also didn't respond to
> > my email with an attempt at Spanish, but I thought I'd mention it
> > here: do you guys have any idea how much breakage is being caused by
> > https blocks in various countries?
>
> We do not have this dataset, but I was at a hackathon last weekend where
> Dan
> Kaminksy showed a nifty hack that Blekko /might/ be able to use to collect
> this data itself under certain circumstances.  If you embedd a simple image
> (such as a favicon, maybe with a query parameter to prevent caching) from a
> site, you can use CSS introspection to see if each copy loaded.  Do this
> for
> http and https in parallel, and you'll spot differential blocking.
>

It's not CSS introspection; you basically create a new image with onload
and onerror handlers, then set the src to favicon.ico.  See
censorsweeper.com.

Peter, that's a hilarious use of my hack, and it'll totally work.


>
> There are a couple of arguments against doing this for your users at
> random:
>
> 1. It will cause them to make requests to YouTube/Twitter/Facebook, which
> causes some privacy loss (especially if you don't have a way to #scrub the
> search terms from the referrer, and if the user has browser link
> prefetching
> disabled)
>

Referer is blocked to HTTPS, at least.


>
> 2. If the user is searching over HTTPS, it will cause a mixed content
> issue.
> But this could be worked-around by only doing it for HTTP users (people who
> search Blekko over HTTPS might be using HTTPS Everywhere anyway, which
> would
> mess up the HTTP/HTTPS differential aspect of the test results).
>

I think maybe you can hide the mixed content warnings if you do this in an
invisible iframe.  Images and mixed content have very weird policies.



>
> You could also do this test for users when they click "report broken
> link", or
> an ad saying "test your network with Blekko", etc, although your sample
> size
> will be much smaller.
>
> >
> > The set of urls which attract complaints is very limited: facebook,
> > twitter, and youtube. Of course, these are the most popular searches
> > at all search engines, "navigational" searches.
> >
> > -- greg
> >
> > _______________________________________________
> > HTTPS-everywhere mailing list
> > HTTPS-everywhere at mail1.eff.org
> > https://mail1.eff.org/mailman/listinfo/https-everywhere
>
> --
> Peter Eckersley                            pde at eff.org
> Technology Projects Director      Tel  +1 415 436 9333 x131
> Electronic Frontier Foundation    Fax  +1 415 436 9993
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.eff.org/pipermail/https-everywhere/attachments/20120421/e1e3fbd9/attachment.html>