[Edbrowse-dev] user agent spoofing

[Edbrowse-dev] user agent spoofing

[Kevin Carhart]

I happened to be in the section on user-agent spoofing in the manual.  I 
recently listened to a fascinating talk from an Infosec dude called Nick 
Nikiforakis, and I can report back with chagrin that in its simplest 
form, just calling yourself a Mozilla may no longer fly depending on what 
kind of website the user is trying to go to.  It's astonishing the kinds 
of devious tricks they use.   In addition to reading the settable user 
agent string, they test hundreds of things over javascript, and create a 
unique hash based on the answers.

For instance, they attempt to generate some text in a given font.  If the 
browser returns "I don't have it," this amounts to the answer to a 
yes-or-no question.  Do that for hundreds of fonts and you have quite a 
unique hash.

Another one is that they will run a script that carries out some floating 
point math.  Then they examine the sequence of numbers in the extended 
decimal places.  This may have a consistency which can be used as part of 
a fingerprint.  What a nuisance.  They overlay as many of these tests as 
they can.  It stands to reason that the most commercial sites would leave 
no stone unturned, but I still slapped my forehead, "of course."

So for pages like startpage.com, which "won't let you in the door unless 
you look like one of the major players," or the old example of an online 
banking site, just setting ua1, ua2, will unfortunately be of declining 
effectiveness if more sites use fingerprinting.  Startpage could have new 
ways of saying "We don't think you are really a new Mozilla as you claim."

Kevin



--------
Kevin Carhart * 415 225 5306 * The Ten Ninety Nihilists

Re: [Edbrowse-dev] user agent spoofing

[Chris Brannon]

Kevin Carhart <kevin@carhart.net> writes:

> In addition to reading the settable user agent string, they test
> hundreds of things over javascript, and create a unique hash based on
> the answers.

Well, this is true.  But spoofing the user-agent header is still often
quite effective.  There are sites out there that block edbrowse for no
good reason.  I seem to remember that in the past an example was
kpfa.org or kpfk.org, don't remember which one, but it was one of the
Pacifica stations.  They fixed this at some point.
But spoofing user-agent turned a 403 to a 200.
There are other sites that autoblock edbrowse; I find them from time to
time.  They don't autoblock lynx.  So I wonder, is someone using
edbrowse for "questionable" purposes?  Five will get you ten that they
are.  It's also possible that someone has preemptively added edbrowse to
some sort of list or lists of "browsers that should be blocked", after
deciding that it had a possibility of "mis-use".  Let's not kid
ourselves here; I'm sure there are those who consider edbrowse highly
subversive.

-- Chris