[Bug 758551] New: A way to block batch users of nominatim.gnome.org proxy

["sysadmin" (GNOME Bugzilla) <bugzilla gnome org>]

Bug ID	758551
Summary	A way to block batch users of nominatim.gnome.org proxy
Classification	Infrastructure
Product	sysadmin
Version	unspecified
OS	Linux
Status	NEW
Severity	normal
Priority	Normal
Component	Other
Assignee	sysadmin-maint@gnome.bugs
Reporter	jonas@threetimestwo.org
QA Contact	sysadmin-maint@gnome.bugs
GNOME version	---

We have been contacted by OpenStreetMap sysadmin team about more control of who
uses our proxy, right now they can only ban none of us or all of us. See mail
below for more information.

Would it be possible to detect and/or ban the current batch users before we are
able to implement something in geocode-glib to identify clients? Or would it be
possible to add some IP-identifier to the user-agent?


-----

Hi,

we've been in contact a while ago about set up some exception for
geocoding traffic from gnome proxies for nominatim.openstreetmap.org.
This has worked out okay so far but lately I started to notice large
batches of bulk requests coming from the gnome servers.

I've also noticed on that occasion that you still haven't implemented
any means to identify the clients you are doing the proxying for. This
is problematic for two reasons. First of all, there are no means for me
to ban misusers on an individual base. If the bulk traffic gets worse,
I will have to ban all traffic from the gnome servers. The second issue
is that due to the massive increase in traffic we have started to put
a blanket ban on all reverse geocoding requests that fail to send user
agent and referer. A the moment the gnome servers are still exempt but
I can't guarantee that this will be technically feasible forever.

So, for the short term, could you please have a look at your servers
and see if you can block the bulk users of the API. Here are some
example log entries for the requests I'm seeing at the moment:

209.132.180.180 - - [23/Nov/2015:13:32:43 +0000] "GET
/search?q=Performance%20Polymers,%20350%20Bellevue%20Rd,%20Newark,%20DE%2019714,%20United%20States%0A&limit=1&bounded=0&addressdetails=1&accept-language=C&format=jsonv2&email=zeeshanak%40gnome.org
HTTP/1.1" 200 3429 "-" "-"
209.132.180.180 - - [23/Nov/2015:13:32:44 +0000] "GET
/search?q=Performance%20Polymers,%20Discovery%20Drive,%20Hopewell,%20VA%2023860-0000,%20United%20States%0A&limit=1&bounded=0&addressdetails=1&accept-language=C&format=jsonv2&email=zeeshanak%40gnome.org
HTTP/1.1" 200 3429 "-" "-"
209.132.180.180 - - [23/Nov/2015:13:32:44 +0000] "GET
/search?q=5401%20Jefferson%20Davis%20Highway,%20Richmond,%20VA%20232340000,%20United%20States%0A&limit=1&bounded=0&addressdetails=1&accept-language=C&format=jsonv2&email=zeeshanak%40gnome.org
HTTP/1.1" 200 3429 "-" "-"
209.132.180.180 - - [23/Nov/2015:13:32:44 +0000] "GET
/search?q=Rt%20141%20Henry%20Clay,%20Wilmington,%20DE%2019880,%20United%20States%0A&limit=1&bounded=0&addressdetails=1&accept-language=C&format=jsonv2&email=zeeshanak%40gnome.org
HTTP/1.1" 200 3429 "-" "-"
209.132.180.180 - - [23/Nov/2015:13:32:44 +0000] "GET
/search?q=Protection%20Technologies,%205401%20Jefferson%20Davis%20Highway,%20Richmond,%20VA%20232340000,%20United%20States%0A&limit=1&bounded=0&addressdetails=1&accept-language=C&format=jsonv2&email=zeeshanak%40gnome.org
HTTP/1.1" 200 3429 "-" "-"

In the long term I would kindly ask you to implement some means to
send the name of the application using the gnome geocoding library
in the user agent. It would greatly help us to filter out the bad
users and keep the service up for the well-behaved ones.

Kind regards

Sarah Hoffmann
(part of the OSM sysadmin team)

---

      You are receiving this mail because:

• You are watching the QA Contact of the bug.
• You are watching the assignee of the bug.