IRC log of musicbrainz-devel on 2011-12-18
Timestamps are in UTC.
- 00:58:48 [the_metalgamer]
- the_metalgamer has joined #musicbrainz-devel
- 04:55:45 [CallerNo6]
- CallerNo6 has joined #musicbrainz-devel
- 05:15:36 [Leftmost]
- Leftmost has joined #musicbrainz-devel
- 06:03:30 [wpl]
- wpl has joined #musicbrainz-devel
- 07:33:19 [djce]
- djce has joined #musicbrainz-devel
- 07:50:04 [kepstin-netbook]
- kepstin-netbook has joined #musicbrainz-devel
- 08:14:42 [bitmap_]
- bitmap_ has joined #musicbrainz-devel
- 09:28:40 [kepstin-laptop]
- kepstin-laptop has joined #musicbrainz-devel
- 09:44:50 [djce]
- * djce hits the big UA Blocking switch
- 09:46:19 [djce]
- Any problems? Shout. Serious problems and I'm not responding? Summit attendees with my phone # should call me. Thanks!
- 09:46:38 [djce]
- * djce thinks: I should make sure that a handful of key people have my phone number.
- 09:51:56 [ruaok]
- the web service seems to work ok. ;)
- 09:52:05 [djce]
- I have tests!
- 09:52:09 [djce]
- prove ftw!
- 09:52:24 [ruaok]
- awesome. :)
- 09:52:50 [djce]
- Ah, now actually, a small caveat: for the "Bad" UAs, I haven't only blocked /ws, I've blocked everything other than "/"
- 09:54:41 [djce]
- * djce eyes the green line dropping sharply: http://stats.musicbrainz.org/mrtg/drraw/drraw.cgi?Mode=view;Template=1262469543.28951;Base=%2Fvar%2Fwww%2Fmrtg%2F%2Flenny_ratelimit-default-wsglobal-count.rrd
- 09:55:32 [ruaok]
- what list of UAs did we start blocking now?
- 09:55:44 [ruaok]
- blocking/throttle
- 09:58:39 [djce]
- python-musicbrainz/0.7.3 of course, and it gets a special 403 which links to http://blog.musicbrainz.org/?p=1111
- 09:58:56 [ruaok]
- blocked to throttled?
- 09:59:19 [djce]
- and also the blank user-agent, and ^(Java|Python-urllib|Jakarta Commons-HttpClient)/[0-9._]+$ , which block with a link to http://blog.musicbrainz.org/?p=839
- 09:59:23 [djce]
- just plain blocked.
- 09:59:30 [djce]
- Every url other than "/" gives a 403.
- 09:59:31 [ruaok]
- we can't really block it.
- 09:59:48 [ruaok]
- lets just throttle it.
- 10:00:02 [djce]
- * djce sobs quietly
- 10:00:11 [ruaok]
- awww, now now.
- 10:00:20 [djce]
- that's gonna need a musicbrainz-server release of course.
- 10:00:43 [ruaok]
- argh.
- 10:00:50 [djce]
- also (gets more professional hat on) we don't have a ticket for this,
- 10:00:59 [djce]
- which is why we've confused our requirements.
- 10:01:08 [djce]
- So mr ruaok, two things for you.
- 10:01:12 [djce]
- #1 please create a ticket
- 10:01:22 [djce]
- and #2 do you need me to revert the block right now?
- 10:02:00 [ruaok]
- * ruaok sobs quietly
- 10:02:31 [ruaok]
- do you have an email at hand that describes the changes to mb-server that are needed?
- 10:03:10 [djce]
- maybe. It's kind of the one that I outlined a month or two ago about completely reworking the way we do blocking.
- 10:03:48 [djce]
- closest ticket I can find is http://tickets.musicbrainz.org/browse/MBS-673 btw
- 10:04:17 [djce]
- although MBS ticket, implemented as MBH ... <meh>. Hence the confusion.
- 10:04:47 [ruaok]
- go ahead and move it.
- 10:05:21 [djce]
- done. http://tickets.musicbrainz.org/browse/MBH-191
- 10:05:24 [ruaok]
- in looking for emails I found: Dave: Please do your part and then create an MBS ticket for oliver on what he needs to do for musicbrainz-server."
- 10:05:32 [ruaok]
- loi
- 10:06:20 [ruaok]
- Thought #2: so that would make each WS request check three rate limits:
- 10:06:20 [ruaok]
- check_ratelimit("ws ip=$ip")
- 10:06:20 [ruaok]
- check_ratelimit("ws ua=$user_agent")
- 10:06:20 [djce]
- Well let's start with collecting requirements.
- 10:06:20 [ruaok]
- check_ratelimit("ws global")
- 10:06:27 [djce]
- then work out the implementation later.
- 10:06:39 [djce]
- I need to head out in the next 5 min.
- 10:06:39 [ruaok]
- * ruaok should really be working on shut-eye
- 10:06:47 [djce]
- Do you need me to revert the block?
- 10:07:04 [ruaok]
- I may regret this decision.
- 10:07:10 [ruaok]
- No. lets keep it in place.
- 10:07:19 [ruaok]
- it will do a few things:
- 10:07:22 [ruaok]
- 1. piss people off
- 10:07:28 [djce]
- ok. I'll be back within a couple of hours
- 10:07:38 [ruaok]
- 2. motivate to get moving on fixing things in the right way
- 10:07:43 [djce]
- nod
- 10:07:47 [ruaok]
- 3. get people to finally contact us.
- 10:08:05 [ruaok]
- can you please collect the requirements and make a high prioroty ticket for ocharles?
- 10:08:22 [ruaok]
- and then we go to throttling as soon as this ticket is done.
- 10:08:43 [djce]
- Clearly I don't know the requirements, unless it's just "like we have right now, but throttling".
- 10:08:44 [djce]
- is that it?
- 10:08:58 [ruaok]
- yes.
- 10:09:02 [djce]
- ok
- 10:09:07 [ruaok]
- and I think the cruz lies here: check_ratelimit("ws ua=$user_agent")
- 10:09:25 [ruaok]
- that needs to be added to mb-server so we can add ua based throttling, right?
- 10:09:35 [ruaok]
- since we have up and global already////
- 10:09:40 [ruaok]
- ...
- 10:09:52 [ruaok]
- *ip
- 10:09:56 [ruaok]
- * ruaok needs sleep
- 10:10:07 [djce]
- I want to change mb-server so that it just sends one ratelimit request per http request
- 10:10:19 [djce]
- but I'll write that up in an hour or two
- 10:10:27 [ruaok]
- with three args to one call?
- 10:10:29 [ruaok]
- sounds great.
- 10:10:35 [djce]
- ip, ua
- 10:10:42 [djce]
- and probably url
- 10:10:43 [ruaok]
- not global?
- 10:10:52 [djce]
- do that in ratelimit-server
- 10:11:00 [ruaok]
- ok, sounds good.
- 10:11:08 [djce]
- * djce goes grocery shopping
- 10:11:13 [ruaok]
- please cc me on the write-up or send me a note on where I can find.
- 10:11:15 [ruaok]
- ta
- 10:11:16 [ruaok]
- nn!
- 10:11:20 [djce]
- sure
- 10:11:22 [djce]
- night!
- 10:11:23 [ruaok]
- thx
- 11:05:19 [nikki]
- ruaok: nobody ever posted on the blog to say that pymb2 now supports setting a user agent
- 11:29:53 [nikki]
- hmm...
- 11:30:14 [nikki]
- it seems you can't import the data dumps using master now
- 11:37:15 [warp]
- did we break it?
- 11:37:54 [nikki]
- the ref_count column has been removed from tag and url, but the data dumps apparently still have those columns
- 11:38:19 [nikki]
- http://forums.musicbrainz.org/viewtopic.php?id=3281
- 11:39:34 [warp]
- ah
- 11:39:45 [warp]
- odd.
- 11:40:05 [warp]
- I would have expected us to not make such changes until the schema change release.
- 11:40:11 [nikki]
- yeah, same
- 11:40:30 [nikki]
- because there's no way to find out which tags will work with the current dumps
- 11:41:19 [warp]
- sure there is, try them each in turn :)
- 11:41:24 [nikki]
- well, technically you can by reading the sql files
- 11:41:31 [nikki]
- you know what I mean though :P
- 11:41:44 [nikki]
- it shouldn't need trial and error, basically
- 11:41:53 [warp]
- * warp nods.
- 12:34:24 [luks]
- isn't the production branch what people should be using?
- 12:34:41 [nikki]
- * nikki has no idea
- 12:34:57 [nikki]
- I understood how it worked with svn :P
- 12:35:19 [luks]
- where you had to be pick the right branch :)
- 12:36:00 [nikki]
- you just copied the version number from the bottom of the website
- 12:36:31 [luks]
- hm, the data dumps used to have a link to the branch as well
- 12:36:35 [luks]
- I wonder if they still point somewhere
- 12:37:16 [warp]
- * warp doesn't like the distinction between production and master.
- 12:37:40 [nikki]
- * nikki kicks all the 502s on beta
- 13:36:39 [Mineo]
- Mineo has joined #musicbrainz-devel
- 14:04:33 [alastairp]
- alastairp has joined #musicbrainz-devel
- 14:25:07 [luks]
- any opinions on http://dl.dropbox.com/u/5215054/tmp/mb_filter.png ? (UI-wise)
- 14:28:25 [nikki]
- * nikki is not sure what the icon next to filter is
- 14:31:18 [luks]
- it's a funnel, often used as an icon for filtering :)
- 14:31:23 [luks]
- but I see that I should remove it :)
- 14:31:56 [Leftmost]
- Leftmost has joined #musicbrainz-devel
- 14:32:08 [nikki]
- oh
- 14:32:37 [luks]
- at first I wanted to have just the icon, but a little larger
- 14:33:16 [nikki]
- I guess I haven't seen enough filtering things, I can't even think of any I could look at
- 14:34:28 [xdnny]
- xdnny has joined #musicbrainz-devel
- 14:40:16 [nikki]
- luks: oh, and I assume "name" refers to the the recording name, not the artist name?
- 14:41:08 [luks]
- yes
- 14:41:34 [luks]
- plus there would be more options for releases
- 14:42:13 [nikki]
- ah
- 15:28:18 [hrglgrmpf]
- hrglgrmpf has joined #musicbrainz-devel
- 16:38:06 [the_metalgamer]
- the_metalgamer has joined #musicbrainz-devel
- 16:38:25 [kepstin-laptop]
- kepstin-laptop has joined #musicbrainz-devel
- 16:44:56 [reosarevok]
- reosarevok has joined #musicbrainz-devel
- 17:21:20 [the_metalgamer]
- the_metalgamer has joined #musicbrainz-devel
- 17:31:39 [the_metalgamer]
- the_metalgamer has joined #musicbrainz-devel
- 17:52:24 [CallerNo6]
- CallerNo6 has joined #musicbrainz-devel
- 18:40:48 [kepstin]
- kepstin has joined #musicbrainz-devel
- 19:40:07 [ruaok]
- ruaok has joined #musicbrainz-devel
- 20:02:27 [hawke_]
- hawke_ has joined #musicbrainz-devel
- 20:10:37 [hawke_]
- hawke_ has joined #musicbrainz-devel
- 20:31:32 [djce]
- djce has joined #musicbrainz-devel
- 20:32:21 [hrglgrmp1]
- hrglgrmp1 has joined #musicbrainz-devel
- 21:12:02 [djce]
- djce has joined #musicbrainz-devel
- 22:05:37 [hawke_]
- hawke_ has joined #musicbrainz-devel
- 22:24:54 [hawke_]
- hawke_ has joined #musicbrainz-devel