IRC log of musicbrainz-devel on 2011-12-18

Timestamps are in UTC.

00:58:48 [the_metalgamer]
the_metalgamer has joined #musicbrainz-devel
04:55:45 [CallerNo6]
CallerNo6 has joined #musicbrainz-devel
05:15:36 [Leftmost]
Leftmost has joined #musicbrainz-devel
06:03:30 [wpl]
wpl has joined #musicbrainz-devel
07:33:19 [djce]
djce has joined #musicbrainz-devel
07:50:04 [kepstin-netbook]
kepstin-netbook has joined #musicbrainz-devel
08:14:42 [bitmap_]
bitmap_ has joined #musicbrainz-devel
09:28:40 [kepstin-laptop]
kepstin-laptop has joined #musicbrainz-devel
09:44:50 [djce]
* djce hits the big UA Blocking switch
09:46:19 [djce]
Any problems? Shout. Serious problems and I'm not responding? Summit attendees with my phone # should call me. Thanks!
09:46:38 [djce]
* djce thinks: I should make sure that a handful of key people have my phone number.
09:51:56 [ruaok]
the web service seems to work ok. ;)
09:52:05 [djce]
I have tests!
09:52:09 [djce]
prove ftw!
09:52:24 [ruaok]
awesome. :)
09:52:50 [djce]
Ah, now actually, a small caveat: for the "Bad" UAs, I haven't only blocked /ws, I've blocked everything other than "/"
09:54:41 [djce]
* djce eyes the green line dropping sharply: http://stats.musicbrainz.org/mrtg/drraw/drraw.cgi?Mode=view;Template=1262469543.28951;Base=%2Fvar%2Fwww%2Fmrtg%2F%2Flenny_ratelimit-default-wsglobal-count.rrd
09:55:32 [ruaok]
what list of UAs did we start blocking now?
09:55:44 [ruaok]
blocking/throttle
09:58:39 [djce]
python-musicbrainz/0.7.3 of course, and it gets a special 403 which links to http://blog.musicbrainz.org/?p=1111
09:58:56 [ruaok]
blocked to throttled?
09:59:19 [djce]
and also the blank user-agent, and ^(Java|Python-urllib|Jakarta Commons-HttpClient)/[0-9._]+$ , which block with a link to http://blog.musicbrainz.org/?p=839
09:59:23 [djce]
just plain blocked.
09:59:30 [djce]
Every url other than "/" gives a 403.
09:59:31 [ruaok]
we can't really block it.
09:59:48 [ruaok]
lets just throttle it.
10:00:02 [djce]
* djce sobs quietly
10:00:11 [ruaok]
awww, now now.
10:00:20 [djce]
that's gonna need a musicbrainz-server release of course.
10:00:43 [ruaok]
argh.
10:00:50 [djce]
also (gets more professional hat on) we don't have a ticket for this,
10:00:59 [djce]
which is why we've confused our requirements.
10:01:08 [djce]
So mr ruaok, two things for you.
10:01:12 [djce]
#1 please create a ticket
10:01:22 [djce]
and #2 do you need me to revert the block right now?
10:02:00 [ruaok]
* ruaok sobs quietly
10:02:31 [ruaok]
do you have an email at hand that describes the changes to mb-server that are needed?
10:03:10 [djce]
maybe. It's kind of the one that I outlined a month or two ago about completely reworking the way we do blocking.
10:03:48 [djce]
closest ticket I can find is http://tickets.musicbrainz.org/browse/MBS-673 btw
10:04:17 [djce]
although MBS ticket, implemented as MBH ... <meh>. Hence the confusion.
10:04:47 [ruaok]
go ahead and move it.
10:05:21 [djce]
done. http://tickets.musicbrainz.org/browse/MBH-191
10:05:24 [ruaok]
in looking for emails I found: Dave: Please do your part and then create an MBS ticket for oliver on what he needs to do for musicbrainz-server."
10:05:32 [ruaok]
loi
10:06:20 [ruaok]
Thought #2: so that would make each WS request check three rate limits:
10:06:20 [ruaok]
check_ratelimit("ws ip=$ip")
10:06:20 [ruaok]
check_ratelimit("ws ua=$user_agent")
10:06:20 [djce]
Well let's start with collecting requirements.
10:06:20 [ruaok]
check_ratelimit("ws global")
10:06:27 [djce]
then work out the implementation later.
10:06:39 [djce]
I need to head out in the next 5 min.
10:06:39 [ruaok]
* ruaok should really be working on shut-eye
10:06:47 [djce]
Do you need me to revert the block?
10:07:04 [ruaok]
I may regret this decision.
10:07:10 [ruaok]
No. lets keep it in place.
10:07:19 [ruaok]
it will do a few things:
10:07:22 [ruaok]
1. piss people off
10:07:28 [djce]
ok. I'll be back within a couple of hours
10:07:38 [ruaok]
2. motivate to get moving on fixing things in the right way
10:07:43 [djce]
nod
10:07:47 [ruaok]
3. get people to finally contact us.
10:08:05 [ruaok]
can you please collect the requirements and make a high prioroty ticket for ocharles?
10:08:22 [ruaok]
and then we go to throttling as soon as this ticket is done.
10:08:43 [djce]
Clearly I don't know the requirements, unless it's just "like we have right now, but throttling".
10:08:44 [djce]
is that it?
10:08:58 [ruaok]
yes.
10:09:02 [djce]
ok
10:09:07 [ruaok]
and I think the cruz lies here: check_ratelimit("ws ua=$user_agent")
10:09:25 [ruaok]
that needs to be added to mb-server so we can add ua based throttling, right?
10:09:35 [ruaok]
since we have up and global already////
10:09:40 [ruaok]
...
10:09:52 [ruaok]
*ip
10:09:56 [ruaok]
* ruaok needs sleep
10:10:07 [djce]
I want to change mb-server so that it just sends one ratelimit request per http request
10:10:19 [djce]
but I'll write that up in an hour or two
10:10:27 [ruaok]
with three args to one call?
10:10:29 [ruaok]
sounds great.
10:10:35 [djce]
ip, ua
10:10:42 [djce]
and probably url
10:10:43 [ruaok]
not global?
10:10:52 [djce]
do that in ratelimit-server
10:11:00 [ruaok]
ok, sounds good.
10:11:08 [djce]
* djce goes grocery shopping
10:11:13 [ruaok]
please cc me on the write-up or send me a note on where I can find.
10:11:15 [ruaok]
ta
10:11:16 [ruaok]
nn!
10:11:20 [djce]
sure
10:11:22 [djce]
night!
10:11:23 [ruaok]
thx
11:05:19 [nikki]
ruaok: nobody ever posted on the blog to say that pymb2 now supports setting a user agent
11:29:53 [nikki]
hmm...
11:30:14 [nikki]
it seems you can't import the data dumps using master now
11:37:15 [warp]
did we break it?
11:37:54 [nikki]
the ref_count column has been removed from tag and url, but the data dumps apparently still have those columns
11:38:19 [nikki]
http://forums.musicbrainz.org/viewtopic.php?id=3281
11:39:34 [warp]
ah
11:39:45 [warp]
odd.
11:40:05 [warp]
I would have expected us to not make such changes until the schema change release.
11:40:11 [nikki]
yeah, same
11:40:30 [nikki]
because there's no way to find out which tags will work with the current dumps
11:41:19 [warp]
sure there is, try them each in turn :)
11:41:24 [nikki]
well, technically you can by reading the sql files
11:41:31 [nikki]
you know what I mean though :P
11:41:44 [nikki]
it shouldn't need trial and error, basically
11:41:53 [warp]
* warp nods.
12:34:24 [luks]
isn't the production branch what people should be using?
12:34:41 [nikki]
* nikki has no idea
12:34:57 [nikki]
I understood how it worked with svn :P
12:35:19 [luks]
where you had to be pick the right branch :)
12:36:00 [nikki]
you just copied the version number from the bottom of the website
12:36:31 [luks]
hm, the data dumps used to have a link to the branch as well
12:36:35 [luks]
I wonder if they still point somewhere
12:37:16 [warp]
* warp doesn't like the distinction between production and master.
12:37:40 [nikki]
* nikki kicks all the 502s on beta
13:36:39 [Mineo]
Mineo has joined #musicbrainz-devel
14:04:33 [alastairp]
alastairp has joined #musicbrainz-devel
14:25:07 [luks]
any opinions on http://dl.dropbox.com/u/5215054/tmp/mb_filter.png ? (UI-wise)
14:28:25 [nikki]
* nikki is not sure what the icon next to filter is
14:31:18 [luks]
it's a funnel, often used as an icon for filtering :)
14:31:23 [luks]
but I see that I should remove it :)
14:31:56 [Leftmost]
Leftmost has joined #musicbrainz-devel
14:32:08 [nikki]
oh
14:32:37 [luks]
at first I wanted to have just the icon, but a little larger
14:33:16 [nikki]
I guess I haven't seen enough filtering things, I can't even think of any I could look at
14:34:28 [xdnny]
xdnny has joined #musicbrainz-devel
14:40:16 [nikki]
luks: oh, and I assume "name" refers to the the recording name, not the artist name?
14:41:08 [luks]
yes
14:41:34 [luks]
plus there would be more options for releases
14:42:13 [nikki]
ah
15:28:18 [hrglgrmpf]
hrglgrmpf has joined #musicbrainz-devel
16:38:06 [the_metalgamer]
the_metalgamer has joined #musicbrainz-devel
16:38:25 [kepstin-laptop]
kepstin-laptop has joined #musicbrainz-devel
16:44:56 [reosarevok]
reosarevok has joined #musicbrainz-devel
17:21:20 [the_metalgamer]
the_metalgamer has joined #musicbrainz-devel
17:31:39 [the_metalgamer]
the_metalgamer has joined #musicbrainz-devel
17:52:24 [CallerNo6]
CallerNo6 has joined #musicbrainz-devel
18:40:48 [kepstin]
kepstin has joined #musicbrainz-devel
19:40:07 [ruaok]
ruaok has joined #musicbrainz-devel
20:02:27 [hawke_]
hawke_ has joined #musicbrainz-devel
20:10:37 [hawke_]
hawke_ has joined #musicbrainz-devel
20:31:32 [djce]
djce has joined #musicbrainz-devel
20:32:21 [hrglgrmp1]
hrglgrmp1 has joined #musicbrainz-devel
21:12:02 [djce]
djce has joined #musicbrainz-devel
22:05:37 [hawke_]
hawke_ has joined #musicbrainz-devel
22:24:54 [hawke_]
hawke_ has joined #musicbrainz-devel