The Wayback Machine - http://web.archive.org/web/20150915090927/http://chatlogs.musicbrainz.org/musicbrainz-devel/2011/2011-01/2011-01-10.html

IRC log of musicbrainz-devel on 2011-01-10

Timestamps are in UTC.

05:15:48 [dinog]
dinog has joined #musicbrainz-devel
05:56:26 [dinog1]
dinog1 has joined #musicbrainz-devel
06:10:35 [dinog]
dinog has joined #musicbrainz-devel
08:07:54 [ruaok]
ruaok has joined #musicbrainz-devel
08:09:37 [djce]
djce has joined #musicbrainz-devel
08:11:35 [luks]
luks has joined #musicbrainz-devel
08:23:55 [djce]
djce has joined #musicbrainz-devel
13:45:20 [kurtjx]
kurtjx has joined #musicbrainz-devel
15:21:05 [ocharles]
anyone around to help me work on http://jira.musicbrainz.org/browse/MBS-840 ?
15:34:39 [luks]
I can take that one
15:35:26 [luks]
or did you just want to ask some quesiton?
15:37:27 [luks]
oh, you already implemented it
15:37:54 [luks]
what do you want to help with?
15:45:19 [ocharles]
heh
15:45:25 [ocharles]
well, I dunno if I've done it the right way
15:45:42 [ocharles]
server is restarted with my implementation now
15:46:34 [ocharles]
This works for now, but we have that credits stuff (performed guitar, tracks 1-5) to bring back in soon, so I'm not sure how it will look with that
15:46:41 [ocharles]
but maybe I should worry about that when I start working on that again
16:08:55 [hawke_]
hawke_ has joined #musicbrainz-devel
17:15:25 [ruaok]
ruaok has joined #musicbrainz-devel
18:12:08 [pronik]
pronik has joined #musicbrainz-devel
18:33:37 [ruaok]
ruaok has joined #musicbrainz-devel
18:35:30 [warp]
hello internet
18:39:38 [ruaok]
hi warp.
18:39:55 [ruaok]
djce: new backup disks arrive tomorrow.
18:40:00 [djce]
cool
18:56:24 [murdos_]
murdos_ has joined #musicbrainz-devel
19:09:33 [ruaok]
LOL http://www.youtube.com/watch?v=3HCiCRJ8kVg
19:10:48 [alastairp]
hi all
19:11:18 [ruaok]
hi alastairp
19:24:26 [ruaok]
djce: have you looked at hobbes at all?
19:24:54 [djce]
not really, it's on the top of my list now though
19:25:01 [ruaok]
great.
19:25:02 [djce]
I just added it to jira, as I guess you saw
19:25:11 [ruaok]
I popped more ram into the box. now up to 8gn.
19:25:14 [ruaok]
gb
19:25:41 [djce]
What do you reckon to the syswiki -> git thing?
19:26:47 [ruaok]
hmmm. one sec.
19:28:20 [ruaok]
feh. I seem to have magically lost a number of emails. and I couldn't put my finger on which ones.
19:28:46 [djce]
I can resend mine...
19:28:50 [ruaok]
it appears that your volley of 3 messages was among them. could you do me afavor and resend those?
19:28:52 [ruaok]
thx
19:29:11 [pronik]
ruaok: "Dieses Video enthält Content von WMG und EMI. Einer oder mehrere dieser Partner haben das Video in deinem Land aus urheberrechtlichen Gründen gesperrt."
19:29:15 [pronik]
Welcome to Germany :(
19:29:26 [ruaok]
welcher video?
19:29:44 [ruaok]
der von oben?
19:29:47 [pronik]
genau
19:29:56 [ruaok]
war nicht sehr wichting.
19:30:01 [pronik]
immerhin :)
19:30:04 [ruaok]
cee low fuck you mit sign language.
19:30:24 [pronik]
EMI, WMG with fuck you, klingt logisch :D
19:31:19 [ruaok]
lol
19:31:24 [ruaok]
djce: "If it's acceptable that all our git administrators (i.e. mb core devs) can
19:31:25 [ruaok]
either access, or grant themselves access to, the syswiki.git repo, then stick
19:31:25 [ruaok]
it on git.musicbrainz.org. If not... github?"
19:31:46 [MBChatLogger]
http://git.musicbrainz.org
19:31:46 [ruaok]
I think git.mb.org is fine. after all ocharles and warp should pretty much have access to that.
19:31:53 [ruaok]
so that approach sounds fine for me.
19:31:54 [djce]
ok will do
19:32:11 [ruaok]
thx
19:32:20 [ruaok]
ocharles: the migration worked without a hitch!
19:32:22 [djce]
syswiki currently has about three passwords in it. I'll leave it as exercise to find them :-)
19:32:22 [ruaok]
:-)
19:40:45 [ruaok]
djce: on hobbes, one of our biggest questions right now is whether or not to install nginx.
19:41:00 [ruaok]
should we use the lenny/carl gninx or should we install one on hobbes?
19:41:09 [djce]
Or both.
19:41:29 [djce]
Presumably hobbes is inaccessible so far from the World?
19:41:29 [ruaok]
* ruaok nods
19:41:34 [ruaok]
correct
19:41:52 [djce]
What dns name is it to be exposed as?
19:42:04 [djce]
i.e. http vhost
19:42:20 [djce]
hobbes.musicbrainz.org?
19:43:18 [MBChatLogger]
http://test.musicbrainz.org
19:43:18 [ruaok]
well, its taking over for lingling, so ultiately it should be test.mb.org
19:43:37 [ruaok]
but until we can complete the switchover, lets go with test2 or newtest or something like htat.
19:43:42 [djce]
ok
19:43:48 [ruaok]
then when hobbes is really ready, then we switch dns.
19:44:00 [ruaok]
hobbes will be on the agenda for today again.
19:44:11 [djce]
Let me go do that on carl/lenny right now.
19:48:22 [djce]
HTTP is open for business. curl -H "Host: test2.musicbrainz.org" http://lb1.musicbrainz.org/foo
19:48:28 [ocharles]
ruaok: at last! :)
19:48:35 [djce]
I'll submit a DNS change request
19:48:43 [ruaok]
ocharles: yes. :)
19:48:49 [ruaok]
djce: thanks
19:52:29 [djce]
FAO ocharles , ruaok and anyone else hacking on mb_server to go onto hobbes: the user's IP address is in the header "X-MB-Remote-Addr", e.g. "X-MB-Remote-Addr: 91.84.196.2"
19:52:45 [djce]
just in case you do anything that needs that
19:53:03 [ruaok]
I dont think we need that
19:53:06 [ruaok]
* ruaok makes a note
19:53:11 [djce]
You will one day.
19:53:26 [ruaok]
yes, my wizard. :)
19:53:30 [alastairp]
cool. new version of google goggles wil solve sudoku :)
19:53:53 [ruaok]
aw. encroacring on the space of our of our own!
19:54:06 [djce]
Well, currently the live mb_server uses it for various things, including rate limiting, and sending out in "please verify your email address" emails (anti-abuse)
19:54:07 [ocharles]
djce: hrm, why the non-standard header?
19:54:11 [outsidecontext]
outsidecontext has joined #musicbrainz-devel
19:54:20 [ocharles]
I thought there was already a forwarded-for header, or something?
19:54:21 [djce]
As opposed to x-forwarded-for?
19:54:24 [ocharles]
right
19:54:32 [djce]
Anything "x-" is non-standard.
19:54:47 [ocharles]
yea, but there are catalyst modules that use x-forwarded-for I believe
19:55:04 [djce]
But the real answer is that it makes processing simpler on nginx.
19:55:20 [djce]
Throw away the incoming x-mb-r-a header. Add a new one. Send.
19:55:53 [djce]
Whereas with x-f-f you get into issues of "which IP networks do I trust?"
19:56:19 [ocharles]
* ocharles isn't enough of a sysadmin to grok
19:56:22 [ocharles]
i'll trust your judgement :)
19:56:56 [djce]
If it's a total PITA to have the mb_server use any header other than x-f-f, we can work on that.
19:57:09 [ocharles]
probably best to wait and see if it's a real problem
19:59:11 [ruaok]
* ruaok picks up the gavel
20:00:08 [ruaok]
<BANG>
20:00:08 [MBChatLogger]
MBChatLogger has changed the topic to: http://musicbrainz.org/#devel
20:00:15 [ruaok]
welcome everyone!
20:00:42 [ruaok]
ruaok has changed the topic to: agenda: review, hobbes, RC1, [what else?]
20:00:54 [ruaok]
who wants to start? ocharles, warp?
20:00:57 [ijabz]
ijabz has joined #musicbrainz-devel
20:01:09 [ocharles]
i can go ahead
20:01:17 [ocharles]
just getting the link now
20:01:23 [warp]
hello!
20:01:24 [ruaok]
k
20:01:31 [ocharles]
http://jira.musicbrainz.org/browse/MBS/fixforversion/10034#selectedTab=com.atlassian.jira.plugin.system.project%3Aversion-issues-panel so, here's the status on the current iteration
20:01:35 [ocharles]
I think we did pretty well!
20:01:59 [ocharles]
I slacked on getting that email out about batch editing (until today) so that didn't get fixed, and I didn't get round to do doing the edit migration stuff
20:02:07 [ocharles]
I think the estimate on that is a little off now that I think about it too
20:02:26 [ocharles]
but of what I did get done, it's all in review, a few have shipits, and a few have some feedback for minor adjustment
20:02:27 [ruaok]
warp has 4 days left on the current iteration...
20:02:44 [ocharles]
warp doesn't track time like I do
20:02:53 [ocharles]
so... i'll hand over to him on that :)
20:03:03 [ruaok]
* ruaok continues listening
20:03:30 [warp]
with my intermittent internet access at the office I'm not as good at logging time into jira
20:03:52 [ruaok]
ah.
20:04:15 [warp]
all the issues listed on the current iteration or on code review. assuming I get ship it's, the remaining time on those is just the few minutes it takes to merge and ship them.
20:04:21 [warp]
s/or/are/
20:04:27 [ruaok]
ah! very good! :)
20:04:41 [ocharles]
nice
20:04:46 [ocharles]
on the remaining issues for rc1
20:04:57 [ocharles]
I have just over a week there. so maybe rc1 will launch with me nearly completing my share :)
20:05:30 [ruaok]
I mostly care about getting all the new features in for RC1.
20:05:38 [ruaok]
so we can qualify as a real RC1.
20:05:45 [ocharles]
* ocharles nods
20:05:59 [ruaok]
do either of you have any potential problems for reaching that goal in the next week?
20:06:21 [ocharles]
only that this week has some annoying things mid working day on weds and thurs (driving)
20:06:43 [ruaok]
but you can make up for it in the evenings?
20:07:01 [ocharles]
hopefully
20:07:06 [ruaok]
k.
20:07:06 [ocharles]
no real reason why not
20:07:11 [ruaok]
anything else from you warp?
20:07:49 [warp]
I finished lazy loading of the release editor recording page at the start of last week, that's in review as I mentioned
20:08:00 [warp]
the rest of the week I've spent tackling all the UX tickets
20:08:31 [djce]
hobbes ssh is open for business. zaphod.musicbrainz.org port 10018.
20:08:45 [warp]
I'm comitting all that to a single 'ux' branch right now, and hope to put that entire thing on code review at some point this week. there's stiff a few tickets left I want to include in that.
20:08:50 [ruaok]
djce: thanks
20:08:54 [djce]
* djce apologises for interrupting. twice now.
20:09:03 [warp]
s/stiff/still/
20:09:05 [ruaok]
djce: no worries. :)
20:09:18 [ocharles]
warp: ah, I saw that branch, wondered what it was
20:09:19 [djce]
biab.
20:09:24 [ocharles]
has it been merged into next?
20:09:27 [ruaok]
warp: ok, looking forward to the imroved RE.
20:10:15 [ruaok]
my week: I've spent a fair amount of time dealing year end paperwork.
20:10:21 [warp]
ocharles: no, I'm still working on it.
20:10:29 [ruaok]
but I got some work on statistic done, which was nice. I'll continue on that later today.
20:10:40 [warp]
ocharles: I'm mainly pushing it to have a backup in case of hardware failure.
20:10:49 [ocharles]
warp: gotcha
20:10:52 [ruaok]
I still need to schedule the AR meeting where we decide some AR issues. hoping to schedule that today.
20:10:54 [warp]
ocharles: ofcourse you're welcome to take a look if you're interested :)
20:11:24 [ruaok]
I got my friend to fix the DIMM that I damaged and I installed it into Hobbes.
20:11:41 [warp]
warp has changed the topic to: agenda: review, hobbes, RC1, MBS-1119, [what else?]
20:11:50 [MBChatLogger]
http://zaphod.musicbrainz.org
20:11:50 [ruaok]
Hobbes is now at 8GB and available on port 10018 on zaphod.mb.org as djce pointed out.
20:12:24 [ruaok]
I've got a pretty packed schedule this week, but RC1 is my top priority.
20:12:58 [ruaok]
has anyone tried the VM I uploaded?
20:13:22 [ruaok]
the VM is a pretty big step for ensuring that we can roll out NGS onto our servers.
20:13:33 [ruaok]
ok, onward. murdos, you here?
20:13:39 [ocharles]
oh, that has downloaded, I will try it later tonight
20:13:48 [ocharles]
if I can get the mac keyboard working again
20:14:01 [ruaok]
kewl.
20:14:16 [ruaok]
I intend to put out an updated one that can be used to sync off test after RC1.
20:14:53 [ruaok]
anyone else for review?
20:14:58 [ruaok]
ijabz? nikki?
20:15:05 [ijabz]
I was updating db to Postgres 9 and messed it up, and then non-mb work mean I havent gone back to it, but I shall try again when I can summon nrg
20:15:14 [ruaok]
ijabz: ok.
20:15:22 [djce]
djce has joined #musicbrainz-devel
20:15:23 [ijabz]
then Ill try and speed up recording index using some temp tables
20:15:27 [ruaok]
I'm still trying to find time to run those index builds as you requested.
20:15:32 [ruaok]
ijabz: awesome.
20:15:43 [ruaok]
right-o. moving on then.
20:15:46 [ruaok]
hobbes!
20:15:53 [ijabz]
that would be appreciated
20:16:09 [ruaok]
ocharles: besides nginx, what else do we have to do with hobbes to get it ready to use it for a new test?
20:16:11 [warp]
(about rc1, looks like my "improvement" tickets scheduled for rc1 are mostly the UX things I'm already working on.. so I may also get all those fixed by next week)
20:16:12 [murdos_]
ruaok: yep?
20:16:16 [ruaok]
I would like to see us move to hobbes for RC1.
20:16:22 [ocharles]
ruaok: I don't think we need to do anything
20:16:22 [ruaok]
warp: +1
20:16:39 [ijabz]
hobbies is a faster test server , right ?
20:16:42 [ruaok]
ocharles: so we have an instance of fastcgi/apache and all that running?
20:17:00 [ruaok]
ijabz: more power friendly, hosting at our facility.
20:17:07 [ruaok]
its about the same speed as lingling.
20:17:17 [ruaok]
ocharles: ok, great.
20:17:20 [ocharles]
ruaok: no
20:17:30 [ruaok]
I will deal with loading new NGS data on hobbes on sunday.
20:17:46 [ruaok]
ocharles: then I guess there *is* stuff to do for that.
20:18:04 [ruaok]
shall we not worry about that now and simply deal with it on monday?
20:18:12 [ocharles]
We need a fastcgi restarter script (but I think we should be able to copy the one on test)
20:18:14 [ruaok]
I think we're close to being ready to go on hobbes -- all the big things are done.
20:18:19 [ocharles]
and we need nginx installed
20:18:32 [djce]
Is it worth thinking soon about how ngs is to be deployed to live? Specifically, do we need to take notes on how Hobbes is being set up, so we know how to set up the other servers that will presumably follow?
20:18:37 [ruaok]
yes, djce will be decided what and how to deal with nginx.
20:18:43 [djce]
or is it so easy that it's not worth noting?
20:18:55 [ruaok]
djce: that is a very valid thing for us to consider.
20:19:13 [ruaok]
but, lets discuss this after the meeting. its a bit detailed right now.
20:19:17 [djce]
sure
20:19:25 [ruaok]
I just wanted to see what is remaining on that front.
20:19:34 [ruaok]
onward. RC1.
20:19:51 [ruaok]
We're prepping hobbes to be ready. I'm loading new data.
20:19:59 [ruaok]
what else do we need in order to be ready for RC1
20:20:00 [ruaok]
?
20:20:40 [ruaok]
djce: it would be nice if hobbes can send email, such as subscription emails and collection emails.
20:20:45 [djce]
From a sysadmin point of view, what is RC1? All servers continue to run the existing live MB, except hobbes?
20:21:11 [ruaok]
correct. only hobbes is affected by RC1. all self contained.
20:21:12 [ruaok]
oh!
20:21:14 [djce]
ruaok: email, yes, will take care of that soon
20:21:24 [ruaok]
the search stuff needs to be installed.
20:21:28 [ruaok]
djce: thanks.
20:21:46 [ruaok]
murdos: can I interest you in looking at setting up search on hobbes?
20:21:53 [djce]
I have another point to make re. the ngs codebase:
20:22:13 [warp]
how different is a production setup of search compared to a setup used by developers?
20:22:13 [djce]
A bunch of commits have been made to svn mb_server since that branch was created,
20:22:22 [djce]
i.e. since the release back in 2009(?)
20:22:30 [ruaok]
warp: not different from what I know.
20:22:41 [djce]
How do we ensure that none of those changes are lost? Or, that we only lose what we /mean/ to lose?
20:23:09 [djce]
Or is it way too early to think about that?
20:23:12 [ruaok]
well since that codebase is being tossed after we release NGS, its of limited importance.
20:23:16 [murdos_]
ruaok: if you've no other choice, sure. but I'd rather spent my free time this week to code the few features I want to add before RC1
20:23:30 [ruaok]
as long as we can go back and NOT go to NGS, then we should be set.
20:23:37 [ruaok]
murdos_: fair nuff. that's the right idea. I'll do it.
20:23:40 [ollie_]
ollie_ has joined #musicbrainz-devel
20:23:48 [jdamcd]
jdamcd has joined #musicbrainz-devel
20:23:54 [ollie_]
urgh, lost irc
20:24:04 [djce]
ruaok: I don't entirely agree. Some of the changes made to live mb_server are made for damn good reason,
20:24:04 [murdos_]
ruaok: ok, thank :)
20:24:07 [djce]
and need to be kept.
20:24:09 [ruaok]
djce: in general you and I need to start a whole series of releasing NGS.
20:24:29 [ruaok]
djce: ok, lets put that up as a point to discuss in our NGS release discussions.
20:24:53 [ruaok]
but there are more than a few things to consider how to get NGS out. its going to be quite the production.
20:24:58 [ollie_]
djce: it's more that we need to make sure we have tickets for those commits open for the ngs codebase
20:25:01 [ollie_]
rather than the code itself
20:25:10 [ollie_]
i'm sure more often than not it will be a copy/paste type fix
20:25:16 [ruaok]
murdos_: how is the improved search stuff doing?
20:25:24 [ruaok]
is that something that we want to push out for NGS or after?
20:25:30 [djce]
ollie_: Sure. We just need to have a plan, and follow it.
20:25:59 [ruaok]
we need more than just ONE plan. a dozen plans. :)
20:26:16 [ruaok]
anymore pressing questions for hobbes or RC1?
20:26:17 [murdos_]
ruaok: it still depends on the speed indexing improvement that ijabz should work on
20:26:30 [ruaok]
interesting.
20:26:47 [ruaok]
if ijabz doesn't succeed you think we can roll out the instant update?
20:27:01 [ruaok]
I trust that we should minimally roll out the fatter replication packets, right?
20:27:51 [murdos_]
we can keep instant update for most entities expected recording and release
20:28:13 [ruaok]
except?
20:28:38 [murdos_]
yes, except
20:28:48 [ruaok]
why not those?
20:29:14 [murdos_]
those are very slow in index, the other are fine
20:29:21 [murdos_]
/in/to
20:29:41 [ruaok]
ah. so, the work that ijabz is doing will really factor into that.
20:29:57 [ruaok]
OK, I need to move the creation of search indexes up on my list of things to do.
20:30:00 [ruaok]
ok, thanks murdos_
20:30:01 [ijabz]
murdos, are we talking about your replication packet stuff
20:30:09 [murdos_]
ijabz: yes
20:30:15 [ijabz]
is that for updating slaves or the server as well
20:30:43 [murdos_]
ijabz: it's mainly for updating the master search server, but can be used to update slave search servers too
20:30:55 [ruaok]
<3
20:31:05 [ruaok]
thats going to be sooo good to roll out. ;)
20:31:13 [ijabz]
so dont yor updates work well for all indexes
20:31:59 [murdos_]
ijabz: what do you mean?
20:32:00 [ruaok]
* ruaok nudges warp to get ready for MBS-1119
20:32:35 [ijabz]
i,e does it matter that much if the main build wjole index takes a while if it s only going to be run rarely
20:33:01 [ijabz]
because once built , your code can update the new edits
20:33:16 [ruaok]
I get the impression that your work on speeding up the build will also greatly impact the speed of updating existing indexes.
20:33:29 [warp]
* warp was nudged.
20:33:36 [ruaok]
because afterall, the slow part is to query the DB, which still has to happen.
20:33:37 [murdos_]
ijabz: the main issue is that updating a few recording or releases is too slow to keep up with real time edits
20:33:50 [ruaok]
:-(
20:33:56 [warp]
* warp nudges ocharles to also be ready for MBS-1119. ;)
20:34:12 [ruaok]
ok, lets leave it for now. I'll get back to ijabz with search index built time data.
20:34:24 [ruaok]
warp, ocharles: MBS-1119
20:34:28 [warp]
right
20:34:32 [ruaok]
ruaok has changed the topic to: agenda: MBS-1119
20:34:36 [ollie_]
ah, this one
20:35:03 [warp]
MBS-1119 is that a user who changes the release artist of a release will expect the track artists to change as well.
20:35:09 [ollie_]
* ollie_ nods
20:35:43 [warp]
but the release editor may not load all mediums, and as such will have to deal with that server side it some fashion
20:36:32 [warp]
I was wondering if we should make 'change release artist' a seperate edit which just does the Right Thing itself. or if the release editor should create all the tracklist edits at the end of the wizard.
20:36:46 [warp]
(I'm leaning toward the first option)
20:36:54 [ollie_]
yea, I thought we were already doing that
20:37:09 [warp]
afaik we don't have anything yet for changing the release artis.t
20:37:19 [ollie_]
no, but we discussed it and I got that ticket :)
20:37:26 [ruaok]
* ruaok looks for an artis template
20:37:27 [ollie_]
(the subtask)
20:37:32 [warp]
oh!
20:37:56 [warp]
doh, you're right.
20:38:00 [ruaok]
lol
20:38:05 [ollie_]
so i'm blocking you i take it?
20:38:14 [warp]
I was looking over the list of tickets for rc1, but didn't realize we already discussed this.
20:38:19 [ollie_]
hehe
20:38:52 [ruaok]
* ruaok wonders if this topic is don now
20:38:57 [warp]
well, it's classified in jira as a bug. so I wasn't planning on solving this.
20:39:03 [warp]
(according to jira)
20:39:10 [warp]
but it sounds like something we should've had for rc1.
20:39:46 [warp]
ollie_: I'll spend most of this week doing UX stuff, so not blocking as such.
20:40:00 [ollie_]
the only problem I see with this is what if I change the release artist and one artist somewhere else
20:40:01 [ruaok]
well, if its assigned to ollie...
20:40:04 [ollie_]
then what happens?
20:40:31 [warp]
ruaok: the edit type is assigned to ocharles, the changes to the release editor are still mine.
20:40:40 [ruaok]
ah
20:40:58 [ollie_]
Ie, the release is by AC1 and all tracks are by AC1. I change the release artist to AC2, and one track to AC1 & AC2. what should happen?
20:41:14 [warp]
ollie_: I would expect a release artist edit on a non-va release to change all the artists credits which are (were) identical to the release artist.
20:41:16 [ollie_]
do all tracks change to AC2 and then we apply the single change on top of that, or do we keep them as AC1?
20:42:31 [warp]
ollie_: preferably the order in which those two edits are applied would not have any effect on the end result.
20:42:54 [ollie_]
Well editing a tracklist changes all tracks
20:42:59 [ollie_]
so it does matter a bit
20:43:04 [warp]
it shouldn't.
20:43:05 [warp]
:)
20:43:12 [ollie_]
oh, I see what you're saying
20:43:22 [ollie_]
ok, I think I can hack something up
20:43:39 [ruaok]
k, then lets wrap up.
20:43:44 [ruaok]
warp, post the link please?
20:43:49 [warp]
yes, will do
20:43:54 [ruaok]
unless anyone has anything else?
20:44:14 [ruaok]
thanks for your time everyone! I'm really looking forward to RC1. Its going to feel very much closer to real. :)
20:44:16 [ruaok]
</BANG>
20:44:16 [MBChatLogger]
MBChatLogger has changed the topic to: http://musicbrainz.org/#devel
20:44:22 [warp]
thank you ruaok :)
20:44:33 [ruaok]
ocharles, warp: shall we chat about the upcoming week or are things clear?
20:44:45 [ollie_]
my plan is more new features etc, and then critical bugs
20:44:59 [warp]
things are fairly clear for me. try to get all the "improvement" tickets done. that's about 1 week of work.
20:45:01 [ruaok]
good. let me know if you need more guidance picking bugs.
20:45:09 [ruaok]
great,
20:45:26 [ruaok]
its going to be a long week for me, but I'm so excited to be back to 110%. :)
20:45:38 [ruaok]
now I just need to stay out of trouble. :)
20:45:39 [warp]
and I noticed ocharles threw a whole bunch of stuff on code review. So tomorrow is probably reviewing code.
20:45:52 [ruaok]
yeah, loads of reviews.
20:46:11 [warp]
(well, not the whole day.. perhaps after dinner when I'm at the better internet)
20:57:11 [ijabz]
ruaok:Two problems with doing the search indexing stuff at the moment
20:58:06 [ijabz]
1. We havent tried building the indexes on a machine equivalent to production so we dont have good idea on how long they will actually take
20:58:59 [ollie_]
ok, up to 4 days. I'll leave it there as monday should be double checking everything is screwed together correctly
20:59:27 [ijabz]
2. The code is designed to add records to the search index whilst waiting for results for next db query, this improvement only works if the database and search indexing are on different machines
20:59:36 [ruaok]
my dev box is reasonably close to a production box, so those numbers will help.
20:59:54 [ruaok]
interesting.
21:00:08 [ruaok]
I can run the indexer in a VM and have the DB live on my dev box.
21:00:16 [ruaok]
let me try that approach when I build my indexes.
21:00:33 [ijabz]
Thats good, so perhaps when hobbes is up you could try building the index on your machine takling to db on hobbes
21:00:55 [ruaok]
hmmm.
21:00:56 [ijabz]
I dont have much faith in VMs, dont they slow everything down greatly compared to a real machine
21:01:14 [ruaok]
there is 3 physical miles and 600 network miles between the two machines.
21:01:27 [ruaok]
they claim to run at 97% of the host machine.
21:01:42 [ruaok]
assuming the host is idle, which I intended to do.
21:01:52 [ruaok]
and the real speed issue comes from the DB, not the indexing.
21:02:33 [ruaok]
in other words, I think this test will be very apt and useful
21:02:47 [ijabz]
Well perhaps you could try both on your computer , no VM and indexer on VM and compare
21:03:03 [ruaok]
seems reasaonable.
21:04:42 [ijabz]
Also I dont think my improvements are going to help murdoses updater because they are simply using some additional temporary tables, and because these take a while to create I was only expecting to build them when rebuilding the whole index, not worth it for updater if only updating a few records.
21:05:18 [ruaok]
true/
21:05:21 [ruaok]
.
21:05:28 [ruaok]
another thing to consider in all of this.
21:05:32 [ijabz]
murdos mentioned a case where the updater is really slow if an artist name is chnaged because then all the release/recordings for that artist have to be uypdated
21:05:43 [ruaok]
yeah, that is worst case.
21:05:49 [ruaok]
but that doesn't happen all that often, fortunately.
21:06:02 [ruaok]
but we can't have a malicious JSB rename crash the site. :)
21:06:50 [ijabz]
In this case, i proposed that he bypasses the database and just querys th existing lucene indexes and modify them with new name
21:07:18 [ruaok]
* ruaok shudders
21:07:27 [ijabz]
More complex to code, but would be much quicker
21:07:36 [ruaok]
* ruaok nods
21:09:01 [ijabz]
Well its an idea, I dont think its particulary difficult
21:10:12 [ijabz]
then possibly, the Updater code will keep up with real edits because that would be so much better than rebuilding the whole index
21:10:26 [ruaok]
it bugs me since lucene isn't an ACID compliant DB.
21:12:10 [ijabz]
true
21:12:47 [ijabz]
Posibly make a copy of the indexes, modfy that, and then swap back in
21:13:25 [ijabz]
Has the index building ever created a corrupt index
21:13:40 [ruaok]
not to my knowedlge.
21:13:51 [ruaok]
but their lifetime is fairly short, so it might be hard to catch that.
21:14:49 [ijabz]
If it failed to write a record it would crash the index build so you would know about it
21:15:41 [ruaok]
well, I've never seen an error report from the index builder, so we can probably assume that its never happened.
21:17:45 [ijabz]
So if it means we can run updater on all indexes might be worth it ...
21:18:26 [ijabz]
... but then the updater might be okay anyway as ther eis a wild disparity between how long it takes me to build indexes and some of the times reported by others
21:19:07 [ruaok]
I'd love to hear what murdos_ has to say about this.
21:20:12 [ijabz]
Yeah murdos doesn't like it , because of duplicate business logic (but he can speak for himself)
21:20:25 [murdos_]
as I said privately to ijabz, I don't like that (editing directly the index without querying database)
21:20:41 [ruaok]
i have strong reservations about this.
21:21:11 [ruaok]
lets press ahead on the "correct" aproach and see if we can find the proper optimzation for it.
21:21:22 [murdos_]
without saying that it's way more complex that what current code is doing: use packet to detect what need to be rebuild, then reuse existing indexing code
21:21:58 [ruaok]
that makes me much more comfortable.
21:23:16 [ijabz]
I wonder if I should do the temporary table stuff, as that is also more complicated than the existing indexing code :;
21:24:22 [ruaok]
I think its worth investigating
21:25:45 [murdos_]
ruaok: so when can we expect the new annual report? :P
21:25:59 [ruaok]
not before rc1. :(
21:26:05 [ruaok]
and I leave for france on the 19th.
21:26:09 [ruaok]
feb?
21:26:59 [ruaok]
I might try and gather data and see if I can work on it on the plane.
21:27:26 [murdos_]
don't worry, last year you published it in March. it was just a gentle reminder :)
21:27:59 [ruaok]
its on my radar: see http://blog.musicbrainz.org/?p=705
21:28:54 [murdos_]
oh... I didn't see the last sentence... :/
21:29:12 [ruaok]
:-)
21:34:42 [murdos_]
my assumption is that I'm #2 editor again that year, the #1 being drsaunde
21:35:00 [ruaok]
oh, i see. :)
22:17:28 [ruaok]
djce: for the backup drives, you want to use the same MBBACKUP# label scheme?
22:17:37 [ruaok]
ext2? 3? 4?
22:17:42 [djce]
lvm
22:17:44 [ruaok]
not 4.
22:17:55 [ruaok]
probably unjournaled 2, I would think
22:18:18 [djce]
As for labels... not fussed, the labels only matter to you, if at all
22:18:46 [djce]
Name after cartoon robots if you wish :-)
22:18:52 [ruaok]
doesn't it look for specific labels at some point?
22:19:14 [djce]
Yes but that's just a three-line config file
22:19:26 [djce]
happy to rewrite that config to reflect the new drives.
22:19:43 [ruaok]
ok, lets pick a new scheme then to not confuse things.
22:20:18 [djce]
fred jim and sheila?
22:20:24 [djce]
* djce wonders if ruaok gets the reference
22:20:56 [ruaok]
I dont.
22:21:32 [ruaok]
I think I am going to skip changing disks today, if the other arrive tomorrow.
22:21:48 [djce]
Three of the core chips in old BBC micros, IIRC
22:22:23 [ruaok]
interesting. I never programmed on any of those, save for a minute or two at selfridges on oxford street.
22:22:42 [djce]
"their uppercase versions were official names given to the 3 memory areas that held I/O status registers on the lovingly-remembered BBC Microcomputer"
22:22:46 [djce]
http://www.answers.com/topic/fred-computer-jargon-1
22:22:49 [djce]
I
22:23:06 [djce]
I'm amazed that googling Fred Jim and Sheila isn't more informative.
22:23:49 [ruaok]
there is the Three Amigos.
22:25:20 [ruaok]
mickey, donald and goofy.
22:26:14 [djce]
As I said, only matters to you really. Just there to try to advise you which disk should go in next.
22:26:34 [ruaok]
ok.
22:26:54 [ruaok]
THC, LSD, Caffeine it is. :)
22:27:08 [djce]
:-D
22:27:26 [djce]
SEX, DRUGS, ROCKNROLL ?
22:27:43 [ruaok]
good one.
22:27:56 [ruaok]
I think that is the keeper.
22:30:04 [djce]
For the first few days I'll probably want to hand-hold the new backups / run them by hand.
22:30:18 [djce]
So no need to format the disks
22:31:07 [djce]
You reckon ext2 better then ext3 for this? I'd be lying if I said I understood the difference, or why it matters.
22:34:15 [ruaok]
* ruaok returns
22:34:54 [ruaok]
ext2 is not journaled.
22:35:05 [ruaok]
ext3 is. ext4 is a faster/better version of ext3
22:35:15 [ruaok]
since this is just for backup, I think ext2 will be fine.
22:35:39 [ruaok]
I've just printed labels for the drives. SEX (1), DRUGS (2), ROCKNROLL (3).
22:35:48 [ruaok]
do you want me to swap them in each day this week?
22:36:01 [ruaok]
tuesday is SEX, wed DRUGS and thu ROCKNROLL?
22:36:10 [ruaok]
* ruaok could use the extra excercise.
22:36:21 [djce]
If you like, sure
22:36:24 [ruaok]
k
22:37:18 [djce]
Can you mail me each time you've changed the drives, to remind me to go set it up?
22:37:26 [ruaok]
will do.
22:37:29 [djce]
thx
22:57:32 [nikki]
djce: what should happen with the tickets on http://bugs.musicbrainz.org/report/35 ?
22:58:51 [nikki]
bah
22:59:54 [djce]
djce has joined #musicbrainz-devel
23:00:58 [djce]
I guess they all need to become MBH jira tickets
23:02:04 [djce]
Some of them may be invalid now, not sure which though so not fussed whether the bad ones are ID'd before migration or after.
23:02:16 [djce]
Are you in a hurry to close down all Trac tickets?
23:04:35 [nikki]
no hurry, I've just been wondering but kept forgetting to ask
23:04:43 [nikki]
but I can just copy them all to jira if you want
23:06:15 [ollie_]
ollie_ has joined #musicbrainz-devel
23:06:44 [djce]
Yes please :-)
23:14:35 [nikki]
ok then
23:19:59 [djce]
test2.musicbrainz.org is in dns now
23:20:20 [ruaok]
nic
23:20:21 [ruaok]
e
23:23:41 [ollie_]
ollie_ has joined #musicbrainz-devel
23:46:47 [djce]
ruaok: So, you plan to put the first new backup disk in just after tomorrow's backup run?
23:47:19 [ruaok]
tomorrow between 4pm-5pm PDT, yes.
23:47:29 [ruaok]
and it will be SEX
23:47:32 [ruaok]
* ruaok snickers
23:47:53 [djce]
ok so one more backup run as normal then. cool.
23:48:10 [djce]
catch you tomorrow! And I'll try to get some hobbes stuff done soon....