IRC log of musicbrainz-devel on 2014-11-10

Timestamps are in UTC.

00:09:29 [dom96]
dom96 has joined #musicbrainz-devel
00:17:18 [Nyanko-sensei]
Nyanko-sensei has joined #musicbrainz-devel
01:09:06 [Freso]
Freso has joined #musicbrainz-devel
01:49:35 [d4rkie]
d4rkie has joined #musicbrainz-devel
02:37:57 [Gentlecat]
Gentlecat has joined #musicbrainz-devel
02:44:26 [CallerNo6]
CallerNo6 has joined #musicbrainz-devel
05:30:40 [coderhead42]
coderhead42 has joined #musicbrainz-devel
06:14:38 [ariscop]
ariscop has joined #musicbrainz-devel
06:33:19 [yeeeargh]
yeeeargh has joined #musicbrainz-devel
07:15:50 [zas]
zas has joined #musicbrainz-devel
08:08:13 [ohrstrom]
ohrstrom has joined #musicbrainz-devel
08:45:45 [ruaok]
ruaok has joined #musicbrainz-devel
09:15:29 [ijabz]
ijabz has joined #musicbrainz-devel
10:13:26 [coderhead42]
coderhead42 has joined #musicbrainz-devel
10:19:25 [ijabz]
ijabz has joined #musicbrainz-devel
10:28:24 [Powlow]
Powlow has joined #musicbrainz-devel
10:32:45 [demonimin]
demonimin has joined #musicbrainz-devel
10:39:13 [ruaok_]
ruaok_ has joined #musicbrainz-devel
10:46:30 [alastairp]
ruaok: you're going to hate me. apparently there's a cmake directive to fix the paths of frameworks/binaries
10:47:23 [ruaok_]
lol
10:47:29 [ruaok_]
no hate. less for us to do. ;)
10:48:45 [ruaok]
* ruaok leaves this for when we return to this topic https://gist.github.com/mayhem/4d56e1ba660cad989d21
11:04:52 [ijabz]
whatdays schema release happening ?
11:05:08 [ruaok]
in one week.
11:05:58 [ijabz]
right, ok I finished everything for search that Im aware of schema related
11:06:15 [ruaok]
:)
11:07:26 [ijabz]
Would be nice if it got tested with the frontend somehow though
11:10:20 [ijabz]
luks getting alot of timeouts on acoustid at the moment
11:12:41 [chirlu`]
ijabz: Could you enable two-letter and three-letter indexing for instruments (and perhaps areas), as it was done for artists in SEARCH-339?
11:12:42 [mb-chat-logger]
http://tickets.musicbrainz.org/browse/SEARCH-339
11:13:55 [ijabz]
chirlu yes that wouldnt be hard, but please raise (or clone and edit) issue
11:16:01 [chirlu`]
OK, I challenged yindesu to open the ticket because he complained about “sax” not finding “saxophone”, but apparently I have to do it myself. :-)
11:18:25 [ijabz]
Yes my stance is if users cant be bothered to open a ticket i cant be bothered to fix it
11:18:50 [yeeeargh]
yeeeargh has joined #musicbrainz-devel
11:35:14 [Nyanko-sensei]
Nyanko-sensei has joined #musicbrainz-devel
12:08:21 [ruaok]
alastairp: any suggestions for which qt5 libs from ubuntu to install?
12:08:49 [alastairp]
for gaia? I'd do qt4
12:08:54 [ruaok]
ok
12:09:09 [alastairp]
apt-get install build-essential libqt4-dev libyaml-dev swig python-dev pkg-config
12:09:15 [alastairp]
(from the gaia readme ;)
12:09:43 [ruaok]
:)
12:10:28 [ruaok]
if I had only scrolled down one line more. :)
12:23:28 [ruaok]
heh, interesting: https://github.com/hughrawlinson/meyda
12:37:49 [ijabz]
ijabz has joined #musicbrainz-devel
12:38:40 [ruaok]
alastairp: essentia built with gaia. what do I need to do to create the high level extractor?
12:46:17 [demonimin]
demonimin has joined #musicbrainz-devel
12:52:34 [alastairp]
ruaok: I think it's just the same streaming_extractor_music but with the other profile file
13:06:15 [kepstin-laptop]
kepstin-laptop has joined #musicbrainz-devel
13:16:54 [kepstin-laptop__]
kepstin-laptop__ has joined #musicbrainz-devel
13:19:08 [ruaok]
alastairp: hmmm https://gist.github.com/mayhem/da3d3106a0ead22f258a
13:19:16 [coderhead42]
coderhead42 has joined #musicbrainz-devel
13:23:39 [ijabz]
ijabz has joined #musicbrainz-devel
13:30:16 [alastairp]
cool
13:30:54 [ijabz]
ijabz has joined #musicbrainz-devel
13:31:22 [ruaok]
cool?
13:31:32 [alastairp]
well. you broke /something/ :)
13:31:36 [alastairp]
I'm not sure
13:31:44 [ruaok]
ok.
13:31:55 [alastairp]
oh, actually. I know what the error is. weird. I thought I fixed it
13:32:05 [ruaok]
oh.
13:32:17 [ruaok]
I don't think I did a git pull on essentia before I recompiled.
13:32:20 [ruaok]
lemme do that
13:32:21 [alastairp]
anyway, maybe you don't run streaming_extractor_music
13:32:29 [alastairp]
oh yeah, when was the last time you compiled it?
13:32:41 [ruaok]
just now, but I didn't git pull
13:32:57 [ruaok]
on audioloader-codec :)
13:32:58 [ruaok]
master right?
13:33:34 [ruaok]
* ruaok git pulls master and builds again
13:34:02 [alastairp]
I mean, the last time you pulled. yeah, master
13:34:24 [ruaok]
* ruaok listens to the new floyd album
13:34:31 [alastairp]
oh, it's out now?
13:34:33 [alastairp]
spotify?
13:34:38 [ruaok]
yeah
13:34:40 [alastairp]
I wish I was in your office
13:34:43 [ruaok]
came out a few days ago.
13:34:45 [ruaok]
heh.
13:34:52 [ruaok]
we should find a way to make that happen.
13:34:53 [alastairp]
I thought the single was out a while back
13:35:00 [ruaok]
* ruaok cranks it
13:35:01 [alastairp]
and the album is out today
13:35:43 [ruaok]
its got fabulous david gilmore guitar.
13:43:58 [kepstin-laptop__]
kepstin-laptop__ has joined #musicbrainz-devel
13:48:10 [ruaok]
woo. high level extractors are happening. :)
13:54:16 [alastairp]
ruaok: labs.abz.org isn't working. Did you create the zone file in gandi but then forget to activate it?
13:54:46 [ruaok]
I thought I did. lemme double check
13:56:03 [ruaok]
> WARNING: This version is in use. To edit this zone, you must first select a different version, or create a new one.
13:56:18 [ruaok]
oh,I think I see the problem
13:56:29 [ruaok]
labs.acousticbrainz.orgA193.145.50.110
13:57:05 [ruaok]
labsA193.145.50.110
13:57:15 [alastairp]
mmmm, trailing dots
14:02:24 [ruaok]
crap. david gilmore is signing.
14:03:36 [alastairp]
signing, even worse than singing!
14:04:07 [ruaok]
signing would be fine. we wouldn't hear it. :)
14:13:09 [ruaok]
alastairp: DNS is now correct on gandis servers.
14:13:21 [ruaok]
in 3 hours should be propagated.
14:13:26 [alastairp]
yeah, dig +trace shows it
14:13:28 [alastairp]
thanks
14:13:45 [ruaok]
dn
14:15:06 [ruaok]
ohhh. AND a new Röyksopp album.
14:22:10 [alastairp]
ruaok: what country is your spotify in? can you send me a spotiyf:// or http:// link to it?
14:22:20 [alastairp]
it looks like it's maybe not available in spain :(
14:22:21 [ruaok]
US
14:24:56 [yeeeargh]
if only there would be some smart people who would invent something like a worl wide web, where you can make digital music available to all humanity at the same time. i bet you could make a lot of money with that idea.
14:25:16 [yeeeargh]
*world
14:25:44 [alastairp]
* alastairp listens to taylor swift
14:25:51 [alastairp]
except I don;t, because you know, I can't
14:32:53 [alastairp]
https://twitter.com/jellea/status/531805778639089664
14:37:42 [ruaok]
heh
15:03:23 [reosarevok]
reosarevok has joined #musicbrainz-devel
15:22:24 [Freso]
yeeeargh: Nah. It wouldn
15:22:26 [Freso]
't take.
15:36:20 [ruaok]
feh. formatted json.
15:43:07 [alastairp]
as the output?
15:43:15 [alastairp]
that's easily fixed
15:43:23 [alastairp]
* alastairp reads code to see if it's easily fixed
15:44:11 [ruaok]
it is.
15:44:20 [ruaok]
I just need to parse the json and then output it again.
15:44:37 [alastairp]
put `indent: 0` in your profile file
15:45:11 [ruaok]
in which section?
15:45:29 [alastairp]
top-level
15:45:36 [alastairp]
so at the top would be fine
15:46:58 [ruaok]
thx
15:48:07 [CallerNo6]
CallerNo6 has joined #musicbrainz-devel
15:49:16 [ruaok]
well, the pink floyd album is no longer available on spotify.
15:50:13 [ruaok]
* ruaok plays his flacs instead
15:50:31 [alastairp]
!
15:50:57 [ruaok]
alastairp: so the mergeValues bit...
15:51:03 [alastairp]
yes
15:51:17 [alastairp]
so anything under mergeValues will get added to the output document
15:51:19 [ruaok]
in order to get the hash of the high level binary in there, I need to modify the profile file?
15:51:24 [alastairp]
yes
15:51:31 [ruaok]
grrr
15:51:39 [alastairp]
https://github.com/MTG/acousticbrainz-client/blob/master/abz/config.py#L54
15:52:14 [ruaok]
lame
15:53:32 [alastairp]
maybe the extractor needs to be able to hash itself
15:53:36 [alastairp]
it'd make it a little easier
15:53:49 [JesseW]
JesseW has joined #musicbrainz-devel
15:54:02 [ruaok]
yeah.
15:54:19 [ruaok]
but in case we do ever want to add anything, then we still have this problem.
15:54:28 [ruaok]
the things that bugs me is that the models need to be listed in the config file.
15:54:47 [ruaok]
so, if someone wants edit the list of models, the they need to change a file that must get generated.
15:55:01 [ruaok]
unnecessarily complex.
15:55:04 [alastairp]
right
15:55:29 [ruaok]
passing a JSON document to be merged would be optimal, mehinks.
15:55:40 [alastairp]
so would you prefer we compiled the models into the extractor?
15:55:45 [alastairp]
that's probably just as easy
15:56:04 [ruaok]
I'm not sure what the exact use case it.
15:56:06 [ruaok]
*is
15:56:11 [ruaok]
so, I dont know.
15:57:07 [alastairp]
it looks like each model creates a different section in the output, right?
15:57:40 [alastairp]
but we could just say that the extractor binary is specific to this project, so we hard-code in all the things that we want
15:57:47 [ruaok]
in the highlevel section, yes.
15:58:03 [alastairp]
yes, specifcally highlevel
15:58:43 [ruaok]
if that is acceptable to you, thats fine by me.
15:59:59 [ruaok]
in fact if the profile file could just go away, that would be grand.
16:00:06 [ruaok]
third arg is a JSON doc to merge.
16:00:22 [ruaok]
who does the hashing of the extractor, matters not.
16:00:34 [ruaok]
since I've got code for it, I'm happy to keep doing it.
16:00:34 [alastairp]
the profile file is the "essentia way" of doing this
16:00:50 [ruaok]
* ruaok shudders
16:06:04 [hawke1]
hawke1 has joined #musicbrainz-devel
16:18:44 [alastairp]
ruaok: oh yes. just tried it on 3g. works fine
16:18:50 [alastairp]
thanks, upf
16:19:11 [alastairp]
http://labs.acousticbrainz.org
16:22:27 [Freso]
Wow. Such cool stuff. Much making. :D
16:27:35 [ijabz]
ijabz has joined #musicbrainz-devel
16:37:08 [Lotheric_]
Lotheric_ has joined #musicbrainz-devel
16:39:11 [Nyanko-sensei]
Nyanko-sensei has joined #musicbrainz-devel
16:43:16 [alastairp]
so, someone needs to make stuff
16:43:32 [alastairp]
I'm going to do a "average tempo of tracks over the years" plot
16:47:48 [ruaok]
the temptation to make a party playlist generator that doesn't suck is quite great. :)
16:48:21 [alastairp]
maybe we should make one just to get it out of our system
16:48:56 [ruaok]
we gonna need a lot of booze, methinks. :)
17:10:41 [nikki]
nikki has changed the topic to: Last exit before schema change week | Agenda: reviews, schema change preparation, bookbrainz update (Leftmost/LordSputnik), more auto-edits (nikki), general discussion, DR: MBS-3150
17:22:03 [MBJenkins]
Project search_server build #37: SUCCESS in 6 min 27 sec: http://ci.musicbrainz.org/job/search_server/37/
17:34:00 [MBJenkins]
Project musicbrainz-server_beta build #312: SUCCESS in 11 min: http://ci.musicbrainz.org/job/musicbrainz-server_beta/312/
17:45:39 [MBJenkins]
Project musicbrainz-server_master build #87: SUCCESS in 11 min: http://ci.musicbrainz.org/job/musicbrainz-server_master/87/
17:57:53 [MBJenkins]
Project musicbrainz-server_schema-change-2014-q4 build #63: SUCCESS in 12 min: http://ci.musicbrainz.org/job/musicbrainz-server_schema-change-2014-q4/63/
18:22:47 [zas]
how should i add an one-man project ? i usually add it as a person, but often the project as a starting date, and when you set the type to person you can only enter birth date. Any advice ? Why https://wiki.musicbrainz.org/Proposal:Artist_Type_Project was abandoned ?
18:29:33 [reosarevok]
Probably because a one-member group works well enough
18:35:05 [ruaok]
ruaok has joined #musicbrainz-devel
18:37:30 [ruaok]
ianmcorvidae: ping
18:58:05 [kepstin-laptop__]
kepstin-laptop__ has joined #musicbrainz-devel
19:23:40 [nikki]
chirlu`: btw I'm also getting 504s for searches for status = failed vote and type = add relationship
19:31:39 [nikki]
nikki has joined #musicbrainz-devel
19:37:08 [LordSputnik]
LordSputnik has joined #musicbrainz-devel
19:44:52 [legoktm]
alastairp: I implemented basic offline support, just haven't had a chance to test it yet: https://github.com/legoktm/acousticbrainz-client/commit/65ecccbe916abadb24152cc6e2f88915b28760b6
19:47:01 [alastairp]
oh, thanks!
19:47:24 [alastairp]
oh, that's a really neat solution
19:47:32 [alastairp]
can you put the files in ~/.cache instead?
19:48:30 [legoktm]
sure
19:48:40 [legoktm]
I guess it should be ~/.cache/abzsumibt/ ?
19:48:41 [alastairp]
oh - I thought they were still in /tmp. it's ok as you have it..
19:48:54 [legoktm]
I'm putting them in ~/.abzsubmit/cache/ right now
19:49:09 [alastairp]
yeah, I was just thinking that we could use the proper standard
19:50:03 [alastairp]
http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html this one -
19:50:35 [alastairp]
but we're not using the standard for the config file anyway, so I guess it's OK either way
19:51:03 [legoktm]
heh, I didn't even know there was a standard
19:51:47 [alastairp]
yeah, not many people do...
19:53:27 [alastairp]
OK, it looks pretty good in principle. I'll take a closer look tomorrow
19:58:34 [ijabz]
ijabz has joined #musicbrainz-devel
20:00:46 [ruaok]
* ruaok saunters in.
20:00:48 [ruaok]
how are things?
20:01:55 [ianmcorvidae]
not bad. slightly sleepy on my part :P
20:02:08 [ruaok]
morning!
20:02:20 [ruaok]
I just had a long call with google re the JSON-LD stuff.
20:02:32 [ruaok]
and as expected, they have changes in mind.
20:02:42 [ruaok]
and want works. and probably labels.
20:03:04 [ruaok]
I bascially told them that we'll respond to their updated doc at the end of next week.
20:03:20 [ruaok]
which means that we don't need to think about this until the schema change is done.
20:03:46 [ruaok]
given the changes they want, I suggested the updated schedule of markup done dec 1, sitemaps done dec 31.
20:03:52 [CatQuest]
CatQuest has joined #musicbrainz-devel
20:04:04 [ianmcorvidae]
heh. I have some stuff already for works and labels, but :P
20:04:15 [ruaok]
they had no issues with that, but nothing is fim until we respond next week.
20:04:24 [ruaok]
as in entities that being marked up?
20:04:27 [ianmcorvidae]
yes
20:04:37 [ruaok]
is that live already?
20:04:40 [ianmcorvidae]
not all of the inter-linking stuff is done is the main thing (e.g. I don't think I have the artist link to albums yet, for example)
20:04:43 [ianmcorvidae]
some of it should be
20:04:51 [ruaok]
ok, I'll have a look tomorrow.
20:05:19 [ruaok]
what you and I need to do is to go over their marked up version of the SOW and discuss.
20:05:39 [ianmcorvidae]
it'll be more "whatever we happen to have" for things they weren't specifying, but e.g. http://beta.musicbrainz.org/label/41915f1c-0cf5-4517-bef3-13a1ac473439 has markup for location, id, name, and sameAs links (i.e. to other databases and such)
20:05:44 [ruaok]
when to do that is going to be tricky. next week is schema change and conference hell.
20:07:46 [ruaok]
cool. I bet that will make them happy. :)
20:08:09 [ruaok]
I'll follow up with them. their action item is to get us markup for labels. they already did markup for works.
20:08:25 [ruaok]
how is the schema change stuff?
20:08:55 [ianmcorvidae]
pretty good, I think, I haven't caught up with the most recent developments, but tests are passing and AFAIK nikki's been testing things on reo's sandbox
20:09:23 [bitmap]
there's a few things for events still in review but that's about it
20:09:41 [ruaok]
anything that is screaming trouble that we need to look at?
20:09:52 [ruaok]
or should we pick a time of day and announce it?
20:09:59 [ianmcorvidae]
not really, this is a pretty minor one overall
20:10:10 [ruaok]
ok, good. as fall ones normally are.
20:10:12 [ianmcorvidae]
the longest-running thing is just because it's adding a column to the track table, and that took 10 minutes on my server
20:10:18 [ianmcorvidae]
er, on my desktop
20:10:25 [ianmcorvidae]
so should be even faster on totoro, probably
20:10:36 [ruaok]
with 48GB of ram, presumably. :)
20:10:48 [ruaok]
so 15 minutes expected run-time?
20:10:55 [ruaok]
call it 30 minute down time?
20:11:11 [chirlu`]
Despite the RAM, it has to be written to disc because of persistence.
20:11:13 [ruaok]
or do we just go with 45 knowing that we're going to run over? :)
20:11:24 [ianmcorvidae]
yeah, chirlu's right as well
20:11:32 [ruaok]
chirlu`: it has screamingly fast drives. :)
20:11:39 [chirlu`]
Good. :-)
20:11:42 [ianmcorvidae]
let's estimate 20 minutes run time, since that's what it took on my desktop and then we're leaving space for screwing things up too
20:11:44 [ruaok]
10K, if not 15K. I forget.
20:11:55 [ianmcorvidae]
which means let's say between 30 minutes and an hour, I think
20:12:03 [ruaok]
ouch long. :(
20:12:14 [ruaok]
ok, when shall we do this?
20:12:23 [ianmcorvidae]
noting that part of that 20 minutes is vacuuming, of course, so it might also be sooner because of that
20:12:26 [ianmcorvidae]
just being safe :)
20:12:32 [ianmcorvidae]
well, other than 'monday', not sure
20:13:01 [ruaok]
well, you're the main drvier and you have the erratic sleep schedule.
20:13:13 [ruaok]
shall we wait another couple of days to make the estimating easier?
20:13:18 [ruaok]
*driver
20:13:20 [ianmcorvidae]
heh, fair enough.
20:13:25 [ianmcorvidae]
well, we have the meeting in any case
20:13:37 [ianmcorvidae]
so after the meeting should be fine, since I should certainly be awake :)
20:14:04 [ruaok]
pretty late for me, but I'm sure I can manage.
20:14:06 [ianmcorvidae]
if that's okay with you; if that's too late I can move things that way
20:14:26 [ianmcorvidae]
setting a time now is probably best since then I know what I'm aiming for, so
20:14:28 [ruaok]
how about doing it in lieu of the meeting at meeting time?
20:14:42 [ruaok]
since I suspect that our status reports will be "schema change".
20:14:50 [ruaok]
well, maybe not mine.
20:14:52 [ianmcorvidae]
that works for me, and then we can have a quick check-in afterwards
20:15:03 [ruaok]
bitmap: ok with you?
20:15:06 [bitmap]
* bitmap nods
20:15:22 [ruaok]
right then. decided.
20:15:36 [ruaok]
I'l'l post a schema change blog entry tomorrow.
20:15:56 [ruaok]
if either of you have any updates beyond what we already covered, jump in...
20:16:35 [ianmcorvidae]
not really. spent a lot of sunday and monday driving or in El Centro :P
20:16:53 [ruaok]
really? that was sunday?
20:17:00 [ruaok]
seems like eons ago.
20:17:25 [ianmcorvidae]
yeah, that was the original day and then I had my delay in El Centro and actually got to you Monday, lol
20:17:40 [ianmcorvidae]
(and got home Tuesday, I guess, heh)
20:17:58 [ruaok]
gah. you were in remarkably good spirits for all that.
20:18:09 [ruaok]
my week was a lot of travel and then desperately trying to stay awake and do something useful.
20:18:26 [ruaok]
I'm still horribly behind on CB stuff and just now catching up on AB.
20:18:36 [ruaok]
the SSL certs got swapped in, ya?
20:18:57 [ruaok]
I got a notification from our other provider that the *mb.org certs are SHA-2 ready now.
20:19:06 [ruaok]
I'll make that one happen after the schema change.
20:19:28 [ianmcorvidae]
yeah, I got those done fairly fast
20:19:54 [ruaok]
I'm traveling to A'dam (again) on thursday evening and I'm hoping to take friday off so I celebrate Aleta's b-day.
20:20:10 [ruaok]
monday - thu am of next week I am in London.
20:20:30 [ruaok]
I'm speaking about AB in London on wednesday and then in BCN on thursday.
20:20:56 [ruaok]
the key for next week is simple: don't die. :)
20:21:16 [ruaok]
right, who else for review stuff?
20:21:29 [ruaok]
ijabz: the search server changes are good to go, yes?
20:22:01 [bitmap]
I'm nearly done converting all of our js to browserify modules, that's all I worked on besides schema change stuff :P
20:22:29 [chirlu`]
Oh, that sounds good.
20:22:59 [bitmap]
plus actual string extraction for gettext + using Jed instead of this MB.text hack
20:23:39 [ijabz]
ruaok they were , but Ive noticed bitmap just made a chnage to mmdschema
20:23:41 [ruaok]
hacks-- :)
20:23:48 [ruaok]
whoops.
20:23:58 [ijabz]
so a little more to do, but not much
20:24:15 [ruaok]
will this be a simple update or a complex one?
20:24:39 [bitmap]
sorry bout that, I was waiting for an approve on it but since there's just a week left I merged it
20:25:03 [ijabz]
I 'think' a simple one as we are just adding data, but because of schema chnages not 100% sure
20:25:19 [ruaok]
ijabz: ok, let me know as soon as you can.
20:25:36 [ianmcorvidae]
heh, whoops. I think I lost track of that PR then, sorry about that :(
20:25:38 [ruaok]
that changes my weekend quite a bit.
20:26:08 [ijabz]
Can we treat as simple, and if problem revert something
20:26:19 [bitmap]
well, data tracks are returned separately from the main tracklist so as not to break disc ids, so it puts those under a data-track-list element. that's what that PR added
20:27:39 [ijabz]
there a re a number of rng schema chnages since the last release, but I don think anything chnages existing elements does it
20:27:59 [ianmcorvidae]
don't think so, other than adding tags to a few things
20:28:12 [ianmcorvidae]
(which isn't quite what you mean by changing, I think)
20:28:15 [ijabz]
adding is fine
20:28:36 [ruaok]
ok, sounds like it should be simple.
20:28:43 [ruaok]
if that turns out not to be the case, please mail me.
20:28:59 [ruaok]
ok, what else do we have before we pick agenda items?
20:29:49 [chirlu`]
chirlu` has changed the topic to: Last exit before schema change week | Agenda: bookbrainz update (Leftmost/LordSputnik), more auto-edits (nikki), general discussion, DR: MBS-3150
20:29:55 [ruaok]
ding. :)
20:30:06 [ruaok]
Leftmost, LordSputnik : bookbrainz update.
20:30:18 [LordSputnik]
I don't think we have leftmost
20:30:21 [LordSputnik]
but I'll go ahead
20:30:41 [LordSputnik]
We've teamed up on BookBrainz, and decided to proceed using node.js
20:31:15 [LordSputnik]
We talked a lot in a couple of meetings last week, (chatlogs available), and decided a lot of stuff
20:31:19 [derwin]
more readable than perl, anyway.
20:31:39 [LordSputnik]
and now I'm working on our editable webservice, while he focuses on the schema package
20:32:16 [LordSputnik]
any questions? :)
20:32:38 [ruaok]
is it done yet?
20:32:40 [ruaok]
:)
20:32:44 [LordSputnik]
ruaok: :P
20:32:50 [LordSputnik]
check here: https://github.com/BookBrainz
20:32:52 [ruaok]
sorry, had to.
20:33:14 [LordSputnik]
The ws does work, but without the schema package, it doesn't serve anything useful yet
20:33:24 [LordSputnik]
It does have OAuth authentication though ;)
20:33:32 [ruaok]
no real comments for me. I'm excited to see you guys picking this up.
20:33:49 [LordSputnik]
Oh, one thing...
20:33:54 [ruaok]
yes, steve?
20:34:02 [LordSputnik]
after the schema change, we'll need a few changes to the bookbrainz.org nginx config
20:34:15 [LordSputnik]
Mainly to get it using https
20:34:16 [ruaok]
k, np.
20:34:28 [LordSputnik]
other than that, fin. :)
20:34:29 [ruaok]
well, that means me buying a cert for it.
20:34:35 [ruaok]
open an MBH ticket for that.
20:34:43 [chirlu`]
chirlu` has changed the topic to: Last exit before schema change week | Agenda: more auto-edits (nikki), general discussion, DR: MBS-3150
20:35:01 [ruaok]
nikki (or ianmcorvidae?): more auto edits?
20:35:03 [nikki]
hi
20:35:23 [nikki]
I was looking for more things we could make auto-edits and had some more suggestions
20:36:10 [nikki]
which are: edit relationship edits which don't change the endpoints (so they can't indirectly cause entities to be deleted) for everyone, since only a tiny proportion of those fail
20:36:41 [ianmcorvidae]
that leaves just attributes changing, I guess?
20:36:50 [ianmcorvidae]
and dates
20:37:02 [nikki]
types, attributes, dates
20:37:14 [nikki]
and possibly also switching the entities around, if it's not too hard
20:37:30 [nikki]
also: remove alias, isrc and iswc for auto-editors, since those can be recovered from the edit history and removing values from an entity is normally an auto-edit for auto-editors
20:37:39 [nikki]
and if we store the toc in the edits, also remove disc id
20:38:35 [ruaok]
* ruaok has no objections
20:38:39 [nikki]
and: move disc id for everyone, since only a tiny proportion of those fail too
20:38:56 [chirlu`]
I think someone objected to the “remove DiscID” thing once
20:38:57 [nikki]
and: edit medium for everyone where the times don't change by, say, more than 5 seconds
20:39:11 [chirlu`]
because the TOC is too hard to find somewhere in the edit history.
20:39:32 [ianmcorvidae]
hm. if we can make an easy "reattach this discID" link with the data then would that be find?
20:39:44 [chirlu`]
That would be only time changes, I assume?
20:39:50 [nikki]
that's why I wanted the toc to be stored in the remove edit, so we can make it easy to re-add it
20:40:11 [nikki]
yeah, only time changes, unless there's also things like capitalisation which are already auto-edits
20:40:18 [chirlu`]
I think the point was that it is difficult to find the (add/remove) edit.
20:40:25 [nikki]
anyway, those were all of the suggestions I had for now
20:40:43 [chirlu`]
Though perhaps for a simple revert, it’s not.
20:41:22 [ianmcorvidae]
those seem good to me, I guess we should make sure when we do the remove discid one that there's an easy one-click revert then
20:41:31 [ruaok]
* ruaok nods
20:41:42 [ianmcorvidae]
(which, we should start prototyping how to make a 'revert' button anyway if we want to do that more broadly, so)
20:42:36 [ruaok]
for now, lets split these into two classes. those not disputed and those that need more work/thinking/etc/
20:43:16 [nikki]
* nikki wonders if chirlu` has a link to the objection he thinks someone had
20:43:35 [chirlu`]
No, only a vague memory. :-)
20:44:13 [ruaok]
well, then lets focus on thinking more on how to do revert right.
20:44:20 [ruaok]
nikki: will you make tickets, please?
20:44:35 [nikki]
sure
20:44:44 [ruaok]
thx.
20:44:45 [chirlu`]
chirlu` has changed the topic to: Last exit before schema change week | Agenda: DR: MBS-3150
20:44:51 [ruaok]
now for … general discussion?
20:44:54 [ruaok]
been there done that.
20:44:57 [ruaok]
shall we do the DR?
20:45:03 [ruaok]
MBS-3150
20:45:04 [mb-chat-logger]
http://tickets.musicbrainz.org/browse/MBS-3150
20:45:23 [nikki]
I was wondering if we should just won't fix it
20:46:00 [ianmcorvidae]
I think I'd support that
20:46:08 [chirlu`]
nikki: I think I had MBS-7408 with the linked edit in mind.
20:46:08 [mb-chat-logger]
http://tickets.musicbrainz.org/browse/MBS-7408
20:46:20 [ruaok]
* ruaok has no opinion on the matter
20:46:33 [ianmcorvidae]
I see the usefulness of it but I don't actually want to use google anyway and I definitely don't want to do it if it's more than one or two links (e.g. for other search engines), so
20:46:47 [bitmap]
+1 wontfix
20:46:58 [reosarevok]
Add a DDG link instead? :p
20:46:59 [ruaok]
fine by me.
20:47:06 [reosarevok]
* reosarevok is fine with wontfixing though
20:47:15 [ruaok]
ok, let's wontfix.
20:47:21 [nikki]
ok
20:47:23 [ruaok]
and be done for tonight.
20:47:27 [ruaok]
thanks everyone!
20:48:50 [chirlu`]
chirlu` has changed the topic to: Schema change freeze week | Agenda:
20:49:04 [chirlu`]
chirlu` has changed the topic to: Schema change freeze week | Agenda: reviews, general discussion
20:49:40 [chirlu`]
chirlu` has changed the topic to: Schema change freeze week | Agenda: reviews, general discussion | 2014-11-17 meeting time: after schema change deployment
20:59:25 [CallerNo6]
CallerNo6 has joined #musicbrainz-devel
21:01:13 [nikki]
chirlu`: are you actually working on mbs-7971? wondering 'cause I have a not quite finished branch that does that already
21:01:13 [mb-chat-logger]
http://tickets.musicbrainz.org/browse/MBS-7971
21:04:03 [chirlu`]
I have a branch for MBS-7903 where that fits in, not finished either, though.
21:04:04 [mb-chat-logger]
http://tickets.musicbrainz.org/browse/MBS-7903
21:04:32 [nikki]
ah, mine didn't include that one
21:04:54 [nikki]
but it did include more data for a bunch of entities
21:05:15 [nikki]
and then I mistakenly tried making the code more generic and less repetitive and never finished
21:06:50 [chirlu`]
My intention was to show year ranges (not complete dates) for all entities that have them, and types, too.
21:06:59 [nikki]
yeah, that's what I did XD
21:07:30 [LordSputnik]
LordSputnik has left #musicbrainz-devel
21:07:31 [nikki]
oh, right, this is the one where I have five different branches since I've been not finishing it for so long >_<
21:07:50 [chirlu`]
So how do we proceed?
21:08:03 [nikki]
good question
21:09:25 [chirlu`]
Whoever is faster to finish? :-)
21:09:29 [ianmcorvidae]
heh
21:09:42 [nikki]
then you win immediately, I imagine
21:09:48 [ianmcorvidae]
I was going to suggest trying to figure out who that would be a priori, but :P
21:10:00 [ianmcorvidae]
t1
21:10:08 [ianmcorvidae]
er, stupid ctrl key
21:10:17 [chirlu`]
No, it may take some days.
21:10:38 [chirlu`]
Didn’t have much time the last two weeks, unfortunately.
21:12:17 [nikki]
the first commit in the first branch says "Date: Sun May 27 05:03:13 2012 +0000"
21:12:47 [nikki]
so you have another two and a half years before you'll be slower than me :P
21:13:32 [chirlu`]
Oh, I meant the finishing point-in-time, not the whole duration between start-of-work and finishing.
21:13:37 [nikki]
oh, right
21:14:36 [chirlu`]
ianmcorvidae: Do you have an idea about how to debug those simple-but-timing-out edit searches?
21:14:52 [ianmcorvidae]
er, I hadn't looked, I guess
21:15:05 [chirlu`]
bitmap ran an EXPLAIN ANALYZE for me last week and it took 16 ms or so, so that should not be the cause.
21:15:18 [ianmcorvidae]
is there a pattern, or?
21:15:43 [chirlu`]
It seems mostly related to the status field.
21:16:16 [ianmcorvidae]
hm
21:16:25 [chirlu`]
It’s fast on sandboxes, so those don’t help in debugging.
21:16:33 [ianmcorvidae]
I'd guess index bloat or something but from what you're saying it's not in the query step at all
21:16:42 [ianmcorvidae]
since explain analyze would catch that
21:17:02 [nikki]
I guess I'll let you work on it then, and I guess ideally if you're not sure if you'll actually finish it soon, keeping it somewhere public would be good so if I get a sudden urge to work on it, I can just continue what you've done
21:17:19 [chirlu`]
Yes, I also thought about something in Perl, but I don’t see anything that would depend on those type of queries.
21:17:37 [chirlu`]
Example: https://beta.musicbrainz.org/search/edits?auto_edit_filter=&order=desc&negation=0&combinator=and&conditions.1.field=status&conditions.1.operator=%3D&conditions.1.args=5&field=Please+choose+a+condition
21:18:20 [chirlu`]
(Just “Status is Error”, there are a few thousand of them only and an index.)
21:22:47 [ianmcorvidae]
heh, taking ages locally, so maybe I'll be able to figure it out
21:23:37 [nikki]
also I pushed my five branches to bitbucket if you happen to be curious what I was doing, they're unimaginatively named search, search2, search3, search4 and search5
21:24:38 [ianmcorvidae]
chirlu`: the edit search query itself is what's taking a really long time, locally
21:24:44 [nikki]
and iirc at some point one or more of them also fixed mbs-7192, mbs-7404 and mbs-3799
21:24:45 [mb-chat-logger]
http://tickets.musicbrainz.org/browse/MBS-7192
21:24:46 [mb-chat-logger]
http://tickets.musicbrainz.org/browse/MBS-7404
21:24:47 [mb-chat-logger]
http://tickets.musicbrainz.org/browse/MBS-3799
21:25:38 [chirlu`]
Hm.
21:26:24 [ianmcorvidae]
locally slow piece is Index Scan Backwards using edit_pkey on edit, with a filter for the status bit
21:27:02 [chirlu`]
Do you have the new indexes, though, locally?
21:27:06 [ianmcorvidae]
gets the same plan on totoro
21:27:14 [ianmcorvidae]
yeah, this is a new DB
21:27:17 [ianmcorvidae]
schema change testing one
21:27:29 [chirlu`]
Ah, that’s interesting.
21:28:02 [chirlu`]
If it doesn’t use the index on (status, id), I am not surprised it is slow.
21:28:14 [chirlu`]
I’m just surprised it doesn’t use the index.
21:29:33 [ianmcorvidae]
jeez, this is taking a really long time
21:29:35 [ianmcorvidae]
crazy
21:29:38 [ianmcorvidae]
9 minutes now I Think
21:30:27 [chirlu`]
Well, because edits with this status are rare, it has to scan most of the table before finding enough of them.
21:30:33 [ianmcorvidae]
yeah
21:30:49 [chirlu`]
That’s why there is an index on them. ;-)
21:32:08 [chirlu`]
nikki: Regarding MBS-7404, did you just drop the sort name completely? I don’t display it when there is a relevant primary alias, but otherwise, it’s still there.
21:32:09 [mb-chat-logger]
http://tickets.musicbrainz.org/browse/MBS-7404
21:32:39 [nikki]
I don't remember, I think I might've kept it
21:32:49 [nikki]
(when there's no alias)
21:33:28 [nikki]
I'm undecided either way about that... not showing the sortname might encourage more aliases, but will probably just encourage more people to put it in the disambiguation comment instead
21:37:34 [reosarevok]
We should remove the sortname as soon as we move aliases into the /edit page
21:37:41 [reosarevok]
But probably not before that
21:38:10 [chirlu`]
nikki: Yes, you did it like me in https://bitbucket.org/96187/musicbrainz-server/commits/24b0a6eaacab2fb8f2d132f35ccd1b3e08d2cacf (i.e. sortname is still there where there is no alias).
21:38:35 [nikki]
ah, so I did XD
21:39:42 [chirlu`]
(Or I did it like you, depending on perspective.)
21:40:12 [hawke1]
Great minds think alike? ;-)
21:46:49 [chirlu`]
ianmcorvidae: Has your query finished yet?
21:46:58 [ianmcorvidae]
nope
21:47:02 [chirlu`]
I wonder if “ANALYZE edit” might help.
21:47:42 [chirlu`]
If the index is present, wrong statistics should be the only reason for the planner not to use it.
21:48:09 [chirlu`]
But I had that in my update script, so there should be recent stats on totoro.
21:48:31 [ianmcorvidae]
and locally is basically a new database
22:02:09 [CatQuest]
CatQuest has joined #musicbrainz-devel
22:02:15 [catgroove]
catgroove has joined #musicbrainz-devel
22:29:56 [coderhead42]
coderhead42 has joined #musicbrainz-devel
22:46:59 [Freso]
Freso has joined #musicbrainz-devel
23:39:50 [ariscop]
ariscop has joined #musicbrainz-devel
23:45:51 [LordSputnik]
LordSputnik has joined #musicbrainz-devel