IRC log of musicbrainz-devel on 2012-08-10
Timestamps are in UTC.
- 00:08:17 [demosdemon]
- demosdemon has joined #musicbrainz-devel
- 00:11:02 [ruaok]
- ruaok has joined #musicbrainz-devel
- 02:04:34 [nikki]
- nikki has joined #musicbrainz-devel
- 04:48:31 [ruaok_]
- ruaok_ has joined #musicbrainz-devel
- 04:52:09 [warp]
- 좋은 아침입니다!
- 04:52:58 [nikki]
- おはようございます :P
- 04:53:57 [warp]
- :D
- 04:57:56 [ianmcorvidae]
- wow, this font has really ugly korean
- 04:59:31 [nikki]
- it looks ok for me, but the font is too small for the first character
- 04:59:46 [warp]
- the first character is a bit squashed in my fixed-width terminal font here as well.
- 04:59:57 [ianmcorvidae]
- mine has ugly bitmap fonts I think
- 05:00:05 [nikki]
- I mean, I can tell it starts with j and then there's a horizontal vowel and then either ng or h
- 05:01:44 [ianmcorvidae]
- http://ianmcorvidae.net/images/uglykorean.png
- 05:03:06 [nikki]
- eww yeah
- 05:03:16 [nikki]
- the japanese is much nicer
- 05:03:36 [ianmcorvidae]
- the japanese is basically how the rest of the font looks, stylistically
- 05:04:01 [ianmcorvidae]
- I suspect other less-used things would get ugly, but
- 05:04:38 [ianmcorvidae]
- georgian, for example, looks largely fine
- 05:05:07 [nikki]
- the second character looks more like meun than eun to me :/
- 05:06:01 [nikki]
- I suppose georgian has an advantage in that there aren't thousands of characters :P
- 05:06:13 [ianmcorvidae]
- yeah, there is that :P
- 05:10:03 [warp]
- seems very bold for no reason.
- 06:05:28 [warp]
- wtf review board.
- 06:06:00 [warp]
- (it doesn't let me type certain characters)
- 06:08:20 [ianmcorvidae]
- warp: get off the diff page
- 06:08:36 [ianmcorvidae]
- warp: it's keyboard shortcuts for that page specifically
- 06:08:39 [ianmcorvidae]
- which is stupid, but yes :P
- 06:09:57 [warp]
- this + pasting being semi broken makes it very frustrating to use review board.
- 06:10:13 [ianmcorvidae]
- as long as you're not on the diff page it works exactly as before :P
- 06:10:34 [ianmcorvidae]
- (comment popups on the diff page work fine, to my knowledge)
- 06:11:05 [warp]
- our previously deployed version of review board didn't have broken pastes :)
- 06:11:17 [ianmcorvidae]
- no, pronik updated it a few months ago
- 06:11:24 [warp]
- yes, I know.
- 06:11:43 [ianmcorvidae]
- and I've never had any trouble with paste -- except, of course, on the diff page, as mentioned
- 06:11:56 [ianmcorvidae]
- I don't paste much though, I guess
- 06:12:11 [nikki]
- I've never had any problems with pasting, but maybe I haven't used it since it was updated
- 06:12:41 [ianmcorvidae]
- I think just your tx config thing was since the update, unless I'm forgetting something
- 06:12:51 [warp]
- Obviously pasting in itself works, but if you do not touch the field after pasting, review board isn't aware there is something in the field, so clicking OK will nuke your paste.
- 06:13:06 [ianmcorvidae]
- ah
- 06:13:15 [ianmcorvidae]
- I have hit that bug, I hadn't realized that's what it was doing
- 06:13:55 [warp]
- so generally I past the summary and description from the git commit. which now means paste summary, click in the field to focus on something harmless, hit a key (space), then click OK. and again for the description or anything else I need to paste.
- 06:14:21 [ianmcorvidae]
- yeah
- 06:14:27 [ianmcorvidae]
- I've only hit it with copy-pasting ticket numbers
- 06:14:43 [warp]
- in itself not _that_ much of a burden, but it's very annoying when I've forgotton review board is broken
- 06:15:07 [ianmcorvidae]
- yeah
- 06:15:13 [ianmcorvidae]
- I should set up post-review
- 06:15:50 [warp]
- I have a post-review python wrapper which I can use for typical cases. but these diffs aren't diffs against master, so I'm not sure my script will work.
- 06:16:13 [ianmcorvidae]
- ah
- 06:16:56 [ianmcorvidae]
- at present I still do all of mine by hand
- 06:17:17 [ianmcorvidae]
- I've gotten very good at the git diff master...mbs-whatever<tab> > <whatever>.diff routine :P
- 06:17:17 [warp]
- I'll push my script somewhere public after I'm done submitting this stuff to code review.
- 06:24:06 [warp]
- unrelated, next weekend is hack weekend!
- 06:25:12 [ianmcorvidae]
- true
- 06:25:15 [ianmcorvidae]
- I should decide what I'm doing
- 06:25:34 [ianmcorvidae]
- "fix a bunch of bugs" isn't quite glamorous enough, given that's what I do with the rest of my time ;)
- 06:30:00 [nikki]
- coding?
- 06:38:13 [warp]
- ianmcorvidae: https://gitorious.org/musicbrainz-post-review/musicbrainz-post-review
- 06:39:01 [warp]
- ianmcorvidae: the only two ideas I had so far would be making picard more modular (but that sounds like a lot of work, and indeed not very glamorous)
- 06:39:15 [warp]
- ianmcorvidae: and adding JSON-LD support to my /ws/2/ json work.
- 06:39:21 [ianmcorvidae]
- more modular as in library vs. frontend stuff?
- 06:39:30 [ianmcorvidae]
- (because if so that's a lot more work than a weekend, heh)
- 06:39:37 [ianmcorvidae]
- JSON-LD might be neat
- 06:40:14 [ianmcorvidae]
- if you can figure out how to encode everything into the relevant RDF
- 06:40:22 [warp]
- ianmcorvidae: yes. the tagging files stuff shouldn't require Qt or any other GUI bits. so you can just import only those bits to write your own tagger.
- 06:40:39 [warp]
- ianmcorvidae: I can just look at the existing RDFa for that.
- 06:40:45 [ianmcorvidae]
- the issue is that at present it uses Qt stuff for all its internal XML and network stuff
- 06:40:57 [ianmcorvidae]
- the current RDFa doesn't cover a lot of stuff, unfortunately, especially ARs
- 06:41:04 [ianmcorvidae]
- but I guess you can just mirror those limitations
- 06:41:37 [warp]
- hm, no RDF.rb on this machine.
- 06:41:42 [warp]
- * warp installs.
- 06:42:29 [ianmcorvidae]
- hm, copy-paste from gitorious doesn't work properly
- 06:42:30 [ianmcorvidae]
- bah
- 06:43:17 [warp]
- ianmcorvidae: I'll be happy if in a weekend a basic release (with artists, mediums, tracks) lookup is complete
- 06:43:33 [ianmcorvidae]
- yeah
- 06:44:28 [ianmcorvidae]
- oh, bah, your script probably says 'python' when I need 'python2'
- 06:44:47 [warp]
- it usually says /usr/bin/env python
- 06:44:52 [ianmcorvidae]
- yeah
- 06:44:56 [ianmcorvidae]
- that's python3, I use Arch
- 06:45:09 [warp]
- right. it shouldn't be difficult to make it compatible with both really.
- 06:45:39 [warp]
- obviously you can run it with python2, but I'd accept patches with python3 compatibility fixes :)
- 06:45:45 [ianmcorvidae]
- hehe
- 06:46:26 [ianmcorvidae]
- heh, also needs 'virtualenv2' apparently
- 06:55:01 [bitmap]
- when I try to run create_test_db on rika I just get failed: FATAL: Ident authentication failed for user "postgres"
- 06:57:28 [bitmap]
- * bitmap wonders if it's possible to run tests on rika or if he should just set up his own server
- 07:11:52 [warp]
- ianmcorvidae: curl 'http://rdf.greggkellogg.net/distiller?format=turtle&in_fmt=rdfa&uri=http://musicbrainz.org/release/aff4a693-5970-4e2e-bd46-e2ee49c22de7'
- 07:11:58 [warp]
- ianmcorvidae: that looks fairly complete
- 07:12:22 [warp]
- (though with some errors)
- 07:16:42 [ianmcorvidae]
- bitmap: it should be possible to run tests, the DB should already be created
- 07:16:55 [ianmcorvidae]
- you need to make a TEST entry in DBDefs of course, but
- 07:17:57 [bitmap]
- * bitmap did so
- 07:18:11 [bitmap]
- I was following the instructions here http://chatlogs.musicbrainz.org/musicbrainz-devel/2012/2012-05/2012-05-24.html
- 07:19:32 [ianmcorvidae]
- yeah, you shouldn't need to run create_test_db.sh
- 07:20:32 [ianmcorvidae]
- pick up after that I gues
- 07:21:16 [ianmcorvidae]
- carton exec -Ilib -- prove t/
- 07:21:32 [ianmcorvidae]
- or whatever filtering you want to do
- 07:21:35 [bitmap]
- hm, the tests run now but it doesn't seem like there's anything in the database
- 07:21:37 [ianmcorvidae]
- t/tests.t is usually all you need to run
- 07:21:40 [ianmcorvidae]
- there won't be
- 07:21:49 [ianmcorvidae]
- it creates an empty DB, and each test does a ROLLBACK after it's done
- 07:22:03 [bitmap]
- ah, neat
- 07:22:08 [bitmap]
- thanks :)
- 07:22:12 [ianmcorvidae]
- basically each test runs in its own transaction which rolls back at the end
- 07:22:18 [ianmcorvidae]
- no problem :)
- 07:31:28 [ianmcorvidae]
- bitmap: see last comment on http://tickets.musicbrainz.org/browse/MBS-5119 -- the rel editor came up on an unrelated ticket, and jesus2099 has some comments that I don't know how to read :P
- 07:32:10 [nikki]
- does anyone ever know? :P
- 07:32:24 [ianmcorvidae]
- well, bitmap's got a better chance than the rest of us on this topic anyway :P
- 08:01:09 [ijabz]
- ijabz has joined #musicbrainz-devel
- 08:03:41 [bitmap]
- ianmcorvidae: added a comment to the whitespace ticket
- 08:03:47 [ianmcorvidae]
- cool
- 08:04:02 [ianmcorvidae]
- just doing ticket maintenance things as I work through scheduling slowly
- 08:04:05 [bitmap]
- I guess the other ticket was http://tickets.musicbrainz.org/browse/MBS-5119 :)
- 08:16:58 [djce]
- djce has joined #musicbrainz-devel
- 08:41:14 [ijabz]
- ijabz has joined #musicbrainz-devel
- 08:49:56 [Mineo]
- warp: there's actually a pep making /usr/bin/python a bit vague, so explicitly specifying a 2 or 3 is the way to go :)
- 08:54:11 [djce]
- djce has joined #musicbrainz-devel
- 09:01:05 [bitmap]
- I'm not sure how to work around https://github.com/SteveSanderson/knockout/issues/132
- 09:01:23 [bitmap]
- whatever html parser the perl tests are using doesn't like that I have && in some attributes, but I can't escape them
- 09:01:49 [bitmap]
- if I escape them, it isn't valid javascript
- 09:06:27 [bitmap]
- oh, escaping them works in normal attributes, just not in <!-- comments -->
- 09:06:40 [bitmap]
- disregard :)
- 09:22:48 [warp]
- Mineo: I wasn't aware that python2 and python3 should be available on a default python install
- 09:22:59 [warp]
- Mineo: good to know. do you know which PEP ?
- 09:24:20 [stefans_]
- stefans_ has joined #musicbrainz-devel
- 09:40:30 [Mineo]
- warp: it's http://www.python.org/dev/peps/pep-0394/
- 09:48:51 [ocharles]
- morning
- 09:52:28 [Leftmost]
- Leftmost has joined #musicbrainz-devel
- 09:54:01 [warp]
- Mineo: thanks.
- 10:08:48 [djce]
- djce has joined #musicbrainz-devel
- 10:15:40 [reosarevok]
- reosarevok has joined #musicbrainz-devel
- 10:16:00 [reosarevok]
- In case nobody has said so already (doubt it, but...) 502 and 500 in live like crazy
- 10:18:57 [ocharles]
- i haven't seen anything reported
- 10:19:13 [ocharles]
- because of course, we still don't have nagios giving me a single alert
- 10:19:44 [ocharles]
- looks like the search server is non responsive?
- 10:20:03 [reosarevok]
- Yeah, search was failing randomly too
- 10:20:08 [MBJenkins]
- Project musicbrainz-server_beta build #69: FAILURE in 3 min 17 sec: http://ci.musicbrainz.org/job/musicbrainz-server_beta/69/
- 10:20:08 [MBJenkins]
- * ollie: MBS-5029: When editing a relationship, refresh the old end points cover art
- 10:20:09 [MBJenkins]
- * ollie: MBS-5029: Add comments to Edit/Relationship/Edit cover art refreshing logic
- 10:20:09 [MBJenkins]
- * ollie: MBS-5078: Add the controlled for whitespace check on various tables
- 10:20:10 [MBJenkins]
- * ollie: MBS-5071: Make sure spaces are collapsed in our custom Field::Text field
- 10:20:10 [MBJenkins]
- * ollie: MBS-5079: Changing a work's language is only an auto-edit under certain conditions
- 10:20:11 [MBJenkins]
- * ianmcorvidae: Disable showing the "Add Note" button when it does nothing.
- 10:20:11 [MBJenkins]
- * nikki: More Amazon URL cleanup (MBS-4431, MBS-4698, MBS-4806)
- 10:20:12 [MBJenkins]
- * nikki: Forgot to escape a dot
- 10:20:12 [MBJenkins]
- * ollie: MBS-5079: Minor correction to test name
- 10:20:34 [reosarevok]
- But that shouldn't cause 500 errors while editing URLs... we don't use the search server there (or do we?)
- 10:20:53 [ocharles]
- no, i haven't seen those errors yet
- 10:20:56 [ocharles]
- still hunting through logs
- 10:21:32 [ocharles]
- i don't see a single timeout that's not /ws/2 in log
- 10:21:33 [ocharles]
- logs*
- 10:21:41 [ocharles]
- so i guess it's not a timeout?
- 10:22:13 [ocharles]
- i don't see any errors in our postgresql log either
- 10:22:43 [ocharles]
- so... need a bit more info
- 10:23:20 [ocharles]
- http://stats.musicbrainz.org/webstats/nginx-rrd/drraw.cgi?Mode=view;Graph=1273836365.880 that certainly confirms what you're saying though
- 10:27:34 [reosarevok]
- Dunno, editing URLs, loads of non-instant 502 coupled with some instant 500s
- 10:27:44 [reosarevok]
- Can try to get some again by editing some more URLs :p
- 10:28:24 [reosarevok]
- Oh, look
- 10:28:27 [reosarevok]
- Didn't even need to edit
- 10:28:34 [reosarevok]
- http://musicbrainz.org/url/01483c17-5364-4578-a323-167d11706862 gave me a 500 just by opening it
- 10:30:06 [reosarevok]
- And 502 in http://musicbrainz.org/edit/relationship/delete?returnto=http%3A%2F%2Fmusicbrainz.org%2Furl%2Fe49b3b61-7e12-4762-bb1b-9aae2e399259&type1=url&type0=release_group&id=3976
- 10:30:12 [ocharles]
- it is all feeling very slow
- 10:30:16 [reosarevok]
- 500 on refresh
- 10:43:33 [ianmcorvidae]
- ocharles: noticing that djce's MBH-270 fix was committed about when this started happening
- 10:43:42 [ianmcorvidae]
- ocharles: not sure if that's relevant, but it does correlate time-wise
- 10:44:02 [djce]
- ianmcorvidae: ping me repeatedly if I need to fix something please
- 10:44:10 [djce]
- in a meeting and reading CVs....
- 10:44:26 [ocharles]
- ianmcorvidae: oh, that's interesting
- 10:44:42 [ocharles]
- ianmcorvidae: what repo/commit?
- 10:44:50 [ianmcorvidae]
- ocharles: frontend nginx 7846d1d
- 10:45:10 [ianmcorvidae]
- ocharles: it's only SSL-related stuff and should only touch test, but
- 10:45:14 [ocharles]
- mmm
- 10:45:35 [ocharles]
- there's ee289 on lenny too
- 10:46:10 [ocharles]
- djce: you only touched carl/lenny with this stuff, i take it?
- 10:46:18 [djce]
- yes
- 10:46:27 [djce]
- and only the hobbes vhost
- 10:46:29 [ianmcorvidae]
- yeah, that's a bit later, though that does correlate with the brief drop in 5xx errors
- 10:46:30 [ocharles]
- ok, I'm going to try temporarily reverting back a few revisions and see what happens
- 10:46:53 [ianmcorvidae]
- (suggesting that the nginx restart there 'fixed' it temporarily)
- 10:53:55 [ocharles]
- http://stats.musicbrainz.org/webstats/nginx-rrd/drraw.cgi?Mode=view;Template=1305563677.21559;Base=%2Fhome%2Fdjce%2Fpostgres-rrd%2Frrd%2F%2Fd%3Dmusicbrainz_db_20110516.rrd dafuq?
- 10:54:03 [ocharles]
- I think it may well just be coincidental, this looks more worrying
- 10:54:49 [ianmcorvidae]
- the huge climb in the past couple days is strange, yeah
- 10:55:05 [ianmcorvidae]
- different timescale than the current 5xx stuff, of course, but
- 10:55:28 [ocharles]
- true, but does match our release on 08-06
- 10:55:51 [ianmcorvidae]
- at least some, yeah
- 10:56:03 [ianmcorvidae]
- weird that it took a day after release to start climbing, I guess, but
- 10:56:27 [ocharles]
- not necessarily a day, we could have just released at the end of the day
- 10:56:32 [ocharles]
- in whatever timezone these charts are in
- 10:56:46 [ocharles]
- hmm, they seem to only be an hour behind my system clock, so maybe that doesn't quite add up
- 10:57:05 [warp]
- ocharles: that would be UTC
- 10:57:16 [warp]
- your BST is one hour ahead of UTC.
- 10:57:29 [ianmcorvidae]
- yeah, they're UTC
- 10:58:19 [warp]
- ocharles: anything I can help with?
- 10:58:28 [ocharles]
- warp: ideas would be good :)
- 10:58:33 [ocharles]
- anything in nagios?
- 10:58:57 [djce]
- djce has joined #musicbrainz-devel
- 10:59:06 [warp]
- SEARCH is/was critical, don't see anything else.
- 10:59:43 [ocharles]
- is or was?
- 11:00:03 [warp]
- both :)
- 11:00:19 [warp]
- zaphod/search-167-148 recovered, roobarb/search is still CRITICAL
- 11:00:25 [ocharles]
- ah, k
- 11:00:48 [warp]
- I only get the e-mails, I don't currently have access to the nagios dashboard
- 11:01:14 [ianmcorvidae]
- 5xx errors rrd graph is dropping slowly but definitely not decisively
- 11:01:55 [ocharles]
- * ocharles nods
- 11:02:39 [warp]
- there is no obviously load or memory problem on any of the hosts I looked at. bit you probably already checked that as well.
- 11:03:00 [ianmcorvidae]
- it's sort of strange how roobarb appears to be doing nothing, given that it's also critical
- 11:03:08 [ianmcorvidae]
- but I guess that could be load-balancing at work, eh
- 11:03:10 [ocharles]
- that's the weird thing, the hosts all look fine, the logs don't contain errors
- 11:03:48 [warp]
- ha, no roobarb is ok as well. I overlooked the recovery e-mail.
- 11:03:53 [ocharles]
- :)
- 11:04:05 [ocharles]
- doesn't nagios give you a dashboard page?
- 11:05:15 [ocharles]
- the search servers are definitely not happy
- 11:05:23 [ocharles]
- run 'tail -n0 -f /usr/local/mb_server-fastcgi/log/oddlog/current | grep -v 'warning' | tai64nlocal' in our frontends, and they are just scrolling errors
- 11:05:28 [warp]
- ocharles: yes. but I do not have access to it.
- 11:05:34 [ocharles]
- fan.tastic.
- 11:05:51 [warp]
- ocharles: as in, I don't know the URL, and if I did I wouldn't know the passwords. (I think I recently dug up the url and tried).
- 11:06:34 [warp]
- so the regular site should be working OK, as long as we don't do searches?
- 11:06:56 [ocharles]
- you'd think so... but reosarevok is reporting timeouts everywhere
- 11:07:25 [warp]
- I clicked on http://musicbrainz.org/release/aff4a693-5970-4e2e-bd46-e2ee49c22de7 and that is definitely taking its time.
- 11:07:39 [ocharles]
- it's just weird that 'tail -f /etc/service/mb_server-fastcgi/log/oddlog/current | tai64nlocal | grep 'error' | grep -v 'ws/2' doesn't show errors for anything else
- 11:07:50 [warp]
- (but it loads without a 5xx)
- 11:08:01 [warp]
- (and a second time it is fast)
- 11:08:03 [ianmcorvidae]
- hm, interestingly I just got that page really fast
- 11:08:04 [ianmcorvidae]
- yeah
- 11:08:18 [ianmcorvidae]
- suggests DB latency, second-and-later hit cache?
- 11:08:28 [ocharles]
- * ocharles is definitely spending more time with graphite next week
- 11:08:57 [ianmcorvidae]
- * ianmcorvidae is definitely getting myself logons for more of our servers so I can be more use than refreshing public graphs :)
- 11:09:03 [warp]
- doing an artist search for "m-flo" also returns without trouble.
- 11:10:54 [warp]
- man, that is a lot of complaining in that oddlog
- 11:13:17 [ocharles]
- for some reason the cover art script seems to be doing queries almost every 5 seconds
- 11:13:18 [ocharles]
- that seems odd
- 11:13:25 [ocharles]
- (find_outdated_release)
- 11:14:30 [warp]
- gah. a query from astro to a search server seemed to take very long. but subsequent queries (with different query= values) are instant.
- 11:14:35 [hrglgrmpf]
- hrglgrmpf has joined #musicbrainz-devel
- 11:14:38 [warp]
- this stuff is difficult to pin point.
- 11:15:43 [ocharles]
- I've just stopped the cover art refresher
- 11:16:38 [ocharles]
- I'm pretty sure that script was just starting up, querying, dieing, and then restarting. there is no logging for it at all
- 11:16:44 [ocharles]
- (the log is discarded)
- 11:17:53 [ocharles]
- no queries are taking over 2 seconds according to totoro, so i doubt that it's anything to do with that machine...
- 11:18:34 [warp]
- the search server is either instant, or takes 30 seconds to respond. for the same query, with the same results.
- 11:21:19 [djce]
- djce has joined #musicbrainz-devel
- 11:21:34 [ocharles]
- warp: if you do tail -f /var/log/nginx/001-musicbrainz.access.log | grep 502 | grep -v '/ws' on all our machines, pingu is the only one struggling
- 11:21:37 [warp]
- hrm, astro is connecting to a search server through lenny?
- 11:21:44 [ocharles]
- so I'm going to take that out of rotation
- 11:22:17 [warp]
- ok
- 11:22:50 [djce]
- * djce is back, snacking on lunch. Ping me if I can help
- 11:23:05 [ocharles]
- yea, that's just shifted it all to astro, as somewhat to be expected
- 11:24:09 [warp]
- oh the search servers are load balanced. so presumably the 30 second responses are from the one which is b0rked, and the other responses are from the good server.
- 11:24:21 [ocharles]
- warp: can we take the b0rked one out of rotation?
- 11:24:36 [warp]
- as soon as I figure out which one, yes, I'd say so.
- 11:24:54 [ocharles]
- :D
- 11:25:14 [warp]
- yeah, .21 responds fast. .22 is slow.
- 11:25:19 [warp]
- (or broken)
- 11:25:43 [ianmcorvidae]
- roobarb is the problem one then, I guess
- 11:25:46 [warp]
- djce: what is a good way to know which of lenny or carl is "active" ?
- 11:25:50 [ocharles]
- warp: do you know what 10.1.1.247 is?
- 11:25:58 [djce]
- warp: sudo ha-statall
- 11:26:00 [warp]
- ocharles: yes, that's lenny.
- 11:26:04 [ocharles]
- ah, ok
- 11:26:09 [djce]
- ALL = active NONE = inactive
- 11:26:11 [warp]
- ocharles: or lenny + carl.
- 11:26:14 [djce]
- anything else = bad
- 11:26:57 [djce]
- 10.1.1.247 is the rate limiter service IP, provided by carl/lenny
- 11:26:59 [warp]
- djce: awesome, thanks.
- 11:27:06 [djce]
- (iir)
- 11:27:07 [djce]
- (c)
- 11:28:05 [warp]
- ocharles: ok, roobarb should be out of rotation.
- 11:28:47 [ocharles]
- we're all quiet for 502s on frontends atm
- 11:28:56 [ocharles]
- warp: hypothesis: the bad search server is constantly timing out, we have a backlog of request handlers that are all waiting for search responses, other requests thus timeout because they can't get serviced
- 11:29:02 [ocharles]
- not entirely convinced, but it's plausable
- 11:30:25 [ocharles]
- http://stats.musicbrainz.org/webstats/nginx-rrd/drraw.cgi?Mode=view;Graph=1273836365.880 i think we got it sorted
- 11:31:18 [warp]
- right. I'm going to see if I can get this roobarb thing running again.
- 11:31:35 [ocharles]
- still not a single 502 on any frontend
- 11:32:40 [warp]
- restarted jetty on roobarb, it responds instantly again.
- 11:32:52 [ocharles]
- so jetty isn't all that magic as we hoped :)
- 11:32:58 [warp]
- nope
- 11:33:07 [warp]
- I'll put it back in rotation.
- 11:33:45 [warp]
- ok, roobarb should be in use again.
- 11:33:51 [ianmcorvidae]
- I guess there's a nagios check we can at least get out of all this, perhaps something about response times for each seach server?
- 11:34:37 [ocharles]
- we're certainly missing some reporting somewhere
- 11:35:27 [ianmcorvidae]
- also: http://stats.musicbrainz.org/webstats/nginx-rrd/drraw.cgi?View=-1&Graph=1273836365.880&Start=end+-+4+hours&End=now&Mode=view&Shift=&Count= definitely fixed :)
- 11:35:34 [warp]
- ocharles: I'll send an e-mail with what I did. roobarb being fussy was the only thing we found right? nothing else was considered troublesome and restarted, etc.. ?
- 11:36:01 [ianmcorvidae]
- cover art script was restarted and the test https stuff was reverted, though I don't know that either of those was the actual problem
- 11:36:10 [ocharles]
- we (me and ianmcorvidae) are a tad concerned by a large growth in tup_return from totoro
- 11:36:13 [ianmcorvidae]
- (those reversions may have since been reverted, ocharles?)
- 11:36:19 [ocharles]
- but totoro was deemed healthy
- 11:36:25 [warp]
- ocharles: oh and you stopped some cover art thing.
- 11:36:26 [ocharles]
- oh yea, I better bring lenny back up to HEAD
- 11:36:58 [ocharles]
- back at ee289bf94b0e32c5e4a4b41d6ea8
- 11:37:45 [ocharles]
- 502's are coming back
- 11:37:53 [ocharles]
- no wait
- 11:37:58 [ocharles]
- i accidently scrolled my terminals :D
- 11:38:02 [ianmcorvidae]
- hah
- 11:38:53 [warp]
- haha
- 11:40:27 [ianmcorvidae]
- hm, not seeing anything in the last release that should have even touched DB things
- 11:40:44 [ocharles]
- nope, i did a diff too
- 11:41:17 [ianmcorvidae]
- * ianmcorvidae sees what hosting things got touched in the last couple days
- 11:43:37 [ocharles]
- rememeber it's not necessarily a change that we made, it could be something fell over and now different code paths are executing
- 11:43:49 [ianmcorvidae]
- sure, but worth ruling it out
- 11:43:57 [ianmcorvidae]
- I don't see anything in MBH
- 11:44:56 [ocharles]
- * ocharles nods
- 11:50:10 [ianmcorvidae]
- hm, could be the cover art script -- there's a big dip where it went down to more normal values while the script was off -- unless it hasn't been restarted?
- 11:50:52 [ianmcorvidae]
- (it's back up to super-high, but it did drop just after it was turned off)
- 11:50:58 [warp]
- ugh. I suck at reading nagios e-mails.
- 11:51:29 [ocharles]
- it has been restarted, about 15 minutes ago
- 11:51:52 [ianmcorvidae]
- http://stats.musicbrainz.org/webstats/nginx-rrd/drraw.cgi?View=-1&Template=1305563677.21559&Base=%2Fhome%2Fdjce%2Fpostgres-rrd%2Frrd%2F%2Fd%3Dmusicbrainz_db_20110516.rrd&Start=end+-+1+hours&End=now&Mode=view&Shift=&Count= so that drop in the middle is while it was off
- 11:52:03 [ocharles]
- interesting
- 11:52:06 [warp]
- so roobarb was CRITICAL, then recovered back to OK, then went CRITICAL again. so when I said "ha, no roobarb is ok as well. I overlooked the recovery e-mail.", I was WRONG.
- 11:52:18 [ocharles]
- so now to find out why that script is not doing mucrh
- 11:52:19 [ocharles]
- much*
- 11:55:13 [ocharles]
- ianmcorvidae: apparently we have no out of date artwork
- 11:55:23 [ocharles]
- so it's just running in a far too tight loop
- 14:23:19 [reoafk]
- reoafk has joined #musicbrainz-devel
- 15:35:00 [stefans_]
- stefans_ has joined #musicbrainz-devel
- 15:47:53 [plaintext]
- plaintext has joined #musicbrainz-devel
- 15:48:52 [reoafk]
- reoafk has joined #musicbrainz-devel
- 15:51:10 [ijabz]
- ijabz has joined #musicbrainz-devel
- 16:08:34 [hawke_1]
- hawke_1 has joined #musicbrainz-devel
- 16:12:05 [Leftmost]
- Leftmost has joined #musicbrainz-devel
- 16:17:26 [hrglgrmp1]
- hrglgrmp1 has joined #musicbrainz-devel
- 16:34:14 [reoafk]
- reoafk has joined #musicbrainz-devel
- 16:41:43 [hrglgrmp1]
- hrglgrmp1 has joined #musicbrainz-devel
- 16:55:39 [ruaok]
- ruaok has joined #musicbrainz-devel
- 16:57:20 [hrglgrmpf]
- hrglgrmpf has joined #musicbrainz-devel
- 17:06:30 [warp]
- * warp got invited to an empty channel.
- 17:09:05 [Muz]
- Maybe everyone left because you smell.
- 17:09:06 [reoafk]
- Is that the "get lost" of IRC?
- 17:09:38 [ocharles]
- warp: i was going to talk to you and ian there about something, but ian cleared it up in pm
- 17:09:48 [ocharles]
- nice 6 hour lag though :P
- 17:10:10 [warp]
- ocharles: invites are not very obvious in this client (with my config)
- 17:10:33 [ocharles]
- evidently :)
- 17:10:55 [reoafk]
- Maybe you should configure them so they mail you!
- 17:10:58 [reoafk]
- * reoafk hides
- 17:11:09 [warp]
- :P
- 17:13:57 [plaintext]
- Hello
- 17:14:05 [plaintext]
- I tried creating a manifest and moving a script to root/static/scripts from the tt file, but it didn't work
- 17:14:12 [plaintext]
- then I tried to revert all the changes I made, and now the original won't work either :)
- 17:14:38 [plaintext]
- I used statistics.js.manifest originally, and it used to work
- 17:15:00 [plaintext]
- now I get this error: Uncaught TypeError: Object [object Object] has no method 'hashchange'
- 17:15:27 [plaintext]
- here is the tt file, I removed all the scripts and added a single alert, but it doesn't get called
- 17:15:28 [plaintext]
- https://github.com/balidani/musicbrainz-server/blob/master/root/logstatistics/category.tt
- 17:15:35 [plaintext]
- does anyone know what's wrong?
- 17:24:57 [reoafk]
- the code!
- 17:25:00 [reoafk]
- *rimshot*
- 17:27:45 [ruaok]
- plaintext: are you in the habit of committing your code at regular intervals?
- 17:27:59 [ruaok]
- because if so, you can go back to the old version and see what you changed.
- 17:28:21 [ruaok]
- you can examine each of the changes, undo each one, run it and narrow down where the trouble is.
- 17:29:42 [ocharles]
- ruaok: guess you're aware of the downtimeyness today?
- 17:29:53 [ruaok]
- I just sent a response in.
- 17:30:03 [ruaok]
- nagios sent mail about what went down.
- 17:30:21 [ruaok]
- it shouldn't have been hard to diagnose, unless the warning came too late.
- 17:30:39 [ruaok]
- looks like we need to examine why nginx didn't fail over to dora.
- 17:33:19 [ocharles]
- i don't get email from nagios
- 17:33:27 [ocharles]
- i don't have any access to nagios, infact
- 17:33:42 [ruaok]
- only dave and I do, since its hosted on non-mb hardware.
- 17:33:54 [ruaok]
- open an MBH ticket for dave to ensure that nagios sends you mail too.
- 17:35:57 [ocharles]
- For those who care: http://ocharles.org.uk/mb-arch-notes-1.pdf
- 17:35:59 [ocharles]
- http://ocharles.org.uk/mb-arch-notes-2.pdf
- 17:36:08 [ocharles]
- not too much there
- 17:36:47 [plaintext]
- ruaok: nope, I didn't commit this yet before :(
- 17:37:11 [ruaok]
- get in the habit of doing that. when you get something working, commit it.
- 17:37:11 [plaintext]
- ruaok: but I don't even know what went wrong
- 17:37:18 [plaintext]
- okay
- 17:37:23 [ruaok]
- then you can always go back and find what went wrong
- 17:37:25 [plaintext]
- lesson learned, I guess
- 17:37:47 [ruaok]
- if you use vi, you can turn on eternal undo.
- 17:37:48 [ocharles]
- you can always rebase to squash commits together
- 17:38:02 [ruaok]
- even after you exit the editor, undo can persist.
- 17:38:15 [ruaok]
- you can go back in time and forward in time. that way you could see what you changed.
- 17:38:29 [ruaok]
- you could go back to an older version that worked, save it, then go forward again.
- 17:38:36 [ruaok]
- then you can diff the old vs the new.
- 17:38:46 [plaintext]
- well I did undo all of the changes up until it was working in category.tt
- 17:39:01 [plaintext]
- then I even removed the js and added a single alert, but for some reason jquery gives a weird error
- 17:39:05 [ruaok]
- for all of you vim users out there, check this out: http://oscon.tumblr.com/post/27706457604/oscon-instantly-better-vim
- 17:39:20 [ruaok]
- paste the error
- 17:39:38 [plaintext]
- Uncaught TypeError: Object [object Object] has no method 'hashchange'
- 17:39:44 [warp]
- ruaok: I always got a response when testing the search server from astro (i.e. connecting to lenny). sometimes the response would take 30 seconds and sometimes it was instant. so, I assume lenny failed over to dora on each request after roobarb didn't respond within 30 seconds?
- 17:40:02 [plaintext]
- and it was in jquery-1.5.1.js:869
- 17:40:15 [ruaok]
- warp: ah, thats good to know.
- 17:40:36 [ruaok]
- it sounds like jetty was partially failing and this not tripping nginx to fail roobarb out.
- 17:40:41 [warp]
- ruaok: but it should probably take roobarb out of rotation after X amount of failed attempts?
- 17:41:00 [ruaok]
- it should, but didn't. lets examine the config.
- 17:41:53 [ruaok]
- max_fails=10 fail_timeout=10s
- 17:43:11 [ruaok]
- 10 fails in 10 seconds until its gets taken out.
- 17:43:33 [ruaok]
- it sounds like it was getting at less than that.
- 17:43:42 [ruaok]
- but how to know what the actual failures were?
- 17:44:30 [ruaok]
- the docs example has 3f and 30s
- 17:44:40 [ruaok]
- that seems more reasonable.
- 17:44:44 [ruaok]
- thoughts?
- 17:45:10 [ocharles]
- thoughts on?
- 17:45:13 [ocharles]
- changing the settings to that?
- 17:45:17 [ruaok]
- yep
- 17:45:27 [ocharles]
- when does it come back into rotation?
- 17:45:36 [ocharles]
- i think it was going out of rotation just fine, but coming back in very quickly
- 17:45:41 [ocharles]
- but i'm not sure how it all works
- 17:47:37 [ruaok]
- no idea. the docs dont say
- 17:48:42 [warp]
- re
- 17:49:34 [warp]
- ruaok: hrm. fail_timeout at 10 seconds seems odd, the request was given 30 seconds to finish, so that clause either isn't working or doesn't mean what I think.
- 17:49:52 [ruaok]
- http://wiki.nginx.org/HttpUpstreamModule
- 17:50:46 [warp]
- ah, that sounds aggressive
- 17:50:55 [ocharles]
- time to get myself home
- 17:50:57 [warp]
- so it needs to fail 10 times within 10 seconds.
- 17:51:13 [ruaok]
- I doubt we ever triggered that.
- 17:51:15 [warp]
- can it even register 1 fail in 10 seconds if a request is given 30 seconds? :)
- 17:51:42 [ruaok]
- I'd be interested in trying 3 and 30s
- 17:51:44 [warp]
- so, naively I would say lower max_fails and increase fail_timeout.
- 17:52:16 [warp]
- i'd say 31s to avoid the edge case
- 17:52:26 [ruaok]
- k
- 17:53:02 [ruaok]
- ok, thats sounds sane enough to try.
- 17:53:16 [ruaok]
- and we can artificially test this by taking one of the servers donw.
- 17:53:34 [warp]
- yeah, probably.
- 17:53:58 [ruaok]
- the question in my mind is where the 30 seconds comes from.
- 17:54:17 [ruaok]
- is that the nginx timeout for a request to be forwarded?
- 17:54:36 [ruaok]
- nothing from the search server should take more than a couple of seconds.
- 17:54:47 [ruaok]
- the front ends only wait 5 seconds for a result.
- 17:55:03 [warp]
- hm
- 17:55:45 [ocharles]
- taking a server down won't give you a 30 second timeout
- 17:55:57 [ocharles]
- replace the server with a 'sleep 40' server
- 17:56:13 [ocharles]
- the search server regularly times out, see astro logs
- 17:56:30 [ruaok]
- regularly when its "working" or "failing" ?
- 17:56:36 [ocharles]
- hmm, actually less today
- 17:57:07 [ocharles]
- i think the logs have just rotated though, which doesn't help ;)
- 17:57:08 [ocharles]
- :)
- 17:57:52 [ocharles]
- 2012-08-10 17:57:33.610451500 LWP::UserAgent::get('LWP::UserAgent=HASH(0xc3893b8)', 'http://10.1.1.247:776/ws/2/recording/?max=15&type=recording&fmt=xml&offset=0&query=%28recording%3A%28Star~0.7%20AND%2069%20AND%20%5C%28Preziosso%20AND%20RMX%5C%29%29%20recording%3A%28Star~0.7%20AND%2069%29%20%29%20AND%20%28artist%3A%28Darek~0.7%29%20%29%20%20')
- 17:57:53 [ocharles]
- there's one
- 17:58:10 [warp]
- ruaok: is there anything on dora/roobarb themselves which would kill a request after 30 seconds?
- 17:58:12 [ocharles]
- 2012-08-10 17:57:35.485276500 LWP::UserAgent::get('LWP::UserAgent=HASH(0xdd062f8)', 'http://10.1.1.247:776/ws/2/recording/?max=25&type=recording&fmt=xml&offset=0&query=tnum%3A%281%29%20qdur%3A%28117%29%20artist%3A%283%20doors%20down%29%20track%3A%28kryptonite%29%20release%3A%28the%20better%20life%29'
- 17:58:15 [ocharles]
- and another
- 17:58:16 [ruaok]
- next time connect to the offending search server directly, without the load balancer.
- 17:58:23 [ruaok]
- if the response is not immediate, it should fail out.
- 17:58:35 [ocharles]
- also, any particular reason pingu has more weight? that difference of 1 weight seems to give a lot more traffic to pingu
- 17:59:29 [ruaok]
- those weights are set so that the load on each server is about the same.
- 17:59:42 [ruaok]
- they were all hovering around 4, so I would say thats perfect.
- 18:00:06 [ruaok]
- why they have to be different, I have no idea.
- 18:00:28 [ocharles]
- hmm, fair enough
- 18:00:32 [warp]
- ruaok: oh wow, that is finetuning with some scary attention to detail :)
- 18:00:34 [ocharles]
- anyway, reallly reallly off now
- 18:01:24 [ruaok]
- :-)
- 18:02:01 [ruaok]
- shall we go with 3 and 31s and see what happens?
- 18:02:20 [warp]
- ocharles: you messed with lenny config at some point as well, right?
- 18:02:29 [warp]
- ocharles: which commit did you checkout?
- 18:06:50 [warp]
- (oh, he's off :)
- 18:07:11 [warp]
- ruaok: are you testing 3 and 31s ?
- 18:07:19 [ruaok]
- still pondering.
- 18:07:47 [hrglgrmp1]
- hrglgrmp1 has joined #musicbrainz-devel
- 18:10:13 [ruaok]
- I'm in #nginx asking questions there
- 18:10:55 [ruaok]
- so far the nearly 400 people are all asleep. :)
- 18:12:32 [warp]
- ruaok: somewhat common on freenode sadly. when it comes to responding to new people on an irc channel on freenode, I think musicbrainz is actually doing quite a good job :)
- 18:12:42 [ruaok]
- srsly.
- 18:26:37 [hawke_1]
- +1
- 18:26:57 [hawke_1]
- without being a firehose like #ubuntu or #debian or any of the *really* populated channels
- 18:27:31 [ruaok]
- even our customers who balked at getting support in IRC are quite happy.:)
- 18:46:05 [plaintext]
- I tried to ask something in #spotify twice, and I didn't get a reply for 6-8 hours :)
- 18:56:23 [dukeleto]
- dukeleto has joined #musicbrainz-devel
- 18:56:42 [dukeleto]
- dukeleto has joined #musicbrainz-devel
- 19:01:28 [ocharles]
- warp: the git reflog should function as a good audit log
- 19:01:31 [ocharles]
- but we're back at master now
- 19:02:04 [djce]
- djce has joined #musicbrainz-devel
- 19:38:17 [ruaok]
- ruaok has joined #musicbrainz-devel
- 19:52:50 [ianmcorvidae]
- plaintext: you need to load the jquery hashchange plugin -- but I don't know that you need that, that's for the fancy hash-urls for the timeline graph
- 19:53:01 [ianmcorvidae]
- this was from a while ago, but )
- 19:53:47 [plaintext]
- ianmcorvidae: I created a new manifest with just a single line for flot and it works now
- 19:54:04 [plaintext]
- I'll try moving the script to a separate file later
- 19:54:55 [ianmcorvidae]
- ah, yeah
- 20:00:16 [ocharles]
- ruaok: one other thought, we might want to add a timeout on LWP for querying the search server
- 20:00:28 [ocharles]
- so at least we don't start using up all the handlers
- 20:00:51 [ruaok]
- ocharles: there is. its set to 5s. or at least we used to have that.
- 20:01:09 [ruaok]
- if not, it needs to come back.
- 20:01:23 [ruaok]
- djce: ping
- 20:02:26 [ocharles]
- it is not timing out after 5 seconds
- 20:02:36 [ocharles]
- because it's raising SIGALRM (which is our 30 second timeout)
- 20:03:00 [ocharles]
- i think it might be worth adding $c->lwp, so we always use an lwp with sane settings
- 20:05:39 [ruaok]
- and the LWP doesn't have an explicit timeout set?
- 20:05:42 [ruaok]
- it should.
- 20:05:51 [ruaok]
- want a ticket?
- 20:13:44 [ocharles]
- if you want it fixed, yes :)
- 20:13:51 [ocharles]
- i don't think lwp has a timeout set for search
- 20:13:56 [ocharles]
- we set one for wikidocs when that fell over
- 20:14:06 [ruaok]
- it did in classic. :) porting fail
- 20:14:34 [ocharles]
- lack of tests to check we ported correctly fail
- 20:14:38 [ocharles]
- and that was before me so.... blame shifted!
- 20:18:27 [ruaok]
- you know? I swear I saw the search timeout after 5s.
- 20:18:32 [ruaok]
- I've seen timeout errors.
- 20:19:17 [ocharles]
- oh, there is one set to timeout, but one not...
- 20:19:19 [ocharles]
- * ocharles investigates
- 20:19:54 [ruaok]
- https://github.com/metabrainz/musicbrainz-server/blob/master/lib/MusicBrainz/Server/Data/Search.pm#L582
- 20:21:40 [ocharles]
- odd, it looks like the web service should have the 5 second timeout, but that's specifically where I saw 30 second timeouts
- 20:21:40 [ocharles]
- generally needs more investigating anyway, a ticket is still worth having
- 20:22:10 [ocharles]
- yes, that's xml_serach
- 20:22:18 [ocharles]
- notice that 'external_search' creates a $ua but doesn't set a timeout
- 20:22:22 [ocharles]
- oh no
- 20:22:23 [ocharles]
- i lie again
- 20:22:33 [ocharles]
- sigh, friday evening, y u no good for programming?
- 20:22:38 [ruaok]
- so, ticket or no ticket? :)
- 20:22:43 [ocharles]
- ticket
- 20:22:47 [ruaok]
- because its good for pub.
- 20:22:50 [ruaok]
- k
- 20:23:01 [ocharles]
- heh, no pub today, just jogging in this lovely hot indian summer evening
- 20:23:17 [ruaok]
- yeah, I can't wait to go for a ride after work
- 20:26:25 [ruaok]
- http://tickets.musicbrainz.org/browse/MBS-5124
- 20:48:11 [ruaok]
- warp: you still up?
- 21:15:10 [djce]
- ruaok: pong
- 21:26:29 [ruaok]
- * ruaok returns
- 21:26:57 [ruaok]
- djce: roobarb stopped responding to search requests earlier today.
- 21:27:18 [ruaok]
- and everything tipped over because roobarb wasn't failed out by nginx.
- 21:28:02 [ruaok]
- so, we're trying to figure out how we should change the proxy config to make it fail when a search server acts up
- 21:29:21 [ruaok]
- we're considering updating the values for max_fails=10 fail_timeout=10s;
- 21:29:38 [ruaok]
- and I'm also looking at proxy_read_timeout 60;
- 21:30:05 [ruaok]
- but the docs are not very detailed, so its hard to figure out how it behaves
- 21:30:17 [ruaok]
- any inights to the fail over process inside nginx?
- 21:31:39 [djce]
- not really sorry, been thinking that we should have a proper test rig for it though
- 21:31:50 [ruaok]
- * ruaok nods
- 21:32:14 [ruaok]
- do you remember your reasoning for the 10, 10s setings?
- 21:33:04 [djce]
- no, too long ago now
- 21:33:54 [ruaok]
- I gotta see if nginx can give us logging info about this stuff
- 21:35:22 [djce]
- I suspect it can if you whack the log level up high enough
- 21:35:27 [djce]
- it can get pretty chatty
- 21:38:30 [ruaok]
- * ruaok reads nginx dosc
- 21:57:33 [ruaok]
- just taking roobarb down for a moment worked like a charm.
- 21:57:45 [ruaok]
- I need to see if *any* errors were thrown by that. I dont think so.
- 21:58:23 [ruaok]
- which is that is of little to no consequence, I'm inclined to restart the servers once a day.
- 21:58:29 [ruaok]
- jetty that is.
- 21:58:38 [djce]
- 12 hours apart?
- 21:59:02 [djce]
- or close together, both in a quiet time?
- 22:00:02 [ruaok]
- I hadn't considered that.
- 22:00:09 [ruaok]
- I'm certainly open for suggestions.
- 22:00:20 [ruaok]
- an hour apart in quiet time sounds sane
- 22:01:22 [djce]
- nod
- 22:02:23 [ruaok]
- what was the unix cmd to remove a column from some data?
- 22:02:50 [djce]
- cut
- 22:03:11 [djce]
- e.g. if tabs then cut -f1-3,5- to remove column 4
- 22:03:24 [ruaok]
- thk
- 22:03:37 [ruaok]
- I want to nuke the url from the nginx logs
- 22:03:45 [ruaok]
- makes them much easier to look for status codes.
- 22:04:01 [ruaok]
- we do throw a lot of 400 errors because of bad queries.
- 22:04:45 [djce]
- time for me to flake out. tomorrow, more house rewiring. fun fun fun
- 22:04:50 [djce]
- nn all
- 22:04:58 [ruaok]
- nn
- 22:05:02 [ruaok]
- joy
- 22:07:33 [plaintext]
- http://plaintext.mbsandbox.org/logstatistic/top_entities
- 22:08:16 [plaintext]
- this works, but when I change jQuery's hide() and show() to hide("slow")/show("slow") the table loses the black line in the bottom after it's opened/closed once
- 22:08:33 [plaintext]
- weird, but I guess the normal hide() and show() is ok
- 22:09:37 [ruaok]
- hi plaintext
- 22:09:45 [plaintext]
- hi ruaok
- 22:09:47 [ruaok]
- I've got logs copying to the splunk machine now.
- 22:09:55 [ruaok]
- next I need to sanitize them.
- 22:09:59 [plaintext]
- great
- 22:10:07 [ruaok]
- then we need to feed them to splunk.
- 22:10:24 [ruaok]
- is there a command like "splunk add <file>" that I can call?
- 22:10:30 [plaintext]
- I have no idea
- 22:10:40 [plaintext]
- should I search for something like that?
- 22:10:47 [ruaok]
- would you please?
- 22:10:51 [plaintext]
- of course
- 22:11:53 [ruaok]
- that page looks ok to me.
- 22:11:58 [ruaok]
- looks really nice, actually. :)
- 22:12:08 [plaintext]
- cool :)
- 22:12:26 [plaintext]
- table column widths need some adjustment with js probably
- 22:12:42 [ruaok]
- one thing that I see now is that we need to know the date ranges that this data applies to.
- 22:12:54 [plaintext]
- hmm yes
- 22:13:03 [ruaok]
- css is probably a better solution for table column widths.
- 22:13:34 [ruaok]
- and for some reason the graph seems to sit a little lower than the table
- 22:14:03 [plaintext]
- well I was thinking about js, because before we had graphs there were many tables underneath each other and different column widths looked ugly
- 22:14:12 [plaintext]
- yeah, I'll look into that
- 22:14:37 [plaintext]
- http://docs.splunk.com/Documentation/Splunk/latest/Data/MonitorfilesanddirectoriesusingtheCLI
- 22:14:45 [ruaok]
- you can still manage that problem with css.
- 22:14:46 [plaintext]
- the "add oneshot" command looks good here
- 22:14:49 [plaintext]
- really?
- 22:14:52 [ruaok]
- you should prefer to do things in css in most cases.
- 22:14:58 [ruaok]
- leave JS to things that really need it
- 22:15:11 [plaintext]
- hmm
- 22:15:14 [ruaok]
- give the table cells a % width.
- 22:15:34 [ruaok]
- and if each table is the same % wrt to its parent, they should all line up nicely.
- 22:16:28 [plaintext]
- I overthought this
- 22:16:42 [ruaok]
- yep, one shot is what I was looking for.
- 22:16:53 [ruaok]
- plaintext: easy to do. always prefer the simplest solution possible.
- 22:17:13 [ruaok]
- that will make life easier and you can have complexity for the really hard stuff.
- 22:17:18 [plaintext]
- yeah
- 22:17:29 [ruaok]
- my CS teacher once said: "Complexity is a scarce resource"
- 22:17:42 [plaintext]
- that's good :)
- 22:17:42 [ruaok]
- you can only have so much before things are just too complicated.
- 22:17:46 [ruaok]
- :-)
- 22:20:24 [plaintext]
- by the way what I currently do is a bit wasteful
- 22:20:35 [plaintext]
- all the data is loaded through tempalte toolkit
- 22:20:50 [plaintext]
- and just for graphs every report is loaded again via "/logstatistic/json/" + id
- 22:20:59 [plaintext]
- just to get the extra details on how to display stuff
- 22:22:22 [ruaok]
- you should populate the tables from the json data.
- 22:22:30 [plaintext]
- from js?
- 22:23:08 [ruaok]
- yeah.
- 22:23:24 [plaintext]
- okay, I'll work on that tomorrow
- 22:23:27 [ruaok]
- this somewhat contradicts what I just said, ironically enough.
- 22:23:41 [plaintext]
- hehe
- 22:23:42 [ruaok]
- but saving bandwidth vs a little bit of code complexity is the right tradeoff here.
- 22:24:17 [plaintext]
- I haven't done much js/jQuery before, but jQuery feels really nice so far
- 22:25:35 [ruaok]
- js without jquery is a real pain. no fun at all.
- 22:25:43 [ruaok]
- but with it, it starts becoming fun.
- 22:25:59 [plaintext]
- hehe
- 22:28:49 [plaintext]
- this is my js signature: http://pastebin.com/raw.php?i=9nqEqzD5
- 22:30:30 [ruaok]
- signature for what?
- 22:30:36 [plaintext]
- my name
- 22:30:50 [plaintext]
- I was really tired the other day :)
- 22:30:58 [ruaok]
- lol
- 22:57:00 [the_metalgamer]
- the_metalgamer has joined #musicbrainz-devel
- 23:32:51 [ruaok]
- plaintext: still here?
- 23:33:08 [ruaok]
- I think I am about to try feeding splunk a log via the oneshot interface
- 23:33:29 [plaintext]
- ruaok: yeah
- 23:33:40 [plaintext]
- hope it works
- 23:34:33 [ruaok]
- I'm wondering how it will know what it import the data as.
- 23:35:23 [plaintext]
- you can specify a sourcetype if that's what you mean
- 23:36:16 [plaintext]
- and I saved an "nginx_log" source type when I was adding logs
- 23:36:31 [ruaok]
- yes, that would be perfect.
- 23:43:07 [ruaok]
- hmm. permissions problems.
- 23:43:20 [ruaok]
- I need to leave now. I'll pick this up when I get home.
- 23:43:25 [plaintext]
- okay
- 23:43:32 [ruaok]
- when you wake up, look for new data. :)
- 23:43:44 [plaintext]
- yay :)
- 23:43:54 [plaintext]
- will you erase the old ones?
- 23:43:55 [ruaok]
- <poof>
- 23:44:07 [plaintext]
- I guess I'll see :P
- 23:45:23 [demosdemon]
- heh, he's a mysterious one