2006-08-14T09:34:31 <xorAxAx> moin
2006-08-14T09:51:05 <Kepplar> moin
2006-08-14T10:01:15 <ThomasWaldmann> moin
2006-08-14T10:42:29 <ffesti> moin
2006-08-14T10:44:53 <ThomasWaldmann> Kepplar: http://storage.wikiwikiweb.de/
2006-08-14T10:45:16 <Kepplar> are error has changed
2006-08-14T10:45:16 <Kepplar> ok
2006-08-14T10:45:20 <Kepplar> ah*
2006-08-14T10:45:43 <Kepplar> ill trawl irc logs a little later for xorAxAx's rune
2006-08-14T10:47:38 <xorAxAx> rune?
2006-08-14T10:48:49 <Kepplar> 1-liner of doom lambda
2006-08-14T10:49:01 <Kepplar> work can be used for unix-1-liners too
2006-08-14T10:49:05 <Kepplar> word*
2006-08-14T10:51:55 <ThomasWaldmann> Kepplar: please put rsplit into MoinMoin/support/string.py and put some comment above it "remove when we require py 2.4"
2006-08-14T10:52:24 <Kepplar> ok
2006-08-14T10:52:30 <ThomasWaldmann> or rather "String.py"
2006-08-14T10:53:00 <ThomasWaldmann> so it doesnt get confused in case we import string somewhere there
2006-08-14T10:54:33 <xorAxAx> yeah, dont call it string
2006-08-14T11:01:53 <xorAxAx> so, nobody here? :)
2006-08-14T11:01:59 <ThomasWaldmann> ok, welcome to your weekly meeting
2006-08-14T11:02:10 <ThomasWaldmann> Kepplar: your time :)
2006-08-14T11:02:45 <Kepplar> Oh
2006-08-14T11:02:49 <Kepplar> i thought it was in an hours time
2006-08-14T11:03:09 <Kepplar> Well, integrations been full stream ahead
2006-08-14T11:03:13 <ThomasWaldmann> it is 9 UTC :)
2006-08-14T11:03:26 <Kepplar> What I can tell from Tests, Page is working at least reading
2006-08-14T11:03:40 <Kepplar> writing I'm not sure as i think tests are stalling before then.. on ACL and user
2006-08-14T11:03:44 <Kepplar> which is what ive mid-way through solving
2006-08-14T11:03:58 <xorAxAx> if you see me writing weird characters, its because of the high load on this server
2006-08-14T11:04:07 <xorAxAx> load avg 12 currently
2006-08-14T11:04:10 <Kepplar> Blockers.. not really any, although I've underestimated the integration
2006-08-14T11:04:27 <Kepplar> because storage changes are needed in a lot more files than I previously thought
2006-08-14T11:04:59 <Kepplar> but theorectically read/write is the same code it should work i think
2006-08-14T11:05:04 <ThomasWaldmann> it is clear to you that you don't need to change all at once?
2006-08-14T11:05:10 <Kepplar> yes
2006-08-14T11:05:32 <Kepplar> although I have to do it on some things because its tightly coupled
2006-08-14T11:06:02 <Kepplar> wait..
2006-08-14T11:06:06 <ThomasWaldmann> did you implement the pseudo rootpage?
2006-08-14T11:06:07 <Kepplar> acl i fixed on sat actually
2006-08-14T11:06:17 <Kepplar> yea, but dont think i needed it in the end
2006-08-14T11:06:38 <ThomasWaldmann> but?
2006-08-14T11:06:39 <Kepplar> so its just user and *maybe* page write.. oh and packages/attachments
2006-08-14T11:07:49 <Kepplar> wikidict is throwing up some errors too
2006-08-14T11:08:31 <Kepplar> Rootpage is just page with a blank reset
2006-08-14T11:08:41 <Kepplar> eg, it doesn't attempt to retrieve the revision
2006-08-14T11:09:32 <ThomasWaldmann> how do you get the pagelist?
2006-08-14T11:10:07 <Kepplar> sem has a new method this week to get list of all items (and you can specify the type) from a particular engine
2006-08-14T11:10:31 <Kepplar> this then iterates through all engines if not specified, or calls a getpage function on siapi
2006-08-14T11:10:34 <Kepplar> er
2006-08-14T11:10:35 <Kepplar> get item
2006-08-14T11:10:43 <Kepplar> which then works out what to look for
2006-08-14T11:10:52 <Kepplar> and returns a list of WikiNames
2006-08-14T11:11:05 <Kepplar> same code is used for userlist
2006-08-14T11:11:10 <Kepplar> except useritems are used not pageitems
2006-08-14T11:11:31 <Kepplar> also simiar code on item object is used for revision lists
2006-08-14T11:13:22 <ThomasWaldmann> 410 if item_revision_type is not "core/page":
2006-08-14T11:13:36 <Kepplar> ok
2006-08-14T11:13:43 <Kepplar> i know what your gonna say :)
2006-08-14T11:13:48 <Kepplar> thanks for that
2006-08-14T11:15:22 <ThomasWaldmann> you are using py 2.4?
2006-08-14T11:15:31 <Kepplar> yep, i know, i will test against 2.3 :)
2006-08-14T11:15:51 <ThomasWaldmann> +if item_revision_type = "core/user":
2006-08-14T11:16:04 <ThomasWaldmann> you are quite creative on that stuff :)
2006-08-14T11:16:15 <Kepplar> :S
2006-08-14T11:16:19 <Kepplar> thats always true
2006-08-14T11:16:40 <Kepplar> sorry - code quality has dived as ive been in a rush for the deadlien
2006-08-14T11:16:52 <ThomasWaldmann> + if re.compile(r'^\d+\.\d+(\.\d+)?$').match(file_name):
2006-08-14T11:17:05 <Kepplar> what about it?
2006-08-14T11:17:20 <ThomasWaldmann> i guess this works, but I dont know if compiling really helps here (if you just use the compiled RE once only)
2006-08-14T11:17:28 <Kepplar> ah i see ok
2006-08-14T11:17:48 <ThomasWaldmann> + if isinstance(file_name, type(u'')):
2006-08-14T11:17:59 <ThomasWaldmann> isinstance(file_name, unicode)
2006-08-14T11:18:34 <ThomasWaldmann> or isinstance(x, str) or isinstance(L, list)
2006-08-14T11:19:12 <ThomasWaldmann> + if (len(group) % 2 != 0):
2006-08-14T11:19:30 <ThomasWaldmann> you dont need the outer brackets
2006-08-14T11:19:40 <fpletz> moin.. I'm sorry for being late. I'll also have to go in about 10 minutes to attend the funeral of my grandmother :/
2006-08-14T11:19:43 <fpletz> now writing my progress report..
2006-08-14T11:19:52 <ThomasWaldmann> (you don't even need the != 0, but that's a matter of taste maybe)
2006-08-14T11:20:31 <ThomasWaldmann> fpletz: oh, bad news. I am sorry with you.
2006-08-14T11:21:00 <ThomasWaldmann> fpletz: if you want to post now, kepplar's time is over anyway
2006-08-14T11:21:18 <fpletz> just a moment
2006-08-14T11:21:54 <xorAxAx> LAST: Continued work on the merge logic, finished prefix handling. Added local conflict detection and logging support in SyncPages. Refactored conflict flag detection from the edit action into the PageEditor class. Enhanced XMLRPC server in Moin to allow XMLRPC functions to return Fault instances. Introduced a new diff3 mode that should reduce the conflicts. Fixed hard to track down bugs in SyncPages. Store full IWIDs and match by either o
2006-08-14T11:22:00 <xorAxAx> ... components when searching for tags. Ignore underlay pages in sync. Filter the remote page list by the prefix and the pageList on the remote side. Finished the direction==DOWN mode. Ergo: SyncPages is working now in BOTH and DOWN mode for pages that exist on both sides.
2006-08-14T11:22:04 <xorAxAx> NEXT: Integrate the other modes that are history sensitive and check for e.g. deleted pages. Start using MultiCalls to parallelise the sync process.
2006-08-14T11:22:07 <xorAxAx> BLOCKERS: -
2006-08-14T11:22:09 <xorAxAx> thats me :)
2006-08-14T11:22:54 <ThomasWaldmann> xorAxAx: does the normal pageeditor also use that diff3 enhancement?
2006-08-14T11:23:08 <ThomasWaldmann> it also often should sub-optimal conflict
2006-08-14T11:23:17 <ThomasWaldmann> s/should/showed/ %)
2006-08-14T11:23:20 <xorAxAx> no, one needs to pass an 2 instead of a 1
2006-08-14T11:23:37 <xorAxAx> ThomasWaldmann: if you ever see an incorrect merge, then please store your version
2006-08-14T11:23:41 <xorAxAx> as i already said earlier
2006-08-14T11:23:59 <xorAxAx> so one can deduce a testcase from wiki history and the manually saved text
2006-08-14T11:24:21 <ThomasWaldmann> ok
2006-08-14T11:25:05 <xorAxAx> but the optimisation only affects pages that are similar in larger parts and i am not sure if it is correct
2006-08-14T11:25:15 <xorAxAx> so its not enabled for pageeditor yet
2006-08-14T11:25:26 <ThomasWaldmann> Kepplar: one last thing:
2006-08-14T11:25:52 <ThomasWaldmann> adding a second @copyright: line does not work
2006-08-14T11:26:03 <Kepplar> ah ok
2006-08-14T11:27:21 <ThomasWaldmann> xorAxAx: do you already transfer metadata as edit comments?
2006-08-14T11:27:54 <xorAxAx> ThomasWaldmann: no, and thats not planned
2006-08-14T11:28:03 <xorAxAx> i.e. it could be added into the comments
2006-08-14T11:28:14 <xorAxAx> but the system does not transfer item revisions, only items
2006-08-14T11:28:16 <fpletz> sorry for interrupting but I'll have to post my lines now
2006-08-14T11:28:25 <xorAxAx> so the next meta data thing is transferring the mime type
2006-08-14T11:28:48 <xorAxAx> that could be actually done and makes even sense :)
2006-08-14T11:28:49 <fpletz> LAST: a bunch of bugfixes (i.e. those littles glitches thomas and richard identified); added some more timers for regression testing; improved highlighting code to work better with stemming and special searches; extended SystemInfo macro; use xapian for sorting (yet to be improved) to get better performance
2006-08-14T11:28:54 <fpletz> NEXT: more performance tweaks (i.e. only get the results displayed on page, problem: acls?); didn't get around to do the advanced ui stuff this weekend as planned, but will be done this week
2006-08-14T11:28:59 <fpletz> BLOCKERS: -
2006-08-14T11:29:37 <xorAxAx> ThomasWaldmann: because this was one of the design goal - not to implement a DVCS
2006-08-14T11:29:42 <ThomasWaldmann> fpletz: see xapian.wikiwikiweb.de for some small fix requests
2006-08-14T11:29:59 <fpletz> ThomasWaldmann: will do, thanks
2006-08-14T11:30:51 <fpletz> hmm
2006-08-14T11:31:25 <fpletz> ok, I will take a look at this probably this evening
2006-08-14T11:31:40 <ThomasWaldmann> ok, thanks fpletz
2006-08-14T11:31:42 <fpletz> any other tings?
2006-08-14T11:32:10 <ThomasWaldmann> no, it worked quite ok in my tests
2006-08-14T11:32:26 <ThomasWaldmann> maybe poll google about an answer
2006-08-14T11:32:56 <Kepplar> brb breakfast
2006-08-14T11:32:59 <fpletz> checking mails..
2006-08-14T11:33:44 <fpletz> ok..
2006-08-14T11:33:49 <fpletz> too bad
2006-08-14T11:33:53 <fpletz> We ask that they not use "Moin" in similar type/color/style as "Google" at the bottom of search pages and ask them not to copy our look and feel for the search results page.
2006-08-14T11:34:27 <ThomasWaldmann> xorAxAx: if you tune the stuff with multicall, keep server load in mind and have some means for the server to get less loaded
2006-08-14T11:34:40 <fpletz> ok, really have to go now.. I'll get in touch with you later today
2006-08-14T11:34:48 <Kepplar> fpletz: you'll just have to come up with something better than googles ;)
2006-08-14T11:34:49 <xorAxAx> ThomasWaldmann: yeah ... what kind of means are you thinking about?
2006-08-14T11:35:26 <xorAxAx> the largest load is currently yielded by the pagelisting
2006-08-14T11:35:36 <xorAxAx> which is only done once
2006-08-14T11:36:08 <xorAxAx> we will see if some kind of throtteling is necessary
2006-08-14T11:36:10 <xorAxAx> i dont think so :)
2006-08-14T11:36:57 <ThomasWaldmann> the problem is not a single syncer, but maybe rather if that gets very "popular"
2006-08-14T11:37:03 <xorAxAx> hehe
2006-08-14T11:37:10 <xorAxAx> yeah, i havent done any benchmarks
2006-08-14T11:37:43 <xorAxAx> at least i think that the highest load is yielded on the server that is managing the sync i.e. running the action
2006-08-14T11:37:55 <ThomasWaldmann> do you do some timestamp exchange, so it can say "nothing changed" without doing much (like 304)
2006-08-14T11:39:24 <xorAxAx> hmm, yeah
2006-08-14T11:39:27 <ThomasWaldmann> (a global timestamp, not per page)
2006-08-14T11:39:44 <xorAxAx> ok, added to the todo list
2006-08-14T11:40:22 * ThomasWaldmann implemented 304 that for rss_rc and AttachFile recently :)
2006-08-14T11:42:50 <ThomasWaldmann> ok, I think we are through. Happy hacking! :)
2006-08-14T13:04:11 * ThomasWaldmann merges xapian -> main
2006-08-14T14:11:48 * ThomasWaldmann plays with pyinotify
2006-08-14T15:10:40 <ThomasWaldmann> hmm, i have another idea: we change global edit-log on every content / namespace change we do
2006-08-14T15:10:58 <ThomasWaldmann> so we have to only monitor that file
2006-08-14T15:11:57 <ThomasWaldmann> if it didn't change, all cached page metadata and data should be correct
2006-08-14T15:13:31 <ThomasWaldmann> there is a small timespan of incorrectness, though: the time between page update and log update
2006-08-14T15:15:15 <xorAxAx> which caches do you want to wire to that trigger?
2006-08-14T15:15:37 <xorAxAx> depending on the answer to that question, you can imagine if an unnoticed update might be problematic
2006-08-14T15:15:50 <ThomasWaldmann> it's just a vague idea, but pagelist and page metadata / page exists stuff
2006-08-14T15:16:05 <xorAxAx> i.e. if you would cache the last revision number of a file before writing it to disk using this trigger, it would be fatal
2006-08-14T15:16:23 <xorAxAx> but as long its ACL, page contents etc. its ok
2006-08-14T15:16:30 <xorAxAx> s/a file/a page/
2006-08-14T15:17:03 <ThomasWaldmann> yeah, when creating new revs, we better lock and directly read from disk
2006-08-14T15:17:47 <xorAxAx> yes, for example
2006-08-14T15:17:59 <xorAxAx> note that this idea doesnt need inotify - you can regularly poll this file as well
2006-08-14T15:18:08 <ThomasWaldmann> sure
2006-08-14T15:18:19 <xorAxAx> which should not be much more slower because its in the OS' cache
2006-08-14T15:18:21 <ThomasWaldmann> one stat per request doesnt matter
2006-08-14T15:38:46 <ThomasWaldmann> i have put those ideas in the wiki. the save even succeeded after a minute.
2006-08-14T15:49:07 <ThomasWaldmann> http://moinmoin.wikiwikiweb.de/ReducingIO
2006-08-14T16:17:57 * xorAxAx added a short comment
2006-08-14T17:05:47 <ThomasWaldmann> fpletz: if you start coding, please notify me and look at http://xapian.wikiwikiweb.de/TestWiki
2006-08-14T19:22:54 <ThomasWaldmann> xorAxAx: sounds good, I'll look at this today evening
2006-08-14T22:44:14 <xorAxAx> ThomasWaldmann: it is not possible to have this sync protocol on that metadict without major complications
2006-08-14T22:44:52 <xorAxAx> because its obvious that two processes having instantiated a metadict and doing changes on them have race conditions that overwrite the changes done by the other process (or even request)
2006-08-14T22:44:57 <xorAxAx> i am currently stripping it down
2006-08-14T22:50:32 <xorAxAx> and fixing that bug ...
MoinMoin: MoinMoinChat/Logs/moin-dev/2006-08-14 (last edited 2007-10-29 19:08:31 by localhost)
