2007-08-08T00:13:42 <DrFrasierCrane> omg
2007-08-08T00:14:08 <nwp> now that's just rude. get us all interested and then quit...
2007-08-08T00:17:31 <grzywacz> ;-)
2007-08-08T00:22:54 * ThomasWaldmann .oO(famous last words)
2007-08-08T00:24:13 <nwp> oh, he is there :-)
2007-08-08T00:26:36 <ThomasWaldmann> http://moinmo.in/LinkMarkupCollection maybe all check if i forgot your favourite broken 1.5 link syntax :)
2007-08-08T00:32:16 <nwp> ThomasWaldmann: ok, I see the wsgi server_* stuff doesn't provide any config stuff
2007-08-08T00:32:45 <nwp> and I'm not 100% clear what config stuff is potentially dealt with in there (would be good to have an exhaustive list somewhere)
2007-08-08T00:33:52 <nwp> so I'm not entirely clear how the wsgi one sets up logging
2007-08-08T00:33:58 <nwp> if it does
2007-08-08T00:35:37 * ThomasWaldmann looks
2007-08-08T00:37:39 <ThomasWaldmann> nwp: yeah, you are right. i have to look at the wsgi stuff. maybe look at another one for examples.
2007-08-08T00:38:41 <nwp> ok, what is the aim then - config to do with getting the thing running is dealt with there, and what it does once it's running in the MoinMoin/config stuff? or should both be merged together somehow at some point?
2007-08-08T00:39:08 <ThomasWaldmann> that Config has nothing to do with wikiconfig
2007-08-08T00:39:21 <nwp> right
2007-08-08T00:39:46 <ThomasWaldmann> it is just to have a place for defaults for the adaptor script
2007-08-08T00:39:57 <ThomasWaldmann> (and init logging etc.)
2007-08-08T00:40:01 <nwp> thought so, just checking
2007-08-08T00:40:37 <ThomasWaldmann> e.g. server_cgi is very easy
2007-08-08T00:40:42 <nwp> yes
2007-08-08T00:41:17 <nwp> I can't think of anything beyond logging that modpython needs at the moment either
2007-08-08T00:42:05 <ThomasWaldmann> debug/profiling options?
2007-08-08T00:42:52 <nwp> it did occur to me while looking at it that it might be nice to do it in the apache config (in the way that _setup_vars_from_std_env does) and read it from there rather than editing these files, but that might overcomplicate things unnecessarily
2007-08-08T00:42:55 <nwp> I guess
2007-08-08T00:43:03 <nwp> not familiar with any profiling for python
2007-08-08T00:43:39 <ThomasWaldmann> well, editing the apache config only works for apache :)
2007-08-08T00:44:13 <nwp> yes, but using the environment would work for cgi, fcgi, wsgi & modpy, wouldn't it?
2007-08-08T00:44:35 <nwp> anyway, I thought it was a bad idea to think too much about that right now ;-)
2007-08-08T00:45:05 <ThomasWaldmann> btw, does moin under modpy also need that properties stuff? this is also in Config.
2007-08-08T00:45:13 <nwp> was looking at that
2007-08-08T00:45:29 <nwp> it could easily be added, but again I wasn't clear what it would be used for
2007-08-08T00:46:08 <ThomasWaldmann> maybe just copy server_cgi to server_mod_python and just edit it
2007-08-08T00:46:18 <nwp> that's what I was doing :-)
2007-08-08T00:47:00 <nwp> and I have something that should at least work, but can't resist trying to tidy it up and make it like the others at the same time
2007-08-08T00:47:52 <grzywacz> argh
2007-08-08T00:49:42 <grzywacz> stupid conflict, I hope I won't mess it up...
2007-08-08T00:53:49 <CIA-28> moin: Karol 'grzywacz' Nowak <grzywacz@sul.uni.lodz.pl> * 2639:11447cada25e 1.7-jabber-knowak/jabberbot/xmlrpcbot.py: Fix layering violation in xmlrpcbot code, as mentioned in TODO.
2007-08-08T00:53:50 <CIA-28> moin: Karol 'grzywacz' Nowak <grzywacz@sul.uni.lodz.pl> * 2640:0608417246ee 1.7-jabber-knowak/jabberbot/xmppbot.py: Use a more generic way to store information about supported extensions.
2007-08-08T00:53:51 <CIA-28> moin: Karol 'grzywacz' Nowak <grzywacz@sul.uni.lodz.pl> * 2641:9efa3194636d 1.7-jabber-knowak/jabberbot/xmppbot.py: Pass URLs using OOB only if it's known to be supported by the client.
2007-08-08T00:53:52 <CIA-28> moin: Karol 'grzywacz' Nowak <grzywacz@sul.uni.lodz.pl> * 2642:82b0e4625f90 1.7-jabber-knowak/MoinMoin/script/migration/_tests/test_conv160_wiki.py: Whitespace fix.
2007-08-08T00:53:53 <CIA-28> moin: Karol 'grzywacz' Nowak <grzywacz@sul.uni.lodz.pl> * 2643:d3503c3c2ba4 1.7-jabber-knowak/MoinMoin/xmlrpc/__init__.py: More whitespace fixes.
2007-08-08T00:53:56 <CIA-28> moin: Karol 'grzywacz' Nowak <grzywacz@sul.uni.lodz.pl> * 2644:459199b59cf7 1.7-jabber-knowak/ (3 files in 3 dirs): Merge changes done in my other branches.
2007-08-08T01:09:01 * ThomasWaldmann looked at creole link markup.
2007-08-08T01:17:03 <ThomasWaldmann> looks like creole link markup could be the easiest - if we accept a radical change in link syntax (and macro syntax)
2007-08-08T01:31:08 <ThomasWaldmann> nobody shocked, crying wtf or whatever? :D
2007-08-08T01:36:47 <nwp> busy
2007-08-08T01:37:28 <nwp> I think if you make significant changes to markup then it needs to become pluggable to allow people with hard-of-learning users to stick with the traditional scheme
2007-08-08T01:37:57 <nwp> but for myself, so long as it can be automatically converted-to from the current system, I don't care too much
2007-08-08T01:41:06 <ThomasWaldmann> the problem is that moin's link syntax is ... grown (not to say fscking complex)
2007-08-08T01:42:18 <ThomasWaldmann> to remove at least some of its strange workarounds I will have to at least introduce quoting for stuff with spaces plus free links with link text
2007-08-08T01:43:23 <ThomasWaldmann> but if I have to convert much of the link stuff anyway, i could also convert to [[link|text]] syntax
2007-08-08T01:44:40 <ThomasWaldmann> solves the same problems, make creole fans happy, makes mediawiki users compatible for internal links
2007-08-08T01:47:24 <starshine> what do dreole link markup look like?
2007-08-08T01:47:56 <starshine> is it easier than mw?
2007-08-08T01:49:09 <starshine> how well does it handle links such as http://some.site.here.org/cgi.pl?foo=bar&thing=whatsit&ouch=bandaid;more.options.here ?
2007-08-08T01:50:05 <starshine> that you'd like to replace with some helpfully plain text like _my mom's first aid kit_
2007-08-08T01:51:10 <starshine> ah it's same as mw for a specific sort of link. ok that might be nice
2007-08-08T01:53:27 <ThomasWaldmann> it is a bit easier than mw as they use double brackets for external links also
2007-08-08T01:53:41 <starshine> ok consistent sounds good
2007-08-08T01:54:13 <ThomasWaldmann> i am currently thinking about what to do with macro syntax.
2007-08-08T01:56:50 <ThomasWaldmann> creole uses <<...>> for plugins
2007-08-08T02:01:12 <ThomasWaldmann> or <<<..>>>, seems not to be decided yet
2007-08-08T02:05:09 <starshine> do you need to decide subclass of macros for anything?
2007-08-08T02:05:27 <starshine> right now Macro resembles a link with programming options.
2007-08-08T02:05:49 <ThomasWaldmann> we dont use [[ for links yet
2007-08-08T02:06:18 <ThomasWaldmann> but we could, if we choose sth else for macros
2007-08-08T02:07:02 <ThomasWaldmann> instead of [[Macro(p,q,r)]] it could be <<<Macro p,q,r>>>
2007-08-08T02:09:09 <ThomasWaldmann> same amount of keystrokes :)
2007-08-08T02:15:20 * ThomasWaldmann looks at TheSheep's creole parser
2007-08-08T02:26:33 <starshine> if I make moin cookies for linux picnic how do you want me to check them in?
2007-08-08T02:26:40 <starshine> ooh I know
2007-08-08T02:26:58 <starshine> I could make some, and take a photo of one, and submit it as an alternate corner-logo.
2007-08-08T02:27:23 <starshine> they would probably be my most complicated moin logo.
2007-08-08T02:28:03 <starshine> .o( advatage is, they'd be round.
2007-08-08T02:30:28 <ThomasWaldmann> hehe
2007-08-08T04:00:31 <CIA-28> moin: Thomas Waldmann <tw AT waldmann-edv DOT de> * 2612:e3e5e2373c60 1.7/MoinMoin/script/migration/ (_conv160_wiki.py _tests/test_conv160_wiki.py): 1.6 converter: cleanup link conversion, cleanup/add tests
2007-08-08T04:00:34 <CIA-28> moin: Thomas Waldmann <tw AT waldmann-edv DOT de> * 2613:ddbbf05d0eac 1.7/MoinMoin/parser/text_creole.py: add creole 0.5 parser (thanks to Radomir Dopieralski)
2007-08-08T06:17:20 <CIA-28> moin: Thomas Waldmann <tw AT waldmann-edv DOT de> * 2614:9a90c11c6c0d 1.7/MoinMoin/parser/text_creole.py: update creole parser to support creole 1.0
2007-08-08T10:37:18 <CIA-28> moin: Heinrich Wendel <h_wendel@cojobo.net> * 2806:f4fb8cb9db2c 1.7-storage-hwendel/MoinMoin/ (19 files in 10 dirs): implement global edit-log and news()
2007-08-08T10:37:47 <ThomasWaldmann> moin
2007-08-08T10:37:51 <lanius> moin
2007-08-08T10:38:44 <ThomasWaldmann> good timings :)
2007-08-08T10:38:48 <lanius> hehe
2007-08-08T10:41:18 <ThomasWaldmann> ehrm, eventlog: if the param is called mtime_usecs, it must be usecs. otherwise it have to be mtime
2007-08-08T10:46:09 <lanius> usecs means what?
2007-08-08T10:46:31 <ThomasWaldmann> micro seconds
2007-08-08T10:47:12 <lanius> ah, ok
2007-08-08T10:47:39 <lanius> i changed it everywhere
2007-08-08T10:47:47 <lanius> so i will rename it evyerwhere
2007-08-08T10:47:48 <ThomasWaldmann> u is usually taken if your kbd misses the greek "mue"
2007-08-08T10:48:16 <lanius> i would take the m because that's where the mue is located on my keyboard :-)
2007-08-08T10:48:46 <xorAxAx> msec is milli second
2007-08-08T10:49:03 <xorAxAx> milli != micro
2007-08-08T10:49:29 <lanius> yes, but still the mue is on the m
2007-08-08T10:49:49 <xorAxAx> thats no reason to call it msec :)
2007-08-08T10:50:26 <ThomasWaldmann> GlobalEditLog.next: item_name, rev, ed_time = self.items[self.pos]
2007-08-08T10:50:42 <ThomasWaldmann> just to avoid using indices and for clarity
2007-08-08T10:51:11 <ThomasWaldmann> + self.pos = self.pos + 1
2007-08-08T10:51:18 <ThomasWaldmann> += exists
2007-08-08T10:53:26 <ThomasWaldmann> wikiutil.timestamp2version(bookmark_usecs) this is also wrong or strangely named
2007-08-08T10:54:10 <ThomasWaldmann> and btw, you can drop that long() stuff in favour for int()
2007-08-08T10:54:21 <ThomasWaldmann> we dont support py 2.2 any more
2007-08-08T10:54:45 <lanius> somewhere in the comments it's mentioned that 2.3 needs it as well
2007-08-08T10:57:12 <lanius> maybe not
2007-08-08T10:58:34 <ThomasWaldmann> lanius: + items.sort(key=lambda x: str(x[2]) + str(x[1]) + x[0], reverse=True)
2007-08-08T10:58:44 <ThomasWaldmann> please explain
2007-08-08T10:59:07 <lanius> it's first sorted by mtime, then by rev and then by the name, everything reversed
2007-08-08T11:01:44 <ThomasWaldmann> >>> "42" > "2323"
2007-08-08T11:01:44 <ThomasWaldmann> True
2007-08-08T11:03:13 <ThomasWaldmann> guess why i used %08d for the revstr :)
2007-08-08T11:03:33 <xorAxAx> ugh, ugly sort
2007-08-08T11:03:47 <ThomasWaldmann> but you can just use int there
2007-08-08T11:04:25 <ThomasWaldmann> and if you can reverse order in that tuple, it will naturally sort right
2007-08-08T11:05:44 <ThomasWaldmann> + mtime = os.path.getmtime(self._get_item_path(item)) wtf?
2007-08-08T11:07:26 <ThomasWaldmann> (i was quite glad about removing mtime stuff a few years ago when i refactored storage)
2007-08-08T11:10:22 <xorAxAx> yep, esp. a windows user should be aware of the problems :)
2007-08-08T11:10:35 <ThomasWaldmann> + TODO: news is too slow currently < i guess it will even get slower now
2007-08-08T11:11:03 <ThomasWaldmann> yeah, M$ fucked it up completely :D
2007-08-08T11:11:16 <xorAxAx> only on fat and with legacy apps, though :)
2007-08-08T11:13:34 <ThomasWaldmann> but you dont need win32 to break mtimes, just using (s)cp for backups is enough
2007-08-08T11:14:15 <lanius> the news implementation has to be changed, it is just a "proof of concept" for the interface
2007-08-08T11:15:08 <ThomasWaldmann> well, proof it correctly, please :)
2007-08-08T11:18:21 <ThomasWaldmann> and please try it for 1000 and 10000 items
2007-08-08T11:20:34 <ThomasWaldmann> + if not os.path.exists(self._get_item_path(item, "revisions", _get_rev_string(revno))):
2007-08-08T11:20:37 <ThomasWaldmann> + continue
2007-08-08T11:20:40 <ThomasWaldmann> + mtime = os.path.getmtime(self._get_item_path(item, "revisions", _get_rev_string(revno)))
2007-08-08T11:20:43 <ThomasWaldmann> revno?
2007-08-08T11:21:01 <ThomasWaldmann> try/except would be much faster
2007-08-08T11:21:33 <lanius> ok, what about revno?
2007-08-08T11:21:41 <ThomasWaldmann> undefined
2007-08-08T11:21:55 <ThomasWaldmann> ehrm, no
2007-08-08T11:23:54 <ThomasWaldmann> you could break out that loop once you get a older rev than timestamp
2007-08-08T11:24:42 <lanius> yes
2007-08-08T11:24:45 <lanius> no
2007-08-08T11:25:04 <lanius> in theory you can edit rev 1 after rev2
2007-08-08T11:25:04 <ThomasWaldmann> :)
2007-08-08T11:26:01 <lanius> since the implementation will change completly, it doesn't matter
2007-08-08T11:26:49 <ThomasWaldmann> how will it change completely?
2007-08-08T11:27:07 <lanius> based on a cache
2007-08-08T11:27:48 <lanius> for a database it will be a one line sql query
2007-08-08T11:27:52 <lanius> should be quite fast
2007-08-08T11:28:15 <ThomasWaldmann> yeah, but the fs stuff is harder
2007-08-08T11:28:19 <lanius> yes
2007-08-08T11:28:32 <lanius> i'm looking into using the global edit log file again
2007-08-08T11:29:04 <ThomasWaldmann> you know it breaks on nfs
2007-08-08T11:29:19 <lanius> that means that the current 1.6 implementaiton is broken as well?
2007-08-08T11:29:36 <ThomasWaldmann> yes, for broken filesystems
2007-08-08T11:29:45 <lanius> so it's no issue ;)
2007-08-08T11:30:29 <xorAxAx> hmm?
2007-08-08T11:30:39 <ThomasWaldmann> but we wanted to get rid of that "global edit-log as a file"
2007-08-08T11:30:56 <lanius> any other idea?
2007-08-08T11:30:57 <xorAxAx> yes, lanius wanted to do so
2007-08-08T11:31:07 <xorAxAx> using news(), its not necessary
2007-08-08T11:31:13 <lanius> i want to get rid of all this filesystem stuff ;)
2007-08-08T11:31:28 <ThomasWaldmann> that's no option
2007-08-08T11:31:34 <lanius> just kidding
2007-08-08T11:31:34 <xorAxAx> ThomasWaldmann: hmm?
2007-08-08T11:31:44 <xorAxAx> and what is broken on nfs?
2007-08-08T11:31:48 <lanius> xorAxAx: any idea for a fast implementation of news()
2007-08-08T11:31:59 <ThomasWaldmann> append mode is broken on nfs
2007-08-08T11:32:03 <xorAxAx> ThomasWaldmann: indeed
2007-08-08T11:32:04 <DrFrasierCrane> guys who made this change: http://tools.assembla.com/newton/wiki/ArchitecturePlan?action=diff&version=6
2007-08-08T11:32:21 <xorAxAx> lanius: well, it gets a global revision or timestamp
2007-08-08T11:32:37 <grzywacz> moin
2007-08-08T11:32:38 <xorAxAx> lanius: the file backend writes a file with append and does a binary search to find the position
2007-08-08T11:32:53 <lanius> global revision or timestamp means what?
2007-08-08T11:33:41 <xorAxAx> lanius: either you have some counter that is incremented for every change in the backend or you just store a timestamp that is increasing automatically
2007-08-08T11:34:00 <xorAxAx> i think its the third time that i am explaining this idea in here
2007-08-08T11:34:30 <xorAxAx> (would be nice if you paid a bit more attention or make notes etc.)
2007-08-08T11:34:39 <lanius> i pay enough attentions
2007-08-08T11:34:41 <lanius> it's the details
2007-08-08T11:34:52 <lanius> what is this counter for?
2007-08-08T11:34:53 <DrFrasierCrane> I guess it was a ghost :-) hmm
2007-08-08T11:34:53 <xorAxAx> mindmapping helps :)
2007-08-08T11:35:01 <DrFrasierCrane> thanks anyway whoever it was
2007-08-08T11:35:08 <lanius> (you never talked about a counter before)
2007-08-08T11:35:10 <xorAxAx> lanius: the idea is that you need some incrementing value to do a binary search
2007-08-08T11:35:19 <xorAxAx> lanius: i talked about revision numbers or timestamps
2007-08-08T11:35:30 <xorAxAx> one idea was to use an item in the backend with meta-data revs only
2007-08-08T11:35:37 <xorAxAx> so the rev no would be incrementing automatically
2007-08-08T11:35:51 <lanius> which is not fast at all
2007-08-08T11:36:34 <ThomasWaldmann> it's quite faster than what you have now and it doesnt need additionaly code for a basic impl.
2007-08-08T11:36:50 <xorAxAx> lanius: why not?
2007-08-08T11:37:03 <lanius> xorAxAx: n file access for n revisions, just like now
2007-08-08T11:37:15 <lanius> ThomasWaldmann: no additionally code? so it is done?
2007-08-08T11:37:31 <ThomasWaldmann> you already have revisioned item metadata
2007-08-08T11:37:41 <xorAxAx> lanius: hmm, no
2007-08-08T11:38:00 <xorAxAx> lanius: currently you need to query n items while in that system, you would query m revs
2007-08-08T11:38:07 <xorAxAx> and m is less than n
2007-08-08T11:38:16 <lanius> m is more then n
2007-08-08T11:38:20 <xorAxAx> no
2007-08-08T11:38:23 <lanius> each item has 1 or more revision
2007-08-08T11:38:27 <ThomasWaldmann> you just need to put it into one place (e.g. the root item)
2007-08-08T11:38:37 <xorAxAx> n is 1000 pages but only lets say 5 changes are new
2007-08-08T11:38:41 <xorAxAx> so 5 < 1000
2007-08-08T11:38:54 <lanius> but you have to check if every change is new
2007-08-08T11:39:39 <xorAxAx> hmm?
2007-08-08T11:39:57 <ThomasWaldmann> you currently even check n items * o revisions
2007-08-08T11:40:03 <lanius> where do the 1000 pages come from
2007-08-08T11:40:18 <lanius> where is the information about the changes stored
2007-08-08T11:40:24 <xorAxAx> lanius: its a large wiki with 1000 pages
2007-08-08T11:40:32 <xorAxAx> lanius: on the filesystem
2007-08-08T11:40:38 <lanius> where, that's important
2007-08-08T11:40:53 <ThomasWaldmann> 1000 is rather medium
2007-08-08T11:41:01 <lanius> that's not the point
2007-08-08T11:41:15 <xorAxAx> lanius: in every single rev
2007-08-08T11:41:17 <lanius> how and where are the changes stored and access, that's the perfromance question
2007-08-08T11:41:28 <xorAxAx> lanius: yes
2007-08-08T11:41:47 <lanius> what does "in every single rev" mean
2007-08-08T11:42:10 <xorAxAx> lanius: that the single fixed-row-width file only points to the meta data revs which have the usual metadata
2007-08-08T11:42:14 <xorAxAx> so you dont have any redudancy
2007-08-08T11:42:25 <xorAxAx> well, recentchagnes is an important page
2007-08-08T11:42:41 <xorAxAx> nevertheless this mustnt be much slower than the old solution of having just one file
2007-08-08T11:43:24 <lanius> must not or should not
2007-08-08T11:43:29 <xorAxAx> if you dislike the indirection, you wont have any other possibility of reusing the global edit log design and add an interface to it
2007-08-08T11:44:18 <lanius> itÄs not about liking anything
2007-08-08T11:44:18 <ThomasWaldmann> must not be much :)
2007-08-08T11:44:23 <xorAxAx> lanius: "wont likely be"
2007-08-08T11:44:30 <lanius> xorAxAx: must not is "darf nicht"
2007-08-08T11:44:38 <xorAxAx> yes
2007-08-08T11:44:58 <lanius> i don't understand the indirection part
2007-08-08T11:45:02 <xorAxAx> sigh
2007-08-08T11:45:12 <xorAxAx> there is a single logging entity
2007-08-08T11:45:12 <lanius> sigh'ing doesn't help
2007-08-08T11:45:18 <xorAxAx> that stores the 3-tuple
2007-08-08T11:45:22 <xorAxAx> do you remember? :)
2007-08-08T11:45:28 <xorAxAx> and its accessed via news()
2007-08-08T11:45:39 <lanius> in the single changes file?
2007-08-08T11:45:44 <xorAxAx> and in order to load the referred data, you need to fetch the revs of the items
2007-08-08T11:45:44 <lanius> where all tuples are stored?
2007-08-08T11:45:59 <xorAxAx> yes, in a file with a fixed row-width
2007-08-08T11:46:13 <lanius> fixed-row-width?
2007-08-08T11:46:14 <ThomasWaldmann> xorAxAx: how fixed for the item name?
2007-08-08T11:46:22 <xorAxAx> ThomasWaldmann: using some maximum length
2007-08-08T11:46:31 <lanius> why do we need that
2007-08-08T11:46:31 <ThomasWaldmann> ugh
2007-08-08T11:46:36 <xorAxAx> lanius: yes, so that you can bisect easily
2007-08-08T11:46:37 <xorAxAx> lanius: speed
2007-08-08T11:46:58 <xorAxAx> ThomasWaldmann: why ugh?
2007-08-08T11:47:25 <ThomasWaldmann> how much is max_path (or whatever it is called) usually?
2007-08-08T11:47:26 <xorAxAx> 255 chars ought to be enogh for everybody, or smoething like that
2007-08-08T11:47:34 <xorAxAx> 1024
2007-08-08T11:47:57 <xorAxAx> lanius: you wanted speed
2007-08-08T11:48:00 <ThomasWaldmann> if you dont want the mossad kill you, you would have to choose 1024
2007-08-08T11:48:09 <xorAxAx> we dont need it, we could use the same code that we have in the logfile modules
2007-08-08T11:48:17 <xorAxAx> using on-the-fly indexing
2007-08-08T11:48:29 <xorAxAx> and over allocation of read buffers
2007-08-08T11:48:35 <lanius> but speed whithout bugs
2007-08-08T11:48:48 <xorAxAx> lanius: bugs?
2007-08-08T11:48:50 <ThomasWaldmann> (or at least about 1000)
2007-08-08T11:49:07 <lanius> theoretically you can always create an item larger than max
2007-08-08T11:49:18 <xorAxAx> ThomasWaldmann: why, do palestinian terrorists have such long names?
2007-08-08T11:49:26 <lanius> so if we enforce a max, it must be enforced throughout the wiki
2007-08-08T11:49:28 <xorAxAx> lanius: ?
2007-08-08T11:49:39 <xorAxAx> lanius: well, thats a detail
2007-08-08T11:49:51 <xorAxAx> lanius: you can try using a non-fixed-width file as well
2007-08-08T11:49:52 <lanius> ok
2007-08-08T11:50:02 <ThomasWaldmann> no, but quoted or utf-8 encoded hebrew (chinese, ...) chars are a bit longer than ascii
2007-08-08T11:50:09 <xorAxAx> and benchmark how large the performance loss of indirection is
2007-08-08T11:50:17 <xorAxAx> ThomasWaldmann: yes
2007-08-08T11:50:45 <lanius> there was no indirection in that explanation until now, just a file with item/revision/mtime tuples
2007-08-08T11:50:48 <ThomasWaldmann> nir reported hitting the maxpath boundary rather easily iirc
2007-08-08T11:51:22 <ThomasWaldmann> the indirection is that you look up the rest in the item's local metadata
2007-08-08T11:51:39 <lanius> on which level does the indirection happen
2007-08-08T11:52:22 <xorAxAx> lanius: the indirection is the fact that you will mostly more parts of the meta data of the revisions
2007-08-08T11:52:27 <xorAxAx> lanius: e.g. the authorname
2007-08-08T11:52:46 <xorAxAx> lanius: therefore, you will need to use the "pointers" returned by news() to query the backend again
2007-08-08T11:52:52 <xorAxAx> for RC
2007-08-08T11:53:09 <lanius> so this is not on the storage level anymore but on the editlog level
2007-08-08T11:53:48 <lanius> right?
2007-08-08T11:53:57 <xorAxAx> or item collection level etc.
2007-08-08T11:54:06 <lanius> ok, i already implementet that
2007-08-08T11:54:08 <ThomasWaldmann> btw, if we go for that 1024 bytes row size, we need some purging more than we needed it for the 1.6 global edit-log (which never was touched in that respect)
2007-08-08T11:55:10 <xorAxAx> ThomasWaldmann: why?
2007-08-08T11:55:21 <ThomasWaldmann> because of size?
2007-08-08T11:55:35 <ThomasWaldmann> i guess it is 10x bigger
2007-08-08T11:56:49 <ThomasWaldmann> hmm, 8MB currently for mm, so it would be 80MB. We could live with that.
2007-08-08T11:57:15 <ThomasWaldmann> but append mode on nfs will still be broken?
2007-08-08T11:57:35 <xorAxAx> it will show less problems :)
2007-08-08T11:57:45 <lanius> will it?
2007-08-08T11:57:49 <xorAxAx> because you would notice a corrupt file directly
2007-08-08T11:57:53 <johill> Timo Sirainen (sp?) just did nfs/append mode analysis iirc
2007-08-08T11:58:16 <xorAxAx> but this idea only makes sense if the bisection time is much more than the time spent on indirection/metadata fetching
2007-08-08T11:58:28 <xorAxAx> so it has to be benchmark
2007-08-08T11:58:29 <xorAxAx> e
2007-08-08T11:58:30 <xorAxAx> d
2007-08-08T11:59:10 <ThomasWaldmann> hmm, we should use a fixed line length text/plain file if we do that
2007-08-08T11:59:24 <ThomasWaldmann> or it will suck if you have to fix it when it gets broken
2007-08-08T11:59:36 <johill> it'll suck anyway because you'd have to get the padding absolutely correct by hand
2007-08-08T11:59:43 <lanius> xorAxAx: the time on metadata fetching should be low since this stuff will be cached then
2007-08-08T12:00:02 <ThomasWaldmann> that is less hard than editing a binary file
2007-08-08T12:00:30 * lanius is getting something to eat
2007-08-08T12:00:37 <johill> depends on the tools; if you use bvi and have 1024 block sizes it'd be quite apparent when something is wrong
2007-08-08T12:00:56 <xorAxAx> ThomasWaldmann: not sure :)
2007-08-08T12:01:01 <xorAxAx> editing a binary file has its advantages
2007-08-08T12:01:06 <ThomasWaldmann> johill: and you can fix it then with bvi?
2007-08-08T12:01:11 <xorAxAx> because you wont try with a non-hex-editor
2007-08-08T12:01:17 <johill> sure, why not?
2007-08-08T12:01:23 <johill> if the blocking is bad insert/remove a few bytes
2007-08-08T12:01:36 <johill> if I want to add/remove something I can see exactly whether I got the block length right
2007-08-08T12:02:08 <xorAxAx> what is bvi? :)
2007-08-08T12:02:32 <johill> binary vi :)
2007-08-08T12:02:35 <xorAxAx> ah
2007-08-08T12:02:56 <ThomasWaldmann> btw, we can have both
2007-08-08T12:03:15 <ThomasWaldmann> it is just a matter of crlf at pos 1022
2007-08-08T12:03:35 <xorAxAx> argh
2007-08-08T12:03:36 <johill> yeah but if you use spaces to pad people won't see them and will use a text editor
2007-08-08T12:03:44 <ThomasWaldmann> and using 0x20 instead of 0x00
2007-08-08T12:03:46 <johill> if you use NULs to pad the text editor will give up right away
2007-08-08T12:04:11 <johill> so then you have to educate users to use a binary editor instead of them noticing
2007-08-08T12:05:07 <xorAxAx> you can also pad with #
2007-08-08T12:05:08 <xorAxAx> :-)
2007-08-08T12:05:43 <johill> :)
2007-08-08T12:06:05 <xorAxAx> well, it would be sensible to store utf-16
2007-08-08T12:06:13 <xorAxAx> or utf-32
2007-08-08T12:06:22 <xorAxAx> if the field is fixed-width anyway
2007-08-08T12:06:54 <ThomasWaldmann> then you have to have 4k row size
2007-08-08T12:07:07 <xorAxAx> ?
2007-08-08T12:07:22 <ThomasWaldmann> because for ascii it will be 4bytes instead of 1
2007-08-08T12:07:28 <xorAxAx> well, so you want to make the limit on bytes and not on chars?
2007-08-08T12:07:40 <xorAxAx> 1k utf-8 will be less bytes than 1k utf-32 in the worst case
2007-08-08T12:07:46 <xorAxAx> s/bytes/chars/
2007-08-08T12:08:03 <ThomasWaldmann> but not in the usual case
2007-08-08T12:08:13 <johill> you have to make a byte limit though, no?
2007-08-08T12:08:14 <ThomasWaldmann> ascii users also should be able to use maxpath
2007-08-08T12:08:28 <xorAxAx> johill: if the encoding is fixed-size, no :)
2007-08-08T12:08:41 <johill> yeah, ok, then you have to use ucs-4
2007-08-08T12:08:59 <ThomasWaldmann> so if they could use 1000 chars for a name, you need 4k utf-32
2007-08-08T12:09:02 <xorAxAx> well, all those implementation details ...
2007-08-08T12:09:11 <xorAxAx> that could be made configurable
2007-08-08T12:10:26 <ThomasWaldmann> btw, that large block size was why I was giving up that idea.
2007-08-08T12:10:47 <ThomasWaldmann> but well, we have the GBytes, so I now think it could be done.
2007-08-08T12:10:50 <CIA-28> moin: Ville-Pekka Vainio <vpivaini AT cs DOT helsinki DOT fi> * 2786:709ad9287c9b 1.7-maninfo-vpv/MoinMoin/script/import/manimport.py: Fix a bug with using info stuff even when the user didn't want info handling, which caused an error.
2007-08-08T12:14:10 <ThomasWaldmann> btw, I am waiting for creole link syntax comments from the usual suspects :)
2007-08-08T12:14:46 <ThomasWaldmann> see http://moinmo.in/LinkMarkupCollection
2007-08-08T12:15:24 <ThomasWaldmann> brb
2007-08-08T12:16:23 <johill> I'm all for redesigning the markup ;) but there are other odd corner cases too
2007-08-08T12:30:32 <ThomasWaldmann> yeah, it needs an escape for |
2007-08-08T12:31:38 <DrFrasierCrane> la la la
2007-08-08T12:31:49 <ThomasWaldmann> btw, did you look at radomir's creole parser (see 1.7 repo)? it builds a domtree internally and then processes it, calling the formatter.
2007-08-08T12:32:03 <johill> gave it a quick look, but not too much
2007-08-08T12:39:14 <ThomasWaldmann> looks much cleaner than the moin wiki parser
2007-08-08T12:39:33 <ThomasWaldmann> (well, it would be hard work to have it look worse :D )
2007-08-08T12:40:30 <johill> :)
2007-08-08T12:45:38 <johill> I quite like the ["xx" yy] syntax though probably the yy should be allowed to be quoted too
2007-08-08T12:46:09 <johill> but you can't use closing brackets anyway since the RE-based parser will terminate at one
2007-08-08T12:49:15 <ThomasWaldmann> the nice thing i found when trying creole alternatively to what we (would) have in 1.6
2007-08-08T12:49:44 <ThomasWaldmann> is that it solves quoting with the [[ | ]] markup
2007-08-08T12:50:10 <ThomasWaldmann> so it is simpler than using [] plus "" in misc. combinations
2007-08-08T12:50:22 <johill> but | isn't allowed in a link
2007-08-08T12:51:17 <johill> and people are already familiar with [""]
2007-08-08T12:51:45 <ThomasWaldmann> well, I guess nobody loves the 1.5 state of moin link markup
2007-08-08T12:51:51 <johill> heh
2007-08-08T12:52:55 <ThomasWaldmann> and if one would have to choose between [[]] and [""] in a clean room, one would go for [[]]
2007-08-08T12:53:11 <ThomasWaldmann> esp. us kbd owners love it
2007-08-08T12:53:11 <johill> oh sure
2007-08-08T12:53:33 <johill> yeah especially since [""] uses the right little finger twice there
2007-08-08T12:53:46 <johill> on different keys and with and without shift
2007-08-08T12:54:49 <ThomasWaldmann> we also have some not supported markup: [CamelCase link text] for example
2007-08-08T12:55:03 <ThomasWaldmann> but people often expect that to work
2007-08-08T12:55:20 <johill> yeah, good point
2007-08-08T12:55:32 <ThomasWaldmann> of course you can use "" to make it work, but it is a bit unnatural
2007-08-08T12:55:45 <johill> but if you're going to revamp the markup then you might want to consider other things like the pre sections too
2007-08-08T12:56:21 <ThomasWaldmann> because of the nested brokenness?
2007-08-08T12:56:26 <johill> for example
2007-08-08T12:56:42 <johill> and if paragraphs work right then you can mandate starting {{{ on a new line
2007-08-08T12:56:45 <johill> i.e. if
2007-08-08T12:56:47 <johill> this is
2007-08-08T12:56:48 <ThomasWaldmann> and <<macro arg,arg,arg>>
2007-08-08T12:56:49 <johill> a single paragraph
2007-08-08T12:56:53 <johill> gives a single paragraph
2007-08-08T12:57:04 <johill> because {{asdf}} is replaced by `asdf` anyway
2007-08-08T12:57:51 <johill> yeah I guess macros can't use [[]] any more then
2007-08-08T12:58:05 <johill> I also dislike the heading strictness
2007-08-08T12:58:19 <johill> it seems useless to have to indicate the heading depth twice
2007-08-08T12:58:35 <ThomasWaldmann> style reasons :)
2007-08-08T12:58:54 <ThomasWaldmann> this is the reasons why creole allows both
2007-08-08T12:59:08 <ThomasWaldmann> i don't like that they allow unbalanced, though
2007-08-08T12:59:14 <johill> I think they just allow any number of ending equal signs
2007-08-08T12:59:21 <johill> actually, I think it makes sense
2007-08-08T12:59:35 <johill> if you want to use the same document as plain text then sometimes it can make sense to indicate first-level headings by
2007-08-08T12:59:40 <johill> = heading =====
2007-08-08T12:59:43 <johill> padding it to some common width
2007-08-08T13:00:11 <ThomasWaldmann> looks dirty for me
2007-08-08T13:01:30 <johill> :)
2007-08-08T13:01:44 <johill> you probably want a text/plain formatter anyway
2007-08-08T13:02:04 <johill> in any case, macro args for example require a smarter parser
2007-08-08T13:02:09 <johill> or you can never use >> in a macro arg
2007-08-08T13:02:22 <johill> or right now you can't use ")]]"
2007-08-08T13:02:42 <johill> anyway, I gotta go
2007-08-08T16:32:38 <CIA-28> moin: Ville-Pekka Vainio <vpivaini AT cs DOT helsinki DOT fi> * 2787:a5b19012e2a7 1.7-maninfo-vpv/MoinMoin/script/import/manimport.py: Fix outdated comment
2007-08-08T16:32:41 <CIA-28> moin: Ville-Pekka Vainio <vpivaini AT cs DOT helsinki DOT fi> * 2788:14c8200c3b0f 1.7-maninfo-vpv/MoinMoin/script/import/manimport.py: Add support for translated man files, validate man XML files too.
2007-08-08T19:02:09 <zenhase> re
2007-08-08T19:02:30 <zenhase> ThomasWaldmann: yeah i think so
2007-08-08T19:03:26 <ThomasWaldmann> zenhase: ok, how far is your stuff?
2007-08-08T19:05:01 <zenhase> tests are working, genshi themes like in the prototype are working including the user selection screen
2007-08-08T19:05:33 <zenhase> last thing i was working on was beautifying the html-code a bit and then add documentation
2007-08-08T19:05:52 <ThomasWaldmann> Help page and docs/CHANGES.genshitheme please
2007-08-08T19:07:14 <zenhase> what should CHANGES.genshitheme contain? same as the help page or something more technical in description?
2007-08-08T19:07:39 <ThomasWaldmann> see what CHANGES contains :)
2007-08-08T19:07:46 <zenhase> ok :)
2007-08-08T19:07:54 <ThomasWaldmann> use the same format, just for your stuff
2007-08-08T19:08:12 <ThomasWaldmann> (it will be moved to CHANGES when the merge happens)
2007-08-08T19:08:32 <ThomasWaldmann> s/moved to/copied into/
2007-08-08T19:09:02 <ThomasWaldmann> you can also refer to your help page from there
2007-08-08T19:09:18 <ThomasWaldmann> ok, gtg to LUG meeting, cu :)
2007-08-08T19:09:26 <zenhase> hf
2007-08-08T19:30:31 <lanius> ThomasWaldmann: the editlog/__init__.py code is really scary
2007-08-08T19:32:02 <lanius> logfile/__init__.py i mean
2007-08-08T19:33:31 <lanius> what is all this buffer stuff for in it?
2007-08-08T19:35:57 <lanius> xorAxAx: any idea?
2007-08-08T20:16:26 <CIA-14> moin: Karol 'grzywacz' Nowak <grzywacz@sul.uni.lodz.pl> * 2645:892055d122fb 1.7-jabber-knowak/jabberbot/ (main.py xmlrpcbot.py xmppbot.py): Handle process/thread termination gracefully.
2007-08-08T20:16:30 <CIA-14> moin: Karol 'grzywacz' Nowak <grzywacz@sul.uni.lodz.pl> * 2649:9a17fabb8008 1.7-jabber-knowak/MoinMoin/ (3 files in 3 dirs): Merge main.
2007-08-08T21:19:02 <ThomasWaldmann> lanius: buffering using 2 buffers
2007-08-08T21:19:17 <ThomasWaldmann> now you see why i want to get rid of it
2007-08-08T22:33:53 * grzywacz grumbles
2007-08-08T22:40:55 <grzywacz> xorAxAx?
2007-08-08T22:41:10 <grzywacz> Anyone familiar with XPath?
2007-08-08T23:01:58 <xorAxAx> neagulm: baebd17538a8 cant be correct (This message has been postponed.)
2007-08-08T23:02:17 <neagulm> hello
2007-08-08T23:09:06 <grzywacz> hi neagulm
2007-08-08T23:11:20 <lanius> ThomasWaldmann: what are the two buffers good for
2007-08-08T23:14:20 <neagulm> ThomasWaldmann, hello
2007-08-08T23:19:34 <CIA-14> moin: Marian Neagul marian@info.uvt.ro work * 2759:a5b259f8f87b 1.7-classify-mneagul/MoinMoin/ (classifier/recommendation_system.py support/art.py): Use numpy. 15xfaster
2007-08-08T23:32:27 <ThomasWaldmann> lanius: if one buffer reads up to pos x, the next will read up to pos x+bufsize if you continue reading
2007-08-08T23:40:55 <lanius> ThomasWaldmann: and why
2007-08-08T23:42:42 <neagulm> ThomasWaldmann, did you have time to check the classifier, the behavior?
2007-08-08T23:55:52 <grzywacz> haha! /me dances a happy dance
2007-08-08T23:57:28 <nwp> grzywacz: XPath? why?
2007-08-08T23:59:11 <grzywacz> nwp, jabber. I wanted to have a one-liner that would return an inner <x> element of a <message> stanza
2007-08-08T23:59:22 <nwp> ah
2007-08-08T23:59:24 <grzywacz> I got owned by evil namespaces, but it seems to work now. ;)
2007-08-08T23:59:39 <nwp> I can't remember it offhand, but I recommend the O'Reilly XSLT book MoinMoin: MoinMoinChat/Logs/moin-dev/2007-08-08 (last edited 2007-10-29 19:08:21 by localhost)
