MoinMoin: MoinMoinChat/Logs/moin-dev/2006-07-19

2006-07-19T00:01:38  <Kepplar> ThomasWaldmann: not if i have to manually edit things
2006-07-19T00:02:27  <Kepplar> felixw: as i said, apart from legacy =)
2006-07-19T00:02:47  <xorAxAx> felixw: he is using hungarian notation
2006-07-19T00:02:51  <Kepplar> just feels wierd to me using underscore for over here ---------> and camelcase over somewhere else
2006-07-19T00:02:57  <Kepplar> xorAxAx: im not
2006-07-19T00:03:09  <Kepplar> xorAxAx: im slowly phasing it out
2006-07-19T00:03:12  <xorAxAx> Kepplar: nice
2006-07-19T00:03:32  <Kepplar> as i said i only support system hungarian notation for mainly dynamic languages :p
2006-07-19T00:18:00  <xorAxAx> hmm, no usable locking in the pageeditor :-(
2006-07-19T00:18:06  <xorAxAx> Kepplar: but you will repair it :)
2006-07-19T00:18:22  <xorAxAx> Kepplar: furthermore, you should change the locking system for the old compat backend
2006-07-19T00:18:29  <xorAxAx> as there is no point in preserving it
2006-07-19T00:19:16  <Kepplar> ??
2006-07-19T00:19:29  <Kepplar> im heading to bed
2006-07-19T00:19:41  <Kepplar> feel free to spill what your thinking.. ill read it up in the morning
2006-07-19T00:19:42  <Kepplar> :)
2006-07-19T00:19:42  <Kepplar> nmn
2006-07-19T00:19:43  <Kepplar> n
2006-07-19T00:19:44  <Kepplar> n
2006-07-19T00:19:47  <Kepplar> even
2006-07-19T00:19:49  <Kepplar> bah
2006-07-19T00:19:52  <xorAxAx> Kepplar: that were my thoughts
2006-07-19T00:20:03  <xorAxAx> have a look at the locking module
2006-07-19T00:20:12  <xorAxAx> and dont try to emulate the old locking
2006-07-19T00:20:21  <xorAxAx> esp. i want to lock from outside as well
2006-07-19T00:36:21  <xorAxAx> 00:32:42 < xorAxAx> c:fOo bar c:Baz
2006-07-19T00:36:21  <xorAxAx> 00:32:48 < xorAxAx> would be transformed into
2006-07-19T00:36:21  <xorAxAx> 00:33:14 < xorAxAx> foo bar baz c:fOo c:Baz
2006-07-19T00:36:21  <xorAxAx> 00:33:26 < xorAxAx> the last two terms would be normal search terms
2006-07-19T00:36:35  <fpletz> my issues with moinSearch were that it was searching the terms all over again in the page
2006-07-19T00:36:46  <fpletz> this data could be saved in the xapian db while indexing
2006-07-19T00:36:48  <xorAxAx> fpletz: can you give an example?
2006-07-19T00:37:13  * johill goes back to lurking
2006-07-19T00:37:19  <fpletz> hmm
2006-07-19T00:37:28  <fpletz> there's not really something to demonstrate
2006-07-19T00:37:32  <fpletz> but let me explain
2006-07-19T00:37:49  <fpletz> moinSearch was basically searching all pages with a regexp
2006-07-19T00:38:16  <fpletz> from xapian we get all the relevant documents so we could completely bypass moinSearch
2006-07-19T00:38:30  <fpletz> originally we used it just for searching the relevant documents
2006-07-19T00:38:41  <fpletz> how many and where the matches where
2006-07-19T00:38:48  <xorAxAx> umm why?
2006-07-19T00:38:52  <fpletz> to do sorting and stuff
2006-07-19T00:38:59  <johill> doesn't xapian do that too?
2006-07-19T00:39:04  <fpletz> yes ;)
2006-07-19T00:39:18  <xorAxAx> i mean, the case is clear: yield moin-terms if xapian cannot handle them
2006-07-19T00:39:26  <xorAxAx> and then feed them to the search machinery
2006-07-19T00:39:33  <fpletz> xorAxAx: full ack ;)
2006-07-19T00:39:46  <xorAxAx> the query optimiser would generate those foo terms for c:FoO queries
2006-07-19T00:40:07  <fpletz> but the good thing is, that we get a preselection of pages from xapian
2006-07-19T00:40:14  <xorAxAx> fpletz: i am talking about that
2006-07-19T00:40:14  <fpletz> so the search should be a lot quicker
2006-07-19T00:40:23  <fpletz> yes ;)
2006-07-19T00:40:24  <xorAxAx> fpletz: and that should be done on the term level IMHO
2006-07-19T00:41:14  <fpletz> what do you mean by term level?
2006-07-19T00:41:31  <johill> you should actually divide up the example though
2006-07-19T00:41:35  <johill> c:fOo bar c:Baz
2006-07-19T00:41:36  <johill> becomes
2006-07-19T00:41:47  <johill> foo bar baz | c:fOo c:Baz
2006-07-19T00:41:54  <xorAxAx> not OR but AND
2006-07-19T00:41:54  <johill> and only the part after | is fed to moinSearch
2006-07-19T00:41:58  <xorAxAx> yes
2006-07-19T00:42:02  <johill> I didn't mean that as binary
2006-07-19T00:42:03  <johill> :)
2006-07-19T00:42:12  <fpletz> uh, yeah
2006-07-19T00:42:16  <xorAxAx> fpletz: that you dont code that into some class but into the query optimiser that operates on term instances
2006-07-19T00:42:23  <xorAxAx> fpletz: thats much more flexible
2006-07-19T00:42:25  <johill> that way, moinSearch doesn't need to see the 'bar' token at all
2006-07-19T00:42:44  <fpletz> ah, I see...
2006-07-19T00:42:57  <xorAxAx> much less code
2006-07-19T00:43:00  <johill> and the query code just says:
2006-07-19T00:43:08  <johill> give me xapian results for 'foo bar baz'
2006-07-19T00:43:19  <xorAxAx> johill: no
2006-07-19T00:43:27  <johill> and then: give me moinSearch results for 'c:fOo c:Baz' on the previous results
2006-07-19T00:43:31  <fpletz> I tend to agree with johill ;)
2006-07-19T00:43:42  <xorAxAx> thats too monolithic
2006-07-19T00:43:46  <xorAxAx> compared to the old model
2006-07-19T00:43:50  <xorAxAx> i.e. a regression
2006-07-19T00:44:01  <xorAxAx> fpletz: is there anything left of the old search engine? :)
2006-07-19T00:44:21  <johill> well
2006-07-19T00:44:23  <fpletz> xorAxAx: I haven't touched most of the old code ;)
2006-07-19T00:44:30  <fpletz> just plugged in the new one
2006-07-19T00:44:34  <johill> you could also assign 'prices' to each search engine
2006-07-19T00:44:35  <xorAxAx> fpletz: hehe ok
2006-07-19T00:44:40  <xorAxAx> johill: thats already done
2006-07-19T00:44:41  <johill> i.e. xapian is cheap, and moinSearch is expensive
2006-07-19T00:44:45  <johill> and then you query them in order
2006-07-19T00:44:49  <johill> each on the results of the previous one
2006-07-19T00:44:50  <xorAxAx> johill: and it has fulltext search hooks etc.
2006-07-19T00:45:11  <xorAxAx> the only problem is generating the xapian query from a tree build of terms
2006-07-19T00:45:25  <xorAxAx> but that should be possible as well
2006-07-19T00:45:44  <fpletz> mmh, this is no problem and implemented
2006-07-19T00:45:46  <johill> alright, I think I understand too little of how you did that all
2006-07-19T00:45:50  <fpletz> if I understood you correctly
2006-07-19T00:45:55  <xorAxAx> johill: the search engine is 1.3.0 :)
2006-07-19T00:46:00  <johill> either you have a query optimiser that queries the different things
2006-07-19T00:46:03  <xorAxAx> johill: so you could have seen it as well :)
2006-07-19T00:46:08  <johill> hehehe
2006-07-19T00:46:15  <johill> never looked, I have to admit
2006-07-19T00:46:32  <johill> or you just pass the query to the backends in order
2006-07-19T00:46:49  <johill> and then it becomes duty of the xapian searcher to convert the c:... things to lowercase
2006-07-19T00:47:01  <johill> and moinSearch starts over with the original query
2006-07-19T00:47:02  <xorAxAx> johill: no!
2006-07-19T00:47:06  <xorAxAx> ugly :)
2006-07-19T00:47:09  <fpletz> this is how it was done and currently could be done ;)
2006-07-19T00:47:10  <johill> yeah
2006-07-19T00:47:15  <xorAxAx> fpletz: "was"?
2006-07-19T00:47:25  <xorAxAx> fpletz: when was it done like that? :)
2006-07-19T00:47:27  <johill> xorAxAx: agreed
2006-07-19T00:47:41  <xorAxAx> in that secret thomas branch that was fortunately nnever published? ;-)
2006-07-19T00:47:41  <fpletz> xorAxAx: before I dropped moinSearch for most of the easy searches
2006-07-19T00:47:43  <johill> xorAxAx: I just don't see how you want an optimiser that isn't aware of the different backends
2006-07-19T00:47:47  <fpletz> thomas implemented this in his xapian branch
2006-07-19T00:47:52  <fpletz> yeah :D
2006-07-19T00:49:06  <xorAxAx> johill: the optimiser traverses all terms, sees that there is a c-term. then it could call some method that replaces that term by  ( c-term AND c-term.lower() )
2006-07-19T00:49:17  <xorAxAx> johill: no idea where you place that interface
2006-07-19T00:49:20  <johill> xorAxAx: I understand that
2006-07-19T00:49:33  <johill> xorAxAx: I just don't understand how you want the optimiser to do that if it doesn't know it needs to
2006-07-19T00:49:39  <xorAxAx> and xapian would traverse all terms and build its query from that
2006-07-19T00:49:43  <xorAxAx> skip the unknown ones
2006-07-19T00:49:44  <fpletz> xorAxAx: this is done for contructing the xapian query
2006-07-19T00:49:48  <johill> i.e. if someone doesn't have xapian search installed it doesn't need to at all
2006-07-19T00:50:01  <xorAxAx> johill: yes, of course
2006-07-19T00:50:10  <fpletz> xapian doesn't build any queries because we can't use its queryparser
2006-07-19T00:50:20  <xorAxAx> fpletz: ah, ok
2006-07-19T00:50:34  <fpletz> I hooked in a custom queryparser in the moin qp
2006-07-19T00:50:34  <xorAxAx> fpletz: and how do you construct the query?
2006-07-19T00:50:36  <fpletz> we contruct xapian query-objects ourselves ;)
2006-07-19T00:50:44  <xorAxAx> by reading the query string?
2006-07-19T00:50:45  <johill> that's ok, then 'xapian' in this context is the python code that constructs it ;)
2006-07-19T00:50:49  <xorAxAx> or the term objects?
2006-07-19T00:50:59  <fpletz> xorAxAx: by traversing the term objects
2006-07-19T00:51:01  <xorAxAx> ok
2006-07-19T00:51:23  <xorAxAx> so terms could have a method supplied by a xapian mix-in ... no idea :)
2006-07-19T00:51:24  <johill> xorAxAx: ok, I suppose what you could do is have a rewrite_query call for each backend
2006-07-19T00:51:46  <johill> rewrite_query returns two new queries: one that it will do, and one that it cannot fulfill
2006-07-19T00:51:46  <xorAxAx> for example
2006-07-19T00:52:06  <xorAxAx> but this "thunking" into another search engine should be controlled in one placde
2006-07-19T00:52:10  <johill> e.g. the xapian.rewrite_query("foo c:bAr c:bAZ") would return
2006-07-19T00:52:14  <xorAxAx> that is not busy parsing or term constructing
2006-07-19T00:52:21  <xorAxAx> johill: no!
2006-07-19T00:52:23  <johill> ("foo bar baz", "c:bAr c:bAZ")
2006-07-19T00:52:29  <fpletz> no ;)
2006-07-19T00:52:30  <xorAxAx> johill: the query is parsed already
2006-07-19T00:52:33  <johill> well
2006-07-19T00:52:35  <fpletz> right ;)
2006-07-19T00:52:40  <johill> then make that the equivalent on the query tree
2006-07-19T00:52:44  <fpletz> we basically already  have that code
2006-07-19T00:53:15  <johill> however that is represented
2006-07-19T00:53:42  <fpletz> yes, we can reduce the query tree to terms xapian cannot handle
2006-07-19T00:53:49  <fpletz> and make a regular moinSearch on them
2006-07-19T00:54:00  <fpletz> merging all matches from both search engines
2006-07-19T00:54:04  <johill> and( [caseinsensitive("foo"), casesensitive("bAr"), ...] )
2006-07-19T00:54:37  <fpletz> yup
2006-07-19T00:54:37  <fpletz> I'm putting this right on top of my todo list
2006-07-19T00:54:44  <fpletz> along the subpage issue
2006-07-19T00:54:57  <johill> and then the xapian.rewrite_query actually returns
2006-07-19T00:55:31  <johill> ( and( [ci("foo"), ci("bar"), ci("baz")] ), and ([cs("bAr"), cs("bAZ") ]) )
2006-07-19T00:55:55  <johill> and the second part of that pair is passed to the next rewrite_query
2006-07-19T00:56:20  <johill> or you could just change the search interface
2006-07-19T00:56:49  <johill> xapian.search(expression-tree) would not just return the list of hits however that is represented, but also a (possibly empty) tree of search terms it couldn't honour
2006-07-19T00:57:27  <johill> and if the tree isn't empty, then you pass the list of hits along with the remaining query tree to the next search
2006-07-19T00:58:03  <johill> note that I'm just talking on a very high level here without having a clue ;)
2006-07-19T00:58:12  <fpletz> right, perfect, that was the idea ;)
2006-07-19T00:58:52  <fpletz> but currently we just have 2 search engines, so a general implemenation of the principle might be overkill
2006-07-19T00:59:08  <johill> it shouldn't be hard
2006-07-19T00:59:21  <johill> you have to have the code somewhere anyway
2006-07-19T00:59:29  <johill> and putting it like this into the relevant modules makes the most sense ;)
2006-07-19T00:59:59  <johill> make sure you think of the corner cases
2006-07-19T01:00:02  <fpletz> mmh, right
2006-07-19T01:00:10  <xorAxAx> fpletz: as long as there is no duplication of code and ideas ....
2006-07-19T01:00:14  <johill> if I enter an expression like "c:fOoOo c:bArAr"
2006-07-19T01:00:22  <johill> eh wait
2006-07-19T01:00:24  <johill> that was dumb
2006-07-19T01:00:46  <fpletz> xorAxAx: there shouldn't as far as I can see now
2006-07-19T01:00:47  <johill> lets say you have some sort of search that xapian cannot fulfill at all
2006-07-19T01:01:01  <johill> not sure if there's any such search but lets assume for the sake of this example that there is
2006-07-19T01:01:04  <fpletz> like certein kinds of regexp
2006-07-19T01:01:08  <fpletz> -e+a
2006-07-19T01:01:16  <johill> right
2006-07-19T01:01:24  <johill> then the xapian searcher should not have to build a list of all pages to pass on to the next searcher
2006-07-19T01:01:40  <johill> it should just return 'not possible' instead of a list of hits/modified terms
2006-07-19T01:01:51  <xorAxAx> NotImplemented :)
2006-07-19T01:01:58  <xorAxAx> yeah, that sounds like coercion protocoll :)
2006-07-19T01:02:09  <fpletz> yes, right
2006-07-19T01:02:10  <johill> no, NotImplemented is bad
2006-07-19T01:02:20  <xorAxAx> johill: why? %-)
2006-07-19T01:02:26  <johill> too generic
2006-07-19T01:02:47  <johill> just personal preference I guess, I don't like NotImplemented for something that depends on the input data
2006-07-19T01:02:50  <fpletz> but xapian could at least search for other terms in the query
2006-07-19T01:03:03  <xorAxAx> johill: do you know the 2.1 coercion protocoll in python?
2006-07-19T01:03:17  <johill> no, any quick link?
2006-07-19T01:03:36  <xorAxAx> http://docs.python.org/ref/coercion-rules.html
2006-07-19T01:03:45  <xorAxAx> one of the dirtiest things :)
2006-07-19T01:05:35  <johill> heh
2006-07-19T01:06:22  <johill> I don't see what part sounds like coercion here though ;)
2006-07-19T01:07:34  <xorAxAx> xapian terms are coerced to normal terms or something like that :)
2006-07-19T01:08:53  <johill> heh
2006-07-19T01:09:05  <xorAxAx> ThomasWaldmann: your i18n script didnt update ar
2006-07-19T01:09:09  <johill> anyway, I just wanted to point out that there must be some protocol to return "I can't do anything"
2006-07-19T01:17:14  <johill> (so that it doesn't have to build a list of all pages and return that long with the same terms)
2006-07-19T01:17:21  <johill> and now, I think I'll get some sleep :)
2006-07-19T01:17:25  <xorAxAx> me too
2006-07-19T01:18:26  <fpletz> good idea.. just have to finish up this movie ;)
2006-07-19T08:57:24  <ThomasWaldmann> moin
2006-07-19T09:29:50  <fpletz> moin
2006-07-19T09:36:12  <ThomasWaldmann> fpletz: you forgot your "daily CHANGES entry" for yesterday
2006-07-19T09:36:58  <ThomasWaldmann> fpletz: ignore, was looking at wrong changeset
2006-07-19T09:37:12  <fpletz> ok :)
2006-07-19T09:37:21  <fpletz> I did a merge after my changes
2006-07-19T09:44:56  <ThomasWaldmann> xorAxAx: I guess you didnt "forget" it, so I added a note to your CHANGES. Feel free to improve that entry today.
2006-07-19T10:21:51  <Kepplar> moin
2006-07-19T22:34:17  <Kepplar> this is stupid
2006-07-19T22:34:22  <Kepplar> its way too hot to concentrate
2006-07-19T22:35:12  <felixw> Yup.  It's like 30 degree C here.
2006-07-19T22:35:29  <felixw> Or 27 maybe.
2006-07-19T22:36:05  <felixw> Yup, it's 27 according to the LuX bot.
2006-07-19T22:37:59  <Kepplar> ive done virtually no work today
2006-07-19T22:38:02  <Kepplar> cant concentrate
2006-07-19T22:38:08  <Kepplar> littereallykeep falling alseep
2006-07-19T22:40:49  <Kepplar> # mimetype stuff ------------------------------------------------------------
2006-07-19T22:40:49  <Kepplar> class MimeType(object):
2006-07-19T22:40:49  <Kepplar> <<<<<<< /home/kepplar/dev/moinmoin/storage/MoinMoin/wikiutil.py """ Represents a mimetype like text/plain """
2006-07-19T22:40:52  <Kepplar> ======= """ represents a mimetype like text/plain """
2006-07-19T22:40:55  <Kepplar> >>>>>>> /tmp/wikiutil.py~other.gjt4ox def __init__(self, mimestr=None, filename=None):
2006-07-19T22:40:58  <Kepplar> what does that mean?
2006-07-19T22:44:52  <felixw> Kepplar: That looks like a merge conflict (did you mean that?).
2006-07-19T22:45:32  <felixw> Just edit it so that it looks "right", if you will.
2006-07-19T22:45:52  <Kepplar> sorted it now
2006-07-19T22:45:56  <Kepplar> wasnt sure ont he toation
2006-07-19T22:46:13  <Kepplar> as its just one letter thats the difference, i didnt see how it was a merge problem O_o
2006-07-19T22:47:04  <Kepplar> searching for changes is hanging
2006-07-19T22:47:07  <Kepplar> when i push
2006-07-19T22:47:30  <Kepplar> ah
2006-07-19T22:47:31  <Kepplar> nm
2006-07-19T23:05:54  <xorAxAx> Kepplar: just use a graphical client as i said
2006-07-19T23:08:12  <Kepplar> if hg had a full graphics UI like svn then I will:)
2006-07-19T23:09:52  <xorAxAx> Kepplar: there are many graphical clients that do the merging for you
2006-07-19T23:10:37  <Kepplar> and they arent fiddily to set up and use?
2006-07-19T23:10:40  <Kepplar> hmm
2006-07-19T23:10:49  <Kepplar> do i have to have an __init__.py for every directory?
2006-07-19T23:10:56  <Kepplar> from MoinMoin.storage import storage, flatfile
2006-07-19T23:10:57  <Kepplar> appears to be invalid
2006-07-19T23:11:56  <xorAxAx> Kepplar: not at all, they work out of the box
2006-07-19T23:12:05  <Kepplar> kdiff3 wasnt it?
2006-07-19T23:12:29  <Kepplar> that sounds like a kde application
2006-07-19T23:12:29  <xorAxAx> for example
2006-07-19T23:12:32  <Kepplar> qt --
2006-07-19T23:12:42  <xorAxAx> there are at least 5 supported ones
2006-07-19T23:12:44  <xorAxAx> RTFM
2006-07-19T23:12:45  <Kepplar> cool