2007-04-24T00:00:21 <lanius> so why not have the Item implement ItemBehaviour and letting the Item call foo(my_item) when view() is called on it
2007-04-24T00:00:22 <xorAxAx> well, you will realise that edit actually needs 2 actions: show_editform and do_edit
2007-04-24T00:00:36 <xorAxAx> lanius: an item doesnt implement that
2007-04-24T00:00:48 <xorAxAx> lanius: you would have to rename classes, please find another nae
2007-04-24T00:01:03 <xorAxAx> because the current rev of an item determines all stuff that can be done on it
2007-04-24T00:01:14 <xorAxAx> (as long as the operations are not binary like diff)
2007-04-24T00:01:17 <lanius> it's the same what you sayed, just the caller of foo(my_item) is different
2007-04-24T00:01:41 <xorAxAx> lanius: no, the large difference here is that item should be data-only, no behaviour
2007-04-24T00:01:49 <lanius> why
2007-04-24T00:01:52 <xorAxAx> and delegating at this point doesnt help
2007-04-24T00:01:58 <xorAxAx> because MVC makes sense here
2007-04-24T00:02:07 <xorAxAx> and, frankly, i dont think that you will finish the VC part
2007-04-24T00:02:15 <xorAxAx> because of time constraints :)
2007-04-24T00:02:18 <xorAxAx> (but we will see)
2007-04-24T00:02:28 <xorAxAx> do you know MVC? :)
2007-04-24T00:02:36 <lanius> i do
2007-04-24T00:03:09 <lanius> where does access control belong in your MVC
2007-04-24T00:03:10 <ThomasWaldmann> xorAxAx: that why i had those levels. the storage items are not related to the ui items
2007-04-24T00:03:18 <xorAxAx> ThomasWaldmann: yeah
2007-04-24T00:03:35 <xorAxAx> well, your idea was to have the abstractitem to implement the behaviour methods which delegate to self.foo(self.current_rev).myname(*args)
2007-04-24T00:03:40 <ThomasWaldmann> we just need good names for everything so that we dont confuse stuff
2007-04-24T00:03:50 <xorAxAx> and i dont see any need for that delegation
2007-04-24T00:04:05 <lanius> it seems easier to use for me
2007-04-24T00:04:23 <xorAxAx> lanius: well, python has better ways for shortcuts, we can talk about that later :)
2007-04-24T00:04:31 <xorAxAx> ThomasWaldmann: thats a good question
2007-04-24T00:04:58 <xorAxAx> i think a request to the revision list-like object should check for ACLs
2007-04-24T00:05:34 <lanius> what's the benefit of that, it confuses behaviour and data ;)
2007-04-24T00:06:05 <ThomasWaldmann> ACL check is not as easy as it seems on first look
2007-04-24T00:06:05 <xorAxAx> good point, but note that this is a different kind of behaviour
2007-04-24T00:06:06 <lanius> i think in your model it belongs into the VC part, just like the mimetypes
2007-04-24T00:06:17 <xorAxAx> lanius: well, do you know VFS?
2007-04-24T00:06:34 <lanius> Vancouver Film School is the first hit on google
2007-04-24T00:06:36 <xorAxAx> there, security checks are in the most inner interfaces that are possible
2007-04-24T00:06:42 <xorAxAx> virtual file system
2007-04-24T00:06:59 <xorAxAx> thats a subsystem in linux, reimplemented in e.g. gnome-vfs
2007-04-24T00:07:08 <xorAxAx> its a set of losely defined posix calls
2007-04-24T00:07:22 <xorAxAx> thats the semantics that is in use nowadays
2007-04-24T00:07:30 <ThomasWaldmann> xorAxAx: we need acls at a higher position
2007-04-24T00:07:35 <xorAxAx> ThomasWaldmann: why?
2007-04-24T00:07:40 <xorAxAx> make an example
2007-04-24T00:07:44 <ThomasWaldmann> because we cant just look at the acl of the revision wanted
2007-04-24T00:07:49 <xorAxAx> why?
2007-04-24T00:08:01 <ThomasWaldmann> it might be less strict than the currently set acl for that item
2007-04-24T00:08:21 <xorAxAx> well
2007-04-24T00:08:26 <ThomasWaldmann> (of course we could define it that way, but it would be different from now)
2007-04-24T00:08:27 <xorAxAx> thats still a requirement on the data level
2007-04-24T00:08:46 <xorAxAx> this can be solved by having a revision proxy that knows its item
2007-04-24T00:09:05 <xorAxAx> can you think of an example that actually carries the acl problem into the behaviour layer?
2007-04-24T00:09:42 <ThomasWaldmann> for current behaviour it need to be on item level, not itemrev
2007-04-24T00:10:08 <xorAxAx> as i said, thats not necessarily true :)
2007-04-24T00:10:18 <ThomasWaldmann> (because we ever look at current rev's acls)
2007-04-24T00:10:27 <xorAxAx> i understood that point
2007-04-24T00:10:44 <xorAxAx> hmm
2007-04-24T00:11:23 <xorAxAx> ThomasWaldmann: a request for a revision returns an instance of a proxy class
2007-04-24T00:11:25 <ThomasWaldmann> for deleted items, we have the additional problem that we dont have metadata in 1.5, but that will be gone with separate metadata
2007-04-24T00:11:39 <xorAxAx> this proxy class knows the items that the revision belongs to
2007-04-24T00:11:50 <xorAxAx> and wraps every call for a method with an ACL check
2007-04-24T00:11:55 <xorAxAx> thats easy to implement in python
2007-04-24T00:12:26 <xorAxAx> the idea is that the item itself cant implement it
2007-04-24T00:12:34 <ThomasWaldmann> sounds ok (we still need another one not doing that :)
2007-04-24T00:12:35 <xorAxAx> because it wont see the operation that will be performed
2007-04-24T00:12:43 <xorAxAx> (it will be performed on the revision)
2007-04-24T00:12:58 <xorAxAx> "another one"?
2007-04-24T00:13:11 <xorAxAx> lanius: still listening? :)
2007-04-24T00:13:12 <ThomasWaldmann> for converters, we dont want to check acls all the time
2007-04-24T00:13:15 <lanius> sure :D
2007-04-24T00:13:31 <xorAxAx> ThomasWaldmann: hmm?
2007-04-24T00:13:41 <xorAxAx> ThomasWaldmann: you mean the background indexer?
2007-04-24T00:14:04 <ThomasWaldmann> converter = loop over all stuff for converting from fs backend to sql backend
2007-04-24T00:14:09 <xorAxAx> ah!
2007-04-24T00:14:20 <xorAxAx> well
2007-04-24T00:14:33 <xorAxAx> thats a matter of the user object that is supplied or whatever it is
2007-04-24T00:14:45 <xorAxAx> i.e. the method where the may call is delegated to
2007-04-24T00:14:50 <xorAxAx> it could return True in every case
2007-04-24T00:15:05 <ThomasWaldmann> i dont mean it is a privilege problem, but doing the checks all the time costs lots of time
2007-04-24T00:15:09 <xorAxAx> then you just have the overhead of the few method calls but no actual checking
2007-04-24T00:15:29 <xorAxAx> i dont see the problem of a few msecs on a use case that is done a single time :)
2007-04-24T00:15:34 <lanius> or if you put in the VC part the converter could just use the M part
2007-04-24T00:15:49 <xorAxAx> lanius: i just realised that we have 2 Ms
2007-04-24T00:15:58 <lanius> do we?
2007-04-24T00:16:00 <xorAxAx> and yes, the converter would use the lower M
2007-04-24T00:16:08 <ThomasWaldmann> ok, if you shortcut the acl check by overriding "may" it is doable.
2007-04-24T00:16:15 <xorAxAx> well, my idea is that Item is the higher M and StorageBackend stuff the lower one
2007-04-24T00:16:15 <lanius> you mean the Item M and the Storage M ?
2007-04-24T00:16:19 <xorAxAx> yes
2007-04-24T00:16:19 <lanius> yes
2007-04-24T00:16:50 <lanius> but i think the converter would use the higher M
2007-04-24T00:16:54 <lanius> because it's just easier to use
2007-04-24T00:16:57 <xorAxAx> yeah
2007-04-24T00:17:15 <lanius> that's why it is there
2007-04-24T00:17:28 <lanius> and if the converter should not respect acl's we can put that in the C part
2007-04-24T00:17:33 <xorAxAx> so, do you feel like updating the chart with our ideas? :)
2007-04-24T00:17:40 <xorAxAx> in the next day(s)
2007-04-24T00:17:51 <lanius> yes
2007-04-24T00:18:03 <lanius> or even nights
2007-04-24T00:18:08 <xorAxAx> i think it makes sense if you look at file in python and at StringIO to see how implementors of the file-like protocol look like
2007-04-24T00:18:35 <xorAxAx> and we havent talked about the operations on the storagemanager class
2007-04-24T00:19:02 <xorAxAx> does it maybe make sense to move the itemsupport stuff to one of both layers?
2007-04-24T00:19:09 <xorAxAx> (no idea)
2007-04-24T00:19:09 <lanius> yes, but an initial design is on the page in the middle
2007-04-24T00:19:24 <xorAxAx> ah
2007-04-24T00:19:27 <lanius> ItemSupport is the connection
2007-04-24T00:19:35 <lanius> it must know both
2007-04-24T00:19:54 <xorAxAx> well, the question if it it is necessary at all :)
2007-04-24T00:20:13 <xorAxAx> but it makes snese to have it exist, yes
2007-04-24T00:20:22 <xorAxAx> e.g. item_exists
2007-04-24T00:20:33 <lanius> it necessary for the design, so you have a clear separation between the two M's
2007-04-24T00:20:35 <xorAxAx> its not implementable on the Item level without creating an item object
2007-04-24T00:20:42 <xorAxAx> (like it is today)
2007-04-24T00:20:59 <lanius> yeah, i saw that
2007-04-24T00:21:51 <xorAxAx> well, i want to have a file-like api for reading at least, and a possiblity to store large items (some write_from_fs_file() method)
2007-04-24T00:22:01 <xorAxAx> that needs to propagate to IStorage
2007-04-24T00:22:31 <xorAxAx> otherwise its a pain to deal with large files (people will love a wiki that can store files easily like moin :))
2007-04-24T00:22:50 <lanius> don't understand that yet, but let's talk about it next time :-)
2007-04-24T00:22:55 <xorAxAx> well
2007-04-24T00:23:08 <xorAxAx> its Stream vs. String for you java guy
2007-04-24T00:23:22 <lanius> yeah
2007-04-24T00:23:37 <lanius> string[] is very wrong there
2007-04-24T00:23:42 <xorAxAx> if somebody uploads a 2 GB jpg file (i have a computer science for medics student in mind)
2007-04-24T00:23:53 <ThomasWaldmann> hehe
2007-04-24T00:23:56 <xorAxAx> then you dont want to keep it in the RAM
2007-04-24T00:24:07 <xorAxAx> (like it is with a string)
2007-04-24T00:24:19 <xorAxAx> but it says " Boolean writeItem(String title, String[] content);"
2007-04-24T00:24:38 <xorAxAx> and for reading, you need an object which has some methods
2007-04-24T00:24:45 <xorAxAx> see StringIO what kind of methods make sense
2007-04-24T00:24:45 <lanius> that what i sayed "initial" :D
2007-04-24T00:24:56 <lanius> i'll take a look at it
2007-04-24T00:25:08 <xorAxAx> and for writing, i think you only need some write() method
2007-04-24T00:25:17 <ThomasWaldmann> otoh, for comfort, we want just some attrs, that do file ops automatically
2007-04-24T00:25:38 <ThomasWaldmann> (if we just have 1kb or so)
2007-04-24T00:26:01 <xorAxAx> well, read_view_on_file.read() vs. get_data
2007-04-24T00:26:15 <xorAxAx> or write_view_on_file.write() vs write_data()
2007-04-24T00:26:35 <ThomasWaldmann> properties
2007-04-24T00:26:46 <xorAxAx> yes, get_data is a property
2007-04-24T00:26:56 <xorAxAx> and it would not save much :)
2007-04-24T00:27:04 <ThomasWaldmann> data would be the property
2007-04-24T00:27:05 <xorAxAx> (ok, rather call it data)
2007-04-24T00:27:11 <ThomasWaldmann> :)
2007-04-24T00:27:23 <xorAxAx> read_view_on_file would be a property as well
2007-04-24T00:27:32 <xorAxAx> one that creates a file-like proxy :)
2007-04-24T00:27:52 <xorAxAx> i wonder how much these ideas match zope 3 .... frankly speaking, i dont want to know :)
2007-04-24T00:28:06 <ThomasWaldmann> ask __doc__ :))
2007-04-24T00:28:11 <ThomasWaldmann> (and duck)
2007-04-24T00:28:15 <xorAxAx> hehe, he doesnt know zope 3 :)
2007-04-24T00:28:24 <xorAxAx> i think they are still using zope 2
2007-04-24T00:28:27 <dreimark> spam/hamcheck needs to be a property too ?
2007-04-24T00:28:49 <xorAxAx> dreimark: well, the model is rather large :)
2007-04-24T00:29:45 <xorAxAx> lanius: i think you have seen that changing the storage doesnt already give you all new features - and the remaining tasks like pulling the new VC stuff into the code will need quite some work as well
2007-04-24T00:30:02 <xorAxAx> but changing the storage will be the first step to the goal :-)
2007-04-24T00:30:11 <xorAxAx> +layers
2007-04-24T00:30:28 <lanius> and it must be the first step
2007-04-24T00:30:37 <xorAxAx> yes, makes sense
2007-04-24T00:31:39 <lanius> ok, i wish you a pleasant night guys
2007-04-24T00:32:50 <xorAxAx> yeah, you too
2007-04-24T00:32:55 <xorAxAx> have fun with argouml
2007-04-24T00:33:06 <lanius> :D
2007-04-24T00:34:37 <dreimark> good night
2007-04-24T00:35:46 <ThomasWaldmann> hehe, bed time
2007-04-24T00:38:55 <ThomasWaldmann> xorAxAx: for the system / help pages we maybe can easily get rid of the current underlay method, but some people want it for farm stuff
2007-04-24T00:39:35 <xorAxAx> ThomasWaldmann: well, you could disallow editing on "unionfs"-like layering scenarios
2007-04-24T00:39:49 <xorAxAx> then its fixed and the orignal intention is restored
2007-04-24T00:40:01 <xorAxAx> in order to get that, you implement a trivial storage backend :)
2007-04-24T00:41:11 <ThomasWaldmann> a readonly backend?
2007-04-24T00:41:37 <xorAxAx> a backend that exposes one backend readonly with a second backend that layers with full features above it
2007-04-24T00:41:52 <xorAxAx> hmm
2007-04-24T00:42:04 <xorAxAx> oh well, thats still broken
2007-04-24T00:42:52 <ThomasWaldmann> we maybe need a layer indicator on the ui :)
2007-04-24T00:43:42 <ThomasWaldmann> or just forget about layers until we know better :D
2007-04-24T00:48:23 <ThomasWaldmann> http://hg.thinkmo.de/moin/1.6/rev/006fe3f373f4 ehrm, wtf?
2007-04-24T00:49:00 <xorAxAx> hmm
2007-04-24T00:49:24 * ThomasWaldmann .oO(don't code at 23:30 :)
2007-04-24T00:50:39 <xorAxAx> i think i will apply the hierachical acl patch next week to 1.6 and track down all bugs
2007-04-24T00:50:56 <ThomasWaldmann> 1.7 or 1.6?
2007-04-24T00:51:00 <xorAxAx> 1.6.
2007-04-24T00:51:28 <xorAxAx> because it will be in use directly after (there is no better way to find bugs :-))
2007-04-24T00:51:41 <xorAxAx> (and fix them)
2007-04-24T00:51:42 <ThomasWaldmann> not sure if that is a good idea
2007-04-24T00:52:00 <xorAxAx> it is rather short ...
2007-04-24T00:52:04 <xorAxAx> its not like its a large one
2007-04-24T00:52:46 <ThomasWaldmann> is there a 1.6 patch already or only 1.5?
2007-04-24T00:52:57 <xorAxAx> 1.5
2007-04-24T00:53:19 <xorAxAx> but note that i will start using it before i commit it
2007-04-24T00:53:31 <xorAxAx> on a production machine
2007-04-24T00:53:38 <xorAxAx> for people with expectations :)
2007-04-24T00:53:57 <xorAxAx> and i will have a review (its really quite limited in length)=
2007-04-24T00:54:36 <ThomasWaldmann> ok, we also need docs then
2007-04-24T00:54:46 <ThomasWaldmann> CHANGES and moinmaster
2007-04-24T00:55:03 <xorAxAx> sure
2007-04-24T00:56:08 <ThomasWaldmann> acl_hierarchic and please check grammar for this :)
2007-04-24T00:57:14 <ThomasWaldmann> + if somehting: and syntax X)
2007-04-24T00:58:09 <ThomasWaldmann> (and lots more ...)
2007-04-24T01:00:13 <ThomasWaldmann> the patch will need a rewrite for 1.6 because acl stuff changed
2007-04-24T08:39:00 <dreimark> moin
2007-04-24T08:54:17 <dreimark> bbl, some talks
2007-04-24T08:59:14 <ThomasWaldmann> moin
2007-04-24T11:56:41 <johill> dreimark: what is no_862? and what does 862 mean?
2007-04-24T11:57:09 <johill> oh that's the css class
2007-04-24T11:57:14 <johill> doesn't make me understand it ;)
2007-04-24T12:01:52 <CIA-28> moin: Johannes Berg <johannes AT sipsolutions DOT net> * 2025:d919b7b7b3e9 1.7/MoinMoin/ (7 files in 2 dirs):
2007-04-24T12:01:52 <CIA-28> moin: auth framework: login() methods return an object now
2007-04-24T12:01:52 <CIA-28> moin: This avoids overloading the returned tuple and simplifies code since
2007-04-24T12:01:52 <CIA-28> moin: they can now just instantiate ContinueLogin(), CancelLogin() etc.
2007-04-24T12:03:54 <CIA-28> moin: Johannes Berg <johannes AT sipsolutions DOT net> * 2026:5aa49f81db45 1.7/MoinMoin/request/__init__.py: finishers: execute only once and make sure request.user is set
2007-04-24T12:54:32 <ThomasWaldmann> johill: it has the name line862 because it was in the src on line 862 at that time
2007-04-24T12:54:56 <ThomasWaldmann> (we could've also named it joe or jim or foo, but i guess that wouldn't be better :)
2007-04-24T13:01:56 <johill> heh
2007-04-24T13:02:07 <johill> something strange is happening
2007-04-24T13:02:45 <johill> my action is invoked twice
2007-04-24T13:02:45 <johill> hrmm
2007-04-24T13:04:01 <johill> for a single post
2007-04-24T13:04:02 <johill> strange
2007-04-24T13:06:41 <johill> fcgi bug?
2007-04-24T13:06:49 <dreimark> bb
2007-04-24T13:12:08 <johill> hmm
2007-04-24T13:14:28 <dreimark> johill:every action?
2007-04-24T13:14:49 <johill> dunno
2007-04-24T13:14:54 <johill> no
2007-04-24T13:15:13 <johill> I have an action that creates a form and that is submitted
2007-04-24T13:15:18 <johill> again back to the action
2007-04-24T13:15:30 <johill> and for some reason it's handled twice under some unknown circumstances
2007-04-24T13:15:47 <johill> very odd
2007-04-24T13:24:08 <johill> apparently this happens when the action raises an exception
2007-04-24T13:29:59 <johill> nah
2007-04-24T13:30:06 <johill> hmmm
2007-04-24T13:33:37 <johill> actually, it does only happen on exceptions
2007-04-24T13:33:39 <johill> but it's still weird
2007-04-24T13:37:19 <johill> ok, openid server works again, though IDP is borked
2007-04-24T15:05:52 <ThomasWaldmann> dreimark: bug fixed see last commit
2007-04-24T15:12:32 <ThomasWaldmann> I just called the wrong plugin loader
2007-04-24T15:40:36 <johill> will you port all those things to 1.7 too or am I the only one using that right now and I should do it?
2007-04-24T15:50:37 <dreimark> ThomasWaldmann: :)
2007-04-24T15:51:52 <dreimark> johill:I usually do port for 1.7 too or backport to 1.6, but I am not currently using 1.7
2007-04-24T15:52:37 <dreimark> moin neagulm
2007-04-24T15:53:15 <johill> dreimark: ok :)
2007-04-24T15:53:28 <johill> we want to integrate openid auth right?
2007-04-24T15:54:27 <johill> what about botbouncer? should probably stay external, no?
2007-04-24T16:01:44 <neagulm> hello
2007-04-24T16:02:51 <johill> hi
2007-04-24T16:04:22 <dreimark> neagulm:I have to leave in a few minutes because of some trombone trainings, can you add some short words about the points we want to discuss later on ~5h with Skip on your page
2007-04-24T16:05:34 <neagulm> dreimark, ok, in 5 minutes
2007-04-24T16:05:52 <dreimark> very good
2007-04-24T16:08:43 <dreimark> bbl ~2h
2007-04-24T16:34:31 <ThomasWaldmann> johill: anyone can pull stuff from 1.6 and merge it into 1.7 (if noone else does, I'll do it from time to time)
2007-04-24T18:53:07 <dreimark> bb
2007-04-24T18:55:35 <dreimark> ThomasWaldmann: Because wikiutil.searchAndImportPlugin will never find the converter text_html_text_moin_wiki
2007-04-24T18:55:49 <dreimark> should we better remove converter from type2classname
2007-04-24T19:06:55 <grzywacz> moin
2007-04-24T19:14:35 <dreimark> moin grzywacz
2007-04-24T19:15:13 <grzywacz> hi dreimark :)
2007-04-24T19:46:20 <skipm> dremark,neagulm: hello
2007-04-24T19:48:17 <dreimark> hi skip
2007-04-24T19:56:15 <neagulm> hello skipm
2007-04-24T19:57:35 <skipm> I have a meeting at 20:00UTC, so I will have to exit a couple minutes early (sorry about the abrupt exit last time)
2007-04-24T19:58:16 <ThomasWaldmann> dreimark: yes (it does no harm, but is somewhat superfluous)
2007-04-24T20:01:42 <dreimark> skipm:may be I did now the time mistake, is it now 2pm CDT ?
2007-04-24T20:02:13 <lanius> hola
2007-04-24T20:02:26 <skipm> nope, 1pm. I just showed up early. I am not real irc savvy, so I figured I'd hang around while I work.
2007-04-24T20:03:58 <ThomasWaldmann> ehlo lanius :)
2007-04-24T20:04:53 <dreimark> ThomasWaldmann:We better remove it it was for me a mousetrap
2007-04-24T20:07:52 <dreimark> skipm: good, I hope you are familiar with the spam classification questions of neagulm ;)
2007-04-24T20:09:25 <neagulm> dreimark, question from an apprentice ;)
2007-04-24T20:10:22 <skipm> I saw the MarianNeagul/ProjectChat page update.
2007-04-24T20:12:06 <dreimark> skipm, neagulm: we can start we don't need to take care on the schedule or ?
2007-04-24T20:12:36 <skipm> i'm game - just going to dash to the refrigerator for a Coke... ;-)
2007-04-24T20:12:45 <dreimark> good idea
2007-04-24T20:13:13 <neagulm> I will be available in 5 minutes... :D
2007-04-24T20:13:40 <lanius> xorAxAx: so in our case the Session would be the ItemSupport
2007-04-24T20:13:57 <xorAxAx> lanius: do you really want to implement UOW? :)
2007-04-24T20:14:11 <xorAxAx> have a look at the code size in the sqlalchemy source code
2007-04-24T20:14:43 <lanius> i did, and i don't understand any bit there ;)
2007-04-24T20:15:00 <lanius> but saving a modified bit and comparing objects does not sound too difficult
2007-04-24T20:15:18 <xorAxAx> oh well
2007-04-24T20:15:22 <xorAxAx> its much more than that :)
2007-04-24T20:15:29 <lanius> what's more?
2007-04-24T20:15:32 <xorAxAx> because objects have child objects etc.
2007-04-24T20:15:40 <xorAxAx> ok, the relational model for moin is simpler
2007-04-24T20:15:52 <lanius> that's what i just wanted to say
2007-04-24T20:16:11 <lanius> we must do something like that since you don't want to save all revisions if you only edit one
2007-04-24T20:16:33 <xorAxAx> btw, i think it makes sense to have a plan about sped up metadata queries
2007-04-24T20:17:00 <xorAxAx> i.e. if it makes sense to design the apis to allow DBs to optimise some operations
2007-04-24T20:17:15 <xorAxAx> or if moin should do it on its own like it is currently
2007-04-24T20:17:19 <lanius> or just query all metadata by default
2007-04-24T20:17:34 <xorAxAx> its not about tat, its about selecting
2007-04-24T20:17:48 <lanius> go on
2007-04-24T20:17:51 <xorAxAx> i.e. something like "give me every page with pagelink to X"
2007-04-24T20:18:15 <xorAxAx> currently that is handled by a fulltext search through a custom file that needs to be stored somwhere as well
2007-04-24T20:18:28 <xorAxAx> (it could be seen as a cache, though)
2007-04-24T20:18:30 <neagulm> skipm, dreimark: i'm back
2007-04-24T20:19:02 <lanius> but we don't need that from the beginning on, it should be no problem to extend the interface for such queries, should it?
2007-04-24T20:19:16 <xorAxAx> no, it should fit into the API
2007-04-24T20:19:24 <xorAxAx> thats a very fundamental quetion
2007-04-24T20:19:31 <xorAxAx> if it should fit and how it should fit
2007-04-24T20:20:00 <lanius> give me all pages to which the user is subscribe and vice versa is another one
2007-04-24T20:20:10 <skipm> dreimark,neagulm: i'm here
2007-04-24T20:20:20 <xorAxAx> lanius: yes
2007-04-24T20:21:02 <lanius> still it can be seen as an extension to the basic api
2007-04-24T20:21:10 <xorAxAx> well, how?
2007-04-24T20:21:19 <lanius> the basic query functionality is by name
2007-04-24T20:21:26 <xorAxAx> the IStorageBcakend interface doesnt allow selecting based on page links :-)
2007-04-24T20:21:44 <lanius> later you can define additionally query functoinality, which of course also must be implemented in the IStorageBackend
2007-04-24T20:22:30 <dreimark> neagulm: skipm: currently I think we have to add the classification feature into Page/PageEditor
2007-04-24T20:22:38 <lanius> you have to add the functionality to the ItemSupport and to the StorageBackend, but you don't need to change old functionality, you can extend the current one
2007-04-24T20:23:21 <dreimark> later on if lanius has refactored storage engine it has to be ported to this
2007-04-24T20:23:34 <xorAxAx> lanius: the problem is that putting links into the storage bacend will give you a layering violation
2007-04-24T20:24:22 <lanius> yeah, two options, links handled as metadata or as fulltext search
2007-04-24T20:24:41 <skipm> dreimark: that's fine. you're the moin architecture expert.
2007-04-24T20:24:57 <neagulm> dreimark, skipm: As i said in the wiki: I think that this approach would allow us the use the feature in most parts of MM
2007-04-24T20:25:22 <xorAxAx> lanius: sure, they are some kind of non-primary metadat
2007-04-24T20:25:23 <xorAxAx> a
2007-04-24T20:25:31 <xorAxAx> i.e. one that can be deduced from the data
2007-04-24T20:25:46 <lanius> yes
2007-04-24T20:26:38 <dreimark> neagulm,skipm:the comment input should be scanned for spam too
2007-04-24T20:26:45 <lanius> and we could add methods to the StorageBackend to define such non-primary-metadata and the Backend then cares for efficient handling of it
2007-04-24T20:27:49 <xorAxAx> lanius: hah
2007-04-24T20:27:52 <skipm> dreimark: neagulm: definitely - Martin v. Loewis hacked that into the Python wiki's code - worked well to avoid blog spammers (whose content is ~100% in the comment)
2007-04-24T20:28:32 <lanius> xorAxAx: like indexes in ldap
2007-04-24T20:28:54 <neagulm> dreimark, All the text can be input for a classifier, even the attachments as long we can obtain pure text from them...
2007-04-24T20:29:44 <dreimark> :neagulm, very good so its your turn to ask whatever you want/need , we try to answer
2007-04-24T20:30:06 <lanius> xorAxAx: you define an index as regex, and the backend takes care to save a cache of pages that match this index. if you query for the index the backend can use the cache and must not search all pages again
2007-04-24T20:30:39 <xorAxAx> lanius: hmm
2007-04-24T20:31:04 <xorAxAx> lanius: can you write that down?
2007-04-24T20:31:15 <lanius> essentially you move the cache on a different layer
2007-04-24T20:31:35 <neagulm> dreimark, skipm: ok. For now we know where to hack the code...
2007-04-24T20:31:41 <skipm> neagulm: getting a pure text stream is no problem - you just subclass SB's tokenizer
2007-04-24T20:31:50 <neagulm> dreimark, skipm: but the primary question: the classifier should be supervised or unsupervised ?
2007-04-24T20:31:56 <lanius> xorAxAx: and make it useable for more than just page links
2007-04-24T20:32:21 <skipm> neagulm: define "supervised"?
2007-04-24T20:32:37 <lanius> xorAxAx: and this way it can be an extension to the basic api
2007-04-24T20:32:55 <lanius> xorAxAx: i'll try to write it down after the rest ;)
2007-04-24T20:33:30 <xorAxAx> :)
2007-04-24T20:33:40 <dreimark> neagulm: supervised - one has to control that feature?
2007-04-24T20:33:46 <neagulm> skipm, the user should train the system ?
2007-04-24T20:34:24 <lanius> xorAxAx: and lazy loading will be hidden in the properties as well?
2007-04-24T20:34:31 <neagulm> skipm, dreimark: the question is general to the text classification feature and not particular to spam filtering
2007-04-24T20:35:04 <neagulm> skipm, dreimark: and the second question: use just the text classifier or use a mix of several spam detection techniques ?
2007-04-24T20:35:05 <skipm> neagulm: i think the wiki admin(s) should train the system
2007-04-24T20:35:47 <xorAxAx> lanius: on which level? :)
2007-04-24T20:35:56 <skipm> neagulm: i have no experience mixing spambayes with other classification schemes - i wouldn't know how to "mix" them.
2007-04-24T20:36:00 <xorAxAx> i think on every level, yes
2007-04-24T20:36:05 <dreimark> skip, neagulm: the best would be to setup a group name for that
2007-04-24T20:36:07 <lanius> xorAxAx: item/revision/data/metadata level
2007-04-24T20:36:08 <xorAxAx> it doesnt make sense to load all revisions
2007-04-24T20:36:23 <xorAxAx> or load the page data if somebody wants to query the mimetype
2007-04-24T20:36:37 <lanius> yes
2007-04-24T20:36:45 <dreimark> skip, neagulm:so we can be variable who did it, on some wikis * known would be enough
2007-04-24T20:36:45 <neagulm> skipm, dreimark: I understand, so we speak about classifying in n-categories ( where n is a fixed number ? ) ?
2007-04-24T20:36:49 <lanius> and does lazy loading use the Backend directly or the ItemSupport
2007-04-24T20:36:58 <skipm> neagulm: dreimark: why not implement the underlying bits necessary in MM then implement one or more security policies (spambayes, spamcop, spamassassin, etc)?
2007-04-24T20:37:03 <xorAxAx> lanius: thats a good question
2007-04-24T20:37:31 <xorAxAx> i have no obvious answer that doesnt involve some kind of callback with another interface
2007-04-24T20:37:35 <neagulm> skipm, dreimark: I suggested this in the wiki
2007-04-24T20:39:42 <dreimark> skipm: I think the question is related more on if there are traps which could be ommited
2007-04-24T20:39:43 <neagulm> skipm, dreimark: The main problem is to chose the best design for this system... afterwards it can be used for any feature the MM Team would like
2007-04-24T20:39:58 <skipm> neagulm: I guess my point was that's where the "mixing" would take place - individual wiki owners could mix things together in different ways (though it's not obvious how pure classification would work, just accept/reject
2007-04-24T20:41:05 <lanius> xorAxAx: how is it done in sqlalchemy
2007-04-24T20:41:06 <neagulm> skipm, I agree
2007-04-24T20:41:36 <xorAxAx> lanius: objects have properties that are handled by mappers which establish the relation and call the lower level of sqlalchemy
2007-04-24T20:42:00 <dreimark> skipm: ok
2007-04-24T20:42:02 <xorAxAx> there is a query class that functions as a factory for those mapper-assisted user-objects
2007-04-24T20:42:19 <lanius> since we don't need mappers this means we would query the backend directly
2007-04-24T20:42:22 <xorAxAx> but there is also an extension that attaches the query-methods to the user-class itself
2007-04-24T20:42:48 <skipm> dreimark: neagulm: do you propose some sort of more general ClassificationPolicy to replace/augment the current SecurityPolicy?
2007-04-24T20:42:48 <xorAxAx> well well well
2007-04-24T20:43:22 <xorAxAx> note that sqlalchemy decorates your classes with the properties - the classes itself can be mostly agnostic to the lower level sqlalchemy interface (which is nice itself)
2007-04-24T20:43:48 <xorAxAx> the query factory attaches the code which knows about the lower levels
2007-04-24T20:44:14 <neagulm> skipm, dreimark: My idea was to remove all the details of interaction with external classifiers from SecurityPolicy and only provide the logic for blocking/allowing
2007-04-24T20:44:18 <xorAxAx> so ItemSupport could do so as well - dynamically write properties that do the access to the backend
2007-04-24T20:44:54 <skipm> neagulm: so n-way classification would be handled separately?
2007-04-24T20:44:55 <neagulm> skipm, dreimark: the inteaction with the classifiers should be done via calls to Page methods
2007-04-24T20:45:37 <neagulm> skipm, dreimark n-way classification and 2-way classification should both happen in Page/PageEditor
2007-04-24T20:46:13 <lanius> xorAxAx: yes but we don't have this separation, so the itemsupport attaches the backend to the item which uses the backend directly
2007-04-24T20:46:36 <neagulm> skipm, dreimark SecurityPolicy should only use the classification results from a page instance. Eg. PageInstance.getCategories() # -> ["SpamPages", ]
2007-04-24T20:49:03 <xorAxAx> lanius: i am looking forward to the UML diagram :)
2007-04-24T20:50:14 <skipm> neagulm: dreimark: irc seems to be the wrong channel to flesh this out, but I would be interested in seeing how a page edit/creation would work its way through the classification system - what it would acquire in the way of attributes, etc.
2007-04-24T20:52:08 <neagulm> skipm, dreimark: you mean a proof of concept ?
2007-04-24T20:52:33 <dreimark> neagulm: yep
2007-04-24T20:53:10 <neagulm> dreimark, skipm: ok
2007-04-24T20:53:16 <dreimark> neagulm: if its done in the wiki we can alter it there easily
2007-04-24T20:53:36 <skipm> neagulm: more of a design document - maybe just flesh out the bullet list in your ProjectChat subpage
2007-04-24T20:54:43 <neagulm> dreimark, skipm: I think that I can write a prof of concept using SB in a few day and provide a wiki page while writing it
2007-04-24T20:55:20 <skipm> neagulm: that would be cool
2007-04-24T20:55:27 <dreimark> :) :)
2007-04-24T20:55:40 <neagulm> dreimark, skipm: after that we could re-factor it or even remove it completely
2007-04-24T20:56:42 <dreimark> neagulm: I think it will be refactored but not removed
2007-04-24T20:57:30 <neagulm> dreimark, skipm: related to your remark: " i think the wiki admin(s) should train the system"...
2007-04-24T20:57:47 <skipm> neagulm: dreimark: carry on - work calls ...
2007-04-24T20:58:02 <dreimark> you are welcome
2007-04-24T20:59:20 <neagulm> dreimark, skipm: I think we should provide the wiki admin the ability to chose who can train the system
2007-04-24T21:00:08 <dreimark> neagulm: yeah and therefore we will add a new var to config.multiconfig
2007-04-24T21:00:38 <neagulm> dreimark, skipm It would be nice to have a wiki that has the ability to adapt to the users, give sens to untagged data
2007-04-24T21:01:10 <neagulm> dreimark, even something like the way "del.icio.us" handles his tags
2007-04-24T21:01:33 <neagulm> dreimark, but this is step 2. first to implement the spam detection :)
2007-04-24T21:02:03 <dreimark> neagulm: its not forbidden to think about ;)
2007-04-24T21:02:52 <neagulm> dreimark, need to leave for a 5 minutes, brb
2007-04-24T21:02:58 <dreimark> ok
2007-04-24T21:06:04 <dreimark> neagulm: what will be the reason for untagged data?
2007-04-24T21:08:52 <neagulm> dreimark, a user that creates a new page without adding it to a specific category.
2007-04-24T21:09:28 <neagulm> dreimark, even detect misclassified pages
2007-04-24T21:10:09 <skipm> neagulm: can you give me some classification examples? Is this like auto-generating the CategoryBlahBlahBlah stuff?
2007-04-24T21:10:28 <neagulm> dreimark, exactly
2007-04-24T21:11:12 <skipm> neagulm: so a page that got classified as "spam" would remain in the system just not be displayed somehow?
2007-04-24T21:11:12 <neagulm> dreimark, skipm well almost
2007-04-24T21:12:03 <dreimark> neagulm: so we should have a default category with a fillvalue, which will be replaced
2007-04-24T21:12:14 <neagulm> dreimark, skipm: automatic grouping of pages in a set of predefined categories
2007-04-24T21:12:41 <dreimark> skipm: neagulm for example if it is renamed to a prefix starting with a dot it is hidden
2007-04-24T21:13:18 <dreimark> on system level
2007-04-24T21:13:29 <neagulm> dreimark, skipm something similar to the suggestions of search engines: similar pages
2007-04-24T21:14:04 <neagulm> dreimark, skipm. not based on a specific keyword but on the content
2007-04-24T21:14:09 <skipm> dreimark: analogous to Unix (but maybe dot files would be viewable to Trusted or Admin users)?
2007-04-24T21:14:49 <dreimark> that needs to be hacked
2007-04-24T21:14:55 <lanius> xorAxAx: i uploaded a new UML diagram, which is essentially the same like yours, just nicer and without methods and attributes
2007-04-24T21:15:06 <lanius> xorAxAx: i will define the methods and attributes as python code like Thomas did
2007-04-24T21:15:30 <dreimark> skipm: the part to make it visible to some users.
2007-04-24T21:17:26 <neagulm> dreimark, skipm do you have some suggestions about nway classification frameworks ?
2007-04-24T21:17:35 <xorAxAx> lanius: ok
2007-04-24T21:18:47 <neagulm> skipm, dreimark: i was thinking about Chih-Chung Chang's libsvm (http://www.csie.ntu.edu.tw/~cjlin/libsvm/)
2007-04-24T21:19:08 <skipm> neagulm: the only one i'm familiar with is crm114 which you already know about. You could try asking on spambayes-dev@python.org or ask the crm114 author
2007-04-24T21:20:10 <neagulm> skipm, dreimark: libsvm has python bindings and was used successfully in some text classification and pattern recognition projects
2007-04-24T21:20:17 <skipm> neagulm: i'm not really familiar with anything other than sb - that's the extent of my classification experience. ;-)
2007-04-24T21:20:41 <dreimark> neagulm:did u checked for unicode support?
2007-04-24T21:22:28 <neagulm> dreimark, skipm: for libsvm it does not realy matter because we need to do the parsing and feature pruning. libsvm handles only the classification and it does it well
2007-04-24T21:22:51 <skipm> neagulm: since unicode support is a requirement, maybe do spam classification first then if you have time and need to you can implement an n-way classifier in python as at least a proof-of-concept
2007-04-24T21:25:48 <dreimark> neagulm:depends all on how fast you are and how complicated one part could become ;)
2007-04-24T21:25:59 <neagulm> dreimark, skipm: I will the text classification part for later, for know I must concentrate on spam filtering and hacking MM to provide the abilities to do that
2007-04-24T21:27:15 <neagulm> dreimark, skipm is right. The text classification might require to much time and testing and finally could be useless to MM users...
2007-04-24T21:28:26 <lanius> xorAxAx: an item copy can be realized as property operation as well, can't it?
2007-04-24T21:28:35 <dreimark> after the spam filter is working and its time left you can work on whatever you want
2007-04-24T21:29:00 <xorAxAx> lanius: hmm, not sure if that is very clean
2007-04-24T21:29:09 <dreimark> neagulm: and i guess if you will be familiar with MMs codebase its even easier to do the next step
2007-04-24T21:29:18 <lanius> you can create a copy of a python object, can't you?
2007-04-24T21:29:18 <xorAxAx> lanius: item copies need new names
2007-04-24T21:29:21 <xorAxAx> so you need a method
2007-04-24T21:29:26 <xorAxAx> yes, but why do you want to ?
2007-04-24T21:29:38 <lanius> good question, so we don't need it?
2007-04-24T21:30:07 <neagulm> dreimark, skipm I agree that spam detection is the main priority but we must prepare also for text classification, prepare the foundations (MoinMoin Foundations :D )
2007-04-24T21:30:27 <xorAxAx> lanius: i think i havent ever written code that copies python objects excepts for a few single times
2007-04-24T21:31:04 <lanius> xorAxAx: the question is if we need to copy Pages, Attachements, Users
2007-04-24T21:32:45 <xorAxAx> no
2007-04-24T21:32:46 <dreimark> neagulm: you can add another proposal for that but don't spend to much time now on that
2007-04-24T21:33:25 <lanius> xorAxAx: that's good
2007-04-24T21:33:29 <neagulm> dreimark, I agree again :)
2007-04-24T21:42:44 <lanius> xorAxAx: I added a description of the classes in textual form, please take a look if something is missing
2007-04-24T21:42:58 <xorAxAx> ok, i will have a lok
2007-04-24T21:42:59 <xorAxAx> look
2007-04-24T21:48:21 <dreimark> lanius: xorAxAx: the new concept needs a flag to hide an item
2007-04-24T21:48:34 <xorAxAx> dreimark: "hide"?
2007-04-24T21:48:56 <dreimark> make it invisible for the common user because it is a spam page
2007-04-24T21:49:10 <dreimark> and needs to be checked to get visible again
2007-04-24T21:49:19 <xorAxAx> thats unrelated to the storage system
2007-04-24T21:49:23 <dreimark> good
2007-04-24T21:49:27 <lanius> it's metadata
2007-04-24T21:49:43 <dreimark> wanted to know that :)
2007-04-24T21:50:10 <neagulm> dreimark, i think that tomorow I will be unable to join #moin-dev
2007-04-24T21:51:07 <dreimark> neagulm, if you work on preparing the proposal and don't have questions about its ok ;)
2007-04-24T21:52:19 <dreimark> you spoke from few days needing for that. Will I see a version before Friday?
2007-04-24T21:53:29 <dreimark> neagulm: ?
2007-04-24T21:53:53 <neagulm> dreimark, other problems: between 15-16 May I will be away for a distributed computing seminar in another town (Cluj) and 20-22 I will be in bucharest for a IBM seminar. In bucharest i have also a meeting with BitDeffenders antispam people.
2007-04-24T21:54:40 <neagulm> dreimark, I hope that until Friday I finish...
2007-04-24T21:54:44 <dreimark> that sounds interesting
2007-04-24T21:55:25 <neagulm> dreimark, but the most probable day would be Sunday
2007-04-24T21:55:58 <dreimark> neagulm: I know, I believe moin-dev will be quite silent
2007-04-24T21:55:58 <skipm> dreimark: neagulm: sorry, i've been distracted and now it's time for my 3pm meeting.
2007-04-24T21:56:29 <dreimark> skipm: ok, thanks for helping !
2007-04-24T21:56:36 <skipm> dreimark: neagulm: send me an email when you want to chat again...
2007-04-24T21:56:41 <neagulm> dreimark, I really want to speak with the BitDeffender friends because over summer they will be at Redmond, MS internship
2007-04-24T21:58:50 <dreimark> neagulm, may be we can get some new ideas from them ;)
2007-04-24T22:00:37 <neagulm> dreimark, skipm is a SpamBayes developer ?
2007-04-24T22:04:46 <dreimark> neagulm:yeah, he is
2007-04-24T22:06:35 <dreimark> http://sourceforge.net/projects/spambayes/index.html
2007-04-24T22:21:22 <dreimark> neagulm:Page._listPageInPath shows the hiding og pages starting with a dot
2007-04-24T22:21:33 <dreimark> of
2007-04-24T22:23:46 <neagulm> is it ok if I clone parts of the logs in a secondary wiki page?
2007-04-24T22:25:37 <dreimark> yeah because they will always be for public access e.g. MoinMoinChat/Logs/moin-dev/2007-04-23
2007-04-24T22:40:32 <lanius> xorAxAx: ThomasWaldmann: I added more stuff to the interface definitions
2007-04-24T22:40:56 <xorAxAx> ok
2007-04-24T22:43:34 <dreimark> good night
2007-04-24T22:43:39 <lanius> sleep well dreimark
2007-04-24T22:57:42 <neagulm> good night
2007-04-24T23:00:05 <dreimark> cu
MoinMoin: MoinMoinChat/Logs/moin-dev/2007-04-24 (last edited 2007-10-29 19:09:02 by localhost)
