MoinMoin: MoinMoinChat/Logs/moin-dev/2007-05-04

2007-05-04T13:46:42  <lanius> hello
2007-05-04T14:04:40  <johill> hi
2007-05-04T14:17:58  <ThomasWaldmann> moin
2007-05-04T14:25:41  <flowhase> moin
2007-05-04T15:18:55  <lanius> xorAxAx: ping
2007-05-04T15:19:42  <xorAxAx> pong
2007-05-04T15:19:58  <xorAxAx> lanius:
2007-05-04T15:20:12  <lanius> let's talk about your comments
2007-05-04T15:20:30  <lanius> Data() is the Data Proxy class
2007-05-04T15:21:00  <xorAxAx> no, its an object that implements a file-like interface with the methods specified in the data proxy interface :)
2007-05-04T15:21:20  <lanius> yes
2007-05-04T15:21:31  <lanius> is there any problem with this?
2007-05-04T15:23:07  <lanius> or just that it wasn't commented
2007-05-04T15:24:01  <xorAxAx> it was not obvious, but its fine
2007-05-04T15:24:13  <lanius> ok
2007-05-04T15:24:31  <lanius> so you would propose to drop the __setitem__ method in the dict
2007-05-04T15:24:41  <lanius> and use an add() method or something like that?
2007-05-04T15:30:48  <johill> lanius: you said: johill: that's what we have, item operations on items, metadata operations on metadata and data operations on data, but you all need to map those to a backend, but that is code you never touch
2007-05-04T15:31:29  <johill> like a few days ago :)
2007-05-04T15:31:32  <lanius> yes
2007-05-04T15:31:39  <johill> I don't quite understand what you mean
2007-05-04T15:32:11  <lanius> in the design you have four classes which are important for the Moin code, that are Item, Revision, Data and Metadata
2007-05-04T15:32:17  <johill> right
2007-05-04T15:32:25  <lanius> with the help of this classes you get access to everything you need
2007-05-04T15:32:38  <johill> ok so say I want to delete an item
2007-05-04T15:32:46  <lanius> and ItemCollection
2007-05-04T15:33:12  <johill> what do I do to delete an item?
2007-05-04T15:33:15  <lanius> deleting an item is like deleting an item from the ItemCollection dict
2007-05-04T15:33:40  <lanius> and calling ItemCollection.save() afterwards
2007-05-04T15:33:47  <johill> so what is it about StorageBackend?
2007-05-04T15:34:01  <johill> ah
2007-05-04T15:34:02  <johill> I see now
2007-05-04T15:34:05  <johill> ok, hmm
2007-05-04T15:34:08  <lanius> ItemCollection then looks to which StorageBackend this item belongs and does the delete operations there
2007-05-04T15:34:17  <johill> ok, gotcha
2007-05-04T15:34:20  <johill> I wouldn't do it that way I guess
2007-05-04T15:34:25  <lanius> how would you do it
2007-05-04T15:34:37  <johill> well I'd have ItemCollection be an abstract class
2007-05-04T15:34:43  <johill> and not define a StorageBackend at all
2007-05-04T15:35:06  <johill> so when you have say sql storage it provides a SQLItemCollection that is created when you want an item
2007-05-04T15:35:19  <johill> and maybe that calls back into some internal SQLStorageBackend, but that's internal
2007-05-04T15:35:38  <lanius> but the thing is that you not only have one backend, but you can have multiple, like pages and underlay
2007-05-04T15:35:38  <johill> iow, I wouldn't define StorageBackend at all since it's the internal call back api
2007-05-04T15:35:44  <johill> sure
2007-05-04T15:35:56  <johill> but that's just a special CombiningItemCollection
2007-05-04T15:36:14  <johill> or StackedItemCollection
2007-05-04T15:36:36  <lanius> but you would have to implement the dictionary methods for all Backends
2007-05-04T15:36:54  <johill> yes
2007-05-04T15:37:05  <lanius> you only must implement them once in this design
2007-05-04T15:37:22  <johill> true
2007-05-04T15:37:53  <lanius> and who would take care which ItemCollection is used
2007-05-04T15:38:07  <johill> well that's a configuration issue anyway
2007-05-04T15:38:12  <lanius> in this case the ItemCollection looks at the config and uses the defined backend
2007-05-04T15:38:17  <johill> who tells you which backend to use :)
2007-05-04T15:38:29  <lanius> how would it work your way
2007-05-04T15:38:33  <johill> well
2007-05-04T15:38:37  <johill> I'd have
2007-05-04T15:38:50  <johill> cfg.item_collection = FileItemCollection()
2007-05-04T15:38:56  <johill> or if you want underlay maybe
2007-05-04T15:39:16  <johill> cfg.item_collection = StackedItemCollection(FileItemCollection(), FileItemCollection(underlay=True))
2007-05-04T15:39:41  <lanius> but you would mix the storage and the moin layer this way, my design has a clear separation between both
2007-05-04T15:41:07  <johill> maybe
2007-05-04T15:41:21  <johill> how would you implement virtual pages with yours?
2007-05-04T15:41:30  <lanius> what are virtual pages
2007-05-04T15:41:46  <johill> basically a backend that doesn't allow you to save things and just pulls them from some other source
2007-05-04T15:41:59  <johill> and takes part of the namespace, say /vp/*
2007-05-04T15:42:21  <johill> or how would you configure underlay?
2007-05-04T15:42:36  <lanius> you have some backends that implement StorageBackend
2007-05-04T15:42:51  <lanius> and you have the DispatchingBackend which is configured for namespaces, underlay, ...
2007-05-04T15:43:04  <lanius> the dispatching backend itself uses the other backends
2007-05-04T15:43:56  <johill> so you just stuff it all into the backend
2007-05-04T15:44:10  <lanius> into different backends
2007-05-04T15:44:16  <johill> right
2007-05-04T15:44:38  <johill> so you'd have a StackedBackend instead of StackedItemCollection I guess
2007-05-04T15:44:48  <lanius> yes, that's the DispatchingBackend
2007-05-04T15:45:02  <johill> ah right, it seems much more generic
2007-05-04T15:45:08  <lanius> yes
2007-05-04T15:45:17  <johill> are you sure that genericity is worth it at that point?
2007-05-04T15:45:26  <lanius> i don't see the downtakes
2007-05-04T15:45:31  <johill> code complexity
2007-05-04T15:45:51  <xorAxAx> lanius: a moment please
2007-05-04T15:45:52  <lanius> i don't think it's more complex, i even think it will be easier to understand
2007-05-04T15:47:23  <johill> dunno. I'd think that having just a single StackedBackend/ItemCollection would be easier to understand than something with arbitrary namespaces ;)
2007-05-04T15:47:54  <johill> and along with NamespaceBackend that takes part of the namespce and maps it to another backend you can do whatever you can do with the dispatcher
2007-05-04T15:48:20  <johill> up to you, I don't want to talk you into anything, just trying to understand why you do it this way
2007-05-04T15:48:20  <xorAxAx> lanius: i will be back in approx. one hour
2007-05-04T15:48:22  <xorAxAx> lanius: is that ok=
2007-05-04T15:48:25  <lanius> sure
2007-05-04T15:48:38  <lanius> johill: that's ok
2007-05-04T15:49:34  <johill> lanius: I think it'd help if you documented the return types for the functions
2007-05-04T15:50:05  <lanius> yes
2007-05-04T15:50:19  <lanius> the advantage of the ItemCollection is that it implements all saving logic
2007-05-04T15:50:34  <johill> there isn't really much logic there htough
2007-05-04T15:50:41  <lanius> it checks what needs to be saved and uses the Backends to call the correct methods
2007-05-04T15:50:44  <lanius> don't think that
2007-05-04T15:50:56  <johill> just backend.something, no?
2007-05-04T15:51:12  <lanius> no, save data, save metadata, which revision changed ...
2007-05-04T15:51:20  <lanius> in your case every backend would have to implement this logic
2007-05-04T15:51:44  <johill> yeah, I suppose I'd push .save() down into the item
2007-05-04T15:51:58  <johill> i.e. if you change an item, you need to .save it
2007-05-04T15:52:07  <lanius> but how would the item then know where to save it
2007-05-04T15:52:28  <johill> well, it'd internally probably call its item collection
2007-05-04T15:52:49  <johill> but from the outside I don't care
2007-05-04T15:53:41  <lanius> so you would need methods like the one defined in the backend for saving, right
2007-05-04T15:53:58  <johill> depends
2007-05-04T15:54:13  <johill> virtual page stuff wouldn't care, it just returns an error on .save
2007-05-04T15:54:18  <lanius> but real stuff
2007-05-04T15:54:51  <johill> again, depends, if you have sql where you can trivially update one revision then you could put the relevant sql for that into .save directly
2007-05-04T15:55:11  <lanius> but you must be independent from the bakend, the item does not care if it is sql or file
2007-05-04T15:55:37  <johill> no no, in my world the Item is an instance of SQLItem or FileItem
2007-05-04T15:55:47  <johill> depending on where you got it from
2007-05-04T15:55:58  <lanius> and what about new items?
2007-05-04T15:56:32  <johill> I'd have something like itemcollection.new_item(name) or something
2007-05-04T15:57:06  <johill> in fact, that'd be about all my item collection does ;)
2007-05-04T15:57:53  <lanius> i think i understand, will think about it
2007-05-04T15:57:59  <johill> btw
2007-05-04T15:58:04  <johill> where's the difference in list_items and keys?
2007-05-04T15:58:33  <lanius> keys lists all items, list_items takes filtering arguments
2007-05-04T15:58:46  <lanius> but i will move this to keys with default arguments
2007-05-04T15:58:51  <johill> ah ok
2007-05-04T16:04:11  <johill> I'm just putting some things onto JohannesBerg/StorageRefactoringIdeas
2007-05-04T16:06:27  <lanius> that's good
2007-05-04T16:07:55  <johill> just saved for a first version
2007-05-04T16:13:05  <johill> and now added a draft for how a stacked collection would work
2007-05-04T16:14:56  <lanius> will take a look later, will go shopping now
2007-05-04T16:16:05  <johill> I'll be around for the next hour or so if you have any questions
2007-05-04T16:16:18  <johill> after that I'll be back after 9pm or so
2007-05-04T16:17:19  <lanius> then i think we'll talk again tomorrow
2007-05-04T16:17:43  <johill> just leave comments on the wiki page if I'm not around
2007-05-04T16:17:50  <lanius> yeah
2007-05-04T16:21:58  <johill> btw
2007-05-04T16:22:06  <ThomasWaldmann> btw, for help and systempages we maybe do not need underlay any more as soon as we have metadata working
2007-05-04T16:22:08  <johill> if you do it my way you can *still* implement the backend stuff ;)
2007-05-04T16:22:53  <ThomasWaldmann> (but some people want it for other stuff, esp. for common content in a set of farm wikis)
2007-05-04T16:23:17  <johill> yeah
2007-05-04T16:25:24  <xorAxAx> ThomasWaldmann: how does "metadata" solve that? :)
2007-05-04T16:25:41  <xorAxAx> lanius: re
2007-05-04T16:28:07  <ThomasWaldmann> we just can select on the metadata instead of the storage layer
2007-05-04T16:32:19  <xorAxAx> ?
2007-05-04T16:32:37  <xorAxAx> how is that related to the reason why underlay exists at all=
2007-05-04T16:32:40  <johill> doesn't really matter since you still want stacked stuff anyway
2007-05-04T17:06:33  <lanius> xorAxAx: re
2007-05-04T17:22:26  <lanius> xorAxAx: ThomasWaldmann what do you think about johill's ideas
2007-05-04T17:23:15  <xorAxAx> lanius: well, your idea about having to reimplement page saving etc. is a pretty strong argument
2007-05-04T17:24:13  <lanius> this logic could be implemented in the abstract class
2007-05-04T17:24:43  <xorAxAx> well
2007-05-04T17:25:06  <xorAxAx> thats no strong separatation, though
2007-05-04T17:25:16  <xorAxAx> did johill point out any disadvantages about the current design?
2007-05-04T17:25:33  <lanius> too complex was his point
2007-05-04T17:26:10  <xorAxAx> hmm, i disagree
2007-05-04T17:27:43  <lanius> i think basicly his idea is that we extend the DataBackend stuff to Revision, Metadata and Item level
2007-05-04T17:28:11  <xorAxAx> hmm?
2007-05-04T17:28:38  <lanius> we have FileData and SQLData clases now
2007-05-04T17:28:53  <xorAxAx> ???
2007-05-04T17:29:00  <lanius> for the data handling stuff
2007-05-04T17:29:06  <lanius> the file like interface
2007-05-04T17:29:10  <xorAxAx> ah, well
2007-05-04T17:29:30  <xorAxAx> thats really just a minor detail
2007-05-04T17:29:34  <lanius> and we could do the same for Metadata, Revision and Item stuff
2007-05-04T17:29:55  <xorAxAx> that doesnt make any sense IMHO
2007-05-04T17:30:21  <lanius> if you look at the diagramm it looks weird somehow to have it only for Data
2007-05-04T17:30:21  <xorAxAx> esp. it will be much more difficult to design a clean stable api
2007-05-04T17:30:26  <xorAxAx> not at all
2007-05-04T17:30:48  <xorAxAx> well, you dont need the class
2007-05-04T17:30:51  <lanius> why would the api differ
2007-05-04T17:30:56  <xorAxAx> you can delegate via storagebackend methods
2007-05-04T17:31:14  <xorAxAx> its just important that somewhere, you get a file like interface at the top of the chain
2007-05-04T17:31:19  <xorAxAx> to be pythonic
2007-05-04T17:33:22  <lanius> where would you implement access control?
2007-05-04T17:34:15  <xorAxAx> in the revision class
2007-05-04T17:34:29  <xorAxAx> or the item class
2007-05-04T17:34:41  <lanius> not the collection?
2007-05-04T17:35:00  <xorAxAx> no
2007-05-04T17:35:27  <lanius> no, you can't access the metadata there
2007-05-04T17:35:34  <lanius> but how would you implement it at the other layr
2007-05-04T17:35:42  <xorAxAx> ?
2007-05-04T17:36:18  <lanius> currently you call Item[000001].data.read(), but there is no user checking involved
2007-05-04T17:36:26  <johill> xorAxAx: I think pushing it all back into the storage backend by design is unnatural
2007-05-04T17:36:37  <xorAxAx> johill: hmm?
2007-05-04T17:36:40  <xorAxAx> johill: what?
2007-05-04T17:36:52  <johill> xorAxAx: it seems that really only the stuff that the moin code interacts with like Item/ItemCollection needs to be defined
2007-05-04T17:37:01  <xorAxAx> lanius: well, the data property will check for read access
2007-05-04T17:37:05  <johill> and then the rest is an implementation detail of how the backend works
2007-05-04T17:37:18  <lanius> xorAxAx: where does the data property get the current user from
2007-05-04T17:37:39  <johill> lanius is of course right in that then all code for a new storage needs to declare a new item subclass etc.
2007-05-04T17:37:48  <xorAxAx> lanius: thats a good question, unrelated to the current class discussion, though
2007-05-04T17:37:56  <xorAxAx> lanius: add it to a todo list and we will solve it later
2007-05-04T17:38:21  <lanius> so you think there will be some kind of session instead of using a parameter to set the user
2007-05-04T17:38:27  <johill> and I'm not saying that there couldn't be a generic item class that does exactly what lanius proposes
2007-05-04T17:38:29  <xorAxAx> johill: well, how do you want to make backends replacable if there is no fixed interface?
2007-05-04T17:38:43  <xorAxAx> lanius: no, a session is not needed here
2007-05-04T17:38:45  <johill> xorAxAx: JohannesBerg/StorageRefactoringIdeas
2007-05-04T17:39:01  <xorAxAx> lanius: thats a question about the initialisation of the item object
2007-05-04T17:39:32  <lanius> so you would rather do an ItemCollection.get("Page", user)
2007-05-04T17:39:36  <lanius> and checking is done on access
2007-05-04T17:40:00  <xorAxAx> lanius: no
2007-05-04T17:40:21  <xorAxAx> lanius: just assume that the collection knows the user
2007-05-04T17:40:40  <xorAxAx> e.g. it could be done by saying request.storage["Page"]
2007-05-04T17:41:06  <lanius> but it's the ItemCollection that knows about the user
2007-05-04T17:41:31  <xorAxAx> johill: well, thats pretty unusable because we have a lot common code. you would need to hash out the interface more explictly
2007-05-04T17:41:41  <xorAxAx> johill: that would be handled by the abstract base class
2007-05-04T17:41:46  <johill> xorAxAx: hm?
2007-05-04T17:41:54  <xorAxAx> like e.g. page creation
2007-05-04T17:42:05  <xorAxAx> lanius: so?
2007-05-04T17:42:13  <xorAxAx> lanius: storage is an ItemCollection instance of course
2007-05-04T17:42:43  <johill> I have to go in a minute or two
2007-05-04T17:42:57  <lanius> xorAxAx: that was just a question
2007-05-04T17:43:05  <xorAxAx> lanius: what? :)
2007-05-04T17:43:14  <lanius> nothing :D
2007-05-04T17:43:14  <johill> I think the main difference is that with lanius's design everything is basically keyed off the page name etc, while with my design a lot more things are implicit by the class
2007-05-04T17:43:39  <xorAxAx> johill: what do you mean by "by the class"?
2007-05-04T17:43:58  <johill> well, I'd have an SQLItem() and an FileStorageItem for example
2007-05-04T17:44:13  <johill> hm
2007-05-04T17:44:18  <xorAxAx> and everyone of them should reimplement storage, acl checking etc.?
2007-05-04T17:44:19  <johill> somewhat like my session code
2007-05-04T17:44:35  <xorAxAx> if not, please add some interface that they are able to call to delegate that work
2007-05-04T17:44:43  <johill> yeah, I guess I haven't thought about acl checking
2007-05-04T17:44:48  <johill> I didn't think it'd be at this point
2007-05-04T17:45:00  <xorAxAx> well, then you would need another wrapper
2007-05-04T17:45:10  <xorAxAx> we want it in the model/data layer this time
2007-05-04T17:45:23  <johill> I guess the main difference is that you're modeling the data layer as one backend class
2007-05-04T17:45:26  <xorAxAx> lanius: can you please continue to comment on my ideas? :)
2007-05-04T17:45:37  <johill> while I prefer to model it as the collection of classes you actually operate on
2007-05-04T17:45:48  <lanius> xorAxAx: what about the __setitem__ method
2007-05-04T17:45:50  <johill> whether it's then a BackendItem() that calls back into a StorageBackend doesn't matter much
2007-05-04T17:46:52  <xorAxAx> lanius: describe the sequence of calls that would be done on page save
2007-05-04T17:46:59  <xorAxAx> using your current meta model
2007-05-04T17:47:08  <lanius> i understand the objection you have, i just want to know what the alternative is
2007-05-04T17:47:20  <johill> anyway, I better go
2007-05-04T17:47:51  <xorAxAx> lanius: note that my objection is none if the Item class code calls the itemcollection method
2007-05-04T17:47:54  <xorAxAx> johill: :-)
2007-05-04T17:48:14  <xorAxAx> lanius: so i want some input about your current idea of saving a page first (i just realised)
2007-05-04T17:48:33  <lanius> col[item.name] = item
2007-05-04T17:49:31  <xorAxAx> well, please including item creation etc.
2007-05-04T17:49:41  <lanius> item = Item()
2007-05-04T17:51:30  <xorAxAx> where is the name? where is the page content? :)
2007-05-04T17:51:42  <xorAxAx> where is the new revision part :)
2007-05-04T17:52:17  <lanius> item.name = "bla"; item[1] = new Revision() ...
2007-05-04T17:53:50  <lanius> that's not good
2007-05-04T17:54:00  <lanius> alternatively you could do a ItemCollection.newItem("Name") method
2007-05-04T17:54:07  <xorAxAx> if the caller should handle the collection registration (as opposed to an item class that would do it automatically, like with sqlalchemy/assignmapper), you would need an add() method, yes
2007-05-04T17:54:17  <xorAxAx> or a method on item
2007-05-04T17:55:07  <lanius> method on item?
2007-05-04T17:55:23  <xorAxAx> Item.register_with_collection(col)
2007-05-04T17:55:31  <lanius> ah, hmm
2007-05-04T17:55:39  <xorAxAx> but note that both solutions are stupid if you want to have a working col.save
2007-05-04T17:55:39  <lanius> how does sqlalchemy do it?
2007-05-04T17:55:54  <xorAxAx> well, two solutions:
2007-05-04T17:56:44  <xorAxAx> a) you use assignmappr. then its like i=Item("foo"); col.save(); done! b) you dont use it. then its session.save(Item("foo"))
2007-05-04T17:57:08  <xorAxAx> i think something assignmapper-like would be nice
2007-05-04T17:57:32  <xorAxAx> also note that you need to do the data management from collection (incl. decoration of properties)
2007-05-04T17:57:53  <xorAxAx> in order to copy the attributes to the backup dict to compare them later for modifications
2007-05-04T17:58:26  * xorAxAx added this issue to the list at the bottom
2007-05-04T17:59:11  <lanius> and what about my proposal with ItemCollection.newItem("Name")
2007-05-04T17:59:19  <lanius> that automatically registers and creates a nice structure
2007-05-04T18:02:12  <xorAxAx> thats ok as well
2007-05-04T18:02:41  <xorAxAx> junger mann starb durch schnüffeln von deo-spray in giffhorn
2007-05-04T18:02:58  <xorAxAx> s/ff/f/
2007-05-04T18:05:12  <lanius> for locking it should be enough to be an Item property which is just available at runtime, no saving needed, right
2007-05-04T18:05:42  <xorAxAx> ?
2007-05-04T18:05:52  <xorAxAx> the same code is running in multiple threads and processes
2007-05-04T18:06:00  <xorAxAx> you need to synchronize via the backend
2007-05-04T18:07:51  <lanius> it's not enough to synchronize via the ItemCollection?
2007-05-04T18:08:06  <xorAxAx> well, the locks needs to be propagated into the backend
2007-05-04T18:08:25  <xorAxAx> because thats the only way of interprocess communication
2007-05-04T18:08:26  <lanius> but if ItemCollection is a singleton
2007-05-04T18:08:33  <xorAxAx> there are multiple processes
2007-05-04T18:08:58  <xorAxAx> or multiple threads ... those are things we cant influence
2007-05-04T18:09:04  <xorAxAx> depends on the deployment
2007-05-04T18:10:07  <lanius> i see, then it could be implemented as metadata key on the current revision?
2007-05-04T18:10:35  <xorAxAx> it needs to be an item-level lock
2007-05-04T18:10:51  <lanius> yes but the item can check on the metadata key of the current revision
2007-05-04T18:10:56  <xorAxAx> i dont see how you can implement a semaphore using an item-level lock :)
2007-05-04T18:11:03  <xorAxAx> s/item/rev/
2007-05-04T18:11:09  <lanius> this way we won't need to add new methods to the backend
2007-05-04T18:11:25  <xorAxAx> sketch the locking scheme and i will break it if necessary
2007-05-04T18:11:27  <xorAxAx> (in the wiki)
2007-05-04T18:11:35  <lanius> :)
2007-05-04T18:12:08  <xorAxAx> :-)
2007-05-04T18:15:45  <lanius> about the name and revision, Thomas proposed to store it as metadata
2007-05-04T18:16:55  <xorAxAx> well
2007-05-04T18:17:13  <xorAxAx> how do you want to implement the storage backend for the current data format? :)
2007-05-04T18:17:31  <xorAxAx> do you want to have some fixed meta-data keys?
2007-05-04T18:17:34  <lanius> haven't thought too much about it
2007-05-04T18:17:46  <xorAxAx> that are shared on all layers
2007-05-04T18:18:15  <lanius> that's what the metadata list is about, not`?
2007-05-04T18:18:27  <xorAxAx> well, somehow, yes
2007-05-04T18:24:08  <lanius> must a revision know about itself
2007-05-04T18:24:14  <lanius> about her number or id
2007-05-04T18:24:59  <xorAxAx> i would say yes
2007-05-04T18:25:18  <lanius> implementing the name as metadata would mean a move operation is just a name change
2007-05-04T18:25:54  <xorAxAx> well, how would you map that to the old data storage?
2007-05-04T18:26:03  <lanius> dunno
2007-05-04T18:26:05  <xorAxAx> hmm
2007-05-04T18:26:18  <xorAxAx> you could say that the old data can only be read using the backend
2007-05-04T18:26:27  <xorAxAx> and then you can convert it
2007-05-04T18:26:44  <lanius> yes, that would be possible
2007-05-04T18:26:44  <xorAxAx> i think that assumption is ok
2007-05-04T18:29:46  <lanius> ok, will add changes, have to leave now, bye
2007-05-04T18:31:47  <xorAxAx> c u
2007-05-04T18:58:50  <ThomasWaldmann> xorAxAx: tja, da siehst Du mal, wie wichtig das richtige Deo ist :D
2007-05-04T18:59:17  <xorAxAx> :-)
2007-05-04T19:00:35  <ThomasWaldmann> btw, if you talk about locking, make sure you are precise about which lock you talk
2007-05-04T19:01:31  <ThomasWaldmann> there are "high level" editor locks (so that the 2nd editor gets warned that the page is already being edited) as well as "low level" locks caring for consistent data structure changes
2007-05-04T19:01:34  <xorAxAx> yeah, moin has 3 different locking kinds
2007-05-04T19:02:04  <ThomasWaldmann> high, low and missing? <g>
2007-05-04T19:02:40  <xorAxAx> current-locks, util.locking-locks, edit-lock-locks
2007-05-04T19:03:30  <ThomasWaldmann> but the first 2 ones are only implementation details
2007-05-04T19:09:34  <xorAxAx> well, i dont know where the second one is used
2007-05-04T19:11:01  <ThomasWaldmann> btw, we maybe could also make use of a wiki wide lock, e.g. for consistent online backups while the wiki is r/o for a short time
2007-05-04T21:09:08  <dreimark> moin
2007-05-04T21:14:17  <ThomasWaldmann> hi dreimark
2007-05-04T21:15:51  <johill> lanius, xorAxAx: I'm not sure that collection.save should save pages, wouldn't item.save make much more sense?
2007-05-04T21:17:20  <xorAxAx> johill: he was planning to implement the UOW pattern
2007-05-04T21:17:50  <johill> uow?
2007-05-04T21:18:15  <johill> google says "university of wollongong" ;)
2007-05-04T21:19:40  <xorAxAx> unit of work
2007-05-04T21:19:50  * johill googles again
2007-05-04T21:20:59  <johill> hm. don't have that book
2007-05-04T21:21:18  <xorAxAx> :)
2007-05-04T21:21:40  <johill> ok, I think I understand
2007-05-04T21:21:58  <johill> does that make sense? do we often change multiple pages in one transaction?
2007-05-04T21:22:47  <xorAxAx> no idea :)
2007-05-04T21:23:09  <xorAxAx> he hasnt yet described how he wants to faciliate that pattern
2007-05-04T21:23:17  <xorAxAx> its on my questions list
2007-05-04T21:24:34  <johill> I guess I'm ambivalent. I like it when you do the operation on the object you're working with
2007-05-04T21:24:52  <johill> what he's proposing is that you modify an item, add revisions and do all the other stuff and then you need to go back to the collection to save the whole lot
2007-05-04T21:24:56  <johill> doesn't seem too intuitive
2007-05-04T21:25:05  <johill> (plus you can't just pass items around, always need both)
2007-05-04T21:25:36  <xorAxAx> johill: ?
2007-05-04T21:25:50  <johill> what was unclear about that? :)
2007-05-04T21:26:26  <xorAxAx> why do you need "both"?
2007-05-04T21:26:42  <xorAxAx> the save would be called "commit"
2007-05-04T21:26:47  <xorAxAx> which is not unintuitive at all
2007-05-04T21:27:17  <johill> you need both the Item and the ItemCollection to do anything useful
2007-05-04T21:27:18  <xorAxAx> (even though there would be no rollback :))
2007-05-04T21:27:31  <johill> hey! if there's no rollback what's the point?
2007-05-04T21:27:46  <xorAxAx> you dont need to pass the collection around
2007-05-04T21:27:49  <xorAxAx> no idea :)
2007-05-04T21:27:54  <xorAxAx> it sounds nice
2007-05-04T21:28:04  <johill> as in "bla pattern bla" ;)
2007-05-04T21:28:11  <xorAxAx> yeah
2007-05-04T21:28:29  <johill> ok what can I do w/o the collection though?
2007-05-04T21:28:41  <johill> if I'm handed an item and add revisions etc. who saves it?
2007-05-04T21:28:50  <johill> would the upstream code that gave me the item call commit?
2007-05-04T21:29:02  <johill> (and if it does again, what's the point?)
2007-05-04T21:29:09  <johill> s/does again/does, again/
2007-05-04T21:29:34  <xorAxAx> i think UOW is indeed a bit problematic here, esp. because of missing atomic transactions
2007-05-04T21:30:14  <johill> from a quick google search uow seems *only* useful with transactions and rollback support
2007-05-04T21:30:26  <johill> in fact, it seems that that's the whole point of it :)
2007-05-04T21:31:07  <xorAxAx> add it to the page in my section with your name at the end :)
2007-05-04T21:40:29  <johill> oh btw
2007-05-04T21:40:41  <johill> shouldn't metadata keys be split up into "per revision" and "per item" metadata?
2007-05-04T21:41:25  <johill> (per-item would be deleted, locked, subscribers, ... while userid, mtime, ... would be per-reivision)
2007-05-04T21:42:29  <xorAxAx> yeah
2007-05-04T21:42:53  <johill> maybe I should add a comments by me category and add some notes there ;)
2007-05-04T21:45:08  <xorAxAx> yes :)
2007-05-04T21:48:53  <johill> doing that, you'll be getting lots of mail ;)
2007-05-04T21:51:34  <johill> what exactly is the data backend btw?
2007-05-04T21:52:08  <johill> What is "Data()" in the Revision class?
2007-05-04T21:52:09  <xorAxAx> its a file-like adapter to the blob in the db or the file on the disk
2007-05-04T21:52:13  <johill> -> I think that's meant to be DataBackend()
2007-05-04T21:52:24  <xorAxAx> he wanted to change that
2007-05-04T21:52:38  <johill> why is the revision itself not just a file-like object?
2007-05-04T21:52:40  <xorAxAx> data is a property that returns the file-like proxy to the data stream
2007-05-04T21:53:00  <xorAxAx> because opening/closing should be explicit
2007-05-04T21:53:21  <johill> but that clashes with that commit idea again
2007-05-04T21:53:29  <xorAxAx> yes
2007-05-04T21:53:46  <xorAxAx> which is broken anyway as we just found out
2007-05-04T21:54:19  <johill> ok, so if we drop the commit idea and push save() down into the item (or the data stream adapter thing) then we basically have my design
2007-05-04T21:54:40  <johill> except that the generic code is pushing things through to the backend
2007-05-04T21:54:42  <ThomasWaldmann> btw, if we store deleted flag in item metadata and acl in revision metadata, fs storage performance might be similar to 1.5
2007-05-04T21:56:43  <xorAxAx> ThomasWaldmann: well, thats necessary if it should make sense :)
2007-05-04T21:57:15  <johill> I shall think about how to handle ACLs with my idea
2007-05-04T21:57:22  <ThomasWaldmann> my last fs idea would need 1 stat for exists() and 1 file open for acl
2007-05-04T21:59:28  <johill> that needs either a current-acl per-item meta data or something similar
2007-05-04T22:01:04  <ThomasWaldmann> that would reduce it to 1 open for exist/may
2007-05-04T22:01:20  <ThomasWaldmann> and 1 open also for just exist
2007-05-04T22:01:26  <ThomasWaldmann> vs. 1 stat for exist
2007-05-04T22:01:46  <xorAxAx> premature optimisation :)
2007-05-04T22:01:56  <johill> heh yeah
2007-05-04T22:02:41  <xorAxAx> usually, you count the opens and stats after the design is fixed enough
2007-05-04T22:03:10  <johill> can all live in the backend anyway, if we figure out (yes, we know already) that a common access pattern is just getting the acl, then the current acl can be cached in per-item metadata without changes to upper layers
2007-05-04T22:03:29  * ThomasWaldmann will remember xorAxAx of his words when he next time complains about bad performance
2007-05-04T22:03:32  <johill> or if you have a sane filesystem you can add a symlink
2007-05-04T22:03:41  <johill> :P
2007-05-04T22:04:07  <xorAxAx> ThomasWaldmann: well, ... currently we are designing. implementing something pretty bad is a whole different issue
2007-05-04T22:04:27  <xorAxAx> i think i wasnt ever talking about speed issues that were by design
2007-05-04T22:05:13  <ThomasWaldmann> the problem will be if the design makes no other implementation easily possible
2007-05-04T22:08:14  <johill> hm, ok, well, I can't come up with anything good for ACLs if I have no idea what you want anyway
2007-05-04T22:08:30  <xorAxAx> johill: nobody has, lanius skipped the requirement analysis
2007-05-04T22:09:03  <johill> I'm not convinced having it at the lowest level is a good idea
2007-05-04T22:09:46  <ThomasWaldmann> what we need is fast exists() and may() because we use it all the time
2007-05-04T22:10:02  <ThomasWaldmann> (much more than reading item's data)
2007-05-04T22:10:06  <johill> well, that doesn't mean it needs to be implemented there
2007-05-04T22:10:21  <johill> that just needs we want fast access to the acl data
2007-05-04T22:12:16  <johill> I suppose with my design you could have an ACLCheckingItem() that is just a proxy for another item or something
2007-05-04T22:12:56  <xorAxAx> johill: that wasnt the plan
2007-05-04T22:13:05  <johill> what was?
2007-05-04T22:13:18  <xorAxAx> acl should be in item or revision
2007-05-04T22:13:33  <johill> why?
2007-05-04T22:13:45  <xorAxAx> because the storagebackend is acl-agnostic
2007-05-04T22:13:50  <johill> ok I think we have a terminology problem here
2007-05-04T22:14:13  <johill> in my terminology "Item" is an abstract concept that belongs to the storage backend
2007-05-04T22:14:27  <johill> and in yours it's actually much more part of the moin code
2007-05-04T22:14:38  <xorAxAx> in our terminology, it can be seen in an UML 2.0 diagram
2007-05-04T22:14:59  <johill> well, that difference can't be seen but hey
2007-05-04T22:15:25  <johill> anyway
2007-05-04T22:21:16  <ThomasWaldmann> btw, i think we should have a layer between most of the moin code and the storage items that checks ACLs
2007-05-04T22:21:31  <johill> the main difference is that I see Item as an abstract class and you see Item as a class that isn't ever inherited from
2007-05-04T22:21:33  <ThomasWaldmann> so we don't need may.xxx everywhere
2007-05-04T22:21:49  <xorAxAx> ThomasWaldmann: yes, Item is only for data storage, it doesnt contain any controller or view code :)
2007-05-04T22:22:35  <ThomasWaldmann> and then s/Item/StorageItem/, s/ProtectedItem/Item/
2007-05-04T22:22:56  <xorAxAx> ThomasWaldmann: i think thats not very nice
2007-05-04T22:23:13  <johill> the whole name "Item" is a bit too generic if you ask me ;)
2007-05-04T22:23:17  <xorAxAx> because security is something essential, pushing it into some remote location makes you lose the overview
2007-05-04T22:23:37  <xorAxAx> johill: yeah, the names need to get reassigned
2007-05-04T22:23:47  <xorAxAx> but UML is name-agnostic
2007-05-04T22:23:49  <johill> that sounds like an acl-checking proxy again
2007-05-04T22:23:52  <johill> heh
2007-05-04T22:24:28  <xorAxAx> ThomasWaldmann: note that the usual code that we have in e.g. send_page nowadays will lose all user.may stuf
2007-05-04T22:25:29  <johill> hmm
2007-05-04T22:25:30  <johill> I wonder
2007-05-04T22:25:40  <ThomasWaldmann> we have it all over the place. sendpage, actions, some macros, ...
2007-05-04T22:25:42  <johill> if the item collection knew about the request
2007-05-04T22:25:53  <xorAxAx> ThomasWaldmann: yep, that will all be unnessary
2007-05-04T22:25:59  <xorAxAx> in theory
2007-05-04T22:26:05  <ThomasWaldmann> why?
2007-05-04T22:26:13  <xorAxAx> because the storage layer will do the checks
2007-05-04T22:26:25  <johill> then acl checking could easily be done by an ACLCheckingItemCollection() that goes down to another item collection and asks it for actual storage and only wraps everything in acl-checking code
2007-05-04T22:26:32  <ThomasWaldmann> and what is the difference to my suggestion?
2007-05-04T22:26:43  <xorAxAx> ThomasWaldmann: you suggested a new class to handle the acl stuff
2007-05-04T22:26:49  <xorAxAx> thats not very sane IMHO
2007-05-04T22:27:02  <ThomasWaldmann> it doesnt make a big difference
2007-05-04T22:27:10  <johill> call it "proxy pattern" ;)
2007-05-04T22:27:59  <ThomasWaldmann> except that you need a backdoor or admin user or another hack if you don't want acl checks in the other approach
2007-05-04T22:28:40  <xorAxAx> ThomasWaldmann: hmm?
2007-05-04T22:28:48  <xorAxAx> what doesnt make any difference?
2007-05-04T22:29:23  <ThomasWaldmann> if you do ACL checks in every case, you need some method to not do them
2007-05-04T22:29:31  <xorAxAx> ???
2007-05-04T22:29:32  <xorAxAx> no
2007-05-04T22:29:38  <ThomasWaldmann> in my approach, you just use the other item class
2007-05-04T22:29:41  <xorAxAx> everything is checked, otherwise its a security hole :)
2007-05-04T22:29:56  <johill> but the conversion script doesn't want security checking
2007-05-04T22:29:58  <johill> for example
2007-05-04T22:30:10  <ThomasWaldmann> if you want to make a wiki backup, you dont want to check acls
2007-05-04T22:30:15  <xorAxAx> johill: well, in that case the checker will notice that
2007-05-04T22:30:26  <ThomasWaldmann> you just want to get everything you want as fast as possible
2007-05-04T22:30:30  <xorAxAx> johill: because e.g. the user object is a virtual root user
2007-05-04T22:30:38  <xorAxAx> ThomasWaldmann: yes, but you are solving it on the wrong level :)
2007-05-04T22:30:40  <johill> xorAxAx: yeah, sucks
2007-05-04T22:30:44  <xorAxAx> johill: what?
2007-05-04T22:30:59  <johill> a concept that requires a "virtual root user"
2007-05-04T22:31:07  <xorAxAx> not at all
2007-05-04T22:31:11  <xorAxAx> thats what posix is about
2007-05-04T22:31:14  <xorAxAx> with its root
2007-05-04T22:31:16  <dreimark> don't forget migrations as we have had before
2007-05-04T22:31:32  <johill> no, posix has an actual root user
2007-05-04T22:31:45  <xorAxAx> just because to get 0.3 s speedup on backup, moving the acl code into some other decorating class is insane
2007-05-04T22:32:02  <xorAxAx> oh well
2007-05-04T22:32:07  <xorAxAx> hi grzyw
2007-05-04T22:32:11  <xorAxAx> and grzywacz
2007-05-04T22:32:20  <johill> heh
2007-05-04T22:33:37  <grzywacz> hi xorAxAx
2007-05-04T22:33:39  <grzywacz> I'm back for good.
2007-05-04T22:33:43  <xorAxAx> cool
2007-05-04T22:35:15  <ThomasWaldmann> well, it doesnt make a big difference if we do it in 1 or 2 classes. we can change that afterwards if needed. important is that it is done somewhere below the usual "item using" code.
2007-05-04T22:36:15  <johill> doing it in a proxy pattern would probably also make the code more readable since then the acl-proxy only deals with acls and refers to some downstream item for the rest
2007-05-04T22:36:27  <xorAxAx> ThomasWaldmann: yes
2007-05-04T22:36:37  <ThomasWaldmann> yeah, that was the idea behind it :)
2007-05-04T22:37:00  <xorAxAx> i think i would use a method-wise decorator pattern
2007-05-04T22:37:06  <xorAxAx> not a class-wide decorator
2007-05-04T22:37:22  <xorAxAx> but we will see
2007-05-04T22:37:39  <ThomasWaldmann> (and we need another exception handling, it would be uncool to see the data that you may not read in some traceback :)
2007-05-04T22:37:42  <johill> how do you do method-wide decorators?
2007-05-04T22:38:08  <johill> ThomasWaldmann: with a proxy pattern it wouldn't be loaded at the point where the exception is thrown that refuses it ;)
2007-05-04T22:38:31  <ThomasWaldmann> oh, sure
2007-05-04T22:38:46  <johill> actually, in either case, not just with that pattern
2007-05-04T22:39:34  <xorAxAx> johill: well, normal decorator pattern for methods
2007-05-04T22:39:50  <xorAxAx> ThomasWaldmann: thats already fixed in 1.6
2007-05-04T22:40:03  <xorAxAx> it doesnt show sensitive data in cgitb
2007-05-04T22:40:51  <ThomasWaldmann> the opinions about what is sensitive can be seen in some open bug reports
2007-05-04T22:42:43  <xorAxAx> yeah, its configurable :)
2007-05-04T22:42:58  <xorAxAx> what are the bug reports talking about e.g.?
2007-05-04T22:44:59  <xorAxAx> (if you can remember)
2007-05-04T22:45:26  <ThomasWaldmann> showing kernel version, os, pathes, ...
2007-05-04T22:45:44  <ThomasWaldmann> moin version, showing backtraces at all, ...
2007-05-04T22:46:56  <ThomasWaldmann> (but of course there is some conflict between security by not giving out information and being able to analyse problems somewhere on the web)
2007-05-04T22:47:30  <xorAxAx> yeah
2007-05-04T22:57:24  <dreimark> good night
2007-05-04T23:13:32  <johill> I added a diagram to my page
2007-05-04T23:21:43  <ThomasWaldmann> looks clean
2007-05-04T23:23:51  <ThomasWaldmann> my general problem with such diagrams is that they often lack details
2007-05-04T23:24:10  <johill> oh, I agree
2007-05-04T23:24:15  <johill> I don't like uml much anyway
2007-05-04T23:24:27  <johill> but xorAxAx seems to think it's a panacea ;)
2007-05-04T23:25:11  <johill> if I knew how this tool allowed me to add methods I would
2007-05-04T23:26:08  <xorAxAx> johill: no, i dont
2007-05-04T23:26:27  <xorAxAx> johill: i am just forced to use it
2007-05-04T23:28:35  <johill> where are you forced to use it?
2007-05-04T23:29:13  <xorAxAx> sopra
2007-05-04T23:29:43  <johill> ah, yeah
2007-05-04T23:30:17  <johill> ok I added some methods and attributes
2007-05-04T23:30:32  <ThomasWaldmann> btw, is there some uml tool with zooming?
2007-05-04T23:30:48  <johill> what kind of zooming?
2007-05-04T23:31:02  <ThomasWaldmann> and able to do pythonic stuff rather than letting look everything like java?
2007-05-04T23:31:36  <ThomasWaldmann> seeing more details of less stuff when you zoom in?
2007-05-04T23:32:04  <johill> hm, not sure, argouml might do that but it doesn't run on my system right now
2007-05-04T23:32:11  <johill> I'm using bouml (never heard of it before, but hey) right now
2007-05-04T23:35:02  <johill> for the fun of it
2007-05-04T23:35:08  * johill adds a backend-item stuff
2007-05-04T23:35:51  <xorAxAx> argouml sucks, no undo
2007-05-04T23:36:01  <xorAxAx> ThomasWaldmann: have you tried dia?
2007-05-04T23:36:28  <xorAxAx> ThomasWaldmann: yes, it has zooom
2007-05-04T23:36:36  <xorAxAx> IMHO its the best tool for small stuf
2007-05-04T23:36:44  <xorAxAx> it can even generate python code
2007-05-04T23:38:10  <ThomasWaldmann> you're sure you mean the same with zoom as me?
2007-05-04T23:38:14  <xorAxAx> it even greets you on the console after using it
2007-05-04T23:38:33  <xorAxAx> well, i just know one kind of zoom
2007-05-04T23:38:44  <ThomasWaldmann> i dont want to have bigger pixels
2007-05-04T23:38:54  <xorAxAx> but?
2007-05-04T23:39:09  <ThomasWaldmann> see more details that are hidden before
2007-05-04T23:39:34  <johill> like if you zoom out at some point you only see class names, not methods any more
2007-05-04T23:40:31  <ThomasWaldmann> yes
2007-05-04T23:47:12  <johill> ah well, I'll leave this for now and see if lanius has any comments. I added some more notes to my page