MoinMoin: MoinMoinChat/Logs/moin-dev/2006-08-31

2006-08-31T08:57:10  <ThomasWaldmann> moin
2006-08-31T09:11:16  <Kepplar> moin
2006-08-31T10:46:40  <Kepplar> So anyone got any ideas on how we should handle the combination/separation of metadata
2006-08-31T10:46:43  <Kepplar> ?
2006-08-31T10:52:39  <mitsuhiko> Kepplar: for flatfile?
2006-08-31T10:53:09  <Kepplar> generally
2006-08-31T10:53:21  <mitsuhiko> imho you only have one way
2006-08-31T10:53:31  <Kepplar> the flatfile is transparent
2006-08-31T10:53:35  <mitsuhiko> since moin supports more than one formatter each formatter might display metadata in the editor different
2006-08-31T10:53:45  <Kepplar> Yea
2006-08-31T10:53:52  <mitsuhiko> for example the normal moin editor requires that you put hashes in the header:
2006-08-31T10:53:58  <mitsuhiko> #mimetype x-something/blub
2006-08-31T10:54:17  <Kepplar> Problem is mimetype was designed to be a separate dicitionary on an item object along side raw data
2006-08-31T10:54:28  <mitsuhiko> that's not a problem
2006-08-31T10:54:41  <Kepplar>  the mimetype bug we have on storage atm stems from it
2006-08-31T10:54:42  <mitsuhiko> just add a method prepare_for_editor(text, mimetype) to the formatter
2006-08-31T10:54:57  <Kepplar> hmm
2006-08-31T10:54:59  <mitsuhiko> and then fetch_metadata_from_text(text) which outputs the stipped text and mimetype again
2006-08-31T10:55:09  <mitsuhiko> and your storage system then saves it like it needs to
2006-08-31T10:55:20  <mitsuhiko> sql for example would have a table with some joines for it
2006-08-31T10:55:29  <Kepplar> yep
2006-08-31T10:55:35  <mitsuhiko> the flatfile system would have a seperated file called "metadata" in the folder of the page
2006-08-31T10:55:50  <Kepplar> well no the legacy system will store it in the same file
2006-08-31T10:56:07  <mitsuhiko> that's an implementation detail
2006-08-31T10:56:19  <Kepplar> this seems like a good intermediate step - personally I dont think any formatter should have metadata and data as one
2006-08-31T10:56:21  <mitsuhiko> basically you don't need to know where the metadata actually is stored in
2006-08-31T10:56:23  <Kepplar> in the future
2006-08-31T10:58:37  <ThomasWaldmann> mitsuhiko: metadata is stored separately and the storage format won't be different for different mimetypes of the data item
2006-08-31T10:59:04  <ThomasWaldmann> (this is the goal, at least)
2006-08-31T10:59:22  <mitsuhiko> ThomasWaldmann: depends on the storage backend doesn't it?
2006-08-31T10:59:36  <ThomasWaldmann> not really
2006-08-31T10:59:48  <mitsuhiko> an sql backend would storage metadata different from an plain text solution
2006-08-31T11:00:05  <ThomasWaldmann> that doesnt change the metadata itself
2006-08-31T11:00:09  <mitsuhiko> but for the editor the mimetype would move into the same textarea
2006-08-31T11:00:28  <mitsuhiko> so the formatter has to strip that out after the saving
2006-08-31T11:00:40  <xorAxAx> mitsuhiko: no, the formatter wont do that
2006-08-31T11:00:45  <xorAxAx> mitsuhiko: its unrelated
2006-08-31T11:00:51  <xorAxAx> it doesnt see them at all
2006-08-31T11:00:52  <ThomasWaldmann> and we can mix data and metadata until we have a better solution - on the UI
2006-08-31T11:01:16  <mitsuhiko> xorAxAx: take mokuwiki as an example
2006-08-31T11:01:22  <mitsuhiko> there comments start with ;;
2006-08-31T11:01:59  <mitsuhiko> or do you want to use "#" just in the header and strip that before giving it the parser?
2006-08-31T11:02:31  <xorAxAx> ?
2006-08-31T11:02:33  <Kepplar> ThomasWaldmann: it will only store item types in the same place for a given engine, another engine instance may store i somewhere else (just saying its possible)
2006-08-31T11:02:40  <xorAxAx> mitsuhiko: the parser handles them currently
2006-08-31T11:02:48  <ThomasWaldmann> metadata is just pairs key, value
2006-08-31T11:03:08  <Kepplar> btw any idea guys why storage is currently printing out the metadata to the screen?
2006-08-31T11:03:13  <Kepplar> (ive not touched any formatters)
2006-08-31T11:03:18  <ThomasWaldmann> UI might present them in a separate editor window as "key: value\nkey2: value2\n..."
2006-08-31T11:03:27  <mitsuhiko> xorAxAx: the moin parser. not the mokuwiki, mediawiki-like parser
2006-08-31T11:03:33  <mitsuhiko> they don't have comments starting with "#"
2006-08-31T11:03:48  <mitsuhiko> in mediawiki is "#" the start of an enumeration afair
2006-08-31T11:04:14  <ThomasWaldmann> or UI might merge them at top of a text page as "#pragma key value\n..." (or whatever...)
2006-08-31T11:04:23  <ThomasWaldmann> (for 1.5 compat)
2006-08-31T11:04:42  <mitsuhiko> ThomasWaldmann: therefore the method on the parser to add and strip the metadata before putting it into the editor
2006-08-31T11:05:03  <ThomasWaldmann> can be done like this
2006-08-31T11:05:28  <ThomasWaldmann> but i guess we have to approach all this stuff differently
2006-08-31T11:05:30  <Kepplar> parser not formatter?
2006-08-31T11:05:44  <ThomasWaldmann> in much smaller steps, or it won't work ever
2006-08-31T11:05:56  <Kepplar> we should have a design first =)
2006-08-31T11:06:06  <Kepplar> hence my diagrams on NextGenerationArchitecture
2006-08-31T11:06:17  <mitsuhiko> Kepplar: you should had the design... hmm... two months ago?
2006-08-31T11:06:24  <Kepplar> i did
2006-08-31T11:06:30  <Kepplar> this is not storage design
2006-08-31T11:06:31  <ThomasWaldmann> Kepplar: i didnt get much out of those yet
2006-08-31T11:06:43  <mitsuhiko> Kepplar: so metadata isn't part of storage?
2006-08-31T11:07:17  <Kepplar> mitsuhiko: storage stores things, it doesnt know what its storing it knows what data is and what is metadata, how its presented, how its handled by moin is outside storage
2006-08-31T11:07:49  <ThomasWaldmann> but an approach (while doing more planning) is to simplify moin before
2006-08-31T11:07:56  <mitsuhiko> so your task was just writing it and not integrating it? Oo
2006-08-31T11:08:20  <Kepplar> mitsuhiko: yes it was, but in hindsight it was more or less impossible in the timeframe
2006-08-31T11:08:24  <ThomasWaldmann> like e.g. removing hierarchical stuff we dont use anyway
2006-08-31T11:08:55  <Kepplar> mitsuhiko: integration was partially and crudly done, its not suitable for production
2006-08-31T11:09:07  <ThomasWaldmann> or moving user profiles to 1.5 page storage
2006-08-31T11:09:17  <Kepplar> usrs are using storage
2006-08-31T11:09:21  <ThomasWaldmann> (we can hide them by acl)
2006-08-31T11:09:27  <Kepplar> im not sure how successful they are atm
2006-08-31T11:10:12  <Kepplar> Anyway I think the first thing we need to do is get a diagram of how Moin works now, and then what we want it to look like
2006-08-31T11:10:12  <ThomasWaldmann> Kepplar: 3 months fulltime are LOTS of time
2006-08-31T11:10:23  <Kepplar> ThomasWaldmann: there was no architecture diagrams though
2006-08-31T11:10:34  <Kepplar> that was my main point
2006-08-31T11:10:43  <Kepplar> i have to work top-down
2006-08-31T11:10:58  <Kepplar> reverse engineering to learn how it works is slow and painful
2006-08-31T11:11:16  <Kepplar> hence now with no deadline we should think about doing it properly
2006-08-31T11:11:43  <Kepplar> *i* can't do the current architecture diagrams
2006-08-31T11:11:55  <Kepplar> I dont really get it
2006-08-31T11:12:00  <ThomasWaldmann> i guess after writing the converter i can work on simplifying moin in some related parts
2006-08-31T11:12:28  <ThomasWaldmann> the main point is we have to keep it working while doing that stuff
2006-08-31T11:12:38  <Kepplar> yea
2006-08-31T11:12:54  <Kepplar> but having specific idea of what its going to look like is nicer for focus :)
2006-08-31T11:13:06  <Kepplar> hence the arch diagrams
2006-08-31T11:13:24  <Kepplar> I've made a start from storage perspective for the NG Architecture
2006-08-31T11:14:04  <Kepplar> at the very least could someone explain it in a way I can write the diagram for existing :)
2006-08-31T11:16:17  <ThomasWaldmann> sorry, but looking at those diagrams on NGA doesn't give me much
2006-08-31T11:17:35  <Kepplar> its a pesudo-object diagram
2006-08-31T11:17:52  <Kepplar> its not complete, just started infact
2006-08-31T11:18:07  <Kepplar> but regardless thats the new architeture, the old one is what we need done before progressing :)
2006-08-31T11:18:24  <Kepplar> basically "how" moin works the question i asked at the start of SOC but didn't really get an answer
2006-08-31T11:18:30  <Kepplar> what happens when a page is pressed
2006-08-31T11:18:34  <Kepplar> what objects get created
2006-08-31T11:18:38  <Kepplar> what is parsed around to where
2006-08-31T11:18:42  <Kepplar> how it all works
2006-08-31T11:20:49  <ThomasWaldmann> you don't expect we paraphrase the whole src on irc for you?
2006-08-31T11:21:24  <Kepplar> Generally speaking (im not talking about moin specifically) i expect this to have been documented at the start =)
2006-08-31T11:21:52  <ThomasWaldmann> there is quite some stuff in the wiki and there is also a search function
2006-08-31T11:22:05  <Kepplar> Hence i think we all need to work together in producing it
2006-08-31T11:22:14  <Kepplar> ThomasWaldmann: oh i tried that
2006-08-31T11:22:17  <ThomasWaldmann> some of that stuff outdates, though, so it is better to look at the src
2006-08-31T11:22:30  <Kepplar> reverse enginering the source will take me a year
2006-08-31T11:23:06  <Kepplar> and seeing as everyone else is so good with the code i think it would be quicker if everyone contributed the bits they know about :)
2006-08-31T11:23:32  <ThomasWaldmann> no, you will know enough after maybe 2 weeks fulltime reading and working with the src.
2006-08-31T11:24:00  <ThomasWaldmann> it isnt THAT complicated
2006-08-31T11:24:06  <Kepplar> i beg to differ
2006-08-31T11:24:33  <Kepplar> remember you see the code much easier because you know it :)
2006-08-31T11:24:55  <xorAxAx> Kepplar: i just needed that amount of time when i got into it a few months ago
2006-08-31T11:25:19  <xorAxAx> Kepplar: it was 2004 ...
2006-08-31T11:25:27  <Kepplar> your a hacky programmer though
2006-08-31T11:25:49  <Kepplar> and you work on the reverse engineering principle
2006-08-31T11:26:05  <ThomasWaldmann> you also don't need to read everything. e.g. formatter base (and maybe text_html) is enough, you dont need to read all formatters
2006-08-31T11:26:35  <Kepplar> i can read a class and object, but i have no idea from the code unless i read everywhere where in the page loading process it is called etc
2006-08-31T11:26:38  <ThomasWaldmann> same is true for macro, action, parser, request, ... (everything that is modular and that's quite much)
2006-08-31T11:27:26  <xorAxAx> Kepplar: not really
2006-08-31T11:28:06  <xorAxAx> Kepplar: as i said, my colleagues in my small job are all physiscians, economics people etc. and they read the code because there are no docs
2006-08-31T11:29:02  <Kepplar> no offense but i find alot of the code obfuscated (Well what ive been working with)
2006-08-31T11:29:18  <Kepplar> plus having a visual memory doesnt help
2006-08-31T11:29:38  <xorAxAx> you can write down your results, then memory problems are gone
2006-08-31T11:29:51  <Kepplar> in diagrams
2006-08-31T11:30:02  <xorAxAx> yeah, where are they?
2006-08-31T11:30:11  <Kepplar> on my desk and in visio
2006-08-31T11:30:20  <Kepplar> but only regarding my code
2006-08-31T11:30:33  <xorAxAx> your code is irrelevant in this discussion
2006-08-31T11:30:50  <xorAxAx> because you are still at the point "understand system"
2006-08-31T11:31:02  <Kepplar> all i see is Page, user, somehow caching and security somehow tied to request in a giant cobweb of functions calls for the rest of the system
2006-08-31T11:31:04  <xorAxAx> not at "design new system"
2006-08-31T11:31:20  <Kepplar> Yes, hence why im asking for documents
2006-08-31T11:31:47  <Kepplar> you yourself said you were against API's and just call code whenever you feel like which i felt a bit shocked about
2006-08-31T11:31:54  <xorAxAx> you said that memory problems stopped you - i said that you can draw your (intermediary) docs
2006-08-31T11:32:05  <xorAxAx> Kepplar: when did i say that?
2006-08-31T11:32:22  <xorAxAx> i dislike the breaking APIs of moin and try to get everybody who breaks them to document changes
2006-08-31T11:32:26  <xorAxAx> and give a reasoning
2006-08-31T11:32:42  <Kepplar> thats ironic
2006-08-31T11:32:47  <xorAxAx> ?
2006-08-31T11:32:51  <Kepplar> i could have sworn you said the opposite last week
2006-08-31T11:33:00  <Kepplar> (thats even gone into my eval)
2006-08-31T11:33:07  <xorAxAx> i would be really interested in the quote
2006-08-31T11:33:17  * ThomasWaldmann works on the step by step stuff
2006-08-31T11:33:25  <Kepplar> ThomasWaldmann: hmm?
2006-08-31T11:33:37  <xorAxAx> its a really bad base to have people in the team defame others
2006-08-31T11:33:38  <Kepplar> xorAxAx: ill have alook around logs later
2006-08-31T11:33:48  <Kepplar> xorAxAx: i didnt mention you
2006-08-31T11:34:01  <Kepplar> xorAxAx: i said i was a little suprised at some of the developers styles and ethos
2006-08-31T11:34:08  <xorAxAx> but even then, you must have misunderstand me
2006-08-31T11:34:26  <xorAxAx> personally, i dont recall a broken API in the moin code because of my changes
2006-08-31T11:34:37  <Kepplar> nooo
2006-08-31T11:34:47  <xorAxAx> ?
2006-08-31T11:35:05  <Kepplar> my point i made a few weeks ago is that moins code appears (at least to me) as a big mess of interdependant method calls without any defined (and controlled) apis on what calls what
2006-08-31T11:35:21  <xorAxAx> oh, then you havent understand it :)
2006-08-31T11:35:25  <xorAxAx> s/and/ood/
2006-08-31T11:35:33  <xorAxAx> there are some dirty edges
2006-08-31T11:35:43  <xorAxAx> but mostly there is a sound concept
2006-08-31T11:36:02  <Kepplar> i'd be very dissapointed if everyone outside storage was fine and the "dirty edge" was storage :>
2006-08-31T11:36:11  <xorAxAx> and no, jürgen wasnt the person who introduced all of those dirty edges. i.e. you can ask people here to give explanations about the dirty edges
2006-08-31T11:36:41  <Kepplar> ah here we go, an analogy
2006-08-31T11:37:14  <Kepplar> Last three months (dispite asking for a map) ive been working surrounded by tower blocks as far as the eye can see trying to map out how everything is put together
2006-08-31T11:37:24  <Kepplar> and getting dazed at the tower blocks around me because im without a map
2006-08-31T11:43:10  <ThomasWaldmann> Kepplar: on the "making it easier by small steps plan"
2006-08-31T11:52:37  <Kepplar> ah
2006-08-31T11:53:03  <Kepplar> so we going to take action on producing a arch diagram?
2006-08-31T11:54:22  <xorAxAx> you will do the first step and we will join in
2006-08-31T11:54:24  * ThomasWaldmann is not used to drawing diagrams
2006-08-31T11:54:47  <Kepplar> ok
2006-08-31T12:00:55  <ThomasWaldmann> Kepplar: see if my last wiki edit addresses some of your problems
2006-08-31T13:01:05  <ThomasWaldmann> ok, finished editing for now. bbl.
2006-08-31T18:31:53  <ThomasWaldmann> re