1 2011-07-26T00:25:40 *** raignarok
2 2011-07-26T01:16:36 *** MattMaker_
3 2011-07-26T01:20:06 *** MattMaker
4 2011-07-26T01:20:06 *** MattMaker_
5 2011-07-26T04:37:45 *** MattMaker
6 2011-07-26T04:38:14 <Marchael> moin
7 2011-07-26T04:46:37 *** MattMaker
8 2011-07-26T05:13:15 *** xorAxAx
9 2011-07-26T05:21:04 *** xorAxAx
10 2011-07-26T05:25:23 <Marchael> ThomasWaldmann: I found a reason
11 2011-07-26T05:25:59 <Marchael> query parser parses "contenttype:text/rst" as Term(u'contenttype', u'text')
12 2011-07-26T05:26:31 <Marchael> so that's why I see unnecessary results
13 2011-07-26T05:30:30 <Marchael> hm, but tokenizer works correct, at least test says so
14 2011-07-26T05:47:10 <Marchael> ThomasWaldmann: for some reason I have different results for test and runtime work
15 2011-07-26T05:47:44 <Marchael> so, in offline it's work well, but in runtime not
16 2011-07-26T06:06:50 <Marchael> ThomasWaldmann: also, I found same problem for acl tokenizer
17 2011-07-26T06:16:49 <Marchael> and for name too =\
18 2011-07-26T06:21:59 *** MattMaker
19 2011-07-26T06:26:16 *** MattMaker
20 2011-07-26T07:15:16 *** xjjk
21 2011-07-26T07:17:46 *** xjjk
22 2011-07-26T07:30:31 *** Marchael
23 2011-07-26T07:35:47 *** pkumar
24 2011-07-26T08:02:48 <dreimark> moin
25 2011-07-26T08:59:16 *** raignarok
26 2011-07-26T09:35:53 *** gwork
27 2011-07-26T09:36:18 <gwork> moin
28 2011-07-26T09:47:23 <ThomasWaldmann> moin
29 2011-07-26T10:02:40 *** raignarok
30 2011-07-26T10:32:57 *** pkumar
31 2011-07-26T10:34:38 *** Marchael
32 2011-07-26T10:48:45 <ThomasWaldmann> moin Marchael
33 2011-07-26T10:49:10 <Marchael> hi ThomasWaldmann
34 2011-07-26T10:49:19 <ThomasWaldmann> what you found out is interesting
35 2011-07-26T10:49:46 <Marchael> but tests working correct
36 2011-07-26T10:50:01 <ThomasWaldmann> before doing anything else, can you please add a repr() of the parsed query to the search results output?
37 2011-07-26T10:50:06 <Marchael> I don't know why parsers work incorrect when moin running
38 2011-07-26T10:50:26 <Marchael> I can provide debug output
39 2011-07-26T10:50:33 <ThomasWaldmann> so that we see how whoosh parsed the query
40 2011-07-26T10:51:07 *** raignarok
41 2011-07-26T10:51:51 <Marchael> ThomasWaldmann: full debug output
42 2011-07-26T10:51:52 <Marchael> http://paste.pocoo.org/show/446585/
43 2011-07-26T10:52:01 <Marchael> sorry, It's rather big
44 2011-07-26T10:52:28 <Marchael> brb
45 2011-07-26T10:52:40 <ThomasWaldmann> (it would be easiest to have it between search query form and search results output)
46 2011-07-26T11:01:21 <ThomasWaldmann> i don't see it working "correct" (== as we expect), no matter how it is run in your debug session
47 2011-07-26T11:05:59 *** greg_f
48 2011-07-26T11:15:01 <Marchael> hm, so what shoud I do?
49 2011-07-26T11:19:20 <Marchael> ThomasWaldmann:
50 2011-07-26T11:19:30 <ThomasWaldmann> i just told you
51 2011-07-26T11:21:32 <Marchael> what is it?
52 2011-07-26T11:21:48 <Marchael> > (it would be easiest to have it between ..
53 2011-07-26T11:22:22 <ThomasWaldmann> (10:48) < ThomasWal> before doing anything else, can you please add a repr() of the parsed query to the search results output?
54 2011-07-26T11:22:22 <Marchael> ah, ok
55 2011-07-26T11:22:27 <ThomasWaldmann> (10:51) < ThomasWal> (it would be easiest to have it between search query form and search results output)
56 2011-07-26T12:19:46 *** greg_f
57 2011-07-26T13:05:45 <Marchael> re
58 2011-07-26T13:11:55 <Marchael> ThomasWaldmann: I added repr() in that place, but still have same results
59 2011-07-26T13:13:07 <Marchael> contenttype:text/x-rst -> Term(u'contenttype', u'text')
60 2011-07-26T13:14:13 <Marchael> same thing for acl and name analyzers
61 2011-07-26T13:14:44 <Marchael> analyzer yields all tokens, but parser use only first
62 2011-07-26T13:26:10 <ThomasWaldmann> you could file an issue in the whoosh tracker.
63 2011-07-26T13:26:57 <ThomasWaldmann> i think there is either missing documentation how it is supposed to work or a bug. in any case, i think matt could help us here.
64 2011-07-26T13:28:22 <ThomasWaldmann> if you maybe could reproduce the issue with a builtin tokenizer first, that could also be helpful (either for us to see what we do wrong or for him to reproduce the issue without our code)
65 2011-07-26T13:28:38 <Marchael> ok
66 2011-07-26T13:32:12 *** greg_f
67 2011-07-26T16:21:42 <Marchael> ThomasWaldmann: multitoken_query = "first"
68 2011-07-26T16:21:45 <Marchael> this?
69 2011-07-26T16:23:04 *** raignarok
70 2011-07-26T16:23:44 <Marchael> seems like I'm need pass there something
71 2011-07-26T16:23:58 <Marchael> different than 'firs'
72 2011-07-26T16:24:07 * Marchael looks in other analyzers
73 2011-07-26T16:29:04 <ThomasWaldmann> i didn't see whoosh using anything else than "first" internally, which explains why it used the first token only
74 2011-07-26T16:29:18 <ThomasWaldmann> so you have to find out now what we really want
75 2011-07-26T16:31:36 *** pkumar
76 2011-07-26T16:38:40 <Marchael> oh cool
77 2011-07-26T16:42:02 <Marchael> ThomasWaldmann: yes, it's working!
78 2011-07-26T16:45:07 <Marchael> ThomasWaldmann: http://codereview.appspot.com/4819047
79 2011-07-26T16:58:14 <ThomasWaldmann> we still don't know why name search worked before that change
80 2011-07-26T16:59:36 <Marchael> ThomasWaldmann: yes, it's worked, but it's worked incorrectly
81 2011-07-26T16:59:50 <Marchael> for the same reason
82 2011-07-26T17:00:12 <ThomasWaldmann> i tried it, it did not seem incorrect
83 2011-07-26T17:00:37 <ThomasWaldmann> maybe i have to try again to make sure...
84 2011-07-26T17:00:51 <Marchael> hm, do you saw what for searching "ThisIsString" it returns "This"?
85 2011-07-26T17:01:19 <Marchael> I put large debug output above with this
86 2011-07-26T17:01:51 <Marchael> name, acl and contentype analyzers workd incorrectly
87 2011-07-26T17:07:54 <ThomasWaldmann> somehow it seems to me that "first" is a stupid default
88 2011-07-26T17:08:45 <Marchael> yes
89 2011-07-26T17:09:31 <ThomasWaldmann> hah
90 2011-07-26T17:10:02 <ThomasWaldmann> i have items "Foo", "FooBar", "BarBaz" now. searching for "FooBar" yields Foo and FooBar.
91 2011-07-26T17:10:39 <ThomasWaldmann> and search for BarFoo yields BarBaz and FooBar.
92 2011-07-26T17:10:46 <ThomasWaldmann> so yes, it is malfunctioning.
93 2011-07-26T17:11:10 <ThomasWaldmann> shall I file the bug about stupid defaults? :)
94 2011-07-26T17:12:09 <Marchael> i guess so :)
95 2011-07-26T17:12:17 <Marchael> s/so/yes/
96 2011-07-26T17:12:51 <ThomasWaldmann> Added per-field multitoken_query attribute to control how the query parser deals with a “term” that when analyzed generates multiple tokens. The default value is “first” which throws away all but the first token (the previous behavior). Other possible values are “and”, “or”, or “phrase”.
97 2011-07-26T17:12:59 <ThomasWaldmann> (from whoosh 1.5 changelog entry)
98 2011-07-26T17:13:15 <Marchael> I read about that from source
99 2011-07-26T17:13:40 <ThomasWaldmann> so it is stupid for compatibility
100 2011-07-26T17:14:01 <ThomasWaldmann> reminds me of microsoft somehow
101 2011-07-26T17:16:51 * dreimark likly is not available today at our meeting
102 2011-07-26T17:18:44 <Marchael> ThomasWaldmann: can I commit my changes?
103 2011-07-26T17:19:11 <Marchael> oops, i don't see comments
104 2011-07-26T17:30:23 *** gwork
105 2011-07-26T17:32:30 <Marchael> ThomasWaldmann: most of fields is ID, so that's only one token to pass :)
106 2011-07-26T17:33:47 <ThomasWaldmann> https://bitbucket.org/mchaput/whoosh/issue/158/multitoken_query-strange-defaults-lead-to
107 2011-07-26T17:33:58 <Marchael> oh, thx
108 2011-07-26T17:36:10 <ThomasWaldmann> Content as other fields using whoosh built in anayzer, and I think what
109 2011-07-26T17:36:11 <ThomasWaldmann> they are working well. at least I don't saw only 'first' token.
110 2011-07-26T17:36:41 <ThomasWaldmann> ^^ i never have seen any other value than "first" for it, so that should be broken also, AFAICS.
111 2011-07-26T17:37:34 <Marchael> ok, I add multitoke to parameters, so we could be sure
112 2011-07-26T17:43:02 <Marchael> I guess "or" will be good for that
113 2011-07-26T17:57:40 <Marchael> ThomasWaldmann: seems like Matt confused by your issue :)
114 2011-07-26T17:59:04 <ThomasWaldmann> https://bitbucket.org/mchaput/whoosh/issue/159/multitoken_query-phrase-gives-typeerror
115 2011-07-26T18:00:38 <waldi> moin
116 2011-07-26T18:07:24 <ThomasWaldmann> moin
117 2011-07-26T18:07:30 <ThomasWaldmann> Marchael: please begin
118 2011-07-26T18:07:44 <Marchael> ok
119 2011-07-26T18:07:46 <Marchael> Last week I'm worked on improving searching and search results template.
120 2011-07-26T18:07:46 <Marchael> I improve search results template so now it's looking like http://i.imgur.com/9OtAC.png
121 2011-07-26T18:07:46 <Marchael> - Added large search input field
122 2011-07-26T18:07:46 <Marchael> - search statistics
123 2011-07-26T18:07:46 <Marchael> - Highlighting working in some cases, but i switch off that due bugs
124 2011-07-26T18:07:46 <Marchael> Althought, I submitted patch to whoosh and it was accepted https://bitbucket.org/mchaput/whoosh/issue/156/with-patch-highlighting-fails-for
125 2011-07-26T18:07:46 <Marchael> but seems like it have more bugs than I thought
126 2011-07-26T18:07:47 <Marchael> We found at least
127 2011-07-26T18:07:47 <Marchael> Searching in content and navigation will coming on this week, also I'll fix issues from previous week.
128 2011-07-26T18:07:48 <Marchael> blockers: None
129 2011-07-26T18:08:19 <Marchael> *We found at least 4 bugs that week
130 2011-07-26T18:08:48 <Marchael> 2 was fixes
131 2011-07-26T18:08:51 <Marchael> *fixed
132 2011-07-26T18:09:30 <ThomasWaldmann> ok, any questions for Marchael?
133 2011-07-26T18:09:52 <waldi> no
134 2011-07-26T18:10:43 <ThomasWaldmann> xorAxAx: are you with us?
135 2011-07-26T18:11:33 <pkumar> this week:
136 2011-07-26T18:11:37 <pkumar> continued the work on improving the test coverage. wrote the tests for all the modules listed in EP under my name(where ever required).
137 2011-07-26T18:11:37 <pkumar> worked on fixing failures/errors from storage modules. Fixed few of them. Now there are no more errors but few failures in storage due to the indexing issue.
138 2011-07-26T18:11:37 <pkumar> Currently working on the improvement of test coverage for the modules earlier assigned to hardy89. Writing tests for already existing tests. After this, once it's over will write tests for the files having no tests at all.
139 2011-07-26T18:11:41 <ThomasWaldmann> ok, I don't have any right now either, so thanks Marchael, waldi / xorAxAx / pkumar please continue.
140 2011-07-26T18:11:41 <pkumar> blockers : None
141 2011-07-26T18:11:57 <Marchael> bbl
142 2011-07-26T18:13:09 <ThomasWaldmann> pkumar: when you "fix" a tests, you make sure you fix at the right place? :)
143 2011-07-26T18:14:00 <pkumar> ThomasWaldmann : yeah, I try to to careful about that
144 2011-07-26T18:14:06 <ThomasWaldmann> i mean some code might be really broken and the test failing, but rightfully failing :)
145 2011-07-26T18:14:29 <pkumar> i.e. it should not effect other execution/process right?
146 2011-07-26T18:14:52 <ThomasWaldmann> what do you mean?
147 2011-07-26T18:15:59 <pkumar> ThomasWaldmann : I meant not changing the code at incorrect place that it changes the purpose of that particular code
148 2011-07-26T18:16:31 <ThomasWaldmann> yes
149 2011-07-26T18:17:28 <ThomasWaldmann> for that tests cleanup it is important not just to try to "fix" it somehow, but to first be clear about how the test should really look like and what exactly it should test.
150 2011-07-26T18:17:47 <ThomasWaldmann> that is not always the same as the existing test code
151 2011-07-26T18:18:33 <pkumar> waldi : hey, though I looked into hardy89's repo I'm not sure about his top three changesets. can you please have a look so that it can be useful to us?
152 2011-07-26T18:18:51 <waldi> which repo?
153 2011-07-26T18:19:29 <pkumar> waldi : his latest one,
154 2011-07-26T18:19:37 <pkumar> lemme check
155 2011-07-26T18:20:05 <waldi> found it
156 2011-07-26T18:20:06 <pkumar> ThomasWaldmann : yeah,
157 2011-07-26T18:20:48 <pkumar> waldi : huh, I've got really slow connection,
158 2011-07-26T18:22:55 <pkumar> waldi : first an the third one I guess it is
159 2011-07-26T18:23:02 <pkumar> from top
160 2011-07-26T18:23:17 <waldi> no minidom. we already use two other implementations
161 2011-07-26T18:24:18 <pkumar> waldi : ok, so you mean the changes with moindom (3rd one) is not useful?
162 2011-07-26T18:24:41 <waldi> i'm saying that it reinvents the wheel
163 2011-07-26T18:25:15 <pkumar> waldi : ok, so do we keep it?
164 2011-07-26T18:25:19 <waldi> no
165 2011-07-26T18:25:26 <pkumar> ok
166 2011-07-26T18:25:41 <pkumar> waldi : what about the 1st from top
167 2011-07-26T18:25:56 <pkumar> one with fixing router backend?
168 2011-07-26T18:26:36 <waldi> i have no idea if it is correct. it changes both the tests and the tested code
169 2011-07-26T18:27:57 <pkumar> waldi : yeah, that's what make me little unsure about that changeset, though it fixes a failure, just fixing is not always correct
170 2011-07-26T18:29:28 <waldi> i have to go, sorry
171 2011-07-26T18:29:43 <pkumar> ok, thanks.
172 2011-07-26T18:29:51 <pkumar> bbl
173 2011-07-26T18:30:00 *** pkumar
174 2011-07-26T18:30:02 <ThomasWaldmann> hmm, bad timing today. dreimark also still away.
175 2011-07-26T18:30:22 <ThomasWaldmann> but sinha is not here anyway, so no problem. session closed for today.
176 2011-07-26T18:56:19 *** greg_f
177 2011-07-26T19:03:21 <brunomartin> ThomasWaldmann: Hi! I was thinking that I can contribute on indexed search in metadata. Seems that this is not the purpose of whoosh, is that right?
178 2011-07-26T19:06:24 <ThomasWaldmann> hi brunomartin
179 2011-07-26T19:06:40 <ThomasWaldmann> we use whoosh to search in metadata
180 2011-07-26T19:07:06 <ThomasWaldmann> it's in marchael's repo, not in main repo yet (he's doing it as a google summer of code project).
181 2011-07-26T19:07:58 <ThomasWaldmann> https://bitbucket.org/marchael/moin-2.0/overview there is moin2 with whoosh. it already works a bit, but not finished yet.
182 2011-07-26T19:08:08 <brunomartin> yes, I take a look at marchael's repo...
183 2011-07-26T19:08:50 <ThomasWaldmann> we just discovered this: https://bitbucket.org/mchaput/whoosh/issue/158/multitoken_query-strange-defaults-lead-to
184 2011-07-26T19:09:01 <ThomasWaldmann> so do not wonder if it behaves a little strange yet
185 2011-07-26T19:09:16 <brunomartin> I see...
186 2011-07-26T19:09:50 <ThomasWaldmann> but I am quite positive about that this will work out in the end
187 2011-07-26T19:10:46 <brunomartin> I was seeing you and Marchael talking early.. ^^
188 2011-07-26T19:10:49 <ThomasWaldmann> whoosh's dynamic fields can likely even replace that kvstore that I hacked a while ago to store arbitrary key/value pairs
189 2011-07-26T19:12:07 <Marchael> re
190 2011-07-26T19:12:21 <brunomartin> thats interesting... so that indexing module is not gonna be used anymore?
191 2011-07-26T19:12:30 <Marchael> ThomasWaldmann: that's not all
192 2011-07-26T19:12:43 <Marchael> I get tracebck for "and" also
193 2011-07-26T19:12:53 <ThomasWaldmann> brunomartin: i hope we can replace sqlalchemy by whoosh
194 2011-07-26T19:12:53 <Marchael> but appears only on contenttype
195 2011-07-26T19:13:23 <ThomasWaldmann> Marchael: "and" worked for me, see that bug report
196 2011-07-26T19:13:48 <Marchael> ThomasWaldmann: try contenttype:text/
197 2011-07-26T19:13:51 <ThomasWaldmann> and it sounds like we'll get sane defaults :)
198 2011-07-26T19:13:52 <Marchael> or contenttype:text
199 2011-07-26T19:14:47 <ThomasWaldmann> brunomartin: the indexing.py backend mixin is still used, but currently has a mix of sqla and whoosh. we'll get rid of sqla as soon as whoosh can do it all.
200 2011-07-26T19:15:41 <Marchael> ThomasWaldmann: there is traceback http://paste.pocoo.org/show/446858/
201 2011-07-26T19:16:07 <Marchael> but seems like it something related with analyzer
202 2011-07-26T19:17:48 <ThomasWaldmann> Marchael: that's rather deep inside whoosh
203 2011-07-26T19:18:49 <Marchael> ThomasWaldmann: hm, if yuo pass string like contenttype:text/ then anayzer parse it as "text", "None"
204 2011-07-26T19:18:59 <Marchael> may be that None cause that
205 2011-07-26T19:19:16 <ThomasWaldmann> it shouldn't give None as text
206 2011-07-26T19:19:23 <brunomartin> that sounds good! well, the point is, I are starting year long project here using moin 2, to make a prototype to end of the year, and possibly go on! it for a great university here for learning porpouses, and I we realy like to contribute with moin in this process... ^^
207 2011-07-26T19:19:50 <Marchael> ThomasWaldmann: yes, so may code what add for code review some days earlier not so crap :)
208 2011-07-26T19:20:04 <brunomartin> sorry about my poor english... I hope you can anderstand...
209 2011-07-26T19:20:23 <ThomasWaldmann> brunomartin: no problem :)
210 2011-07-26T19:20:46 <brunomartin> you are going to say we are nuts, but there already 300 users testing a hacked moin 2 today!
211 2011-07-26T19:20:51 <brunomartin> :-o
212 2011-07-26T19:20:51 <ThomasWaldmann> brunomartin: maybe you can help by looking at the code we already have.
213 2011-07-26T19:21:04 <brunomartin> exactly!
214 2011-07-26T19:21:04 <ThomasWaldmann> there is also an etherpad about it there: http://etherpad.osuosl.org/whoosh-moin
215 2011-07-26T19:21:47 <ThomasWaldmann> currently it is mostly about searching in metadata, content search hopefully soon coming :)
216 2011-07-26T19:22:23 <Marchael> yeah :)
217 2011-07-26T19:22:53 <ThomasWaldmann> if you want to play with it, look at the schema. e.g. you can search for language:de
218 2011-07-26T19:23:10 <ThomasWaldmann> (left side = field name from schema, right side = wanted value)
219 2011-07-26T19:23:32 <ThomasWaldmann> mtime:... is also interesting, range search works :)
220 2011-07-26T19:25:28 <brunomartin> metadata search is what I need now... I'm gonna merge Marchael's repo today...
221 2011-07-26T19:26:10 <brunomartin> ^^
222 2011-07-26T19:26:34 <Marchael> brunomartin: you're welcome :)
223 2011-07-26T19:27:07 <brunomartin> thanks
224 2011-07-26T19:28:02 <ThomasWaldmann> brunomartin: if you want to contribute, you can fork it on bitbucket, so we can easily see what you have
225 2011-07-26T19:28:56 <brunomartin> soon I are going to post some info about our project! we think it can be used as a case to promote moin here in brasil! ;-)
226 2011-07-26T19:29:44 <ThomasWaldmann> cool :)
227 2011-07-26T19:30:06 <Marchael> \o/
228 2011-07-26T19:30:17 <brunomartin> I think that now the code is too 'hacked'... :-P I will organize it for the next weeks an publish it... but we are using hg here already! ;-)
229 2011-07-26T19:30:20 <ThomasWaldmann> brunomartin: if you like, you could look at the more exotic features of whoosh
230 2011-07-26T19:31:13 <ThomasWaldmann> == stuff that is not directly related to search (and thus not in scope of marchael's work), but nice nevertheless
231 2011-07-26T19:33:44 <brunomartin> we have to fix some things thare are annoying users first, like speed it up (the index problem, couse i do some metadata serching here) and some problem with inserting images using html editor...
232 2011-07-26T19:34:39 <ThomasWaldmann> try to keep in mind that release target of moin 2.0 is a basic wiki :)
233 2011-07-26T19:36:02 <brunomartin> I know that... we think we are going to extend it with plugins, right?
234 2011-07-26T19:36:46 <ThomasWaldmann> we don't really have a plugin system yet
235 2011-07-26T19:37:05 <ThomasWaldmann> but we still search for good ideas for that
236 2011-07-26T19:38:32 <Marchael> brunomartin: We haven't test for offline index building script now, if you want to have fun with moin and whoosh, you could try write one :)
237 2011-07-26T19:39:18 <Marchael> you could also understand how we do indexing .
238 2011-07-26T19:39:29 <ThomasWaldmann> Marchael: don't outsource stuff in scope of your project :)
239 2011-07-26T19:39:40 <Marchael> ah
240 2011-07-26T19:40:37 <Marchael> I just want to give point for start ;)
241 2011-07-26T19:41:07 <brunomartin> ThomasWaldmann: yeah, I see that... that's why I hacked it all over.... :-P thats a place (plugins) where we can really contribute.... I'm starting with flask yet, but we have some hackers with great ideas here! :-P
242 2011-07-26T19:44:01 <ThomasWaldmann> brunomartin: the point about plugins is that we don't want 1.9-style plugins any more, but there is no good idea / plan yet about how to make better ones
243 2011-07-26T19:44:37 * ThomasWaldmann looked at some 3rd party plugin systems, but wasn't too impressed
244 2011-07-26T19:45:06 <brunomartin> and we really are going to extend moin 2 anyway... so if we can help on plugin system, we can try it while we help to build it... ;-)
245 2011-07-26T19:46:29 <brunomartin> I will discuss here some plugin system for the next days, and come back whit suggestions... is there any plugin system that you like?
246 2011-07-26T19:53:25 <Marchael> I made some fixes in analyzer, now it isn't crash
247 2011-07-26T19:53:35 <Marchael> *isn't crash moin
248 2011-07-26T19:55:03 *** raignarok
249 2011-07-26T19:55:29 <brunomartin> Marchael: I will pull your repo now... I will import 1GB of data, and do some search on that, and I give feedback to you... I think i'm going to start with that and than we talk about how can i help you! ;-)
250 2011-07-26T19:55:39 <Marchael> ThomasWaldmann: http://codereview.appspot.com/4819047/ patchset 2
251 2011-07-26T19:56:07 <Marchael> brunomartin: oh great thx!
252 2011-07-26T19:57:08 <brunomartin> Marchael: thank you too!
253 2011-07-26T19:59:34 <ThomasWaldmann> brunomartin: cool, testing with big dataset is still todo :)
254 2011-07-26T20:00:13 <brunomartin> ThomasWaldmann: we have very good designers here... you can assign leiaute tasks to us too.... ;-)
255 2011-07-26T20:04:02 <ThomasWaldmann> "leiaute"?
256 2011-07-26T20:04:47 <brunomartin> hahahaha... sory... its layout in portuguese... :-P
257 2011-07-26T20:04:55 <ThomasWaldmann> ah :)
258 2011-07-26T20:05:18 <Marchael> ThomasWaldmann: please review that small fix, I want to commit this stuff
259 2011-07-26T20:05:22 <ThomasWaldmann> i could have guess it by just reading it aloud
260 2011-07-26T20:06:36 <ThomasWaldmann> Marchael: done
261 2011-07-26T20:07:19 <ThomasWaldmann> Marchael: about "or": some years ago i had an encounter with an online shop selling electronic components and it used "or" for search terms
262 2011-07-26T20:07:54 <ThomasWaldmann> i could've killed whoever made that while trying to find some specific part.
263 2011-07-26T20:07:57 <Marchael> what's hapend with this shop?
264 2011-07-26T20:08:17 <Marchael> ah
265 2011-07-26T20:08:53 <ThomasWaldmann> if you search for a part that has properties A, B and C, you don't want to see thousands of parts that have property A, B or C
266 2011-07-26T20:09:27 <Marchael> so use "and'
267 2011-07-26T20:09:30 <Marchael> ?
268 2011-07-26T20:09:34 <ThomasWaldmann> and if you really have some "or" scenario, you can as well do multiple searches one after the other
269 2011-07-26T20:09:43 <ThomasWaldmann> yes
270 2011-07-26T20:12:24 *** sinha
271 2011-07-26T20:13:00 <Marchael> ThomasWaldmann: ID doesn't accept 'multitoken_query'
272 2011-07-26T20:13:10 <Marchael> so I can't use it for tags
273 2011-07-26T20:13:42 <Marchael> mmm
274 2011-07-26T20:14:33 <Marchael> me accidently found strage what in latest revs schema i use ID for tags, and TEXT in all revs
275 2011-07-26T20:14:44 <Marchael> *strange
276 2011-07-26T20:16:11 <Marchael> so, what I should use: ID or TEXT?
277 2011-07-26T20:17:10 <Marchael> I guess ID
278 2011-07-26T20:20:29 <Marchael> ThomasWaldmann: http://codereview.appspot.com/4819047/ patchset 3
279 2011-07-26T20:40:28 <Marchael> ThomasWaldmann: ?
280 2011-07-26T20:41:25 <brunomartin> Marchael: the option index_rebuild = True in wikiconfig already rebuild index with whoosh, or a have to do a moin index --for both --action build?
281 2011-07-26T20:42:25 <Marchael> brunomartin: you could disable that option because index building process can eat large amount of time
282 2011-07-26T20:42:46 <Marchael> and also you could build indexes in temp directory
283 2011-07-26T20:42:54 <Marchael> and then --action move
284 2011-07-26T20:43:03 <Marchael> will replace that
285 2011-07-26T20:43:10 <ThomasWaldmann> Marchael: ok
286 2011-07-26T20:43:14 <ThomasWaldmann> done
287 2011-07-26T20:44:42 <brunomartin> Marchael: yes, I reed that in docs! ;-) but I do want to do both things together... import and build index...
288 2011-07-26T20:46:14 <brunomartin> i got this trying that: http://paste.pocoo.org/show/446916/
289 2011-07-26T20:47:51 <Marchael> hm
290 2011-07-26T20:48:25 <Marchael> what command cause this/
291 2011-07-26T20:48:27 <Marchael> ?
292 2011-07-26T20:50:09 <Marchael> brunomartin: moin index --for both --action build works ok for me
293 2011-07-26T20:50:33 <dreimark> re
294 2011-07-26T20:55:01 <CIA-114> Michael Mayorov <marchael@kb.csu.ru> * d9d8e978323a r355 moin-2.0/MoinMoin/search/ (analyzers.py indexing.py):
295 2011-07-26T20:55:01 <CIA-114> Improvments in contenttype analyzer, fixes in schema
296 2011-07-26T20:55:01 <CIA-114> - contenttype analyzer doesn't crash moin with query cotenttype:text/
297 2011-07-26T20:55:01 <CIA-114> - added additional parameter 'multitoken_query' for improving search
298 2011-07-26T20:57:28 <brunomartin> Marchael: just moin... I removed the wiki dir, and set load_xml to my data, and index_rebuild = True...
299 2011-07-26T20:58:32 <Marchael> brunomartin: do you use hg clone to take latest changes? That's look strange
300 2011-07-26T20:59:53 <Marchael> for first run will be better use index_rebuild = True, later you can turn off this in wikiconfig
301 2011-07-26T21:00:17 <Marchael> as load_xml
302 2011-07-26T21:01:51 <ThomasWaldmann> oh, the acl analyser only works withing a request
303 2011-07-26T21:01:57 <sinha> dreimark: I was also not there in meeting time
304 2011-07-26T21:02:39 <Marchael> ThomasWaldmann: yes, but my moin starts ok
305 2011-07-26T21:03:01 <ThomasWaldmann> Marchael: that's because there are no acls in the default preload
306 2011-07-26T21:03:10 <ThomasWaldmann> but likely brunomartin has some in his xml, right?
307 2011-07-26T21:03:19 <brunomartin> yes, I have!
308 2011-07-26T21:03:23 <brunomartin> ;-)
309 2011-07-26T21:03:23 <Marchael> ah
310 2011-07-26T21:03:27 <ThomasWaldmann> brunomartin: good tester :)
311 2011-07-26T21:03:49 <brunomartin> ThomasWaldmann: thanks!
312 2011-07-26T21:03:55 <brunomartin> :-)
313 2011-07-26T21:04:39 <ThomasWaldmann> brunomartin: you could file a bug in marchaels issue tracker so he does not forget
314 2011-07-26T21:07:27 <Marchael> yes, I never forget this
315 2011-07-26T21:07:33 <Marchael> :)
316 2011-07-26T21:09:23 <brunomartin> ok, I file it! ;-) I was looking at code, and seems that is not possible to use stored.terms with whoosh yet, right?
317 2011-07-26T21:11:13 <Marchael> ThomasWaldmann: m, this problem very similiar on problem with content indexing
318 2011-07-26T21:16:52 <ThomasWaldmann> yes
319 2011-07-26T21:17:38 <ThomasWaldmann> but iirc you did not try my new code for content conversion yet
320 2011-07-26T21:18:02 <Marchael> I neccesarily try that
321 2011-07-26T21:20:38 <dreimark> sinha: ThomasWaldmann can we continue with the meeting in a quarter ?
322 2011-07-26T21:20:47 * dreimark had read backlog
323 2011-07-26T21:20:59 <sinha> yes i will be here
324 2011-07-26T21:21:29 <brunomartin> Marchael: bug reported!
325 2011-07-26T21:21:33 <ThomasWaldmann> dreimark: maybe (still at work, but want to go home at some time :)
326 2011-07-26T21:21:40 <Marchael> brunomartin: thx!
327 2011-07-26T21:22:53 <brunomartin> Marchael: I was looking at code, and seems that is not possible to use stored.terms with whoosh yet, right? will stored.terms still be used or should I avoid using it?
328 2011-07-26T21:23:41 <Marchael> you could try
329 2011-07-26T21:24:08 <Marchael> But I didn't test that stuff yet
330 2011-07-26T21:26:26 <ThomasWaldmann> brunomartin: what's the problem with stored?
331 2011-07-26T21:27:38 <Marchael> ThomasWaldmann: I saw your last changes. I found converter, but how can i access to item content at start up? MoinMoin.apps.frontend.views.indexable() using flaskg which unreacheble in that time
332 2011-07-26T21:28:22 <brunomartin> Marchael: I don't know how to do that... :-| I have to build that index and do some search before... I will try to solve this bug I have found... everything else in search seems to be working...
333 2011-07-26T21:28:48 <brunomartin> homasWaldmann: no problems, just trying to understand the big picture...
334 2011-07-26T21:29:09 <ThomasWaldmann> well, if you can index metadata, you already have the revision (otherwise you could not access metadata either)
335 2011-07-26T21:37:50 <Marchael> gn/bbl
336 2011-07-26T21:40:48 <dreimark> brunomartin: if you can make a break we could have a meeting with sinha
337 2011-07-26T21:41:28 <dreimark> sinha: please start when you are ready
338 2011-07-26T21:41:32 <brunomartin> dreimark: oh... sorry...
339 2011-07-26T21:41:38 <sinha> okay
340 2011-07-26T21:41:59 <dreimark> brunomartin: no problem we both missed the today meeting
341 2011-07-26T21:42:16 <dreimark> and because of the time difference we should do it
342 2011-07-26T21:43:18 <sinha> LAST: i) Moved the fileupload functionality from index2 to index, and thus removed the index2 completely.
343 2011-07-26T21:43:21 <sinha> Changeset: https://bitbucket.org/sinha/moin2/changeset/17132086b9d6
344 2011-07-26T21:43:21 <sinha> ii) Moved the paging code for item history to a new function under util, so that it could be used by some other views.
345 2011-07-26T21:43:21 <sinha> Changeset: https://bitbucket.org/sinha/moin2/changeset/b9f0ba2ba779
346 2011-07-26T21:43:21 <sinha> iii) Updated the contenttype filter code, because of lack of encoding information for uploaded files, some extra checks have been added.
347 2011-07-26T21:43:21 <sinha> (Pushing it after some small corrections) Currently at codereview: http://codereview.appspot.com/4813050
348 2011-07-26T21:43:21 <sinha> BLOCKER: the correct solution for this encoding problem would be to add encoding on file upload, otherwise this contenttye filter has became too much complex.
349 2011-07-26T21:43:21 <sinha> CURRENT: Currently i am working on the view part at item index page, So now i will be adding the file download functionality at index page.
350 2011-07-26T21:43:22 <sinha> NEXT: Will work on some other collective actions on item index (ex. Delete, Destroy, Move(maybe))
351 2011-07-26T21:46:33 <dreimark> sinha: we discussed some time ago to have the upload box extended to the whole page, can you also work on that. so that one can drop something into the white are between items
352 2011-07-26T21:47:33 <dreimark> for file download we should define a contenttype or test with application/octet-stream
353 2011-07-26T21:47:51 <sinha> dreimark: yes i will do that, actually the problem of drag and drop on my PC was due to some problem with nautilus itself, it also doesnt work with media players and other tools
354 2011-07-26T21:48:14 <dreimark> the latter one has on my firefox an issue, i can't tell don't ask me again
355 2011-07-26T21:49:12 <dreimark> but we should think on a mimetype for sending all downloads, because than this mimetype can be configured on the browser for download
356 2011-07-26T21:49:45 <dreimark> if we enable all mimetypes possible this will ever download everywhere - and some downloads you don't like
357 2011-07-26T21:49:54 <sinha> okay so for multiple downloads are we sending them all together or one by one ( as that multiDownload does)
358 2011-07-26T21:50:07 <dreimark> as multiDownload does
359 2011-07-26T21:50:43 <dreimark> but I think we should send it as a different mimetype as it is. and we should invent one which is for sending from moin
360 2011-07-26T21:51:39 <dreimark> if i see this correct then the mimetype is only needed for configurarion by the browser
361 2011-07-26T21:52:01 <dreimark> for download.
362 2011-07-26T21:52:10 <sinha> I havent undertood this different mimetype thing, you mean for the file "text.jpg" we will send it as "application/octet-strem" or something and noot like "image/jpeg" ?
363 2011-07-26T21:52:16 <sinha> s/undetood/understood
364 2011-07-26T21:52:35 <dreimark> i try again
365 2011-07-26T21:53:31 <dreimark> if we send text.jpg as image/jpeg the user has to set always save for this mimetype to get not annoyed by lots of windows
366 2011-07-26T21:53:45 <sinha> yes
367 2011-07-26T21:53:47 <dreimark> the same for zip, pdf etc
368 2011-07-26T21:54:00 <sinha> true
369 2011-07-26T21:54:25 <dreimark> if we send it as something else e.g. application/x-moin-save
370 2011-07-26T21:54:48 <dreimark> then the user has only to set for this mimetype that he wants it stored
371 2011-07-26T21:55:21 <sinha> okay, i get it now
372 2011-07-26T21:55:26 <sinha> we will say this as "Moin type file"
373 2011-07-26T21:55:33 <dreimark> yes
374 2011-07-26T21:57:30 <dreimark> sinha: for other collective actions you should also think how to filter
375 2011-07-26T21:57:41 <dreimark> so that one can select by such a filter
376 2011-07-26T21:58:09 <ThomasWaldmann> sinha: after you finished your current topic, look there https://bitbucket.org/sinha/moin2/changeset/b9f0ba2ba779#chg_MoinMoin/util/__init__.py_newline100
377 2011-07-26T21:58:10 <dreimark> less clicks to select all images
378 2011-07-26T21:59:05 <ThomasWaldmann> (that loop looks rather strange, like it should be rather results[offset:offset+results_per_page] or so
379 2011-07-26T21:59:16 <sinha> dreimark: yes one way is we could use the same contenttype filter which is at global index, and one i can add Javascript contenttype selector
380 2011-07-26T22:00:17 <dreimark> sinha: for that refactoring thomas askes a unit test would be nice too
381 2011-07-26T22:02:22 <sinha> ThomasWaldmann: whats in that ? are you pointing towards variable naming ?
382 2011-07-26T22:02:50 <dreimark> sinha: it is the loop itself, it may be that it can be much simplified
383 2011-07-26T22:02:54 <ThomasWaldmann> no, but I don't understand why you use that loop
384 2011-07-26T22:03:07 <ThomasWaldmann> you know l[from:to] syntax?
385 2011-07-26T22:04:09 <sinha> ThowasWaldmann: yes i know, but there the results is supposed to be a generator not a list
386 2011-07-26T22:04:22 <sinha> If it would have been a list (as in index) i wont be using any loop
387 2011-07-26T22:06:15 <dreimark> sinha: can global index and item index ne unified, so that we have only one of these functions
388 2011-07-26T22:06:31 <dreimark> otherwise some of the code must be duplicated on both
389 2011-07-26T22:06:42 <dreimark> s/ne/be/
390 2011-07-26T22:07:10 <ThomasWaldmann> sinha: hmm, i somehow thought it would work the same way then
391 2011-07-26T22:07:35 <sinha> dreimark: yes it could be, but there is some extra features at global index now, say startswith selection and contenttype filtering, so should also be everywhere ?
392 2011-07-26T22:07:56 <dreimark> yes
393 2011-07-26T22:07:56 <sinha> otherwise there isnt any problem in unifying
394 2011-07-26T22:09:30 <dreimark> i think unifying should be done before further extensions, less work then
395 2011-07-26T22:09:37 <sinha> dreimark: okay and file upload also ?
396 2011-07-26T22:09:38 <sinha> we can add items directly to top level
397 2011-07-26T22:09:44 <dreimark> yes
398 2011-07-26T22:10:30 <ThomasWaldmann> that stuff was unified in the past iirc
399 2011-07-26T22:10:42 <sinha> okay, cool so should i start with the unifying now, before doing any more changes /
400 2011-07-26T22:10:44 <sinha> ?
401 2011-07-26T22:10:47 <ThomasWaldmann> one just needs the concept of a virtual root item, e.g. named ""
402 2011-07-26T22:11:37 <dreimark> ThomasWaldmann: yes, but is this an argument for or against having only one view for both?
403 2011-07-26T22:12:08 <ThomasWaldmann> i didn't read all of your discussion. is there some argument against?
404 2011-07-26T22:12:27 <dreimark> no
405 2011-07-26T22:12:46 <dreimark> i just ordered the work, first unifying then new features
406 2011-07-26T22:12:58 <ThomasWaldmann> btw return (selected_result, next_offset, previous_offset) < no need for () there
407 2011-07-26T22:13:51 <ThomasWaldmann> that's because "," is the tuple operator, not ()
408 2011-07-26T22:13:52 <dreimark> it is ever a tuple
409 2011-07-26T22:14:17 <ThomasWaldmann> not ever, but if you use a comma :)
410 2011-07-26T22:15:08 <sinha> ThomasWaldmann: okay, i wasn't aware of that
411 2011-07-26T22:15:09 <dreimark> ever with comma without clamps
412 2011-07-26T22:15:10 <sinha> i will remove that
413 2011-07-26T22:15:41 <ThomasWaldmann> sinha: that getPageContent maybe should get a better name
414 2011-07-26T22:15:53 <ThomasWaldmann> with that name, it could be also something rather different
415 2011-07-26T22:15:58 * dreimark strange I thought clamps is somehting else, that was google translator
416 2011-07-26T22:16:19 <ThomasWaldmann> parens :)
417 2011-07-26T22:17:00 <dreimark> that word is unknows
418 2011-07-26T22:17:08 <dreimark> s/s/n/
419 2011-07-26T22:17:14 <dreimark> however
420 2011-07-26T22:17:50 <ThomasWaldmann> the long form is parentheses
421 2011-07-26T22:19:30 <dreimark> sinha: order is unifying, beautifying, new functions
422 2011-07-26T22:19:39 <ThomasWaldmann> sinha: yes, you're right about grammar. it is either "is" or possessive.
423 2011-07-26T22:19:58 <sinha> :)
424 2011-07-26T22:20:36 <ThomasWaldmann> http://www.angryflower.com/bobsqu.gif
425 2011-07-26T22:20:59 <dreimark> ah a new onw
426 2011-07-26T22:21:10 <ThomasWaldmann> not really
427 2011-07-26T22:22:01 <dreimark> nice example btw. with my name
428 2011-07-26T22:22:55 <sinha> dreimark: i thought you must have heard that earlier :)
429 2011-07-26T22:23:41 <dreimark> not in en
430 2011-07-26T22:25:07 <dreimark> i still struggle by reading the item's sentence, but ignore me
431 2011-07-26T22:25:56 <dreimark> further questions to sinha ?
432 2011-07-26T22:26:42 <ThomasWaldmann> not from me right now
433 2011-07-26T22:27:01 <dreimark> ok, thanks for attending the late meeting ;)
434 2011-07-26T22:27:33 <sinha> ofcourse
435 2011-07-26T22:27:37 * ThomasWaldmann should stop using/selling harddisks, they are wasting too much time
436 2011-07-26T22:29:23 <sinha> dreimark: okay i better get started with unifying them. Thanks.
437 2011-07-26T22:29:47 <ThomasWaldmann> how many indexes do we have right now? 2?
438 2011-07-26T22:30:07 <sinha> yes global and item;s
439 2011-07-26T22:30:29 <ThomasWaldmann> ok. have a good plan. :)
440 2011-07-26T22:30:37 * ThomasWaldmann drives home
441 2011-07-26T22:30:43 <sinha> after unifying will have just 1
442 2011-07-26T22:32:17 <dreimark> sinha: please add the concept to ep, roughly
443 2011-07-26T22:36:13 <sinha> okay, will do
444 2011-07-26T22:56:00 <CIA-114> Akash Sinha <akash2607@gmail.com> * a286df67f294 r338 default/MoinMoin/items/__init__.py: contenttype filter code updated, due to lack of encoding information for uploaded files, some extra checks have been added.
445 2011-07-26T23:54:03 *** sinha
446
