1 2011-08-07T00:48:14 <ThomasWaldmann> ok, sqlalchemy based indexing gone \o/
2 2011-08-07T03:00:20 *** grzywacz
3 2011-08-07T06:12:50 *** Marchael
4 2011-08-07T06:21:01 <Marchael> moin
5 2011-08-07T06:25:44 <ThomasWaldmann> moin Marchael
6 2011-08-07T06:27:16 <Marchael> ThomasWaldmann: for sorting may be use something like sorted(docs, key=itemgetter("rev_no"), reverse=reverse)[start:end] ?
7 2011-08-07T06:29:58 <ThomasWaldmann> yes
8 2011-08-07T06:30:27 <ThomasWaldmann> ehrm, for history?
9 2011-08-07T06:30:47 <ThomasWaldmann> then it is rather mtime
10 2011-08-07T06:31:23 <ThomasWaldmann> or first mtime, then rev_no
11 2011-08-07T06:32:08 <Marchael> seems like rev_no ok
12 2011-08-07T06:33:23 <Marchael> at least all revisions ordere
13 2011-08-07T06:33:28 <Marchael> *ordered
14 2011-08-07T06:33:32 <ThomasWaldmann> it won't work for global history
15 2011-08-07T06:33:45 <Marchael> ah
16 2011-08-07T06:33:48 <Marchael> ok
17 2011-08-07T06:34:17 <ThomasWaldmann> but you can use it as 2nd sort criteria to maybe avoid mtime granularity issues
18 2011-08-07T06:36:22 <ThomasWaldmann> btw, see pull request in your repo
19 2011-08-07T06:36:54 <Marchael> hm, /me doesn't revieve any notice to email
20 2011-08-07T06:37:07 <ThomasWaldmann> yes, because your email does not work
21 2011-08-07T06:37:11 <Marchael> ok, I check this
22 2011-08-07T06:44:08 <CIA-114> Thomas Waldmann <tw AT waldmann-edv DOT de> * 951c8edf9776 r401 moin-2.0/MoinMoin/storage/backends/indexing.py:
23 2011-08-07T06:44:08 <CIA-114> remove ItemIndex.history (was replaced by history_whoosh)
24 2011-08-07T06:44:08 <CIA-114> remove all kvstore usage (was intended to store user-defined metadata,
25 2011-08-07T06:44:08 <CIA-114> but we'll use whoosh dynamic fields for that soon)
26 2011-08-07T06:44:09 <CIA-114> Thomas Waldmann <tw AT waldmann-edv DOT de> * 2dc35ebbc43a r402 moin-2.0/MoinMoin/util/kvstore.py: removed MoinMoin.util.kvstore - not used any more
27 2011-08-07T06:44:09 <CIA-114> Thomas Waldmann <tw AT waldmann-edv DOT de> * 256c6e0ae8bf r403 moin-2.0/MoinMoin/storage/backends/indexing.py: removed all_tags / tagged_items, replaced by *_whoosh
28 2011-08-07T06:44:10 <CIA-114> Thomas Waldmann <tw AT waldmann-edv DOT de> * 72278e056ec4 r404 moin-2.0/MoinMoin/storage/backends/indexing.py: indexing: removed/commented unused code
29 2011-08-07T06:44:10 <CIA-114> Thomas Waldmann <tw AT waldmann-edv DOT de> * 8cc72401fc25 r405 moin-2.0/ (9 files in 6 dirs): (log message trimmed)
30 2011-08-07T06:44:11 <CIA-114> indexing: remove sqlalchemy based indexing code (is done by whoosh now)
31 2011-08-07T06:44:11 <CIA-114> rename some methods from *_whoosh to * (as there are no others any more,
32 2011-08-07T06:44:12 <CIA-114> all indexing is based on whoosh now).
33 2011-08-07T06:44:12 <CIA-114> added some "XXX wrong" comments to indexing, where item and revision level
34 2011-08-07T06:44:13 <CIA-114> was confused.
35 2011-08-07T06:44:13 <CIA-114> cfg.index_uri was removed as well as index_uri Backend kwarg (both were only
36 2011-08-07T06:44:58 <CIA-114> Thomas Waldmann <tw AT waldmann-edv DOT de> * 598b1731069b r406 moin-2.0/MoinMoin/ (config/default.py app.py storage/backends/indexing.py): remove cfg.index_rebuild and related code (not needed any more)
37 2011-08-07T06:45:54 <Marchael> ThomasWaldmann: great thanks!
38 2011-08-07T06:49:25 <ThomasWaldmann> still lots to do left :)
39 2011-08-07T06:49:50 <Marchael> heh
40 2011-08-07T06:54:14 <Marchael> ThomasWaldmann: there is small cs http://codereview.appspot.com/4832056
41 2011-08-07T06:54:29 <Marchael> I think that mtime enough for sorting
42 2011-08-07T07:06:37 <Marchael> ThomasWaldmann: ping
43 2011-08-07T07:31:58 <Marchael> argh, server goind down due costetic work in lab.
44 2011-08-07T07:45:15 *** MattMaker
45 2011-08-07T07:45:37 *** MattMaker
46 2011-08-07T08:04:52 *** pkumar
47 2011-08-07T08:14:35 <pkumar> xorAxAx : hi, read the logs. sorry for the delay. so I need to make a changeset in which the test fails and then the next one in which it passes right?
48 2011-08-07T08:16:27 <pkumar> for changeset 35939c292237, will make it better so that it won't be related to repr
49 2011-08-07T08:17:41 <pkumar> will backout hardy89's changesets :)
50 2011-08-07T08:20:25 <pkumar> thanx
51 2011-08-07T08:46:01 *** pkumar
52 2011-08-07T08:58:49 *** pkumar
53 2011-08-07T09:13:55 *** m4k3r
54 2011-08-07T09:27:19 *** m4k3r
55 2011-08-07T09:28:20 *** m4k3r
56 2011-08-07T09:32:39 *** pkumar
57 2011-08-07T09:43:11 *** raignarok
58 2011-08-07T09:46:53 *** m4k3r
59 2011-08-07T09:47:28 *** m4k3r
60 2011-08-07T10:22:24 *** Marchael
61 2011-08-07T10:23:49 *** Marchael
62 2011-08-07T10:35:53 *** raignarok
63 2011-08-07T11:25:21 *** raignarok
64 2011-08-07T12:16:29 *** greg_f
65 2011-08-07T12:28:24 <ThomasWaldmann> Marchael: how big is the maximum mtime granularity so you can be sure that mtime is enough for sorting?
66 2011-08-07T12:35:37 *** raignarok
67 2011-08-07T12:37:20 <ThomasWaldmann> Marchael: if the answer is "i don't know", maybe you should consider using mtime and revno, as I advised
68 2011-08-07T12:38:05 <ThomasWaldmann> otherwise sort order could be wrong for fast changes (more than one change within mtime granularity time span)
69 2011-08-07T13:01:17 *** raignarok
70 2011-08-07T13:07:26 <ThomasWaldmann> Marchael: btw, where are your diary entries?
71 2011-08-07T13:07:46 <Marchael> hm, oops
72 2011-08-07T13:08:54 <ThomasWaldmann> if i look at the diary entries of last 4 weeks, they're telling something
73 2011-08-07T13:09:28 <Marchael> I guess better see on my repo on bb :)
74 2011-08-07T13:10:29 <ThomasWaldmann> still much too slow progress
75 2011-08-07T13:10:47 <ThomasWaldmann> you realized it's over in 2 weeks?
76 2011-08-07T13:11:17 <ThomasWaldmann> or rather "final evals begin in 2 weeks" (not "over")
77 2011-08-07T13:16:09 <Marchael> yes, I'll try to work faster.
78 2011-08-07T13:17:10 <Marchael> datetime safe time with precision up to a second
79 2011-08-07T13:17:19 <Marchael> may it'll be enough?
80 2011-08-07T13:17:43 <Marchael> s/safe/saved/
81 2011-08-07T13:17:44 <ThomasWaldmann> if you automatically edit/upload stuff, you can save same item multiple times a second
82 2011-08-07T13:23:12 <Marchael> ThomasWaldmann: ps 2 http://codereview.appspot.com/4832056
83 2011-08-07T13:26:56 <ThomasWaldmann> http://codereview.appspot.com/4832056/diff2/1:2001/MoinMoin/storage/backends/indexing.py wtf?
84 2011-08-07T13:30:11 <Marchael> I forget to pull my test repo
85 2011-08-07T13:30:43 <Marchael> ah, no
86 2011-08-07T13:31:43 <Marchael> ThomasWaldmann: you're looking on delta with previous cs, I pulled your changes after that code
87 2011-08-07T13:33:05 <Marchael> so that diff showing changes before pull and after.
88 2011-08-07T13:33:23 <Marchael> I could start new issue if it confusing you.
89 2011-08-07T13:35:04 *** greg_f
90 2011-08-07T13:35:41 *** greg_f
91 2011-08-07T13:37:22 <Marchael> hm, no I'm wrong. still need pull in my test repo.
92 2011-08-07T13:38:39 <xorAxAx> pkumar, no, you dont need two changesets, you just need to do it locally
93 2011-08-07T13:42:04 <Marchael> ThomasWaldmann: ps 3
94 2011-08-07T13:45:00 <ThomasWaldmann> Marchael: why are you showing my own changes to me?
95 2011-08-07T13:47:20 <Marchael> it was in ps 2, because I forgot make pull
96 2011-08-07T13:47:48 *** m4k3r
97 2011-08-07T13:48:37 <ThomasWaldmann> if you like a review, make a new clean one
98 2011-08-07T13:50:07 <Marchael> ThomasWaldmann: http://codereview.appspot.com/4852044
99 2011-08-07T13:51:18 <dreimark> may be i am blind but what is the differentce to 4832056/patch/5001/6001
100 2011-08-07T13:54:35 *** m4k3r
101 2011-08-07T13:55:28 <dreimark> Marchael: ^
102 2011-08-07T13:58:46 <Marchael> dreimark: that was another repo
103 2011-08-07T13:58:55 <ThomasWaldmann> Marchael: ok
104 2011-08-07T13:59:00 <ThomasWaldmann> commit
105 2011-08-07T13:59:31 <Marchael> ok
106 2011-08-07T14:01:41 <CIA-114> Michael Mayorov <marchael@kb.csu.ru> * 0f9a97f74a8d r407 moin-2.0/MoinMoin/storage/backends/indexing.py: Fixed https://bitbucket.org/marchael/moin-2.0/issue/3/dicts-are-bad-for-sorting Fixes #3.
107 2011-08-07T14:02:07 <Marchael> now issue closed.
108 2011-08-07T14:02:28 <ThomasWaldmann> good, next one :)
109 2011-08-07T14:02:46 <ThomasWaldmann> #21
110 2011-08-07T14:03:11 <ThomasWaldmann> see the comments there, fix sourcecode failures. 10 minutes.
111 2011-08-07T14:07:42 *** pkumar
112 2011-08-07T14:15:04 <pkumar> xorAxAx : hi, I generally makes a test to fail by incorrect assert or other similar ways after your suggestion in previous review. Is it the same you are referring to or I'm missing something?
113 2011-08-07T14:15:41 <Marchael> ThomasWaldmann: http://codereview.appspot.com/4810079
114 2011-08-07T14:15:48 <xorAxAx> pkumar: not by an assert, in the best case, you modify the code of moin to behave incorrectly and then check whether the test catches the bug
115 2011-08-07T14:16:50 <ThomasWaldmann> Marchael: test_sourcecode has no fails any more after that?
116 2011-08-07T14:17:32 <Marchael> yes
117 2011-08-07T14:17:36 <pkumar> xorAxAx: ah, got it :) will do it.
118 2011-08-07T14:18:21 *** m4k3r
119 2011-08-07T14:18:31 <ThomasWaldmann> ok, commit, next one
120 2011-08-07T14:20:11 <ThomasWaldmann> Marchael: did you look at the acl tokenizer / search tests?
121 2011-08-07T14:20:49 <CIA-114> Michael Mayorov <marchael@kb.csu.ru> * 6f6a29825a1e r408 moin-2.0/MoinMoin/ (3 files in 3 dirs): Fixed test_sourcecode failures
122 2011-08-07T14:21:08 <Marchael> yes, It failed with due ACL tokenizer
123 2011-08-07T14:21:22 <ThomasWaldmann> test_indexing.py
124 2011-08-07T14:21:56 <Marchael> ah, yes
125 2011-08-07T14:24:36 <Marchael> hm, seems like that happening due updating AccessControlList
126 2011-08-07T14:28:23 *** m4k3r
127 2011-08-07T14:28:45 <ThomasWaldmann> you found why the test fails?
128 2011-08-07T14:29:54 <Marchael> incorrect search string
129 2011-08-07T14:30:01 <Marchael> s/search/query/
130 2011-08-07T14:30:35 <ThomasWaldmann> what exactly is incorrect?
131 2011-08-07T14:31:37 <Marchael> ThomasWaldmann: see http://paste.pocoo.org/show/454133/
132 2011-08-07T14:31:51 <Marchael> > u'JoeDoe:-write'
133 2011-08-07T14:31:51 <Marchael> > u'JoeDoe:+write'
134 2011-08-07T14:32:24 <ThomasWaldmann> that pastebin looks correct. so?
135 2011-08-07T14:34:00 <Marchael> hm, do you see traceback?
136 2011-08-07T14:34:04 <Marchael> of py.test
137 2011-08-07T14:34:33 <ThomasWaldmann> yes, i wanted your analysis :)
138 2011-08-07T14:34:35 <Marchael> it rather obviuos And([Term(u'acl', u'JoeDoe:-read'), Term(u'acl', u'JoeDoe:-...'JoeDoe:-admin'), Term(u'acl', u'JoeDoe:-destroy')]) runtime=0.000859022140503>) == 1
139 2011-08-07T14:34:40 <Marchael> > And
140 2011-08-07T14:35:26 <Marchael> token set are different
141 2011-08-07T14:35:39 <Marchael> and may be too strict
142 2011-08-07T14:35:47 <ThomasWaldmann> assert len(<Top 0 Results for And([Term(u'acl', u'JoeDoe:+read'), Term(u'acl', u'JoeDoe:-... u'JoeDoe:-admin'), Term(u'acl', u'JoeDoe:-destroy')]) runtime=0.0013861656189>) == 1
143 2011-08-07T14:35:57 <ThomasWaldmann> note that it is +read
144 2011-08-07T14:36:06 <ThomasWaldmann> so the query is correct
145 2011-08-07T14:36:13 <Marchael> query is correct
146 2011-08-07T14:36:25 <Marchael> but JoeDoe:+read != JoeDoe:read,write
147 2011-08-07T14:37:00 <Marchael> so if I want to solve this I shold correct search query
148 2011-08-07T14:37:18 <ThomasWaldmann> JoeDoe:read,write is tokenized into: JoeDoe:+read JoeDoe:+write JoeDoe:-anythingelse
149 2011-08-07T14:37:29 <ThomasWaldmann> so why is JoeDoe:+read not found?
150 2011-08-07T14:37:39 <Marchael> because And
151 2011-08-07T14:38:06 <ThomasWaldmann> exactly. do we want that?
152 2011-08-07T14:38:21 <Marchael> I guess no
153 2011-08-07T14:38:22 <ThomasWaldmann> https://bitbucket.org/marchael/moin-2.0/issue/13/acl-search
154 2011-08-07T14:38:42 <ThomasWaldmann> the point here is to specify first what we expect from acl search
155 2011-08-07T14:39:09 <Marchael> need use MultiFilter
156 2011-08-07T14:40:09 <Marchael> ThomasWaldmann: or may be use "or"
157 2011-08-07T14:40:17 <Marchael> I mean multitoken_query="or"
158 2011-08-07T14:40:17 <ThomasWaldmann> no
159 2011-08-07T14:40:31 <Marchael> why?
160 2011-08-07T14:40:44 <ThomasWaldmann> if you are searching for JoeDoe:read, you do not want to find where he is not admin
161 2011-08-07T14:40:50 <ThomasWaldmann> but where he may read :)
162 2011-08-07T14:41:44 <ThomasWaldmann> so, i think we must process input differently for query than for indexing
163 2011-08-07T14:41:52 <Marchael> so then MultiFilter
164 2011-08-07T14:42:24 <ThomasWaldmann> afaik our indexing is fine, it expands everything so it is explicit
165 2011-08-07T14:43:08 <ThomasWaldmann> so rights given are explicitely there as +right, rights denied are expl. there as -right
166 2011-08-07T14:43:10 <Marchael> so tokenizer in query mode should just extract one token?
167 2011-08-07T14:43:45 <ThomasWaldmann> query mode shall operate differently. not make up things that one did not state
168 2011-08-07T14:43:47 <Marchael> For example if it [u'JoeDoe:+read', u'JoeDoe:-write', u'JoeDoe:-create', u'JoeDoe:-admin', u'JoeDoe:-destroy']
169 2011-08-07T14:43:48 <Marchael> it yield u'JoeDoe:+read
170 2011-08-07T14:44:25 <ThomasWaldmann> so i guess we want to just use the expanded syntax for user queries and use them "as is"
171 2011-08-07T14:44:40 <ThomasWaldmann> i.e. JoeDoe:+read
172 2011-08-07T14:45:10 <Marchael> hm, you mean not tokenize them?
173 2011-08-07T14:45:16 <ThomasWaldmann> yes
174 2011-08-07T14:45:27 <Marchael> and user want to search for something complex?
175 2011-08-07T14:45:34 <ThomasWaldmann> like?
176 2011-08-07T14:45:56 <Marchael> like JoeDoe:+read,write for example
177 2011-08-07T14:46:14 <ThomasWaldmann> JoeDoe:+read JoeDoe:+write
178 2011-08-07T14:46:14 <Marchael> I guess then search doesn't find anything
179 2011-08-07T14:46:27 <Marchael> and what about whitespace?
180 2011-08-07T14:46:47 <ThomasWaldmann> we can also use a similar tokenizer than for index. but do not add stuff that is not there.
181 2011-08-07T14:46:49 <Marchael> I guess whoosh will be think that JoeDoe:+write is name
182 2011-08-07T14:48:18 <Marchael> you mean second tokenizer or?
183 2011-08-07T14:48:57 <ThomasWaldmann> i guess we rather do the "as is" approach
184 2011-08-07T14:49:23 <ThomasWaldmann> because you can't use AccessControlList class for this special interpretation
185 2011-08-07T14:49:49 <Marchael> I don't want that.
186 2011-08-07T14:50:22 <Marchael> I just suggested to use MultiFilter with two modes
187 2011-08-07T14:51:05 <Marchael> so, in query mode tokenizer yield only tokens what contain in query
188 2011-08-07T14:52:47 <ThomasWaldmann> yes, of course you need MultiFilter
189 2011-08-07T14:53:06 <ThomasWaldmann> and for query, it just uses stuff as is
190 2011-08-07T14:54:15 <Marchael> are you sure what searcher can find results with query JoeDoe:+read,write ?
191 2011-08-07T14:54:29 <Marchael> or rather JoeDoe:+read JoeDoe:+write
192 2011-08-07T14:55:35 <ThomasWaldmann> quite sure, yes, only the 2nd query will work
193 2011-08-07T14:55:57 <ThomasWaldmann> btw, ACLTokenizer.__call__ docstring looks outdated
194 2011-08-07T14:57:05 <ThomasWaldmann> or rather mismatching with the code
195 2011-08-07T14:57:43 <ThomasWaldmann> https://bitbucket.org/marchael/moin-2.0/issue/13/acl-search there is a bug for that, btw
196 2011-08-07T14:58:58 <Marchael> ok
197 2011-08-07T15:01:05 <ThomasWaldmann> bug updated
198 2011-08-07T15:02:00 <ThomasWaldmann> i suggest you first fix the docstring to reflect code behaviour, commit.
199 2011-08-07T15:02:20 <ThomasWaldmann> then adjust the tests, so they test for the behaviour we want, commit.
200 2011-08-07T15:02:37 <ThomasWaldmann> then use Multifilter to make the test succeed
201 2011-08-07T15:03:31 <ThomasWaldmann> check if that fixes #13
202 2011-08-07T15:03:31 <Marchael> ok
203 2011-08-07T15:03:40 <ThomasWaldmann> if so, commit & close 13
204 2011-08-07T15:04:30 *** pkumar
205 2011-08-07T15:08:26 <Marchael> ThomasWaldmann: there is new docstring http://codereview.appspot.com/4852045
206 2011-08-07T15:11:32 <ThomasWaldmann> ok
207 2011-08-07T15:13:16 <CIA-114> Michael Mayorov <marchael@kb.csu.ru> * 0173477e11bc r409 moin-2.0/MoinMoin/search/analyzers.py: Update docstring
208 2011-08-07T15:15:08 <Marchael> ThomasWaldmann: there is new query string for acl http://codereview.appspot.com/4856042/patch/1/2
209 2011-08-07T15:17:03 <ThomasWaldmann> yes, sure
210 2011-08-07T15:17:20 <ThomasWaldmann> you could also add the one checking for read and write
211 2011-08-07T15:19:25 <Marchael> ThomasWaldmann: ps 2
212 2011-08-07T15:21:34 <ThomasWaldmann> no
213 2011-08-07T15:22:17 <ThomasWaldmann> https://bitbucket.org/marchael/moin-2.0/issue/13/acl-search just read that
214 2011-08-07T15:22:45 <ThomasWaldmann> and don't overuse codereview. it doesn't replace thinking by yourself.
215 2011-08-07T15:25:47 *** sinha
216 2011-08-07T15:26:12 <Marchael> ThomasWaldmann: you mean add All:read ?
217 2011-08-07T15:26:22 <Marchael> s/read/+read/
218 2011-08-07T15:29:16 <Marchael> no
219 2011-08-07T15:29:22 <Marchael> it won't working
220 2011-08-07T15:31:18 <ThomasWaldmann> why?
221 2011-08-07T15:32:03 <Marchael> it won't working for All:read,write
222 2011-08-07T15:32:32 <Marchael> if I just pass it wothout tokenization
223 2011-08-07T15:32:41 <Marchael> s/wothout/without/
224 2011-08-07T15:33:53 <ThomasWaldmann> https://bitbucket.org/marchael/moin-2.0/issue/13/acl-search#comment-591804 what exactly is unclear there?
225 2011-08-07T15:34:47 <Marchael> ah
226 2011-08-07T15:35:01 <ThomasWaldmann> we are not required to use exactly the same syntax for acl queries as for acls
227 2011-08-07T15:41:00 <CIA-114> Michael Mayorov <marchael@kb.csu.ru> * 6c407c842f4c r410 moin-2.0/MoinMoin/search/_tests/test_indexing.py: Updated query string for acl tokenizer
228 2011-08-07T15:54:11 <ThomasWaldmann> from http://packages.python.org/Whoosh/highlight.html#pinpointfragmenter-limitations :
229 2011-08-07T15:54:20 <ThomasWaldmann> "(This can be embarassing when the word fragments form dirty words!)"
230 2011-08-07T15:54:24 <ThomasWaldmann> oh noes! :D
231 2011-08-07T15:56:40 <Marchael> lol
232 2011-08-07T16:06:27 <Marchael> ThomasWaldmann: metadata coming from web UI doesn't converting to unicode?
233 2011-08-07T16:08:22 <Marchael> then may be analyzer should just use unicode on that?
234 2011-08-07T16:09:07 <Marchael> because I'm trying add "acl": "JoeDoe:read,write" from editor and recieve traceback from tokenizer
235 2011-08-07T16:09:22 <Marchael> assert isinstance(value, unicode) # so you'll notice if it blows up
236 2011-08-07T16:14:27 <ThomasWaldmann> stuff coming from url or forms should be unicode
237 2011-08-07T16:14:51 <ThomasWaldmann> use u"..."
238 2011-08-07T16:15:19 *** m4k3r
239 2011-08-07T16:15:20 <ThomasWaldmann> (for the value)
240 2011-08-07T16:15:49 <Marchael> JSON Invalid
241 2011-08-07T16:16:11 <Marchael> when I tried to save this
242 2011-08-07T16:16:34 <ThomasWaldmann> if you are editing metadata with the UI, it is all unicode
243 2011-08-07T16:16:52 <ThomasWaldmann> but if you use tests, you need to use u"JoeDoe:..."
244 2011-08-07T16:17:13 *** m4k3r
245 2011-08-07T16:37:24 <Marchael> ThomasWaldmann: that's happening when I use query like "acl:JoeDoe:read " http://paste.pocoo.org/show/454178/
246 2011-08-07T16:40:22 <Marchael> ah, that <itertools.chain object at 0x4082210>
247 2011-08-07T16:40:38 <Marchael> I mean value is itertools.chain object
248 2011-08-07T16:41:21 <Marchael> ThomasWaldmann: ignore it
249 2011-08-07T17:00:22 <Marchael> ThomasWaldmann: yeah, I make it work
250 2011-08-07T17:00:55 <Marchael> and what's happening with initial rebuild?
251 2011-08-07T17:01:06 <Marchael> I don't see items after rebuild
252 2011-08-07T17:01:14 <Marchael> (I don't mean offline script)
253 2011-08-07T17:03:33 *** pkumar
254 2011-08-07T17:10:29 <ThomasWaldmann> https://bitbucket.org/site/master/issue/2927/priority-values-seem-to-mix-up-things
255 2011-08-07T17:10:58 <Marchael> ThomasWaldmann: http://codereview.appspot.com/4838051
256 2011-08-07T17:11:07 <ThomasWaldmann> there is no rebuild at app startup any more
257 2011-08-07T17:11:15 <Marchael> hm, ok
258 2011-08-07T17:11:25 <ThomasWaldmann> but, when it loads items from xml and saves them to backend, of course they get indexed
259 2011-08-07T17:13:37 <ThomasWaldmann> nope
260 2011-08-07T17:13:46 <ThomasWaldmann> why are you creating a mess?
261 2011-08-07T17:14:08 <Marchael> ?
262 2011-08-07T17:15:10 *** m4k3r_
263 2011-08-07T17:17:09 *** m4k3r
264 2011-08-07T17:17:26 <ThomasWaldmann> Marchael: just read what you gave me for review and think about it
265 2011-08-07T17:18:43 <Marchael> I gave you working code and I don't understand what makes you dislike it
266 2011-08-07T17:20:22 <ThomasWaldmann> the docstring and names are not consistent, the names and concepts are not consistent, you use a multifilter to call the same code for both cases, ...
267 2011-08-07T17:22:31 <ThomasWaldmann> if value.mode in "query": and stuff like that also doesn't impress
268 2011-08-07T17:37:38 *** m4k3r_
269 2011-08-07T17:39:39 *** m4k3r
270 2011-08-07T18:24:50 *** Marchael
271 2011-08-07T18:25:09 *** Marchael
272 2011-08-07T18:46:53 *** m4k3r_
273 2011-08-07T18:48:56 *** m4k3r
274 2011-08-07T18:56:23 *** m4k3r_
275 2011-08-07T18:57:48 *** raignarok
276 2011-08-07T18:58:11 *** m4k3r
277 2011-08-07T19:05:44 <ThomasWaldmann> yay, interwiki has_item :)
278 2011-08-07T19:05:59 <ThomasWaldmann> at least for farms sharing same index
279 2011-08-07T19:13:21 <Marchael> ThomasWaldmann: yes, you was right. please review http://codereview.appspot.com/4838051/ ps 4
280 2011-08-07T19:13:45 <Marchael> MultiFilter doesn't need there
281 2011-08-07T19:25:19 <ThomasWaldmann> review done
282 2011-08-07T19:27:18 *** m4k3r
283 2011-08-07T19:28:20 <dreimark> sinha: working on the mimetype?
284 2011-08-07T19:33:22 <Marchael> ThomasWaldmann: ps 5
285 2011-08-07T19:40:54 <ThomasWaldmann> hmm, we have a "cache consistency"-like issue with the whoosh index
286 2011-08-07T19:43:38 <Marchael> mm
287 2011-08-07T19:44:45 <ThomasWaldmann> Marchael: ok, fine now
288 2011-08-07T19:45:15 <ThomasWaldmann> we'll maybe need some sort of "refresh" in case the backend contents change behind our back
289 2011-08-07T19:45:30 <ThomasWaldmann> like for fileserver backend
290 2011-08-07T19:47:21 <Marchael> brb
291 2011-08-07T19:54:52 <CIA-114> Michael Mayorov <marchael@kb.csu.ru> * 7d79c684123d r411 moin-2.0/MoinMoin/search/analyzers.py: Fixed search in acl, fixed docstings Fixes #13.
292 2011-08-07T19:59:25 <Marchael> https://bitbucket.org/marchael/moin-2.0/issue/26/wildcard-search-in-item-names-does-not
293 2011-08-07T19:59:25 <Marchael> > hmm, instead of just searching in name and content, we could maybe also search in name_exact, so the user does not need to think about this issue.
294 2011-08-07T19:59:25 <Marchael> ThomasWaldmann: that won't work
295 2011-08-07T19:59:45 <Marchael> because name_exact is ID
296 2011-08-07T20:01:07 <ThomasWaldmann> why doesn't it work?
297 2011-08-07T20:01:33 <Marchael> (23:55:44) Marchael: because name_exact is ID
298 2011-08-07T20:01:39 <ThomasWaldmann> so?
299 2011-08-07T20:02:24 <ThomasWaldmann> maybe just try entering a query like name_exact:foo*
300 2011-08-07T20:02:25 <Marchael> hm, as you know we have analyzer for parsing names, but ID hasn't
301 2011-08-07T20:02:29 <ThomasWaldmann> it'll find foobar
302 2011-08-07T20:02:49 <ThomasWaldmann> you don't need it, even the contrary is true
303 2011-08-07T20:02:50 <Marchael> then user should enter * each time for any query
304 2011-08-07T20:03:06 <ThomasWaldmann> hm?
305 2011-08-07T20:03:25 <Marchael> except when he knows search result before
306 2011-08-07T20:03:31 <ThomasWaldmann> no, we need both
307 2011-08-07T20:03:58 <ThomasWaldmann> the name analyzer is useful, just not for wildcard search
308 2011-08-07T20:04:29 <ThomasWaldmann> the name_exact searching is useful for wildcard, but not for inexact searches other than wildcard
309 2011-08-07T20:04:58 <Marchael> name_exact is dirty hack to avoid UUID issue, I don't sure what using dirty hack for another dirty hack for avoiding bugs is good
310 2011-08-07T20:05:21 <ThomasWaldmann> no
311 2011-08-07T20:05:29 <ThomasWaldmann> we need name_exact in any case
312 2011-08-07T20:05:48 <ThomasWaldmann> because you need to be able to lookup precisely a give item by name
313 2011-08-07T20:06:02 <Marchael> yes, but I can do it using uuids
314 2011-08-07T20:06:07 <ThomasWaldmann> without having something tokenizing that name
315 2011-08-07T20:06:31 <ThomasWaldmann> you can't do a name lookup using uuid
316 2011-08-07T20:06:47 <ThomasWaldmann> e.g. if you have the name and you want the uuid
317 2011-08-07T20:07:20 <Marchael> why we need do this name lookup? If we know item uuid then we can know it name.
318 2011-08-07T20:08:05 <Marchael> *do we need do
319 2011-08-07T20:10:15 <dreimark> Marchael: the user don't know the uuid
320 2011-08-07T20:10:37 <Marchael> in which cases we know name and don't know uuid?
321 2011-08-07T20:10:47 <Marchael> *do we know
322 2011-08-07T20:10:47 <ThomasWaldmann> wiki urls have the name, not the uuid
323 2011-08-07T20:10:47 <dreimark> in the case when you search something
324 2011-08-07T20:11:05 <ThomasWaldmann> wiki markup also has the name, not the uuid
325 2011-08-07T20:11:47 <dreimark> the uuid is always only internal of course it is in meta
326 2011-08-07T20:12:01 <dreimark> but you address the item by its name
327 2011-08-07T20:14:39 <Marchael> ThomasWaldmann: yes, wiki urls containing names, but how it makes sense on indexing and searching?
328 2011-08-07T20:15:07 <Marchael> when we operating Item we have all metadata
329 2011-08-07T20:15:10 <dreimark> a name can have more than one uuid
330 2011-08-07T20:15:37 <Marchael> dreimark: hm, how?
331 2011-08-07T20:16:41 <dreimark> in history, not in always all revs
332 2011-08-07T20:17:08 <dreimark> if you rename an item and create a new one with its old name
333 2011-08-07T20:18:29 <dreimark> then you have a new uuid with its name and another for the history of the older revisions
334 2011-08-07T20:19:08 <Marchael> ah, yes.
335 2011-08-07T20:20:17 <ThomasWaldmann> Marchael: because we will use the index to lookup the uuid for some name
336 2011-08-07T20:20:24 <dreimark> i have not read all backlog, usually you don't enter the uuid to the querystring but its name
337 2011-08-07T20:20:38 <ThomasWaldmann> that requires that we can look for that exact name
338 2011-08-07T20:22:21 *** raignarok
339 2011-08-07T20:22:55 <Marchael> ThomasWaldmann: so, you suggest me use name_exact instead name for searching?
340 2011-08-07T20:27:18 <ThomasWaldmann> no
341 2011-08-07T20:27:48 <Marchael> then how we can mix name_exact and name?
342 2011-08-07T20:28:07 <ThomasWaldmann> https://bitbucket.org/marchael/moin-2.0/issue/26/wildcard-search-in-item-names-does-not#comment-591858
343 2011-08-07T20:28:29 <ThomasWaldmann> you "mix" it the same way as for name and content
344 2011-08-07T20:28:36 <ThomasWaldmann> which you already do
345 2011-08-07T20:30:06 * ThomasWaldmann found new index issues after replacing has_item by an indexed one
346 2011-08-07T20:31:09 <Marchael> ThomasWaldmann: there is for mparser = MultifieldParser(["name_exact", "name", "content"], schema=latest_index.schema) http://paste.pocoo.org/show/454276/
347 2011-08-07T20:31:29 <Marchael> I just try "Home*"
348 2011-08-07T20:32:52 <ThomasWaldmann> you had that already
349 2011-08-07T20:33:12 <ThomasWaldmann> it is if you give unicode key names
350 2011-08-07T20:33:48 *** m4k3r
351 2011-08-07T20:34:33 <ThomasWaldmann> python 2.6 just wants str as keys
352 2011-08-07T20:43:19 *** pkumar
353 2011-08-07T20:46:39 <Marchael> hm, and how I should convert it to str?
354 2011-08-07T20:49:44 <ThomasWaldmann> did you already find what "it" is?
355 2011-08-07T20:58:41 <Marchael> no
356 2011-08-07T20:58:58 <Marchael> sorry, need wake up early, gn/bbl
357 2011-08-07T20:59:12 <ThomasWaldmann> gn
358 2011-08-07T20:59:12 *** Marchael
359 2011-08-07T21:23:12 <ThomasWaldmann> shit, using indexed has_item creates ~50 test fails
360 2011-08-07T21:51:40 *** greg_f
361 2011-08-07T21:52:38 *** greg_f
362 2011-08-07T22:32:51 *** raignarok
363 2011-08-07T22:44:42 <sinha> dreimark: i have added that mimetype thing but its not working well for all the items, event if i open the link in different window it sends the default contenttype, this is the changes i made http://codereview.appspot.com/4810068
364 2011-08-07T22:48:24 * dreimark looks
365 2011-08-07T22:53:14 *** m4k3r
366 2011-08-07T22:56:07 <dreimark> sinha: don't forget to fix setup.py
367 2011-08-07T22:56:25 <sinha> i did and commited it
368 2011-08-07T22:56:38 <dreimark> ah ok, my clone is old
369 2011-08-07T22:56:58 <sinha> i didnt push it yet, will push it with this
370 2011-08-07T23:03:15 <dreimark> sinha: you need to define the mimetype parameter also for frontend route similar to rev
371 2011-08-07T23:03:31 <dreimark> otherwiser somthing like this http://localhost:8080/+download/0/mimetype/Example/100_6379.JPG
372 2011-08-07T23:03:44 <dreimark> gives a traceback
373 2011-08-07T23:05:33 <sinha> dreimark: i am passing it by the query, such as +download/item/mimetype=application/x-moin, this should work, isnt it ?
374 2011-08-07T23:05:50 <sinha> +download/item?mimetype...
375 2011-08-07T23:06:37 <dreimark> looking
376 2011-08-07T23:08:21 <dreimark> i get always TypeError: do_get() got an unexpected keyword argument 'mimetype'
377 2011-08-07T23:09:12 <dreimark> i guess u missed it on one place
378 2011-08-07T23:10:10 <sinha> dreimark: yes at NonExistent item class
379 2011-08-07T23:11:07 <sinha> but if a item exists it shouldnt give any error, right ? infact for some item like application/x-tar i am getting mimetype as the passed one, but for others i am not getting what desired
380 2011-08-07T23:11:29 <dreimark> hmm
381 2011-08-07T23:11:46 <dreimark> you can try to add some logging info
382 2011-08-07T23:11:54 <dreimark> it needs to send the mimetype
383 2011-08-07T23:12:45 <dreimark> sinha: if the item exists, yes but it looks my url is wrong
384 2011-08-07T23:12:51 <dreimark> The requested URL was not found on the server.
385 2011-08-07T23:14:20 <sinha> dreimark: http://localhost:8080/+download/<itemname>?mimetype=application%2Fx-moin
386 2011-08-07T23:14:27 <sinha> try the above one
387 2011-08-07T23:15:19 <dreimark> this did not set the mimetype parameter
388 2011-08-07T23:16:05 <dreimark> if I call something from a sub page
389 2011-08-07T23:18:52 <dreimark> hmm, for a text item it works
390 2011-08-07T23:19:34 <sinha> yes for some its working, but for majority it isnt
391 2011-08-07T23:19:50 <dreimark> sinha: i removed the extension and then it works for an image too
392 2011-08-07T23:20:06 <sinha> which extension ?
393 2011-08-07T23:20:12 <sinha> that x-moin ?
394 2011-08-07T23:20:15 <dreimark> i have uloaded test.jpg
395 2011-08-07T23:20:25 <dreimark> and renamed the item then to test
396 2011-08-07T23:20:32 <dreimark> test works and test.jpg not
397 2011-08-07T23:20:38 <sinha> yes gotcha
398 2011-08-07T23:20:47 <sinha> i have the same thing here
399 2011-08-07T23:21:04 <dreimark> looks like browser to clever
400 2011-08-07T23:21:27 <sinha> yes, it needs to open it in default application
401 2011-08-07T23:21:49 <sinha> any other way to override this ?
402 2011-08-07T23:21:58 <dreimark> but why does the browser ignore the mimetype sended
403 2011-08-07T23:29:40 <sinha> dreimark: chrome doesnt ask for anything, it simply saves the file
404 2011-08-07T23:30:09 <sinha> and also how many cases are there on a wiki that a item name use to have these file extension ?
405 2011-08-07T23:30:14 <dreimark> yes, i recogn. chrome tells something like do you want the download
406 2011-08-07T23:30:43 <dreimark> extensions happens with the upload feature
407 2011-08-07T23:31:40 <dreimark> because of the files.
408 2011-08-07T23:35:10 <dreimark> sinha: is js working for you in chromium?
409 2011-08-07T23:35:39 <sinha> i have google chrome and yes it is working in it
410 2011-08-07T23:35:48 <sinha> chromium is similar to chrome ? isnt it ?
411 2011-08-07T23:37:20 <dreimark> ubunutu 10.10 for me here it is chromium_browser
412 2011-08-07T23:37:31 <dreimark> I guess that is something on my box
413 2011-08-07T23:40:04 <sinha> so what should be the action for now ? the only reason we are doing this, such that user wont get annoyed by the "Save File" popup
414 2011-08-07T23:40:47 <dreimark> i try currently different mimetypes
415 2011-08-07T23:42:30 <dreimark> http://localhost:8080/+download/test.jpg?mimetype=text%2Fplain
416 2011-08-07T23:42:48 <dreimark> sinha: what happens for you
417 2011-08-07T23:43:58 <dreimark> for me this works - it did not recog. the jpegg
418 2011-08-07T23:44:42 <sinha> yes for me also it output as plain text
419 2011-08-07T23:45:21 <dreimark> if i set to save this mimetype it seems to work. it stores the image
420 2011-08-07T23:47:17 <dreimark> application%2Fplain works too
421 2011-08-07T23:47:25 <dreimark> moment
422 2011-08-07T23:50:03 <dreimark> sinha: find out if there is an other than text/plain been accepted
423 2011-08-07T23:50:35 <sinha> okay
424 2011-08-07T23:50:57 <dreimark> I am looking too
425 2011-08-07T23:53:02 <sinha> dreimark: what i am seeing is, if we send the standard(known) mimetypes it works fine, but if we send something new, it uses the defaults one
426 2011-08-07T23:54:42 <dreimark> rephrase please
427 2011-08-07T23:56:03 <dreimark> i see the following. without extensions it works, with not. and if the browser is to clever it ignores other sended mimetypes than text/plain
428 2011-08-07T23:56:52 <sinha> i mean i have set mimetype as application/pdf, image/png and other known mimetype, it outputs fine with them, but if i set something like application/moin (unknown) it simply ignores it and show the default one
429 2011-08-07T23:57:44 <dreimark> yes seeing this too. but if you set text/plain instead of application/moin i guess it works
430 2011-08-07T23:57:46 <sinha> i have tried the above with item name *.png
