2008-06-10T00:26:44 <TheSheep> PawelPacana: lets just continue here
2008-06-10T00:28:01 <PawelPacana> ok, we stopped on choosing method to safely store item names on various filesystems
2008-06-10T00:28:28 <PawelPacana> (as long as wroking copy of repo lives on fs)
2008-06-10T00:29:43 <PawelPacana> you mentioned urlib.quote
2008-06-10T00:31:25 * PawelPacana is searching for posssible alternatives
2008-06-10T00:36:00 <TheSheep> I think that cgi module has some quoting mechanism too
2008-06-10T00:36:09 <TheSheep> but it might use the same as urllib.quote
2008-06-10T00:36:23 <TheSheep> we could use moin's quoting from wikiutil, I think
2008-06-10T00:37:05 <PawelPacana> just found that, http://moinmo.in/QuotingWikiNames
2008-06-10T00:39:12 <PawelPacana> ok, lets stick to internal quoting
2008-06-10T00:40:03 <TheSheep> ouch
2008-06-10T00:40:27 <PawelPacana> ?
2008-06-10T00:40:53 <TheSheep> maybe we should check the resulting fiel name length and trim it, appending a suffix for disambiguation?
2008-06-10T00:42:31 <PawelPacana> not following you
2008-06-10T00:43:15 <TheSheep> s/fiel/file/
2008-06-10T00:43:40 <TheSheep> windows has a limit on the length of filenames
2008-06-10T00:45:44 <PawelPacana> how does it work now in fs backend?
2008-06-10T00:47:58 <TheSheep> I think there is just a limit on the length of the page title
2008-06-10T00:48:03 <TheSheep> let me check
2008-06-10T00:48:08 <PawelPacana> to totally obfuscate working copy and provide some slowdown we could use sha1 hashes from names ;)
2008-06-10T00:48:38 <TheSheep> better DSA keys
2008-06-10T00:48:39 <TheSheep> ;)
2008-06-10T00:48:46 <ThomasWaldmann> :P
2008-06-10T00:53:37 <TheSheep> PawelPacana: looks like it doesn't do anything with file length
2008-06-10T00:54:00 <PawelPacana> got "Nie mogę zmienić nazwy strony ze względu na błąd systemu plików: [Errno 36] File name too long." when entering veery long name
2008-06-10T00:54:01 <TheSheep> PawelPacana: it's MoinMoin.wikiutil.QuoteWikinameFS()
2008-06-10T00:54:52 <TheSheep> sorry, quoteWikinameFS
2008-06-10T00:56:22 <TheSheep> we could do something like this: if the quoted filename is longer than, say, 240 characters, truncate it to 220 characters and append 20 characters of a checksum of the complete name
2008-06-10T00:56:46 <TheSheep> note that we need to store the original page name somewhere
2008-06-10T00:57:05 <TheSheep> and that it could have changed, so we need to store it in the revision
2008-06-10T00:59:47 <PawelPacana> antoher line in message, huh, this doesnt seem clean
2008-06-10T01:00:47 * PawelPacana wonders how does mercurial handle such cases
2008-06-10T01:02:40 <TheSheep> we could make the commit message look like e-mail/http headers
2008-06-10T01:02:55 <TheSheep> it can be easily parsed with ConfigParser
2008-06-10T01:03:18 <TheSheep> the first line is what appears in shortlog
2008-06-10T01:03:36 <TheSheep> so it should probably be the comment
2008-06-10T01:07:35 <PawelPacana> much cleaner, not relying on order
2008-06-10T01:07:54 <PawelPacana> bbim
2008-06-10T01:19:58 <TheSheep> I wonder if whole revision metainfo could be stored in this way
2008-06-10T01:31:22 <PawelPacana> back
2008-06-10T01:32:04 <PawelPacana> minute has stretched
2008-06-10T01:33:25 <PawelPacana> TheSheep: gtg and finish tomorrows presentation
2008-06-10T01:34:19 <PawelPacana> i`ll drop our previous an current conversation extracts on wiki
2008-06-10T01:37:36 <PawelPacana> just to finish for today, i thought earlier about flat repo structure. if all meta go into hg and only Item content is left on working copy, there is no need for nesting
2008-06-10T01:38:36 <PawelPacana> when some pages are nested we could map this nesting in Item's name on working copy, just like pages are presented now
2008-06-10T01:39:14 <PawelPacana> so any Item moves will be renames
2008-06-10T01:39:26 <PawelPacana> what do you think of it?
2008-06-10T01:40:12 <TheSheep> yeah
2008-06-10T01:40:32 <TheSheep> but keeping too much info in the commit messages may make it hard to edit pages with external tools
2008-06-10T01:47:09 <PawelPacana> that is a drawback of lower magnitude, however 'no longer easy ctags iface' will be sad enough :|
2008-06-10T01:47:25 <PawelPacana> ok, enough for today
2008-06-10T01:47:31 <PawelPacana> good night
2008-06-10T07:10:00 <gayane> хи
2008-06-10T07:10:02 <gayane> hi
2008-06-10T07:10:15 <gayane> can anyone help me with mail support in moinmoin
2008-06-10T08:24:04 <dreimark> moin
2008-06-10T08:24:24 <dreimark> ThomasWaldmann: currently all attachfile test do have request as param
2008-06-10T08:44:47 <CIA-51> Reimar Bauer <rb.proj AT googlemail DOT com> default * 3687:207ae45bab18 1.7/MoinMoin/action/_tests/test_attachfile.py:
2008-06-10T08:44:47 <CIA-51> test_attachfile: added a test for add_attachment using a file object / removed
2008-06-10T08:44:47 <CIA-51> the duplicated str using test
2008-06-10T08:45:09 <dreimark> ThomasWaldmann: I do refactor it to a class definition
2008-06-10T09:10:23 <CIA-51> Reimar Bauer <rb.proj AT googlemail DOT com> default * 3688:57d85b82bc3e 1.7/MoinMoin/action/_tests/test_attachfile.py: test_attachfile: refactored to test class
2008-06-10T11:01:07 <xorAxAx> hmm, my student is not here
2008-06-10T11:08:59 <ThomasWaldmann> his irc client was here until 10:03
2008-06-10T11:09:13 <xorAxAx> yes
2008-06-10T11:09:33 <xorAxAx> its 18:xx local time for him
2008-06-10T11:35:18 <zenhase> moin
2008-06-10T11:35:26 <ThomasWaldmann> hi zenhase
2008-06-10T11:40:26 <ThomasWaldmann> xorAxAx: the last hunk in his last changeset looks like he is using tabs or wrong indentation
2008-06-10T11:54:42 <ThomasWaldmann> 2008-06-10 11:54:37,138 WARNING MoinMoin.search.builtin:1072 can't index: can't acquire lock
2008-06-10T11:54:58 <ThomasWaldmann> looks like we need a different method to rebuild the index for busy sites
2008-06-10T12:30:48 * dreimark tries to fix AttachmentUnzipCanCauseUnicodeDecodeError
2008-06-10T12:34:39 <ThomasWaldmann> but please fix it at the right place :)
2008-06-10T12:35:20 <ThomasWaldmann> (that is not where the crash happens that's shown in the traceback)
2008-06-10T12:35:30 <dreimark> I know
2008-06-10T12:36:19 <dreimark> ThomasWaldmann: it is not only the zip file you can get with add_atachment too
2008-06-10T12:40:11 <ThomasWaldmann> the current code does not use add_attachment
2008-06-10T12:40:31 <ThomasWaldmann> (but it better should :)
2008-06-10T12:40:39 <ThomasWaldmann> outfile.write(zf.read(origname))
2008-06-10T12:40:46 <ThomasWaldmann> that sucks for big stuff
2008-06-10T12:42:31 <ThomasWaldmann> # it's not allowed to zip a zip file so it is dropped
2008-06-10T12:42:34 <ThomasWaldmann> if zipfile.is_zipfile(new_file):
2008-06-10T12:42:35 <ThomasWaldmann> why is that?
2008-06-10T12:44:19 <waldi> ThomasWaldmann: shutil.*?
2008-06-10T12:44:27 <dreimark> I wonder on that line too, it should be above. it should not first unpack and then remove it
2008-06-10T12:44:43 <dreimark> we do not want to have zip bombs
2008-06-10T12:44:59 * waldi .o0( 23.zip ... )
2008-06-10T12:45:16 <ThomasWaldmann> waldi: add_attachment already handles files in a sane way
2008-06-10T12:45:23 <waldi> ThomasWaldmann: okay
2008-06-10T12:45:51 <ThomasWaldmann> dreimark: a zip bomb is handled by other code
2008-06-10T12:47:28 <dreimark> I know but there is no reason to have zip files in one zip archive
2008-06-10T12:47:54 <ThomasWaldmann> you can't know if someone has a reason for it
2008-06-10T12:48:59 <ThomasWaldmann> offerering some download files as zip seems even rather reasonable
2008-06-10T12:49:21 <dreimark> no
2008-06-10T12:49:32 <ThomasWaldmann> sure :)
2008-06-10T12:49:46 <dreimark> if you do this you can get 42.zip unpacked manual
2008-06-10T12:50:17 <ThomasWaldmann> imagine someone wants to offer 50 zip files for download, but doesn't want to upload them all one-by-one
2008-06-10T12:51:01 <ThomasWaldmann> it won't unpack a zip bomb because it has too many files inside, or too big files or both
2008-06-10T12:51:20 <ThomasWaldmann> the uncompressed zip bomb itself is no problem as it is usually rather small
2008-06-10T12:51:34 <dreimark> it does unpack if you do it slow enough until you reach the limit
2008-06-10T12:51:58 <ThomasWaldmann> ?
2008-06-10T12:52:29 <dreimark> the first layer unpacks 50 zip files
2008-06-10T12:52:43 <ThomasWaldmann> that's allowed
2008-06-10T12:53:20 <dreimark> if they have stacked zip files you get with the next request the file limit exceeded
2008-06-10T12:53:33 <dreimark> and you have no real file
2008-06-10T12:53:57 <dreimark> the next request on each zip file will unpack 50 zip files again
2008-06-10T12:54:59 <ThomasWaldmann> if you can do it that (cumbersome) way, it just means that our limits are not completely doing what we want
2008-06-10T12:55:14 <ThomasWaldmann> it doesn't mean that zip files within a zip are a bad thing
2008-06-10T12:56:01 <ThomasWaldmann> but if I look at _do_unzip, the limits are per page, not per zip file
2008-06-10T12:56:31 <ThomasWaldmann> so there is no way beyond it
2008-06-10T12:56:31 <dreimark> yes, you can get for each page e.g. 200MB
2008-06-10T12:57:05 <dreimark> and we have this kind of quota only for zip files
2008-06-10T12:57:58 <ThomasWaldmann> so again,what's the problem with unpacking a zip member of a zip file?
2008-06-10T12:58:45 <dreimark> I do see only the problem that one can do exceed the 200MB easily.
2008-06-10T12:58:47 <ThomasWaldmann> (it seems to be rarely done, otherwise we would have a open bug about missing zip files maybe :)
2008-06-10T12:59:04 <ThomasWaldmann> dreimark: how?
2008-06-10T13:00:09 <dreimark> one can upload a small file and can umnapck it
2008-06-10T13:00:16 <dreimark> unpack
2008-06-10T13:00:31 <ThomasWaldmann> that won't get you beyond the limit
2008-06-10T13:01:18 <dreimark> if you don't get the size limit you will catch the no of files limit
2008-06-10T13:01:45 <ThomasWaldmann> maybe we can have another approach for that:
2008-06-10T13:02:07 <dreimark> yeah
2008-06-10T13:02:08 <ThomasWaldmann> remove the code denying extraction of a zip member file
2008-06-10T13:02:17 <ThomasWaldmann> and then try to exploit it
2008-06-10T13:02:33 <ThomasWaldmann> exploiting means going beyond the limits in your config
2008-06-10T13:02:43 <ThomasWaldmann> and then tell me how you did it
2008-06-10T13:02:54 <dreimark> hmm, I think I have done this sometime ago
2008-06-10T13:03:17 <ThomasWaldmann> I didn't try, but looking at the code you can't go beyond the limit.
2008-06-10T13:03:54 <ThomasWaldmann> (and this is completely unrelated to whether the member file is a zip or not)
2008-06-10T13:08:48 * ThomasWaldmann hopes dennda does a good job and we can get rid of the whole AttachFile stuff soon :)
2008-06-10T13:13:09 <dreimark> yeah you can't exceed the limit you can reach it only with stupid files
2008-06-10T13:13:43 <CIA-51> Bastian Blank <bblank@thinkmo.de> default * 3649:49499eedcde9 1.8-dom-bblank/MoinMoin/converter2/ (_tests/test_html_out.py html_out.py):
2008-06-10T13:13:43 <CIA-51> Support heading outline-level.
2008-06-10T13:13:43 <CIA-51> * MoinMoin/converter2/html_out.py
2008-06-10T13:13:43 <CIA-51> Check the outline-level attribute and crop to [1-6].
2008-06-10T13:13:43 <CIA-51> * MoinMoin/converter2/_tests/test_html_out.py:
2008-06-10T13:13:44 <dreimark> if I think those zip files could be packages I think I do agree one qants them in one file
2008-06-10T13:13:45 <CIA-51> Add tests for outline-level.
2008-06-10T13:13:51 <dreimark> ThomasWaldmann:
2008-06-10T13:15:15 <dreimark> if removed the docs needs to be changed for "zip files in zip file will be ignored"
2008-06-10T13:15:41 <dreimark> and one i18n string could be killed too
2008-06-10T13:16:03 <dreimark> hmm, others have to be changed then too
2008-06-10T13:16:08 <ThomasWaldmann> killing i18n strings is no problem, just changing and adding them is
2008-06-10T13:16:22 <dreimark> HelpOnActions/AttachFile
2008-06-10T13:16:55 <dreimark> unfortunately zip files only is often used
2008-06-10T13:17:03 <dreimark> -> 1.8 ?
2008-06-10T13:17:07 <ThomasWaldmann> and I guess noone will complain that it didn't nuke his zip member ;)
2008-06-10T13:18:06 <dreimark> because they didn't complain or they did have accepted it
2008-06-10T13:18:56 <dreimark> bbl ~2h
2008-06-10T15:01:55 <johill> PawelPacana: was that you updating the hg backend page?
2008-06-10T15:17:37 <johill> ThomasWaldmann: it does look like the new account notification happens twice if the browser has a non-default language (i.e. not en)
2008-06-10T15:25:25 <ThomasWaldmann> that's strange :) it shouldn't care for the browser language at all. is that tip?
2008-06-10T15:27:15 <johill> pretty much, sec
2008-06-10T15:28:41 <johill> 1.7.0rc2
2008-06-10T15:30:32 <ThomasWaldmann> so the 2 mails you get are in which language?
2008-06-10T15:31:19 <ThomasWaldmann> http://hg.moinmo.in/moin/1.7/log/57d85b82bc3e/MoinMoin/events/emailnotify.py see the 2 changes 2 weeks ago
2008-06-10T15:31:45 <johill> both in english
2008-06-10T15:33:04 <ThomasWaldmann> i don't see how the code should send 2 emails for the same userid
2008-06-10T15:33:27 <ThomasWaldmann> but you maybe have 2 userids with same username?
2008-06-10T15:33:33 <johill> no, I checked all that
2008-06-10T15:33:39 <johill> only a single account has new user notification enabled
2008-06-10T15:34:36 <johill> and those changes should be in 1.7.0rc2 anyway, no
2008-06-10T15:34:39 <johill> ?
2008-06-10T15:34:52 <ThomasWaldmann> and aside from that: are there 2 different uids for the same email?
2008-06-10T15:35:11 <ThomasWaldmann> (no matter whether they have subscribed to that event)
2008-06-10T15:35:14 <johill> nope
2008-06-10T15:37:09 <ThomasWaldmann> i can try to reproduce it later. i don't see what could go wrong.
2008-06-10T15:41:17 <ThomasWaldmann> (yes, those changes are in rc2)
2008-06-10T15:44:47 <johill> just upgraded to the latest
2008-06-10T15:44:59 <johill> it doesn't _always_ happen which leads me to believe that it happens when the browser language is not en
2008-06-10T15:45:09 <johill> (it happened so far with people whose email address is @gmx etc)
2008-06-10T15:48:07 <ThomasWaldmann> i did get 3 "new user" mails yet, no dupes
2008-06-10T15:49:12 <johill> dunno
2008-06-10T15:56:10 <ThomasWaldmann> 4 :)
2008-06-10T15:56:47 <ThomasWaldmann> johill: do the dupes have same Message-ID?
2008-06-10T15:58:31 <ThomasWaldmann> or is it maybe a problem with the way the mailings are done?
2008-06-10T15:59:14 <ThomasWaldmann> moin's mails have from: wiki_from to: wiki_from and all real destinations are done by bcc
2008-06-10T15:59:21 <johill> let me see
2008-06-10T15:59:32 <johill> well the thing is I don't _always_ get dupes
2008-06-10T15:59:38 <johill> only on new user, and then not on all
2008-06-10T16:00:06 <johill> no, they do not have the same message ID, but the message ID is added by exim apparently
2008-06-10T16:00:37 <ThomasWaldmann> what happens if someone double clicks "create user"? :P
2008-06-10T16:00:55 <johill> the second ought to be refused since it already exists, no?
2008-06-10T16:01:33 <ThomasWaldmann> maybe some race condition
2008-06-10T16:02:31 <johill> but I have a hard time believing that so many people double-click
2008-06-10T16:02:41 <johill> I see that in maybe 1 out of 2 or 3 new users
2008-06-10T16:10:52 <johill> I added a small patch to add a traceback to the email... let's see
2008-06-10T16:16:52 <johill> can't reproduce...
2008-06-10T16:17:00 <johill> and double submit doesn't seem to lead to errors either
2008-06-10T16:18:25 <johill> I'll wait for it to happen again
2008-06-10T18:16:39 <ThomasWaldmann> the docs about the jabber bot are a bit thin...
2008-06-10T18:46:59 <PawelPacana> johill: that was me, i forgot to log in after upgrade :o
2008-06-10T18:56:19 <johill> PawelPacana: I added a few notes
2008-06-10T19:08:31 <ThomasWaldmann> http://www.southpolestrip.com/comic/51/South-Pole-50-Being-different/ < LOL
2008-06-10T20:31:11 <ThomasWaldmann> > AssertionError: (inconsistently failed then succeeded)
2008-06-10T20:32:30 <johill> ?
2008-06-10T20:33:26 <ThomasWaldmann> search query parser tests
2008-06-10T20:33:36 <ThomasWaldmann> (after fixing some of the stuff)
2008-06-10T21:11:17 <dreimark> we need a shorter way to get to the advanced search if you've done one title search already
2008-06-10T21:15:21 <dreimark> in the form the "Go get it!" should be more visible
2008-06-10T21:15:32 <dreimark> e.g. bold
2008-06-10T21:23:01 <CIA-51> Thomas Waldmann <tw AT waldmann-edv DOT de> default * 3689:23a5fefa9553 1.7/MoinMoin/search/_tests/test_search.py: search query parser tests: adapt to current query parser behaviour
2008-06-10T21:28:11 <dreimark> just only an idea http://moinmo.in/ReimarBauer?action=AttachFile&do=view&target=codec_problem.diff (not completed for the msg)
2008-06-10T21:29:16 <dreimark> for this http://moinmo.in/MoinMoinBugs/AttachmentUnzipCanCauseUnicodeDecodeError and http://moinmo.in/MoinMoinBugs/UnzipFilesWithNonAsciiName
2008-06-10T21:29:51 <CIA-51> Florian Krupicka <florian.krupicka@googlemail.com> default * 3661:30ac757e72de 1.8-wsgi-fkrupicka/MoinMoin/ (web/apps.py wsgiapp.py): Added HTTPException catching middleware
2008-06-10T21:29:53 <CIA-51> Florian Krupicka <florian.krupicka@googlemail.com> default * 3662:ddcd459c3255 1.8-wsgi-fkrupicka/MoinMoin/ (web/contexts.py wsgiapp.py): Restructured contexts classes for wrapping contexts in each other
2008-06-10T21:29:54 <CIA-51> Florian Krupicka <florian.krupicka@googlemail.com> default * 3663:75b0b7f32423 1.8-wsgi-fkrupicka/MoinMoin/web/ (contexts.py exceptions.py): Replace makeForbidden with HTTPException variant for WSGI
2008-06-10T21:29:55 <CIA-51> Florian Krupicka <florian.krupicka@googlemail.com> default * 3664:fbbc33a3d40a 1.8-wsgi-fkrupicka/MoinMoin/ (web/contexts.py wsgiapp.py): New isSpiderAgent property on RequestContext(and fixed some copy'n'paste messup)
2008-06-10T21:29:57 <CIA-51> Florian Krupicka <florian.krupicka@googlemail.com> default * 3665:7e59a7989b23 1.8-wsgi-fkrupicka/MoinMoin/ (web/contexts.py wsgiapp.py): Make getText into a context property
2008-06-10T21:30:00 <CIA-51> Florian Krupicka <florian.krupicka@googlemail.com> default * 3666:40501b3e2176 1.8-wsgi-fkrupicka/MoinMoin/ (web/contexts.py wsgiapp.py): Moved some attributes on Request into RenderContext
2008-06-10T21:30:03 <CIA-51> Florian Krupicka <florian.krupicka@googlemail.com> default * 3667:264e5687ef1c 1.8-wsgi-fkrupicka/MoinMoin/web/utils.py: Factoring out more parts of request.run
2008-06-10T21:30:34 <zenhase> sorry if i don't always push right away
2008-06-10T21:31:10 <zenhase> sometimes i restructure commits after i've done them to my local repo
2008-06-10T21:41:51 <dreimark> entered the patch to UnzipFilesWithNonAsciiName too
2008-06-10T21:41:59 * dreimark htg
2008-06-10T21:42:03 <dreimark> gn
2008-06-10T21:42:49 <ThomasWaldmann> using decodeUnknown would be too easy
2008-06-10T22:00:24 <dreimark> decodeUnknownInput does not work because of isinstance(text, unicode)
2008-06-10T22:00:38 <dreimark> everything you enter to the form is by default unicode
2008-06-10T22:01:11 <dreimark> return text will not remove any of the wrong signs
2008-06-10T22:01:18 <dreimark> ThomasWaldmann:
2008-06-10T22:03:50 <dreimark> I would like a solution which replaces those signs
2008-06-10T22:07:04 <dreimark> it is an idea about replacing them similiar to taintfilename
2008-06-10T22:08:28 * dreimark has not slept much the last nights, sorry
2008-06-10T22:10:17 <ThomasWaldmann> then it is already decoded wrong
2008-06-10T22:10:30 <ThomasWaldmann> s/signs/chars/
2008-06-10T22:10:44 <ThomasWaldmann> I'll have a look.
2008-06-10T22:12:50 <ThomasWaldmann> the error msgs of unzip are wrong
2008-06-10T22:33:48 * ThomasWaldmann rewrite unzipping code
MoinMoin: MoinMoinChat/Logs/moin-dev/2008-06-10 (last edited 2008-06-09 22:30:02 by IrcLogImporter)