Description

If you generate Table-of-contents for Headings generated by Include, then the links in the TOC and the anchors of Headings mismatch.

And it seems, if you have Includes in your included pages, then the headings of the included pages of the included pages doesn't appear in the TOC.

Example


TOC:


/IncludedPageWhichIncludesAnotherPage :

Heading with mismatching anchor

text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text.

text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text.

/IncludedPage

Heading which didn't appear in TOC

text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text.

text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text text.


Details

This Wiki.

Workaround

Use full paths to included pages, e.g., on page ParentPage, you'd write [[Include(ParentPage/SubPage)]], not [[Include(/SubPage)]]

Discussion

That Include/TOC code is a mess. If somebody wants it fixed, debug it yourself. :) -- ThomasWaldmann 2005-08-17 10:09:36

Here is a possible fix:

  1. Create the toc while formatting the page, by hooking to formatter.heading and text. Each time heading is opened, an item will be added to the toc. Then all the text will go to both heading and toc item.
  2. The toc will be collected and saved for all pages, in request.toc
  3. TableOfContents macro will put a place holder into the page: <<<TableOfContentsPlaceHolder>>>

  4. Page will be rendered into a buffer, always
  5. After the page is rendered, the table of contents placeholder will be replaced by the content of the toc, and the text sent to the client.
  6. Only the first toc macro will place a toc, next calls will add nothing. Multiple toc on one page does not make sense.

Just for the record, this problem (or a very similar one) has returned with a vengeance in 1.9.0. Even fully specified included pages fail to be linked correctly from the TOC. You can see it for example in the second TOC on MoinMoinQuestions. See Fix for Moin 1.9 below.

text_python output

The fix will create the text_python output bellow. headings can be static items, rendered on compile time, while tocItem will be dynamic call, made when the cache is executed, adding items to a toc object.

Alt 1: adding toc hook in the formatter

Add call to tocItem to formatter.base.heading

I'm not sure it will work, as text or _text is static.

formatter.tocItem(1, 1, "id")
request.write('<h1 id="id">)
formatter.text('heading')
request.write('heading')
formatter.tocItem(0)
request.write('</h1>')

Alt 2: adding toc hook in the parser

Call formatter.tocItem from the parser, just before or after the heading

This seems to be the easiest solution.

formatter.tocItem(1, "id", "heading")
request.write('<h1 id="id">)
request.write('heading')
request.write('</h1>')

Partial Fix

This code fixes the problem with the mis-pointed anchor. Isolating the problem, I noticed that if the included page was a complete Pagename - in this example

It appears that the anchor in the TableOfContents is built based on the included page name and that mismatches when you have leading slash subpages or regular expressions in the Include. Change TableOfContents.py seems to work in my 1.5.3 Wiki.

   1 @@ -114,9 +114,35 @@
   2                  else:
   3                      inc_page_lines = []
   4 
   5 -                inc_page_lines = inc_page_lines + self.IncludeMacro(self.macro, match.group(1), called_by_toc=1)
   6 -
   7                  self.process_lines(inc_page_lines, inc_pagename)
   8 +
   9 +                # get list of pages to include
  10 +                inc_name = wikiutil.AbsPageName(self.macro.request, self.macro.formatter.page.page_name, match.group(1) )
  11 +                pagelist = [inc_name]
  12 +                if inc_name.startswith("^"):
  13 +                    try:
  14 +                        inc_match = re.compile(inc_name)
  15 +                    except re.error:
  16 +                        pass # treat as plain page name
  17 +                    else:
  18 +                        # Get user filtered readable page list
  19 +                        pagelist = self.macro.request.rootpage.getPageList(filter=inc_match.match)
  20 +
  21 +                # sort and limit page list
  22 +                pagelist.sort()
  23 +                sort_dir = tmp.group('sort')
  24 +                if sort_dir == 'descending':
  25 +                    pagelist.reverse()
  26 +                max_items = tmp.group('items')
  27 +                if max_items:
  28 +                    pagelist = pagelist[:int(max_items)]
  29 +
  30 +                for inc_name in pagelist:
  31 +                   if not self.macro.request.user.may.read(inc_name):
  32 +                       continue
  33 +                    self.process_lines( self.IncludeMacro(self.macro, inc_name, called_by_toc=1),
  34 +                                       inc_name)
  35 +
  36              else:
  37                  self.parse_line(line, pagename)

Another partial fix (works for regexes)

-- MaximYanchenko 2008-05-27 09:57:11

The following fix changes the Include macro behavior when it's called called_by_toc parameter. In the current version, it returns raw text (so it can be only wiki text, you can't get headings from other markups). I changed the protocol to return just the information about the headers in this case, i.e. a list of tuples like (level, id, text).

So the TableOfContents macro get the heading information directly from the Include macro.

I failed to get correct heading ids inside the Include macro, so now I just get it directly from the rendered page. I believe MoinMoin gurus can easily fix this.

Good

Bad

All bad things come from my fault to get correct heading ids inside the Include macro without rendering the page (I'm not a MoinMoin guru, I believe there are ways to do this without rendering the page). Namely:

   1 --- moin-1.6.3/MoinMoin/macro/Include.py	2008-02-20 06:46:33.000000000 +0900
   2 +++ pkg/x86_64/lib/python2.4/site-packages/MoinMoin/macro/Include.py	2008-05-27 12:15:42.507439000 +0900
   3 @@ -32,12 +32,15 @@
   4  
   5  _title_re = r"^(?P<heading>\s*(?P<hmarker>=+)\s.*\s(?P=hmarker))$"
   6  
   7 +_heading_re = r'<h(?P<level>\d) id="(?P<id>[^"]+?)">(?P<title>.+?)</h(?P=level)>'
   8 +
   9  def extract_titles(body, title_re):
  10      titles = []
  11      for title, _ in title_re.findall(body):
  12 @@ -169,10 +180,6 @@
  13          ##result.append("*** f=%s t=%s ***" % (from_re, to_re))
  14          ##result.append("*** f=%d t=%d ***" % (from_pos, to_pos))
  15  
  16 -        if called_by_toc:
  17 -            result.append(inc_page.get_raw_body())
  18 -            continue
  19 -
  20          if not hasattr(request, "_Include_backto"):
  21              request._Include_backto = this_page.page_name
  22  
  23 @@ -219,7 +226,7 @@
  24              result.append(strfile.getvalue())
  25          finally:
  26              request.redirect()
  27 -
  28 +            
  29          # decrement or remove include marker
  30          if this_page._macroInclude_pagelist[inc_name] > 1:
  31              this_page._macroInclude_pagelist[inc_name] = \
  32 @@ -237,6 +244,12 @@
  33              ])
  34          # XXX page.link_to is wrong now, it escapes the edit_icon html as it escapes normal text
  35  
  36 +    if called_by_toc:
  37 +        # find all headings in the HTML result and return them as a list of tuples (level, id, text)
  38 +        heading_re=re.compile(_heading_re)
  39 +        match = heading_re.findall( ''.join(result) )
  40 +        return match
  41 +
  42      # return include text
  43      return ''.join(result)
  44  
  45 --- moin-1.6.3/MoinMoin/macro/TableOfContents.py	2008-03-29 06:15:00.000000000 +0900
  46 +++ pkg/x86_64/lib/python2.4/site-packages/MoinMoin/macro/TableOfContents.py	2008-05-27 12:10:51.572445000 +0900
  47 @@ -14,21 +14,7 @@
  48  #Dependencies = ["page"]
  49  Dependencies = ["time"] # works around MoinMoinBugs/TableOfContentsLacksLinks
  50  
  51 -# from macro Include (keep in sync!)
  52 -_arg_heading = r'(?P<heading>,)\s*(|(?P<hquote>[\'"])(?P<htext>.+?)(?P=hquote))'
  53 -_arg_level = r',\s*(?P<level>\d*)'
  54 -_arg_from = r'(,\s*from=(?P<fquote>[\'"])(?P<from>.+?)(?P=fquote))?'
  55 -_arg_to = r'(,\s*to=(?P<tquote>[\'"])(?P<to>.+?)(?P=tquote))?'
  56 -_arg_sort = r'(,\s*sort=(?P<sort>(ascending|descending)))?'
  57 -_arg_items = r'(,\s*items=(?P<items>\d+))?'
  58 -_arg_skipitems = r'(,\s*skipitems=(?P<skipitems>\d+))?'
  59 -_arg_titlesonly = r'(,\s*(?P<titlesonly>titlesonly))?'
  60 -_arg_editlink = r'(,\s*(?P<editlink>editlink))?'
  61 -_args_re_pattern = r'^(?P<name>[^,]+)(%s(%s)?%s%s%s%s%s%s%s)?$' % (
  62 -    _arg_heading, _arg_level, _arg_from, _arg_to, _arg_sort, _arg_items,
  63 -    _arg_skipitems, _arg_titlesonly, _arg_editlink)
  64 -
  65 -# from Include, too, but with extra htext group around header text
  66 +# from Include macro, but with extra htext group around header text
  67  _title_re = r"^(?P<heading>(?P<hmarker>=+)\s(?P<htext>.*)\s(?P=hmarker))$"
  68  
  69  class TableOfContents:
  70 @@ -41,7 +27,6 @@
  71          self._ = self.macro.request.getText
  72  
  73          self.inc_re = re.compile(r"^\<\<Include\((.*)\)\>\>")
  74 -        self.arg_re = re.compile(_args_re_pattern)
  75          self.head_re = re.compile(_title_re) # single lines only
  76          self.pre_re = re.compile(r'\{\{\{.+?\}\}\}', re.S)
  77  
  78 @@ -67,7 +52,7 @@
  79          if self.include_macro is None:
  80              self.include_macro = wikiutil.importPlugin(self.macro.request.cfg,
  81                                                         'macro', "Include")
  82 -        return self.pre_re.sub('', self.include_macro(*args, **kwargs)).split('\n')
  83 +        return self.include_macro(*args, **kwargs)
  84  
  85      def run(self):
  86          _ = self._
  87 @@ -92,31 +77,16 @@
  88              match = self.inc_re.match(line)
  89              if match:
  90                  # this is an <<Include()>> line.
  91 -                # now parse the included page and do the work on it.
  92 -
  93 -                ## get heading and level from Include() line.
  94 -                tmp = self.arg_re.match(match.group(1))
  95 -                if tmp and tmp.group("name"):
  96 -                    inc_pagename = tmp.group("name")
  97 -                else:
  98 -                    # no pagename?  ignore it
  99 -                    continue
 100 -                if tmp.group("heading") and tmp.group("hquote"):
 101 -                    if tmp.group("htext"):
 102 -                        heading = tmp.group("htext")
 103 -                    else:
 104 -                        heading = inc_pagename
 105 -                    if tmp.group("level"):
 106 -                        level = int(tmp.group("level"))
 107 -                    else:
 108 -                        level = 1
 109 -                    inc_page_lines = ["%s %s %s" % ("=" * level, heading, "=" * level)]
 110 -                else:
 111 -                    inc_page_lines = []
 112 -
 113 -                inc_page_lines = inc_page_lines + self.IncludeMacro(self.macro, match.group(1), called_by_toc=1)
 114 -
 115 -                self.process_lines(inc_page_lines, inc_pagename)
 116 +                #
 117 +                # save and restore request._page_headings, otherwise all heading ids
 118 +                # will receive counters due to multiple IncludeMacro call
 119 +                saved_page_headings = self.macro.request._page_headings.copy()
 120 +                # get headings from the included page
 121 +                inc_result = self.IncludeMacro(self.macro, match.group(1), called_by_toc=1)
 122 +                self.macro.request._page_headings = saved_page_headings
 123 +                # add the headings to the TOC
 124 +                for h in inc_result:
 125 +                    self.add_toc_line( int(h[0]), h[1], h[2] )
 126              else:
 127                  self.parse_line(line, pagename)
 128  
 129 @@ -130,8 +100,14 @@
 130          self.titles.setdefault(pntt, 0)
 131          self.titles[pntt] += 1
 132  
 133 -        # Get new indent level
 134 -        newindent = len(match.group('hmarker'))
 135 +        unique_id = ''
 136 +        if self.titles[pntt] > 1:
 137 +            unique_id = '-%d' % (self.titles[pntt],)
 138 +            
 139 +        self.add_toc_line( len(match.group('hmarker')), "head-" + sha.new(pntt.encode(config.charset)).hexdigest() + unique_id, title_text )
 140 +
 141 +
 142 +    def add_toc_line(self, newindent, anchor, title_text):
 143          if newindent > self.maxdepth or newindent < self.mindepth:
 144              return
 145          if not self.indent:
 146 @@ -148,20 +124,15 @@
 147              self.result.append(self.macro.formatter.number_list(1))
 148              self.result.append(self.macro.formatter.listitem(1))
 149  
 150 -        # Add the heading
 151 -        unique_id = ''
 152 -        if self.titles[pntt] > 1:
 153 -            unique_id = '-%d' % (self.titles[pntt],)
 154 -
 155          # close last listitem if same level
 156          if self.indent == newindent:
 157              self.result.append(self.macro.formatter.listitem(0))
 158  
 159 +        # Add the heading
 160          if self.indent >= newindent:
 161              self.result.append(self.macro.formatter.listitem(1))
 162 -        self.result.append(self.macro.formatter.anchorlink(1,
 163 -            "head-" + sha.new(pntt.encode(config.charset)).hexdigest() + unique_id) +
 164 +        self.result.append(self.macro.formatter.anchorlink( 1, anchor ) +
 165                             self.macro.formatter.text(title_text) +
 166                             self.macro.formatter.anchorlink(0))
 167  
 168          # Set new indent level
toc_for_included_pages.patch

Fix for Moin 1.9

RenatoSilva - This bug is not reproducible in Moin 1.8.5 but came back in 1.9.0, as reported by GreyCat. I propose the following patch for 1.9 (looks like someone forgot to update this part in uid refactoring):

   1 --- MoinMoin/macro/TableOfContents.py   2009-12-19 19:49:35 +0000
   2 +++ MoinMoin/macro/TableOfContents.py   2009-12-19 19:49:31 +0000
   3 @@ -197,7 +197,7 @@
   4              continue
   5  
   6          # will be reset by pop_unique_ids below
   7 -        macro.request.include_id = incl_id
   8 +        macro.request.uid_generator.include_id = incl_id
   9  
  10          need_li = lastlvl >= lvl
  11          while lastlvl > lvl:

Plan


CategoryMoinMoinBugFixed

MoinMoin: MoinMoinBugs/TableOfContentsBrokenForIncludedPages (last edited 2010-01-22 00:00:57 by RenatoSilva)