Macro SearchInPagesAndSort

Description

Goal

The aim of the macro SearchInPagesAndSort is to:

• Search for pages that match a given first pattern
• Inside these pages, search for text matching a second given pattern
• All lines found (=hits) are included in place of the macro call, sorted by a key given as a third pattern
• Each hit can be followed by a link to its page, and by a heading (a heading text found on its page, matching an optional given pattern)

I developed this macro to generate a page summarizing action items and deadlines from meeting minutes and project status pages. Its versatility is illustrated in the examples below.

Download & Release Notes

Usage

Macro synopsis

Usage:

•   [[ SearchInPagesAndSort (KEYWORD=VALUE ... ) ]] 

Search for 'searchtext' regex in pages marching 'pages' regex, and sort the found lines (=hits) in this order:

1. substring of the hit matching 'sortkey'; group same matches of 'sortkey' by a header
2. substring of the hit matching 'searchtext'
3. the hit itself

Possible keywords:

  Help           = 0, 1, 2         Displays 1:short or 2:full help in the page.
                                   Default: 0 (i.e. no help).

  Pages          = 'PAGES REGEX'   Pages in which the text is sought. If
                                   empty (default) search in the current page
                                   and defaults 'NoLinks' to 1.
                                   Default: empty (i.e. current page).

  ExcludePages   = 'PAGES REGEX'   Exclude these pages (i.e. remove these pages
                                   from the list collected by 'Pages').
                                   Default: empty (i.e. don't exclude any).

  SearchText     = 'TEXT REGEX'    To search for lines in matching pages.
                                   Mandatory!

  SortKey        = 'TEXT REGEX'    Criterion to sort matching lines (=hits).
                                   Default: empty (i.e. no sorting).

  Heading        = 'TEXT REGEX'    Follow each hit by the text maching Regex,
                                   that preceeds the hit in its source page.
                                   Default: empty (i.e. no headings).

  UnassignedText = 'WIKI TEXT'     Header for hits not matching the sort key.
                                   Default: '[unassigned]'.

  Reverse        = 0 or 1          Reverse-sort the hits.
                                   Default: 0 (i.e. forward sort).

  RawText        = 0 or 1          Do not format found text.
                                   Default: 0 (i.e. formatted).

  Format         = 'STRING'        Explicitely format the output using this
                                   string, which can contain wiki formatting
                                   as well as these tokens:
                                     @KT@ : text matching 'SortKey'
                                     @ST@ : text matching 'SearchText'
                                     @FT@ : line of text
                                     @PN@ : page name
                                     @HT@ : heading text
                                     @SU@ : subtext
                                     @@   : the '@' character
                                     \\n  : newline (of wiki source text).

                                   Each token can contain a regex acting as
                                   a filter for displaying the value, e.g:
                                     @FT:{[123]}@      displays the prio smiley

                                   Multiple groups can be defined, in which
                                   case the text matching them will be
                                   displayed, e.g:
                                     @FT:{[123]}(.*)@  displays text after prio

                                   Default: '' (i.e. auto-formatting).

  HeaderFormat   = 'STRING'        If specified, use this instead of 'Format'
                                   for headers.
                                   Default: '' (i.e. do not display headers).

  FormatSort     = 0 or 1          If 1, sort the output generated by 'Format'
                                   (if 'Reverse' is 1, reverse-sort). If 0,
                                   leave the output sorted by the 'SortKey'
                                   criterion (if specified).
                                   Default: 0 (i.e. no sorting).

  Unique         = 0 or 1          If 1, make formatted output lines unique.
                                   Default: 0 (i.e. no filtering).

  NbSubs         = 0, N, or 'all'  Follow each hit by max N sub lines (i.e.
                                   next lines with greater indent) of source
                                   text following the hit. If N is 'all', take
                                   *all* sub lines. If N is positive, take N
                                   *first* sub lines. If N is negative, take
                                   the |N| *last* sub lines.
                                   Default: 0 (i.e. do *not* include subs).

  MoreSubsText   = 'WIKI TEXT'     If there are more sub lines than 'NbSubs',
                                   follow/preceed the last/first sub lines by
                                   this text.
                                   Default: '...'.

  NoHeader       = 0 or 1          Disable showing the headers as subtitles.
                                   Default: 0 (i.e. show headers).

  NoLinks        = 0 or 1          Disable following each hit by a link to its
                                   page.
                                   Default: 0 (i.e. show links) or 1 if
                                   'Pages' is omitted.

  NoPageText     = 'HTML TEXT'     Text displayed if no page match 'Pages'.
                                   Default: an error message w/ Page regex

  NoText         = 0 or 1          Disables showing the found text.
                                   Default: 0 (i.e. show found text).

Keywords can be also given in upper or lower cases, or abbreviated. Example: SearchText, searchtext, SEARCHTEXT, st, ST, Pages, p, etc.

HOWTO

Installing the macro

As usual, for any new macro:

1. Download the latest code: (See Dowload and Release Notes - above)

2. Install in the Python location for MoinMoin macros (on my Debian system: /usr/lib/python2.x/site-packages/MoinMoin/macro/)

Examples

See /UsageExamples.

Migration from version prior 0.3

The arguments syntax has been improved in v0.3, but is incompatible with previous versions. I chose not to provide backwards compatibility.

• The old macro can be still found in the attachments ( SearchInPagesAndSort.py--old-0.2.4 )

• To help finding old macro calls in your Wiki, the new macro itself can be used to find the old one:

   * Pages in which the macro SearchInPagesAndSort (old and new) appears:
    [[SearchInPagesAndSort(p=".*", st="SearchInPagesAndSort", nt=1)]]

   * Pages in which the macro SearchInPagesAndSort appears, old version:
    [[SearchInPagesAndSort(p=".*", st="SearchInPagesAndSort\([^=]*?\)", rt=1)]] 

• Old arguments can be converted as follows:

Tips

• For help, insert just  [[SearchInPagesAndSort]]  (i.e. without arguments) in your page and click Preview. The macro will insert its usage synopsis.

• When sub-pages are sought unintentionally, it may be hard to understand why the results are unexpected. So be sure to terminate the Pages regex argument by '$' to limit the page matches.

• Modifications in preview mode of pages targetted by the macro will not be seen by the macro until the changes are saved.

Misc

Todos

Things I am considering:

• nothing currently

Author, contact

• PascalBauermeister / Date: 2003-03-17, 2004-06-07, 2004-11-15

• For questions/wishes, or just if you like this macro, contact me: <pascal DOT bauermeister AT gmail DOT com>

FeedBack

Integration with MoinMoinAsPim?

Maybe this macro could be used to integrate MoinMoinAsPim with the addressbook data (or maybe even calendar too) having people freeform info in Moin, but with the updated contact data that depends on our addressbook. For this, we could:

1. export our addressbook to a page named AddressBook in, say, one register per line.

2. Include in each node whose type is person (searching the AddressBook) the contact information.

Did I understood OK the macro use? What do you think about this? Does it looks feasible? -- EduardoMercovich 2005-03-16 13:53:45

---

• See also ../LookupPagesAndSort - based on this macro, but instead of operating on lines (using regular expressions), it operates on pages (using dict lookups for definitions::).

• Perhaps one useful thing to add would be an option to return (just) the count of objects returned. One could then display info such as This page has n subpages (oddly useful at times) -- WalterAprile 2006-02-17 17:00:00

Bug with Non-Ascii Characters?

When searched page has non-ascii characters, this macro doesn't work for me. It crashes by calling „urllib.pyo in quote_plus“ and „urllib.pyo in quote“:

• KeyError
  u'\u017e'
  args = (u'\u017e',)

-- Anonymous 2006-04-20 16:44:00

• For this (I think) I commented out part of line 755, to read:

  •    1                         pageurl = pagename
       2                         # XXX: creeps out on unicode:
       3                         #pageurl = '%s?highlight=%s' % (
       4                         #    pagename,
       5                         #    urllib.quote_plus (re.escape (text)))
    

    • That fixed it for me. (Moin 1.5.3) -- TheAnarcat 2007-02-05 22:11:59

Reverse Sorting Bug

• The reverse sorting bug can be corrected by changing line 631 from

    if not opt_notext and not opt_reverse: all_matches.sort ()

to

    if not opt_notext: all_matches.sort ()

• because to reverse sort, you must sort first.

  • Answer: I think I have fixed this issue in the latest version, which is maintained there: http://ten.homelinux.net/productivity/MoinMoinExtensions/macro/SearchInPagesAndSort -- PascalBauermeister 2012-08-22 08:55:27

MM193 compatibility

Can someone update this macro for moin193???

• Answer: This macro is maintained there: http://ten.homelinux.net/productivity/MoinMoinExtensions/macro/SearchInPagesAndSort -- PascalBauermeister 2012-08-22 08:55:27