Creating easier to use unified wiki markup for release 2.0

Problems with current markup

<!> refactor, collect more links to related bugs, feature requests and user observations. This part should be short and clear: what are the REAL problem that we need to solve?

There are also comments or complaints about Moin's WikiMarkup. These can be defined into some categories:

Markup should be selected because we have decided it is reasonable, by our own reasons, not just because someone else thought so. In short if we cannot express the reasoning here I would not see the point of making a change.

There is another page, MarkupProposals, which is heavily edited (hooray for the wiki way) but suffers the problem of trying to be all three of those, with a bit of online help page thrown in. It also starts off abrasively, with professorial diagrams and a decidedly non-neutral point of view.

There are also people reporting feeling lost in the wealth of our HelpContents pages.

So, it's clear the refactoring under and over the hood as it were, need to happen at the same time. We need to unify the markup... so the new, unified markup becomes part of the design too.

Related work

There are currently efforts to unify a number of things:

Parsers, Processors and Macros are called sometimes PPM, or simply Extensions. See also: FeatureRequests/PPMCalledFromPragma.

Goals for new markup

<!> replace most of the text with common goals from the proposal page. It must be short and clear: without clear goals we can't evaluate any proposal.

Unifying the markup needs the following traits:

I'd like this page to describe specific suggested markup, in a style rather similar to that found in the various HelpOnEditing pages, with discussion following, in the style of the discussion about any possible new MoinDev/MoinMoinLogo.

Initial examples are from the IRC log that spurred me to write this up, more is up to you.

How to tell if it makes sense?

When we've decided for ourselves why it does make sense, people can read these defining thoughts at the top of the related HelpContents page and not feel nearly so lost as they do today.

How can we tell if it can make sense so easily? Try to describe it in plain language to people whose idea of "computer savvy" is that they can find their web browser icon without being told what Earth look like. For a good test I recommend trying to say your description out loud, and suppose your audience is such an ordinary person.

In this regard all wiki are imperfect. It was also well noted on the IRC channel that redefining the markup can actually effect how the code under the hood looks - in order to handle it. Also, if it can't be coded,efficiently, it's possible that describing it won't be so effective either.

We were talking about merging the SectionParser, and it wandered in the direction of what code makes sense or not. Yes, this is very long. I have ellided intervening threads about different topics and people popping in and out with the assorted hello and bye noises.

moin_markup_ideas_irc_29apr05.txt

Hmm. This means we have three things to consider:

The topic turned (twisted? :-? ) a few other directions, then back again. Eventually I dug out a browser and started this page. Hope it helps :D

Conclusions I think I saw:

Markup Specification

Markup proposals

Edit Section

Easy markup

PositionalMarkup is not a good name for the proposal, because it was never intended to allow only positional arguments in extensions.

  • But Easy Markup is also a bad name, you can call any proposal "Easy".
    • If you can think of a good name, just change it.

Goals

  • Easy to learn
    • only a few simple familiar concepts
    • as consistent as possible
  • Easy to use
  • Easy to type
    • compact syntax (less then html)
  • Easy to read
    • clean syntax
    • uses symbols in a familiar way
    • no unnecessary redundancy
  • Small MoinMoin-Core, use of plugins where it makes sense

  • Similar to MediaWiki when possible, as some new MoinMoin users already know MediaWiki

    • The other stated goals are more important than similarity to MediaWiki

Rules:

  • Anything inside [[]] is a link

  • A Link will never include (unlike current markup)
  • The second optional argument can be used as the caption for the link.
  • The second optional argument can also be an image (attachment or page, ends with .gif, .png or .jpg). This image will then be turned into a link.
  • The arguments are separeted by a commma.
  • Parser should discard spaces after the comma; Both [[Page Name, Title]] and [[Page Name,Title]] should both be displayed as Title

Markup

Display

Comments

[[PageName]]

PageName

[[Page Name]]

Page Name

[[PageName, Title]]

Title

[[Page Name, Title]]

Title

[[Page Name, moinmoin.png]]

moinmoin.png (links to "Page Name")

http://example.com

http://example.com

same for all other schemas

[[http://example.com]]

http://example.com

same for all other schemas

[[http://example.com, Example]]

Example

same for all other schemas

[[moinmoin.png]]

moinmoin.png

link to an attachment, will open the image in a window. Assume that attachments are pages.

[[moinmoin.png, 1000 words]]

1000 words

same with custom title

[[http://example.com, moinmoin.png]]

moinmoin.png (links to http://example.com)

[[http://example.com/moinmoin.png]]

http://example.com/moinmoin.png

link to external image - does not include the image!

[[http://example.com/moinmoin.png, Funny Man]]

Funny Man

link to external image with title

[[wiki:MoinMaster/Remote Page Name]]

interwiki link to: MoinMaster/Remote Page Name

interwiki link

[[wiki:MoinMaster/Remote Page Name, page from Master]]

page from Master (links to MoinMaster/Remote Page Name)

interwiki link

info@somedomain.org

info@somedomain.org

Extensions (macros, parsers)

Rules:

  • Anything inside {{ }} is an extension ( / plugin / PPM )

  • Extensions support positional arguments, keyword arguments and keywords as arguments.
  • Arguments are in parentheses
  • The arguments are separeted by a commma
  • Parser should discard spaces after the comma. Both {{Image(sunset.jpg, left)}} and {{Image(sunset.jpg,left)}} should get the same result.

description:
markup example

linebreak macro:
{{BR}}

WordCount macro:
{{ WordCount(FrontPage, 1000) }}

indent table parser:
{{IndentTable
1                               
 2                       
  3                               
   4
}}

Python code:
{{python
import sys, os
print 'hello!'
}}

SectionParser, uses keyword arguments:
{{section(class=cssclass, format=rst)
RST markup here...
}}

Image macro, uses keywords as arguments, for example the keyword "right" means to add an image on the right side of the page:
{{ Image(sunset.jpg, thumb, right) }}
{{ Image(sunset.jpg, left) }}
{{ Image(sunset.jpg, left, thumb) }}

Empty extension default to plain plugin:
{{
plain text as is
}}

Inclusion

A stated goal is to reduce the number of concepts for the user and the moin developer. IMHO the moin core should be as small as possible. As a consequence, inclusion should be implemented as macros.

markup example

description: includes ...

Include(Featured Article)

page "Featured Article"

Include(moinmoin.png)

"moinmoin.png"

Image(moinmoin.png)

"moinmoin.png"

Image(sunset.jpg, thumb)

"sunset.jpg" as thumbnail

Image(sunset.jpg, thumb,"a sunset")

"sunset.jpg" as thumbnail, image caption "a sunset"

Image(sunset.jpg, right)

"sunset.jpg", the image is sent over to the right

Image(sunset.jpg, left)

"sunset.jpg", forced to the left

Image(sunset.jpg, 100)

"sunset.jpg", scales image to maximum size 100

Image(sunset.jpg, 200, "a sunset")

"sunset.jpg", scales image to maximum size 200, image caption "a sunset"

If images are pages or sub pages, then we don't need Image, only Include:

  • Include(Some Page) - include the page content

  • Include(Some Image) - include an image saved as "Some Image"

But then size and alignment don't not make sense for Include.

Alternative Extension Syntax

Use a colon instead of brackets to separate the macro name from the macro arguments.

{{ Image: sunset.jpg, right }}
{{ Image: sunset.jpg }}

{{ section: class=cssclass, format=rst
RST markup here...
}}

{{BR}}

{{IndentTable
1                               
 2                       
  3                               
   4
}}

Backward compability

  • a conversion script is needed, which converts the old markup into the new markup
  • as positional arguments, keyword arguments and keywords as arguments are supported for PPMs, it is possible to convert the old syntax for parsers and macros to the new syntax for PPMs
  • tbc

Discussion

This proposal is more complex than KeywordMarkup, as you have both positional markup and keywords, its more like a programing language. For example, the function calling convetion: name(value, value), compared with simpler html: <tag key="value" key="value">. This also add the complexity of dealing with keyword and positional arguments in same call, just like in Python. For example, is this call correct? name(key=value, value)?

This will be harder to use for the new user, easier for a programmer. -- NirSoffer 2005-07-11 07:30:30

  • Yes, the proposal is more complex than KeywordMarkup, because it offers someone who creates a macro more choice. If we take away that choice, we loose backward compability with the current MoinMoin markup. Keyword arguments make sense if you have a macro with many arguments and you can't use "keywords as arguments". When you have to use keyword arguments where they are not nesessary (for example with macros with only one or two arguments), this constraint adds unnesessary complexibility from the perspective of the user of your extension. New users normally don't use macros or parsers with 5, 10 or 15 arguments. To offer them a simple syntax for the extensions they normally use by allowing positional arguments and keywords as arguments, makes it IMHO easier for them.

Edit Section

Keyword markup

This proposal try to:

  • Be clear, don't use funny characters if there is a readable way
  • Be consistent as possible, so user learn only few things
  • Use current syntax or syntax we use in other parts of the system
  • Be like WikiPedia markup if it make sense

  • Be like html - well known, simple, easy to merge in stuff like class:value in a natural way

The basic idea is [[]] make link, {{}} include extensions. Everything but links is implemented with extensions and can be replaced with farm or wiki plugins.

  • (!) Technical note: http, page, image and like will be plugins. To have external links that use "rel=nofollow", simply install a 4 line nofollow http plugin, that extend the builtin http handler by sending the attribute to the formatter. To have fancy include with 17 arguments, install a custom page plugin.

This proposal try to use key:value pairs when possible. We already use this for ACL:

#acl user:read,write All:

and search:

[[FullSearch(linkto:PageName title:Help)]]

And we many want to use this for meta data:

Author: Name
Status: Draft
...

Rules:

  • Anything inside [[]] is a link

  • Link will never include (unlike current markup)
  • title:, image: keywords used just like in our search for special cases.

  • No keyword in the first item means page: - [[PageName]] == [[page:PageName]]. Because the most important markup in a wiki is a page link.

  • http://, ftp:// are (funny) keywords too :)

  • Support old link types with no change, until at least until next major release

Markup

Display

Comments

[[PageName]]

PageName

[[Page Name]]

Page Name

[[PageName title:Title]]

Title

[[Page Name title:Title]]

Title

[[Page Name image:moinmoin.png]]

moinmoin.png (links to "Page Name")

Its not hard to parse but smell

[[Page Name title:Title image:moinmoin.png]]

moinmoin.png
Title

[[http://example.com]]

http://example.com

same for all other schemas

[[http://example.com title:Example]]

Example

same for all other schemas

[[moinmoin.png]]

moinmoin.png

link to an attachment, will open the image in a window. Assume that attachments are pages.

[[moinmoin.png title:1000 words]]

1000 words

same with custom title

[[http://example.com/moinmoin.png]]

http://example.com/moinmoin.png

link to external image - does not include the image!

[[http://example.com/moinmoin.png title:Funny Man]]

Funny Man

link to external image with title

[[wiki:MoinMaster/RemotePageName]]

RemotePageName

interwiki link

[[wiki:MoinMaster/RemotePageName title:page from Master]]

page from Master

interwiki link

Backward compatibility:

Markup

Display

Comments

PageName

PageName

works unless turned off in config

http://example.com

http://example.com

same for all other schemas

Additions:

Markup

Display

Comments

[[Page Name class:value]]

Page Name (use css class)

add the css class of the link <a class="original-class value">

[[Page Name accesskey:value]]

Page Name (use accesskey)

Set accesskey for the link <a accesskey="value">

Problems:

  • Can't have page named "Page Name title:Something" :-)

    • Unless we allow: [[page:"Page Name" title:Title]]

Include

Rules:

  • Anything inside {{}} will include the content in the page

  • The content can be some extenstion (called now macro/parser)
  • Include of pages and attachments is done with extenstions
  • Will not support old call syntax because links took the [[]]`

  • page:, image:, media: are plugins

  • The first item is the plugin, the next are the arguments
    • This is little wired, but thats how http://domain title:text works in the links, if you see http: as a plugin.

  • Order of arguments does not matter

Markup

Display

Comments

text {{BR}} text

text
text

include <br>

{{PageCount}}

15698

simple macro

{{page:Page Name from:"^= heading 1 =$" to:"^= heading 2 =$"}}

(include parts from "Page Name")

macro with arguments

{{page:Page Name}}

(include the contents of "Page Name")

include a page

{{image:Image Name}}

(include "Image Name")

or an image

{{media:Media.mpeg}}

(include "Media.mpeg")

include time based media

{{http://example.com/moinmoin.png}}

moinmoin.png

or an external image

{{http://example.com/}}

(include the html from example.com/)

<!> unsafe, unless we sanitize the html

{{image:Image Name class:cssclass title:Simple Figure}}

(Figure section)

a simpler way to do figures {{{#!figure\n#class cssclass\n}}}

Parsers extenstions:

Empty extension default to plain plugin:
{{
plain text as is
}}

Python code:
{{python
import sys, os
print 'hello!'
}}

New SectionParser:
{{section:cssclass format:rst
RST markup here...
}}

or:
{{section class:cssclass format:rst
RST markup here...
}}
?

{{figure
some markup
 <!> we must have nesting if we want to include here!
}}

Backward compatibility:

Markup

Display

Comments

attachment:file.txt

file.txt

inline:file.txt

   1 I'm a file.
file.txt

drawing:drawing.png

drawing.png

http://moinmoin.wikiwikiweb.de/wiki/classic/img/moinmoin.png

http://moinmoin.wikiwikiweb.de/wiki/classic/img/moinmoin.png

Maybe we should use the same words: inline, attachment and drawing instead of page and image?

Discussion

IMHO key:value pairs should only be used when nessesary. One advantage of a wiki markup language is that the user has to type less when he uses it instead of html. With a wiki markup similar to html you loose that advantage.

I like
{{Image(sunrise.jpg,right,nofloat,thumb,"A sunrise")}}
more than
{{Image:sunrise.jpg alignment:right float:nofloat size:thumb caption:"a sunrise"}}.
I also think that a syntax without key:value pairs is easier to learn for a new user.

WikkaWiki uses a syntax similar to html ({{image class="center" alt="DVD logo" title="An Image Link" url="images/dvdvideo.gif" link="RecentChanges"}}) and one of the reasons I chose MoinMoin was because I liked the MoinMoin markup more than the WikkaWiki markup.

Because of these reasons, I prefer the proposed PositionalMarkup to the proposed KeywordMarkup.

  • Try to explain a new user how to call this:

    {{Image(sunrise.jpg, right, nofloat, thumb, "A sunrise")}} The order of the arguments is important? How do you leave the default value? How do you find the function signature? You must add keywords for this type of call.

    • The order of the arguments is not improtent. {{Image(sunrise.jpg, nofloat, thumb, "A sunrise", right )}} also works.

      As a consequence the plugin is easy to use, even for new users. A new user would probably use not all optional arguments, she would probably use {{Image(sunrise.jpg)}} or {{Image(sunrise.jpg, "A sunrise" )}}.

      The syntax {{Image:sunrise.jpg alignment:right float:nofloat size:thumb caption:"a sunrise"}} is IMHO harder to learn, because a user has to know the keys and the values which are allowed for each key. The argument "right" to send an image over to the right is easy to remember. If a user also needs to remember the keyword "alignment", it gets harder.

      In KeywordMarkup there is also no clear distinction between the name of the macro and the arguments of the macro, which might be confusing for some users.

I don't think it will work - arguments without any specific order that are magically bound to the correct variable?

What if I have some arbitary strings like:

  • {{section, cssclass, cssid, title}}
    {{Action, edit, Edit, BacktoPage}}
    {{Include, from here, up to there}}

Either order matter, or you need keys. keys are easier to read.

  • Agreed, in these cases you have to use keyword arguments or positional arguments.

Discussion


I prefer the usage of ["Moin Moin Etymology"] displaying as Moin Moin Etymology, ["Moin Moin Etymology" Etymology] displaying as Etymology, and urls being the way they are now. This is how we have implemented it at DavisWiki.org. The only possible confusion is that sometimes people put quotes around external links, such as ["http://daviswiki.org" page], which would be solved (apparently) in this proposed markup, but there's no reason the same couldn't be done with our quoted markup -- just automatically route "http://*" to be external. We have chosen to notify the user when they make this mistake, which teaches them how to make internal links. Maybe I'll change this. I do like the idea of unified [[Image]] macro that can toggle thumb/no thumb. Right now we have our [[Thumbnail]] macro in addition to the attachment: syntax, which is ugly, but we will unify soon.

I would also propose that if you want to simplify markup you turn off WikiNames by default. Giving the user two radically different linking mechanisms is very counterintuitive. Also, I noticed that in the above examples commas were used within the macros. As was noted, this is inconsistant -- but does it matter? I think if you are going to use "|" as the delimator of links that it ought to delimate everything, just as wikipedia does it. I like the use of " quotes because they are used in natural english to quote things, and commas are used to seperate ideas, not | bars. --PhilipNeustrom

Refactoring proposals

The goals stated in both proposal are duplicated, we should factor them out to the main page, leaving only the specific goals of each proposal.

It will be interesting to try a version which is even more like WikiPedia, not because its better, but because its poplar, and the best interface is the familiar interface. Here is a MediaWiki syntax reference.

A html like (WikaWiki) syntax also can be nice and very easy to learn. And there are more proposals on the wiki, maybe its good to collect them and compare here.

Python like calls

Since some users prefer positional arguments, and some keywords, and calls with one or two arguments seems bloated with keywords, and calls with more arguments are impossible without keywords, we need both :)

Maybe use the python tokenizer that Oliver wrote, so both [["Page Name", image="moinmoin.png"]] and [["Page Name", "moinmoin.png"]] will work. This will create a more complex system, but can satisfy all users.

The problems with Python like calls is its too complex. In Python you have both strings and bound names. In a wiki page, everything is a string, so we don't need any quoting:

  1. .

    {{plugin arg1, kw=value}}
  2. .

    call = "plugin"
    argskw = ["arg1", "kw=value"]
  3. .

    args" = ["value",]
    kargs = {"kw": "value"}

MoinMoin: UnifyMarkup (last edited 2014-05-15 14:01:49 by RogerHaase)