<!> We should move the setup instructions and tips into HelpOnWikiFarm, since it's too big to stuff into HelpOnConfiguration. The questions and support forum could stay as FarmQuestions for now.

Contents

  1. Setup instructions
    1. Using Apache and CGI
      1. Grouping rules for automation
      2. Top level wikis
      3. Putting the wikifarm offline
      4. Using rewrite rules instead of aliases
      5. Taking robots.txt and favicon.ico out of the wiki processing
    2. Using Fastcgi
    3. Using Standalone or Twisted server
  2. Useful Farming tips and tools
    1. Sharing users accross the wikifarm
    2. Sharing plugins across the wikifarm
    3. Automating instance creation
    4. Farm statistics
    5. Upgrades and maintenance across the wikifarm
  3. Support forum
    1. Problem setting up a farm
      1. moin.py contains:
      2. farmconfig.py contains:
      3. alpha.py is:
      4. The Problem
      5. Trying different data directories
      6. Solution
    2. Miscellaneous questions
    3. Some answers
    4. backlinks and search across wikis in a farm
    5. Farm with Desktop Edition
    6. Farm on IIS
    7. Directory path of farm instance
      1. Web interface for wikifarm?
    8. Wiki Farm URL Problems

Setup instructions

Using Apache and CGI

This is an example setup tested on Mac OS X with Apache 2. It has been slightly modified for a Apache 1.3 setup on FreeBSD, and should therefore work on most Apache platforms.

Download this example farm to use as base for your farm: wikifarm.tar.gz

Grouping rules for automation

In the Koumbit wiki, we have a seperate config for the "virtual hosts" configuration and the general listing. This is because some wikis (like http://rocococamp.info/) are not within the normal http://wikifarm.koumbit.net/ hierarchy. To accomplish this, the apache.conf file is split in two, the apache.conf file, which simply contains the list of url mappings and the apache-common.conf file which contains configuration common for all virtual hosts. In httpd.conf, you therefore need: 

<VirtualHost *:80>
    ServerName wiki.example.com
    Include /opt/wikifarm/config/apache.conf
    Include /opt/wikifarm/config/apache-common.conf
</VirtualHost>
apache.conf
. 
ScriptAlias /redwiki /opt/wikifarm/bin/moin.cgi
ScriptAlias /bluewiki /opt/wikifarm/bin/moin.cgi
apache-common.conf
. 
# common configuration for wikifarm virtual hosts
Alias /wikifarm/ "/usr/local/share/moin/htdocs/"

## map everything else to moin.cgi/moin.fcg
ScriptAlias / /opt/wikifarm/bin/moin.cgi/

<Directory "/usr/local/share/moin/htdocs/>
   Order allow,deny
   # ad-hoc blacklist
   Deny from 1.2.3.4
   Allow from all
</Directory>

<Directory "/opt/wikifarm">
        AllowOverride None
        Options ExecCGI FollowSymLinks
        SetHandler fastcgi-script
        Order allow,deny
        # ad-hoc blacklist
        Deny from 1.2.3.4
        Allow from all
</Directory>

Top level wikis

Using the above setup, wikis are installed in http://example.com/wiki. If you want to use a FQDN for your wiki, you'll have to add a special Apache config to your httpd.conf: 

<VirtualHost *:80>
    ServerName www.rocococamp.info
    #ServerAlias rocococamp.info # optional
    Include /opt/config/apache-common.conf
</VirtualHost>

Putting the wikifarm offline

With this configuration, to put the wikifarm offline, you need to comment out some rewrite rules and add a DocumentRoot: 

# put the wiki offline, also comment out the next rewrite rule
DocumentRoot /usr/local/www/data/maintenance/

#ScriptAlias / /opt/wikifarm/server/moin.cgi/

The apache.conf include must also be commented out.

Using rewrite rules instead of aliases

Rewrite rules a bit more flexible than the ScriptAlias directive, so they can be interesting to use. Simply replace: 

ScriptAlias /redwiki /opt/wikifarm/bin/moin.cgi

... with: 

RewriteRule ^/redwiki(.*)$ /opt/wikifarm/server/moin.cgi$1 [type=application/x-httpd-cgi,L]

Taking robots.txt and favicon.ico out of the wiki processing

To save processing time, it is preferable to take add static rules for regularly requested files like robots.txt: 

Alias /robots.txt "/usr/local/share/moin/htdocs/robots.txt"
Alias /favicon.ico "/usr/local/share/moin/htdocs/favicon.ico"

or: 

RewriteRule ^/robots.txt$ /usr/local/share/moin/htdocs/robots.txt [last]
RewriteRule ^/favicon.ico$ /usr/local/share/moin/htdocs/favicon.ico [last]

... for rewrite rules. Those rules must be added to the apache-common.conf configuration file so they apply to all wikis.

Using Fastcgi

Very similar to the above, you just need to change the .cgi extensions to .fcgi. The rewrite rules are different too: 

RewriteRule ^/redwiki(.*)$ /opt/wikifarm/server/moin.cgi$1 [type=application/x-httpd-cgi,L]

becomes: 

RewriteRule ^/redwiki(.*)$ /opt/wikifarm/server/moin.fcg$1 [type=application/x-httpd-fcgi,L]

Using Standalone or Twisted server

This is not much harder. Two things should be changed:

  1. url_prefix must be '/wiki' - its currently hardcoded into these servers
  2. You cant access the wiki with /redwiki, you must define multiple hosts, for example add few hosts to your host file: 
    • 127.0.0.1 redwiki bluewiki
80.205.68.2 redwiki.wikicolors.org bluewiki.wikicolors.org
      Then update your wikis list in farmconfig according to the new urls.

Useful Farming tips and tools

Sharing users accross the wikifarm

See CommonUserAccounts -- OliverSiemoneit 2006-11-13 20:04:06

Sharing plugins across the wikifarm

See FarmPlugins. The Koumbit wikifarm just uses symlinks to a common plugin/ directory when a new instance is created. See also FeatureRequests/WikifarmPluginDirectory.

Automating instance creation

Two approaches here:

Farm statistics

For now, there's a MuninPlugin that graphs the size of the wikis in the wikifarm.

The ScriptMarket/UserStatsScript could probably be generalized to a farm.

Upgrades and maintenance across the wikifarm

Koumbit uses the ScriptMarket/AllWikisHack to upgrade all wikis at once when the MoinMoin codebase is upgraded.

Support forum

Problem setting up a farm

Many thanks for the comments.

The problem is that I don't know how to set up a farm. There are no docs I've found, and following the example in farmconfig.py leads to odd results.

Here are my various config files ...

moin.py contains:

# Path of the directory where wikiconfig.py is located.
# YOU NEED TO CHANGE THIS TO MATCH YOUR SETUP.
## Dummied out since farmconfig.py seems to override it.
##sys.path.insert(0, '/path/to/wikiconfig')

# Path to MoinMoin package, needed if you installed with --prefix=PREFIX
# or if you did not use setup.py.
sys.path.insert(0, '/home/ss/ftp/pythonmods/moinmoin/tdi/lib/python2.4/site-packages')

# Path of the directory where farmconfig is located (if different).
sys.path.insert(0, '/home/ss/ftp/pythonmods/moinmoin/tdi/share/moin/config')

farmconfig.py contains:

wikis = [
    # wikiname,     url regular expression (no protocol)
    # Standalone server needs the port e.g. localhost:8000
    # Twisted server can now use the port, too.
    ("alpha",    r"^localhost:11000/alpha.*$"),
    ("beta",    r"^localhost:11000/beta.*$"),
    ("central",    r"^localhost:11000/.*$"),
    ##("moinmaster",  r"^moinmaster.wikiwikiweb.de/.*$"),
    ##("moinmoin",    r"^moinmoin.wikiwikiweb.de/.*$"),
]
...
    data_dir = '/home/ss/ftp/pythonmods/moinmoin/tdwiki/data/'

alpha.py is:

# we import the FarmConfig class for common defaults of our wikis:
from farmconfig import FarmConfig

# now we subclass that config (inherit from it) and change what's different:
class Config(FarmConfig):

    show_timings = 1
    sitename = u'Alpha' # [Unicode]
    interwikiname = 'Alpha'
    mail_from = "The Alpha Wiki <noreply@ucar.edu>"

beta.py and central.py are identical to alpha.py, except "alpha" is changed to "beta" or "central".

The Problem

In alpha's FrontPage I create a link to OurScope, and go create the page to describe the alpha team's scope.

When I try to do the same thing for the beta team, I get the alpha team's scope.

They share the same scope ... the alpha and beta sites are not separate. How can I make them separate? I notice the example in farmconfig.py uses separate host names, but I don't have access to make virtual hosts.

My directory structure follows the farmconfig.py example: 

   .../tdwiki/      # main wiki dir
       data/
       underlay/

Trying different data directories

I tried having a separate data directory for alpha, beta, and central. But that doesn't work either. When I create a user profile, it's only known to central, not alpha or beta. So nobody can create or edit pages for alpha or beta.

The configuration files are the same as above except alpha.py contains: 

    data_dir = '/home/ss/ftp/pythonmods/moinmoin/tdwiki/alpha/data/'

beta.py contains: 

    data_dir = '/home/ss/ftp/pythonmods/moinmoin/tdwiki/beta/data/'

central.py contains: 

    data_dir = '/home/ss/ftp/pythonmods/moinmoin/tdwiki/data/'

Does anyone have a complete example - config files, directory structure, etc - of a farm that uses urls like mysite/alpha and mysite/beta instead of separate host names?

Many many thanks! Steve

Solution

I had the same problem. Here is the recipe that worked for me: HelpOnConfiguration/ApacheCgiWikiFarm -- -- UcheOgbuji 2008-08-29 19:49:49

Miscellaneous questions

When I make a farm using farmconfig.py, the ACLs specified there:

simply don't work. Even though I'm signed in I cannot edit any page.

Reply: the acl_rights_default above works with a single site; why not with farmconfig.py? None of my pages have "#acl" lines. My goal is to have known users be able to edit pages, and all else read pages, with no '#acl' commands sprinkled throughout the pages.

Second problem: All the builtin buttons, like UserPreferences, my login name, etc, refer to addresses like localhost:8000/UserPreferences,, localhost:8000/MyName, etc. But this means the wikis variable in farmconfig.py must contain an entry for each user...

Reply: If I set up farmconfig.py with: 

# farmconfig.py:
wikis = [
    ("mds",         r"^localhost:10000/mds.*$") ]

# mds.py:
    data_dir = '/d1/steves/wiki/mds/data/'

then when I click "login" I get: MoinMoin Configuration Error Could not find a match for url: "localhost:10000/UserPreferences".

So I create a new entry in wikis:

and a new UserPreferences.py file:

and set up the new data dir and try again. Now I can create a profile and login. But when I click on my name at the top of the screen I get:

So now I need to create a wikis entry for StevesSmith, and a SteveSmith.py, and a new data dir for SteveSmith. And I have to do this for each new user. Yikes!

I've looked for docs that better describe farm configuration, but not found any.

Many thanks,

Steve

Some answers

Provide a detailed description of your setup. Farmconfig in moin 1.3.3 works fine for my cgi/Apache setup and I cannot remember any problems from setting it up.

Maybe simply try to do a tested (and supported) configuration. I never tried a config like you try to do. The usual farm config here is like shown in the sample moinmaster.py and farmconfig.py config files as provided in the distribution archive. That means:

This all doesn't strictly mean it won't work in any other config, but it was primarily tested (and is running productively) in this config.

If you want to run on localhost only and have no DNS, you maybe simply can put some entries into your hosts file, all pointing to 127.0.0.1.

-- ThomasWaldmann 2005-03-14 00:29:07

How to run wikifarm with Apache + Mod_python without virtual hosts? I have followed the cgi example on top of this page, but I don't know how to go forward. -- ZhangYunfeng 2006-10-10 07:54:18

Is it possible to make backlinks work across wikis in a farm? I'd also like to search across wikis (almost the same question since backlinks uses "linkto") and while we're at it having visual site map work as well would be nice :) I'm considering splitting up my wiki into a farm in order to clean up the namespace as well as make the purpose of each wiki more focuses and understandable. I would still like to be able to navigate easily between them and show how they are interconnected. -- GregWhittier 2024-04-16 11:20:56

Farm with Desktop Edition

You may wonder why would anyone need to run a wikifarm on a DesktopEdition but the truth is that I would like to keep my personal wiki and my work wiki completely separate (work wiki can then be synchronized with a remote server, my personal stuff can stay my own). Can anyone give me a brief outline (provided it is possible to run this with the DesktopEdition on Windows) of how to accomplish this? I am a total newbie with python so instructions as simple as the like of "open eye, insert fork, twist, call 911" would be greatly appreciated. Ivaylo, 2006-02-07

Farm on IIS

I have been trying without success to get a MoinMoin Farm running under IIS. running a single wiki or a farm the defaults to a single wiki works fine, but I can't get multiple wikis working.

Directory structure 

Moin\
    \Lib
    \mywiki\
           \mywiki_data
           \RedWiki_data
           \BlueWiki_data
    \share

the farmconfig file contains: 

wikis = [
    # wikiname,     url regular expression (no protocol)
    # ---------------------------------------------------------------
    ("RedWiki",     r"^svwebserver.ABC.com/RedWiki.*$"),
    ("BlueWiki",    r"^svwebserver.ABC.com/BlueWiki.*$"),
    # default
    ("mywiki",  r".*"),   # this is ok for a single wiki
]

mywiki.py 

from farmconfig import FarmConfig
class Config(FarmConfig):
    sitename = u'SV Wiki' # [Unicode]
    interwikiname = 'SV Wiki'
    logo_string = """<img src="/SV-Wiki.png" width="60" alt="%s">%s""" % (sitename,sitename)
    page_front_page = u"FrontPage"
    data_dir = './mywiki_data/'

RedWiki.py 

from farmconfig import FarmConfig
class Config(FarmConfig):
    sitename = u'RedWiki' # [Unicode]
    interwikiname = 'RedWiki'
    page_front_page = u"FrontPage"
    data_dir = './RedWiki_data/'

BlueWiki.py 

from farmconfig import FarmConfig
class Config(FarmConfig):
    sitename = u'BlueWiki' # [Unicode]
    interwikiname = 'BlueWiki'
    page_front_page = u"FrontPage"
    data_dir = './BlueWiki_data/'

if I hit the default wiki it works fine http://svwebserver.ABC.com/moin.cgi/FrontPage but trying to hit http://svwebserver.ABC.com/RedWiki/moin.cgi/FrontPage or http://svwebserver.ABC.com/moin.cgi/RedWiki/FrontPage I have also tried having ^svwebserver.ABC.com/moin.cgi/RedWiki.*$ in the farmconfig wiki's list I suspect that it is somthing in the IIS configuration but have not been able to figure it out yet.

-- JamesSpears 2006-03-02 19:16:06

Directory path of farm instance

I'm trying to adapt the VisualSiteMap action in order to use it in several instances of a wiki farm. The main problem I have is that my ignorance of Python programming is only paired by my ignorance about MoinMoin's API and data structures :-)

Let's see if someone with a little more knowledge than I can help me with this.

The problem that VisualSiteMap has, when used in a farm is that it needs a path within the public http directory in order to generate the image files and then refer them from within the page. The current version, simply asks you to edit the VisualSiteMap.py file and fill in to variables (constants?) named CACHE_DIR and CACHE_URL with the absolute path of the directory to put the files in (since they are generated by a program invoked via a couple of plain os.system() calls), and the URL by which they'll be referred then.

I don't have a problem with CACHE_URL since I can use request.cfg.url_prefix and add it what I want (maybe configuring the suffix of that the same way, that is via a variable in the same file).

Now, what I need is a way to know, programmatically, where in the filesystem lies this directory (the absolute path of it). Is there a way to know that from within moin?

The current version has this at the begginning: 

   1 # This should be a public path on your web server. The dot files, images and map files are created in this directory and
   2 # served from there.
   3 #CACHE_DIR  = "C:/DocumentRoot/cache"
   4 #CACHE_URL  = "http://my-server/cache"
   5 CACHE_DIR  = "/org/de.wikiwikiweb.moinmaster/htdocs/cache"
   6 CACHE_URL  = "http://moinmaster.wikiwikiweb.de/wiki/cache"
   7 
   8 # Absolute location of the dot (or neato) executable.
   9 #DOT_EXE    = "C:/Programme/ATT/GraphViz/bin/dot.exe"
  10 #DOT_EXE    = "/usr/bin/dot"
  11 DOT_EXE    = "/usr/bin/neato"

where you're supposed to edit these variables (constants? -I should start learning Python).

And then it uses this within the execute() function like this: 

   1     wikinamefs = wikiutil.quoteWikinameFS(pagename)
   2     wikinameurl = wikiutil.quoteWikinameURL(pagename)
   3     fnprefix = os.path.join(CACHE_DIR, '%s_%s' % (wikinamefs, maxdepth))
   4     dotfilename = '%s.%s' % (fnprefix, 'dot')
   5     imagefilename = '%s.%s' % (fnprefix, OUTPUT_FORMAT)
   6     mapfilename = '%s.%s' % (fnprefix, 'cmap')
   7     imageurl = '%s/%s_%s.%s' % (CACHE_URL, wikinameurl, maxdepth, OUTPUT_FORMAT)

What I need to do is to know programmatically where this instance has its directory... for example, if I have to wikis in a farm, one with: 

   1 CACHE_DIR  = "/org/de.wikiwikiweb.moinmaster/htdocs/cache"
   2 CACHE_URL  = "http://moinmaster.wikiwikiweb.de/wiki/cache"

and the other one with: 

   1 CACHE_DIR  = "/org/de.wikiwikiweb.other/htdocs/cache"
   2 CACHE_URL  = "http://other.wikiwikiweb.de/wiki/cache"

how do I find out that http://moinmaster.wikiwikiweb.de/ is at /org/de.wikiwikiweb.moinmaster/htdocs/ and http://other.wikiwikiweb.de/ is at /org/de.wikiwikiweb.other/htdocs/ in the filesystem (the /wiki thing I can manage with request.cfg.url_prefix and I actually don't need the domain name, since using "/" instead of "http://moinmaster.wikiwikiweb.de/" works just fine.

TIA

-- MarianoAbsatz 2007-05-15 20:23:52

Thanx, Anacrat... but it didn't do, since what I needed was the local (unix) path and NOT the URL path.

But I just found out how to solve it... I can write the CACHE_DIR as relative to the directory where moin.cgi is, since that is the CWD for the script. As I have a different instance of moin.cgi for each instance of the farm, using relative paths solved it easily.

-- MarianoAbsatz 2007-05-26 23:02:48

Web interface for wikifarm?

I would most appreciate a web interface that allows to create and configure new wikis in a wiki farm. Are there any extensions doing this?

-sboehringer

Wiki Farm URL Problems

I've created a path based wiki farm: 

www.website.com/wiki/instance1
www.website.com/wiki/instance2

Everything seems to work great but when I click through on any links on the wiki they don't find a matching wiki config and I get the message No wiki configuration matching the URL found!.

The links appear as: 

http://www.website.com/cgi-bin/moin.cgi/existing-wiki-page/

When I would like them to appear as: 

First wiki instance
http://www.website.com/instance1/existing-wiki-page/

Second wiki instance
http://www.website.com/instance2/existing-wiki-page/

I'm sure I'm doing something stupid here :)

MoinMoin: FarmQuestions (last edited 2012-10-05 15:09:20 by c-76-117-90-149)