Table of Contents

Concurrency Issues

Many things can happen in a Moin wiki at the same time. This includes multiple CGI calls, multiple threads in a Twisted setup or multiple forked processes (the same as cgi...) in a mod_python setup.

To be fast, Moin tries to cache all kinds of things. Sometimes in memory (only good for long-running processes -- Twisted, fast_cgi, mod_python, standalone) sometimes on disc (good for every setup). Sometime these caches can run out of sync.

This page is focused on finding those places and getting them fixed soon.

Open Issues

Memory storage

For long running processes objects can be caching in memory. By now there is a proper infrastructure missing. The infrastructure must work in a MultiConfig setup! So the object must be attached to a Site object or to the site configuration object.

Persisted storage

All objects that cache stuff in memory and on disk. Example: wikidicts.py

A process/thread should read the cache which should have a time stamp. The data is modified. Now the cache should be write. But is the cache on disk the same that was read by this process? Or is some new data inside we would overwrite?

For threads, use the LockClass from below. For forks we need some lock files or file locking.

Possible solution:

   1 class ThreadLock(BaseLock):
   2 
   3     def __init__(self):
   4         import thread
   5         self._lock = thread.allocate_lock()
   6 
   7     def acquire(self, blocking=1):
   8         if blocking:
   9             return self._lock.acquire()
  10         else:
  11             return self._lock.acquire(0)
  12 
  13     def release(self):
  14         return self._lock.release()
  15 
  16 #
  17 # change this for another method of locking...
  18 #
  19 LockClass = ThreadLock
  20 
  21 
  22 #
  23 # a late init re. needs some more methods, perhaps...
  24 #
  25 # as seen in wikiutil.py:
  26 # _TEMPLATE_RE=None
  27 # ...
  28 #    global _TEMPLATE_RE
  29 #    if _TEMPLATE_RE is None:
  30 #        _TEMPLATE_RE = re.compile(config.page_template_regex)
  31 #    return _TEMPLATE_RE.search(pagename) is not None
  32 # ...
  33 #
  34 # could be replaced with:
  35 # _TEMPLATE_RE=LateRe(config.page_template_regex)
  36 # ...
  37 #    global _TEMPLATE_RE
  38 #    return _TEMPLATE_RE.search(pagename) is not None
  39 # ...
  40 #
  41 class LateRe:
  42 
  43     def __init__(self, *args, **kwargs):
  44         self._args=args
  45         self._kwargs=kwargs
  46         self._re=None
  47         self._lock=LockClass()
  48 
  49     def get(self):
  50         import re
  51         if self._re is None:
  52             self._lock.acquire()
  53             if self._re is None:
  54                 self._re=apply(re.compile,self._args,self._kwargs)
  55             self._lock.release()
  56         return self._re
  57 
  58     def search(self, *args, **kwargs):
  59         return apply(self.get().search,args,kwargs)

Questions

• Is that also an issue when running it stand-alone?
  • Many things on this page are outdated, but there are indeed known concurrency issues in the current code base that affect multi-process and multi-thread setups.