Description

The migration process is likely to bother most wiki admins. "Is the data ok?" is a question they'll probably have constantly in mind. So, any typo in the docs or comments in the migration scripts, any forgotten small side effect is likely to vastly decrease their trust in the process. This page logs these issues as I found them in a recent "paranoid-mode" migration.

Steps to reproduce

1. Fear
2. Backup
3. Migrate

Details

• The scripts often say that the file size should get a bit larger because of latin1 to utf8 conversion, etc. A note should be added explaining that the data dir size (reported by du) will vary a lot between migration steps, sometime becoming quite larger, sometimes quite smaller, because at some point many empty dirs are created, and removed at another. This of course depends on the filesystem. Other methods should be used to check things are still ok, such as ls -1 | wc -l to cound the number of pages, md5sum to quickly check that files that shouldn't have changed haven't changed indeed, and diff to check that changes made to files are ok.

• Some migrations scripts do not copy the error.log file. This is a user data file and we must copy it from the original directory without change.
  • You don't need to copy it at all. It just contains error msgs of a moin version you currently replace by a newer one...

• data.pre-mig1 (the original data dir) will likely be much larger than data.pre-mig2 because it still contains cached data, which is rightfully dropped by the migration (to be recreated next time the wiki is launched).

  • "Some_Page already exists" error occurs if there are two pages with names like "Some Page" and "Some_Page" (i.e. identical save for the space / underscore). Rename these pages before starting the migration.

• data.pre-mig4 will likely be much larger than data.pre-mig3 because it contains a ton of empty cache directories.

• data.pre-mig5 will likely be smaller than data.pre-mig4 because it drops those empty cache directories (to be created next time the wiki is launched).

• data.pre-mig6 will likely be much larger than data.pre-mig5 because many small files are created (increase depends on the filesystem, of course).

• data.pre-mig7 should be the same size as data.pre-mig6, up to one or two filesystem blocks.

• data.pre-mig8 should be slightly larger than data.pre-mig7 because of the conversion from LF to CRLF line endings.

• data.pre-mig9 should be the same size as data.pre-mig8, only a few bytes changed in the edit-logs.

• the final data dir should be the same size as data.pre-mig9, except if you have many users.

• 12_to_13_mig5.py drops MoinEditorBackups without warnings. A decrease in the page count should be expected.

  • There is no need for special handling of MoinEditorBackup files. Just migrate them as any other file.

  • Moineditorbackup pages have no edit-log, and mig5 needs an editlog for processing.

• line 18 of 12_to_13_mig5.py is incorrect. It should read -> data/pages/PageName/revisions/<revno> (PageName added), of course.

• 12_to_13_mig6.py comments contain a reference to 12_to_13_mig1.py and data.pre-mig1, which should be mig6 instead.

• 12_to_13_mig8.py (rightfully) adds a \n at the end of the current files. A change in their md5sums should be expected.

• 12_to_13_mig9.py drops the final \n at the end of the user files, and also drops error.log. Diff will report that.

  • fixed in patch-445

• Information about the repair_language script should be added to README.migration.

  • Note that this script is not needed unless you want to use the #language feature. This is why its not a migration script. Its just a way to make it easier for wiki admins to use the new feature on old data files.

• repair_language drops the final \n of every revision in the wiki. Sometimes, this means the final blank line is removed.

  • fixed in patch-447

• The upgrader should be told to copy intermap.txt from the distributed data directory.

  • No, he should not, intermap.txt is located in the data directory, and is copied when you create an instance.
    • The upgrader should think about merging the file with his own. This might result in overwriting in most cases.

• He should be told to copy the BadContent page, too.

  • No, he should not, bad content is updated automatically by antispam.
• After upgrading your data directory with the migration scripts you might get this error (happened to me when upgrading from 1.2.3 to 1.3.3)

  {{{MoinMoin Configuration Error

Could not import plugin package from "./data/" because of ImportError: No module named plugin.

Make sure your data directory path is correct, check permissions, and that the data/plugin directory has an init.py file.}}}

• To solve this I copied the plugin-directory from share/moin/data to the data-directory of the updated wiki-instance. It would be nice to have this or just the creation of the file __init__.py integrated into the migrations scripts. -- DavidLinke 2005-02-13 12:07:46

Workaround

Relax, have a cookie.

Discussion

Richard Reitmeyer <reitmeyer_richard@emc.com> wrote an additional routine to get attachments stored in a seperated directory right translated into UTF-8. While he has some timeout problems a while ago I was aked to add it. Sorry I missed it till now. I rememebered this issue now by a migration I did at the moment. This routine must be copied into the migrationscripts directory and then called

python ~/src/moin-1.3.5/MoinMoin/scripts/migration/migrate_attachments.py pathname_of_attachments

migrate_attachments.py

Afterwards you need only to copy this tree to your mywiki/data/pages and set the right owner.

Plan

• Priority: Medium

• Assigned to: ThomasWaldmann

• Status: fixed the serious stuff. whoever reopens this, volunteers for fixing everything he wants.

CategoryMoinMoinBugFixed