You can use MoinMoin to organize your contacts. For every contact you create a new page. Use the preinstalled NewPage macro to use specific names, places and templates, so that you just fill out the spaces under headings or make lists.
You can then use the FullSearch macro or something similar to automatically list contact pages. On the contact pages you can attach files that are related to the contact, make subpages like talking protocols or take notes of when you communicated last.
Be aware that clicking on a link in your wiki transfers referer information to the target server, this server than knows that you might have a page on your server with a name of the contact. This is information that you eventually might not wanted to give away. You will need to take steps to prevent this from happening (like using a proxy that filters the referer).