How to import mbox files into GroupServer¶

The following describes all steps before mbox2gs can be called. This document is intended for advanced users only. Only attempt to use this if you are really sure you know what you are doing! You have been warned.

1. (Optional) Disable SMTP (sending emails) in your GroupServer instance. You can disable SMTP in one of two ways.

   1. Modify etc/gsconfig.ini in your GroupServer installation

      • In etc/gsconfig.ini you have sections [config-default], which has set smtp = on by default. Set smtp = off.
      • Stop and start again your instance so the configuration change takes effect.
      • Test sending of email via your GroupServer instance. No message should be sent.

   2. Re-configure your GroupServer instance to use Python’s native SMTP daemon running in debug mode:

      • SMPT port set to 2525 in config.cfg

      • Run Buildlout:

        $ buildout -N
        

      • Start the server:

        $ sudo python -m smtpd -n -c DebuggingServer localhost:2525
        

      • Test sending of email via your GroupServer instance. No message should be sent, but you should see the email in the DebuggingServer that you are running.

        Important:	You may want to disable SMTP in your GroupServer instance while you are testing, or even migrating archivesd from old Mailman mailing-lists, because every email you import would be send to your mailing list members. Which is something you may wish to avoid.

2. Prepare Mailman archives for migration.

   Long story short, things are not in ideal shape in any Mailman archive out there, because:

   • Email addresses are usually obfuscated (eg. some.email at example.com, some.email).
   • A typical Mailman archive (usually gzipped) is often an invalid mbox file.

   The mailman2mbox Python script deals with the issues and helps you to convert your Mailman archives to mbox format <https://gist.github.com/corydolphin/1728592>.

3. Create a new GroupServer group to which you intend to import mbox archive(s) in to.

   Example: http://groups.example.com/groups/my-group/

4. Get a list of all email addresses from your Mailman mailing-list, or extract the list from mbox archives.

5. Load all email addresses to the group via the Add members in bulk page, linked from the Admin section of the group.

   Note:	Unless you disable SMTP everyone on the list will get an email about the fact that you added them to the group.

6. All prepared. You are now ready to migrate your mbox archive(s) using mbox2gs (see mbox2gs).

mbox2gs¶

$ mbox2gs -v -l my_group -f /tmp/test.mbox http://groups.example.com

Configuration file¶

The configuration for the mbox2gs script is handled by the gs.config module [1]. It is entirely concerned with token authentication [2]. To authenticate script needs to pass a token to the web pages that are used to add a post [3]. The pages compare the token that was passed in to one that is stored in the database. If they match the script is allowed to post.

[webservice-default]
token = theValueOfTheToken

[config-testing]
...
webservice = testing

[config-production]
...
webservice = production

[webservice-testing]
token = theValueOfTheTokenForTesting

[webservice-production]
token = theValueOfTheTokenForProduction

[config-firstSite]
...
webservice = default

[config-secondSite]
...
webservice = default

[webservice-default]
token = theValueOfTheDefaultToken

The code¶

The mbox2gs script is provided by the module gs.group.messages.add.mbox2gs.script. The main function takes the name of the default configuration file a single argument, which is normally supplied by buildout when it generates the mbox2gs script from the entry point.

The script parses the command-line arguments, and calls two further functions:

gs.group.messages.add.smtp2gs.servercomms.get_group_info_from_address:

The script posts to the page /gs-group-messages-add-group-exists.html to check if the group exists, and to get some information about the group.

gs.group.messages.add.smtp2gs.servercomms.add_post:

The script posts to the page /gs-group-messages-add-email.html to actually add the post.

Both pages are provided by the gs.group.messages.add.base module [1]; the data is sent by the gs.form.post_multipart function [2], with gs.auth.token [3] providing authentication (see the section Configuration File below).

Changelog¶