Difference between revisions of "Retain Module Utilities"

Revision as of 17:56, 18 November 2015

Retain Connects to many different systems and there are utilities that exist to help with troubleshooting.

Troubleshooting Exchange

Test Utilities

One of the challenges to testing Retain is being able to send enough data. A few utilities have been created to make that easier. Make sure to only use these in your lab system as these will get your IP blocked by ISPs that block spammers automatically.

GoPostal

To create large amounts of unique messages for Retain you can use these utilities[[1]]:

• GoPostal.sh sends a user supplied number of unique emails to ten users.
• GoPostalFullAuto.sh sends 100 unique emails to ten users. This is best used with cron to run daily creating dynamic data.

PREREQUISITES:

• A Linux computer or VM. It does not have to be very powerful.
• sendmail needs to be enabled and Outgoing Mail set to the IP address of your receiving mail server.
• The receiving mail server must have 10 users, named user0-user9.

REQUIREMENTS:

• You need to edit the script to provide it with the domain of your test email server.

OPTIONS:

infile.txt contains the text of "Alice in Wonderland" from gutenberg.org as it is a public domain document. This file can be replaced with any large text document. The larger the better. You can change the username and number of users with that username. You must create them in your email system or the mail will not be delivered. The size of the email gopostal sends can be changed, by default it is set to 4k. Rule of Thumb: 1 character is 1 byte.

Text sources:

• Project Gutenberg is a good place for public domain text [[2]]

FileBlaster

To send large amounts of existing text data to Retain, you can use this utility[[3]]:

• FileBlaster.sh sends text files from a data directory.

PREREQUISITES:

• A Linux computer or VM. It does not have to be very powerful.
• sendmail needs to be enabled and Outgoing Mail set to the IP address of your receiving mail server.
• The receiving mail server must have a user available to receive the data.

REQUIREMENTS:

• Edit the script to provide it with an email address of your test email server.
• Place text data in fileblaster/data for sending. By default it has the top 20 ebooks from gutenberg.org
• For a larger data set you can use the Enron Email Data Set [[4]]

OPTIONS:

You can fill the data directory with any text data, it will be posted as the body of the message.

Data set for fileblaster

• Enron Email Data Set [[5]] Contains 1.2 millions messages. This is what I used, but is too large to be worth adding to the fileblaster download.

Other Email Datasets

• Apache Software Foundation Public Mail Archives [[6]]
• EU email communication network email data set [[7]]

Links to other data sets. Any text-based data set should work.

• Quora Links [[8]]
• Amazon data sets links [[9]]
• YCombinator links [[10]]

Text sources:

• Project Gutenberg is a good place for public domain text [[11]]
• Wikipedia Downloads [[12]]

• gopostal [[13]]
• fileblaster [[14]]

Data set for fileblaster

• Enron Email Data Set [[15]] This is what I used and it is 1.2 million messages.

Other Email Datasets

• Apache Software Foundation Public Mail Archives [[16]]
• EU email communication network email data set [[17]]

Links to other data sets. Any text-based data set should work.

• Quora Links [[18]]
• Amazon data sets links [[19]]
• YCombinator links [[20]]

Text sources:

• Project Gutenberg is a good place for public domain text [[21]]
• Wikipedia Downloads [[22]]