Monitoring the SASI update

SASI (Sophos Anti Spam Interface) for iQ.Suite Wall is an interface that can be used as of iQ.Suite version 10 as an add-on function to keep out spam and other mass mailings. Users must have a valid licence, which comes as an option with the iQ.Suite Wall module.

During analysis, SASI checks emails for the known patterns of typical spam mails for iQ.Suite Wall. The pattern database resides locally on the server on which iQ.Suite is installed. The database is updated automatically at regular intervals.

SASI consists of the pattern files asdb.antispam and db.summary and the program pmx_engine.dll.

Spam mutates - sometimes pretty slowly, sometimes a whole new wave of spam will flood into users’ mailboxes within a single day. The conventional way of dealing with this is to update blacklists and word lists manually as quickly as possible so that users are not buried under a new pile of junk and have to spend valuable time sorting it manually. Spam mails that contain little (e.g. just an html link) or no text at all (just an image) are very difficult to identify and filter on the quick. Using SASI makes it possible to kill two birds with one stone:

1. Analysing a broad array of different criteria makes it possible to identify emails with little or no text as spam. This includes checks on the email header, message text and file attachment information as well as a query on the standard RBL server.
2. The fully automatic update of the anti-spam engine and patterns on the basis of standard protocols (HTTP or FTP) guarantees that information about spam mails currently in circulation is always up-to-date. There is no longer any need for corrective intervention by the administrator.

The SASI update process

The SASI update takes place in two phases (from a customer viewpoint):

1. Download of the new anti-spam patterns from the GROUP server via http or ftp. Up until version 11.1 this process is handled via a scheduled task under Windows. In this case, the administrator has to configure the file ..iQSuiteSASIUpdatesasi_update.cmd as a scheduled task (on Linux as a cron job). From version 11.2 onwards the update takes place automatically. In this case, the main features are that the SASI Update Service (sasi_updateService.exe) is launched with the file settings.xml as parameters and logged in the following files:
   
   ../iQSuite/SASI/Update/sasi_updateservice.log or updateserv.log when the update is launched manually
   ../iQSuite/SASI/ntk_sasi.dll.update.log (only temporary, is always deleted again) 
   
   The entries discussed under this point refer exclusively to the file sasi_updateservice.log or updateserv.log. Other log files are dealt with below. 
   
   The following entries are important once a successful connection to the download server has been established:
   08:07:40.953 | [cURL] CURLINFO_RESPONSE_CODE=(200)
   08:07:40.953 | [cURL] Status Code is 'Successful 2xx' (200)! 
   
   You can trace its progress as the required files are downloaded:
   08:07:40.953 | [PROC] Processing contents-file...
   08:07:40.953 | [PROC] Processing contents logic... !
   08:07:40.953 | [PROC] Processing contents logic finised !
   08:07:40.953 | [PROC] Processing contents-file finished !
   08:07:40.953 | Downloading file - 'http://httpupdate.group-technologies.com/sasi/win32/antispam-2008.8.08.151007-MSWin32-x86.zip' ...
   08:08:01.781 | Downloading Hash-File 'http://httpupdate.group-technologies.com/sasi/win32/antispam-2008.8.08.151007-MSWin32-x86.zip.md5' ...
   08:08:01.781 | Downloading file - 'http://httpupdate.group-technologies.com/sasi/win32/antispam-2008.8.08.151007-MSWin32-x86.zip.md5' ...
   
   If there are no new pattern files, no download takes place:
   10:29:43.515 | File 'antispam-2008.8.08.71618-MSWin32-x86.zip' does not need to be updated ! (time-diff-ratio=-11436.00) Local timestamp='Wed Aug 08 07:18:00 2008
   ' / Remote timestamp='Wed Aug 08 07:18:00 2008
   '
   10:29:43.515 | [PROC] Processing contents logic finised !
   10:29:43.515 | [PROC] Processing contents-file finished !
   10:29:43.515 | No files downloaded ! Problem might be one of the following:
   * No matching patterns in contents file
   * Invalid URL
   * Invalid proxy settings 
   
   The "problem" in this case is "No matching patterns in contents file". No suitable (new) pattern files were found for download. 
   
   The files that reside locally are verified, unzipped and reorganised. The current files are now located in the directory ..iQSuiteSASIUpdateExtract:
   08:08:01.828 | [PROC] Verifying file 'antispam-2008.8.08.151007-MSWin32-x86.zip' against 'antispam-2008.8.08.151007-MSWin32-x86.zip.md5' ...
   08:08:01.937 | [PROC] Verifying file 'antispam-2008.8.08.151007-MSWin32-x86.zip' successful !
   08:08:01.937 | Opening archive file - 'C:/Lotus/Domino/iQSuite/SASI/Update/temp/antispam-2008.8.08.151007-MSWin32-x86.zip' ...
   08:08:01.937 | Extracting file contents 'C:/Lotus/Domino/iQSuite/SASI/Update/temp/antispam-2008.8.08.151007-MSWin32-x86.zip' to 'C:/Lotus/Domino/iQSuite/SASI/Update/Extract' ...
   08:08:01.937 | ->extracting: 'C:/Lotus/Domino/iQSuite/SASI/Update/Extract/MANIFEST' ...
   08:08:01.937 | ->extracting: ''C:/Lotus/Domino/iQSuite/SASI/Update/Extract/pmx_engine.dll' ...
   08:08:01.984 | ->extracting: ''C:/Lotus/Domino/iQSuite/SASI/Update/Extract/asdb.antispam' ...
   08:08:02.234 | ->extracting: ''C:/Lotus/Domino/iQSuite/SASI/Update/Extract/db.summary' ...
   08:08:02.265 | File 'C:/Lotus/Domino/iQSuite/SASI/Update/temp/antispam-2008.8.08.151007-MSWin32-x86.zip' extracted successfully .
   08:08:02.265 | Removing old files from working directory - 'C:/Lotus/Domino/iQSuite/SASI/Update/temp' 
   
   You will find the following message if a connection could not be established to the download server:
   08:08:17.593 | Error performing lib cURL operation (code 7)='couldn't connect to server' 
   
   In this case check the following: 
   a) File ../iQSuite/SASI/Update/settings.xml: Is the correct download server entered in the file?
   Correct is:
   <Url>http://httpupdate.group-technologies.com</Url> oder
   <Url>ftp://ftpupdate.group-technologies.com</Url> 
   
   b) Can you access the above server from the iQ.Suite server per http or ftp? 
   
   c) File ../iQSuite/SASI/Update/settings.xml: Does the service require a proxy to access the above server? If a proxy is not needed, the following setting must be made <Proxy enabled="false">.
   Example: 
   <Proxy enabled="true">
   <Url>192.168.1.11</Url>
   <Port>8080</Port>
   <Username>domainuser</Username>
   <Password>geheim</Password> 
   
   d) Is DNS configured correctly? Can the DNS name of the download server (httpupdate.group-technologies.com or ftpupdate.group-technologies.com) be resolved into an IP address? Is Port 53 TCP outbound activated for the server?
   
   As a test, you can input the URL "ftp://ftpupdate.group-technologies.com" in your browser. Authenticate yourself with the user name "sasi" and the password "groupsasi".
2. Copy the new patterns to the SASI working directory
   The second phase involves updating the SASI patterns (asdb.antispam, db.summary) and the program (pmx_engine.dll) via a GROUP sandbox implementation. This implementation is configured so that it searches for new files under ../iQSuite/SASI/Update/Extract. As soon as files are found there (new file versions are available), the sandbox implementation transfers all the necessary files to the SASI directory (../iQSuite/SASI) that is used by a standard SASI job.
   In order to execute the sandbox implementation, activate the preconfigured (custom) SASI standard job. The sandbox tries to update existing pattern and program files each hour or each time a job is initialised. This process is logged in the file ../iQSuite/SASI/ntk_sasi_server.log. In between we have the SMTP dialog, which sends the email that informs us whether an update was or was not successful - depending on what is configured under "email options" in the control file settings.xml. It is important to know that this section has run successfully - the notification emails are intended for information purposes and should not just disappear.
   
   08/13/08 11:16:50 (0552/3320): Need to copy new file C:/Lotus/Domino/iQSuite/sasi/./Update/Extract/asdb.antispam to C:/Lotus/Domino/iQSuite/sasi/asdb.antispam
   08/13/08 11:16:50 (0552/3320): Need to copy new file C:/Lotus/Domino/iQSuite/sasi/./Update/Extract/db.summary to C:/Lotus/Domino/iQSuite/sasi/db.summary
   08/13/08 11:16:50 (0552/3320): Need to copy new file C:/Lotus/Domino/iQSuite/sasi/./Update/Extract/pmx_engine.dll to C:/Lotus/Domino/iQSuite/sasi/pmx_engine.dll
   08/13/08 11:16:50 (0552/3320): ---------- Initial Run ----------
   08/13/08 11:16:50 (0552/3320): Copy new file C:/Lotus/Domino/iQSuite/sasi/./Update/Extract/asdb.antispam to C:/Lotus/Domino/iQSuite/sasi/update_tmp/asdb.antispam ...
   08/13/08 11:16:50 (0552/3320): ... done
   08/13/08 11:16:50 (0552/3320): Copy new file C:/Lotus/Domino/iQSuite/sasi/./Update/Extract/db.summary to C:/Lotus/Domino/iQSuite/sasi/update_tmp/db.summary ...
   08/13/08 11:16:50 (0552/3320): ... done
   08/13/08 11:16:50 (0552/3320): Copy new file C:/Lotus/Domino/iQSuite/sasi/./Update/Extract/pmx_engine.dll to C:/Lotus/Domino/iQSuite/sasi/update_tmp/pmx_engine.dll ...
   08/13/08 11:16:50 (0552/3320): 11:16:40.671 | [cURL] CURLINFO_RESPONSE_CODE=(200)
   08/13/08 11:16:50 (0552/3320): 11:16:40.890 | [cURL] Status Code is 'Successful 2xx' (200)!
   08/13/08 11:16:50 (0552/3320): 11:16:40.890 | [PROC] Processing contents-file...
   08/13/08 11:16:50 (0552/3320): 11:16:40.890 | [PROC] Processing contents logic... !
   08/13/08 11:16:50 (0552/3320): 11:16:40.890 | [Local-time=1218608400] [Server-time=1218618926] [time-diff-ratio=-10526.00]
   08/13/08 11:16:50 (0552/3320): 11:16:40.890 | File 'antispam-2008.8.13.81921-MSWin32-x86.zip' does not need to be updated ! (time-diff-ratio=-10526.00) Local timestamp='Wed Aug 13 08:20:00 2008
   08/13/08 11:16:50 (0552/3320): ' / Remote timestamp='Wed Aug 13 08:20:00 2008
   08/13/08 11:16:50 (0552/3320): '
   08/13/08 11:16:50 (0552/3320): 11:16:40.890 | [PROC] Processing contents logic finised !
   08/13/08 11:16:50 (0552/3320): 11:16:40.890 | [PROC] Processing contents-file finished !
   08/13/08 11:16:50 (0552/3320): 11:16:40.890 | No files downloaded ! Problem might be one of the following: 08/13/08 11:16:50 (0552/3320): * No matching patterns in contents file 08/13/08 11:16:50 (0552/3320): * Invalid URL 08/13/08 11:16:50 (0552/3320): * Invalid proxy settings
   08/13/08 11:16:50 (0552/3320): 11:16:40.890 | Removing old files from working directory - 'C:/Lotus/Domino/iQSuite/SASI/Update/temp'
   08/13/08 11:16:50 (0552/3320): 11:16:40.890 | Sending [Info] mail...
   08/13/08 11:16:50 (0552/3320): 11:16:40.890 | [tcp] connecting to 'iqsuite.training.local' ...
   08/13/08 11:16:50 (0552/3320): 11:16:40.890 | [smtp] waiting for 'Ok' status ...
   08/13/08 11:16:50 (0552/3320): 11:16:40.890 | SMTP-Response: 220 iqsuite.training.local ESMTP Service (Lotus Domino Release 7.0.1) ready at Wed, 13 Aug 2008 11:16:40 +0200
   08/13/08 11:16:50 (0552/3320): 11:16:40.890 | [smtp] sending 'EHLO' ...
   08/13/08 11:16:50 (0552/3320): 11:16:40.890 | SMTP-Response: 250-iqsuite.training.local Hello iqsuite.training.local ([127.0.0.1]), pleased to meet you
   08/13/08 11:16:50 (0552/3320): 11:16:40.890 | SMTP-Response: 250-HELP
   08/13/08 11:16:50 (0552/3320): 11:16:40.890 | SMTP-Response: 250-VRFY
   08/13/08 11:16:50 (0552/3320): 11:16:40.890 | SMTP-Response: 250-SIZE
   08/13/08 11:16:50 (0552/3320): 11:16:40.890 | SMTP-Response: 250 PIPELINING
   08/13/08 11:16:50 (0552/3320): ... done
   08/13/08 11:16:50 (0552/3320): ---------- Waiting 10 seconds ----------
   08/13/08 11:16:50 (0552/3320): 11:16:45.890 | tcp_getline(): No data.
   08/13/08 11:16:50 (0552/3320): 11:16:45.890 | [smtp] err=(status(-1)!=250)
   08/13/08 11:16:50 (0552/3320): 11:16:45.890 | [smtp] sending 'mail from' ...
   08/13/08 11:16:50 (0552/3320): 11:16:45.890 | SMTP-Response: 250 sasi-info@training.local... Sender OK
   08/13/08 11:16:50 (0552/3320): 11:16:45.890 | [smtp] sending 'rcpt to' ...
   08/13/08 11:16:50 (0552/3320): 11:16:45.890 | SMTP-Response: 250 iqsuite-admin@training.local... Recipient OK
   08/13/08 11:16:50 (0552/3320): 11:16:45.890 | [smtp] sending 'data' ...
   08/13/08 11:16:50 (0552/3320): 11:16:45.890 | SMTP-Response: 354 Enter message, end with "." on a line by itself
   08/13/08 11:16:50 (0552/3320): 11:16:45.906 | SMTP-Response: 250 Message accepted for delivery
   08/13/08 11:16:50 (0552/3320): 11:16:45.906 | [smtp] sending 'quit' ...
   08/13/08 11:16:50 (0552/3320): 11:16:45.906 | SMTP-Response: 221 iqsuite.training.local SMTP Service closing transmission channel
   08/13/08 11:16:50 (0552/3320): 11:16:45.906 | Mail sent successfully to 'iqsuite-admin@training.local' .
   08/13/08 11:16:50 (0552/3320): 11:16:45.906 |
   08/13/08 11:16:50 (0552/3320): _Application-ExitCode=0
   08/13/08 11:16:50 (0552/3320): ---------- Run 1 ----------
   08/13/08 11:16:50 (0552/3320): File set is stable.
   08/13/08 11:16:50 (0552/3320): ---------- Final Move ----------
   08/13/08 11:16:50 (0552/3320): Move file C:/Lotus/Domino/iQSuite/sasi/update_tmp/pmx_engine.dll to C:/Lotus/Domino/iQSuite/sasi/pmx_engine.dll ...
   08/13/08 11:16:50 (0552/3320): ... done
   08/13/08 11:16:50 (0552/3320): Move file C:/Lotus/Domino/iQSuite/sasi/update_tmp/db.summary to C:/Lotus/Domino/iQSuite/sasi/db.summary ...
   08/13/08 11:16:50 (0552/3320): ... done
   08/13/08 11:16:50 (0552/3320): Move file C:/Lotus/Domino/iQSuite/sasi/update_tmp/asdb.antispam to C:/Lotus/Domino/iQSuite/sasi/asdb.antispam ...
   08/13/08 11:16:50 (0552/3320): ... done
   08/13/08 11:16:50 (0552/3320): Finished update
   08/13/08 11:16:50 (0552/3320): Timestamp 2008-08-13T09:16:40 written to C:/Lotus/Domino/iQSuite/sasi/ntk_sasi_ref.cfg.timestamp
   08/13/08 11:16:50 (0552/3320): ---- done
   08/13/08 11:16:50 (0552/3320): Result: 0

Invocation of the interface by iQ.Suite

If the update was successful or when the required files are located in the SASI directory, iQ.Suite can access the interface.
This process is implemented via the SOAP protocol in the form of a client-server connection, where iQ.Suite is the SOAP client (soap.ntk_sasi.dll) and the SASI interface (ntk_sasi.dll.exe) the server.

Since the SOAP server is not always running, the client needs to be able to launch it when necessary. However, first the client tries to connect directly to the server. If the server is not up and running, this first connection attempt fails. This fact is recorded in the client log (../iQSuite/SASI/ntk_sasi_client.log):
"No connection could be made because the target machine actively refused it."
Detail: connect failed in tcp_connect()

Only now does the client launch the SOAP server, see ntk_sasi_client.log:
08/08/08 08:08:08 (1160/3288): trying to start C:/Lotus/Domino/iQSuite/sasintk_sasi.dll.exe C:/Lotus/Domino/iQSuite/sasi/ntk_sasi.dll ...
08/08/08 08:08:09 (1160/3288): started talking to C:/Lotus/Domino/iQSuite/sasi/ntk_sasi.dll.exe pid 1436

The connection is now established and emails are transferred over the interface to the SASI engine (pmx_engine.dll), which uses the pattern files (asdb.antispam and db.summary) to check the message for the likelihood of spam.

Apart from logging the local update, the connection attempts made by the SOAP client(s) are listed in the file ntk_sasi_server.log:
08/13/08 11:16:50 (0552/3320): OK (providing GAPI interface)
08/13/08 11:16:51 (0552/3444): Sandbox: open request from client "3832-240311564-003462D0"
08/13/08 11:16:51 (0552/3444): Sandbox: now 1 session(s) open
08/13/08 11:16:51 (0552/3244): GAPI: InitializeDLL requested
08/13/08 11:16:51 (0552/3244): GAPI: InitializeDLL finished
08/13/08 11:16:51 (0552/3444): GAPI: InitializeSession requested
08/13/08 11:16:51 (0552/3444): GAPI: InitializeSession succeeded 

The server terminates on expiry of a defined timeout if the client does not maintain the connection. It is launched again by the client’s next connection attempt.
Error 28 fault: SOAP-ENV:Server [no subcode]
"Timeout"
Detail: accept failed in soap_accept()
Error 28 fault: SOAP-ENV:Server [no subcode]
"Timeout"
Detail: accept failed in soap_accept()
Error 28 fault: SOAP-ENV:Server [no subcode]
"Timeout"
Detail: accept failed in soap_accept()
08/13/08 12:58:26 (4088/3448): going to terminate because idle timeout exceeded
08/13/08 12:58:26 (4088/3448): C:/Lotus/Domino/iQSuite/sasi/ntk_sasi.dll.exe - Waiting for threads to terminate
08/13/08 12:58:27 (4088/3448): C:/Lotus/Domino/iQSuite/sasi/ntk_sasi.dll.exe - Going to terminate socket connection
08/13/08 12:58:27 (4088/3448): C:/Lotus/Domino/iQSuite/sasi/ntk_sasi.dll.exe - SOAP service terminated 

Combating spam is still one of the administrator’s top priorities. Once the filter has been configured, it does not usually take long for spammers to think up a new strategy that circumvents the current detection mechanisms. And again the lines are blocked with calls from dissatisfied users complaining about the deluge of spam mails in their mailboxes.

At our training courses we develop a multiphase concept that helps you combat spam successfully in the long term without the need to make permanent readjustments.

Come and see. We look forward to your visit!