Cluster Transfer

SMTP Relaying
Local Delivery
Backend Queues
Remote Queue Processing

This section explains how Message Transfer operations work in the CommuniGate Pro Cluster environment.

SMTP Relaying

Incoming SMTP connections are received with a TCP Load Balancer and sent to Cluster Frontends. A Frontend Server receives a message in the same way as it does in the single-server mode, but it may contact the Backend Servers (via CLI) when it needs to:

authenticate the sender
verify the sender and recipient addresses
check the recipient status
retrieve the sender limits

Received messages are enqueued. If a message is directed to an external address, it can be relayed by the same Frontend:

Local Delivery

A message directed to a local recipient can be enqueued on a "wrong" Server, i.e. on a Server that cannot open the target Account and deliver the message to that Account.

This situation can happen when a message is enqueued on a Frontend Server (Frontend Servers cannot directly open any Account in Shared Domains), or a message is enqueued on a Backend Server that either does not host the target Account (in a Static Cluster), or it cannot open it, because the Account is opened by some other Backend Server (in a Dynamic Cluster).

To solve the problem, the Local Delivery module uses a Delivery channel connection to the proper Backend Server and passes the message there. The receiving Backend immediately opens the target Account, applies its Account-Level Rule and stores the transferred message. This Backend does not enqueue the message.

If there is a temporary problem or a delivery failure, the receiving Backend Server reports the error back, and the message is either delayed in Queue or it is removed from the Queue (with error report messages generated).

Backend Queues

WebUser Interface sessions, Rules, MAPI sessions, and other modules and components can generated E-mail messages on Backend Servers.

Backend Server often do not have direct access to the Internet, in this case they cannot deliver generated messages to remote systems. To solve this problem, the Backend Servers can be configured to relay all messages to the Frontend Servers first, using the * symbol as the SMTP Forwarding Server name.

In this case the message is submitted to the Backend Queue, where it is processed using the Server-wide and Cluster-wide Rules, and if it is not directed to a local recipient, it is directed to the SMTP module which sends it to one of the Frontend Servers:

In this setup, each messages generated on a Backend Server is processed twice. If Cluster Rules invoke content management Plugins, double-processing can create a substantial overhead. To avoid this overhead, the Remote Queue processing method can be used.

Remote Queue Processing

Most of the Queue processing takes place on the Frontend Servers. Frontend Servers accept incoming SMTP messages and either relay them to remote locations or deliver them to local Accounts via Backends, using the special inter-cluster protocol, without placing messages into the Backend Server Queue.

A certain amount of messages can be generated directly on the Backend Servers. They include messages:

created with Account-level Rules;
composed within WebUser sessions;
retrieved using the RPOP module;
submitted using the MAPI connector and XTND XMIT POP3 mechanism;
submitted using the PIPE module.

You may want to avoid processing Message Queues on Backends for various reasons, including:

lack of Internet connectivity for Backends;
requirement to use anti-spam and/or anti-virus Plugins installed on Frontends only;
shortage of processing power and/or disk space on Backend Servers.

You may also want to process Message Queues on some Frontends only.

To specify Queue processing options, open the Settings WebAdmin realm and select the Cluster link on the General page. Find the Queue Processing panel:

Submit Messages

This setting specifies how the Queue should be processed by this Cluster member.

If the Locally option is selected, messages will be processed in the "regular" way: when a message is composed in a temporary file, it is submitted into the Queue maintained by this Server.

If the Locally for Others option is selected, messages will be processed in the "regular" way, too. In addition, this Server will accept messages generated on other Cluster members and submit them into its own Queue - for processing and delivery. The Dynamic Cluster Controller collects and distributes information about all running Cluster members that have this option selected.

If the Remotely option is selected, this Cluster member will try to connect to any other Cluster member that has the "Locally for Others" option selected. The temporary file content (the message envelope and the message itself) is sent to that other Cluster member via a special protocol that uses the SMTP port. If the remote message submission does not work (the Server failed to connect to the Cluster member(s) or the message file transfer fails), the message is submitted to the Server own Queue to ensure that no message is lost: