PBS Email Service

Send and Receive Email from Delphi Servers

Sooner or later every Delphi server has to send email - password resets, confirmations, reports with a PDF attached - and often read it too. The usual answer is to drop a TIdSMTP into the server. It works, until it doesn't: OpenSSL DLLs to keep with every exe, a slow mail host holding up a request, a failed send lost because nobody retried it, and every application that sends mail carrying its own copy of the same code.

PBS Email Service takes a different approach. One Windows service is the only mail client in the system. Everything else writes a small text file into a queue folder, and reads received mail as plain files from another. This article explains the design and what the service does, and the full source is free to download.

Why Keep Email Out of Your Server

• No Indy or OpenSSL in your applications. Only the service needs the DLLs, so there is one place to update them.
• Queueing never waits on the mail host. Writing a file takes milliseconds; the service deals with slow servers, timeouts and retries.
• Nothing is lost. A message is a file until it is sent or set aside with the reason, so a restart, a network outage or a full mailbox at the other end loses nothing.
• Any language can send mail. A PHP site, a batch file, a watchdog or a backup script writes the same file as your Delphi server.
• One log and one journal of everything sent, however many applications send it.
• The mail side can be stopped, updated or moved to another machine without touching your servers.

The Queue File

To send an email, write one UTF-8 text file into the To_Send folder:

TO=recipient@example.com, second@example.com
FROMNAME=Your Company
SUBJECT=Booking 1234 confirmed
HTML=1
NOREPLY=1
PRIORITY=1
ATTACH=C:\Data\pdf\booking_1234.pdf
ATTACH=C:\Data\pdf\terms.pdf
BODY=
<p>Everything after BODY= is the body.</p>

Only TO and SUBJECT are required. TO takes several addresses separated by commas or semicolons. NOREPLY sets Reply-To to noreply@ your domain, PRIORITY sends it at highest priority, and ATTACH may repeat for as many files as you like - they are read when the message is sent and never deleted.

The one rule that matters: write the file as .tmp and rename it to .eml when it is complete. A rename is atomic, so the service never picks up a half-written file. Name it yyyymmdd_hhnnss_<GUID>.eml; the timestamp is how the service judges its age.

A file that is not valid UTF-8 is read in the ANSI code page instead, so a writer that got the encoding wrong still sends.

Queueing from Delphi

The download includes Email.Send.pas, which writes the file for you. It has no Indy and no other dependencies, so it can go into any application. Set it up once:

uses
  Email.Send;

TEmail_Send.Company_Name   := 'Your Company';
TEmail_Send.Company_Colour := '#0066cc';
TEmail_Send.Path_To_Queue  := 'C:\Mail\Emails\To_Send\';

then queue from any thread:

if not TEmail_Send.Queue_Email ('user@example.com',
                                'Your booking',
                                LHtml_Body,
                                True,          { HTML }
                                True,          { no-reply }
                                False,         { high priority }
                                [LPdf_File])   { attachments }
then Log (TEmail_Send.Last_Error);

Company_Colour is there for your own HTML templates: inherit from TEmail_Send and add methods such as Queue_Password_Reset that build a branded body and call Queue_Email.

Retrying Properly: 4xx and 5xx

Most send-and-forget code treats every failure the same way. SMTP does not. A reply from 500 to 599 is permanent - the address does not exist, the message was rejected - and retrying will never help. A 4xx reply is temporary - greylisting, a full mailbox, a busy server - and the same message will usually go through a minute later. A timeout, socket or TLS error has no reply code at all and is also worth retrying.

Indy raises EIdSMTPReplyError with the reply code in ErrorCode, so the distinction is a few lines:

except
  on E : EIdSMTPReplyError
  do begin
       Result        := E.Message;
       AIs_Permanent := (E.ErrorCode >= 500) and (E.ErrorCode <= 599);
     end;
  on E : Exception
  do Result := E.Message;   { timeouts, sockets, TLS: try again }
end;

What the service does with each outcome:

Two details stop a bad day from becoming a worse one. Each pass takes new files first, oldest first, and only then files whose backoff has expired, so a pile of failing mail never holds up today's password reset. And after any pass that set files aside, one summary email listing them goes to an administrator address - through the same queue, so it follows the same rules.

Sending over TLS 1.2 with Indy

Most mail hosts now want port 587 with STARTTLS (explicit TLS), and many cloud hosts block port 25 outbound altogether. The Indy setup is short once you know it:

LSSL.SSLOptions.Method      := sslvTLSv1_2;
LSSL.SSLOptions.Mode        := sslmClient;
LSSL.SSLOptions.SSLVersions := [sslvTLSv1_2];

LSMTP.Host      := 'mail.yourdomain.com';
LSMTP.Port      := 587;
LSMTP.IOHandler := LSSL;
LSMTP.UseTLS    := utUseExplicitTLS;

LMsg.CharSet    := 'UTF-8';

With attachments the message goes as multipart/mixed: the body as the first part, plain text or HTML, then each file as its own part under its own name. POP3 on port 995 uses utUseImplicitTLS with the same TLS 1.2 options.

"Could not load SSL library"

The error everyone meets once. Delphi's standard TIdSSLIOHandlerSocketOpenSSL loads the OpenSSL 1.0.2 DLLs, libeay32.dll and ssleay32.dll. They are not part of Windows and not part of Delphi, and they must match the exe: a 64-bit exe needs the 64-bit DLLs, a 32-bit exe the 32-bit ones. The mismatch is the usual cause, since both sets have the same names. Put the right pair beside the exe.

The service checks for the DLLs when it starts and stops with a message naming what is missing, instead of failing on the first email an hour later. With a mail service that is one folder to get right, not one per application.

Testing Without Emailing Real People

A test system running on a copy of live data has real customers' addresses in it. Set Redirect_Domain and every recipient ending in it goes to one address of yours instead, with the original recipients in square brackets at the front of the subject so you can see who would have received it. Leave Redirect_To blank and those recipients are simply dropped.

A Journal of Everything Sent

Each send outcome - Sent, Retry or Failed - is one JSON line in a daily journal file, with the time in UTC, the recipients, the subject and the reason. Put a tag such as [TEST] or [TRAINING] in the subject and that mail goes into its own journal folder, so live, test and training systems can share one service and still be reported apart. It is easy to show on a web page as a list of emails sent. The journal, Sent and Send_Errors folders tidy themselves after a set number of days.

Receiving Mail as Plain Files

Every minute, or whatever interval you set, the service collects the POP3 mailbox and writes each message to disk as a set of files sharing one name:

Your application needs no mail library to use them: wait for the .eml, read the .txt, open the attachments, and move the set away once processed. That is how a server turns an emailed booking form or invoice into a database row without ever touching POP3.

Two things make the metadata more useful than the raw headers:

• Forwarded mail is unwrapped. When someone forwards a message on, the sender, subject, date and Cc come from the original, the original recipient is kept as SENT_TO, and the forwarding headers are stripped from the body.
• The envelope recipient is kept. Point several aliases - bookings@, accounts@, support@ - at one mailbox and your application can still tell them apart from TO.

For testing, every received file can also be copied to a second folder so a test system processes exactly the same mail as live. Messages can be left in the mailbox rather than deleted, and a replay mode feeds a saved .eml through the parser instead of the mailbox, so a message that was handled wrongly can be run again until it is right. An optional .dec file and the included Test_Email_Parser show exactly how a message was put together.

Knowing It Is Working

A mail service that has quietly stopped is worse than none. So:

• Daily log files, kept for a set number of days, with every line marked [SEND], [RECV] or [SVC]. Every send failure is one line naming the file, its subject and the actual error, ending will retry or Moved to Send_Errors.
• Each thread logs an "alive" heartbeat every hour, so a silent log means a stopped thread, not a quiet day.
• The settings, without passwords, are logged at every start.
• An optional JSON status file, rewritten regularly and soon after any error, so a watchdog or status page can check the service by the file's age without reading logs.
• A crash that escapes everything, or a failure during startup before the log exists, goes to the Windows Application event log.

What Is in the Download

Full source for all four, Email.Send.pas for your applications, the shared library units they use, a sample email.ini, and PBS_Email_Service.md documenting every setting, the queue and received file formats and troubleshooting. The sender and receiver can each be turned off in the ini, for example to run a send-only service.

Requirements

• Delphi 12 or 13, with the Indy components that ship with it
• The OpenSSL 1.0.2 DLLs, libeay32.dll and ssleay32.dll, for the exe's platform
• A mailbox with SMTP and, for receiving, POP3 access

Installing

1. Build the projects, keeping the folder layout of the download.
2. Copy PBS_Email_Service.exe, email.ini and the two OpenSSL DLLs to a folder.
3. Fill in the mail host, logins and folders in email.ini, and restrict access to it - it holds the passwords.
4. Run Email_Test.exe there to prove the SMTP login.
5. From an administrator prompt run PBS_Email_Service.exe /install, then start "PBS Email Service".

If you found this useful, then please consider chipping in.
Chip In with

Please chip in if helpful