How to Verify STARTTLS Encryption for Sendmail Email Transmissions: A Comprehensive Guide #

Securing email communications is paramount, especially when dealing with sensitive data. If you’re using Sendmail on your VPS (like a Debian server) to send notifications, enabling STARTTLS is crucial for encrypting the email transmission between your server and the recipient’s mail server. This guide, brought to you by revWhiteShadow, a personal blog site by revWhiteShadow and kts, delves into verifying STARTTLS usage within your Sendmail configuration.

Understanding STARTTLS and Its Importance #

STARTTLS (Opportunistic TLS) is a protocol command that tells an email server that the client wants to upgrade to a secure TLS (Transport Layer Security) connection. This encryption method protects the email content and headers from eavesdropping during transit. Without STARTTLS, your email communication is sent in plain text, making it vulnerable to interception.

Configuring Sendmail for STARTTLS: A Detailed Walkthrough #

Before diving into verification, let’s ensure Sendmail is correctly configured to use STARTTLS. We’ll expand on the basic steps you’ve already taken to provide a more robust and error-resistant approach.

Modifying the Sendmail Configuration Files #

1. Accessing the Configuration Files: Log in to your VPS as a user with sufficient privileges (usually root or a user with sudo access).

2. Editing sendmail.mc and submit.mc: Open the sendmail.mc and optionally the submit.mc file using your favorite text editor (e.g., nano, vim). These files are typically located in /etc/mail/.

   sudo nano /etc/mail/sendmail.mc
   sudo nano /etc/mail/submit.mc
   

3. Adding the STARTTLS Inclusion: Add the following line to both files, ensuring it’s placed before the MAILER definitions:

   include(`/etc/mail/tls/starttls.m4')dnl
   

   This line includes the STARTTLS configuration macro.

4. Optional: Configuring Certificate Verification (Recommended): To further enhance security, configure certificate verification. This requires having valid TLS certificates installed. Add the following lines before the MAILER definitions:

   define(`confCACERT_PATH', `/etc/ssl/certs')dnl
   define(`confCACERT_FILE', `/etc/ssl/certs/ca-certificates.crt')dnl
   define(`confSERVER_CERT', `/etc/ssl/certs/your_domain.crt')dnl
   define(`confSERVER_KEY', `/etc/ssl/private/your_domain.key')dnl
   

   Important: Replace /etc/ssl/certs/your_domain.crt and /etc/ssl/private/your_domain.key with the actual paths to your certificate and private key files. These paths depend on how you obtained and installed your TLS certificates (e.g., Let’s Encrypt, self-signed, etc.).

5. Optional: Enforcing TLS for Specific Domains: You can enforce TLS for specific recipient domains, ensuring that emails to those domains are only sent if a TLS connection can be established. Add the following line, replacing example.com with the desired domain:

   FEATURE(`enforce_tls', `example.com')dnl
   

6. Saving the Changes: Save the modified files in your text editor.

Rebuilding the Sendmail Configuration #

1. Generating the sendmail.cf File: Use the sendmailconfig command or the m4 macro processor to generate the sendmail.cf file from the sendmail.mc file. We recommend using m4 directly for greater control:

   sudo m4 /etc/mail/sendmail.mc > /etc/mail/sendmail.cf
   sudo m4 /etc/mail/submit.mc > /etc/mail/submit.cf
   

2. Adjusting Permissions (If Necessary): Ensure the sendmail.cf file has the correct permissions:

   sudo chown root:smmsp /etc/mail/sendmail.cf
   sudo chmod 640 /etc/mail/sendmail.cf
   sudo chown root:smmsp /etc/mail/submit.cf
   sudo chmod 640 /etc/mail/submit.cf
   

Restarting Sendmail #

1. Restarting the Sendmail Service: Restart the Sendmail service to apply the changes:

   sudo systemctl restart sendmail
   

Verifying STARTTLS Usage: Comprehensive Methods #

Now that Sendmail is configured, let’s verify that STARTTLS is actually being used. We’ll cover several methods, including examining email headers, using netcat, and analyzing verbose output.

Analyzing Email Headers #

Examining the email headers is a reliable way to determine if STARTTLS was used during transmission.

1. Sending a Test Email: Send a test email from your Sendmail server to an external email address (e.g., your Outlook address).

2. Viewing the Email Headers: Open the email in your email client (e.g., Outlook, Gmail, Thunderbird). Look for an option to “View Source,” “View Headers,” or similar. The location of this option varies depending on your email client.

3. Identifying STARTTLS Indicators: Look for the following indicators in the headers:

   • Received Header with TLS Information: A Received: header that includes ESMTPS (Explicit SMTP over TLS) and details about the TLS version and cipher suite. For example:

     Received: from mail.YOURDOMAIN.com (localhost [127.0.0.1])
         by mail.YOURDOMAIN.com (8.15.2/8.15.2/Debian-14~deb10u1) with ESMTPS id 054G4khN002213
         (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT)
         for <RECIPIENT@example.com>; Thu, 4 Jun 2020 16:04:46 GMT
     

     The presence of ESMTPS and the version=, cipher=, and bits= attributes strongly indicate that STARTTLS was used. The verify=NOT indicates that the certificate was not verified. Ideally, this should be verify=OK if you configured certificate verification.

   • X-TLS-Cipher and X-TLS-Version Headers: Some email servers may add specific headers indicating the TLS cipher and version used:

     X-TLS-Cipher: TLS_AES_256_GCM_SHA384
     X-TLS-Version: TLSv1.3
     

   • Absence of Cleartext Keywords: Ensure there are no headers indicating the email was processed or relayed without encryption (e.g., headers containing phrases like “sent in cleartext”).

Using netcat to Simulate an SMTP Conversation #

The netcat utility allows you to manually interact with an SMTP server and observe the STARTTLS negotiation. This method is useful for troubleshooting and understanding the underlying protocol exchange.

1. Initiating a netcat Connection: Open a terminal and use netcat to connect to your Sendmail server on port 25 (SMTP). Replace mail.YOURDOMAIN.com with your server’s hostname or IP address:

   netcat -Cw 60 mail.YOURDOMAIN.com 25
   

2. Issuing SMTP Commands: Type the following commands, pressing Enter after each one:

   • EHLO your_client_hostname (Replace your_client_hostname with a suitable hostname, such as your server’s hostname or localhost.)
   • Examine the response. Look for the 250-STARTTLS line. If it’s present, the server supports STARTTLS.
   • STARTTLS
   • The server should respond with 220 2.0.0 Ready to start TLS.
   • EHLO your_client_hostname (Issue the EHLO command again after the STARTTLS command.)
   • Examine the response again. This time, you should see the supported authentication mechanisms (e.g., 250-AUTH PLAIN LOGIN) after the TLS handshake.

3. Example Conversation:

   220 mail.YOURDOMAIN.com ESMTP Sendmail 8.15.2/8.15.2/Debian-14~deb10u1; Fri, 5 Jun 2020 14:40:15 GMT; (No UCE/UBE) logging access from: your_client_hostname(OK)-your_client_hostname [YOUR_IP_ADDRESS]
   EHLO your_client_hostname
   250-mail.YOURDOMAIN.com Hello your_client_hostname [YOUR_IP_ADDRESS], pleased to meet you
   250-ENHANCEDSTATUSCODES
   250-PIPELINING
   250-EXPN
   250-VERB
   250-8BITMIME
   250-SIZE
   250-DSN
   250-ETRN
   250-AUTH DIGEST-MD5 CRAM-MD5
   250-STARTTLS
   250-DELIVERBY
   250 HELP
   STARTTLS
   220 2.0.0 Ready to start TLS
   EHLO your_client_hostname
   250-mail.YOURDOMAIN.com Hello your_client_hostname [YOUR_IP_ADDRESS], pleased to meet you
   250-ENHANCEDSTATUSCODES
   250-PIPELINING
   250-EXPN
   250-VERB
   250-8BITMIME
   250-SIZE
   250-DSN
   250-ETRN
   250-AUTH DIGEST-MD5 CRAM-MD5
   250-DELIVERBY
   250 HELP
   

   If the conversation proceeds as shown, STARTTLS is working.

4. Troubleshooting: If the STARTTLS command fails or the second EHLO doesn’t show the authentication mechanisms, there’s likely a problem with your TLS configuration.

Analyzing Sendmail’s Verbose Output #

Running Sendmail in verbose mode (-v parameter) provides detailed information about the email transmission process, including the STARTTLS negotiation.

1. Sending a Test Email in Verbose Mode: Use the sendmail command with the -v option to send a test email:

   echo 'Subject: Sendmail test' | sudo sendmail -v -f sender@example.com recipient@example.com
   

   Replace sender@example.com and recipient@example.com with appropriate email addresses.

2. Examining the Output: Analyze the verbose output for the following:

   • STARTTLS Command and Response: Look for the >>> STARTTLS command and the 220 2.0.0 Ready to start TLS response. This confirms that the STARTTLS negotiation was initiated.

   • Subsequent EHLO Command: After the STARTTLS command, there should be another >>> EHLO command.

   • TLS Cipher and Version Information: Sometimes, the verbose output will include information about the TLS cipher and version used.

   • Connection to the Recipient’s Server: The output will also show the connection to the recipient’s mail server, including whether STARTTLS was used for that connection as well. Look for similar STARTTLS commands and responses in that section of the output.

Using tcpdump or Wireshark to Capture Network Traffic (Advanced) #

For a more in-depth analysis, you can use network packet capture tools like tcpdump or Wireshark to examine the raw network traffic between your Sendmail server and the recipient’s server. This method requires a good understanding of network protocols.

1. Capturing Traffic: Use tcpdump or Wireshark to capture traffic on port 25 while sending a test email.

   sudo tcpdump -i any port 25 -w capture.pcap
   

   Replace any with the appropriate network interface if needed.

2. Analyzing the Capture: Open the capture file in Wireshark and filter for SMTP traffic. Look for the STARTTLS command and the subsequent TLS handshake. You can examine the TLS packets to verify the cipher suite and TLS version being used.

Troubleshooting STARTTLS Issues #

If STARTTLS is not working as expected, consider the following troubleshooting steps:

• Certificate Issues: Ensure your TLS certificates are valid, correctly installed, and properly configured in Sendmail. Verify the permissions on the certificate and private key files.

• Firewall Rules: Make sure your firewall is not blocking port 25 or port 587 (submission port, often used with TLS).

• SELinux/AppArmor: If you’re using SELinux or AppArmor, ensure that Sendmail has the necessary permissions to access the TLS certificates and keys.

• Configuration Errors: Double-check your sendmail.mc and submit.mc files for any typos or configuration errors.

• Recipient Server Requirements: Some recipient mail servers may require specific TLS versions or cipher suites. Ensure your Sendmail configuration is compatible with the recipient server’s requirements.

• DNS Issues: Verify that your server’s hostname resolves correctly in DNS.

Conclusion #

By following this comprehensive guide, brought to you by revWhiteShadow, you can effectively configure and verify STARTTLS encryption for your Sendmail email transmissions. Regularly testing your configuration is essential to maintain secure email communications and protect sensitive data. We, at revWhiteShadow, hope this article helps you to secure your email communication.