Smile Internet Status & News

Smile experienced an issue with Real Time Blacklists which caused a delay in incoming mail. While some mail came through in a timely manner, many emails were delayed. The servers are now churning through the queue and should have all mail delivered shortly.

Some mail may be delayed further, depending on the sending mail server. You may also see a higher incidence of spam temporarily as we track down what RBL caused the problem and put the others in place.

We should have more posted shortly.

Smile Internet, like most mail providers, rely on a group of organizations that monitor spammers and maintain a list of bad addresses that spammers send spam through. One of these organizations was dsbl.org which recently discontinued service. Today, (March 11) they took their servers offline. This caused a problem for the Smile mail servers, where requests to the blacklist piled up, and the servers had trouble responding to new incoming mail.

The RBL requests eventually timed out, allowing some mail to come through, but the backup of incoming mail quickly piled up requests again. We were unlucky in troubleshooting the problem, because our test emails came in in a reasonable amount of time, which sent us searching in the wrong direction. After combing logs, restarting services, and some specific error reports from customers, we isolated the issue and removed all of the blacklists. Because of the specific way this issue occurred, our alerting system yep

We then checked the blacklist services and found that dsbl.org had stopped service, and we re-enabled the other services.

Currently the servers are working through a very large queue of mail that is coming in. Delayed mail should continue into the evening. We are monitoring the problem, and enhancing our alerting to provide us with better information on a queue issue, so if another blacklist goes down, we can quickly remedy the problem, and keep service at 100%.