5

I have been wanting to run my own mailserver at home for some time now. I don't necessarily absolutely need such a thing but it would be very useful to have as I am working running my own business now and really I need a more professional email than just a personal Hotmail account.

I have been working on this on and off for quite a few weekends and haven't managed to make much progress. I started by following one of Luke Smiths guides, which is available on YouTube. (Link will be attached at the end of this post.)

Here is an overview of what I have tried to do:

  • Install required software on a Raspberry Pi. Setup required port forwarding on my home network. Discovered that my ISP router prevents port 25 from being forwarded, so tried using 2525 instead. (Read online that 2525 was a common alternative port if 25 is blocked. Not sure how valid this information is.)
  • Tried moving R-Pi to be directly attached to ISP router rather than being behind another router. (pfSense box)
  • Put ISP router into modem mode, passing everything to pfSense. Enabled port forwarding and required firewall rules for traffic. (See below) Of course I put the R-Pi back on one of the pfSense interfaces when putting the ISP router into modem mode.

Here's a list of ports I am currently forwarding:

  • 25 (SMTP)
  • 143 (IMAP)
  • 465 (SMTP/S)
  • 587 (SUBMISSION)
  • 993 (IMAP/S)

Being perfectly honest about this I don't understand what all of these are for. I understand some of them are secure versions, eg: SMTP/S is the secure version of SMTP, and I understand that some are for communication with other mail servers and some are for communication with other mail clients. This is about the limit of my understanding.

Additionally I have installed postfix and dovecot. Again, I am not 100 % certain on what each of them does, I believe one is a piece of software which moves mail to/from different mail servers, and the other is a server which allows email clients to login and send/retrieve mail.

I am using Epik as my domain name registrar. I have set the MX records and ip addresses.

  • If I login to a remote machine on a different network (such as a University computer) I am able to run dig with my mail.mydomainname.com and obtain the expected ip address.

  • I can also use nmap to see that the ports listed above are open.

  • I tried running this mail diagnostics tool https://mxtoolbox.com/diagnostic.aspx. It produces a warning for the SMTP banner. I am not sure if this is a problem or not.

  • Using this tool, it does not work for mydomain.com, and only works for mail.mydomain.com. I don't know if this indicates an issue?

There are a few things which don't work.

Firstly, until today I was able to get Thunderbird to connect to my R-Pi. Now I can't. My IP address has changed, but I have updated this on Epik, so I don't understand why there is suddenly some problem with this. (Again, I have no idea what steps to take to diagnose this - no errors are produced, other than I cannot connect to download/sync mail.)

I was able to send mail from my mail server to other servers. (For example outlook or gmail.) But I was NOT able to send mail from gmail or outlook to my own mail server. I assumed this was due to not being able to forward port 25. But again I do not know for sure.

Sorry for the absolute wall of text here. I've been working on this for weeks using up most of my free time trying to get this to work and I seem to be getting no where. The frustrating thing is that I can get things like ssh to work without issue. Also the fact that I have no idea how to start with diagnostics isn't helping either.

So my question is: What tools are available to help me diagnose this issue and what further steps should I take to diagnose this?

Currently I am aware of

  • nmap to do port scans to check for a route to a port where some server software (eg postfix/dovecot) is listening
  • this online mxtoolbox thing, althought I don't fully understand what it does

Any help / advice appreciated.

It could be that Luke gave me bad advice. Perhaps I should be using another resource as my reference for how to setup a mail server instead of some guy on YouTube?

https://www.youtube.com/watch?v=3dIVesHEAzc

Edit: Further Info

I have found that if I put in the local ip address into the Thunderbird configuration, then the "Re-test" button indicates that the setup config is ok.

However if I put in mail.mydomain.com, the setup fails. Does this indicate a DNS setup issue?

Vomit IT - Chunky Mess Style
  • 40,038
  • 27
  • 84
  • 117
user3728501
  • 2,486
  • 6
  • 28
  • 46
  • 1
    Lots of ISPs and mail providers block dynamic/residential addresses https://flurdy.com/docs/postfix/#test is a good troubleshooting guide, and an *excellent* guide to setting up a mail server but.... generally, doing it at home's going to have a lot of issues cause spammers ruin everything. – Journeyman Geek May 22 '21 at 13:46

2 Answers2

5

Sending mail from your email server and receiving email to your email server are definitely not the same so that sounds correct that you could have issues with one and not the other.

You say your IP address changed and then you started having trouble receiving email so I suspect this was your public IP that your ISP assigned. If you do not have static IP addresses from the ISP which the email server sits behind, then you will have issues resolving mail.yourdomain.com until you update your public DNS records to point to the new IP address.

I would put a TTL on your mail host records of 1 hour if you deal with changing public IP addresses often. If the IP changes and you update the DNS record and there is still 23 hours before an already connected client goes to email your server, their DNS records pointing to your email server won't get the updated for 23 hours and sending email to the old IP wont work.

Consider either getting a static IP address from your ISP and reach out to them to see what all that entails, or consider using Dynamic DNS and give that domain name our for your email address domain.

Update

After further discussion and troubleshooting with the OP, it was determined the issue was related to the the pfSense firewall hairpinning not being enabled which is called NAT Reflection in the pfSense configurations.

Once the OP enabled the pfSense "NAT Reflection" feature, email was then able to be sent to the mailbox on that domain.

Supporting Resources

  • What’s The Difference Between An Incoming Mail Server And An Outgoing Mail Server?

    Most email accounts have two servers: one that lets you send emails to other people, and another that lets you receive the emails that other people send you. The server that lets you send mail is called an outgoing, or SMTP server. The server that lets you receive mail is called an incoming, POP, or just Mail server.

  • The top 25 DDNS providers

  • DNS TTL

    DNS TTL (time to live) is a setting that tells the DNS resolver how long to cache a query before requesting a new one. The information gathered is then stored in the cache of the recursive or local resolver for the TTL before it reaches back out to collect new, updated details.

  • Hairpinning

    Hairpinning is where a machine on the LAN is able to access another machine on the LAN via the external IP address of the LAN/router (with port forwarding set up on the router to direct requests to the appropriate machine on the LAN).

  • NAT Reflection

    NAT reflection refers to the ability to access external services from the internal network using the external (usually public) IP address, the same as if the client were on the Internet.

  • Split DNS

    A preferable alternative to NAT reflection is deploying a split DNS infrastructure. Split DNS refers to a DNS configuration where, for a given hostname, public Internet DNS resolves to public IP address, and DNS on the internal network resolves to the internal, private IP address.

    Split DNS allows servers to see the true client IP address, and connections between servers and clients in the same subnet will go directly, rather than unnecessarily involving the firewall.

Vomit IT - Chunky Mess Style
  • 40,038
  • 27
  • 84
  • 117
  • @user3728501... I'd use dig or nslookup to check the DNS domain and mail host records from a machine which you are having trouble sending email to your domain. It could be one machine has gotten the DNS per TTL and email to you from it works but another machine or external testing tool is using the old DNS record. Try flushing DNS and reregistering it and see if that helps from any client you have issues with. Consider making the TTL 1 hour moving forward if you cannot go with a static on dynamic dns solution. Basic DNS troubleshooting should suffice to show you the problem here though. – Vomit IT - Chunky Mess Style May 22 '21 at 14:26
  • Since you said it was working and then just stopped when your IP changed, to me that means DNS and not something on your internal network since nothing there changed. I apologize if I've misinterpreted your detail though. The different ports and protocols are for sending or receiving and using TLS encryption or not with those connections basically without getting too technical. – Vomit IT - Chunky Mess Style May 22 '21 at 14:28
  • On Epik my TTL is 300 - but I'm not sure exactly what are the "units" for that – user3728501 May 22 '21 at 14:58
  • To clarify: Thunderbird broke after the IP address change. However I could not receive mail on my mail server, before or after this change. – user3728501 May 22 '21 at 14:59
  • `nslookup mydomain.com` says: `can't find mydomain.com: no answer`. however `nslookup mail.mydomain.com` obtains the correct IP. Does this indicate an issue? – user3728501 May 22 '21 at 15:01
  • Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/124553/discussion-between-pillsbury-it-doughboy-and-user3728501). – Vomit IT - Chunky Mess Style May 22 '21 at 15:39
  • 1
    @user3728501 Our discussion with full detail for troubleshooting and such is in the chat also linked to my edited answer. I think you have the issue resolved now. and if so I encourage you to consider opening additional questions for other aspects of configuration issues you may run into if your main issue is resolved for this question. As per "*Any help / advice or steps to take to diagnose*", it has been a pleasure assisting here as best I could so you could get email sent to your email server to your mailbox. – Vomit IT - Chunky Mess Style May 23 '21 at 01:26
  • Very helpful thanks - regarding the split DNS feature. Does this mean I have to run my own DNS server on pfSense? In which case, do I need to set Epik to treat my public IP address as the start of authority (?) for every host and subdomain which I might have behind my public IP? – user3728501 May 23 '21 at 10:48
  • @user3728501 You just need to have any internal DNS server on your internal network/subnet that the host machines point to which has a host A records for those same hosts e.g. mail, WWW but point those to the LAN private IPs of those machines. I believe the SOA record on the internal DNS server machine can point to itself with the private IP. You can try to also create an internal A host record with no host specified, the name will be mydomain.com and the IP will be the public IP of your domain. This is how it appears an environment I support is configured. You could test edit hosts file too. – Vomit IT - Chunky Mess Style May 23 '21 at 11:58
1

Install required software on a Raspberry Pi. Setup required port forwarding on my home network. Discovered that my ISP router prevents port 25 from being forwarded, so tried using 2525 instead. (Read online that 2525 was a common alternative port if 25 is blocked. Not sure how valid this information is.)

Not very. For server-to-server mail exchange, 25 is the only port that is used. If you're trying to receive mail from other domains, you will need port 25 inbound to work. Listening on any other port will only work if the sender has that port manually configured for your domain – and that's just not going to happen. And similarly, if you're trying to send mail to other domains, you will need port 25 outbound to be allowed, because that's the only port that other mail systems are guaranteed to listen on.

(Previously port 25 was the all-purpose SMTP port, so due to it being blocked some services might have offered 2525 for client-to-server mail submission, i.e. from desktop mail apps. However, nowadays 587 (together with 465) is the dedicated mail submission port and 25 is the server-to-server mail exchange port, while 2525 is only seen in custom configurations.)

u1686_grawity
  • 426,297
  • 64
  • 894
  • 966
  • Yes that's what I thought as well. The only way it could have worked is if "most" mail servers on the net would try 25 by default and then try 2525 if they couldn't reach 25. But clearly this isn't how most mailservers are setup. – user3728501 May 22 '21 at 15:02
  • 2
    @user3728501 My guess is, if this were the case then your ISP would also block port 2525. They block ports for a reason and that reason is that it’s very easy to misconfigure an email server, if you’re not absolutely sure what you’re doing, so that it can become a part of a spam or phishing network. – StarCat May 22 '21 at 19:51
  • @StarCat is there any information about that? Surely if the server is password protected then it cannot be (or is not likely) to be hacked to become part of a spam botnet? – user3728501 May 22 '21 at 21:59
  • @user3728501: The mail submission service (port 587) can be password-protected, but the mail exchange service (port 25) cannot, as other systems wouldn't know what password to use; instead it checks whether the recipient domain is local (i.e. only accepts mail in, not out). – u1686_grawity May 22 '21 at 22:10
  • Password protection of IMAP and Submission is only a very small part of securing a mail server. Securing against open relaying, spoofing (SPF, DKIM, DMARC), software vulnerabilities etc. are also important. – StarCat May 22 '21 at 22:11
  • @user3728501: (Which is normal, that's not the point I want to make...) the point I want to make is that most smtp services don't have this separation hardcoded, instead they support flexible "relay policies" in order to allow various combinations (e.g. the same port 25 allowing optional authentication to provide both services on same port)... and the more flexible configuration is, the easier it is to accidentally get wrong and make it accept too much (or too little). Postfix _tries_ to catch the common misconfigurations, but not all. – u1686_grawity May 22 '21 at 22:11
  • @StarCat are there any good online available references to understand this and take whatever measures are required to prevent problems? – user3728501 May 22 '21 at 22:15
  • @user3728501 [How to Set up SPF and DKIM with Postfix](https://www.linuxbabe.com/mail-server/setting-up-dkim-and-spf) and [Creating DMARC Record to Protect Your Domain Name From Email Spoofing](https://www.linuxbabe.com/mail-server/create-dmarc-record) might be two good basic starting points for those other things for email server security you implement. Google "configure postfix to ensure it is not a relay" and you will find some basic detail there too for a starting point. Then there is antispam and AV too for incoming email to protect for email security in general too. – Vomit IT - Chunky Mess Style May 23 '21 at 01:21