Can't get @mx_any to get MX lookup failures to be ignored

[Eli]

I've got a domainlist that I use to figure out if a domain is local or not -
it's based on a local list of domains I know are local, and also does an MX
lookup to see if the domain has an MX record that points to our server
(can't steal email hosting this way though - still have to pass local
account checks). I have a feeling I don't even need the MX lookup part of
the list any more, but that's for me to figure out later :)

Anyways, my problem comes in when email comes in and the sender specifies
their mail from and gives a domain name that Exim needs to verify with my
dnslookup router (since I do domain name verification on senders). If one
of the MX records that the domain has however can't be resolved, Exim ends
up giving a "temporary local problem" error since it is trying to resolve
all MX records to compare against the domainlist to even figure out if it
should use the dnslookup router (so basically the router isn't the problem,
but the fact that @mx_any is making Exim look up all MX records).

I noticed in the documentation that if it were a *host*list, I could use
"+ignore_unknown" in the list to have it fix my problem, but this isn't
allowed in domainlists. The only other settings I could think of were the
"mx_fail_domains" and "srv_fail_domains" settings for the dnslookup router -
I put them in just to see if it made a difference, but I didn't think it
would fix my problem since it seems to be before the router is executed that
the problem happens.

Here are the parts of my config file:


domainlist treat_as_local = ${lookup {$domain} dbmnz
{/etc/exim.host.db} {${if > {$value}{0} {$domain}{}}}{}} :
@mx_any/ignore=!1.2.3.4
remote_address:
driver = dnslookup
domains = !+treat_as_local
mx_fail_domains = *
srv_fail_domains = *
ignore_target_hosts = 0.0.0.0 : 127.0.0.0/8
cannot_route_message = Insufficient DNS information found for
$domain
transport = remote_delivery
more = no


... And here is the debug output:


7821 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
7821 Verifying ***@cpp-db.com
7821 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
7821 Considering ***@cpp-db.com
7821 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
7821 routing ***@cpp-db.com
7821 --------> remote_address router <--------
7821 local_part=user domain=cpp-db.com
7821 checking domains
7821 search_open: dbmnz "/etc/exim.host.db"
7821 search_find: file="/etc/exim.host.db"
7821 key="cpp-db.com" partial=-1 affix=NULL starflags=0
7821 LRU list:
7821 2/etc/exim.host.db
7821 :/etc/exim.whitelist
7821 6/etc/exim.blacklist
7821 End
7821 internal_search_find: file="/etc/exim.host.db"
7821 type=dbmnz key="cpp-db.com"
7821 file lookup required for cpp-db.com
7821 in /etc/exim.host.db
7821 lookup failed
7821 DNS lookup of cpp-db.com (MX) succeeded
7821 199.0.58.101 in "!66.165.106.125"? yes (end of list)
7821 ignored host mail.cpp-db.com [199.0.58.101]
7821 66.88.134.59 in "!66.165.106.125"? yes (end of list)
7821 ignored host mail01.cpp-db.com [66.88.134.59]
7821 DNS lookup of mail.cpp-db.us (A) gave TRY_AGAIN
7821 mail.cpp-db.us in dns_again_means_nonexist? no (option unset)
7821 returning DNS_AGAIN
7821 host_find_bydns yield = HOST_FIND_AGAIN (1); returned hosts:
7821 mail.cpp-db.us <null> MX=20 *
7821 cpp-db.com in " : @mx_any/ignore=!1.2.3.4"? lookup deferred for
@mx_any/ignore=!1.2.3.4
7821 cpp-db.com in "!+treat_as_local"? lookup deferred for !+treat_as_local
7821 domains check lookup or other defer
7821 ----------- end verify ------------
7821 deny: condition test deferred
7821 SMTP>> 451 Temporary local problem - please try later
7821 LOG: MAIN REJECT
7821 H=(test) [216.209.84.151] I=[66.165.125.115]:2525 temporarily
rejected MAIL <***@cpp-db.com>: DNS lookup of "cpp-db.com" deferred


You can test with cpp-db.com as well if you'd like (I don't host it, just a
domain I know has an unresolvable MX record). It's the mail.cpp-db.us
domain that can't be resolved. The really odd part is that when I use dig
to try to look it up, I get a SRVFAIL for the DNS response, however Exim
says that DNS was deferring (though maybe this was due to a change made to
Exim for a purpose - I remember several discussions about how failed DNS
should be handled.

Is there any way I can have Exim ignore an MX lookup failure for @mx_any ?

Eli

[Kjetil Torgrim Homme]

Post by Eli
Anyways, my problem comes in when email comes in and the sender specifies
their mail from and gives a domain name that Exim needs to verify with my
dnslookup router (since I do domain name verification on senders). If one
of the MX records that the domain has however can't be resolved, Exim ends
up giving a "temporary local problem" error since it is trying to resolve
all MX records to compare against the domainlist to even figure out if it
should use the dnslookup router (so basically the router isn't the problem,

excuse me, but why do you want to accept e-mail from a broken system?
you won't be able to send them e-mail anyway.

are you using verify = sender?

[Eli]

Post by Kjetil Torgrim Homme
excuse me, but why do you want to accept e-mail from a broken system?
you won't be able to send them e-mail anyway.

In this case, I still can as it is a backup MX that isn't resolving. I know
what accepting the email would do, and still debating whether I even want to
(I rather enjoy telling people to fix their stupid mistakes, but stupid
people usually don't know how to fix them, which is why their mistakes exist
:P). I was just slightly curious if there was any way to really just have
Exim stop from reporting an internal error and instead report something a
bit more meaningful so people understand what the problem is rather than
emailing me (in this case, about 10 times and they still can't figure it
out).

Post by Kjetil Torgrim Homme
are you using verify = sender?

Yes, with /no_details. It uses the dnslookup router I gave in my first
email, however it's not even that causing the problem - it's the @mx_any in
my +treat_as_local domainlist as it tries to figure out if cpp-db.com is in
the list or not (which it should not be).

Eli.

[Brian Candler]

Post by Eli
domainlist treat_as_local = ${lookup {$domain} dbmnz
@mx_any/ignore=!1.2.3.4

...

Post by Eli
7821 H=(test) [216.209.84.151] I=[66.165.125.115]:2525 temporarily

...
I don't know if there is - but I'd just observe that Exim seems to be doing
The Right Thing[TM] here.

As you know, the decision as to whether a domain is local or not is
important, as it fundamentally influences the routing and ACL decisions
(i.e. should I try looking up names in the local account database? or should
I try a remote delivery? Is this an attempt to relay, and if so, is it
authorised?)

Given that this domain does not exist in your exim.host.db, then whether or
not the hostname has an MX record which resolves to your machine is
critical, and can't be left to chance in the event that the DNS can't tell
whether it does or not.

However, in this particular example, there might be a simple solution. If
your "treat_as_local" domainlist means addresses which should do a database
lookup and perform local delivery, then presumably you only want to do this
if this machine is the *primary* MX for this domain. Otherwise, it's just
acting as a backup/relay host, and should relay via SMTP.

So you could use @mx_primary instead of @mx_any - and since the cpp-db.com
failure was on a backup MX record, this particular problem might be solved.

Sorry that doesn't answer your actual question though. I do think however
that @mx_any/defer_ok would be (in my opinion) a dangerous option. Actually
you'd need two options: (1) defer implies this domain *is* included in
mx_any, and (2) defer implies this domain *isn't* included in mx_any.

To solve it in your way, you could perhaps rearrange the router logic:

1. Try a local database lookup always (regardless of whether domain is in
exim.host.db and regardless of DNS). If found, then deliver.
2. Otherwise, check if the domain is in exim.host.db or @mx_primary, and
bounce if it is.
3. Otherwise, this is a non-local delivery.

However, you're still likely to have a problem with ACLs, since your
anti-relaying controls are going to need to know whether a domain is local
or not. So this is really just a demonstration of why using @mx_any or
@mx_primary is a dubious thing to do, and you'd be better just listing all
local domains in your exim.host.db anyway.

Just a few thoughts,

Brian.

[Eli]

Post by Brian Candler
I don't know if there is - but I'd just observe that Exim
seems to be doing The Right Thing[TM] here.

In a way yes, however the error reported due to Exim having no way to ignore
or reject a DNS error in an @mx_any (or _primary/_secondary) is what's
causing me some grief.

Post by Brian Candler
As you know, the decision as to whether a domain is local or
not is important, as it fundamentally influences the routing
and ACL decisions (i.e. should I try looking up names in the
local account database? or should I try a remote delivery? Is
this an attempt to relay, and if so, is it
authorised?)

Yes - the domain in question (cpp-db.com) is not local, and I know what
would happen if I had Exim ignore and accept a broken DNS entry - with my
config, Exim would not accept the email and would instead default to my
final routers error message (not sure what that is at the moment), which
would make me happier than Exim deferring temporarily with a "temporary
local problem" message. This has caused the domain to retry email countless
times to my systems, and although not annoying to me, does cause users to
wonder whats up with their email, and ask me what this "local problem" is
(and trying to teach them about DNS isn't so fun either).

Post by Brian Candler
the cpp-db.com failure was on a backup MX record, this
particular problem might be solved.

I do have domains that do not use my systems as primary MX records and just
POP off messages (I don't offer ETRN) if there are any. For this reason I
can't use @mx_primary or @mx_secondary. I had noticed that in Exim docs as
well, but it won't suit my situation unfortunately.

Post by Brian Candler
Sorry that doesn't answer your actual question though. I do
a dangerous option. Actually you'd need two options: (1)
defer implies this domain *is* included in mx_any, and (2)
defer implies this domain *isn't* included in mx_any.

True, which was why maybe allowing those two neat DNS switches that you can
have in hosts lists in to domain lists might be a better choice?

Post by Brian Candler
From what I understand, it doesn't matter how I organize it, as long as I

rely on ever doing an @mx_* lookup, if there's any broken DNS in the domain,
Exim will issue a local problem if it can't resolve an MX record (unless
it's only due to my exclusion modifier, which I need though).

Post by Brian Candler
However, you're still likely to have a problem with ACLs,
since your anti-relaying controls are going to need to know
whether a domain is local or not. So this is really just a
dubious thing to do, and you'd be better just listing all
local domains in your exim.host.db anyway.

True - it would be nice if it were usable without this type of problem
though... And I do think I have all my hosts in the local file, so I'm
*still* not sure why I even have this mx lookup thinger in my config -
probably just left overs from previous configs.

Eli.