Discussion:
SMTP timeout sending mail to gmail
(too old to reply)
Daniel Webb
2006-04-07 00:52:27 UTC
Permalink
Nearly all (although not 100%) of my outgoing messages to gmail are giving
this (some are continuing to do this for 7 days now):

2006-04-06 18:48:45 1FRLeI-0006Wx-9T SMTP timeout while connected to
gmail-smtp-in.l.google.com [64.233.185.114] after end of data (1593 bytes
written): Connection timed out

How can I track this down? I'm using Exim 4.50 in Debian stable on Linux
2.6.15. I have a vague understanding of tcpdump, but not enough to give a
good trace to debug (let me know the right incantation and I'll send it
along).

I have no problems delivering to any other SMTP servers as far as I can tell.
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/
Tony Finch
2006-04-07 10:23:17 UTC
Permalink
Post by Daniel Webb
2006-04-06 18:48:45 1FRLeI-0006Wx-9T SMTP timeout while connected to
gmail-smtp-in.l.google.com [64.233.185.114] after end of data (1593 bytes
written): Connection timed out
How can I track this down?
The problem is at the gmail end so you'll have to ask them what's going on.

Tony.
--
<***@exim.org> <***@dotat.at> http://dotat.at/ ${sg{\N${sg{\
N\}{([^N]*)(.)(.)(.*)}{\$1\$3\$2\$1\$3\n\$2\$3\$4\$3\n\$3\$2\$4}}\
\N}{([^N]*)(.)(.)(.*)}{\$1\$3\$2\$1\$3\n\$2\$3\$4\$3\n\$3\$2\$4}}
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/
Daniel Webb
2006-04-08 00:17:57 UTC
Permalink
Post by Tony Finch
The problem is at the gmail end so you'll have to ask them what's going on.
Is this a known problem? I would think that if every exim user is getting all
their mail to gmail bouncing I wouldn't be the only one reporting it (and yes
I searched the archives).

So I'm wondering if there's something on my end that brings out the problem
where no one else does. I don't even know how to go about figure out what
that would be though, so I'm wondering if anyone here knows how to go about
it.

Based on my searching so far it looks like emailing gmail support would be a
complete waste of my time. Other posters on other lists who are more
important than me (ISP admins and so on) have reported they were completely
ignored by gmail support. Power breeds arrogance, eh?
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/
Peter Bowyer
2006-04-08 07:11:25 UTC
Permalink
Post by Daniel Webb
Post by Tony Finch
The problem is at the gmail end so you'll have to ask them what's going on.
Is this a known problem? I would think that if every exim user is getting all
their mail to gmail bouncing I wouldn't be the only one reporting it (and yes
I searched the archives).
One of my Exim servers forwards all of my mail to Gmail for
road-warrior access - never seen a problem.
Post by Daniel Webb
So I'm wondering if there's something on my end that brings out the problem
where no one else does. I don't even know how to go about figure out what
that would be though, so I'm wondering if anyone here knows how to go about
it.
Debug a delivery, run tcpdump on it, etc. Try lots of interactive
telnets to their port 25.
Post by Daniel Webb
Based on my searching so far it looks like emailing gmail support would be a
complete waste of my time. Other posters on other lists who are more
important than me (ISP admins and so on) have reported they were completely
ignored by gmail support. Power breeds arrogance, eh?
Not really arrogance, just scale.

Peter

--
Peter Bowyer
Email: ***@bowyer.org
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/
Avleen Vig
2006-04-08 07:26:55 UTC
Permalink
Post by Peter Bowyer
Post by Daniel Webb
Based on my searching so far it looks like emailing gmail support would be a
complete waste of my time. Other posters on other lists who are more
important than me (ISP admins and so on) have reported they were completely
ignored by gmail support. Power breeds arrogance, eh?
Not really arrogance, just scale.
That, at it helps to know who to talk to :-)

I run an exim server in the same way as you do, forwarding a few
thousand mails a day to various Gmail accounts. I have a problem when I
send mail too fast to one account (mail loops), and when I forward too
much spam in a very short amount of time (broken dspam setup), but
apart from that I have no problems.

Daniel, throw me the file created by this off-list, and I'll take a
look:

tcpdump -ni <ext intreface, eth0/fxp0/etc> -s 1500 -w <some_file_to_save_to>
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/
Oliver Heesakkers
2006-04-08 12:16:55 UTC
Permalink
Post by Daniel Webb
Post by Tony Finch
The problem is at the gmail end so you'll have to ask them what's going on.
Is this a known problem? I would think that if every exim user is getting all
their mail to gmail bouncing I wouldn't be the only one reporting it (and yes
I searched the archives).
So I'm wondering if there's something on my end that brings out the problem
where no one else does. I don't even know how to go about figure out what
that would be though, so I'm wondering if anyone here knows how to go about
it.
Based on my searching so far it looks like emailing gmail support would be a
complete waste of my time. Other posters on other lists who are more
important than me (ISP admins and so on) have reported they were completely
ignored by gmail support. Power breeds arrogance, eh?
What I get is this:
(look at times and serveraddresses explicitly)
-------------------
17:12:09 1FRsd7-000HMF-99 <= ***@example.tld

17:17:09 1FRsd7-000HMF-99 SMTP timeout while connected to
gmail-smtp-in.l.google.com [66.249.93.114] after initial connection:
Operation timed out

17:17:09 1FRsd7-000HMF-99 => ***@gmail.com H=gmail-smtp-in.l.google.com
[66.249.93.27]

17:17:09 1FRsd7-000HMF-99 Completed
-------------------

Time-outs happen with hotmail too, but they usually deliver on the
second server, one or two times I've seen two time-outs in a row with
hotmail, but then the third server took it.

In any way, never has a mail to a gmail- or hotmailaccount been rejected
because of a timeout.
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/
John W. Baxter
2006-04-09 03:33:27 UTC
Permalink
Post by Daniel Webb
Post by Tony Finch
The problem is at the gmail end so you'll have to ask them what's going on.
Is this a known problem? I would think that if every exim user is getting all
their mail to gmail bouncing I wouldn't be the only one reporting it (and yes
I searched the archives).
As with several others, our sends to Gmail go fine. (Both from our general
outgoing mail servers and from our mailing list server.)

It's quite possible Gmail doesn't like this combination (assuming that you
send to this list via the same machine your server sends to Gmail:

$host danielwebb.us
danielwebb.us has address 64.81.101.157
[$host 64.81.101.157
157.101.81.64.in-addr.arpa domain name pointer
dsl081-101-157.den1.dsl.speakeasy.net.

Speakeasy will provide explicit DNS, assuming you are on a fixed IP, which
is probably a good assumption. We bought a line from competitor Speakeasy
to have a DSL connection which did not depend on our DSL equipment or our
ATM feed...when turned up, it had the prior customer's reverse DNS (which
Speakeasy quickly fixed when told of it).

--John
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/
Daniel Webb
2006-04-09 05:44:29 UTC
Permalink
Post by John W. Baxter
As with several others, our sends to Gmail go fine. (Both from our general
outgoing mail servers and from our mailing list server.)
I figured out how to make it not happen:

I had the MTU for my mail server set to 1500 while the router was 400 (I use
such a small MTU because it greatly improves the quality of my VoIP calls).
When I changed the mail server MTU to 400 as well, the mails to gmail go
through again.

I'm still unsure if this is a problem on my end or gmail's. Why would all
other mails go through fine but not to gmail?
Post by John W. Baxter
It's quite possible Gmail doesn't like this combination (assuming that you
$host danielwebb.us
danielwebb.us has address 64.81.101.157
[$host 64.81.101.157
157.101.81.64.in-addr.arpa domain name pointer
dsl081-101-157.den1.dsl.speakeasy.net.
Speakeasy will provide explicit DNS, assuming you are on a fixed IP, which
is probably a good assumption. We bought a line from competitor Speakeasy
to have a DSL connection which did not depend on our DSL equipment or our
ATM feed...when turned up, it had the prior customer's reverse DNS (which
Speakeasy quickly fixed when told of it).
Thanks for the info. For some reason I thought they only did reverse DNS for
business customers, but that's not true. I put in a request just now.

That wasn't the cause of the connection problem, but it will probably keep me
out of some spam folders (my mails have gone to "bulk" at Yahoo on an off for
a couple of years).
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/
Daniel Webb
2006-04-09 07:37:38 UTC
Permalink
Post by Daniel Webb
I'm still unsure if this is a problem on my end or gmail's. Why would all
other mails go through fine but not to gmail?
Here's what I think is happening, and why it was only manifesting with Google:
Google is filtering ICMP packets used for path MTU discovery and thus breaking
emails that go over 400 bytes (only with my system). Because the mail server
had an MTU of 1500, Google was trying to use 1500 byte packets.

See the "Example Path MTU Discovery Failure Scenario" here:

http://www.netheaven.com/pmtu.html

I think that's exactly what's happening.

I'm an amateur at networking and this isn't my day job, so I'm open to other
possibilities.

P.S. I checked my firewall and I'm not blocking any ICMP packets.
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/
W B Hacker
2006-04-09 08:00:41 UTC
Permalink
Post by Daniel Webb
Post by Daniel Webb
I'm still unsure if this is a problem on my end or gmail's. Why would all
other mails go through fine but not to gmail?
Google is filtering ICMP packets used for path MTU discovery and thus breaking
emails that go over 400 bytes (only with my system). Because the mail server
had an MTU of 1500, Google was trying to use 1500 byte packets.
http://www.netheaven.com/pmtu.html
I think that's exactly what's happening.
I'm an amateur at networking and this isn't my day job, so I'm open to other
possibilities.
P.S. I checked my firewall and I'm not blocking any ICMP packets.
I suspect you are correct, and tcpdump may show it clearly.

But - while we had a 'related' problem here with a broadband
supplier, initially supported by tcpdump records, and reduced,
but not fully resolved by both re-compiling Exim to use smaller
MTU AND setting the client(s) to match, we ultimately found that
a dodgy SiS 10/100 NIC & its 4.* FreeBSD driver combo was a
major contributor.

Switched the cable and IP to the Intel 10/100/Gig-E NIC in the
same server, and the problem went away and took some other
problems away with it.

So don't rule out 'hardware', or hardware+driver problems,
especially when it seems one is the 'Lone Ranger' on an issue.

JFWIW, YMMV

Bill
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/
Avleen Vig
2006-04-09 13:07:58 UTC
Permalink
Post by W B Hacker
I suspect you are correct, and tcpdump may show it clearly.
I don't think that should be the case.
If Daniel is sending packets larger than 400 bytes, it is his OWN router
that should complain that the packets are too large before they leave
the network.
By the time they get to the gmail servers, they should either be <= 400
bytes, or they shouldn't leave his router.
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/
Marc Sherman
2006-04-09 13:31:49 UTC
Permalink
Post by Avleen Vig
I don't think that should be the case.
If Daniel is sending packets larger than 400 bytes, it is his OWN router
that should complain that the packets are too large before they leave
the network.
By the time they get to the gmail servers, they should either be <= 400
bytes, or they shouldn't leave his router.
Yeah, I've had MTU problems with mail in the past, but they've always
caused timeouts on inbound mail, never outbound mail. Unless gmail is
trying to send you a reply packet larger than 400B, and that's what's
timing out, but I can't imagine a reply larger than 400B in a standard
SMTP conversion unless there's a large rejection message.

- Marc
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/
W B Hacker
2006-04-09 14:23:51 UTC
Permalink
Post by Marc Sherman
Post by Avleen Vig
I don't think that should be the case.
If Daniel is sending packets larger than 400 bytes, it is his OWN router
that should complain that the packets are too large before they leave
the network.
By the time they get to the gmail servers, they should either be <= 400
bytes, or they shouldn't leave his router.
Yeah, I've had MTU problems with mail in the past, but they've always
caused timeouts on inbound mail, never outbound mail. Unless gmail is
trying to send you a reply packet larger than 400B, and that's what's
timing out, but I can't imagine a reply larger than 400B in a standard
SMTP conversion unless there's a large rejection message.
- Marc
There may be a thought. One might be able to trigger such a
reply...

Bill
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/
W B Hacker
2006-04-09 14:22:48 UTC
Permalink
Post by Avleen Vig
Post by W B Hacker
I suspect you are correct, and tcpdump may show it clearly.
I don't think that should be the case.
If Daniel is sending packets larger than 400 bytes, it is his OWN router
that should complain that the packets are too large before they leave
the network.
By the time they get to the gmail servers, they should either be <= 400
bytes, or they shouldn't leave his router.
Still worth a look.

Keeping in mind that a session should be two-way, what we saw in
our case was an imbalance, wherein the session fired up, but
then the distant end began to slow down and finally time-out.

Grant - in our case, we were looking at *incoming* traffic,
where the balance of packets was expected to be heaviest inbound.

His situation is, of course, the reverse.

- But I doubt gmail techs would run any sort of analysis, so..

Bill
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/
Jakob Hirsch
2006-04-09 14:44:28 UTC
Permalink
Post by Avleen Vig
I don't think that should be the case.
If Daniel is sending packets larger than 400 bytes, it is his OWN router
that should complain that the packets are too large before they leave
the network.
If Google/gmail is really blocking icmp fragmentation-needed (which I
doubt), it's the gmail server not receiving the icmp packet (telling it
that its packets are too big) and therefore breaking path MTU discovery.
I only wonder why the timeout is happening on outgoing mail, that would
indicate that the problem is on the Daniel's own mail server, but then
the issue would be with all outgoing mail.

Anyway, I wouldn't set the MTU to 400, it's a waste of bandwith and it
probably even violates RFC 791, which says the minimum supported
datagram size must 576 octets (with or without fragmentation, but with
PMTU all packets have the don't-fragment bit set).
A router with decent traffic shaping would be a much better thing.
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/
Alan J. Flavell
2006-04-09 09:37:26 UTC
Permalink
Post by Daniel Webb
Here's what I think is happening, and why it was only manifesting
with Google: Google is filtering ICMP packets used for path MTU
discovery and thus breaking emails that go over 400 bytes (only with
my system). Because the mail server had an MTU of 1500, Google was
trying to use 1500 byte packets.
We used to encounter very similar symptoms with a very small number of
remote sites. In our case, it wasn't unusual for small mails to get
through, while large ones were retried indefinitely and finally
timed-out.

On investigation it usually transpired that the destination site was
using a Cisco PIX firewall, with what I suppose was a default (but in
any case inept!) configuration which, as you say, defeated MTU
discovery. Temporarily reducing the MTU size in our networking stack
would get the items through, but we refused to nobble our whole
networking performance just for the sake of one or two misbehaved
peers. I played around for a while trying to configure explicit
routes to the rogue destinations and give them an explicit MTU, but
never came up with an answer that I could really recommend to anyone
else.

The problem seemed to fade away with time, so I stopped looking for
Wuergarounds (to use a rather apt piece of German techno-slang).

Seems you may have encountered it in a different context...

OT here, but surely it must be possible to control your networking
software's use of RTP (UDP) packet size as desired for VoIP,
separately from the values used by TCP?

good luck
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/
Tony Finch
2006-04-09 16:54:51 UTC
Permalink
Post by Alan J. Flavell
OT here, but surely it must be possible to control your networking
software's use of RTP (UDP) packet size as desired for VoIP,
separately from the values used by TCP?
The problem is probably that a large TCP MTU causes undesirable jitter on
the RTP stream.

Tony.
--
<***@exim.org> <***@dotat.at> http://dotat.at/ ${sg{\N${sg{\
N\}{([^N]*)(.)(.)(.*)}{\$1\$3\$2\$1\$3\n\$2\$3\$4\$3\n\$3\$2\$4}}\
\N}{([^N]*)(.)(.)(.*)}{\$1\$3\$2\$1\$3\n\$2\$3\$4\$3\n\$3\$2\$4}}
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/
Daniel Webb
2006-04-09 19:13:11 UTC
Permalink
Wow, thanks for all the great help!

Like I said, this isn't my day job, and it looks like it was all my fault
after all. I have discovered "IPv4 requires a minimum MTU of 576 bytes. Below
it linux uses the minimum MTU and turns off path mtu discovery (drops DF)"[1],
which I definitely did not know. So it was my systems that were ignoring path
MTU discovery, even though I don't block ICMP and didn't (knowingly) turn off
path MTU discovery?

Sure enough, the tcpdump of a session shows that the packets are close to 576
bytes.

I'm not really sure why this only borked with gmail... a packet dump of a
SMTP session with Gmail is at http://danielwebb.us/misc/gmail.tcpdump if
anyone knows what's going on. It looks to me like my system receives an ACK
and then ignores it and retries the packet forever.

As for the VoIP jitter: I read that Asterisk uses a hard-coded RTP packet size
of 0.02 seconds[2], so my UDP packets heading out to my provider are already
small. Here's (maybe) the problem, though: suppose a 1500 byte TCP packet
starts out on the wire, and immediately after a new VoIP packet comes in. The
VoIP packet is going to have to wait an absolute minimum of 35 ms before
starting transmission (my connection is about 340kbit outbound). That
actually doesn't sound too bad to me, but when I throttled down my MTU the
VoIP quality increased dramatically.

I do have QoS with VoIP packets given highest priority, but given my
experience here it's plenty likely I messed that up somehow too.
One thing I'm not sure about is where and how many queues are involved in the
packet traversal. For example, should I do:

ip link set dev eth0 qlen 1

or does the eth0 qdisc have access to that queue anyhow and that's a stupid
thing to do?

[1] http://www.ussg.iu.edu/hypermail/linux/kernel/0302.0/0129.html
[2] http://www.voip-info.org/wiki/view/Asterisk+SPA-841+Sound+Quality
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/
W B Hacker
2006-04-09 19:36:51 UTC
Permalink
Daniel Webb wrote:

*trimmed*
Post by Daniel Webb
Sure enough, the tcpdump of a session shows that the packets are close to 576
bytes.
*trimmed*
Post by Daniel Webb
I do have QoS with VoIP packets given highest priority, but given my
JMNSHO, but trying to run an MTA *and* VoIP on the same slender
resources you describe, (...about 340kbit outbound) does not
strike me as a good idea, regardless of settings - quite aside
from being way off-topic here.. ;-)

You may not generate much traffic, nor receive all that much in
'legit' incoming, but the spam-wanking zombies that will hammer
on your MTA's door often arrive in bunches, and Exim's filtering
kicks in only after they have grabbed a TCP connection, so you
don't have all that much control over link load.

No amount of fiddling will make that go away.

Better to use a (leased? ISP?) remote MTA on a fat pipe, (100
Mb/s bothway for ours) and stick with a decent MUA-only on the
lowball remote link.

That way, you can take the MUA 'offline' while using the phone,
yet not lose any messages.

JM2CW,

Bill
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/
Continue reading on narkive:
Loading...