SA Bugzilla – Bug 6841
Spamd cannot listen to more than one IP address (or to v4/v6 at the same time)
Last modified: 2014-02-02 22:03:51 UTC
Pretty simple, but a big one: Spamd cannot listen to more than one (specified) IP address. If you specify multiple -i options, only the last one is used. While it's possible to spin up multiple spamd processes, each bound to a separate IP address, it feels as though there's a cleaner way to do this. The larger problem here is that if you specify simply "-i" spamd will not listen on "all addresses" but only "all ipv4 addresses". If you specify something like "-i localhost" or "-i [hostname]" this will only be a single-stack bind according to the standard lookup rules (typically v4 first, then v6). To explain this further, assume you're doing your spamc calls with -d hostname. This should simply work in the same way (if there's a AAAA and the host has v6 connectivity, use it, else use v4). I've seen other daemons able to bind to * on TCP46 (dual-stack). If this is too complicated to do in current spamd code, then it should be noted in the docs section and perhaps the issues of potential file locking should be noted.
As noted on the mailing list, this issue might be OS specific. Per DFS: I think this is a FreeBSDism. On Linux, something listening on :: will answer both IPv4 and IPv6 connection attempts. Maybe FreeBSD has a way to emulate that? And continued with Greg Toxel: It's not quite right to call that a FreeBSDism; it's much messier than that. IPv6 supports a concept called mapped addresses, where v4 addresses can be represented in v6 addresses. A system can be configured to have sockets that listen on :: also listen on INADDR_ANY and present the v4 addresses as mapped v6 addresses. This feature is somewhat controversial, because of security concerns (if the program didn't open a v4 socket, why is it possible to connect to it over the net via v4?): http://en.wikipedia.org/wiki/IPv4_mapped_address#IPv4-mapped_IPv6_addresses On NetBSD, the default is that v6 sockets are only v6 (via sysctl): "net.inet6.ip6.v6only = 1", and I believe OpenBSD and FreeBSD are the same way. See http://tools.ietf.org/html/rfc3493 http://tools.ietf.org/html/rfc3542#section-13 So I suspect that on Linux, v6only defaults to off (while on *BSD it defaults to on). Apparently on some systems it's always off because the stacks are separate IMHO, portable software should have two sockets, one on INADDR_ANY and one on IN6ADDR_ANY. But, setting the socket option may be a workaround. It's certainly wrong to assume that an OS has a particular default.
It's worth it to note here that proftpd and apache manage to do it with a single bind. I should also ask how difficult it would be to solve this by simply making the -i option work multiple times, as opposed to fixing the socket semantics? I don't know the internal locking/queueing well enough to know if that's an "easier" answer. -Dan
I checked an apache server that listens on both v4 and v6 and the config file has Listen 0.0.0.0:80 Listen [::]:80 So I don't follow "single bind". (I suspect that on systems that have v6only set to 0, listening on [::] will result in handling v4 connections as mapped addresses.)
Good comment on listserv from John Wilcock: The documentation for apache's Listen directive is actually very helpful in explaining the situation with regard to IPv4-mapped IPv6 addresses and BSD's different default. http://httpd.apache.org/docs/2.2/en/bind.html#ipv6
Greg, on the other hand, on my machine (FreeBSD by the way), with apache: %cd /usr/local/etc/apache22 %grep -Ri listen * httpd-webmail.conf:Listen 149.20.61.46:80 httpd-webmail.conf:Listen 149.20.61.46:443 httpd.conf:# Listen: Allows you to bind Apache to specific IP addresses and/or httpd.conf:# Change this to Listen on specific IP addresses as shown below to httpd.conf:#Listen 12.34.56.78:80 httpd.conf:Listen 80 vhost.conf:#Listen 8090 vhost.conf:listen [2001:470:1f07:8d8:9746:1ac4:a56d:b20c]:80 vhost.conf.original:Listen 8090 %netstat -na|grep LIST tcp4 0 0 149.20.61.46.443 *.* LISTEN tcp4 0 0 149.20.61.46.80 *.* LISTEN tcp6 0 0 2001:470:1f07:8d.80 *.* LISTEN tcp46 0 0 *.80 *.* LISTEN prime# sysctl -a|grep -i v6only net.inet6.ip6.v6only: 1 While I wouldn't argue that this may be different on different OSes, I seem to think that apache's doing the right thing (I specified listen 80, without specifying an address family, and it listened on v4 and v6), despite the "v6only" setting above. I'm not trying to argue that all these things should be the same or that the apache way is the One True Way, mind you -- just trying to point out that it's actually possible, despite what I think has been previously said. If anyone wants a shell to see how apache was built, whatnot, let me know.
So apache when told to listen on 80 opened two sockets. That sounds exactly right, and SA should default to two sockets, one on each AF, unless given specific instructions. Really the thing that is needed for these issue is to change programs that have a 'one socket' concept into a list of sockets, and then it's easy.
> Really the thing that is needed for these issue is to change programs that > have a 'one socket' concept into a list of sockets, and then it's easy. Getting there, brace yourself for a hefty change to internals in spamd ...
Ok, here comes the biggie: Bug 6841 - Spamd cannot listen to more than one IP address (or to v4/v6 at the same time) Sending lib/Mail/SpamAssassin/Util/DependencyInfo.pm Sending spamd/spamd.raw Committed revision 1396626. - allows spamd to listen on multiple sockets - adds spamd options -4 and -6 to prefer one or the other protocol - widens the syntax of a spamd option --listen (alias -i) to accept any type of a socket (its IPv4/IPv6 address or a host name (with possibly multiple addresses), or a Unix socket) along with an optional port number to override the global default --port or --ssl-port option - uses module IO::Socket::IP if available, otherwise falls back to IO::Socket::INET6, or else to IO::Socket::INET What would still be desired is: - provide round-robin scheduling in a select() to avoid one busy socket monopolizing the server - extend the use of IO::Socket::IP to the async DNS resolver and to a plugin like DCC
Created attachment 5098 [details] Enables spamd to listen on multiple sockets: IPv6 or IPv4 or Unix sockets The diff is in the attachment.
Created attachment 5100 [details] Add options -4 and -6 to spamassassin, use IO::Socket::IP for DNS if available > What would still be desired is: > - extend the use of IO::Socket::IP to the async DNS resolver > and to a plugin like DCC Ok, here comes the rest - attached. Bug 6841: Add options -4 and -6 to spamassassin, use IO::Socket::IP for DNS if available Sending lib/Mail/SpamAssassin/DnsResolver.pm Sending lib/Mail/SpamAssassin.pm Sending lib/spamassassin-run.pod Sending spamassassin.raw Committed revision 1398905. Changes: - adds option -4 to spamassassin as an alias for --ipv4only for consistency with spamd, spamc and other network tools; - adds option -6 to spamassassin, mirroring that of spamd; - adjust DnsResolver.pm to understand force_ipv4 and force_ipv6 - extend DnsResolver.pm to use a module IO::Socket::IP if available, or fall back to IO::Socket::INET6, or else to IO::Socket::INET
Done, closing.
trunk: related to Bug 6841: decouple choosing a Socket vs. Socket6 module from choosing an IO::Socket::{IP,INET6,INET} module, improve compatibility with old versions of these modules Sending lib/Mail/SpamAssassin/Util/DependencyInfo.pm Sending spamd/spamd.raw Committed revision 1450858.
trunk: related to Bug 6841: testing for PF_INET and PF_INET6 availability in spamd allows automatic choice of default protocol families for listen sockets (like on an IPv6-only host); documentation details; more informative debugging Sending spamd/spamd.raw Committed revision 1451230.
(In reply to comment #8) > What would still be desired is: > - provide round-robin scheduling in a select() to avoid one > busy socket monopolizing the server Btw, this has now been taken care of by revision 1563172 (Bug 6996, 3.4.0). Spamd now randomly picks one of the ready sockets, in case there is more than one ready and we are using autonomous child processes (--round-robin). Not expected to be a very likely event, but better safe than sorry.