With Ivo unavailable, I don't know that we have a huge amount of input available.  However, as a volunteer project, we do welcome patches if you have thoughts on how it should work.

Well, regarding of how it should work, here is my vision of the algoryhtm that solving this issue: 

- For the first of all the records addded by outgoing whitelist feature should be stored separately in the db and not mixed to other records.

- For example field "ip" in the case of outgoing white listing should have some personal value like "WHITEOUT" that identifies this records as outgoing whitelist record type.

- The "signedby" field fot such records should be always blank and not depend on auto_whitelist_distinguish_signed value.

- When analyzing new incoming messages and finding DB for the records we should check for the whiteout records first: try to find the records that hit the local user name (or overrided user name if set) and email fields (correspondent to currently analyzing message) AND field "ip" = WHITEOUT. such records should hit the saerch result not depending on "signedby" field and nevermind of dkim signature of the email being analyzed. The checking of the WHITEOUT records should be done separately from checking all other records (and of course when analyzing other records that are done now - such WHITEOUT records should be ignored to prevent double hits).

- WHITEOUT records scores values can be changed when analyzing new incoming and ourgoing mail as done now with generic records.

what do you think about it ?

According to the docs, auto_whitelist_distinguish_signed was added solely for 3.3.0 database compatibility:

 auto_whitelist_distinguish_signed
         (default: 1 - enabled)

        Used by the SQLBasedAddrList storage implementation.

        If this option is set the SQLBasedAddrList module will keep separate
        database entries for DKIM-validated e-mail addresses and for
        non-validated ones. A pre-requisite when setting this option is that
        a field awl.signedby exists in a SQL table, otherwise SQL operations
        will fail (which is why we need this option at all - for
        compatibility with pre-3.3.0 database schema). A plugin DKIM should
        also be enabled, as otherwise there is no benefit from turning on
        this option.


So if you are using 3.4.X, I'm expecting this option to be off and perhaps the database key on email is incorrect which explains the duplicate SQL errors as well.

So yes, I think the whitelisting of outbound only with this feature disabled makes sense.  It was a transitional option from what I gather.

(In reply to Kevin A. McGrail from comment #3)
> According to the docs, auto_whitelist_distinguish_signed was added solely
> for 3.3.0 database compatibility:
...
>         If this option is set the SQLBasedAddrList module will keep separate
>         database entries for DKIM-validated e-mail addresses and for
>         non-validated ones. A pre-requisite when setting this option is that
>         a field awl.signedby exists in a SQL table, otherwise SQL operations
>         will fail (which is why we need this option at all - for
>         compatibility with pre-3.3.0 database schema). A plugin DKIM should
Sorry, but I think that it means that pre-3.3.0 db doesn't have the field "signed by" and this option should be set to zero when using old scheme (turned off). So I think the documentation means "why we need this options to be TURNED OFF". By default this option is turned on (for the latest DB that has signedby field).
the DKIM separate records is a feature that helps to prevent of trashing the score for good senders when spammers use someone's "from" field but can not provide the users correct DKIM signature. when this option is not used and spammer user the email of some good user we have bad reputation for this good user just because spamnmer used his email and this is not correct, we will score the good emails with this incorrect score. For example an intruder can generate much spam and use email of a victim, the system will learn that this email is spam by txrep. then our victim writes us a good email but it is banned by txrep.
so I think this is a feature to prevent such security holes in spam protection.. and it can be turned off for old db when there were no support of this feature in txrep/awl.

(In reply to Andrew from comment #4)
> (In reply to Kevin A. McGrail from comment #3)
> > According to the docs, auto_whitelist_distinguish_signed was added solely
> > for 3.3.0 database compatibility:
> ...
> >         If this option is set the SQLBasedAddrList module will keep separate
> >         database entries for DKIM-validated e-mail addresses and for
> >         non-validated ones. A pre-requisite when setting this option is that
> >         a field awl.signedby exists in a SQL table, otherwise SQL operations
> >         will fail (which is why we need this option at all - for
> >         compatibility with pre-3.3.0 database schema). A plugin DKIM should
> Sorry, but I think that it means that pre-3.3.0 db doesn't have the field
> "signed by" and this option should be set to zero when using old scheme
> (turned off). So I think the documentation means "why we need this options
> to be TURNED OFF". By default this option is turned on (for the latest DB
> that has signedby field).
> the DKIM separate records is a feature that helps to prevent of trashing the
> score for good senders when spammers use someone's "from" field but can not
> provide the users correct DKIM signature. when this option is not used and
> spammer user the email of some good user we have bad reputation for this
> good user just because spamnmer used his email and this is not correct, we
> will score the good emails with this incorrect score. For example an
> intruder can generate much spam and use email of a victim, the system will
> learn that this email is spam by txrep. then our victim writes us a good
> email but it is banned by txrep.
> so I think this is a feature to prevent such security holes in spam
> protection.. and it can be turned off for old db when there were no support
> of this feature in txrep/awl.


Then have you set option to 0 when using a DB design from 3.4.1?

Yes, sure, it is working when I turn off this feature. In that case The field signedby is empty in all of the records (compatibility mode with old db). But it is better to use Txrep with this feature because of the holes that I described in previous post

(In reply to Andrew from comment #6)
> Yes, sure, it is working when I turn off this feature. In that case The
> field signedby is empty in all of the records (compatibility mode with old
> db). But it is better to use Txrep with this feature because of the holes
> that I described in previous post

Did you install DKIM?  I believe that's needed for the pre-db design to populate singnedby.

(In reply to Kevin A. McGrail from comment #7)
> (In reply to Andrew from comment #6)
> Did you install DKIM?  I believe that's needed for the pre-db design to
> populate singnedby.
yes, sure. and DKIM is looks like working in TxRep. The incoming mail is getting signedby fields whet I turn on this feature. BUT the outgoing white list
 scores are not recognised because outgoin mail can not have the DKIM signature of the remote address (the address that is added to the whitelist. it should be signed by the SENDER's server and its key.). the outgoing mail can only be signed by LOCAL (our domain) signature that is not related to the remote address. local dkim signature is useless for the spam checking and should not be assigned to the remote addr.. Actually in my configuration the DKIM signs the outgoing mail AFTER processing with SA. and it is okay. So the current logics for the DKIM isolation of the records and the auto whitelist outgoing mail is broken cause we can not sign outgoing mail with someone's key. That is why I proposed the algorythm where outgoing whitelist records should be processed separately ignoring the signedby field

Sending        lib/Mail/SpamAssassin/Plugin/TxRep.pm
Sending        lib/Mail/SpamAssassin/SQLBasedAddrList.pm
Transmitting file data ..done
Committing transaction...
Committed revision 1909609.