Mangled "From" field in mailing list posts

newer
[poly_collection] Help with Intel...

Vladimir Prus

10 Jun 2017 10 Jun '17

8:28 p.m.

Hi, it seems that most of recent posts to the mailing lists have rather strange "From" field, e.g. From: Daniel James via Boost I see this problem both via NTTP and regular mail, including emails that appear to be posted directly with a mail client. That extra "via Boost" adds no value, and clutters mailbox. Is there any change to return back to having "From" field have just the name of the person? Thanks in advance, Volodya

Show replies by date

Olaf van der Spek

11 Jun 11 Jun

7:19 a.m.

On Sat, Jun 10, 2017 at 10:28 PM, Vladimir Prus via Boost wrote:

...

Hi,

it seems that most of recent posts to the mailing lists have rather strange "From" field, e.g.

From: Daniel James via Boost

I see this problem both via NTTP and regular mail, including emails that appear to be posted directly with a mail client.

That extra "via Boost" adds no value, and clutters mailbox. Is there any change to return back to having "From" field have just the name of the person?

+1 -- Olaf

Daniel James

8:33 a.m.

On 10 June 2017 at 21:28, Vladimir Prus via Boost wrote:

...

Hi,

it seems that most of recent posts to the mailing lists have rather strange "From" field, e.g.

From: Daniel James via Boost

I see this problem both via NTTP and regular mail, including emails that appear to be posted directly with a mail client.

That extra "via Boost" adds no value, and clutters mailbox. Is there any change to return back to having "From" field have just the name of the person?

Problem is that would link my name with the mailing list address. Email programs often automatically fill their address books from emails.

Stefan Seefeld

1:48 p.m.

...

On 10 June 2017 at 21:28, Vladimir Prus via Boost wrote:

...
Hi,

it seems that most of recent posts to the mailing lists have rather strange "From" field, e.g.

From: Daniel James via Boost

I see this problem both via NTTP and regular mail, including emails that appear to be posted directly with a mail client.

That extra "via Boost" adds no value, and clutters mailbox. Is there any change to return back to having "From" field have just the name of the person? Problem is that would link my name with the mailing list address. Email programs often automatically fill their address books from emails. The "From:" field could contain the full address of the original poster, not just his name. That's how things were before the change, IIUC. But, AFAIU, that had to change because some mail servers would refuse to serve mail whose "From:" address differed from the "sender" field (which is the list address in our case). Am I describing this correctly ? I wonder how others handle this situation (in particular, how mailman and similar tools deal with this themselves), given how frequent a use-case

On 11.06.2017 04:33, Daniel James via Boost wrote: this is... Stefan -- ...ich hab' noch einen Koffer in Berlin...

Rene Rivera

2 p.m.

On Sun, Jun 11, 2017 at 8:48 AM, Stefan Seefeld via Boost < boost@lists.boost.org> wrote:

...

On 11.06.2017 04:33, Daniel James via Boost wrote:

...
On 10 June 2017 at 21:28, Vladimir Prus via Boost wrote:

...
Hi,

it seems that most of recent posts to the mailing lists have rather strange "From" field, e.g.

From: Daniel James via Boost

I see this problem both via NTTP and regular mail, including emails that appear to be posted directly with a mail client.

That extra "via Boost" adds no value, and clutters mailbox. Is there any change to return back to having "From" field have just the name of the person?

...

The "From:" field could contain the full address of the original poster, not just his name. That's how things were before the change, IIUC. But, AFAIU, that had to change because some mail servers would refuse to serve mail whose "From:" address differed from the "sender" field (which is the list address in our case). Am I describing this correctly ? I wonder how others handle this situation (in particular, how mailman and similar tools deal with this themselves), given how frequent a use-case this is...

They deal with it by not setting "Sender:" at all. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail

Stefan Seefeld

2:07 p.m.

On 11.06.2017 10:00, Rene Rivera via Boost wrote:

...

...
The "From:" field could contain the full address of the original poster, not just his name. That's how things were before the change, IIUC. But, AFAIU, that had to change because some mail servers would refuse to serve mail whose "From:" address differed from the "sender" field (which is the list address in our case). Am I describing this correctly ? I wonder how others handle this situation (in particular, how mailman and similar tools deal with this themselves), given how frequent a use-case this is...

They deal with it by not setting "Sender:" at all.

Then why wouldn't that work for us ? Stefan -- ...ich hab' noch einen Koffer in Berlin...

Olaf van der Spek

12 Jun 12 Jun

7:30 a.m.

On Sun, Jun 11, 2017 at 4:00 PM, Rene Rivera via Boost wrote:

...

They deal with it by not setting "Sender:" at all.

Doesn't that run into problems with SPF? -- Olaf

Michael Caisse

8:32 p.m.

On 6/11/17 06:48, Stefan Seefeld via Boost wrote:

...

The "From:" field could contain the full address of the original poster, not just his name. That's how things were before the change, IIUC. But, AFAIU, that had to change because some mail servers would refuse to serve mail whose "From:" address differed from the "sender" field (which is the list address in our case). Am I describing this correctly ? I wonder how others handle this situation (in particular, how mailman and similar tools deal with this themselves), given how frequent a use-case this is...

Stefan

With the old system, many people were having issues with DMARC filtering emails as-if they were spoof'd. In the recent couple years many corporate accounts have moved to utilize DMARC as part of their inbound authentication and the popularity continues to increase. Unfortunately, Mail Lists normally break because the original sender's domain DKIM signature doesn't match the Mail List. The most popular work around is rewriting the From header field. We are doing that in the most basic manner. We can consider some other possible changes; however, it is impolite to modify the From field with an address that would indicate an individual but represents the entire list. -- Michael Caisse Ciere Consulting ciere.com

Vladimir Prus

13 Jun 13 Jun

8:22 a.m.

On 12/06/2017 23:32, Michael Caisse via Boost wrote:

...

On 6/11/17 06:48, Stefan Seefeld via Boost wrote:

...
The "From:" field could contain the full address of the original poster, not just his name. That's how things were before the change, IIUC. But, AFAIU, that had to change because some mail servers would refuse to serve mail whose "From:" address differed from the "sender" field (which is the list address in our case). Am I describing this correctly ? I wonder how others handle this situation (in particular, how mailman and similar tools deal with this themselves), given how frequent a use-case this is...

Stefan

With the old system, many people were having issues with DMARC filtering emails as-if they were spoof'd. In the recent couple years many corporate accounts have moved to utilize DMARC as part of their inbound authentication and the popularity continues to increase.

Unfortunately, Mail Lists normally break because the original sender's domain DKIM signature doesn't match the Mail List. The most popular work around is rewriting the From header field. We are doing that in the most basic manner.

Hi Michael, thanks for the explanation. So, if I understand correctly, the problem is that some *senders* have their domains configured to ask recipients to reject emails that don't pass DKIM or SPF? In other words, the question is not how many organizations have DMARC for inbound authentication, but how many users are sending emails to a mailing list (which, by definition, forwards email with modifications) while also requesting than any forwared with modifications emails are rejected by recipients? How many such sending users/domains do we have? I personally think it would be reasonable to just require that posters don't use such domain configuration. If that's not possible, can't we make Mailman not add any footers, and don't add any DKIM signature of its own. Maybe, that will cause original DKIM signature to remain valid and DMARC check to pass? Thanks, Volodya

Olaf van der Spek

11:59 a.m.

On Tue, Jun 13, 2017 at 10:22 AM, Vladimir Prus via Boost wrote:

...

thanks for the explanation. So, if I understand correctly, the problem is that some *senders* have their domains configured to ask recipients to reject emails that don't pass DKIM or SPF? In other words, the question is not how many organizations have DMARC for inbound authentication, but how many users are sending emails to a mailing list (which, by definition, forwards email with modifications) while also requesting than any forwared with modifications emails are rejected by recipients? How many such sending users/domains do we have?

I personally think it would be reasonable to just require that posters don't use such domain configuration.

Doesn't gmail also use dmarc?

...

If that's not possible, can't we make Mailman not add any footers, and don't add any DKIM signature of its own. Maybe, that will cause original DKIM signature to remain valid and DMARC check to pass?

The original From header is still problematic AFAIK. -- Olaf

Vladimir Prus

14 Jun 14 Jun

6:22 a.m.

On 13/06/2017 14:59, Olaf van der Spek via Boost wrote:

...

On Tue, Jun 13, 2017 at 10:22 AM, Vladimir Prus via Boost wrote:

...
thanks for the explanation. So, if I understand correctly, the problem is that some *senders* have their domains configured to ask recipients to reject emails that don't pass DKIM or SPF? In other words, the question is not how many organizations have DMARC for inbound authentication, but how many users are sending emails to a mailing list (which, by definition, forwards email with modifications) while also requesting than any forwared with modifications emails are rejected by recipients? How many such sending users/domains do we have?

I personally think it would be reasonable to just require that posters don't use such domain configuration.

Doesn't gmail also use dmarc?

It does, as can be seen at https://dmarcian.com/dmarc-inspector/gmail.com But that configuration (p=none) asks recipient servers to report back to gmail about any problems they see, not drop email, whereas some other email providers have "p=reject".

...

...
If that's not possible, can't we make Mailman not add any footers, and don't add any DKIM signature of its own. Maybe, that will cause original DKIM signature to remain valid and DMARC check to pass?

The original From header is still problematic AFAIK.

It is my understanding that if you don't modify From header and don't modify body, then DKIM will pass and SPF will fail, and it's enough for one of the tests to pass. - Volodya

Michael Caisse

13 Jun 13 Jun

10:17 p.m.

On 6/13/17 01:22, Vladimir Prus via Boost wrote:

...

On 12/06/2017 23:32, Michael Caisse via Boost wrote:

...
On 6/11/17 06:48, Stefan Seefeld via Boost wrote:

...
The "From:" field could contain the full address of the original poster, not just his name. That's how things were before the change, IIUC. But, AFAIU, that had to change because some mail servers would refuse to serve mail whose "From:" address differed from the "sender" field (which is the list address in our case). Am I describing this correctly ? I wonder how others handle this situation (in particular, how mailman and similar tools deal with this themselves), given how frequent a use-case this is...

Stefan

With the old system, many people were having issues with DMARC filtering emails as-if they were spoof'd. In the recent couple years many corporate accounts have moved to utilize DMARC as part of their inbound authentication and the popularity continues to increase.

Unfortunately, Mail Lists normally break because the original sender's domain DKIM signature doesn't match the Mail List. The most popular work around is rewriting the From header field. We are doing that in the most basic manner.

Hi Michael,

thanks for the explanation. So, if I understand correctly, the problem is that some *senders* have their domains configured to ask recipients to reject emails that don't pass DKIM or SPF? In other words, the question is not how many organizations have DMARC for inbound authentication, but how many users are sending emails to a mailing list (which, by definition, forwards email with modifications) while also requesting than any forwared with modifications emails are rejected by recipients? How many such sending users/domains do we have?

I might have explained poorly. When the ML sends emails, it is the receiving side (inbound) that is doing the check. The receiving server confirms headers, checks the signature against what is in the original sender's domain entries and then fails the message. Some of the organizations/services that utilize DMARC: Microsoft, Yahoo, Pixar, any thing through Rackspace, and gmail. We are talking about some other solutions... but most of them are horrible or short lived until the entire world moves to DMARC. michael -- Michael Caisse Ciere Consulting ciere.com

Vladimir Prus

14 Jun 14 Jun

6:40 a.m.

On 14/06/2017 01:17, Michael Caisse via Boost wrote:

...

On 6/13/17 01:22, Vladimir Prus via Boost wrote:

...
On 12/06/2017 23:32, Michael Caisse via Boost wrote:

...
On 6/11/17 06:48, Stefan Seefeld via Boost wrote:

...
The "From:" field could contain the full address of the original poster, not just his name. That's how things were before the change, IIUC. But, AFAIU, that had to change because some mail servers would refuse to serve mail whose "From:" address differed from the "sender" field (which is the list address in our case). Am I describing this correctly ? I wonder how others handle this situation (in particular, how mailman and similar tools deal with this themselves), given how frequent a use-case this is...

Stefan

With the old system, many people were having issues with DMARC filtering emails as-if they were spoof'd. In the recent couple years many corporate accounts have moved to utilize DMARC as part of their inbound authentication and the popularity continues to increase.

Unfortunately, Mail Lists normally break because the original sender's domain DKIM signature doesn't match the Mail List. The most popular work around is rewriting the From header field. We are doing that in the most basic manner.

Hi Michael,

thanks for the explanation. So, if I understand correctly, the problem is that some *senders* have their domains configured to ask recipients to reject emails that don't pass DKIM or SPF? In other words, the question is not how many organizations have DMARC for inbound authentication, but how many users are sending emails to a mailing list (which, by definition, forwards email with modifications) while also requesting than any forwared with modifications emails are rejected by recipients? How many such sending users/domains do we have?

I might have explained poorly. When the ML sends emails, it is the receiving side (inbound) that is doing the check. The receiving server confirms headers, checks the signature against what is in the original sender's domain entries and then fails the message.

According to what I read, only if *sending side* requests to fail the message with "p=reject" in DMARC DNS entry. Is it not true? Also, according to what I read, if ML does not modify any headers and does not modify body, and does not add its own DKIM signature, then DKIM test will pass. Is it not true? At present, it seems that mailing list: - Adds footer (which breaks original DKIM signature) - Adds its own DKIM signature (in fact, two) - Modifies From header to "fix" things up. I am asking whether we've tried to configure Mailman to try not modifying anything at all, and act as close to perfect forwarding as possible.

...

Some of the organizations/services that utilize DMARC: Microsoft, Yahoo, Pixar, any thing through Rackspace, and gmail.

According to: https://dmarcian.com/dmarc-inspector/outlook.com https://dmarcian.com/dmarc-inspector/exchange.microsoft.com Microsoft has "p=none" as well. Gmail likewise. Only Yahoo has "p=reject".

...

We are talking about some other solutions... but most of them are horrible or short lived until the entire world moves to DMARC.

I am not 100% sure that Mailman can be configured to keep original DKIM signature valid (and seems like its developers don't know either), but it seems to me that loosing mailing list footer is better than mangling From field, and therefore worth a try? - Volodya

Olaf van der Spek

10:13 a.m.

On Wed, Jun 14, 2017 at 8:40 AM, Vladimir Prus via Boost wrote:

...

On 14/06/2017 01:17, Michael Caisse via Boost wrote: At present, it seems that mailing list: - Adds footer (which breaks original DKIM signature) - Adds its own DKIM signature (in fact, two) - Modifies From header to "fix" things up.

[boost] subject prefix -- Olaf

Michael Caisse

13 Jun 13 Jun

10:22 p.m.

On 6/13/17 01:22, Vladimir Prus via Boost wrote:

...

If that's not possible, can't we make Mailman not add any footers, and don't add any DKIM signature of its own. Maybe, that will cause original DKIM signature to remain valid and DMARC check to pass?

I meant to respond to this part too: This is what we used to do with the old ML when it was ran by IU. We were dropping thousands of emails a day because of DMARC and we had quite a number of complaints from individuals who used to be involved with the list but could no longer receive emails. As more companies are using DMARC, we will have to find a better solution. michael -- Michael Caisse Ciere Consulting ciere.com

Olaf van der Spek

14 Jun 14 Jun

10:15 a.m.

On Mon, Jun 12, 2017 at 10:32 PM, Michael Caisse via Boost wrote:

...

We can consider some other possible changes; however, it is impolite to modify the From field with an address that would indicate an individual but represents the entire list.

Is it? I get notifications from github with From: User Name It seems like the best solution. -- Olaf

Asbjørn

10:55 a.m.

On 14.06.2017 12:15, Olaf van der Spek via Boost wrote:

...

On Mon, Jun 12, 2017 at 10:32 PM, Michael Caisse via Boost wrote:

...
We can consider some other possible changes; however, it is impolite to modify the From field with an address that would indicate an individual but represents the entire list.

Is it?

I get notifications from github with From: User Name It seems like the best solution.

I think it would greatly increase the chance of someone mistakenly sending private mails to the list. That's not an issue with the github mail address. Regards - Asbjørn

Olaf van der Spek

11:01 a.m.

On Wed, Jun 14, 2017 at 12:55 PM, Asbjørn via Boost wrote:

...

On 14.06.2017 12:15, Olaf van der Spek via Boost wrote:

...
On Mon, Jun 12, 2017 at 10:32 PM, Michael Caisse via Boost wrote:

...
We can consider some other possible changes; however, it is impolite to modify the From field with an address that would indicate an individual but represents the entire list.

Is it?

I get notifications from github with From: User Name It seems like the best solution.

I think it would greatly increase the chance of someone mistakenly sending private mails to the list. That's not an issue with the github mail address.

Why not? If you reply it ends up in the public issue tracker. -- Olaf

Asbjørn

11:13 a.m.

On 14.06.2017 13:01, Olaf van der Spek via Boost wrote:

...

On Wed, Jun 14, 2017 at 12:55 PM, Asbjørn via Boost

...
I think it would greatly increase the chance of someone mistakenly sending private mails to the list. That's not an issue with the github mail address.

Why not? If you reply it ends up in the public issue tracker.

In that case it has the same issue, one which I'd consider a defect. YMMV. Regards - Asbjørn

2722

Age (days ago)

2726

Last active (days ago)

List overview

Download

18 comments

7 participants

participants (7)

Asbjørn
Daniel James
Michael Caisse
Olaf van der Spek
Rene Rivera
Stefan Seefeld
Vladimir Prus