Anyone having problems with sending contact emails?

[nzyeehaa]

contact form uses senders email address as "from", which confuses dmarc filters...i.e: yahoo rejects mail not sent from its own domain, so anyone sending a contact form with yahoo address is rejected, possibly microsoft and gmail would go to spam due to their respective DMARC policies

yahoo: v=DMARC1; p=reject; pct=100; rua=mailto:dmarc_y_rua@yahoo.com;
microsoft: v=DMARC1; p=reject; pct=100; rua=mailto:d@rua.agari.com; ruf=mailto:d@ruf.agari.com; fo=1
gmail: v=DMARC1; p=none; sp=quarantine; rua=mailto:mailauth-reports@google.com

so customer comes to example.com website, fills the contact form with customer@yahoo.com and yahoos DMARC check fails because email is originating from example.com and not yahoo.com... therefore - email is rejected

is best practice to use own address as "from"?
and use customer address as reply-to..?
e.g:...

$mailTo = mb_encode_mimeheader("$author")." <$email>";
$mailSubject = mb_encode_mimeheader($this->yellow->page->get("title"));
$mailHeaders = mb_encode_mimeheader("From: $author")." <$email>\r\n";
$mailHeaders = mb_encode_mimeheader("Reply-To: $name")." <$from>\r\n";

[nzyeehaa]

...not all email clients may respect the Reply-To field when we actually click "Reply"

[nzyeehaa]

From RFC 5322
The "Sender:" field specifies the mailbox of the agent responsible for the actual transmission of the message.
In all cases, the "From:" field SHOULD NOT contain any mailbox that does not belong to the author(s) of the message.

therefore...

$mailTo = mb_encode_mimeheader("$author")." <$email>";
$mailSubject = mb_encode_mimeheader($this->yellow->page->get("title"));
$mailHeaders = mb_encode_mimeheader("From: $name")." <$from>\r\n";
$mailHeaders = mb_encode_mimeheader("Sender: $author")." <$email>\r\n";
$mailHeaders = mb_encode_mimeheader("Reply-To: $name")." <$from>\r\n"; {optional}

may be the most politicaly correct version, and should work for the majority of mail clients

UPDATE...
this would be incorrect as shown here - see below for a fix

[nzyeehaa]

further to the above...
with identical mail server settings...

yellow v 7.6 / contact v 7.3 sends fine with yahoo address
yellow v 8.9 / contact v 8.7 still awaiting confirmation
yellow v 8.11 / contact v 8.8 does not send with yahoo address

[markseuffert]

Let me guess... your webmaster email address is your yahoo address?

If this is true, then the problem is your webmaster email address and you can try the following:

1. If your website is hosted at http://example.com then use an email address with the same domain name, for example webmaster@example.com. Many service providers run both web server and mail server inside their own infrastructure, so there should never be any problem with mail sending and spam filtering. This is the recommended setup for the contact extension.
2. If your website is hosted on a free hosting provider or one that doesn't come with an email address, then use a different email service than Yahoo. It's probably time to face the unpleasant news, that another service provider would work better.
3. Ask your email service provider if there's an authentication token that you can add to outgoing email headers. You have to extend the contact extension yourself or suggest the authentication token as a new idea. We don't have such a feature yet.

Let us know what you find out, hope this helps.

[nzyeehaa]

@markseu hi mark, thanks for taking the time to respond.

• over three tested sites,
• all are on the same server/ IP
• all use the same mail server / IP (they are all different domains)
• all sites have identical server/ mail server setup
• all share the same webmaster address,
• all have an additional and different domain based email address setup for the contact form.
• all sites use DKIM SPF and DMARC with comparable settings
• all sites use different yellow and contact versions

• currently one site fails if a site visitor uses the contact form and has a yahoo address (Yellow 8.11 / contact 8.8)
• another site is awaiting a response from testing, but there has been no bounce email (yellow 8.9 / contact 8.7)
• another site works fine with yahoo email address (yellow 7.6 / contact 7.3)

• I adjusted the failing site with...
  $mailHeaders = mb_encode_mimeheader("From: $author")." <$email>\r\n"; $mailHeaders = mb_encode_mimeheader("Reply-To: $name")." <$from>\r\n";
  and while this is not technically correct implementation - there has been no bounce email using a yahoo address.

my reasoning suggests there is something in the contact form which is contributing to the email rejection, or moreover something that could be further optimised.

Yahoo has made bold use of DMARC rejection, as has microsoft, and even gmail "pushed the envelope" for a while.
part of the bounce message says
Unauthenticated email from yahoo.com is not accepted due to domain's DMARC policy. Please contact the administrator of yahoo.com domain if this was a legitimate mail.

in the case of the contact form - the mail of course does not originate from yahoo... the mail is originating through a private domain

bear in mind this is one visitor to the site who happens to have a yahoo address ~ therefore every visitor using yahoo email will fail

[markseuffert]

I think the problem is the webmaster email address, not the visitor email address. I recommend to configure a non-yahoo email address for the webmaster in the settings, use an email address that matches the domain name of your website. You can also make your own contact extension, with workarounds for a specific email service provider. Please contact your email service provider and ask for recommendations.

I recommend to configure a webmaster email address that matches the domain name of your website and to not forward emails across domain names. As long as you keep emails inside your own infrastructure, there shouldn't be any problems at all. Even if you're sending emails outside of your own infrastructure, there shouldn't be many problems.

Here are some explanations with technical details. Let's assume you configure a webmaster email address that doesn't match the domain name of your website, then emails might be transferred to another email server, which might apply filters to reject incoming emails. Let's assume you configure your webmaster email address to be forwarded across domain names, for example to Gmail or Yahoo, then emails will be first accepted by your own mail server, so far so good, but later transferred to another email server and pretty much the same problem arises. Once emails leave your own infrastructure, things get more complicated and mail filtering rules become more aggressive. Policies like those from Yahoos break traditional email forwarding. They must have been really desperate when they introduced these policies and the responses at the time were not positive. In theory you could use headers to convince your MTA/MUA that you're sending emails in behalf of someone else. The approach we are using is not pretending to be sending emails as someone else. We are sending emails as ourselves and then state inside the email that we have a contact message from someone else. Hope this design works better in practice. Tested with Datenstrom Yellow 0.8.20.

Edit 2020-09-04: Updated recommended setup and added technical details.

Edit 2022-05-17: Updated recommended setup and explained current approach.

[nzyeehaa]

@markseu
then you are wrong and you have missed some crucial information in the previous posts

the webmaster is a private domain
the visitor is a yahoo address

further information...

if ($this->yellow->page->isExisting("author") && !$this->yellow->system->get("contactEmailRestriction")) {
            $author = $this->yellow->page->get("author");
        }
        if ($this->yellow->page->isExisting("email") && !$this->yellow->system->get("contactEmailRestriction")) {
            $email = $this->yellow->page->get("email");
        }

the above overrides the system.ini webmaster email when an author and email are set on the contact page

author and email are set in the contact page. Authors email is a gmail address

The problem exists when author is using a gmail address and the visitor is using yahoo
when these conditions exist - the email is bounced
(spf has include:_spf.google.com)

'''
//$mailHeaders = mb_encode_mimeheader("From: $name")." <$from>\r\n";
$mailHeaders = mb_encode_mimeheader("From: $author")." <$email>\r\n";
$mailHeaders = mb_encode_mimeheader("Reply-To: $name")." <$from>\r\n";
'''
fixes the initial problem and the mail is delivered, but it is not technically correct

the way it was originally set is technically correct - however a visitor using yahoo is the sender of the mail on another domain therefore it is unauthenticated, and it is rejected by yahoo's DKIM / SPF / DMARC policies
and i suspect the same will happen with microsoft, but havent tested it

There is no way to keep it technically correct and solve the problem...
this is a problem with webforms in general and the current state of anti-spam measures.

however the problem exists, and it will continue to affect yellow users.
with 3.9-4.5 billion email users in the world - at yahoo's famous data breach in 2013 - they had 3 billion accounts
even if there are only 10% of users that now have a yahoo address - it could still be 10% of your custom gone, and a bad reputation building.

drupal have an implementation to change any 'from' email to a root domain -no-reply email.
this also wont be technically correct - but it will aid in deliverability of webforms

[markseuffert]

Have you tried to set an email that matches the domain name of your website?

[nzyeehaa]

I have tried 19 hours worth of experiments to diagnose and find a way past it, as i have customers losing business, and bounce emails backing up.

in the case quoted...
i can setup an email on the domain / server to receive the contact form email, and add a forwarder to gmail = fails
i can setup an email on the domain / server to receive the contact form email, and have gmail use pop3 to retrieve it = soft fails / spam
i can change "From" to a server domain (or Gmail) and set up "reply-to" and it works

$mailHeaders = mb_encode_mimeheader("From: $author")." <$email>\r\n";
$mailHeaders = mb_encode_mimeheader("Reply-To: $name")." <$from>\r\n";

yahoo suggests

If you are sending on behalf of individuals, we recommend that you send mail from your own domain. You can set a Reply-To: header with their email address so that people can reply to the sharer instead of to you. Let’s pretend you are “sharing.example”:

Reply-to: “Example Sender” <example-sender@yahoo.com>

From: “Example Sender” <noreply@sharing.example>

again - while this is not technically correct as per RFC,
both yahoo and microsoft will reject mail with their domains - that are not originating from their servers.

from a technical standpoint - the way the contact form is setup is correct.
from the evolving DMARC restrictions - change is urgently needed - or lose the contact form

[markseuffert]

Have you tried to not forward emails -or- only to an email that matches the domain name of your website?

[nzyeehaa]

yes, and in one word - deliverability

i live in the country.
if i send a letter to mark in sweden, i write the letter, sign the letter, put my return address on the back, put it in my (domain)mailbox and put a flag up.
the mailman comes, sees the flag, gets the letter, takes the letter to the post office, who sends it to sweden's post office for another mailman to deliver to mark, mark reads the mail and sees it is signed by me, and has my return address to reply to.(or if mark has moved - the letter can be returned to me so I know mark has moved and the letter was not delivered)

If i use a webform to write a letter, and it can not go anywhere because i have signed the letter (a signature someone doesnt like) and put my return address on it, the mailman can not pick up the letter, the letter can not get to sweden, mark can not get the letter and the entire system is broken.

for this webform - we are only talking about the signature on the letter and the return address.

all yahoo/microsoft/gmail is saying is that we cant use their pen to write the letter, and we cant say we are sending the letter through their post office (unless we are)- thats all fair enough.

essentially - a visitor to a website/webform is using the (domains) pen to write a letter and they put their own reply address on it...
simple...

the rest should be automatic and gauranteed - the letter should be delivered no matter what colour the mailbox is, who the mailman is, what brand of mail-car the mail man drives, or what language the post office speaks.

{spam excepted}

[nzyeehaa]

When you are sending email using mail() function in PHP, the mail is sent from your web server. Sometimes it may cause issues on sending an email and fails to deliver mail to the recipient. When you send an email via SMTP, email is sent from the mail server rather than the web server.

according to the guy that made swiftmail...

In my experience -- and others' -- the mail() function is not particularly predictable, or helpful.

Quite notably, the mail() function behaves entirely differently between Linux and Windows servers. On linux it uses sendmail, but on Windows it uses SMTP. ...

Serious drawbacks when using this "mail()" Transport are:

Unpredictable message headers
Lack of feedback regarding delivery failures
Lack of support for several plugins that require real-time delivery feedback
It's a last resort, and we say that with a passion!

therefore in order to fix the non-deliverability / non-forwardability of the contact form...

• you would need to use authenticated SMTP details inputed for webmaster or author and use SMTP transport to activate the MAIL server
  OR
• the "from" field needs to contain two emails, one from the form filler and one from the author or webmaster and a sender field would then contain a single email of webmaster or author see this post
  - the "from" field would need to be from the originating domain - NOT the form fillers email address
  OR
• use MSMTP or swiftmailer or zend_mail or PEAR or CURL

and considering the last options require modifications to the server, you would need one of the first two.

ref:
https://www.lifewire.com/send-email-from-php-script-4764146
https://www.codexworld.com/send-html-email-php-gmail-smtp-phpmailer/
https://github.com/PHPMailer/PHPMailer
http://reusableforms.com/d/b/phpmailer-contact-form

[GiovanniSalmeri]

you would need to use authenticated SMTP details inputed for webmaster or author and use SMTP transport to activate the MAIL server
I.E: use PHPmailer

The extension Mailer also allows to use SMTP (or sendmail, or mail), more or less like PHPmailer. It is not extensively tested, though.

[nzyeehaa]

@GiovanniSalmeri you may not have heard it, but I gave you a standing ovation when you created that, love your work.

IMO mail() may be ok for testing but SMTP SPF DMARC is required for live situations to ensure deliverability and reduce spam.

I got part way through creating one when yours turned up, but since covid arrived - I havent had time for anything, not even to test yours, but hope to soon.

[annaesvensson]

For your information, the following has been discussed on Discord.

The core has been updated and there's a new event handler onMail(). Now it should be possible to write an extension that sends emails via an alternative transport. SMTP comes to mind. Perhaps someone wants to split Giovanni's mailer extension and move the SMTP part into an own extension? Then all email sending extensions can take advantage of the alternative transport, not just the contact extension. Hopefully this solves the remaining problems with mail server setups.

[GiovanniSalmeri]

Perhaps someone wants to split Giovanni's mailer extension and move the SMTP part into an own extension?

I hope to do that myself in the next few days, I am glad that a part of my old extension can be useful!

[annaesvensson]

I think you would make a couple of users happy, would be lovely to have a SMTP extension in the future.

[nzyeehaa]

If you’ve created a PHP mail form and find it’s not sending email, then it’s most often due to the FROM address the form is using in its headers.

Emails should not be sent with a FROM address hosted somewhere else. For example:

user@hotmail.com
user@comcast.net
user@gmail.com

For example, if you’re sending from a site called ‘example.com’, the FROM address in your PHP Mail form must be something like admin@example.com.

REF: dreamhost

[schulle4u]

The contact extension has been updated with new headers for SPF. Hopefully this will work better in some setups.

[nzyeehaa]

@schulle4u thanks for the updates,
I havent tested it, but I have some concerns...

• From: sitename<noreply> isnt a valid email address so will it fail in some cases?
  vs: From: sitename<noreply@mywebsite.com>

• I have posted an extensive review of an earlier post below... but a summary

  RFC
  The "From:" field specifies the author(s) of the message...the mailbox(es) of the person(s) or system(s) responsible for the writing of the message.
  The "Sender:" field specifies the mailbox of the agent responsible for the actual transmission of the message.
  If the "From:" field contains more than one mailbox ... then the sender field... and a single mailbox MUST appear

therefore this may resolve it...
(subject to testing)

$mailTo = mb_encode_mimeheader("$author")." <$email>";
$mailSubject = mb_encode_mimeheader($this->yellow->page->get("title"));
$mailHeaders = mb_encode_mimeheader("From: $name")." <$from>,<$email>\r\n";
$mailHeaders = mb_encode_mimeheader("From: ")." <$from>,<$email>\r\n";
$mailHeaders .= mb_encode_mimeheader("Sender: $author")." <$email>\r\n";
$mailHeaders .= mb_encode_mimeheader("Reply-To: $name")." <$from>\r\n";

UPDATED...
in some cases microsoft servers dont like names in the "from:" field, they only want to see email addresses - or it may fail before handover to php

Caveat:

RFC
If the originator of the message can be indicated by a single mailbox and the author and transmitter are identical, the "Sender:" field SHOULD NOT be used.

therefore someone testing their own mail may experience bad results, and this may need an IF( a=b){ then }

[nzyeehaa]

works fine on local mail server testing,
"From:" is supplanted with "sender" and "reply to"

[annaesvensson]

That extension is now at https://github.com/annaesvensson/yellow-contact

[nzyeehaa]

as per #542 (comment) above...

possibly the reason $mailHeaders = mb_encode_mimeheader("Sender: $author")." <$email>\r\n"; did not work in one instance, is because the "from:" field would need to be a list of emails as per RFC. which then enables sender field to be activated.

The folowing may work but is untested...
(using old contact extension code)

$mailTo = mb_encode_mimeheader("$author")." <$email>";
$mailSubject = mb_encode_mimeheader($this->yellow->page->get("title"));
$mailHeaders = mb_encode_mimeheader("From:")." <$from>,<$email>\r\n";
$mailHeaders = mb_encode_mimeheader("Sender: $author")." <$email>\r\n";
$mailHeaders = mb_encode_mimeheader("Reply-To: $name")." <$from>\r\n"; {optional}

and the relevant RFC...

3.6.2. Originator fields

The originator fields of a message consist of the from field, the
sender field (when applicable), and optionally the reply-to field.
The from field consists of the field name "From" and a
comma-separated list of one or more mailbox specifications. If the
from field contains more than one mailbox specification in the
mailbox-list, then the sender field, containing the field name
"Sender" and a single mailbox specification, MUST appear in the
message. In either case, an optional reply-to field MAY also be
included, which contains the field name "Reply-To" and a
comma-separated list of one or more addresses.

from = "From:" mailbox-list CRLF

sender = "Sender:" mailbox CRLF

reply-to = "Reply-To:" address-list CRLF

The originator fields indicate the mailbox(es) of the source of the
message. The "From:" field specifies the author(s) of the message,
that is, the mailbox(es) of the person(s) or system(s) responsible
for the writing of the message. The "Sender:" field specifies the
mailbox of the agent responsible for the actual transmission of the
message. For example, if a secretary were to send a message for
another person, the mailbox of the secretary would appear in the
"Sender:" field and the mailbox of the actual author would appear in
the "From:" field. If the originator of the message can be indicated
by a single mailbox and the author and transmitter are identical, the
"Sender:" field SHOULD NOT be used. Otherwise, both fields SHOULD
appear.

The originator fields also provide the information required when
replying to a message. When the "Reply-To:" field is present, it
indicates the mailbox(es) to which the author of the message suggests
that replies be sent. In the absence of the "Reply-To:" field,
replies SHOULD by default be sent to the mailbox(es) specified in the
"From:" field unless otherwise specified by the person composing the
reply.

In all cases, the "From:" field SHOULD NOT contain any mailbox that
does not belong to the author(s) of the message. See also section
3.6.3 for more information on forming the destination addresses for a
reply.