Page MenuHomePhabricator

Mail handler unable to parse e-mail from MS Outlook 12, Failed to JSON encode value
Closed, WontfixPublic

Description

If a regular mail is received from Outlook 12, Phabricator is unable to parse it and crashed with the following error message:

 # cat pfmail1.txt | /storage/web/dazzler.sk/phabricator/phabricator/scripts/mail/mail_handler.php
[2017-05-18 20:05:37] EXCEPTION: (Exception) Failed to JSON encode value (#5: Malformed UTF-8 characters, possibly incorrectly encoded): Dictionary value at key "html" is not valid UTF8, and cannot be JSON encoded: <html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta h. at [<phutil>/src/utils/utils.php:1194]
arcanist(head=master, ref.master=3c4735795a29), phabricator(head=master, ref.master=75fb1a0327cb), phutil(head=master, ref.master=a900d7b63e95)
  #0 phutil_json_encode(array) called at [<phabricator>/src/infrastructure/storage/lisk/LiskDAO.php:1701]
  #1 LiskDAO::applyLiskDataSerialization(array, boolean) called at [<phabricator>/src/infrastructure/storage/lisk/LiskDAO.php:1359]
  #2 LiskDAO::willWriteData(array) called at [<phabricator>/src/infrastructure/storage/lisk/PhabricatorLiskDAO.php:315]
  #3 PhabricatorLiskDAO::willWriteData(array) called at [<phabricator>/src/infrastructure/storage/lisk/LiskDAO.php:1233]
  #4 LiskDAO::insertRecordIntoDatabase(string) called at [<phabricator>/src/infrastructure/storage/lisk/LiskDAO.php:1106]
  #5 LiskDAO::insert() called at [<phabricator>/src/infrastructure/storage/lisk/LiskDAO.php:1075]
  #6 LiskDAO::save() called at [<phabricator>/scripts/mail/mail_handler.php:92]

This e-mail was created in Outlook. It is correctly shown in Gmail, for example (I'm uploading copy of this e-mail which was sent to my Gmail, so you can reproduce the problem).

Event Timeline

The problem is a outdated version of phabricator/externals/mimemailparser/MimeMailParser.class.php.

A patch is attached, unfortunatelly I 'arc diff-ed' it in same revision as other function here:
https://secure.phabricator.com/D17989

Hi,

is there a chance that this problem will get fixed?
The change D17989 is already from May 2017 without any reviewer...

epriestley claimed this task.
epriestley added a subscriber: epriestley.

This is almost certainly "PhabricatorMetaMTAReceivedMail incorrectly assumes email is valid UTF8", and the solution begins by making all the associated storage binary, but it's hard to imagine that ever happening now.