Page MenuHomePhabricator

Historic Phriction documents do not have mailKeys populated
Closed, ResolvedPublic

Description

I'm getting an exception message (see below), when trying to change existing Phriction document content (100% english content, but in any case content should be saved) to this new one (see below).

Exception message:

Attempting to construct a query containing characters outside of the Unicode Basic Multilingual Plane. MySQL will silently truncate this data if it is inserted into a `utf8` column. Use the `%B` conversion to escape binary strings data.

Page Content:

# commit is created by a **developer** on a team
# commit is then audited by **auditor**
# if no suspicious code is found, then **auditor** approves commit
# if suspicious code is found, then **auditor** raises a concern (from commit page) and **developer** gets e-mail notification
# **developer** must take action **within a week** upon receiving such notification:
 * ask **auditor** a question by replying to individual comment (in code or above file list)
 * commit a fix and specify original commit (e.g. `[fixes: rESE7281] `) at the first line (along with task number) of his commit message

If there are multiple unresolved audits, then **developer** must process them by FILO logic (most recent audit get processed first).

== What is checked in audit ==

# matches coding standards
# performs task in most efficient way
# doesn't duplicate any existing code
# ...

P.S.
@alex audits all commits by default, but anyone can voluntarily audit a commit.

== Recommended E-mail Preferences ==
It's recommended to change e-mail preferences (http://qa.intechnic.com/settings/panel/emailpreferences/) to these:
{F93}

I've recently did ./bin/storage adjust and some columns were converted to support UTF-8 emotions and maybe this is the cause.

Event Timeline

aik099 raised the priority of this task from to Needs Triage.
aik099 updated the task description. (Show Details)
aik099 added a project: Phriction.
aik099 updated the task description. (Show Details)
aik099 added a subscriber: aik099.

This will be fixed by D10798, which is waiting for D10800.

Is there anything wrong in the content I'm trying to save? I've executed all storage adjustments, so I shouldn't be getting this error at all.

Ah, now I get it after reading description of D10798. I shouldn't have executed storage adjustments, because that resulted in that exception in first place.

I'm not sure what's causing the issue, but also can't reproduce it by reverting D10798 locally:

Screen_Shot_2014-11-07_at_4.15.14_AM.png (1×1 px, 250 KB)

(I can reproduce by reverting D10798 and trying to edit the page to include emoji, as expected.)

I create page some time ago. Then made storage adjustments recently and tried to edit page content and got that error. In fact if I even add one word at the end I'm getting that error.

I have 2 installs and on one it's working, but on another it's not.

I've just verified that all existing documents before storage adjust are not editable, even if I just have "test" word in them. I can create new and edit these new documents without issues.

So it's storage adjust, who did this to existing documents.

Maybe storage adjust was changing collation of columns without actually converting content inside them to use that new collation?

It's like having UTF-8 text stored with latin1_swedish_ci collation and then change the column collation to utf8_general_ci - text will be mess.

epriestley renamed this task from Getting "AphrontCharacterSetQueryException" when attempting to save Phriction page to Historic Phriction documents do not have mailKeys populated.Nov 7 2014, 12:28 PM
epriestley claimed this task.
epriestley triaged this task as Normal priority.
epriestley added a subscriber: btrahan.

Ah, this is D10756.

And these mailKeys cause document save exception, how so?

Specifically:

  • D10756 added mailKey.
  • But does not retroactively assign mail keys.
  • After adjustment, the column type is bytes20, which will fill with "\0\0\0\0..." if not filled.
  • That's not valid BMP utf8.
  • D10756 added it as varchar(20) (equivalent to text20, not bytes20) so no issue. It was still incorrect, but not as problematic.