Page MenuHomePhabricator

Make libphutil compatible with utf8mb4
ClosedPublic

Authored by epriestley on Sep 28 2014, 9:12 PM.
Tags
None
Referenced Files
F12827787: D10604.id25441.diff
Thu, Mar 28, 10:23 AM
Unknown Object (File)
Mon, Mar 25, 6:15 PM
Unknown Object (File)
Mon, Mar 25, 6:15 PM
Unknown Object (File)
Mon, Mar 25, 6:15 PM
Unknown Object (File)
Fri, Mar 22, 12:55 PM
Unknown Object (File)
Thu, Mar 21, 12:05 PM
Unknown Object (File)
Sat, Mar 16, 4:33 AM
Unknown Object (File)
Mon, Mar 4, 4:53 PM

Details

Summary

Ref T1191. Two changes here:

  • Try to use the utf8mb4 character set. Fall back to utf8 if it's not available. This may be slightly worse for old MySQL, but is far, far easier than trying to keeep a cache around somewhere with availability information.
  • Prepare to transition to utf8mb4. I'm just going to switch this over in the future. I considered trying to save the migration state in the environment somewhere, but it gets really, really messy because we have to inspect the migration state in order to build a connection object. There's also no advantage in the overwhelming majority of cases. We can do something more messy here eventually if it ends up being more complicated than I think.
Test Plan

Wrote comments full of emoji locally.

Diff Detail

Repository
rPHU libphutil
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

epriestley retitled this revision from to Make libphutil compatible with utf8mb4.
epriestley updated this object.
epriestley edited the test plan for this revision. (Show Details)
epriestley added a reviewer: btrahan.
btrahan edited edge metadata.
btrahan added inline comments.
src/aphront/storage/connection/mysql/AphrontBaseMySQLDatabaseConnection.php
339

there's an extra %

This revision is now accepted and ready to land.Sep 30 2014, 4:56 PM
src/aphront/storage/connection/mysql/AphrontBaseMySQLDatabaseConnection.php
339

This is intentional, since we don't want pht() to interpret %B as a conversion.

src/aphront/storage/connection/mysql/AphrontBaseMySQLDatabaseConnection.php
339

oh, whoops, righto

This revision was automatically updated to reflect the committed changes.

@epriestley, @btrahan - wasn't sure if I should open a new task for this or put it here

src/aphront/storage/connection/mysql/AphrontMySQLiDatabaseConnection.php
70

This is causing massive haemorrhage when it fails... (Server version: 5.1.73)

In DarkConsole:

mysqli::set_charset() [<a href='mysqli.set-charset'>mysqli.set-charset</a>]: Error executing query at [AphrontMySQLiDatabaseConnection.php:70] (12 times)

Same when running CLI scripts.

Can you clarify what "massive haemorrhage" means? Just errors in the log, or is the page fataling?

If it's errors, can you try adding @ in front of the call line, like this?

$ok = @$conn->set_charset('utf8mb4');
     ^^^

I can't reproduce any issues locally (I have utf8mb4, but setting it to "zebra" or whatever) -- the call fails and returns an error, but doesn't spew. Could just be version differences or whatever, though.

Let me move this to a task so I don't lose track of it, since it seems like the resolution is at least slightly involved.

Apologies @epriestley
Yep, it was just erroring out multiple times but no fatal, @ fixed it and latest pull is fine. Thanks!