There is currently no way to set the default locale, and for my specific installation I want all users to use the English (Very Wow) translation.
This is a task for the TODO in the following file: https://secure.phabricator.com/diffusion/P/browse/master/src/applications/settings/panel/PhabricatorAccountSettingsPanel.php;410380be3f7a4371876af357b7776a70e3026a63$85