The "Must Encrypt" change will necessarily link into the existing /mail/ web UI so users can read the mail content, so it's probably worth considering possible improvements.
- Queries
- All Stories
- Search
- Advanced Search
- Transactions
- Transaction Logs
Advanced Search
Feb 1 2018
Although I'm suppressing the X-Phabricator-To and X-Phabricator-Cc headers for "Must Encrypt" mail (and the "To: ..." and "Cc: ..." footers in the body), the recipients will still be visible in the actual "To" and "Cc" lines if metamta.one-mail-per-recipient has been configured to false instead of the default (which is true).
On Thread-Topic, this very old thread suggests that Outlook can show the header in some cases: https://bugzilla.mozilla.org/show_bug.cgi?id=411601
@cmmata here is my diff. It can probably be improved, but it works for me:
| 1 | diff --git a/src/unit/engine/PhpunitTestEngine.php b/src/unit/engine/PhpunitTestEngine.php |
|---|---|
| 2 | index 8206b787..ef01fda4 100644 |
| 3 | --- a/src/unit/engine/PhpunitTestEngine.php |
| 4 | +++ b/src/unit/engine/PhpunitTestEngine.php |
| 5 | @@ -52,7 +52,7 @@ final class PhpunitTestEngine extends ArcanistUnitTestEngine { |
| 6 | if (!Filesystem::pathExists($test_path)) { |
| 7 | continue; |
| 8 | } |
| 9 | - $json_tmp = new TempFile(); |
| 10 | + $xml_tmp = new TempFile(); |
| 11 | $clover_tmp = null; |
| 12 | $clover = null; |
| 13 | if ($this->getEnableCoverage() !== false) { |
| 14 | @@ -64,10 +64,10 @@ final class PhpunitTestEngine extends ArcanistUnitTestEngine { |
| 15 | |
| 16 | $stderr = '-d display_errors=stderr'; |
| 17 | |
| 18 | - $futures[$test_path] = new ExecFuture('%C %C %C --log-json %s %C %s', |
| 19 | - $this->phpunitBinary, $config, $stderr, $json_tmp, $clover, $test_path); |
| 20 | + $futures[$test_path] = new ExecFuture('%C %C %C --log-junit %s %C %s', |
| 21 | + $this->phpunitBinary, $config, $stderr, $xml_tmp, $clover, $test_path); |
| 22 | $tmpfiles[$test_path] = array( |
| 23 | - 'json' => $json_tmp, |
| 24 | + 'xml' => $xml_tmp, |
| 25 | 'clover' => $clover_tmp, |
| 26 | ); |
| 27 | } |
| 28 | @@ -81,7 +81,7 @@ final class PhpunitTestEngine extends ArcanistUnitTestEngine { |
| 29 | |
| 30 | $results[] = $this->parseTestResults( |
| 31 | $test, |
| 32 | - $tmpfiles[$test]['json'], |
| 33 | + $tmpfiles[$test]['xml'], |
| 34 | $tmpfiles[$test]['clover'], |
| 35 | $stderr); |
| 36 | } |
| 37 | @@ -90,17 +90,17 @@ final class PhpunitTestEngine extends ArcanistUnitTestEngine { |
| 38 | } |
| 39 | |
| 40 | /** |
| 41 | - * Parse test results from phpunit json report. |
| 42 | + * Parse test results from phpunit XML report. |
| 43 | * |
| 44 | * @param string $path Path to test |
| 45 | - * @param string $json_tmp Path to phpunit json report |
| 46 | + * @param string $xml_tmp Path to phpunit XML report |
| 47 | * @param string $clover_tmp Path to phpunit clover report |
| 48 | * @param string $stderr Data written to stderr |
| 49 | * |
| 50 | * @return array |
| 51 | */ |
| 52 | - private function parseTestResults($path, $json_tmp, $clover_tmp, $stderr) { |
| 53 | - $test_results = Filesystem::readFile($json_tmp); |
| 54 | + private function parseTestResults($path, $xml_tmp, $clover_tmp, $stderr) { |
| 55 | + $test_results = Filesystem::readFile($xml_tmp); |
| 56 | return id(new ArcanistPhpunitTestResultParser()) |
| 57 | ->setEnableCoverage($this->getEnableCoverage()) |
| 58 | ->setProjectRoot($this->projectRoot) |
| 59 | diff --git a/src/unit/parser/ArcanistPhpunitTestResultParser.php b/src/unit/parser/ArcanistPhpunitTestResultParser.php |
| 60 | index 5ccff970..3d7fcd77 100644 |
| 61 | --- a/src/unit/parser/ArcanistPhpunitTestResultParser.php |
| 62 | +++ b/src/unit/parser/ArcanistPhpunitTestResultParser.php |
| 63 | @@ -9,10 +9,10 @@ |
| 64 | final class ArcanistPhpunitTestResultParser extends ArcanistTestResultParser { |
| 65 | |
| 66 | /** |
| 67 | - * Parse test results from phpunit json report |
| 68 | + * Parse test results from phpunit XML report |
| 69 | * |
| 70 | * @param string $path Path to test |
| 71 | - * @param string $test_results String containing phpunit json report |
| 72 | + * @param string $test_results String containing phpunit XML report |
| 73 | * |
| 74 | * @return array |
| 75 | */ |
| 76 | @@ -25,7 +25,7 @@ final class ArcanistPhpunitTestResultParser extends ArcanistTestResultParser { |
| 77 | return array($result); |
| 78 | } |
| 79 | |
| 80 | - $report = $this->getJsonReport($test_results); |
| 81 | + $report = simplexml_load_string($test_results); |
| 82 | |
| 83 | // coverage is for all testcases in the executed $path |
| 84 | $coverage = array(); |
| 85 | @@ -36,56 +36,36 @@ final class ArcanistPhpunitTestResultParser extends ArcanistTestResultParser { |
| 86 | $last_test_finished = true; |
| 87 | |
| 88 | $results = array(); |
| 89 | - foreach ($report as $event) { |
| 90 | - switch (idx($event, 'event')) { |
| 91 | - case 'test': |
| 92 | - break; |
| 93 | - case 'testStart': |
| 94 | - $last_test_finished = false; |
| 95 | - // fall through |
| 96 | - default: |
| 97 | - continue 2; // switch + loop |
| 98 | - } |
| 99 | - |
| 100 | + foreach ($report->testsuite as $test_suite) { |
| 101 | $status = ArcanistUnitTestResult::RESULT_PASS; |
| 102 | $user_data = ''; |
| 103 | |
| 104 | - if ('fail' == idx($event, 'status')) { |
| 105 | + if ((int)$test_suite['failures'] > 0) { |
| 106 | $status = ArcanistUnitTestResult::RESULT_FAIL; |
| 107 | - $user_data .= idx($event, 'message')."\n"; |
| 108 | - foreach (idx($event, 'trace') as $trace) { |
| 109 | - $user_data .= sprintf( |
| 110 | - "\n%s:%s", |
| 111 | - idx($trace, 'file'), |
| 112 | - idx($trace, 'line')); |
| 113 | + foreach ($test_suite->testcase as $test_case) { |
| 114 | + foreach ($test_case->failure as $failure) { |
| 115 | + $user_data .= sprintf( |
| 116 | + "\n%s", |
| 117 | + (string)$failure); |
| 118 | + } |
| 119 | } |
| 120 | - } else if ('error' == idx($event, 'status')) { |
| 121 | - if (strpos(idx($event, 'message'), 'Skipped Test') !== false) { |
| 122 | - $status = ArcanistUnitTestResult::RESULT_SKIP; |
| 123 | - $user_data .= idx($event, 'message'); |
| 124 | - } else if (strpos( |
| 125 | - idx($event, 'message'), |
| 126 | - 'Incomplete Test') !== false) { |
| 127 | - $status = ArcanistUnitTestResult::RESULT_SKIP; |
| 128 | - $user_data .= idx($event, 'message'); |
| 129 | - } else { |
| 130 | - $status = ArcanistUnitTestResult::RESULT_BROKEN; |
| 131 | - $user_data .= idx($event, 'message'); |
| 132 | - foreach (idx($event, 'trace') as $trace) { |
| 133 | - $user_data .= sprintf( |
| 134 | - "\n%s:%s", |
| 135 | - idx($trace, 'file'), |
| 136 | - idx($trace, 'line')); |
| 137 | + } else if ($test_suite['errors'] > 0) { |
| 138 | + $status = ArcanistUnitTestResult::RESULT_BROKEN; |
| 139 | + foreach ($test_suite->testcase as $test_case) { |
| 140 | + foreach ($test_case->error as $error) { |
| 141 | + $user_data .= sprintf( |
| 142 | + "\n%s", |
| 143 | + (string)$error); |
| 144 | } |
| 145 | } |
| 146 | } |
| 147 | |
| 148 | - $name = preg_replace('/ \(.*\)/s', '', idx($event, 'test')); |
| 149 | + $name = preg_replace('/ \(.*\)/s', '', $test_suite['name']); |
| 150 | |
| 151 | $result = new ArcanistUnitTestResult(); |
| 152 | $result->setName($name); |
| 153 | $result->setResult($status); |
| 154 | - $result->setDuration(idx($event, 'time')); |
| 155 | + $result->setDuration((float)$test_suite['time']); |
| 156 | $result->setCoverage($coverage); |
| 157 | $result->setUserData($user_data); |
| 158 | |
| 159 | @@ -95,7 +75,7 @@ final class ArcanistPhpunitTestResultParser extends ArcanistTestResultParser { |
| 160 | |
| 161 | if (!$last_test_finished) { |
| 162 | $results[] = id(new ArcanistUnitTestResult()) |
| 163 | - ->setName(idx($event, 'test')) // use last event |
| 164 | + ->setName($test_suite['name']) // use last event |
| 165 | ->setUserData($this->stderr) |
| 166 | ->setResult(ArcanistUnitTestResult::RESULT_BROKEN); |
| 167 | } |
| 168 | @@ -161,28 +141,4 @@ final class ArcanistPhpunitTestResultParser extends ArcanistTestResultParser { |
| 169 | return $reports; |
| 170 | } |
| 171 | |
| 172 | - /** |
| 173 | - * We need this non-sense to make json generated by phpunit |
| 174 | - * valid. |
| 175 | - * |
| 176 | - * @param string $json String containing JSON report |
| 177 | - * @return array JSON decoded array |
| 178 | - */ |
| 179 | - private function getJsonReport($json) { |
| 180 | - |
| 181 | - if (empty($json)) { |
| 182 | - throw new Exception( |
| 183 | - pht( |
| 184 | - 'JSON report file is empty, it probably means that phpunit '. |
| 185 | - 'failed to run tests. Try running %s with %s option and then run '. |
| 186 | - 'generated phpunit command yourself, you might get the answer.', |
| 187 | - 'arc unit', |
| 188 | - '--trace')); |
| 189 | - } |
| 190 | - |
| 191 | - $json = preg_replace('/}{\s*"/', '},{"', $json); |
| 192 | - $json = '['.$json.']'; |
| 193 | - return phutil_json_decode($json); |
| 194 | - } |
| 195 | - |
| 196 | } |
@mcorteel I have the same problem, arc unit only works with phpunit 5.7. If I use 6.5, it fails. Do you have that patch anywhere with public access so we can use it while it's not published?
Here I was thinking I found a helpful public bug.
From rereading D18600, maybe "ridiculously strong" is an understatement and it was actually so powerful it took effect even if you didn't enable it.
Locally, seems like "No Notifications" correctly has no effect on the "Upload" bubble being shown at HEAD of master.
Hm, actually the problem here is that the notification bubble doesn't show up for any file uploads on our current version. I suppose it could actually be "on our current version + extensions". I'll get a reproduction matrix together tomorrow.
Not unless something super weird/buggy is going on.
We don't have a notifications server, does that make it like that?
It's possible that adjusting Settings → Notifications to No Notifications disables this bubble, if that's a compounding factor here. D18600 pulled back some of the aggressiveness of that setting but I'd guess it might still be hitting the upload progress bubbles.
Ok I actually get a dialog on secure., it's probably because we're out of date with the upstream.
What browser are you using? I get a progress bubble in the lower left immediately. Here's a screenshot -- note the blue upload progress element in the lower left corner.
Jan 31 2018
I think T11934 and an explicit !claim Txxx action (which would mean "reassign + ref"; maybe with a !claim-and-fix Txxx to mean "reassign + fixes" or some kind of !claim+fix X syntax which expands to !claim <args>; !fix <args>) is a possible way forward on this, but I don't want to make "ref" automatically mean "claim".
Because of the high degree of complexity there's basically zero chance we'll ever build this without active customer interest, and we have none today. Closing this until interest arises.
We don't have any use cases for this today so I don't plan to pursue it. It's conceptually reasonable, but complex, and there's no customer interest.
T11934 is a possible way forward on this (and other niche things) without adding a lot of Javascript and UI complexity, by supporting it through !fixes.
One possible approach for this is:
I'm going to roll this forward into T13057.
We shipped "Mentions" and "Duplicates" some time ago.
See T13056 for followup.
One attack is that you can override the content of email and then send invite or welcome mail that says whatever you want, whether the victim has a Phabricator account (welcome) or not (invite).
After T13049 this is relatively within reach, but we haven't seen other requests for it so I don't currently plan to pursue it.
Jan 30 2018
This is relevant now that work related to T13049 has added CSV support.
More or less obsoleted by work in T13049.
That's as far as I plan to take this for now. I think the additional features described above (filter, reorder, reformat, and broader field and application support) are all reasonable if use cases arise, but am not currently aware of use cases. We can return to that stuff when use cases arise.
I just let that run for a while but it finished at some point:
(They got back to me and we're scheduling a call.)
Mailgun has yet to respond to me after about three weeks, so I send them a followup.
... OPTIMIZE Optimizing table "<instance>_audit"."audit_transaction"... DONE Compacted table by 139 GB in 910,219ms. ...
The compaction completed overnight. I'm optimizing the tables now.
Pool is full again, repo is upgrading, edges are compacting on the instance shard.
web004 is deploying now.
web004 died abruptly so I'm going to fix that and deploy these changes at the same time.
Jan 29 2018
Resolved by D18959 + D18960. See also T13049.
Even on our fairly normal data, the effect was a little bit more dramatic than I'd expected:
D18962 uses this to implement "Export Data" for large result sets.
I'm going to optimize + probe secure001 now and see if any of the tables above shrunk. I'm expecting a very modest effect combined with zero user visible changes in the UI despite throwing away a bunch of data.
Took like 3-ish minutes and did this:
I double checked that our backups are working.
Editing some edges on the new code as a sanity check before I compact things.
(Pushing this to secure, stuff might be funky for a minute while I gently massage the database.)
It looks like PHPExcel was formally deprecated on December 24, 2017 but I'm not planning to switch to its successor, PhpSpreadsheet, until we run into some specific motivator.
See PHI324.
Jan 28 2018
See PHI323.
This hasn't had any stability issues for a while.
transaction.search now returns comments and inlines, including path, line, length, and reply information.
I think everything we know about has been resolved.