Page MenuHomePhabricator

Implement modular transactions for application policy changes
ClosedPublic

Authored by amckinley on Apr 21 2017, 7:46 PM.
Tags
None
Referenced Files
F14089535: D17757.id42862.diff
Sun, Nov 24, 11:32 AM
Unknown Object (File)
Fri, Nov 22, 8:33 AM
Unknown Object (File)
Fri, Nov 22, 7:31 AM
Unknown Object (File)
Fri, Nov 22, 7:13 AM
Unknown Object (File)
Fri, Nov 22, 1:55 AM
Unknown Object (File)
Wed, Nov 20, 10:29 AM
Unknown Object (File)
Sat, Nov 16, 1:40 PM
Unknown Object (File)
Thu, Nov 14, 5:11 AM

Details

Summary

Still needs some cleanup, but ready for review in broad outline form.

Test Plan

Made lots of policy changes to the Badges application and confirmed expected rows in application_xactions, confirmed expected changes to phabricator.application-settings.

See example output (not quite working for custom policy objects) here:

Screen Shot 2017-04-24 at 2.59.55 PM.png (1×2 px, 527 KB)

Diff Detail

Repository
rP Phabricator
Branch
arcpatch-D17757
Lint
Lint Passed
Unit
Tests Passed
Build Status
Buildable 16787
Build 22402: Run Core Tests
Build 22401: arc lint + arc unit

Unit TestsFailed

TimeTest
951 msPhabricatorLibraryTestCase::Unknown Unit Message ("")
EXCEPTION (PhutilBootloaderException): Include of '/core/data/drydock/workingcopy-83/repo/phabricator/src/applications/slowvote/xactions/PhabricatorSlowvoteTransactionType.php' failed! #0 /core/data/drydock/workingcopy-83/repo/libphutil/src/symbols/PhutilSymbolLoader.php(381): PhutilBootloader->loadLibrarySource('phabricator', 'applications/sl...') #1 /core/data/drydock/workingcopy-83/repo/libphutil/src/symbols/PhutilSymbolLoader.php(256): PhutilSymbolLoader->loadSymbol(Array)
786 msPhabricatorLibraryTestCase::Unknown Unit Message ("")
Assertion failed, expected 'true' (at PhutilLibraryTestCase.php:51): The library map is out of date. Rebuild it with `arc liberate`. These entries differ: class.PhabricatorSlowvoteCloseTransaction, class.PhabricatorSlowvoteDescriptionTransaction, class.PhabricatorSlowvoteQuestionTransaction, class.PhabricatorSlowvoteResponsesTransaction, class.PhabricatorSlowvoteShuffleTransaction, class.PhabricatorSlowvoteTransactionType, xmap.PhabricatorSlowvoteCloseTransaction, xmap.PhabricatorSlowvoteDescriptionTransaction, xmap.PhabricatorSlowvoteQuestionTransaction, xmap.PhabricatorSlowvoteResponsesTransaction, xmap.PhabricatorSlowvoteShuffleTransaction, xmap.PhabricatorSlowvoteTransactionType.
22 msPhabricatorLibraryTestCase::Unknown Unit Message ("")
EXCEPTION (PhutilBootloaderException): Include of '/core/data/drydock/workingcopy-83/repo/phabricator/src/applications/slowvote/xactions/PhabricatorSlowvoteCloseTransaction.php' failed! #0 /core/data/drydock/workingcopy-83/repo/libphutil/src/symbols/PhutilSymbolLoader.php(381): PhutilBootloader->loadLibrarySource('phabricator', 'applications/sl...') #1 /core/data/drydock/workingcopy-83/repo/libphutil/src/symbols/PhutilSymbolLoader.php(256): PhutilSymbolLoader->loadSymbol(Array)
1 msAlmanacNamesTestCase::Unknown Unit Message ("")
30 assertions passed.
0 msAlmanacServiceTypeTestCase::Unknown Unit Message ("")
1 assertion passed.
View Full Test Results (3 Failed · 366 Passed)

Event Timeline

This only exists to move the code from old to new laptop.

  • implements transaction support for application policy changes
amckinley edited the test plan for this revision. (Show Details)
amckinley added a reviewer: epriestley.
src/applications/meta/controller/PhabricatorApplicationEditController.php
70–73

This was failing the request, but returning an HTTP 200 and not showing an error like it was before. Need to investigate.

src/applications/meta/xactions/PhabricatorApplicationPolicyChangeTransaction.php
41

This is just a hack. I was tempted to just hardcode it to a web request, but this xaction flow will (eventually) be accessible via Conduit as well.

95

Caching of this object at the per-xaction level, but would be nice if we could fetch this once and inject it into all the xactions when constructing the timeline.

src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php
548–550

I had to add this in addition to the other two places you added a setStorage() call. Pretty sure we should also be calling setViewer() on those sites as well.

chad added inline comments.
src/applications/transactions/storage/PhabricatorModularTransactionType.php
318–321

oh i need this too

The "simultate" stuff should be unnecessary with the move to proper transactions: they already have code which prevents you from setting invalid policies on things (specifically, policies which would prevent you from viewing or editing the object). I'm not totally sure if it will work as-is, but in theory the Editor should already prevent you from setting the "Can View Application" policy to something which excludes you. If it allows you to make this change, let me know and I can see what's going on.

src/applications/meta/controller/PhabricatorApplicationEditController.php
38

If $new is null it means that the user did something like right click -> open inspector and deleted half the form. In this case, we should detect that null isn't a valid policy below (near line 56) and just leave the setting as-is.

src/applications/meta/xactions/PhabricatorApplicationPolicyChangeTransaction.php
41

You can probably get the right source like this:

$editor = $this->getEditor();
$content_source = $editor->getContentSource();
64

For consistency with getTitle(), drop quotes around old/new?

src/applications/meta/xactions/PhabricatorApplicationPolicyChangeTransaction.php
19

Swap this to getActor() and it should work as-is.

src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php
548–550

Oh -- during editing, transactions should be using getActor() rather than getViewer(). This is an unintuitive distinction, but "actors" are a little more complicated than viewers.

src/applications/meta/controller/PhabricatorApplicationEditController.php
38

Anecdotally I saw some nulls coming through when I was testing (without any tricky form manipulation). I'll see if I can repro consistently.

Right now it's a little hard to distinguish between user-only, project-scoped, custom-policy scoped, etc. Can I embed little icons into the timeline text to indicate "project"/"person"/"policy", etc?

In theory, yes, but I don't think we do that with any other transaction type so it might cause a bunch of weird layout issues or something like that. You can try just rendering something like this and see what it looks like:

phutil_tag(
  'span', 
  array(
    'class' => 'timeline-policy',
  ),
  array(
    id(new PHUIIconView())->setIcon('fa-user'),
    $actual_policy_name,
  ));

(There might be a better CSS class name to use, similar to other related CSS.)

If that looks OK, I think Policy objects and Policy handles have an "icon" property.

We could also do a hovercard or tooltip or something.

Current:

epriestley changed the "Can Use Application" policy from "All Users" to "Phacility".

Proposed:

epriestley changed the "Can Use Application" policy from " All Users" to " Phacility".

This doesn't look too awful, but I'll leave it for sharper minds to improve the spacing between the icon and the text:

Screen Shot 2017-04-25 at 11.55.54 AM.png (912×1 px, 329 KB)

src/applications/meta/controller/PhabricatorApplicationEditController.php
38

So here's what I'm getting in my config on master when I change the "Default Edit Badges" policy to allow "All Users":

{
   "PHID-APPS-PhabricatorBadgesApplication" : {
      "policy" : {
         "badges.default.edit" : "users",
         "edit" : null,
         "badges.default.create" : "users"
      }
   }
}

My code dutifully considers $newValue to be null when running my branch instead of master.

The "simultate" stuff should be unnecessary with the move to proper transactions: they already have code which prevents you from setting invalid policies on things (specifically, policies which would prevent you from viewing or editing the object). I'm not totally sure if it will work as-is, but in theory the Editor should already prevent you from setting the "Can View Application" policy to something which excludes you. If it allows you to make this change, let me know and I can see what's going on.

Doesn't seem to work out of the box. Without putting any code in validateTransactions(), I can set the "View" policy on the Badges application to "No One" without being stopped. I end up at the "You Shall Not Pass: Restricted Application" dialogue when trying to view the config.

Let's just move forward without that check and then I can counter-diff with whatever dark magic we need to get it working. It's probably something simple but I'm not sure offhand.

  • requested changes, adds rendering of policy icons selectively

This is all working except for the issue where the new value for a policy can be null.

New UI:

Screen Shot 2017-04-25 at 1.09.29 PM.png (450×1 px, 146 KB)

To tackle the null thing first, I think there are two issues:


First, we try to edit every capability the object has, even capabilities which should not be editable. Here, notably, this includes CAN_EDIT, which is always locked to "Administrators". We check for this correctly when building the form (to figure out whether the UI will show an editable control or not) by calling:

$can_edit = $application->isCapabilityEditable($capability);

...but we don't perform a similar check before trying to actually apply the edit. To fix this, I think:

  • Since we're applying the edit from within the transaction, let's move the validation logic (the stuff between $request->getStr('policy:'.$capability); and where we build the transaction) to validateTransactions() on the TransactionType.
  • validateTransactions() should read the 'capability.name' metadata from each transaction and make sure that the capability is valid (exists in $application->getCapabilities()) and editable (isCapablityEditable()), and raise an "invalid" error if it isn't. If you try to do a null edit after this change, we should get a proper error.
  • Then, the $request->getStr(...) loop should test that capabilities are editable before building transactions for them, so only valid capability edits get submitted to the editor.

Second, the logic around PhabricatorPolicyQuery is currently constructed incorrectly, because PhabricatorPolicyQuery always loads some policy object -- it's just possible that the object it loads is an invalid/incomplete/restricted object. Specifically, this condition will never be met because $policy will always be a valid object:

if (!$policy) {
  // Not a custom policy either. Can't set the policy to something
  // invalid, so skip this.
  continue;
}

99% of the time, I think this (always loading an object) is the right behavior for policies and just makes things work properly, but in this case we need to do a little more work.

To step back slightly, for Handles, we have two general classes of "invalid" object:

  • Filtered/Restricted: The PHID is valid, but the viewer doesn't have permission to see it (for example, we tried to load the handle for a project you can't see). This usually renders like "Restricted Project". (Viewers are allowed to know that something they can't see exists, they just aren't allowed to know any of the content.)
  • Incomplete: We failed to load the object completely. This happens if you, e.g., load "cat" as a PHID or something like that. This usually renders as "Unknown Object".

In Policies, we don't currently make this distinction. Some day, we possibly should, but for now we can just lump these into one "invalid" bucket since neither is valid to select when editing something.

In the normal case (for example, when you set "Visible To" on a task), we validate policies implicitly by making sure that the user still satisfies the policy check. If the user provides an invalid policy (like null or "cat" or the PHID of something they can't see) they'll fail the check anyway, so we don't need to explicitly do a validity test.

However, it's legitimate to set a policy like "Default task view policy" to something which excludes you (for example, if I set it to "Default View Policy: Only user alincoln" that's silly, but perfectly fine) so we can't rely on this same check. Instead, I think we can do this:

  • Per above, move this validation logic to the TransactionType.
  • Add a isValidPolicyForEdit() or similar method to PhabricatorPolicy, and just have it return $this->getType() !== PhabricatorPolicyType::TYPE_MASKED;.
  • Test each policy by calling that method, raising an invalid error if the policy isn't valid.

This will make it easier to refine "restricted" vs "incomplete" policies later if we want by putting the logic in one place and making it relatively grep-able.

One way to to test this is to right-click/control-click the form and edit the hidden input for a control to read "cat" in the inspector:

Screen Shot 2017-04-26 at 4.29.35 AM.png (282×1 px, 89 KB)

When you submit the form, you should get an "invalid policy" error.


Sort that out first and then we can look at the issue with setting "Can Use Application" to something which excludes you.

src/applications/meta/xactions/PhabricatorApplicationPolicyChangeTransaction.php
71–78

You should be able to get rid of this, PhabricatorPolicy has a getIcon() method already which you can just call.

100–109

Transactions may render to different targets. We currently support three-ish, in some cases:

  • Phabricator itself, like the transaction timeline.
  • HTML mail, which can use HTML but can't use images or external CSS.
  • Text mail, which can't use HTML.
  • (In some cases, external APIs like JIRA or Asana, which generally are also plain text.)

Today, no transactions actually do anything interesting in "HTML mail" mode, and the don't have a convenient way to test for it, so the only modes you'll actually see are "Web HTML" and "Plain Text". (We might do fancier stuff with HTML mail eventually.)

If you add HTML rendering like this, you need to add a text fallback by testing for $this->isTextMode(). Rendering support methods like renderValue() already do this for you where they can, but renderValue() can't know how to remove HTML if you pass it a complex value.

Fix here is just something like:

if ($this->isTextMode()) {
  return $this->renderValue($plain_text_label);
}

// Keep going and build the icon...

This is probably difficult to test here since Applications don't actually generate email today, but you could make isTextMode() just return true; and make sure the page looks OK and doesn't have a bunch of HTML as a reasonable approximation.

src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php
2723–2725

Instead of doing this, override getMailCC() in the subclass and just have it return array();.

Optionally, change the "Capability not supported." exception message to read something like this:

The object being edited does not implement any standard interfaces (like PhabricatorSubscribableInterface) which allow CCs to be generated automatically. Override the "getMailCC()" method and generate CCs explicitly.

That is, the intent here is "we can't do something sensible by default, so we're making you define an explicit behavior instead of guessing what you want".

src/applications/meta/xactions/PhabricatorApplicationPolicyChangeTransaction.php
71–78

That was my first iteration, but the timeline looked super-busy with every line having two different icons. In particular, the global policies (which I'm going to submit without evidence are the most frequently used) looked pretty bad. I can put up a screenshot if you want to see what it looks like, though.

src/applications/meta/xactions/PhabricatorApplicationPolicyChangeTransaction.php
71–78

Maybe we should just make the policy "amckinley" render as "User: amckinley" in this context, and avoid the whole icon/html issue? And likewise for "Members of Project: Secret Cabal"?

src/applications/meta/xactions/PhabricatorApplicationPolicyChangeTransaction.php
71–78

That's fine with me. Looks like either PhabricatorPolicy::getShortName() or PhabricatorPolicy::getFullName() will do what I want.

Just because I dug into this: I think the reason we aren't throwing "can't lock yourself out" errors is because PhabricatorApplicationTransactionEditor is only invoking validatePolicyTransaction() for transaction objects that were constructed with ->setTransactionType(PhabricatorTransactions::TYPE_EDIT_POLICY) (or similar).

If I change PhabricatorApplicationEditController to construct xactions with either TYPE_EDIT_POLICY or TYPE_VIEW_POLICY, I trigger the expected validation errors.

  • refactor validity checking code from the edit controller to the transaction
  • adding code to prevent setting of policies of type MASKED

User austin viewing the timeline after bob switched to a project members policy that austin isn't a member of (and a transaction at the end that attempts to change the transaction to 'cat'):

Screen Shot 2017-04-26 at 4.29.07 PM.png (217×868 px, 56 KB)

User bob viewing the same timeline:

Screen Shot 2017-04-26 at 4.31.01 PM.png (191×827 px, 52 KB)

This looks good and the explanation about CAN_VIEW makes perfect sense, nice job hunting that down. I'm probably going to call it a night once the import/backup stuff settles but I'll give this a more detailed look tomorrow, try to break it, and see if I can come up with a clean approach to get the CAN_VIEW logic running on this pathway. Those screeshots look good, and I think I like the "User/Members" compromise -- feels more clear without looking weird to me.

I think something like this is reasonable to make the CAN_VIEW validation work, and it seems to work properly:

diff --git a/src/applications/meta/xactions/PhabricatorApplicationPolicyChangeTransaction.php b/src/applications/meta/xactions/PhabricatorApplicationPolicyChangeTransaction.php
index 50e4f15afa..7cb496fffe 100644
--- a/src/applications/meta/xactions/PhabricatorApplicationPolicyChangeTransaction.php
+++ b/src/applications/meta/xactions/PhabricatorApplicationPolicyChangeTransaction.php
@@ -119,6 +119,40 @@ final class PhabricatorApplicationPolicyChangeTransaction
       }
     }
 
+    // If we're changing these policies, the viewer needs to still be able to
+    // view or edit the application under the new policy.
+    $validate_map = array(
+      PhabricatorPolicyCapability::CAN_VIEW,
+      PhabricatorPolicyCapability::CAN_EDIT,
+    );
+    $validate_map = array_fill_keys($validate_map, array());
+
+    foreach ($xactions as $xaction) {
+      $capability = $xaction->getMetadataValue(self::METADATA_ATTRIBUTE);
+      if (!isset($validate_map[$capability])) {
+        continue;
+      }
+
+      $validate_map[$capability][] = $xaction;
+    }
+
+    foreach ($validate_map as $capablity => $xactions) {
+      if (!$xactions) {
+        continue;
+      }
+
+      $editor = $this->getEditor();
+      $policy_errors = $editor->validatePolicyTransaction(
+        $object,
+        $xactions,
+        self::TRANSACTIONTYPE,
+        $capability);
+
+      foreach ($policy_errors as $error) {
+        $errors[] = $error;
+      }
+    }
+
     return $errors;
   }
 
diff --git a/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php b/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php
index df9cc76844..314f6db1b1 100644
--- a/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php
+++ b/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php
@@ -2157,7 +2157,7 @@ abstract class PhabricatorApplicationTransactionEditor
     return array_mergev($errors);
   }
 
-  private function validatePolicyTransaction(
+  public function validatePolicyTransaction(
     PhabricatorLiskDAO $object,
     array $xactions,
     $transaction_type,

The actual exception we get out of it is a little ugly since the UI doesn't use EditEngine yet, but that's fine for now.

src/applications/meta/xactions/PhabricatorApplicationPolicyChangeTransaction.php
83

This should retain the if ($old === $new) { continue; } check which skips the other validation.

The general idea here is that if an object is in a partially restricted state, you're still allowed to edit it as long as you don't try to change that state.

For example, if a task is tagged with a "Restricted Project" which you don't have permission to see, even though you can see the task, you can still edit the task.

Here, a super-administrator might have set some policy default to "High-Access Secret Cabal". When a low-power administrator views the UI they'll see "Restricted Project", and when they save the form it will submit a transaction like "old = PHID-PROJ-xzy, new = PHID-PROJ-xyz", where both values are the same. If we block this, the low-power administrator won't be able to make changes without adjusting that policy, even though the policy is valid and they could make other reasonable changes.

To reproduce this:

  • As a high-power administrator, set some policy other than "View" or "Edit" to "Members of: Secret Cabal".
  • As a low-power administrator who can not see "Secret Cabal", make a mundane change to a different policy.
  • The edit should be permitted.

(I didn't actually run through this test case myself so it's possible I'm wrong/crazy about the behavior here.)

137–148

This can't be translated since the strings aren't wrapped in pht().

It would also probably be a little cleaner on PhabricatorPolicy as a method like getFullName(), as I could imagine using it elsewhere.

To do the actual string stuff, do something like $label = pht('User: %s', $this->getName()); instead of using string concatenation. Then it can eventually be translated into "El Usero: ...", "alincoln a la Userétte", etc.

152–163

As you noted elsewhere this isn't terribly efficient, but I think this is fine for the moment (we'll just do some extra queries when looking at an application detail page, which isn't a huge issue). Once we collect a couple more weird transactions like this which need better batching of fetches for external objects (other than Handles) we can add a proper mechanism for it.

This revision now requires changes to proceed.Apr 29 2017, 6:03 PM
src/applications/meta/xactions/PhabricatorApplicationPolicyChangeTransaction.php
83

My thought was that if $old === $new, we shouldn't even be creating a transaction for it, since that would just clutter up the timeline by showing Foo changed "Bar" to "Bar". That's why I kept the check/continue in PhabricatorApplicationEditController.

On master, if you try and change the view policy to something that prevent you from seeing it, you just get the unhandled exception dialogue:

Screen Shot 2017-05-03 at 12.42.32 PM.png (261×1 px, 23 KB)

I copy/pasted some code I found, and now it shows a little dialogue box:

Screen Shot 2017-05-03 at 12.43.50 PM.png (236×619 px, 17 KB)

amckinley edited edge metadata.
  • requested changes

Oh, I actually missed that the old old/new code was still around.

The transaction should get dropped automatically a little later on anyway, we just validate it first so we can give you a better error if you're doing something wrong. Depending on how the edit is being applied, we also sometimes tell you that the action will have no effect, explicitly.

An example of this is if you try to add a tag which an object already has by using the comment form (this isn't the greatest example, but conveys the general idea):

Screen Shot 2017-05-03 at 12.52.40 PM.png (1×1 px, 180 KB)

The general goal of this feature is to prevent surprises where two users assign a task to the same victim or add the same tag at the same time, and then end up with silly-looking similar comments or no-op transactions or an empty effect (all of which could happen long ago, and confused users).

Putting the $old == $new code in the validate block supports these things and lets the same code work for API, form, and comment cases.

Since we only have a form case right now and don't care about this fine-grained edit feedback there's no actual effect yet and your version is equally correct, but moving it into validation will improve behavior if we eventually expose this over the API, let you change policies in the comment form (seems very unlikely) or maaaaaybe add a more full-featured CLI tool (maybe a little more likely?). In this case these are all a bit far-fetched, of course, but for many edits they're more realistic.

Some minor excitement making this work until I found the typo. On the last line, it should be $capability, not $capablity:

I think something like this is reasonable to make the CAN_VIEW validation work, and it seems to work properly:

diff --git a/src/applications/meta/xactions/PhabricatorApplicationPolicyChangeTransaction.php b/src/applications/meta/xactions/PhabricatorApplicationPolicyChangeTransaction.php
index 50e4f15afa..7cb496fffe 100644
--- a/src/applications/meta/xactions/PhabricatorApplicationPolicyChangeTransaction.php
+++ b/src/applications/meta/xactions/PhabricatorApplicationPolicyChangeTransaction.php
@@ -119,6 +119,40 @@ final class PhabricatorApplicationPolicyChangeTransaction
       }
     }
 
+    // If we're changing these policies, the viewer needs to still be able to
+    // view or edit the application under the new policy.
+    $validate_map = array(
+      PhabricatorPolicyCapability::CAN_VIEW,
+      PhabricatorPolicyCapability::CAN_EDIT,
+    );
+    $validate_map = array_fill_keys($validate_map, array());
+
+    foreach ($xactions as $xaction) {
+      $capability = $xaction->getMetadataValue(self::METADATA_ATTRIBUTE);
+      if (!isset($validate_map[$capability])) {
+        continue;
+      }
+
+      $validate_map[$capability][] = $xaction;
+    }
+
+    foreach ($validate_map as $capablity => $xactions) {
  • fixing typo, moving old vs new comparison
  • whoops, wrong library map

One possible inline but I don't see any other issues. Thanks!

src/applications/meta/xactions/PhabricatorApplicationPolicyChangeTransaction.php
82

This might have the wrong (i.e., empty) value here -- does the "less privileged admin" test case work?

If it doesn't, $old = $this->generateOldValue($object) likely fixes it. I believe validation runs before old values are populated.

This revision is now accepted and ready to land.May 3 2017, 10:34 PM
src/applications/meta/xactions/PhabricatorApplicationPolicyChangeTransaction.php
82

You're right that the $old === $new check isn't working as-is inside validateTransactions() because old is always null. Doing generateOldValue($object) instead doesn't work, because nothing has called setStorage() before validateTransactions() gets invoked. Doing the repro you suggested (edit an unrelated policy for an app with a policy you cant see) works as-is without the $old === $new check. I'm now confused about how I was ever hitting this condition.

I'm going to put the old === new check back the way it was and land this, because in practice I couldn't break it and the current version is definitely broken (because old is always null).

  • reverting to previous old v new check
This revision was automatically updated to reflect the committed changes.