Page MenuHomePhabricator

Integrate Conduit API for Policy Object Creation
Open, Needs TriagePublic

Description

Especially for generating large numbers of groups and repositories from the API, it would be super helpful to have a policy.create API function. In our scenario, we want to create custom policies to restrict the visibility of student repositories.
As far as I can see, creating custom policies from the API cannot be achieved at the moment. Wikimedia has already an extension to achieve this goal.

https://phabricator.wikimedia.org/T135249
https://github.com/wikimedia/phabricator-extensions/blob/master/src/conduit/CreatePolicyConduitAPIMethod.php

Root Problem: I run a class of 50 students every semester. We want to use Phabricator to learn concepts of operating system constructions and the students get graded on the code they commit into Phabricator. We don't want students seeing each others work, so we want to pre-fill repositories with policies that enforce this visibility so no cheating occurs. Furthermore, a tutor group should also have the possibility to view the repository. Because of the large number of repositories and groups, we have to ease production as much as possible with Conduit.

Event Timeline

We'd want to know the root problem you're having with Phabricator specifically so that we could decide the best way to resolve the issue in the upstream. Can you go into more detail about the custom policies you're wanting to create, how often, what they do, etc?

(Are you associated with WMF, or just noting that they have an extension for this?)

Root of the Problem: The conduit API does not provide the possibility to create custom policies. Therefore, the fine-grained policy model of phabricator cannot be used in automated workflows.

My usage scenario: student lab at the university, where always two students are assigned to a group and have a shared repository. For the assignments, students as well as the tutor group should have view access to the repository, but no other group.

I'm not associated to WMF, but have seen their custom extension and the related Maniphest task.

I don't think this is related to wikimedia since we already have the functionality live in the WMF production install.

For what it's worth, the use-case for wikimedia was the same as stated in this task - we have on-demand repository creation for 'tools' projects and we wanted to automatically set the policy for these repositories to be editable and pushable by members_of($repository_admins) + users($requesting_user).

I don't know how useful it would be to upstream the WMF component since it's not very sophisticated nor very generic. It would be straightforward for any downstream to copy the extension from wikimedia's repository and modify it to suit their needs.

What I would find especially useful would be a way to create named policies and then reference them when creating objects. That might warrant a separate feature-request task though.

From an usability side, it would also be a possibility to create policy objects just in time, when the API user submits a transaction touching a policy:

[ { 'type': 'view', 
   'value':   {'default': deny, 
     'rules': [
        {'type': 'members_of_group', 'value': PHID}
     ],
   }

Since the WMF implementation currently leaks phabricator implementation details (class names). Furthermore, I can imagine that there is a race condition between policy object creation and assignment to another object due to garbage collection (I don't know if phabricator has a garbage collection for unreferenced policy objects).

A root problem to us describes the issue you're trying to solve with the request, in full detail. In general we can't accept "feature doesn't exist" as a root problem. Here's an example of a root problem:

I run a class of 50 students every quarter. We use Phabricator to learn concepts of programming and the students get graded on the code they commit into Phabricator. We don't want students seeing each others work, so we want to pre-fill repositories with policies that enforce this visibility so no cheating occurs. Because of the large number of repositories we need to create each time, we want to ease production as much as possible with Conduit.

I think this scenario is better solved using Spaces, which are intended to be a "policy wall" around large groups of content in Phabricator (including tasks, repositories, etc).

Ok, so root problem is more of a user story. Sorry to not getting this right. But this was my first feature request on Phabricator.

So, the problem with spaces at this point is that their creation is also not exposed as an API (as far as I can tell). However, even if there would be a spaces creation API, how would I instanciate it in my usage scenario?

repo group 1 - visible to S1 all users
                          - editable to tutor group
                          - pushable by student group
S1 - editable by tutor group 1
      - visible to [student group 1, tutor group]

As far as i can tell, I would need the same complex policy API when using spaces, since the tutor group also has to see the space (as edit rights require view rights).

This comment was removed by chad.

Why do you need separated edit/push policies if it's locked to a space with just the specific tutors and students? Is there an actual abuse case, or is this just anticipatory / hypothetical?

The different edit/view policy is not the actual problem here. The problem with using spaces to solve the described scenario in an automatic fashion is that a) there is no spaces API b) the view policy for the space would also be a complex policy b.1) There is no API to create complex policies.

The different edit/view policies are more anticipatory at the moment for the current course. However, we plan to use Phabricator for larger courses (>300 students) in the future, where manual detecting abuse is impossible for the tutors.

Some workarounds in the short term:

  • Use the WMF extension (from cursory review, it appears to be well-constructed).
  • Use project.edit to create projects for each student / student group, consisting of the students and authorized faculty/tutors individually. Then, use repository.edit to create a repository with the "View" Policy set to that project's PHID, making the effective policy "Members of X". This will be somewhat cumbersome (since faculty/tutors will end up in 300+ projects if you have 300+ students, and adding a new TA halfway through the semester will require a large number of updates) but should be manageable if you're scripting the whole thing via the API. Basically, the only thing you lose here is a way to conveniently refer to all the faculty/tutors by reference to a group.

In the long term, both Spaces and Policies should get modern APIs, but I think the majority of use cases we've seen for "light instancing" like this are related to student groups so these features are difficult for us to prioritize. We'd eventually like to be able to spend more resources supporting students/education (and other open source projects) but we don't currently have the resources.

Then, I will use the WMF extension for now. I understand that you won't prioritize this. Just for future reference, I will document the structure we have come up with : I currently employ already a more hierarchical project/subproject structure that can be use over several semesters and makes it possible for me to reference all students at once:

V_BSB                       # Every course becomes a top-level project that will be existing forever
    - V_BSB Tutors          # All Tutors that are currently employed
    - V_BSB 17              # For V_BSB (Betriebssystembau == operating-system engineering)
         -V_BSB 17 - Group 1
              View: [V_BSB 17, V_BSB Tutors] |  Edit: V_BSB Tutors 

git repo: v_bsb17_group1
    View: [V_BSB  17 - Group1, V_BSB Tutors] | Edit: V_BSB Tutors | Push: V_BSB 17 - Group 1

Furthmore, my scripting is idempotent, and can therefore be run at any point during the semester to bring everything to the desired state.

@stettberger: If there is anything missing from the WMF policy API extension, please feel free to file tasks under https://phabricator.wikimedia.org/tag/wikimedia_phabricator_extensions/ or open a differential revision against https://phabricator.wikimedia.org/source/phab-extensions/. I will be glad to review patches, I can't make any promises about feature requests but reasonable changes will be considered.