Scripts to migrate old "points" fields and move projects beneath other projects
Open, NormalPublic

Tokens
"Party Time" token, awarded by MatanUbe."Mountain of Wealth" token, awarded by CodeMouse92."Like" token, awarded by cmmata."Yellow Medal" token, awarded by Luke081515.2."Pterodactyl" token, awarded by ftdysa."Yellow Medal" token, awarded by chad.
Assigned To
Authored By
jdforrester, Feb 12 2016

Description

This task contains:

  • A script for copying values from an older "points" field in Maniphest using Custom Fields to the new formal "points" field.
  • A script for moving existing, top-level projects underneath other existing, top-level projects as milestones or subprojects.

Neither workflow is formally supported by the upstream, and we do not currently plan to maintain these scripts. The were written circa February 2016, and may not work (and may even be dangerous) if run against future versions of Phabricator.


Script For Copying Points From a Custom Field

This script emits SQL you can use to copy an existing custom "Points" field into the new first-class "Points" field.

To run this script:

  • Put it in phabricator/
  • Make it executable with chmod +x copy_points.php
  • Run it as ./copy_points.php
  • It will walk you through giving it the information it needs, primarily a --field <field key> argument to pick which field to copy from.

The script just prints SQL statements to stdout. It does not touch the database, so running it won't make any actual changes to the data. Here's an example run:

$ ./copy_points.php --field std:maniphest:birthday
UPDATE `phabricator_maniphest`.`maniphest_task` SET points = 238 WHERE id = 216;

Generally, the migration process will probably look something like this:

  • Run the script.
  • Look at the output SQL to make sure it seems OK.
  • Load a couple of tasks to double-check (e.g., for the output above, load T216 in the web UI and make sure it should have 238 points).
  • If everything looks like you expect it to, pipe the SQL into MySQL to actually run it.

That might look like this:

$ ./copy_points.php --field whatever > points.sql
$ cat points.sql # Look at the results and sanity check them.
$ cat points.sql | mysql -u root

The script will not copy points to tasks that already have a "points" value. If you make a mistake (for example, copy the wrong field) and need to wipe out all the "new" points values in all tasks so you can do another copy of everything, you can use this query:

/* DANGEROUS! DESTROYS DATA! */ UPDATE phabricator_maniphest.maniphest_task SET points = null;

That will destroy the "new" points, but leave any custom points untouched. Then you can use copy_points.php again to do a fresh copy.

Here's the actual script:

1​#!/usr/bin/env php
2<?php
3
4// See <https://secure.phabricator.com/T10350> for discussion.
5
6require_once 'scripts/__init_script__.php';
7
8$args = new PhutilArgumentParser($argv);
9$args->parseStandardArguments();
10$args->parse(
11array(
12array(
13'name' => 'field',
14'param' => 'key',
15'help' => pht('Field to migrate.'),
16)
17));
18
19$task = new ManiphestTask();
20$fields = PhabricatorCustomField::getObjectFields(
21$task,
22PhabricatorCustomField::ROLE_EDIT);
23
24$field_map = $fields->getFields();
25$field_list = implode(', ', array_keys($field_map));
26
27if (!$field_map) {
28throw new PhutilArgumentUsageException(
29pht(
30'You do not have any custom fields defined in Maniphest, so there is '.
31'nowhere that points can be copied from.'));
32}
33
34$field_key = $args->getArg('field');
35if (!strlen($field_key)) {
36throw new PhutilArgumentUsageException(
37pht(
38'Use --field to specify which field to copy points from. Available '.
39'fields are: %s.',
40$field_list));
41}
42
43$field = idx($field_map, $field_key);
44if (!$field) {
45throw new PhutilArgumentUsageException(
46pht(
47'Field "%s" is not a valid field. Available fields are: %s.',
48$field_key,
49$field_list));
50}
51
52$proxy = $field->getProxy();
53if (!$proxy) {
54throw new PhutilArgumentUsageException(
55pht(
56'Field "%s" is not a standard custom field, and can not be migrated.',
57$field_key,
58$field_list));
59}
60
61if (!($proxy instanceof PhabricatorStandardCustomFieldInt)) {
62throw new PhutilArgumentUsageException(
63pht(
64'Field "%s" is not an "int" field, and can not be migrated.',
65$field_key,
66$field_list));
67}
68
69$storage = $field->newStorageObject();
70$conn_r = $storage->establishConnection('r');
71
72$value_rows = queryfx_all(
73$conn_r,
74'SELECT objectPHID, fieldValue FROM %T WHERE fieldIndex = %s
75 AND fieldValue IS NOT NULL',
76$storage->getTableName(),
77$field->getFieldIndex());
78$value_map = ipull($value_rows, 'fieldValue', 'objectPHID');
79
80$id_rows = queryfx_all(
81$conn_r,
82'SELECT phid, id, points FROM %T',
83$task->getTableName());
84$id_map = ipull($id_rows, null, 'phid');
85
86foreach ($value_map as $phid => $value) {
87$dict = idx($id_map, $phid, array());
88$id = idx($dict, 'id');
89$current_points = idx($dict, 'points');
90
91if (!$id) {
92continue;
93}
94
95if ($current_points !== null) {
96continue;
97}
98
99if ($value === null) {
100continue;
101}
102
103$sql = qsprintf(
104$conn_r,
105'UPDATE %T.%T SET points = %f WHERE id = %d;',
106'phabricator_maniphest',
107$task->getTableName(),
108$value,
109$id);
110
111echo $sql."\n";
112}


Script For Moving Projects Underneath Other Projects

This script moves an existing, top-level project underneath another existing project, turning it into either a subproject or a milestone.

To run this script:

  • Put it in phabricator/.
  • Make it executable with chmod +x move_beneath.php.
  • Run it as ./move_beneath.php.
  • It will walk you through giving it the information it needs.
IMPORTANT: This script mutates data immediately, without prompting, and can not be undone. Double check your command line before running it!

Here's an example run:

$ ./move_beneath.php --parent p1 --child p2 --keep-members both --subproject
Done.

Broadly, you will use these flags:

  • --parent <id|phid|hashtag> Choose which project will be the parent. This project must not already be a milestone.
  • --child <id|phid|hashtag> Choose which project will be the child. This project must be a top-level project with no children of its own -- this script can not move project trees. Plan ahead! This project must not be the same as the parent project.
  • --subproject or --milestone Choose whether the child should become a subproject or a milestone.
  • --keep-members <both|child|parent> Choose which members to keep (and, implicitly, which to destroy). See below for discussion.

These are mostly straightforward, except for --keep-members. Broadly, when moving projects, we must adjust membership because subprojects and milestones have these special rules:

  • Milestones can not have their own members (they have the same members as their parent).
  • Projects with subprojects can not have their own members (they have the union of all children as members).

When you move a child under a parent, one of them either becomes a milestone or a project with subprojects, and thus can not have members. We have to do something with the invalid members.

With --keep-members both, overall membership will be preserved. If the child is becoming a milestone, members will be copied to the parent. If the child is becoming a subproject, members will be copied to the child. This mode is safe, and won't destroy data, although it could make users members of projects you don't want or expect them to be members of.

With --keep-members parent, the child's members will be wiped out. If the child is becoming a milestone, nothing else happens. If the child is becoming a subproject, the parent's members are then copied to the child.

With --keep-members child, the parent's members will be wiped out. If the child is becoming a milestone, the members are then copied. If the child is becoming a subproject, nothing else happens.

Note that when you are making the child a milestone of a parent with subprojects, neither project may have members! The script will only permit this operation with --keep-members parent, which wipes child members and leaves parent members untouched (the union of all subprojects). If a milestone has unique members and you want to perform this operation and retain them, you need to manually move them to some subproject of the parent.

WARNING: I've made a reasonable effort to test this locally, and will make a reasonable effort to help repair any damage it causes, but this is basically a big hack. So, be careful with it, make a backup first, do some small runs on test data before converting hundreds of projects with thousands of members, let someone else run it first and complain that I wiped their install, etc., etc.

Here's the actual script:

1​#!/usr/bin/env php
2<?php
3
4// See <https://secure.phabricator.com/T10350> for discussion.
5
6require_once 'scripts/__init_script__.php';
7
8$args = new PhutilArgumentParser($argv);
9$args->parseStandardArguments();
10$args->parse(
11array(
12array(
13'name' => 'milestone',
14'help' => pht(
15'Turn the project into a milestone. Or, use --subproject.'),
16),
17array(
18'name' => 'child',
19'param' => 'project',
20'help' => pht('The project to make a child of the --parent project.'),
21),
22array(
23'name' => 'parent',
24'param' => 'project',
25'help' => pht('The project to make a parent of the --child project.'),
26),
27array(
28'name' => 'subproject',
29'help' => pht(
30'Turn the project into a subproject. Or, use --milestone.'),
31),
32array(
33'name' => 'keep-members',
34'param' => 'mode',
35'help' => pht('Choose which members to keep: both, child, parent.'),
36),
37));
38
39
40$parent_name = $args->getArg('parent');
41$child_name = $args->getArg('child');
42
43if (!$parent_name) {
44throw new PhutilArgumentUsageException(
45pht(
46'Choose which project should become the parent with --parent.'));
47}
48
49if (!$child_name) {
50throw new PhutilArgumentUsageException(
51pht(
52'Choose which project should become the child with --child.'));
53}
54
55$keep_members = $args->getArg('keep-members');
56switch ($keep_members) {
57case 'both':
58case 'child':
59case 'parent':
60break;
61default:
62if (!$keep_members) {
63throw new PhutilArgumentUsageException(
64pht(
65'Choose which members to keep with --keep-members.'));
66} else {
67throw new PhutilArgumentUsageException(
68pht(
69'Valid --keep-members settings are: both, child, parent.'));
70}
71}
72
73$want_milestone = $args->getArg('milestone');
74$want_subproject = $args->getArg('subproject');
75if (!$want_milestone && !$want_subproject) {
76throw new PhutilArgumentUsageException(
77pht(
78'Use --milestone or --subproject to select what kind of child the '.
79'project should become.'));
80} else if ($want_milestone && $want_subproject) {
81throw new PhutilArgumentUsageException(
82pht(
83'Use either --milestone or --subproject, not both, to select what kind '.
84'of project the child should become.'));
85}
86$is_milestone = $want_milestone;
87
88$parent = load_project($parent_name);
89$child = load_project($child_name);
90
91if ($parent->isMilestone()) {
92throw new PhutilArgumentUsageException(
93pht(
94'The selected parent project is a milestone, and milestones may '.
95'not have children.'));
96}
97
98if ($child->getParentProjectPHID()) {
99throw new PhutilArgumentUsageException(
100pht(
101'The selected child project is already a child of another project. '.
102'This script can only move root-level projects beneath other projects, '.
103'not move children within a hierarchy.'));
104}
105
106if ($child->getHasSubprojects() || $child->getHasMilestones()) {
107throw new PhutilArgumentUsageException(
108pht(
109'The selected child project already has subprojects or milestones '.
110'of its own. This script can not move entire trees of projects.'));
111}
112
113if ($parent->getPHID() == $child->getPHID()) {
114throw new PhutilArgumentUsageException(
115pht(
116'The parent and child are the same project. There is no conceivable '.
117'physical interpretation of what you are attempting to do.'));
118}
119
120
121if ($is_milestone) {
122if (($keep_members != 'parent') && $parent->getHasSubprojects()) {
123throw new PhutilArgumentUsageException(
124pht(
125'You can not use "child" or "both" modes when making a project a '.
126'milestone of a project with existing subprojects: there is nowhere '.
127'to put the members.'));
128}
129
130$copy_parent = false;
131$copy_child = ($keep_members != 'parent');
132$wipe_parent = ($keep_members == 'child');
133$wipe_child = true;
134} else {
135$copy_parent = ($keep_members != 'child');
136$copy_child = false;
137$wipe_parent = true;
138$wipe_child = ($keep_members == 'parent');
139}
140
141$child->setParentProjectPHID($parent->getPHID());
142$child->attachParentProject($parent);
143
144if ($is_milestone) {
145$next_number = $parent->loadNextMilestoneNumber();
146$child->setMilestoneNumber($next_number);
147}
148
149$child->save();
150
151$member_type = PhabricatorProjectProjectHasMemberEdgeType::EDGECONST;
152
153$parent_members = PhabricatorEdgeQuery::loadDestinationPHIDs(
154$parent->getPHID(),
155$member_type);
156
157$child_members = PhabricatorEdgeQuery::loadDestinationPHIDs(
158$child->getPHID(),
159$member_type);
160
161if ($copy_parent) {
162edit_members($parent_members, $child, true);
163}
164
165if ($copy_child) {
166edit_members($child_members, $parent, true);
167}
168
169if ($wipe_parent) {
170edit_members($parent_members, $parent, false);
171}
172
173if ($wipe_child) {
174edit_members($child_members, $child, false);
175}
176
177id(new PhabricatorProjectsMembershipIndexEngineExtension())
178->rematerialize($parent);
179
180id(new PhabricatorProjectsMembershipIndexEngineExtension())
181->rematerialize($child);
182
183echo tsprintf(
184"%s\n",
185pht('Done.'));
186
187
188function load_project($name) {
189$viewer = PhabricatorUser::getOmnipotentUser();
190
191$project = id(new PhabricatorProjectQuery())
192->setViewer($viewer)
193->withSlugs(array($name))
194->executeOne();
195if ($project) {
196return $project;
197}
198
199$project = id(new PhabricatorProjectQuery())
200->setViewer($viewer)
201->withPHIDs(array($name))
202->executeOne();
203if ($project) {
204return $project;
205}
206
207$project = id(new PhabricatorProjectQuery())
208->setViewer($viewer)
209->withIDs(array($name))
210->executeOne();
211if ($project) {
212return $project;
213}
214
215throw new Exception(
216pht(
217'Unknown project "%s"! Use a hashtags, PHID, or ID to choose a project.',
218$name));
219}
220
221function edit_members(array $phids, PhabricatorProject $target, $add) {
222if (!$phids) {
223return;
224}
225
226$member_type = PhabricatorProjectProjectHasMemberEdgeType::EDGECONST;
227
228$editor = id(new PhabricatorEdgeEditor());
229foreach ($phids as $phid) {
230if ($add) {
231$editor->addEdge($target->getPHID(), $member_type, $phid);
232} else {
233$editor->removeEdge($target->getPHID(), $member_type, $phid);
234}
235}
236$editor->save();
237}


Additional Links

There are a very large number of changes, so older changes are hidden. Show Older Changes

My tentative plan here is to write a script that does it and make it available in this task. I want to do this a bit on this install too, mostly so we can have better direct exposure to these features ourselves.

If there's widespread/long-term interest we can look at bringing it upstream, but hopefully this problem is mostly a one-time problem caused by the introduction of subprojects. It's going to be somewhat hard to test/maintain in the future (e.g., no one is going to remember to go move subprojects when they make some unrelated change, and writing a unit test for "subprojects moved without any problems" seems infeasible to me) so if I'd be happy if I can get away without committing to it working forever.

One issue is that there's no unambiguous way to resolve memberships. Superproject members are the union of all subproject members, but if you ./make-subproject-of A B and they have disjoint membership sets, we need to do something weird.

Membership is generally less necessary at HEAD than it previously was (you no longer need to be a member to watch a project) so maybe we just prevent the script from running if B has members and is not already a superproject. Then you'd have two options to resolve membership:

  • kick everyone out of B, so there's no ambiguity; or
  • create a "B > Old Team Members" subproject first.

@jdforrester, in your situation, how would you want to handle members?

@jdforrester, in your situation, how would you want to handle members?

I don't know. Kicking everyone out and manually cross-checking that everyone in B is in A sounds lame; creating a sub-team seems… too formal for any use case of mine (absent ACL projects, but I'd really hope that anyone using this script Knew What They Were Doing™). Possibly just add all members of A to B?

We have like 50 "Team X Sprint 123" projects that would be nice to turn into "actual" sprints. But they are all old sprints so that's just for my need to keep things looking tidy.

chad added a subscriber: chad.Feb 12 2016, 11:11 PM

@epriestley I need to suggest a profile photo to @cburroughs

Haha. I really do think "suggest a profile picture" might turn out to be pretty fun. Maybe I'll kick out that and {button} if I want to take a break before doing callsigns.

ftdysa added a subscriber: ftdysa.Feb 12 2016, 11:37 PM

Commenting to show my interest in such a script. I'd be completely fine if the migration script did what @jdforrester described and merged members of A to B.

epriestley updated the task description. (Show Details)Feb 13 2016, 12:17 PM
epriestley renamed this task from Provide a tool (script?) to mark an existing project as a sub-project of another one to Scripts to migrate old "points" fields and move projects beneath other projects.
epriestley claimed this task.
epriestley triaged this task as Normal priority.
epriestley updated the task description. (Show Details)
epriestley updated the task description. (Show Details)
epriestley updated the task description. (Show Details)Feb 13 2016, 1:23 PM
epriestley updated the task description. (Show Details)
epriestley updated the task description. (Show Details)Feb 13 2016, 1:25 PM

This task now has scripts to migrate points and move projects underneath other projects. Let me know if you run into issues.

epriestley updated the task description. (Show Details)Feb 13 2016, 1:36 PM
chad awarded a token.Feb 13 2016, 3:28 PM
nochum added a subscriber: nochum.Feb 15 2016, 2:23 PM
joshma added a subscriber: joshma.EditedFeb 17 2016, 2:47 AM

We're using the wikimedia sprint extension, which adds a isdc:sprint:storypoints field - upon running ./copy_points.php --field isdc:sprint:storypoints, we get

[2016-02-16 18:46:22] EXCEPTION: (PhutilArgumentUsageException) Field "isdc:sprint:storypoints" is not an "int" field, and can not be migrated. at [<phabricator>/copy_points.php:62]
arcanist(head=master, ref.master=fcc11b3a2781), phabricator(head=master, ref.master=608fcdd9dd70, custom=2), phutil(head=master, ref.master=f43291e99d36), sprint(head=master, ref.master=802afc636035)

Is there a way to "cast" to an int?

You may be able to just comment out that check, but I'm not sure how the extension stores its data. You'd have to check with the authors to be sure.

Ah cool, I think that'll work, noticing this:

UPDATE `phabricator_maniphest`.`maniphest_task` SET points = 2 WHERE id = 4130;
UPDATE `phabricator_maniphest`.`maniphest_task` SET points = 2 WHERE id = 4129;
HAS POINTSHAS POINTSUPDATE `phabricator_maniphest`.`maniphest_task` SET points = 0 WHERE id = 4201;
HAS POINTSHAS POINTSHAS POINTSHAS POINTS

Are the "HAS POINTS" supposed to be printing? (L97)

No, that was some stray debugging code. I updated the script.

👍 worked fine for me, thank you!

That 'int' requirement is definitely misleading, since the field supports fractional point values (which makes me happy, since we have .5-point tasks in our workflow).

Yeah, the int is mostly out of an abundance of caution. There's currently no float custom field type, and if you previously used string you probably need to massage values anyway since some of the fields may have strings in them (I think WMF had a few strings like huge, epic, etc). You can comment out the int check safely, but may need to add some code to examine $value for usual values if you do (or accept that they'll probably be cast to 0).

I can update the script with a "no int check + warn on weird values" version if anyone wants that but isn't too comfortable with PHP.

I had to add an explicit cast to int when I ran the conversion script, but other than the 3 or 4 tasks which lost their points value, everything else worked nicely.

urzds added a subscriber: urzds.Feb 22 2016, 11:55 AM
urzds added a comment.EditedFeb 22 2016, 2:14 PM

I was using the move_underneath.php script and noticed that now some tasks are assigned to several milestones in weird ways.

Walking through this list, you can hopefully reproduce this:

  1. Have an old project A that is assigned some tasks. These tasks are also assigned to a milestone B(Z) of another project.
  2. Call move_underneath.php --parent B --child A --milestone --keep-members both
  3. See that the task shows up on the new (A) column on the B workboard.
  4. Try to move the task from column (A) to (Z) on the B workboard - this makes the UI hang and the result is not permanent.
  5. Open the task. See that it is assigned to project B in the (A) column.
  6. Edit the task. See that it is assigned to both projects A and B.
  7. Remove assignment of a task to A, but leave the assignment to B. Result: In the view UI it appears as assigned to B on both the (Z) and (A) columns (note that Phab does not use the milestone tag, but the project tag plus the column).
  8. Back in the edit UI, remove the assignment to B. Result: In the view UI it appears as assigned to no projects ("None").
  9. Add the assignment to B again. Result: Appears to be assinged to B on the (Z) and (A) columns again.

It appears impossible to clean up the relation between the task and B(Z) / B(A) from the UI.

Would be nice to be able to skip --keep-members altogether and not move the members over at all. Thanks for the script!

@nochum, can you explain what you're trying to do in more detail?

I believe --keep-members already allows you to select all legal behaviors. If you don't want to move members from the child, use --keep-members parent.

With --keep-members parent, the child's members will be wiped out. If the child is becoming a milestone, nothing else happens. If the child is becoming a subproject, the parent's members are then copied to the child.

I want to have it not copy anything to the child as I don't want any membership on the subproject. My members will be defined on one of the subproject's siblings and this subproject's membership will be empty. So, eg, my structure would be:

  • Parent
    • Members (here go all the memberships for this parent)
    • Subproject A (no members)
    • Subproject B (no members)

Hope that makes sense.

If the parent is already a parent project of some other project, it has no direct members, so nothing will be copied. --keep-members parent should do what you want, provided you move "Members" first, before "Subproject A" or "Subproject B".

urzds added a comment.Feb 23 2016, 5:09 PM

[...]
It appears impossible to clean up the relation between the task and B(Z) / B(A) from the UI.

I tried today using the batch editor:

  1. Select the task e.g. by showing Hidden Columns and All Tasks on the workboard.
  2. Batch Edit Tasks in the column menu for the column you want to change.
  3. Remove Project: The milestone you want to remove.
  4. Update Tasks
  5. Back at the workboard show hidden columns and closed tasks
  6. Notice that the tasks are still shown in the milestone column you just removed them from.

tl;dr: I am still searching for a way to clean up this mess.

urzds added a comment.EditedFeb 23 2016, 5:28 PM

If I was to delete the broken project (bin/remove destroy PHID-PROJ-...), would that clean up the database in such a way that the broken links were also removed? Or would that break the database even more?

guayosr moved this task from Backlog to Chad on the Projects (v3) board.Feb 25 2016, 9:41 PM
guayosr moved this task from Chad to Backlog on the Projects (v3) board.

Worked like a charm! Thanks.

cmmata added a subscriber: cmmata.Mar 23 2016, 9:10 AM

How (or where) can I remove "old" points custom field? I successfully migrated all points to the new Maniphest field, but now I have two story points fields. I'm taking a look at the database, but there's no direct field in maniphest_task. I suppose it's the reference in maniphest_customfieldstorage with fieldIndex yERhvoZPNPtM (at least in my case), but I prefer not to alter a database without knowing all the places I have to make changes.

@cmmata Configuring Custom Fields may help. In your installation, you'd navigate to /config/edit/maniphest.custom-field-definitions/ to edit the Maniphest custom field configuration.

@cmmata Configuring Custom Fields may help. In your installation, you'd navigate to /config/edit/maniphest.custom-field-definitions/ to edit the Maniphest custom field configuration.

I'm sorry, I totally forgot I'm using Wikimedia's Sprint Extension and not only the main project. If I navigate there, I can see my own custom fields, but I can't see that extension's story points field. I asked here because I was taking a look at the script, but I think I should ask this in wikimedia's instance and not here.

If anyone else has the same problem, here is the solution. You have to edit maniphest.fields and not maniphest.custom-field-definitions.

Isn't that about editing the config to have maniphest.fields say "isdc:sprint:storypoints": { "key": "isdc:sprint:storypoints", "disabled": true } ?
rbalik added a subscriber: rbalik.Mar 31 2016, 1:35 AM

This is useful, but are there any plans to add this functionality to the GUI? Would be nice to have for when we're doing some housekeeping

This is useful, but are there any plans to add this functionality to the GUI? Would be nice to have for when we're doing some housekeeping

No. See description:

Neither workflow is formally supported by the upstream, and we do not currently plan to maintain these scripts. The were written circa February 2016, and may not work (and may even be dangerous) if run against future versions of Phabricator.

Pawka added a subscriber: Pawka.Apr 13 2016, 2:05 PM

I'm in the position where I need to re-parent an existing hierarchy of projects, which isn't supported by @epriestley's move_beneath.php above. Rather than attempt to figure out everything involved in doing that (which I think is mostly messing with the various projects' path, pathKey, and depth fields, in case anyone wants to try) I threw this script together to move an existing subproject or milestone to root level, where it can then be moved using move_beneath.php.

Please back up your phabricator_project database before using this. It's highly experimental. Use it in a similar fashion:

  • Put it in phabricator/
  • chmod +x move_to_root.php to make it executable
  • Run it with ./move_to_root.php --project your_project_slug

{P1975}

Oh, one problem with that script: when you move all the subprojects or milestones out from under a parent, it doesn't set the parent's hasMilestones or hasSubprojects fields to 0 for you. That'll have to be done by hand in the database.

beber added a subscriber: beber.Jun 9 2016, 9:25 PM

It seems the move_beneath.php doesn't work any more with the latest code.

# ./move_beneath.php --parent infra_dev --child backend --keep-members both --subproject
[2016-07-07 16:34:03] EXCEPTION: (AphrontParameterQueryException) Expected a numeric scalar or null for %Ld conversion. Query: id IN (%Ld) at [<phutil>/src/xsprintf/qsprintf.php:294]
arcanist(head=master, ref.master=7b0aac5c6f31), phabricator(head=cyyun, ref.master=1558175ec84c, ref.cyyun=4276472310ea, custom=1), phutil(head=cyyun, ref.master=ad458fb7df59, ref.cyyun=be57e3e80ea1), sprint(head=master, ref.master=df6e9dee03e4)
  #0 qsprintf_check_scalar_type(string, string, string) called at [<phutil>/src/xsprintf/qsprintf.php:267]
  #1 qsprintf_check_type(array, string, string) called at [<phutil>/src/xsprintf/qsprintf.php:134]
  #2 xsprintf_query(AphrontMySQLiDatabaseConnection, string, integer, array, integer) called at [<phutil>/src/xsprintf/xsprintf.php:70]
  #3 xsprintf(string, AphrontMySQLiDatabaseConnection, array) called at [<phutil>/src/xsprintf/qsprintf.php:64]
  #4 qsprintf(AphrontMySQLiDatabaseConnection, string, array) called at [<phabricator>/src/applications/project/query/PhabricatorProjectQuery.php:429]
  #5 PhabricatorProjectQuery::buildWhereClauseParts(AphrontMySQLiDatabaseConnection) called at [<phabricator>/src/infrastructure/query/policy/PhabricatorCursorPagedPolicyAwareQuery.php:266]
  #6 PhabricatorCursorPagedPolicyAwareQuery::buildWhereClause(AphrontMySQLiDatabaseConnection) called at [<phabricator>/src/infrastructure/query/policy/PhabricatorCursorPagedPolicyAwareQuery.php:96]
  #7 PhabricatorCursorPagedPolicyAwareQuery::loadStandardPageRows(PhabricatorProject) called at [<phabricator>/src/infrastructure/query/policy/PhabricatorCursorPagedPolicyAwareQuery.php:82]
  #8 PhabricatorCursorPagedPolicyAwareQuery::loadStandardPage(PhabricatorProject) called at [<phabricator>/src/applications/project/query/PhabricatorProjectQuery.php:223]
  #9 PhabricatorProjectQuery::loadPage() called at [<phabricator>/src/infrastructure/query/policy/PhabricatorPolicyAwareQuery.php:227]
  #10 PhabricatorPolicyAwareQuery::execute() called at [<phabricator>/src/infrastructure/query/policy/PhabricatorPolicyAwareQuery.php:167]
  #11 PhabricatorPolicyAwareQuery::executeOne() called at [<phabricator>/move_beneath.php:210]
  #12 load_project(string) called at [<phabricator>/move_beneath.php:88]
chad added a comment.Jul 7 2016, 1:59 PM

phabricator(head=cyyun, ref.master=1558175ec84c, ref.cyyun=4276472310ea, custom=1), phutil(head=cyyun, ref.master=ad458fb7df59, ref.cyyun=be57e3e80ea1), sprint(head=master, ref.master=df6e9dee03e4)

@RuralHunter these aren't versions of Phabricator we produce. How have you modified your local install?

Yes, It was some local encoding handling related and should not be related to the error. My code base is this:

commit 1558175ec84c7b13ba3f692b897464901bc00578
Author: epriestley <git@epriestley.com>
Date:   Sat Jun 11 04:44:40 2016 -0700

    (stable) Promote 2016 Week 24
chad added a comment.Jul 8 2016, 1:12 AM

I can't reproduce any errors with the script against a new/clean install. Any ideas on what I can do differently to reproduce this?

Thanks, I will try to test more.

OK, I got it worked for 2 other projects. So there must be something special with the 2 projects I tried previously.

RuralHunter added a comment.EditedJul 8 2016, 2:58 AM

Well, it turns out I mis-typed the project name! It's just the error message not so friendly. I can reproduce the error with this:

./move_beneath.php --parent aaa --child bbb --keep-members both --subproject
tigzav added a subscriber: tigzav.Feb 24 2017, 10:36 AM

Thanks @epriestley and @rfreebern - your scripts worked perfectly! Between them both (and tweaking that one "hasSubprojects" field in the database myself), I was able to move an entire project tree. Yay!