Page MenuHomePhabricator

Write some custom Herald glue for driving builds through Owners
Closed, ResolvedPublic

Assigned To
Authored By
epriestley
Sep 6 2015, 7:27 PM
Referenced Files
F864068: victory.gif
Oct 5 2015, 7:47 PM
F858910: buildvar.png
Oct 2 2015, 12:42 PM
F858918: Screen Shot 2015-10-02 at 5.41.42 AM.png
Oct 2 2015, 12:42 PM
F858920: Screen Shot 2015-10-02 at 5.41.11 AM.png
Oct 2 2015, 12:42 PM
F846659: Screen Shot 2015-09-28 at 12.01.22 PM.png
Sep 28 2015, 7:05 PM
F846663: Screen Shot 2015-09-28 at 12.02.40 PM.png
Sep 28 2015, 7:05 PM
F846666: Screen Shot 2015-09-28 at 11.48.22 AM.png
Sep 28 2015, 7:05 PM
F846669: Screen Shot 2015-09-28 at 12.01.04 PM.png
Sep 28 2015, 7:05 PM

Description

Until Harbormaster is actually smart, we can fake build smartness with a small amount of custom glue in Herald.

Revisions and Commits

Event Timeline

epriestley raised the priority of this task from to Normal.
epriestley updated the task description. (Show Details)
epriestley moved this task from Backlog to The Queue on the Prioritized board.
epriestley added a subscriber: epriestley.

Quick progress update here: Owners packages now have configurable custom fields, so you can add one like this:

Screen Shot 2015-09-28 at 12.01.22 PM.png (1×1 px, 178 KB)

That shows up in the UI like this, and you could list a bunch of build names in it:

Screen Shot 2015-09-28 at 12.02.40 PM.png (1×1 px, 151 KB)

Then you drop this into phabricator/src/extensions/:

1<?php
2
3final class CustomDifferentialRunPackageBuildsHeraldAction
4 extends HeraldAction {
5
6 const ACTIONCONST = 'custom.differential.run-package-builds';
7
8 const DO_BUILD = 'do.custom-package-build';
9
10 public function getHeraldActionName() {
11 return pht('Run builds for affected packages');
12 }
13
14 public function getActionGroupKey() {
15 return HeraldApplicationActionGroup::ACTIONGROUPKEY;
16 }
17
18 public function supportsObject($object) {
19 return ($object instanceof DifferentialRevision);
20 }
21
22 public function supportsRuleType($rule_type) {
23 return ($rule_type == HeraldRuleTypeConfig::RULE_TYPE_GLOBAL);
24 }
25
26 public function applyEffect($object, HeraldEffect $effect) {
27 $viewer = PhabricatorUser::getOmnipotentUser();
28
29 // Replace this with the full field key of the custom field you want to
30 // examine. Generally, "std:owners:" plus whatever you named the field.
31 $field_key = "std:owners:mycompany:lore";
32
33 // Replace this with the PHID of the build plan you want to execute.
34 $buildplan_phid = "PHID-HMCP-vadfbuqvdbx5gy65pfba";
35
36 $adapter = $this->getAdapter();
37 $packages = $adapter->loadAffectedPackages();
38
39 $package_phids = mpull($packages, 'getPHID');
40 $this->logEffect(self::DO_BUILD, $package_phids);
41
42 $all = array();
43 foreach ($packages as $package) {
44 $field_list = PhabricatorCustomField::getObjectFields(
45 $package,
46 PhabricatorCustomField::ROLE_VIEW);
47 $field_list
48 ->setViewer($viewer)
49 ->readFieldsFromStorage($package);
50
51 foreach ($field_list->getFields() as $field) {
52 $key = $field->getFieldKey();
53 if ($key == $field_key) {
54 $all[] = $field->getValueForStorage();
55 }
56 }
57 }
58
59 $all = implode("\n", $all);
60 $all = phutil_split_lines($all, false);
61 $all = array_filter($all);
62 $all = array_unique($all);
63
64 if (!$all) {
65 return;
66 }
67
68 foreach ($all as $parameter) {
69 $request = id(new HarbormasterBuildRequest())
70 ->setBuildPlanPHID($buildplan_phid)
71 ->setBuildParameters(
72 array(
73 'build-name-from-owners' => $parameter,
74 ));
75
76 $adapter->queueHarbormasterBuildRequest($request);
77 }
78 }
79
80 public function getHeraldActionStandardType() {
81 return self::STANDARD_NONE;
82 }
83
84 public function renderActionDescription($value) {
85 return pht('Run builds for affected pakages.');
86 }
87
88 protected function getActionEffectMap() {
89 return array(
90 self::DO_BUILD => array(
91 'icon' => 'fa-play',
92 'color' => 'green',
93 'name' => pht('Ran Builds'),
94 ),
95 );
96 }
97
98 protected function renderActionEffectDescription($type, $data) {
99 switch ($type) {
100 case self::DO_BUILD:
101 if (!$data) {
102 return pht('No affected packages.');
103 } else {
104 return pht(
105 'Ran builds for affected packages: %s.',
106 $this->renderHandleList($data));
107 }
108 }
109 }
110
111}

...and write a rule to run the action unconditionally:

Screen Shot 2015-09-28 at 11.48.22 AM.png (878×1 px, 148 KB)

This almost works. It starts the builds correctly (here, the three "HTTP request to Google" builds were started because of this chain of configuration/magic):

Screen Shot 2015-09-28 at 12.01.04 PM.png (1×1 px, 177 KB)

However, there's currently no way to parameterize at the build level, so these three builds are identical instead of differing by a parameter. Fixing that will require some upstream adjustments.

My plan is to allow parameterization at the build level; D13635 is an implementation of that but it has some other stuff mixed in and needs to be pulled apart a bit. I think this change is straightforward overall, though.

After D14222, the updated P1860 should pass parameters properly. Here's a target getting a build parameter:

buildvar.png (989×1 px, 136 KB)

So your target can use ${build/build-name-from-owners} to pass this to something like Jenkins:

Screen Shot 2015-10-02 at 5.41.42 AM.png (205×620 px, 22 KB)

...and it should pop out on the other side:

Screen Shot 2015-10-02 at 5.41.11 AM.png (486×1 px, 69 KB)

  • The way to do multiple builds per package is to use a remarkup custom field and separate them with newlines, correct?
  • The herald rule should be "Always" instead of something to do with packages because CustomDifferentialRunPackageBuildsHeraldAction has sufficient logic of it's own?

The way to do multiple builds per package is to use a remarkup custom field and separate them with newlines, correct?

Yeah, at least as written. You could tweak the logic or I can swap it for something else if that's easier.

The herald rule should be "Always" instead of something to do with packages because CustomDifferentialRunPackageBuildsHeraldAction has sufficient logic of it's own?

Yeah. You could add additional conditions, but the action only runs build plans per configuration in affected packages, so there's no way to run "too many" plans. But you could add rules like "Day of week is not tuesday" if you have "No Tests Tuesdays" in your company or whatever.

Yeah, at least as written. You could tweak the logic or I can swap it for something else if that's easier.

A list feels ideal, but that's not an existing custom field type. It's mildly unsightly to have the full remarkup bar there but that isn't a big deal at all.

I've promoted this to our primary installation and arranged all the sticky gluey ends so they connect together and did a full end to end test. It worked, and everything seems good in theory, yeah!

victory.gif (48×32 px, 873 B)

My next step is to start adding more builds and confirming everything holds up in practice.

At HEAD, you can now edit this field via the owners.edit Conduit API method.

epriestley claimed this task.

Custom fields (and paths, too) can now be read with owners.search and written with owners.edit. There's a description of a more complex edit (against paths) here, but the same general workflow applies to examining and updating custom fields:

https://secure.phabricator.com/phame/post/view/752/reading_and_writing_paths_in_owners/

We ran into a case where archived packages were trigger builds, which didn't feel right and was causing problkems. I'm not sure what the default behavior loadAffectedPackages should be or what would be most consistent with how archiving is handled elsewhere.

diff --git a/config/phabricator/extensions/CustomDifferentialRunPackageBuildsHeraldAction.php b/config/phabricator/extensions/CustomDifferentialRunPackageBuildsHeraldAction.php
index 5bf3593..366b06d 100644
--- a/config/phabricator/extensions/CustomDifferentialRunPackageBuildsHeraldAction.php
+++ b/config/phabricator/extensions/CustomDifferentialRunPackageBuildsHeraldAction.php
@@ -34,7 +34,7 @@ final class CustomDifferentialRunPackageBuildsHeraldAction
     $buildplan_phid = "PHID-HMCP-ue7dbgw57fwxzmeadqfx";
 
     $adapter = $this->getAdapter();
-    $packages = $adapter->loadAffectedPackages();
+    $packages = mfilter($adapter->loadAffectedPackages(), 'isArchived', true);
 
     $package_phids = mpull($packages, 'getPHID');
     $this->logEffect(self::DO_BUILD, $package_phids);