Page MenuHomePhabricator

Harbormaster working copy build step shouldn't crash when user don't set any also clone repository
Closed, ResolvedPublic

Description

EDIT : As bugreports goes for this bug, i identify two concerns (see comments):

  • DrydockLease::setAllowedBlueprintPHIDs() is not an array when the field also clone is empty thus the exception is handled not correctly and this gives cryptic error : this can be fix easily (see comment? my diff was tested in production with succes)
  • this doesn't seems the correct way to do : also clone means that it should by default clone buildable associated repo and also, if needed other repo(like if you build rP here you will clone by default rP and you may want to also clone rPHU and rARC) : this can be easily fixed by mimicing HarbormasterLeaseWorkingCopyBuildStepImplementation::buildRepositoryMap into HarbormasterLeaseWorkingCopyBuildStepImplementation::execute(). If also clone is not clone rX also clone rY, the step needs to be renamed and make required.

I would like to contribute on this !


I have Drydock working correctly (I mean Almanac works, Almanac Based Host Blueprint works, Working Copy Works, Stagging area and I'm capable of landing pasta in my personal repo).

This is the data I have.

USE phabricator_harbormaster;
SELECT * FROM harbormaster_buildstep WHERE className = "HarbormasterLeaseWorkingCopyBuildStepImplementation";
idphidbuildPlanPHIDclassNamedetailsdateCreateddateModifiedsequencenamedescriptionstepAutoKey
5PHID-HMCS-dxekix3bwt22yk3u6xq4PHID-HMCP-viylmanphmudsc4wiuhoHarbormasterLeaseWorkingCopyBuildStepImplementation{"dependsOn":[],"name":"wc","blueprintPHIDs":"[\"PHID-DRYB-nc3ceb6udcgce6alluh5\"]"}144611663114461177580Lease Working CopyNULL
USE phabricator_drydock;
SELECT * FROM drydock_blueprint;
idphidclassNameblueprintNameviewPolicyeditPolicydetailsdateCreateddateModifiedisDisabled
1PHID-DRYB-c7fvz5ka4doxwbxmwb3sDrydockAlmanacServiceHostBlueprintImplementationhigh trusted almanac hostusersadmin{"almanacServicePHIDs":"[\"PHID-ASRV-dsqpay3ivuwuj6zr4t7n\"]","credentialPHID":"PHID-CDTL-qbmxmx24js6suezzvlux"}144603776314460381600
2PHID-DRYB-nc3ceb6udcgce6alluh5DrydockWorkingCopyBlueprintImplementationhigh trusted working copyusersadmin{"blueprintPHIDs":"[\"PHID-DRYB-c7fvz5ka4doxwbxmwb3s\"]"}144603813414460381340
SELECT * FROM drydock_authorization;
idphidblueprintPHIDblueprintAuthorizationStateobjectPHIDobjectAuthorizationStatedateCreateddateModified
1PHID-DRYA-qgnpalkgkjfeigfk4vnjPHID-DRYB-c7fvz5ka4doxwbxmwb3sauthorizedPHID-DRYB-nc3ceb6udcgce6alluh5active14460381341446038144
3PHID-DRYA-z7r64liiqrmwgex55jjqPHID-DRYB-nc3ceb6udcgce6alluh5authorizedPHID-REPO-hbsyooqi3rjbxpq675m3active14460568811446056892
4PHID-DRYA-sduc27zrt4hhhrvb4zuyPHID-DRYB-nc3ceb6udcgce6alluh5authorizedPHID-HMCS-dxekix3bwt22yk3u6xq4active14461166311446116645

To sum it up an harbormaster working copy build step, producing an wc artifact, connected to a authorized working copy blueprint phid.
The working copy blueprint also authorized the repo and the the almanac host blueprint authorize the working copy blueprint.

But when I test this single build step against a commit running manually a build plan i get :

bin/harbormaster build rPXf53ae675fd0ad9cb8c6c99d98b7fdb0f1d738a58 --plan 4 --trace
>>> [2] <connect> phabricator_repository
<<< [2] <connect> 920 us
>>> [3] <query> SELECT `r`.* FROM `repository` r  WHERE ((r.callsign IN ('PX')))   ORDER BY `r`.`id` DESC 
<<< [3] <query> 170 us
>>> [4] <query> SELECT `commit`.* FROM `repository_commit` commit  WHERE (((commit.repositoryID = 2 AND commit.commitIdentifier LIKE 'f53ae675fd0ad9cb8c6c99d98b7fdb0f1d738a58%')))   ORDER BY `commit`.`id` DESC 
<<< [4] <query> 550 us
>>> [5] <query> SELECT `r`.* FROM `repository` r  WHERE (r.id IN (2))   ORDER BY `r`.`id` DESC 
<<< [5] <query> 125 us
>>> [6] <connect> phabricator_harbormaster
<<< [6] <connect> 260 us
>>> [7] <query> SELECT * FROM `harbormaster_buildplan`   WHERE (id IN (4))   ORDER BY `id` DESC 
<<< [7] <query> 117 us
>>> [8] <connect> phabricator_harbormaster
<<< [8] <connect> 246 us
>>> [9] <query> INSERT INTO `harbormaster_buildable` (`buildablePHID`, `containerPHID`, `buildableStatus`, `isManualBuildable`, `phid`, `dateCreated`, `dateModified`) VALUES ('PHID-CMIT-fwgv3mj3bx5qi3ips5he', 'PHID-REPO-hbsyooqi3rjbxpq675m3', 'building', '1', 'PHID-HMBB-7ogxe5gkv27zwbgfjo75', '1446121190', '1446121190')
<<< [9] <query> 21,119 us
Applying plan 4 to new buildable B9...

    http://redacted.url/B9

>>> [10] <query> INSERT INTO `harbormaster_build` (`buildablePHID`, `buildPlanPHID`, `buildStatus`, `buildGeneration`, `buildParameters`, `planAutoKey`, `phid`, `dateCreated`, `dateModified`) VALUES ('PHID-HMBB-7ogxe5gkv27zwbgfjo75', 'PHID-HMCP-viylmanphmudsc4wiuho', 'pending', '0', '[]', NULL, 'PHID-HMBD-oxzm26jlbqlhxsz5geii', '1446121190', '1446121190')
<<< [10] <query> 12,367 us
>>> [11] <query> SELECT * FROM `harbormaster_build`   WHERE (id IN (9))   ORDER BY `id` DESC 
<<< [11] <query> 297 us
>>> [12] <query> SELECT * FROM `harbormaster_buildable`   WHERE (phid IN ('PHID-HMBB-7ogxe5gkv27zwbgfjo75'))   ORDER BY `id` DESC 
<<< [12] <query> 209 us
>>> [13] <query> SELECT `commit`.* FROM `repository_commit` commit  WHERE (commit.phid IN ('PHID-CMIT-fwgv3mj3bx5qi3ips5he'))   ORDER BY `commit`.`id` DESC 
<<< [13] <query> 115 us
>>> [14] <query> SELECT `r`.* FROM `repository` r  WHERE (r.id IN (2))   ORDER BY `r`.`id` DESC 
<<< [14] <query> 111 us
>>> [15] <query> SELECT * FROM `harbormaster_buildplan`   WHERE (phid IN ('PHID-HMCP-viylmanphmudsc4wiuho'))   ORDER BY `id` DESC 
<<< [15] <query> 106 us
>>> [16] <query> SELECT * FROM `harbormaster_buildcommand` WHERE targetPHID IN ('PHID-HMBD-oxzm26jlbqlhxsz5geii') ORDER BY id ASC 
<<< [16] <query> 253 us
>>> [17] <lock> ph:phabric-6rILUDHKjHKu:harbormaster.build:9
>>> [18] <connect> phabricator_repository
<<< [18] <connect> 279 us
>>> [19] <query> SET wait_timeout = 2147483
<<< [19] <query> 97 us
>>> [20] <query> SELECT GET_LOCK('ph:phabric-6rILUDHKjHKu:harbormaster.build:9', 15)
<<< [20] <query> 123 us
>>> [21] <query> SELECT * FROM `harbormaster_build` WHERE `id` = 9 
<<< [21] <query> 186 us
>>> [22] <query> SELECT * FROM `harbormaster_buildtarget`   WHERE (buildPHID in ('PHID-HMBD-oxzm26jlbqlhxsz5geii')) AND (buildGeneration in (0))   ORDER BY `id` DESC 
<<< [22] <query> 463 us
>>> [23] <query> UPDATE `harbormaster_build` SET `buildablePHID` = 'PHID-HMBB-7ogxe5gkv27zwbgfjo75', `buildPlanPHID` = 'PHID-HMCP-viylmanphmudsc4wiuho', `buildStatus` = 'building', `buildGeneration` = '1', `buildParameters` = '[]', `planAutoKey` = NULL, `id` = '9', `phid` = 'PHID-HMBD-oxzm26jlbqlhxsz5geii', `dateCreated` = '1446121190', `dateModified` = '1446121190' WHERE `id` = '9'
<<< [23] <query> 12,564 us
>>> [24] <query> SELECT * FROM `harbormaster_buildtarget`   WHERE (buildPHID in ('PHID-HMBD-oxzm26jlbqlhxsz5geii')) AND (buildGeneration in (1))   ORDER BY `id` DESC 
<<< [24] <query> 249 us
>>> [25] <query> SELECT * FROM `harbormaster_buildstep` WHERE (buildPlanPHID in ('PHID-HMCP-viylmanphmudsc4wiuho')) ORDER BY `id` DESC 
<<< [25] <query> 115 us
>>> [26] <query> SELECT * FROM `harbormaster_buildplan`   WHERE (phid IN ('PHID-HMCP-viylmanphmudsc4wiuho'))   ORDER BY `id` DESC 
<<< [26] <query> 111 us
>>> [27] <query> SELECT * FROM `harbormaster_buildstep` WHERE (buildPlanPHID in ('PHID-HMCP-viylmanphmudsc4wiuho')) ORDER BY `id` DESC 
<<< [27] <query> 149 us
>>> [28] <query> SELECT * FROM `harbormaster_buildplan`   WHERE (phid IN ('PHID-HMCP-viylmanphmudsc4wiuho'))   ORDER BY `id` DESC 
<<< [28] <query> 107 us
>>> [29] <query> SELECT * FROM `harbormaster_buildstep` WHERE (buildPlanPHID in ('PHID-HMCP-viylmanphmudsc4wiuho')) ORDER BY `id` DESC 
<<< [29] <query> 106 us
>>> [30] <query> SELECT * FROM `harbormaster_buildplan`   WHERE (phid IN ('PHID-HMCP-viylmanphmudsc4wiuho'))   ORDER BY `id` DESC 
<<< [30] <query> 114 us
>>> [31] <query> INSERT INTO `harbormaster_buildtarget` (`name`, `buildPHID`, `buildStepPHID`, `className`, `details`, `variables`, `targetStatus`, `dateStarted`, `dateCompleted`, `buildGeneration`, `phid`, `dateCreated`, `dateModified`) VALUES ('Lease Working Copy', 'PHID-HMBD-oxzm26jlbqlhxsz5geii', 'PHID-HMCS-dxekix3bwt22yk3u6xq4', 'HarbormasterLeaseWorkingCopyBuildStepImplementation', '{\"dependsOn\":[],\"name\":\"wc\",\"blueprintPHIDs\":\"[\\\"PHID-DRYB-nc3ceb6udcgce6alluh5\\\"]\"}', '{\"buildable.commit\":\"f53ae675fd0
<<< [31] <query> 13,558 us
>>> [32] <query> SELECT RELEASE_LOCK('ph:phabric-6rILUDHKjHKu:harbormaster.build:9')
<<< [32] <query> 174 us
<<< [17] <lock> 50,952 us
>>> [33] <query> SELECT * FROM `harbormaster_buildtarget`   WHERE (id IN (14))   ORDER BY `id` DESC 
<<< [33] <query> 247 us
>>> [34] <query> SELECT * FROM `harbormaster_build`   WHERE (phid in ('PHID-HMBD-oxzm26jlbqlhxsz5geii'))   ORDER BY `id` DESC 
<<< [34] <query> 196 us
>>> [35] <query> SELECT * FROM `harbormaster_buildable`   WHERE (phid IN ('PHID-HMBB-7ogxe5gkv27zwbgfjo75'))   ORDER BY `id` DESC 
<<< [35] <query> 108 us
>>> [36] <query> SELECT `commit`.* FROM `repository_commit` commit  WHERE (commit.phid IN ('PHID-CMIT-fwgv3mj3bx5qi3ips5he'))   ORDER BY `commit`.`id` DESC 
<<< [36] <query> 124 us
>>> [37] <query> SELECT `r`.* FROM `repository` r  WHERE (r.id IN (2))   ORDER BY `r`.`id` DESC 
<<< [37] <query> 121 us
>>> [38] <query> SELECT * FROM `harbormaster_buildplan`   WHERE (phid IN ('PHID-HMCP-viylmanphmudsc4wiuho'))   ORDER BY `id` DESC 
<<< [38] <query> 106 us
>>> [39] <query> SELECT * FROM `harbormaster_buildcommand` WHERE targetPHID IN ('PHID-HMBD-oxzm26jlbqlhxsz5geii') ORDER BY id ASC 
<<< [39] <query> 108 us
>>> [40] <query> SELECT * FROM `harbormaster_buildstep` WHERE (phid in ('PHID-HMCS-dxekix3bwt22yk3u6xq4')) ORDER BY `id` DESC 
<<< [40] <query> 102 us
>>> [41] <query> UPDATE `harbormaster_buildtarget` SET `name` = 'Lease Working Copy', `buildPHID` = 'PHID-HMBD-oxzm26jlbqlhxsz5geii', `buildStepPHID` = 'PHID-HMCS-dxekix3bwt22yk3u6xq4', `className` = 'HarbormasterLeaseWorkingCopyBuildStepImplementation', `details` = '{\"dependsOn\":[],\"name\":\"wc\",\"blueprintPHIDs\":\"[\\\"PHID-DRYB-nc3ceb6udcgce6alluh5\\\"]\"}', `variables` = '{\"buildable.commit\":\"f53ae675fd0ad9cb8c6c99d98b7fdb0f1d738a58\",\"repository.callsign\":\"PX\",\"repository.phid\":\"PHID-REPO-hbsyooqi3rjbxpq
<<< [41] <query> 5,974 us
[2015-10-29 13:19:50] EXCEPTION: (InvalidArgumentException) Argument 1 passed to DrydockLease::setAllowedBlueprintPHIDs() must be of the type array, null given, called in /opt/phabricator/src/applications/harbormaster/step/HarbormasterLeaseWorkingCopyBuildStepImplementation.php on line 51 and defined at [<phutil>/src/error/PhutilErrorHandler.php:200]
arcanist(head=master, ref.master=411a4f4a3917), phabricator(head=master, ref.master=d92f7a14739a), phutil(head=master, ref.master=59f5a8d2bb82)
  #0 <#2> PhutilErrorHandler::handleError(integer, string, string, integer, array) called at [<phabricator>/src/applications/drydock/storage/DrydockLease.php:399]
  #1 <#2> DrydockLease::setAllowedBlueprintPHIDs(NULL) called at [<phabricator>/src/applications/harbormaster/step/HarbormasterLeaseWorkingCopyBuildStepImplementation.php:51]
  #2 <#2> HarbormasterLeaseWorkingCopyBuildStepImplementation::execute(HarbormasterBuild, HarbormasterBuildTarget) called at [<phabricator>/src/applications/harbormaster/worker/HarbormasterTargetWorker.php:64]
  #3 phlog(InvalidArgumentException) called at [<phabricator>/src/applications/harbormaster/worker/HarbormasterTargetWorker.php:93]
  #4 PhabricatorWorker::scheduleTask(string, array) called at [<phabricator>/src/applications/harbormaster/engine/HarbormasterBuildEngine.php:84]
  #5 HarbormasterBuildEngine::continueBuild() called at [<phabricator>/src/applications/harbormaster/worker/HarbormasterBuildWorker.php:25]
  #6 HarbormasterBuildWorker::doWork() called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorWorker.php:161]
  #7 PhabricatorWorker::scheduleTask(string, array) called at [<phabricator>/src/applications/harbormaster/storage/HarbormasterBuildable.php:181]
  #8 HarbormasterBuildable::applyPlan(HarbormasterBuildPlan, array) called at [<phabricator>/src/applications/harbormaster/management/HarbormasterManagementBuildWorkflow.php:101]
  #9 HarbormasterManagementBuildWorkflow::execute(PhutilArgumentParser) called at [<phutil>/src/parser/argument/PhutilArgumentParser.php:406]
  #10 PhutilArgumentParser::parseWorkflowsFull(array) called at [<phutil>/src/parser/argument/PhutilArgumentParser.php:301]
  #11 PhutilArgumentParser::parseWorkflows(array) called at [<phabricator>/scripts/setup/manage_harbormaster.php:21]
>>> [42] <query> INSERT INTO `harbormaster_buildlog` (`buildTargetPHID`, `logSource`, `logType`, `duration`, `live`, `phid`, `dateCreated`, `dateModified`) VALUES ('PHID-HMBT-iycovncaofj4hqemmmca', 'core', 'exception', NULL, '0', 'PHID-HMCL-mmaxcqatraazghglg2jx', '1446121190', '1446121190')
<<< [42] <query> 14,057 us
>>> [43] <query> UPDATE `harbormaster_buildlog` SET `buildTargetPHID` = 'PHID-HMBT-iycovncaofj4hqemmmca', `logSource` = 'core', `logType` = 'exception', `duration` = NULL, `live` = '1', `id` = '5', `phid` = 'PHID-HMCL-mmaxcqatraazghglg2jx', `dateCreated` = '1446121190', `dateModified` = '1446121190' WHERE `id` = 5
<<< [43] <query> 10,542 us
>>> [44] <query> SELECT id, size, encoding FROM harbormaster_buildlogchunk WHERE logID = 5 ORDER BY id DESC LIMIT 1
<<< [44] <query> 255 us
>>> [45] <query> INSERT INTO harbormaster_buildlogchunk (logID, encoding, size, chunk) VALUES (5, 'text', 2129, 'exception \'InvalidArgumentException\' with message \'Argument 1 passed to DrydockLease::setAllowedBlueprintPHIDs() must be of the type array, null given, called in /opt/phabricator/src/applications/harbormaster/step/HarbormasterLeaseWorkingCopyBuildStepImplementation.php on line 51 and defined\' in /opt/libphutil/src/error/PhutilErrorHandler.php:200\nStack trace:\n#0 /opt/phabricator/src/applications/drydock/sto
<<< [45] <query> 10,116 us
>>> [46] <query> SELECT * FROM `harbormaster_buildlog` WHERE `id` = 5 
<<< [46] <query> 204 us
>>> [47] <query> UPDATE `harbormaster_buildlog` SET `buildTargetPHID` = 'PHID-HMBT-iycovncaofj4hqemmmca', `logSource` = 'core', `logType` = 'exception', `duration` = '0', `live` = '0', `id` = '5', `phid` = 'PHID-HMCL-mmaxcqatraazghglg2jx', `dateCreated` = '1446121190', `dateModified` = '1446121190' WHERE `id` = '5'
<<< [47] <query> 10,141 us
>>> [48] <query> UPDATE `harbormaster_buildtarget` SET `name` = 'Lease Working Copy', `buildPHID` = 'PHID-HMBD-oxzm26jlbqlhxsz5geii', `buildStepPHID` = 'PHID-HMCS-dxekix3bwt22yk3u6xq4', `className` = 'HarbormasterLeaseWorkingCopyBuildStepImplementation', `details` = '{\"dependsOn\":[],\"name\":\"wc\",\"blueprintPHIDs\":\"[\\\"PHID-DRYB-nc3ceb6udcgce6alluh5\\\"]\"}', `variables` = '{\"buildable.commit\":\"f53ae675fd0ad9cb8c6c99d98b7fdb0f1d738a58\",\"repository.callsign\":\"PX\",\"repository.phid\":\"PHID-REPO-hbsyooqi3rjbxpq
<<< [48] <query> 10,261 us
>>> [49] <lock> ph:phabric-6rILUDHKjHKu:harbormaster.build:9
>>> [50] <connect> phabricator_repository
<<< [50] <connect> 336 us
>>> [51] <query> SELECT GET_LOCK('ph:phabric-6rILUDHKjHKu:harbormaster.build:9', 15)
<<< [51] <query> 123 us
>>> [52] <query> SELECT * FROM `harbormaster_build` WHERE `id` = 9 
<<< [52] <query> 213 us
>>> [53] <query> SELECT * FROM `harbormaster_buildtarget`   WHERE (buildPHID in ('PHID-HMBD-oxzm26jlbqlhxsz5geii')) AND (buildGeneration in (1))   ORDER BY `id` DESC 
<<< [53] <query> 300 us
>>> [54] <query> SELECT * FROM `harbormaster_build`   WHERE (phid in ('PHID-HMBD-oxzm26jlbqlhxsz5geii'))   ORDER BY `id` DESC 
<<< [54] <query> 110 us
>>> [55] <query> SELECT * FROM `harbormaster_buildable`   WHERE (phid IN ('PHID-HMBB-7ogxe5gkv27zwbgfjo75'))   ORDER BY `id` DESC 
<<< [55] <query> 108 us
>>> [56] <query> SELECT `commit`.* FROM `repository_commit` commit  WHERE (commit.phid IN ('PHID-CMIT-fwgv3mj3bx5qi3ips5he'))   ORDER BY `commit`.`id` DESC 
<<< [56] <query> 134 us
>>> [57] <query> SELECT `r`.* FROM `repository` r  WHERE (r.id IN (2))   ORDER BY `r`.`id` DESC 
<<< [57] <query> 114 us
>>> [58] <query> SELECT * FROM `harbormaster_buildplan`   WHERE (phid IN ('PHID-HMCP-viylmanphmudsc4wiuho'))   ORDER BY `id` DESC 
<<< [58] <query> 114 us
>>> [59] <query> SELECT * FROM `harbormaster_buildcommand` WHERE targetPHID IN ('PHID-HMBD-oxzm26jlbqlhxsz5geii') ORDER BY id ASC 
<<< [59] <query> 103 us
>>> [60] <query> SELECT * FROM `harbormaster_buildstep` WHERE (buildPlanPHID in ('PHID-HMCP-viylmanphmudsc4wiuho')) ORDER BY `id` DESC 
<<< [60] <query> 111 us
>>> [61] <query> SELECT * FROM `harbormaster_buildplan`   WHERE (phid IN ('PHID-HMCP-viylmanphmudsc4wiuho'))   ORDER BY `id` DESC 
<<< [61] <query> 108 us
>>> [62] <query> UPDATE `harbormaster_build` SET `buildablePHID` = 'PHID-HMBB-7ogxe5gkv27zwbgfjo75', `buildPlanPHID` = 'PHID-HMCP-viylmanphmudsc4wiuho', `buildStatus` = 'failed', `buildGeneration` = '1', `buildParameters` = '[]', `planAutoKey` = NULL, `id` = '9', `phid` = 'PHID-HMBD-oxzm26jlbqlhxsz5geii', `dateCreated` = '1446121190', `dateModified` = '1446121190' WHERE `id` = '9'
<<< [62] <query> 6,484 us
>>> [63] <query> SELECT RELEASE_LOCK('ph:phabric-6rILUDHKjHKu:harbormaster.build:9')
<<< [63] <query> 117 us
<<< [49] <lock> 14,986 us
>>> [64] <lock> ph:phabric-6rILUDHKjHKu:harbormaster.buildable:9
>>> [65] <connect> phabricator_repository
<<< [65] <connect> 238 us
>>> [66] <query> SELECT GET_LOCK('ph:phabric-6rILUDHKjHKu:harbormaster.buildable:9', 15)
<<< [66] <query> 108 us
>>> [67] <query> SELECT * FROM `harbormaster_buildable`   WHERE (id IN (9))   ORDER BY `id` DESC 
<<< [67] <query> 190 us
>>> [68] <query> SELECT `commit`.* FROM `repository_commit` commit  WHERE (commit.phid IN ('PHID-CMIT-fwgv3mj3bx5qi3ips5he'))   ORDER BY `commit`.`id` DESC 
<<< [68] <query> 113 us
>>> [69] <query> SELECT `r`.* FROM `repository` r  WHERE (r.id IN (2))   ORDER BY `r`.`id` DESC 
<<< [69] <query> 111 us
>>> [70] <query> SELECT * FROM `harbormaster_build`   WHERE (buildablePHID IN ('PHID-HMBB-7ogxe5gkv27zwbgfjo75'))   ORDER BY `id` DESC 
<<< [70] <query> 239 us
>>> [71] <query> SELECT * FROM `harbormaster_buildable`   WHERE (phid IN ('PHID-HMBB-7ogxe5gkv27zwbgfjo75'))   ORDER BY `id` DESC 
<<< [71] <query> 108 us
>>> [72] <query> SELECT * FROM `harbormaster_buildplan`   WHERE (phid IN ('PHID-HMCP-viylmanphmudsc4wiuho'))   ORDER BY `id` DESC 
<<< [72] <query> 105 us
>>> [73] <query> SELECT * FROM `harbormaster_buildcommand` WHERE targetPHID IN ('PHID-HMBD-oxzm26jlbqlhxsz5geii') ORDER BY id ASC 
<<< [73] <query> 102 us
>>> [74] <query> UPDATE `harbormaster_buildable` SET `buildablePHID` = 'PHID-CMIT-fwgv3mj3bx5qi3ips5he', `containerPHID` = 'PHID-REPO-hbsyooqi3rjbxpq675m3', `buildableStatus` = 'failed', `isManualBuildable` = '1', `id` = '9', `phid` = 'PHID-HMBB-7ogxe5gkv27zwbgfjo75', `dateCreated` = '1446121190', `dateModified` = '1446121190' WHERE `id` = '9'
<<< [74] <query> 12,329 us
>>> [75] <query> SELECT RELEASE_LOCK('ph:phabric-6rILUDHKjHKu:harbormaster.buildable:9')
<<< [75] <query> 113 us
<<< [64] <lock> 18,450 us
>>> [76] <query> SELECT * FROM `harbormaster_buildtarget`   WHERE (buildPHID in ('PHID-HMBD-oxzm26jlbqlhxsz5geii')) AND (buildGeneration in (1))   ORDER BY `id` DESC 
<<< [76] <query> 141 us
>>> [77] <query> SELECT * FROM `harbormaster_build`   WHERE (phid in ('PHID-HMBD-oxzm26jlbqlhxsz5geii'))   ORDER BY `id` DESC 
<<< [77] <query> 195 us
>>> [78] <query> SELECT * FROM `harbormaster_buildable`   WHERE (phid IN ('PHID-HMBB-7ogxe5gkv27zwbgfjo75'))   ORDER BY `id` DESC 
<<< [78] <query> 188 us
>>> [79] <query> SELECT `commit`.* FROM `repository_commit` commit  WHERE (commit.phid IN ('PHID-CMIT-fwgv3mj3bx5qi3ips5he'))   ORDER BY `commit`.`id` DESC 
<<< [79] <query> 109 us
>>> [80] <query> SELECT `r`.* FROM `repository` r  WHERE (r.id IN (2))   ORDER BY `r`.`id` DESC 
<<< [80] <query> 113 us
>>> [81] <query> SELECT * FROM `harbormaster_buildplan`   WHERE (phid IN ('PHID-HMCP-viylmanphmudsc4wiuho'))   ORDER BY `id` DESC 
<<< [81] <query> 106 us
>>> [82] <query> SELECT * FROM `harbormaster_buildcommand` WHERE targetPHID IN ('PHID-HMBD-oxzm26jlbqlhxsz5geii') ORDER BY id ASC 
<<< [82] <query> 102 us
>>> [83] <query> SELECT * FROM `harbormaster_buildartifact`   WHERE (buildTargetPHID IN ('PHID-HMBT-iycovncaofj4hqemmmca'))   ORDER BY `id` DESC 
<<< [83] <query> 228 us
>>> [84] <connect> phabricator_worker
<<< [84] <connect> 389 us
>>> [85] <query> START TRANSACTION
<<< [85] <query> 95 us
>>> [86] <query> INSERT INTO `worker_taskdata` (`data`) VALUES ('{\"targetID\":14}')
<<< [86] <query> 289 us
>>> [87] <query> INSERT INTO `lisk_counter` (counterName, counterValue) VALUES
          ('worker_activetask', LAST_INSERT_ID(1))
        ON DUPLICATE KEY UPDATE
          counterValue = LAST_INSERT_ID(counterValue + 1)
<<< [87] <query> 288 us
>>> [88] <query> INSERT INTO `worker_activetask` (`failureTime`, `taskClass`, `leaseOwner`, `leaseExpires`, `failureCount`, `dataID`, `priority`, `objectPHID`, `id`) VALUES (NULL, 'HarbormasterTargetWorker', NULL, NULL, '0', '1930', '2000', NULL, '1932')
<<< [88] <query> 188 us
>>> [89] <query> SAVEPOINT Aphront_Savepoint_1
<<< [89] <query> 108 us
>>> [90] <query> DELETE FROM `worker_activetask` WHERE id = 1932
<<< [90] <query> 215 us
>>> [91] <query> INSERT INTO `worker_archivetask` (`duration`, `result`, `taskClass`, `leaseOwner`, `leaseExpires`, `failureCount`, `dataID`, `priority`, `objectPHID`, `id`, `dateCreated`, `dateModified`) VALUES ('0', '0', 'HarbormasterTargetWorker', NULL, NULL, '0', '1930', '2000', NULL, '1932', '1446121191', '1446121191')
<<< [91] <query> 185 us
>>> [92] <query> COMMIT
<<< [92] <query> 42,507 us
>>> [93] <lock> ph:phabric-6rILUDHKjHKu:harbormaster.buildable:9
>>> [94] <connect> phabricator_repository
<<< [94] <connect> 494 us
>>> [95] <query> SELECT GET_LOCK('ph:phabric-6rILUDHKjHKu:harbormaster.buildable:9', 15)
<<< [95] <query> 173 us
>>> [96] <query> SELECT * FROM `harbormaster_buildable`   WHERE (id IN (9))   ORDER BY `id` DESC 
<<< [96] <query> 313 us
>>> [97] <query> SELECT `commit`.* FROM `repository_commit` commit  WHERE (commit.phid IN ('PHID-CMIT-fwgv3mj3bx5qi3ips5he'))   ORDER BY `commit`.`id` DESC 
<<< [97] <query> 145 us
>>> [98] <query> SELECT `r`.* FROM `repository` r  WHERE (r.id IN (2))   ORDER BY `r`.`id` DESC 
<<< [98] <query> 118 us
>>> [99] <query> SELECT * FROM `harbormaster_build`   WHERE (buildablePHID IN ('PHID-HMBB-7ogxe5gkv27zwbgfjo75'))   ORDER BY `id` DESC 
<<< [99] <query> 115 us
>>> [100] <query> SELECT * FROM `harbormaster_buildable`   WHERE (phid IN ('PHID-HMBB-7ogxe5gkv27zwbgfjo75'))   ORDER BY `id` DESC 
<<< [100] <query> 108 us
>>> [101] <query> SELECT * FROM `harbormaster_buildplan`   WHERE (phid IN ('PHID-HMCP-viylmanphmudsc4wiuho'))   ORDER BY `id` DESC 
<<< [101] <query> 120 us
>>> [102] <query> SELECT * FROM `harbormaster_buildcommand` WHERE targetPHID IN ('PHID-HMBD-oxzm26jlbqlhxsz5geii') ORDER BY id ASC 
<<< [102] <query> 101 us
>>> [103] <query> SELECT RELEASE_LOCK('ph:phabric-6rILUDHKjHKu:harbormaster.buildable:9')
<<< [103] <query> 123 us
<<< [93] <lock> 6,525 us
>>> [104] <query> START TRANSACTION
<<< [104] <query> 121 us
>>> [105] <query> INSERT INTO `worker_taskdata` (`data`) VALUES ('{\"buildID\":9}')
<<< [105] <query> 186 us
>>> [106] <query> INSERT INTO `lisk_counter` (counterName, counterValue) VALUES
          ('worker_activetask', LAST_INSERT_ID(1))
        ON DUPLICATE KEY UPDATE
          counterValue = LAST_INSERT_ID(counterValue + 1)
<<< [106] <query> 196 us
>>> [107] <query> INSERT INTO `worker_activetask` (`failureTime`, `taskClass`, `leaseOwner`, `leaseExpires`, `failureCount`, `dataID`, `priority`, `objectPHID`, `id`) VALUES (NULL, 'HarbormasterBuildWorker', NULL, NULL, '0', '1931', '2000', NULL, '1933')
<<< [107] <query> 173 us
>>> [108] <query> SAVEPOINT Aphront_Savepoint_1
<<< [108] <query> 99 us
>>> [109] <query> DELETE FROM `worker_activetask` WHERE id = 1933
<<< [109] <query> 163 us
>>> [110] <query> INSERT INTO `worker_archivetask` (`duration`, `result`, `taskClass`, `leaseOwner`, `leaseExpires`, `failureCount`, `dataID`, `priority`, `objectPHID`, `id`, `dateCreated`, `dateModified`) VALUES ('0', '0', 'HarbormasterBuildWorker', NULL, NULL, '0', '1931', '2000', NULL, '1933', '1446121191', '1446121191')
<<< [110] <query> 172 us
>>> [111] <query> COMMIT
<<< [111] <query> 17,174 us
Done.

After discussing privatly with @yelirekim and seeking for a while I decided to publish this.
Basically I see two problems:

  • the fact that null isn't a normal argument and should be and empty array : it thus for me seems very similar with T9631: Don't crash when users fail to specify additional working copies and should be easy to fix
  • the fact that it is not working while all seem's to be set correctly (that is the reason I posted all this (let me know if you need more). It seem's to be a reall problem too but it is probably just me being dumb !

Event Timeline

Basically I see two problems:

  • the fact that null isn't a normal argument and should be and empty array : it thus for me seems very similar with T9631: Don't crash when users fail to specify additional working copies and should be easy to fix
  • the fact that it is not working while all seem's to be set correctly (that is the reason I posted all this (let me know if you need more). It seem's to be a reall problem too but it is probably just me being dumb !

I work around the second problem and makes all works by adding a repository to be clone. Maybe I must have done it but then the term Also Clone in the build step and the fact that this build step is not mandatory is misleading. Basically I understood also clone as clone the buildable repositorylike rP and you may also want to clone rARC or rPHU.

If you agree can I submit a fix :

  • same as rP43569d4e278c365a1feb9497acfa4ce336a8262c for first part
  • if needed (maybe first part solve all later on by adding buildable.repository if no repository are selected) a modification of UI that say that clone instead of also clone and make it required.

Doing

diff --git a/src/applications/harbormaster/step/HarbormasterLeaseWorkingCopyBuildStepImplementation.php b/src/applications/harbormaster/step/HarbormasterLeaseWorkingCopyBuildStepImplementation.php
index 3c4d18f..9e20b6c 100644
--- a/src/applications/harbormaster/step/HarbormasterLeaseWorkingCopyBuildStepImplementation.php
+++ b/src/applications/harbormaster/step/HarbormasterLeaseWorkingCopyBuildStepImplementation.php
@@ -43,7 +43,9 @@ final class HarbormasterLeaseWorkingCopyBuildStepImplementation
 
       $allowed_phids = $build_target->getFieldValue('repositoryPHIDs');
       $authorizing_phid = $build_target->getBuildStep()->getPHID();
-
+      if (!is_array($allowed_phids)){
+          $allowed_phids = array(); 
+      }
       $lease = DrydockLease::initializeNewLease()
         ->setResourceType($working_copy_type)
         ->setOwnerPHID($build_target->getPHID())

get me

[2015-10-29 15:33:35] EXCEPTION: (Exception) Trying to queue a lease for activation without any allowed Blueprints. Use "setAllowedBlueprintPHIDs()" to specify allowed blueprints. The authorizing object must be approved to use the allowed blueprints. at [<phabricator>/src/applications/drydock/storage/DrydockLease.php:156]
arcanist(head=master, ref.master=411a4f4a3917), phabricator(head=master, ref.master=d92f7a14739a), phutil(head=master, ref.master=59f5a8d2bb82)
  #0 <#2> DrydockLease::queueForActivation() called at [<phabricator>/src/applications/harbormaster/step/HarbormasterLeaseWorkingCopyBuildStepImplementation.php:76]
  #1 <#2> HarbormasterLeaseWorkingCopyBuildStepImplementation::execute(HarbormasterBuild, HarbormasterBuildTarget) called at [<phabricator>/src/applications/harbormaster/worker/HarbormasterTargetWorker.php:64]
  #2 phlog(Exception) called at [<phabricator>/src/applications/harbormaster/worker/HarbormasterTargetWorker.php:93]
  #3 PhabricatorWorker::scheduleTask(string, array) called at [<phabricator>/src/applications/harbormaster/engine/HarbormasterBuildEngine.php:84]
  #4 HarbormasterBuildEngine::continueBuild() called at [<phabricator>/src/applications/harbormaster/worker/HarbormasterBuildWorker.php:25]
  #5 HarbormasterBuildWorker::doWork() called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorWorker.php:161]
  #6 PhabricatorWorker::scheduleTask(string, array) called at [<phabricator>/src/applications/harbormaster/storage/HarbormasterBuildable.php:181]
  #7 HarbormasterBuildable::applyPlan(HarbormasterBuildPlan, array) called at [<phabricator>/src/applications/harbormaster/management/HarbormasterManagementBuildWorkflow.php:101]
  #8 HarbormasterManagementBuildWorkflow::execute(PhutilArgumentParser) called at [<phutil>/src/parser/argument/PhutilArgumentParser.php:406]
  #9 PhutilArgumentParser::parseWorkflowsFull(array) called at [<phutil>/src/parser/argument/PhutilArgumentParser.php:301]
  #10 PhutilArgumentParser::parseWorkflows(array) called at [<phabricator>/scripts/setup/manage_harbormaster.php:21]

wich definitively is better for 1) (still a bit misleading cause one ask me to authorize what was already authorized)

But I kinda think that with actual UI it should by default clone the repository which goes with the buildable (or at least allow ${repository}).
That would makes build plan not repository specific (I imagine a step publish to npm that could works for all the 100 repository one may have and would did the same think in each case, ie read the package.json file and make npm publish). I think having one npm publish build plan is way better than having 100 build plan (one for each repo).

Moreover I don't find a case where we want to have a working copy of something without cloning it first.

But it is basically what seem's to be done here : https://secure.phabricator.com/diffusion/P/browse/master/src/applications/harbormaster/step/HarbormasterLeaseWorkingCopyBuildStepImplementation.php;724f6ddda58d27edcee3a755f238179ebf0a2d7b$129-143
I still don't get why it isn't executed earlier on.

tycho.tatitscheff renamed this task from Dont crash when testing harbormaster steps on repository commit to Harbormaster working copy build step shouldn't crash when user don't set any also clone repository.Oct 30 2015, 9:20 AM
epriestley triaged this task as Normal priority.Oct 30 2015, 2:53 PM