Page MenuHomePhabricator

Unprototype Harbormaster (v1)
Closed, ResolvedPublic

Tokens
"Love" token, awarded by marklu."Love" token, awarded by ciarand."Love" token, awarded by tycho.tatitscheff."Love" token, awarded by johnny-bit.
Assigned To
Authored By
epriestley, May 6 2015

Description

Umbrella task for moving Harbormaster out of prototype. This iteration only includes support for external build systems (like Jenkins or a similar custom system), not hosted builds.

The major changes are:

  • Change Handoff (T8090): Make it easy to hand changes off to Jenkins or a similar system without requiring any fragile or messy intermediate steps (no arc patch, no side-effect pushes to real branches).
  • Unit/Lint Ownership (T8095): Move unit and lint results entirely into Harbormaster and let it own them completely.
  • Reporting (T8097): Provide better tools for external build systems to report detailed build results (not just pass/fail).
  • UI/UX (T8096): Generally modernize and improve the Harbormaster UI and UX, including integration with applications like Differential/Diffusion.

This would roughly leave us with a reasonable data model, reasonable UI, good application integration, and the ability to drive Jenkins-like external test environments in a plausible way. Future iterations would build on Drydock and the APIs to offer hosted builds, more powerful builds, and possibly integration with less flexible external build systems like Travis/Circle.

Related Objects

StatusAssignedTask
DuplicateNone
Resolvedepriestley
Resolvedepriestley
Resolvedepriestley
Resolvedepriestley
ResolvedNone
Resolvedhach-que
Wontfixepriestley
Wontfixepriestley
Resolvedepriestley
ResolvedNone
Resolvedepriestley
Resolvedepriestley
Wontfixhach-que
Resolvedhach-que
DuplicateNone
Resolvedepriestley
DuplicateNone
Resolvedhach-que
ResolvedNone
ResolvedNone
Duplicatehach-que
OpenNone
InvalidNone
Resolvedhach-que
Resolvedepriestley
OpenNone
Resolvedhach-que
OpenNone
ResolvedNone
DuplicateNone
Resolvedhach-que
Wontfixepriestley
Wontfixepriestley
Wontfixepriestley
ResolvedNone
Resolvedepriestley
OpenNone
OpenNone
OpenNone
Resolvedepriestley
Resolvedbtrahan
Resolvedepriestley
Duplicateepriestley
ResolvedNone
Resolvedepriestley
ResolvedNone
Resolvedepriestley
Resolvedepriestley

Event Timeline

epriestley raised the priority of this task from to Normal.
epriestley updated the task description. (Show Details)
epriestley added a project: Harbormaster.
epriestley added a subscriber: epriestley.
epriestley moved this task from Backlog to v1 on the Harbormaster board.May 6 2015, 2:48 PM
epriestley updated the task description. (Show Details)May 6 2015, 3:17 PM
epriestley updated the task description. (Show Details)May 6 2015, 3:28 PM
epriestley updated the task description. (Show Details)May 6 2015, 3:37 PM
chad added a subscriber: chad.May 6 2015, 4:03 PM
epriestley updated the task description. (Show Details)May 6 2015, 4:07 PM
epriestley updated the task description. (Show Details)May 6 2015, 4:22 PM
epriestley updated the task description. (Show Details)May 6 2015, 6:51 PM
joshma added a subscriber: joshma.Jun 1 2015, 6:23 AM
epriestley added a comment.EditedJun 4 2015, 4:24 PM

There's some level of general interest in this, so I figured I'd centralize the rough level-of-effort estimates I've been throwing around.

There are some large pieces that aren't technical blockers which we can leave out of scope for now, depending on needs:

  • Virtual Refs (T8093). This makes handoff (sending changes to a build/CI system) nicer (basically, more automatic/invisible) for users.
    • If you don't host repositories on Phabricator, this won't do anything for you and you don't need it.
    • If you do use hosted repositories, but you're fine with side-band handoff for now (see T8238), you don't need this yet. If you want nicer handoff later later, we could build it later.
    • If you do use hosted repositories, but you want to use arc patch handoff or some other kind of rickety-cart-based handoff, you don't need this.
    • If you definitely want nice handoff now, we need it now.
    • This is basically just polish and it's fairly involved so I suspect no one cares for now.
  • Full Result Reporting (Part of T8097). This lets engineers review all test results in Phabricator, without going to an external tool, instead of just seeing pass/fail in Phabricator and clicking through.
    • If you're happy with having users click through to an external tool like Jenkins for now to get test details, and don't plan to CI lint yet/ever, you don't need this for now.
    • Instead, we'd just build enough of this to let you link to an external tool's result page.

All told, I think this roughly breaks out like this:

WhatTaskEstimateRequiredDescription
Virtual RefsT809312 HoursOptionalMake handoff nicer for hosted repositories.
Diff PropertiesT51383 HoursRequiredSeparate "diff" and "revision" properties.
Unit/Lint OwnershipT80954 HoursRequiredMove storage of lint/unit to Harbormaster.
Conduit InfrastructureT58732 Hours?Blocker?Some risk we need to clear a bit of T5873 before moving forward.
Reporting APIT80972-4 HoursRequiredThis is more like 2 hours and less likely to trigger T5873 if we don't do full reporting for now.
UI/UX WorkT8096??OptionalNot a blocker for moving forward.

I'm pretty sure no one cares about T8093 right now since the tradeoffs for the side-band stuff in T8238 are pretty reasonable/moderate, so this is something like 10-15 hours of work, depending on how much reporting you care about having right now vs later.

Development Log

DateTaskHoursWorkNotes
Jun 14T8095-T5138Initial work to split diff properties.

Total Hours: 0

Available Now

  • (After Merge/Push) Old lint/unit/build results are reviewable in Differential.

Discussion

Feeble start on this, but getting the UI split went cleanly.

Next Up

The next major thing I want to do is have arc push lint/unit results into Harbormaster, and fall back to the current mechanism only if it fails.

Rough steps here are:

Build Plans for arc: We need a build to attach results. We need a build plan in order to create a build. We don't currently have one we can use, since the user may never have touched Harbormaster when they first run arc.

I think I'm going to add a specialKey to build plans and create "real" arc lint and arc unit plans implicitly. They will be immutable and implicit but have real PHIDs and IDs. I think this is the shortest path to everything just working properly.

Third-party tests in a similar situation which run proactively for some reason (I expect this to be very rare) can just hard-code plan PHIDs, but I don't want users to have to do a bunch of error-prone setup steps just to keep arc unit working. The alternatives (like virtual PHIDs and virtual objects) seem much messier and I think the UI/UX complexity of this approach will be very low ("it just works like you expect it to") and the internal complexity will be fairly low.

APIs: I'm going to build out modern-enough versions of the required core APIs, then special APIs for publishing results. This may derail slightly or significantly into T7715 but I want whatever I build now to survive the coming API apocalypse mostly unscathed. I expect this to be largely straightforward, the new SearchFields abstraction which supports this feels pretty clean to me and I think the boundaries are in a good place.

Result Storage: I need to plan this more specifically. I'm not sure if logs, test results, and unit results are all flavors of the same object, or if logs are different but test and unit results are the same thing, or if all three are different. I lean toward all three being different (and "artifacts" being a fourth, also different category, and build status being a fifth, different sort of result), I just want to think about this some more because I think the "0, 1, or N" rule is often a pretty good one, and "5" isn't any of those numbers. I think there are almost certainly strong use cases for each result/output type being a separate, distinct data type, though.

(There's also a sort of nebulous 6th result type, which is "data/properties", particularly links to external result set(s). I doubt these are a 6th distinct type -- maybe they're just special artifacts?)

Once I convince myself that "5" or "6" or something in this realm is a good value for "1", I'll build the actual storage and connect it to the API, then put UI on it.

The visible effect will be that arc reports results into Harbormaster instead of Differential (using generic APIs which are equally available to other build software), and both Harbormaster and Differential display this information in a reasonable way, and we're generally well-positioned to start making Harbormaster do more interesting things than sleep for 30 seconds.

This stuff is still behind Spaces in queue, so progress will be slow for a bit.

epriestley moved this task from Backlog to The Queue on the Prioritized board.Jun 16 2015, 9:39 PM
tycho.tatitscheff moved this task from Backlog to The Queue on the Prioritized board.

Development Log

DateTaskHoursWorkNotes
Jun 17T80953T8095Prepared for Harbormaster lint/unit ownership.

Total Hours: 3

Available Now

  • Nothing user-facing yet.

Discussion

Slow week in general, but I'm pretty happy with the actual code here. D13345 adds "autoplans", which feel pretty reasonable as a way to solve the "where does arc actually put results?" problem. These roughly follow the outline described above.

Next Up

Connecting arc to autoplans from the CLI, and the UI to Harbormaster. I expect this to be straightforward and resolve T8095.

Development Log

DateTaskHoursWorkNotes
Jun 21T80952D13377 .. D13383Almost gave Harbormaster ownership of lint/unit.

Total Hours: 5

Available Now

  • Nothing user-facing.

Discussion

This stuff basically all works now but I want to clean it up a bit more before landing it.

Next Up

Finishing this up so we can cut over to Harbormaster ownership.

revi added a subscriber: revi.Jun 28 2015, 1:51 PM
ftdysa added a subscriber: ftdysa.Jul 1 2015, 1:29 AM
elesh added a subscriber: elesh.Jul 20 2015, 7:09 PM
fcoelho added a subscriber: fcoelho.Aug 7 2015, 6:17 AM
mrjoops added a subscriber: mrjoops.Aug 9 2015, 5:20 AM
epriestley closed this task as Resolved.Aug 10 2015, 10:40 PM
epriestley claimed this task.

This stuff is now in relatively good shape in HEAD. Some followups:

  • T9123 is the direct followup for the next iteration on Harbormaster. Currently, the next iteration focuses on getting arc unit working via Harbormaster so we can build the upstream and build without Jenkins or a similar external build system.
  • We have a pathway forward on T8090 (change handoff) but haven't walked very far down that pathway yet. This probably won't happen for a while, since "staging area" handoff isn't too terrible.
  • Design of lint/unit summary results in Differential (T7739) is okay for now but can use some more work in the future.
urzds added a subscriber: urzds.Jul 12 2017, 11:12 AM