Page MenuHomePhabricator
Differential D10395

[drydock/aws] Implement Amazon EC2 blueprint for Drydock
AbandonedPublic
Actions

Authored by hach-que on Aug 30 2014, 8:41 AM.
Tags
Referenced Files
Unknown Object (File)
Tue, Jan 7, 2:17 PM
Unknown Object (File)
Tue, Jan 7, 2:03 PM
Unknown Object (File)
Mon, Jan 6, 9:56 PM
Unknown Object (File)
Wed, Jan 1, 4:48 PM
Unknown Object (File)
Tue, Dec 24, 4:34 PM
Unknown Object (File)
Fri, Dec 20, 6:52 PM
Unknown Object (File)
Tue, Dec 17, 4:24 AM
Unknown Object (File)
Sun, Dec 15, 12:06 AM
View All Files
Subscribers
claassen
epriestley
joshuaspence
Korvin
revi

Details

Reviewers
epriestley
Group Reviewers
Blessed Reviewers
Maniphest Tasks
T5544: Implement EC2 blueprint for Drydock
T2015: Implement Drydock
Summary

Ref T2015. Depends on D10495. Depends on D10394.

This implements an Amazon EC2 blueprint that allocates and leases hosts from EC2. It starts EC2 instances according to the parameters specified on the blueprint, using the public key of a Passphrase key as the key pair for AWS.

For Linux, this connects over SSH using the private key pair, which is automatically configured by AWS.

For Windows, this connects over WinRM, using the specified WinRM password credential. The user must configure a Windows image in AWS with remoting enabled and with an account that matches the username / password in the credential.

It handles allocation of Elastic IPs, which provides a public IP address for connecting to the allocated machine when Phabricator is not hosted in the same VPC (or on Amazon).

This is built to allocate into a VPC; it doesn't support allocating a classic EC2 instance (and mainly because AWS is trying to deprecate that model anyway).

Test Plan

Tested with both Windows and Linux AMIs in the us-west-2 region.

Diff Detail

Repository
rP Phabricator
Branch
drydock-reconstructed-stack
Lint
Lint Passed
SeverityLocationCodeMessage
Advicesrc/applications/drydock/blueprint/DrydockAmazonEC2HostBlueprintImplementation.php:191XHP16TODO Comment
Unit
Tests Passed
Build Status
Buildable 6848
Build 6870: [Placeholder Plan] Wait for 30 Seconds

Event Timeline

hach-que updated this revision to Diff 25015.Aug 30 2014, 8:41 AM
hach-que retitled this revision from to Implement Amazon EC2 blueprint for Drydock.
hach-que updated this object.
hach-que edited the test plan for this revision. (Show Details)
hach-que added a reviewer: epriestley.
hach-que added a task: T2015: Implement Drydock.
hach-que added a parent revision: D10394: [drydock/core] Abstract min-max blueprint and implement expiry logic.
Herald added a reviewer: Blessed Reviewers. · View Herald TranscriptAug 30 2014, 8:41 AM
Herald added subscribers: Korvin, epriestley. · View Herald Transcript
Harbormaster completed remote builds in B2399: Diff 25015.Aug 30 2014, 8:42 AM
hach-que updated this revision to Diff 25048.Sep 4 2014, 5:23 AM
hach-que edited edge metadata.

Provide additional status information during allocation, and prevent crashes when deleting storage

Harbormaster completed remote builds in B2423: Diff 25048.Sep 4 2014, 5:24 AM
hach-que mentioned this in D10460: Allow custom attributes against blueprints and when leasing hosts.Sep 10 2014, 7:03 AM
hach-que updated this revision to Diff 25176.Sep 11 2014, 12:12 AM

Configure the service to run as delayed start instead of waiting for instance storage

Harbormaster completed remote builds in B2500: Diff 25176.Sep 11 2014, 12:12 AM
hach-que added a task: T5544: Implement EC2 blueprint for Drydock.Nov 3 2014, 5:00 AM
hach-que updated this revision to Diff 27528.Jan 21 2015, 12:01 AM

Rebase

hach-que mentioned this in D10396: Implement support for launching spot instances in Amazon EC2.Jan 21 2015, 12:06 AM
joshuaspence added a subscriber: joshuaspence.Feb 27 2015, 5:59 AM
hach-que updated this revision to Diff 32281.Jun 18 2015, 3:46 PM

Update to support WinRM, drop old SSH on Windows stuff

hach-que added a parent revision: D10495: [drydock/core] Use WinRM instead of SSH for Windows hosts.Jun 18 2015, 3:47 PM
hach-que updated this object.
hach-que added a project: Drydock.
hach-que added a comment.Jun 18 2015, 4:26 PM

Tested this on HEAD with the latest changes for a Linux AMI and it works.

Testing for a Windows AMI now.

hach-que updated this revision to Diff 32284.Jun 18 2015, 4:27 PM

Remove call to setExecTimeout since it's introduced in a later diff

hach-que added a comment.Jun 18 2015, 5:27 PM

Tested this on HEAD with the latest changes for a Windows AMI and it works.

hach-que retitled this revision from Implement Amazon EC2 blueprint for Drydock to [drydock/aws] Implement Amazon EC2 blueprint for Drydock.Jul 1 2015, 4:46 AM
epriestley requested changes to this revision.Aug 8 2015, 5:48 PM
epriestley edited edge metadata.

I want an Almanac-based version of DrydockPreallocatedHostBlueprintImplementation in production and stable before we pursue this.

This revision now requires changes to proceed.Aug 8 2015, 5:48 PM
epriestley mentioned this in D10458: Allow private IP address to always be used, even if elastic IP is allocated.Aug 8 2015, 5:49 PM
hach-que added a comment.Aug 8 2015, 11:28 PM
In D10395#150396, @epriestley wrote:

I want an Almanac-based version of DrydockPreallocatedHostBlueprintImplementation in production and stable before we pursue this.

I'm not clear on what this means. Is there a task filed somewhere that describes the work that needs to be done?

hach-que mentioned this in D10304: [drydock/core] Implement resource constraints and correct locking mechanisms with yielding for Drydock.Aug 24 2015, 5:07 AM
revi added a subscriber: revi.Oct 7 2015, 2:05 PM
hach-que abandoned this revision.Dec 3 2015, 8:45 AM

Needs to be completely rewritten for the new API; D14583 has a good structure for implementing host allocation on non-static services.

claassen added a subscriber: claassen.Apr 13 2016, 2:36 PM

Revision Contents
Changeset List

Diff 32284

View Options

src/__phutil_library_map__.php

Loading...
View Options

src/applications/drydock/blueprint/DrydockAmazonEC2HostBlueprintImplementation.php

Loading...
View Options

src/applications/drydock/blueprint/DrydockBlueprintImplementation.php

Loading...
View Options

src/applications/drydock/worker/DrydockAllocatorWorker.php

Loading...
Phabricator · Built by Phacility