This document explains how to add new hardware to the [[ phacility_cluster | Phacility Cluster ]].
Overview
======
To add hardware to the cluster:
- Log in to the AWS console using your IAM role.
- Launch a new instance into the VPC.
- Add a `phacility.net` DNS entry for the device in Route 53.
- Launch and attach any required EBS storage.
- Using the [[phacility_cluster/cli | CLI Tools]], `deploy` the device.
- If the device receives traffic from load balancers, add the host to the load balancer pool.
- If the device hosts a service, create a new Almanac device entry and bind it to new or existing services.
This process is not as automated as it could be, but we an automate it as the need arises. The more complicated and error-prone parts //are// automated.
Launching Instances
======
Before launching instances, consider these cases:
| Device | Considerations |
|---|---|
| `admin` | Do not launch multiple `admin` devices without making software changes to support this first. |
| `bastion` | Requires master key. Use `--master`. Copy master key to `/core/conf/keys/master.key` after deployment. |
| `notify` | These need software changes to scale up. |
| `secure` | Requires software changes to scale up. |
| `vault` | These are tricky to launch properly. |
Launch instances using these settings:
| Device | AMI | Type | Security Group | Key | VPC |
|---|---|---|---|---|---|
| `admin` | ami-5c120b19 (Ubuntu 14) | m3.large | instance.admin | phacility-master | block001 |
| `aux` | ami-5c120b19 (Ubuntu 14) | m3.large | instance.aux | phacility-master | block001 |
| `bastion` | ami-5c120b19 (Ubuntu 14) | m3.large | instance.bastion | phacility-master | block001 |
| `corp` | ami-5c120b19 (Ubuntu 14) | m3.large | instance.corp | phacility-master | block002 |
| `db` | ami-5c120b19 (Ubuntu 14) | m3.large | instance.db | phacility-master | block001 |
| `notify` | ami-5c120b19 (Ubuntu 14) | m3.large | instance.notify | phacility-master | block001 |
| `repo` | ami-5c120b19 (Ubuntu 14) | m3.large | instance.repo | phacility-master | block001 |
| `secure` | ami-5c120b19 (Ubuntu 14) | m3.large | instance.secure | phacility-master | block001 |
| `vault` | ami-5c120b19 (Ubuntu 14) | m3.large | instance.vault | phacility-master | block001 |
| `web` | ami-5c120b19 (Ubuntu 14) | m3.large | instance.web | phacility-master | block001 |
Adding DNS
======
For all devices, add a `phacility.net` DNS entry in Route 53 with the internal (VPC) device IP address. This should be an "A" record, similar to the other device records. The UI defaults are generally correct.
Set the name of the device in the EC2 console to be the same as the DNS entry you selected.
For example, if you launch a new web instance, you might call it `web007.phacility.net` (you should use the next number which doesn't have a DNS entry yet, of course).
EBS Storage
======
These types of devices have persistent EBS storage:
| Device | Storage Types |
|---|---|
| `admin` | `adata`, `abak` |
| `db` | `ddata`, `dbak` |
| `repo` | `rdata`, `rbak`, `rlog` |
| `secure` | `sdata`, `srepo`, `sbak` |
If you're launching a device which uses one or more kinds of persistent EBS storage, you need to create and attach EBS volumes to the device. Use these settings for each storage type:
| Storage | Size | Attach To |
|---|---|---|
| `abak` | 64GB | `/dev/sdf` |
| `adata` | 64GB | `/dev/sdh` |
| `dbak` | 64GB | `/dev/sdf` |
| `ddata` | 64GB | `/dev/sdh` |
| `rbak` | 256GB | `/dev/sdf` |
| `rdata` | 64GB | `/dev/sdg` |
| `rlog` | 32GB | `/dev/sdj` |
| `sbak` | 64GB | `/dev/sdf` |
| `sdata` | 32GB | `/dev/sdh` |
| `srepo` | 32GB | `/dev/sdg` |
| //swap// | - | `/dev/sdi` |
For example, if you are brining up a `db` device, create an `adata` volume and attach it to `/dev/sdh`. Then create an `abak` volume and attach it to `/dev/sdf`.
Name any volumes you create with `phacility.net` names, for example `dbak007.phacility.net`.
Storage devices should have the same number as the hosts they are attached to. For example, `db007` should use `ddata007` and `dbak007`.
Volumes which will have backups written to them need device entries in Almanac on `admin.phacility.com`. These entries are used to store backup logs.
Deploy
======
Using the CLI tools in rCORE, deploy the device (for example, `web007`):
core/ $ ./bin/remote deploy web007
The `deploy` workflow automatically runs the `upgrade` and `restart` workflows.
Add Devices to Load Balancers
=======
If you've launched a device which is part of a load balancer pool, add it to the appropriate load balancer:
| Device | Add To | Notes |
|---|---|---|
| `admin` | `alb` | Should you really be launching these? |
| `corp` | `clb` | |
| `web` | `lb` | Should also be added to `vault` but that's muddy. |
| `notify` | `nlb` | No process for putting multiple `nlb` nodes into production yet. |
Other devices do not need to be added to load balancer pools.
Update Almanac Services
======
If you've launched a device which provides a cluster service, create a device record for it in Almanac on `admin.phacility.com`, then attach it to an existing Almanac Service (to add or replace a device in a service pool) or create a new Almanac Service for it (to add a new service pool).
Create device records for any storage devices you created, too.
//All services have only one host in them today, so this is largely future work.//