Page MenuHomePhabricator
Differential D15703 Diff 37833 src/docs/user/cluster/cluster_databases.diviner

Changeset View

Standalone View

View Options

src/docs/user/cluster/cluster_databases.diviner

Show First 20 Lines • Show All 63 Lines • ▼ Show 20 Lines
Once you've configured this option, restart Phabricator for the changes to take Once you've configured this option, restart Phabricator for the changes to take
effect, then continue to "Monitoring Replicas" to verify the configuration. effect, then continue to "Monitoring Replicas" to verify the configuration.
Monitoring Replicas Monitoring Replicas
=================== ===================
You can monitor replicas in {nav Config > Cluster Databases}. This interface You can monitor replicas in {nav Config > Database Servers}. This interface
shows you a quick overview of replicas and their health, and can detect some shows you a quick overview of replicas and their health, and can detect some
common issues with replication. common issues with replication.
The table on this page shows each database and current status. The table on this page shows each database and current status.
NOTE: This page runs its diagnostics //from the web server that is serving the NOTE: This page runs its diagnostics //from the web server that is serving the
request//. If you are recovering from a disaster, the view this page shows request//. If you are recovering from a disaster, the view this page shows
may be partial or misleading, and two requests served by different servers may may be partial or misleading, and two requests served by different servers may
▲ Show 20 Lines • Show All 60 Lines • ▼ Show 20 Lines
If things have been set up properly, Phabricator should degrade to a temporary If things have been set up properly, Phabricator should degrade to a temporary
read-only mode immediately. After a brief period of unresponsiveness, it will read-only mode immediately. After a brief period of unresponsiveness, it will
degrade further into a longer-term read-only mode. For details on how this degrade further into a longer-term read-only mode. For details on how this
works interanlly, see "Unreachable Masters" below. works interanlly, see "Unreachable Masters" below.
Once satisfied, turn the master back on. After a brief delay, Phabricator Once satisfied, turn the master back on. After a brief delay, Phabricator
should recognize that the master is healthy again and recover fully. should recognize that the master is healthy again and recover fully.
Throughout this process, the {nav Cluster Databases} console will show a Throughout this process, the {nav Database Servers} console will show a
current view of the world from the perspective of the web server handling the current view of the world from the perspective of the web server handling the
request. You can use it to monitor state. request. You can use it to monitor state.
You can perform a more narrow test by enabling `cluster.read-only` in You can perform a more narrow test by enabling `cluster.read-only` in
configuration. This will put Phabricator into read-only mode immediately configuration. This will put Phabricator into read-only mode immediately
without turning off any databases. without turning off any databases.
You can use this mode to understand which capabilities will and will not be You can use this mode to understand which capabilities will and will not be
▲ Show 20 Lines • Show All 86 Lines • ▼ Show 20 Lines
of time. This mechanism is very similar to the health checks that most HTTP of time. This mechanism is very similar to the health checks that most HTTP
load balancers perform against web servers. load balancers perform against web servers.
If a database fails several health checks in a row, Phabricator will mark it as If a database fails several health checks in a row, Phabricator will mark it as
unhealthy and stop sending all traffic (except for more health checks) to it. unhealthy and stop sending all traffic (except for more health checks) to it.
This improves performance during a service interruption and reduces load on the This improves performance during a service interruption and reduces load on the
master, which may help it recover from load problems. master, which may help it recover from load problems.
You can monitor the status of health checks in the {nav Cluster Databases} You can monitor the status of health checks in the {nav Database Servers}
console. The "Health" column shows how many checks have run recently and console. The "Health" column shows how many checks have run recently and
how many have succeeded. how many have succeeded.
Health checks run every 3 seconds, and 5 checks in a row must fail or succeed Health checks run every 3 seconds, and 5 checks in a row must fail or succeed
before Phabricator marks the database as healthy or unhealthy, so it will before Phabricator marks the database as healthy or unhealthy, so it will
generally take about 15 seconds for a database to change state after it goes generally take about 15 seconds for a database to change state after it goes
down or comes up. down or comes up.
▲ Show 20 Lines • Show All 50 Lines • Show Last 20 Lines
Phabricator · Built by Phacility