Never sever non-cluster database; write more read-only documentation

Summary:
Ref T4571. Write more of the missing documentation sections and clarify a few things.

Since the "replicating master" check needs a special permission, imposes a performance penalty, is probably very difficult to misconfigure, and likely not a big deal anyway, just drop the idea of trying to automatically detect + prevent it. We still show if it's an issue on the status page, provided we have permission to check.

When you don't have any cluster databases configured, never stop trying to connect to the default master database. We might want to do this eventually as load reduction, but just don't muddy the waters too much for now while things stabilize.

Test Plan:
  - Tested functionality in cluster, non-cluster, and degraded-cluster modes.
  - Used status console to monitor a health check cycle.
  - Read docs.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4571

Differential Revision: https://secure.phabricator.com/D15679

src/applications/config/controller/PhabricatorConfigClusterDatabasesController.php

@@ -35,6 +35,8 @@ final class PhabricatorConfigClusterDatabasesController
 
     $rows = array();
     foreach ($databases as $database) {
+      $messages = array();
+
       if ($database->getIsMaster()) {
         $role_icon = id(new PHUIIconView())
           ->setIcon('fa-database sky')
@@ -125,6 +127,9 @@ final class PhabricatorConfigClusterDatabasesController
       } else {
         $health_icon = id(new PHUIIconView())
           ->setIcon('fa-times red');
+        $messages[] = pht(
+          'UNHEALTHY: This database has failed recent health checks. Traffic '.
+          'will not be sent to it until it recovers.');
       }
 
       $health_count = pht(
@@ -138,8 +143,6 @@ final class PhabricatorConfigClusterDatabasesController
         $health_count,
       );
 
-      $messages = array();
-
       $conn_message = $database->getConnectionMessage();
       if ($conn_message) {
         $messages[] = $conn_message;