Make SQL patch management DAG-based and provide namespace support

Summary:
This addresses three issues with the current patch management system:

  # Two people developing at the same time often pick the same SQL patch number, and then have to go rename it. The system catches this, but it's silly.
  # Second/third-party developers can't use the same system to manage auxiliary storage they may want to add.
  # There's no way to build mock databases for unit tests that need to do reads.

To resolve these things, you can now name your patches whatever you want and conflicts are just merge conflicts, which are less of a pain to fix than filename conflicts.

Dependencies are now a DAG, with implicit dependencies created on the prior patch if no dependencies are specified. Developers can add new concrete subclasses of `PhabricatorSQLPatchList` to add storage management, and define the dependency branchpoint of their patches so they apply in the correct order (although, generally, they should not depend on the mainline patches, presumably).

The commands `storage upgrade --namespace test1234` and `storage destroy --namespace test1234` will allow unit tests to build and destroy MySQL storage.

A "quickstart" mode allows an upgrade from scratch in ~1200ms. Destruction takes about 200ms. These seem like fairily reasonable costs to actually use in tests. Building from scratch patch-by-patch takes about 6000ms.

Test Plan:
  - Created new databases from scratch with and without quickstart in a separate test namespace. Pointed the webapp at the test namespaces, browsed around, everything looked good.
  - Compared quickstart and no-quickstart dump states, they're identical except for mysqldump timestamps and a few similar things.
  - Upgraded a legacy database to the new storage format.
  - Destroyed / dumped storage.

Reviewers: edward, vrana, btrahan, jungejason

Reviewed By: btrahan

CC: aran, nh

Maniphest Tasks: T140, T345

Differential Revision: https://secure.phabricator.com/D2323

src/docs/configuration/configuration_guide.diviner

@@ -8,17 +8,6 @@ This document contains basic configuration instructions for Phabricator.
 This document assumes you've already installed all the components you need.
 If you haven't, see @{article:Installation Guide}.
 
-= Configuring MySQL =
-
-Get MySQL running and verify you can connect to it. Consult the MySQL
-documentation for help. When MySQL works, you need to load the Phabricator
-schemata into it. First, load the initial database schema.
-
-  mysql -uroot < path/to/phabricator/resources/sql/init/initialize.sql
-
-After this you need to upgrade the schema (see @{article:Upgrading Schema}),
-but you need to finish the rest of the configuration first.
-
 = Configuring Phabricator =
 
 Create a new file here:
@@ -59,6 +48,43 @@ For the last line, you can also use ##'development'## instead of
 ##'production'## if you are planning to develop Phabricator itself. This will
 turn on some debugging features.
 
+= PHABRICATOR_ENV Environment Variable =
+
+When running Phabricator scripts, they will ask you to set the `PHABRICATOR_ENV`
+environment variable to point at your config. If you put your script in
+`custom/myconfig.conf.php`, you can identify the config with
+`custom/myconfig`, like this:
+
+  $ PHABRICATOR_ENV=custom/myconfig ./some_phabricator_command
+
+NOTE: Make sure you put 'PHABRICATOR_ENV=...' at the beginning of the line, not
+in the middle. The shell won't parse environmental variables declared after the
+command. You can also ##export PHABRICATOR_ENV=...## in your ~/.bashrc or
+~/.profile or similar, depending on which shell you use and how your system is
+configured.
+
+= Storage: Configuring MySQL =
+
+Get MySQL running and verify you can connect to it. Consult the MySQL
+documentation for help. When MySQL works, you need to load the Phabricator
+schemata into it. To do this, run:
+
+  phabricator/ $ ./bin/storage upgrade
+
+If your configuration uses an unprivileged user to connect to the database, you
+may have to override the default user so the schema changes can be applied with
+root or some other admin user:
+
+  phabricator/ $ ./bin/storage upgrade --user <user> --password <password>
+
+You can avoid the prompt the script issues by passing the ##--force## flag (for
+example, if you are scripting the upgrade process).
+
+  phabricator/ $ ./bin/storage upgrade --force
+
+NOTE: When you update Phabricator, run `storage upgrade` again to apply any
+new updates.
+
 = Webserver: Configuring Apache =
 
 Get Apache running and verify it's serving a test page. Consult the Apache
@@ -145,12 +171,6 @@ For nginx, use a configuration like this:
     }
   }
 
-= Upgrading Schema =
-
-Now, it's time for you to upgrade the database schema -- see
-@{article:Upgrading Schema}. You'll also need to do this after you update the
-code in the future.
-
 = Setup =
 
 Now, restart your webserver and navigate to whichever subdomain you set up. You
@@ -200,7 +220,6 @@ that will make upgrading Phabricator more difficult in the future.
 
 Continue by:
 
-  - upgrading the database schema with @{article:Upgrading Schema}; or
   - setting up your admin account and login/registration with
     @{article:Configuring Accounts and Registration}; or
   - configuring where uploaded fils and attachments will be stored with

src/docs/configuration/upgrade_schema.diviner

@@ -1,36 +0,0 @@
-@title Upgrading Schema
-@group config
-
-This document describes how to upgrade the database schema.
-
-= Prerequisites =
-
-This document assumes you've already initialized the MySQL database and
-configured your Phabricator environment. If you haven't, see
-@{article:Configuration Guide}.
-
-= Loading patches =
-
-To upgrade your database schema to the latest version, just run this command:
-
-  PHABRICATOR_ENV=<your_config> path/to/phabricator/scripts/sql/upgrade_schema.php
-
-NOTE: Make sure you put 'PHABRICATOR_ENV=...' at the beginning of the line, not
-in the middle. The shell won't parse environmental variables declared after the
-command. You can also ##export PHABRICATOR_ENV=...## in your ~/.bashrc or
-~/.profile or similar, depending on which shell you use and how your system is
-configured.
-
-This will install all the patches that are new since you installed, or since the
-last time you ran this script.
-
-If your configuration uses an unprivileged user to connect to the database, you
-may have to override the default user so the schema changes can be applied with
-root or some other admin user:
-
-  PHABRICATOR_ENV=<your_config> path/to/phabricator/scripts/sql/upgrade_schema.php -u <user> -p <pass>
-
-You can avoid the prompt the script issues by passing the ##-f## flag (for
-example, if you are scripting the upgrade process).
-
-  PHABRICATOR_ENV=<your_config> path/to/phabricator/scripts/sql/upgrade_schema.php -f

src/docs/installation_guide.diviner

@@ -118,8 +118,17 @@ See <https://bugs.php.net/bug.php?id=59747> for more information.
 
 = Updating Phabricator =
 
-Since Phabricator is under active development, you should update frequently.
-You can use a script similar to this one to automate the process:
+Since Phabricator is under active development, you should update frequently. To
+update Phabricator:
+
+  - Stop the webserver.
+  - Run `git pull && git submodule update --init` in `libphutil/`,
+    `arcanist/` and `phabricator/`.
+  - Run `phabricator/bin/storage upgrade`.
+  - Restart the webserver.
+
+For more details, see @{article:Configuration Guide}. You can use a script
+similar to this one to automate the process:
 
   http://www.phabricator.com/rsrc/install/update_phabricator.sh