1fd2d28b11ef8b647bcc035cb84ee305527cf776
9 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
epriestley
|
42c0f060d5 |
Push feed publishing deeper into the task queue
Summary:
Ref T2852. I want to model Asana integration as a response to feed events. Currently, we queue one feed event for each HTTP hook.
Instead, always queue one feed event and then have it queue any necessary followup events (now, http hooks; soon, asana).
Add a script to make it easy to reproducibly fire feed event publishing.
Test Plan:
Republished a feed event and verified it hit configured HTTP hooks correctly.
$ ./bin/feed republish 5765774156541908292 --trace
>>> [2] <connect> phabricator2_feed
<<< [2] <connect> 1,660 us
>>> [3] <query> SELECT story.* FROM `feed_storydata` story JOIN `feed_storyreference` ref ON ref.chronologicalKey = story.chronologicalKey WHERE (ref.chronologicalKey IN (5765774156541908292)) GROUP BY story.chronologicalKey ORDER BY story.chronologicalKey DESC
<<< [3] <query> 595 us
>>> [4] <connect> phabricator2_differential
<<< [4] <connect> 760 us
>>> [5] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
<<< [5] <query> 478 us
>>> [6] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
<<< [6] <query> 449 us
>>> [7] <connect> phabricator2_user
<<< [7] <connect> 1,062 us
>>> [8] <query> SELECT * FROM `user` WHERE phid in ('PHID-USER-lqiz3yd7wmk64ejugvov')
<<< [8] <query> 540 us
>>> [9] <connect> phabricator2_file
<<< [9] <connect> 951 us
>>> [10] <query> SELECT * FROM `file` WHERE phid IN ('PHID-FILE-gq6dlsysvxbn3dgwvky7')
<<< [10] <query> 498 us
>>> [11] <query> SELECT * FROM `user_status` WHERE userPHID IN ('PHID-USER-lqiz3yd7wmk64ejugvov') AND UNIX_TIMESTAMP() BETWEEN dateFrom AND dateTo
<<< [11] <query> 507 us
Republishing story...
>>> [12] <query> SELECT story.* FROM `feed_storydata` story JOIN `feed_storyreference` ref ON ref.chronologicalKey = story.chronologicalKey WHERE (ref.chronologicalKey IN (5765774156541908292)) GROUP BY story.chronologicalKey ORDER BY story.chronologicalKey DESC
<<< [12] <query> 685 us
>>> [13] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
<<< [13] <query> 489 us
>>> [14] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
<<< [14] <query> 512 us
>>> [15] <query> SELECT * FROM `user` WHERE phid in ('PHID-USER-lqiz3yd7wmk64ejugvov')
<<< [15] <query> 601 us
>>> [16] <query> SELECT * FROM `file` WHERE phid IN ('PHID-FILE-gq6dlsysvxbn3dgwvky7')
<<< [16] <query> 405 us
>>> [17] <query> SELECT * FROM `user_status` WHERE userPHID IN ('PHID-USER-lqiz3yd7wmk64ejugvov') AND UNIX_TIMESTAMP() BETWEEN dateFrom AND dateTo
<<< [17] <query> 551 us
>>> [18] <query> SELECT story.* FROM `feed_storydata` story JOIN `feed_storyreference` ref ON ref.chronologicalKey = story.chronologicalKey WHERE (ref.chronologicalKey IN (5765774156541908292)) GROUP BY story.chronologicalKey ORDER BY story.chronologicalKey DESC
<<< [18] <query> 507 us
>>> [19] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
<<< [19] <query> 428 us
>>> [20] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
<<< [20] <query> 419 us
>>> [21] <query> SELECT * FROM `user` WHERE phid in ('PHID-USER-lqiz3yd7wmk64ejugvov')
<<< [21] <query> 591 us
>>> [22] <query> SELECT * FROM `file` WHERE phid IN ('PHID-FILE-gq6dlsysvxbn3dgwvky7')
<<< [22] <query> 406 us
>>> [23] <query> SELECT * FROM `user_status` WHERE userPHID IN ('PHID-USER-lqiz3yd7wmk64ejugvov') AND UNIX_TIMESTAMP() BETWEEN dateFrom AND dateTo
<<< [23] <query> 593 us
>>> [24] <http> http://127.0.0.1/derp/
<<< [24] <http> 746,157 us
[2013-06-24 20:23:26] EXCEPTION: (HTTPFutureResponseStatusHTTP) [HTTP/500] Internal Server Error
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2852
Differential Revision: https://secure.phabricator.com/D6291
|
||
|
epriestley
|
c94ef134e4 |
Add bin/auth refresh for debugging OAuth token refresh issues
Summary: Ref T2852. Provide a script for inspecting/debugging OAuth token refresh. Test Plan: Ran `bin/auth refresh` with various arguments, saw token refreshes. Reviewers: btrahan Reviewed By: btrahan CC: aran Maniphest Tasks: T2852 Differential Revision: https://secure.phabricator.com/D6276 |
||
|
epriestley
|
c0cc7bbfdf |
Provide bin/auth ldap for LDAP diagnostics
Summary: Ref T1536. Test Plan: Ran `bin/auth ldap`. Reviewers: mbishopim3, chad Reviewed By: mbishopim3 CC: aran Maniphest Tasks: T1536 Differential Revision: https://secure.phabricator.com/D6218 |
||
|
epriestley
|
278905543e |
Add very basic bin/auth tool
Summary: Ref T1536. This script basically exists to restore access if/when users shoot themselves in the foot by disabling all auth providers and can no longer log in.
Test Plan: {F46411}
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T1536
Differential Revision: https://secure.phabricator.com/D6205
|
||
|
epriestley
|
baa9d96082 |
Increase the power of bin/config
Summary: Fixes T2254. Make the CLI for config more powerful: - Add validation for `set`. - Add `get`. - Add `list`. - Add `delete`. The `get` command produces fairly verbose JSON to support flags like `--all`, or `--source database` later. The other commands are straightforward. Test Plan: Tested `config set`: $ ./bin/config set Usage Exception: Specify a configuration key and a value to set it to. $ ./bin/config set x Usage Exception: Specify a value to set the key 'x' to. $ ./bin/config set phabricator.base-uri Usage Exception: Specify a value to set the key 'phabricator.base-uri' to. $ ./bin/config set phabricator.base-uri x Usage Exception: Config option 'phabricator.base-uri' is invalid. The URI must start with 'http://' or 'https://'. $ ./bin/config set phabricator.base-uri http://x Usage Exception: Config option 'phabricator.base-uri' is invalid. The URI must contain a dot ('.'), like 'http://example.com/', not just a bare name like 'http://example/'. Some web browsers will not set cookies on domains with no TLD. $ ./bin/config set phabricator.base-uri http://x.com Set 'phabricator.base-uri' in local configuration. $ Tested `config get`: $ ./bin/config get pygments.enabled { "config" : [] } $ ./bin/config set pygments.enabled true Set 'pygments.enabled' in local configuration. $ ./bin/config get pygments.enabled { "config" : [ { "key" : "pygments.enabled", "source" : "local", "value" : true } ] } $ Tested `config delete`: $ ./bin/config delete Usage Exception: Specify a configuration key to delete. $ ./bin/config delete x x Usage Exception: Too many arguments: expected one key. $ ./bin/config delete x Usage Exception: No such configuration key 'x'! Use `config list` to list all keys. $ ./bin/config delete pygments.enabled Deleted 'pygments.enabled' from local configuration. $ ./bin/config delete pygments.enabled Usage Exception: Configuration key 'pygments.enabled' is not set in local configuration! $ Tested `config list`: $ ./bin/config list account.editable account.minimum-password-length amazon-ec2.access-key amazon-ec2.secret-key amazon-s3.access-key amazon-s3.endpoint amazon-s3.secret-key amazon-ses.access-key amazon-ses.secret-key aphront.default-application-configuration-class audit.can-author-close-audit auth.email-domains auth.login-message auth.password-auth-enabled auth.require-email-verification auth.sessions.conduit auth.sessions.web auth.sshkeys.enabled cache.enable-deflate celerity.force-disk-reads celerity.minify celerity.resource-hash celerity.resource-path config.hide config.lock config.mask controller.oauth-registration darkconsole.always-on darkconsole.enabled debug.profile-rate debug.stop-on-redirect differential.allow-reopen differential.allow-self-accept differential.always-allow-close differential.anonymous-access differential.custom-remarkup-block-rules differential.custom-remarkup-rules differential.days-fresh differential.days-stale differential.enable-email-accept differential.expose-emails-prudently differential.field-selector differential.generated-paths differential.require-test-plan-field differential.revision-custom-detail-renderer differential.show-host-field differential.show-test-plan-field differential.whitespace-matters disqus.application-id disqus.application-secret disqus.auth-enabled disqus.auth-permanent disqus.registration-enabled disqus.shortname environment.append-paths events.listeners facebook.application-id facebook.application-secret facebook.auth-enabled facebook.auth-permanent facebook.registration-enabled facebook.require-https-auth feed.http-hooks feed.public files.image-mime-types files.viewable-mime-types gcdaemon.ttl.daemon-logs gcdaemon.ttl.differential-parse-cache gcdaemon.ttl.general-cache gcdaemon.ttl.herald-transcripts gcdaemon.ttl.markup-cache gcdaemon.ttl.task-archive github.application-id github.application-secret github.auth-enabled github.auth-permanent github.registration-enabled google.application-id google.application-secret google.auth-enabled google.auth-permanent google.registration-enabled ldap.activedirectory_domain ldap.anonymous-user-name ldap.anonymous-user-password ldap.auth-enabled ldap.base_dn ldap.hostname ldap.port ldap.real_name_attributes ldap.referrals ldap.search-first ldap.search_attribute ldap.start-tls ldap.username-attribute ldap.version load-libraries log.access.format log.access.path maniphest.custom-fields maniphest.custom-task-extensions-class maniphest.default-priority maniphest.enabled metamta.can-send-as-user metamta.default-address metamta.differential.attach-patches metamta.differential.inline-patches metamta.differential.patch-format metamta.differential.reply-handler metamta.differential.reply-handler-domain metamta.differential.subject-prefix metamta.differential.unified-comment-context metamta.diffusion.attach-patches metamta.diffusion.byte-limit metamta.diffusion.inline-patches metamta.diffusion.reply-handler metamta.diffusion.reply-handler-domain metamta.diffusion.subject-prefix metamta.diffusion.time-limit metamta.domain metamta.herald.show-hints metamta.insecure-auth-with-reply-to metamta.macro.reply-handler-domain metamta.macro.subject-prefix metamta.mail-adapter metamta.maniphest.default-public-author metamta.maniphest.public-create-email metamta.maniphest.reply-handler metamta.maniphest.reply-handler-domain metamta.maniphest.subject-prefix metamta.one-mail-per-recipient metamta.package.reply-handler metamta.package.subject-prefix metamta.pholio.reply-handler-domain metamta.pholio.subject-prefix metamta.placeholder-to-recipient metamta.precedence-bulk metamta.public-replies metamta.re-prefix metamta.recipients.show-hints metamta.reply.show-hints metamta.send-immediately metamta.single-reply-handler-prefix metamta.user-address-format metamta.vary-subjects mysql.configuration-provider mysql.host mysql.implementation mysql.pass mysql.user notification.client-uri notification.debug notification.enabled notification.log notification.pidfile notification.server-uri notification.user phabricator.application-id phabricator.application-secret phabricator.auth-enabled phabricator.auth-permanent phabricator.base-uri phabricator.csrf-key phabricator.env phabricator.mail-key phabricator.oauth-uri phabricator.production-uri phabricator.registration-enabled phabricator.serious-business phabricator.setup phabricator.show-beta-applications phabricator.show-error-callout phabricator.show-stack-traces phabricator.timezone phame.skins phd.log-directory phd.pid-directory phd.start-taskmasters phd.trace phd.verbose phid.external-loaders phpmailer.mailer phpmailer.smtp-host phpmailer.smtp-password phpmailer.smtp-port phpmailer.smtp-protocol phpmailer.smtp-user phriction.enabled policy.allow-public pygments.dropdown-choices pygments.enabled recaptcha.enabled recaptcha.private-key recaptcha.public-key remarkup.enable-embedded-youtube repository.default-local-path search.elastic.host search.engine-selector security.alternate-file-domain security.hmac-key security.require-https sendgrid.api-key sendgrid.api-user storage.default-namespace storage.engine-selector storage.local-disk.path storage.mysql-engine.max-size storage.s3.bucket storage.upload-size-limit style.monospace syntax-highlighter.engine syntax.filemap test.value tokenizer.ondemand translation.override translation.provider uri.allowed-protocols $ Reviewers: btrahan, codeblock Reviewed By: btrahan CC: aran Maniphest Tasks: T2254 Differential Revision: https://secure.phabricator.com/D4570 |
||
|
epriestley
|
d5b006b2cc |
Remove pcntl/php CLI setup checks
Summary: As far as I know, we never actually need `php` to be available from the web UI. I think the history here is: - Long ago, we checked for 'pcntl' as an extension during setup. - Someone had an install where 'pcntl' was available from the CLI, but not the web UI. So we switched the check to use the CLI. - Someone had an install where the CLI binary was php-fpm, which caused the 'pcntl' check to loop endlessly, so we added more checks. But we don't actually need to do any of this -- when the user tries to run the daemons, they get an explicit message that they need to install pcntl already, and we never (as far as I know) try to run PHP scripts from the web UI other than the pcntl_available.php check (we only run `git`, `svn`, `hg`, `ssh-agent`, `diff`, `xhpast` and `pygmentize`, I think). Test Plan: Thought carefully about places we might execute PHP scripts from the web UI. Looked through /scripts/ to try to identfiy anything we might execute. Reviewers: btrahan, vrana Reviewed By: btrahan CC: aran Differential Revision: https://secure.phabricator.com/D4568 |
||
|
epriestley
|
ba489f9d85 |
Add a local configuration source and a non-environmental ENV config source
Summary:
See discussion in T2221. Before we can move configuration to the database, we have a bootstrapping problem: we need database credentials to live //somewhere// if we can't guess them (and we can only really guess localhost / root / no password).
Some options for this are:
- Have them live in ENV variables.
- These are often somewhat unfamiliar to users.
- Scripts would become a huge pain -- you'd have to dump a bunch of stuff into ENV.
- Some environments have limited ability to set ENV vars.
- SSH is also a pain.
- Have them live in a normal config file.
- This probably isn't really too awful, but:
- Since we deploy/upgrade with git, we can't currently let them edit a file which already exists, or their working copy will become dirty.
- So they have to copy or create a file, then edit it.
- The biggest issue I have with this is that it will be difficult to give specific, easily-followed directions from Setup. The instructions need to be like "Copy template.conf.php to real.conf.php, then edit these keys: x, y, z". This isn't as easy to follow as "run script Y".
- Have them live in an abnormal config file with script access (this diff).
- I think this is a little better than a normal config file, because we can tell users 'run phabricator/bin/config set mysql.user phabricator' and such, which is easier to follow than editing a config file.
I think this is only a marginal improvement over a normal config file and am open to arguments against this approach, but I think it will be a little easier for users to deal with than a normal config file. In most cases they should only need to store three values in this file -- db user/host/pass -- since once we have those we can bootstrap everything else. Normal config files also aren't going away for more advanced users, we're just offering a simple alternative for most users.
This also adds an ENVIRONMENT file as an alternative to PHABRICATOR_ENV. This is just a simple way to specify the environment if you don't have convenient access to env vars.
Test Plan: Ran `config set x y`, verified writes. Wrote to ENVIRONMENT, ran `PHABRICATOR_ENV= ./bin/repository`.
Reviewers: btrahan, vrana, codeblock
Reviewed By: codeblock
CC: aran
Maniphest Tasks: T2221
Differential Revision: https://secure.phabricator.com/D4294
|
||
|
vrana
|
ef85f49adc |
Delete license headers from files
Summary: This commit doesn't change license of any file. It just makes the license implicit (inherited from LICENSE file in the root directory). We are removing the headers for these reasons: - It wastes space in editors, less code is visible in editor upon opening a file. - It brings noise to diff of the first change of any file every year. - It confuses Git file copy detection when creating small files. - We don't have an explicit license header in other files (JS, CSS, images, documentation). - Using license header in every file is not obligatory: http://www.apache.org/dev/apply-license.html#new. This change is approved by Alma Chao (Lead Open Source and IP Counsel at Facebook). Test Plan: Verified that the license survived only in LICENSE file and that it didn't modify externals. Reviewers: epriestley, davidrecordon Reviewed By: epriestley CC: aran, Korvin Maniphest Tasks: T2035 Differential Revision: https://secure.phabricator.com/D3886 |
||
|
epriestley
|
ead9bbfeb1 |
Test for pcntl availability from the command line, not Apache
Summary: In RHEL6 at the least, pcntl installs from distro package management to the CLI but not to Apache. Since we don't need it in apache and it's a pain to build manually, just verify it exists on the CLI. Test Plan: Simulated script failures and verified setup output. Reviewed By: codeblock Reviewers: codeblock, aran, jungejason, tuomaspelkonen CC: aran, epriestley, kevinwallace, codeblock Differential Revision: 380 |