archive/phabricator
1
0
Fork 0
Code Issues 1 Pull Requests Packages Projects Releases Wiki Activity

Faster Query for Differential Updates

Browse Source 
Summary:
The old query was effectively

SELECT DISTINCT revision.* FROM `differential_revision` revision
        JOIN `differential_relationship` relationship ON
(relationship.revisionID = revision.id
            AND relationship.objectPHID in
('PHID-USER-a113b9ae4ee9524d0a20'))
          OR revision.authorPHID = 'PHID-USER-a113b9ae4ee9524d0a20'
        LEFT JOIN `differential_viewtime` viewtime ON
viewtime.viewerPHID in ('PHID-USER-a113b9ae4ee9524d0a20')
          AND viewtime.objectPHID = revision.phid
          AND GREATEST(1304022277, IFNULL(viewtime.viewTime, 0)) <
revision.dateModified
        ORDER BY dateModified DESC;

I'm not a db performance expert but it looks like the problem is that we
have to scan all revisions

  mysql> EXPLAIN SELECT DISTINCT revision.* FROM `differential_revision`
revision         JOIN `differential_relationship` relationship ON
(relationship.revisionID = revision.id             AND relationship.objectPHID
in ('PHID-USER-a113b9ae4ee9524d0a20'))           OR revision.authorPHID =
'PHID-USER-a113b9ae4ee9524d0a20'         LEFT JOIN `differential_viewtime`
viewtime ON viewtime.viewerPHID in ('PHID-USER-a113b9ae4ee9524d0a20')
AND viewtime.objectPHID = revision.phid           AND GREATEST(1304022277,
IFNULL(viewtime.viewTime, 0)) < revision.dateModified         ORDER BY
dateModified DESC;

+----+-------------+--------------+-------+--------------------+------------+---------+-------+--------+------------------------------------+
  | id | select_type | table        | type  | possible_keys      | key        |
key_len | ref   | rows   | Extra                              |

+----+-------------+--------------+-------+--------------------+------------+---------+-------+--------+------------------------------------+
  |  1 | SIMPLE      | revision     | ALL   | PRIMARY,authorPHID | NULL       |
NULL    | NULL  | 254127 | Using temporary; Using filesort    |
  |  1 | SIMPLE      | viewtime     | ref   | PRIMARY            | PRIMARY    |
66      | const |     17 | Distinct                           |
  |  1 | SIMPLE      | relationship | index | PRIMARY,objectPHID | objectPHID |
72      | NULL  | 966900 | Using where; Using index; Distinct |

+----+-------------+--------------+-------+--------------------+------------+---------+-------+--------+------------------------------------+

The new query is a lot faster

  mysql> EXPLAIN SELECT revs.* FROM ( (SELECT revision.* FROM
`differential_revision` revision WHERE revision.authorPHID in
('PHID-USER-a113b9ae4ee9524d0a20'))       UNION       (SELECT revision.* FROM
`differential_revision` revision JOIN differential_relationship rel WHERE
rel.revisionId = revision.Id AND rel.objectPHID =
'PHID-USER-a113b9ae4ee9524d0a20')) as revs     LEFT JOIN `differential_viewtime`
viewtime ON viewtime.viewerPHID = 'PHID-USER-a113b9ae4ee9524d0a20' AND
viewtime.objectPHID = revs.phid    WHERE GREATEST(1304022277,
IFNULL(viewtime.viewTime, 0)) < revs.dateModified   ORDER BY revs.dateModified;

+----+--------------+------------+--------+--------------------+------------+---------+-----------------------------------------+------+--------------------------+
  | id | select_type  | table      | type   | possible_keys      | key        |
key_len | ref                                     | rows | Extra
   |

+----+--------------+------------+--------+--------------------+------------+---------+-----------------------------------------+------+--------------------------+
  |  1 | PRIMARY      | <derived2> | ALL    | NULL               | NULL       |
NULL    | NULL                                    | 3021 | Using filesort
   |
  |  1 | PRIMARY      | viewtime   | ref    | PRIMARY            | PRIMARY    |
66      | const                                   |   17 | Using where
   |
  |  2 | DERIVED      | revision   | ref    | authorPHID         | authorPHID |
67      |                                         | 1040 | Using where
   |
  |  3 | UNION        | rel        | ref    | PRIMARY,objectPHID | objectPHID |
66      |                                         | 3822 | Using where; Using
index |
  |  3 | UNION        | revision   | eq_ref | PRIMARY            | PRIMARY    |
4       | phabricator_differential.rel.revisionID |    1 |
   |
  | NULL | UNION RESULT | <union2,3> | ALL    | NULL               | NULL
| NULL    | NULL                                    | NULL |
     |

+----+--------------+------------+--------+--------------------+------------+---------+-----------------------------------------+------+--------------------------+

Test Plan:
Loaded differential updates with new query, made sure page loaded quickly. Ran
the query from the command-line, it took about .4 seconds.

Reviewed By: Girish
Reviewers: tuomaspelkonen, jungejason, Girish
Commenters: btrahan
CC: aran, btrahan, elynde, Girish
Differential Revision: 181
This commit is contained in:
elynde
2011-04-29 15:37:15 -07:00
parent 2e96565f67
commit 72dec7cd25
1 changed files with 21 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

29
src/applications/differential/data/revisionlist/DifferentialRevisionListData.php
View File

@@ -216,17 +216,30 @@ class DifferentialRevisionListData {
$data = queryfx_all( $data = queryfx_all(
$revision->establishConnection('r'), $revision->establishConnection('r'),
'SELECT DISTINCT revision.* FROM %T revision 'SELECT revs.* FROM (
JOIN %T rel ON rel.revisionID = revision.id (
AND (rel.objectPHID in (%Ls) OR revision.authorPHID in (%Ls)) SELECT revision.*
LEFT JOIN %T viewtime ON viewtime.viewerPHID in (%Ls) FROM %T revision
AND viewtime.objectPHID = revision.phid WHERE revision.authorPHID in (%Ls)
WHERE GREATEST(%d, IFNULL(viewtime.viewTime, 0)) < revision.dateModified )
%Q', UNION
(
SELECT revision.*
FROM %T revision
JOIN %T rel
WHERE rel.revisionId = revision.Id AND rel.objectPHID in (%Ls)
)
) as revs
LEFT JOIN %T viewtime ON
viewtime.viewerPHID in (%Ls)
AND viewtime.objectPHID = revs.phid
WHERE GREATEST(%d, IFNULL(viewtime.viewTime, 0)) < revs.dateModified
%Q',
$revision->getTableName(),
$this->ids,
$revision->getTableName(), $revision->getTableName(),
DifferentialRevision::RELATIONSHIP_TABLE, DifferentialRevision::RELATIONSHIP_TABLE,
$this->ids, $this->ids,
$this->ids,
DifferentialRevision::TABLE_VIEW_TIME, DifferentialRevision::TABLE_VIEW_TIME,
$this->ids, $this->ids,
$min_view_time, $min_view_time,