Page MenuHomePhabricator

Truncated repository browser view in Diffusion for large Mercurial repositories
Closed, ResolvedPublic

Description

Description:

The Diffusion repository browser does not show all folders of a Mercurial repository. The bug occurs for repositories with more than 100 files organised in several subdirectories. The files nevertheless exist in the repository (also in those hosted by Phabricator) and can be located via the Diffusion page view.

Example:

Code root comprises 5 subdirectories with 30 files each. The repository browser in the Diffusion repository overview page will show only the first three of the subdirectories with no possibility to navigate to the remaining two - not even when selecting "Browse Repository".

Background:

The Diffusion Repository browser truncates the file browser view according to a page limit (100) defined in the PHUIPagerView. The Mercurial browser adapter in <install_dir>/src/applications/diffusion/conduit/DiffusionBrowseQueryConduitAPIMethod.php executes truncation by comparing a count against the limit.

For Mercurial repositories the count in function getMercurialResult is computed incorrectly, as it counts also files in subdirectories, which are not displayed in the page view. Background is, that the underlying hg manifest of hg files command returns a flat list of all files. Counting is executed over all files, although files in subdirectories will not be shown on the page. Thus, for Mercurial repositories with a large number of files in subdirectories, the browser view becomes incomplete.

Proposed solution:

The count computation should increment the count only once per directory. The fix proposet below remembers the last registered directory and ignores any counter increments on the same directory. This simple fix works because hg manifest outputs always files ordered by path.

Patch:

diff --git a/src/applications/diffusion/conduit/DiffusionBrowseQueryConduitAPIMethod.php b/src/applications/diffusion/conduit/DiffusionBrowseQueryConduitAPIMethod.php
index f325837..ff12e6f 100644
--- a/src/applications/diffusion/conduit/DiffusionBrowseQueryConduitAPIMethod.php
+++ b/src/applications/diffusion/conduit/DiffusionBrowseQueryConduitAPIMethod.php
@@ -231,6 +231,9 @@ final class DiffusionBrowseQueryConduitAPIMethod
     $trim_len = $match_len ? $match_len + 1 : 0;
 
     $count = 0;
+    // remember the  last directory captured such as not to count files in subdirectories
+    // which are not displayed but are used to limit the output
+    $last_dir = '';
     foreach ($entire_manifest as $path) {
       if (strncmp($path, $match_against, $match_len)) {
         continue;
@@ -250,6 +253,11 @@ final class DiffusionBrowseQueryConduitAPIMethod
       if (count($parts) == 1) {
         $type = DifferentialChangeType::FILE_NORMAL;
       } else {
+        // count subdirectories only once
+        if ($parts[0] == $last_dir) {
+          continue;
+        }
+        $last_dir = $parts[0];
         $type = DifferentialChangeType::FILE_DIRECTORY;
       }

Event Timeline

We mitigated with:

diff --git a/src/applications/diffusion/conduit/DiffusionBrowseQueryConduitAPIMethod.php b/src/applications/diffusion/conduit/DiffusionBrowseQueryConduitAPIMethod.php
index f325837..a71152b 100644
--- a/src/applications/diffusion/conduit/DiffusionBrowseQueryConduitAPIMethod.php
+++ b/src/applications/diffusion/conduit/DiffusionBrowseQueryConduitAPIMethod.php
@@ -253,6 +253,11 @@ final class DiffusionBrowseQueryConduitAPIMethod
         $type = DifferentialChangeType::FILE_DIRECTORY;
       }
 
+      $name = reset($parts);
+      if (isset($results[$name])) {
+        continue;
+      }
+
       if ($count >= $offset) {
         $results[reset($parts)] = $type;
       }

… but whatever works, probably.

Your proposal seems even better than mine as it does not depend on the manifest ordering. I will be happy to see the fix showing up in the repository.

epriestley edited projects, added Diffusion (v3); removed Diffusion.
epriestley moved this task from Backlog to Junk Behavior on the Diffusion (v3) board.

I think this should be fixed in HEAD, let me know if I missed anything. Thanks for the report (and the patches; I did approximately the same thing in D15282).

Thank you very much for the quick fix. I already updated my Phabricator installation and it works perfectly now.