Page MenuHomePhabricator

Timeout error when loading "All Audits" query from a user who should not have access to any commits/audits
Closed, ResolvedPublic

Description

I'm testing out Spaces for isolating a group of users to only have access to a few select sections of our Phabricator install (namely Maniphest, Macros, Herald -- however I'm not yet denying access to other applications).

When logged in as an "outside user" I navigate to the Diffusion application and when I browse through the queries I verify that I have access to no repositories. When navigating to the Audit application I browse through the queries and verify that no audits/commits are visible -- however if I try the "All Commits" query the page attempts to load ~30sec and then reports:

phabricator.html
>>> UNRECOVERABLE FATAL ERROR <<<

Maximum execution time of 30 seconds exceeded

/usr/local/phacility/libphutil/src/parser/PhutilTypeSpec.php:1909


┻━┻ ︵ ¯\_(ツ)_/¯ ︵ ┻━┻

From nginx's error.log there was not a lot of information:

/var/log/nginx/error.log
2015/09/15 01:34:18 [info] 51945#0: *110 client 10.0.0.100 closed keepalive connection
2015/09/15 01:34:41 [info] 51945#0: *108 epoll_wait() reported that client prematurely closed connection, so upstream connection is closed too while sending request to upstream, client: 10.0.0.100, serv
er: phab.company.com, request: "POST /dashboard/panel/render/2/ HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "phab.company.com"
2015/09/15 01:34:49 [info] 51945#0: *111 client 10.0.0.100 closed keepalive connection
2015/09/15 01:34:49 [info] 51945#0: *109 client 10.0.0.100 closed keepalive connection
2015/09/15 01:34:51 [info] 51945#0: *254 epoll_wait() reported that client prematurely closed connection, so upstream connection is closed too while sending request to upstream, client: 10.0.0.100, serv
er: phab.company.com, request: "POST /dashboard/panel/render/2/ HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "phab.company.com"
2015/09/15 01:35:34 [info] 51945#0: *264 client closed connection while waiting for request, client: 10.0.0.100, server: 0.0.0.0:443
2015/09/15 01:35:35 [error] 51945#0: *91 FastCGI sent in stderr: "PHP message: PHP Fatal error:  Maximum execution time of 30 seconds exceeded in /usr/local/phacility/phabricator/src/infrastructure/quer
y/policy/PhabricatorPolicyAwareQuery.php on line 0
PHP message: >>> UNRECOVERABLE FATAL ERROR <<<

Maximum execution time of 30 seconds exceeded

/usr/local/phacility/phabricator/src/infrastructure/query/policy/PhabricatorPolicyAwareQuery.php:0


┻━┻ ︵ ¯\_(ツ)_/¯ ︵ ┻━┻" while reading response header from upstream, client: 10.0.0.100, server: phab.company.com, request: "GET /audit/query/all/ HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "phab.company.com"

There was no relevent (or recent from today) output from /var/log/mysql.d nor /var/log/php-fpm/error.log.

I just noticed that the two errors are different - the logs have multiple instances of the errors which reference both files.

Event Timeline

cspeckmim updated the task description. (Show Details)
cspeckmim added a project: Audit.
cspeckmim added a subscriber: cspeckmim.

Forgot to mention that a user who has access to all the repositories can load the "All Commits" query results in ~1-2secs.

Also, the install is at:

epriestley moved this task from Backlog to EditEngine on the Audit board.Jan 10 2017, 4:38 PM
epriestley closed this task as Resolved.Jan 10 2017, 5:04 PM
epriestley claimed this task.
epriestley added a subscriber: epriestley.

I believe this was resolved by the introduction of the "query overheating" mechanism, described in T11773.

Users will now receive a warning that the query overheated an recommendation that they refine their search parameters.

This is a fairly broad response, and we could do more to improve this, but it sounds like the issue was largely just that this fataled; it no longer should. If you're still seeing issues at HEAD, feel free to file a new report.