Details
Hello,
We have an install of Phabricator for Swiss univercities (https://c4science.ch) and we're facing some scaling issues. I know importing commits takes a long time in phab, and I'm looking to improving it. I also wonder how does the task daemons works, because increasing the number of daemons doesn't help executing task faster so much. Should it be the case ?
Here's for instance the state of our daemons, with 5 taskmaster running (limit is set to 20, but it's not spawning more apparently..)
Any other inputs on the subject is welcome.
Regards,
Jean-Baptiste
Answers
I must add that we'll have a lot of users (1000+) importing existing repositories over the summer. We're considering creating a dedicated import page that doesn't do the Herald rule checks, but I'm not sure it will be enough to have good performances while keeping the platform in good shape (db, filesystem, ...).
Can you file a bug report for this? If "Queued Tasks" is not empty and "Queue Utilization" is not in the realm of 100%, something is wrong.
Please include current screenshots of "Recently Completed Tasks", "Active Daemons", "Queued Tasks", "Leased Tasks", and "Next in Queue", the output of bin/phd log, and the version of Phabricator you are running.
From the screenshots in the original question, I see a lot of queued SearchWorker tasks but none completed in the last 15 minutes. That might be coincidental, or might suggest that, e.g., you have an issue with your search index configuration which is preventing those tasks from completing and clogging up the queue. But please provide up-to-date information and I'll look at what's happening.
(In general, if Phabricator isn't performing well or scaling under reasonable workloads, that's a bug and you should file a bug report to make sure it gets looked at.)