A bad half-fix for this is here:

I can reproduce this. Fixing it will take a bit of work.

Resolved by D16488.

We have a parser for comment docblocks, used to generate documentation and extract metadata from source code. This is a docblock:

I'm going to retarget this to make the API more consistent.

We should probably change this API to always return array(), the "sometimes array, sometimes scalar" return value is pretty unintuitive. And any code which fixed this by adding (array) will just work.

I'm going to close this for now since there's no more upstream action available given that the issue either no longer reproduces or disabling APCu appears to fix it and be a reasonable workaround.

Yeah, I think this didn't crop up sooner through a combination of the cache and there being relatively few files with multiple @phutil-external-symbol tags.

I'm not sure how this wasn't reported earlier, I guess the module cache prevented everyone from encountering it?

FWIW, PHP 7.1.0 RC1 was released today, and it includes the async signal change. The following amended script from T11270#184087 seems to work:

We don't really use APCu for very much, so if you are able to isolate it as the issue (or are just suspicious, e.g. based on that launchpad bug looking pretty similar) it's probably OK to turn it off. I'd expect you to lose something like 5% performance with it off, not 50%+.

Just some quick googling revealed this issue with a prior version of APCu.

I turned it back on and ran additional load tests but Apache was able to handle everything. To be honest, I was surprised how well the site held up with APCu turned off. The numbers are about the same.

If you turn APCu back on, does it freeze up again?

I've been running load tests for the past 2-3 hours and for some reason, I can't bring it down anymore. You can probably mark this bug as 'unable to reproduce'. I'll continue looking into the issue and see if there's a definitive way to reproduce.

I'm not immediately sure about the daemon confusion, although I suspect it's not likely to be a hint at the root issue. Caching for setup issues falls back to some pretty whacky shenanigans with APC/APCu disabled.

That's probably a webhook URI configured in feed.http-hooks (which may no longer be configured).

I've started the load tests without APC. One odd item I didn't notice before that may or may not be related is that Phabricator is having a hard time knowing that daemons are running (e.g. you have 2 unresolved setup issues instead of just 1 because APC is turned off). I checked the daemon logs and see they are flooded with this with one entry occurring a lot. I can't track down the source because I don't know what address it's trying to reach out to (is this a repository pull or a herald rule?). Again, this may be completely unrelated and a different issue all together.

I don't see anything interesting in Multimeter, it just shows that the host doesn't see much traffic and that we probably need to update some package definitions.

In T11573#192996, @epriestley wrote:

Are you running APC (Config → Cache Status shows APC as the active "Data Cache")?

If so, can you reproduce it if you disable APC (apc.enabled=0 in php.ini), restart apache, verify that no "Data Cache" is available any more in Config → Cache Status, and then use the takedown script?

Here's the output. I removed "events" from some of them to improve formatting.

Are you running APC (Config → Cache Status shows APC as the active "Data Cache")?

I've added a simple javascript file on the installation I grabbed this report from. As part of the troubleshooting process, I installed a vanilla installation of Phabricator on a separate machine with no modifications (version referenced in the bug report) and was able to reproduce the problem.

Also, can you show me what these pages look like?

GET /res/phabricator/86668b8e/rsrc/js/phui/rdar.js HTTP/1.1

This is a bug with how relative/object policies and inherited/indirect policies interact.

looks like a duplicate of T8647

Okay, thank you for the pointers, we have indeed a huge performance improvement using standard mysql replication, which seems to increase the queue usage.

I'm going to close this for now since I don't plan to take any further action as the upstream; this appears to be an environmental issue (which we don't offer support for) rather than a scalability limitation (which we do).

I've just dumped this on the pile of arc patch-related bugs. Some of them are likely duplicates, possibly including this one, but I don't expect to dig through that pile until the next major iteration on arc, since at least some of them are likely not addressable without other changes. T11429 has a rough overview of upcoming plans.

In T10456#184273, @jack.chung wrote:

if it helps anyone, the workaround "fix" is to place the following code in the file "src/applications/conduit/parametertype/ConduitBoolParameterType.php "

$value = (boolean) $value;

at about line 9 just before the following piece of code:

if (!is_bool($value)) {

Its a bit of a hack, so I'm not sure how much to trust this workaround for a production instance of Phabricator, but hopefully, it helps someone out there. The issue is (I think) basically related to http://stackoverflow.com/questions/27446946/preserve-data-type-of-post-fields-when-sending-data-using-php-and-curl where PHP converts the curl input as a string.

I'm not entirely sure, but it seems plausible. We compute "Queue Utilization" as basically:

Okay, seems bad indeed, do you think it might explain why the queue is capped at 20-30% ?

"Failed builds show as building" is actually very different from "Aborted builds show as building". Handling the Aborting one is a more involved than I'd like to dig into right now.

Yes, here it is.

Okay. Can you run this command?

Ah yes sorry, I saw some other reference to Nuance today and thought it was for that.

Oh, no, this is the right task. Your logs:

I think you may have answered to the wrong task :)

Someone has configured a bad Nuance source, so let's start by fixing that.

The reason we suggest @ as a delimiter is that you don't have to escape forward slashes if you use a delimiter other than /, so you get a more readable regexp.

I think I'm going insane. I was wrangling with that for ages and as you pointed out /\.js$/ does in fact work. I have no idea how I did every permutations of that regex except that one.

I believe this is fine; I can't immediately come up with an attack using -.

Form (1) is expected to work, which is why we don't document a need for form (7): one is not expected to exist. If form (1) does not work, this is a bug with Herald itself, not with the Herald documentation.

Thanks for the report. This should be fixed in HEAD of master, and fixed in production some time in the next 24 hours when the docs regenerate.

Not yet, but soon. New code should use project.edit.

Did the project.create API deprecated?

T10423 was resolved indirectly (in connection with T11524) so we've sort of inched forward here.

Try using project.edit instead.

du -sh returns 1.3GB for that repository so it should be below that limit.

This is almost certainly T4369. I would expect that this repository is larger than 2GB?