Ok, i've checked all the implementations of getTitleForFeed and getApplicationTransactionTitleForFeed which get $story passed and it is never used anywhere...
Want me to submit a diff removing it?

Remove all the hacks from last year ;)
calling getTitleForFeed without a Parameter is still bad and might break other use cases.
However making up a dummy $story is tricky as it expects a PhabricatorFeedStoryData Object which is a DAO...

The SaveController block is needed to prevent the user mentioned being added as a CC if she has no permission.
Without it they are greyed out correctly but are still added as CC.
In the TransactionEditor we are only catching mentions and do not look at already added subscriber transactions.
Normally i'd think the block in the ManiphestTransactionSaveController should be removed completely as the mentions are extracted in the TransactionEditor anyway.
However this seems not to be the case for maniphest comments. When i remove the block in question mentioning no longer works at all.

sry, i'll have to do this tomorrow. Somehow the RepositoryRemarkupRule doesen't allow me not to specify a word prefix (just like in the commit Remarkup rule)...

PhabricatorRepositoryQuery now disallows leading 'r' and classes using it have to strip / attach it themselves. Preserving it in the identifierMap leads to much worse problems.
Refactored the Query to use it's own private variables and added phid support.
I'll change the r1 rTEST vs. R1 rTEST next

updating for style and inlines

oops

yep you were right. just missed a case.

Ah ok. rendering policy aware stuff in preview is problematic as the preview renderer just creates a bogus task and adds the text to it...
To make the preview consider policies it would have to load the complete real task. Not sure if we actually want to do this.

added a few more test cases for the new repos without callsigns

found out about non capturing syntax for regexes and fixed the unit test :)

clean up CommitQuery identifierMap generation.
I tried to refactor it completely but was unable to achieve O(n) in any other way as it is.

api actually was that way all along

first stage of cleanup. PhabricatorRepositoryQuery now builds an identifierMap (just like the DiffusionCommitQuery supports)
and also allows searching for multiple different kinds of identifiers at the same time (ids, phid, callsigns)
building the identifierMap in CommitQuery is still a mess and work in progress

changed wording

Display user having no permissions to a task with a grey background. (actual style TBD ;) )
For a start now only in maniphest. If this is ok i'll track down all the other uses of MarkupEngine.

the problem when creating a diff still persists but is not so easy to fix. The problem is that the diff is created with default permissions (public) and then of course the user can be mentioned in the comments too. Maybe a diff should inherit it's policies from the repository it's attached to?

execute instead of loadpage and check if policy is available for the object

I'll add the other changes. Would you already like handling the 'explicit cc' case? (It was indicated in the task that it might be too severe for the start.

As far as i can see at the moment a remarkup rule is not aware of what context it is rendered in (which specific task / diff / commit and even which type).
I think it would generelly be useful to have that information especially for policy stuff but I'm not sure how to approach this.

add policy check when commenting on tasks

Ah i forgot about the greying out users... That should provide enough feedback why a user is not mentioned

Yeah i've got no real idea there either. Might be a good idea to move the whole 'run things as another user' stuff into the ExecFuture (or have an SudoExecFuture) and try to handle it centrally in the future.

"a password is required" might be a locale specific string. (I've seen users have german text there).
Just checking for sudo: however might match some pam warnings that start the daemon anyway which might result in two daemons.. :(

Yep works fine now..

Ok, i can reproduce it with a user who's shell is set to /sbin/nologin ...

When you start the daemons as btrahan it'll try to do a sudo -u _phd .... which you might not have the permissions for.
Normally this should result in an exception and the code will start the daemon as your normal user as a fallback.
If your sudo fails without the "CommandException" phd will think the daemons have started normally.
Can you try to manually run:
sudo -En -u _phd -- scripts/daemon/phd-daemon 'PhabricatorTaskmasterDaemon' --trace --verbose --phd='/var/tmp/phd/pid'
from your phabricator dir?

I should add that the current setup check only alerts if there's a mismatch between config and running daemon.
If phd.user is not configured at all (default) it will not trigger.

fix inlines and add user field to webui

give me a moment and i'll send a new diff which fixes this and also adds the user to the web ui.

Yeah you're right. the cd %s && is redundant. I added it when i had some problems with 'su'. But for sudo it works fine without.

add the missing setup check

I think using runuser would limit the portability quite a bit and most admins are familiar with sudo.

use sudo and fix debug. also try without sudo if it fails.
Log runningAsUser to the daemon log

Sounds good. I'm not quite sure about the "if sudo fails" part though as i can't really differentiate between sudo failing and the phd daemon failing for another reason.
Worst that can happen though is that the daemon fails to start twice.
The problem with the %s for the command is though that it was built in the csprintf just before and the double escaping will produce things like ('./phd-daemon '\''PhabricatorRepositoryPullLocalDaemon'\''). %C for the command should be safe though.

I initially didn't get the '-1000' being the phabricator limit exception and wondered why there were two identical error cases ;)
But to differentiate here totally makes sense.
As i'm not a native speaker, feedback about my wording is always appreciated even when it might seem nitpicky.

adjust wording and seperate the two error conditions

I think the main difference here was to show the user the actual size limit. Otherwise he'd have to guess, maybe compress the pdf/txt whatever and reupload to get the same error.

• add new db schema config
• populate blame cache on reparse
• replaced some code with an already existing conduit call
• rebasing and fix db validity issues

remove mapping stuff

This diff is not really meant to be merged. (And i'm not sure if can even create a diff on top of another not yet landed diff?)
But you're right that if we want the numbers to be correct you should apply the patch from D10955 and then this one on top
and then only run: ./bin/search find_duplicates
I'll just remove the mapping stuff from this diff.

Attached diff allows for batch edits of all tasks in a workboard column (only all, no selection possible). And please note that you have to reload the board when you make column moves and want to batch edit afterwards!
Very handy to close all tasks in a column or add/remove a project.

It should also check then if the mapping is correct and warn if a mismatch is detected.
I think the typical problem would be that someone has an phabricator install, enables elasticsearch and then some random task/commit gets indexed automatically and creates the index without the mapping. Even if the user then deletes the index. In the time between running init / reindex just a single object needs to be reindexed to create the index without its mapping again and i don't like the idea of deleting it in an init/setup script. So just a setupcheck telling the user how to set it up in a clean way is probably best.
@WikiChad wannna add the check? otherwise i could do it if you like.

I thought about doing something just like this. However i didn't like the fact that every search request now does an additional elasticsearch roundtrip to check if the index already exists (which it usually will except on the first request). Most elasticsearch client libraries do this and i've no real idea how to avoid it since when a user deletes the index it will be automatically recreated without phabricator ever knowing... So if we don't check it might not recreate the mapping

rebaseing

One way of doing this is by adjusting the mapping for the elasticsearch index (actually way more powerful than a wildcard search).
I currently switched to the mapping below (added as a template expecting the index to be named 'phabricator' and with an english language setting).
Feel free to change the min/max ngrams setting (e.g. 4 instead of 3 letter ngrams) if you get too many / few results.
If you like to switch to another language the docs to do so are here: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/analysis-lang-analyzer.html
For it to take effect you have to delete the index and then reindex all objects (bin/index --all).

• do not use array_merge

ah ok i see.
I'll just change it to a loop. I guess that also makes it safe if someone reuses that code.
And the difference is minimal

libphutil sure has some very nice utility functions :D

• simplify code

My dev env is up2date.
But you're right that all the tasks are assigned to one user and that causes the problem within loadStatus on every page.
However as far as i can see in the reports the getDestinationPHIDs is called for every user there is and therefore causes the same problem there.
I've rearranged the way getDestinationPHIDs works a bit and it went from 60s to a 2s load time (for loadStatus as well). Which is still not great but might be acceptable for a reports page or a user who has 10k assigned tasks.

Ok, i've created ~10k tasks in my dev installation to test this more thoroughly.
I still see the full table scan on the edge table (tested on mysql 5.1 & 5.6). E.g.:

If the reports feature is what the user expects to find or is helpful is a whole different question.

The attached revision will get all tasks marked as duplicate in the maniphest db and then tries to find the task it was merged into in the elasticsearch just using the title.
By default it will use the standard analyzed field in elasticsearch as it is right now.
If you run installmapping and reindex the tasks the results should get better because it will utilize english stopwords and stemming.
Let's see what this yields. If it still find too less results we can try to combine an ngrams / edge_ngrams analyzer into it.

In ManiphestTaskQuery it says:

// TODO: Eventually, we should make this optional and introduce a
// needProjectPHIDs() method, but for now there's a lot of code which
// assumes the data is always populated.

We have the same problem with more casual submitters.
I could think of a fairly easy but powerful implementation of this when elasticsearch as a backend is used.
The custom scoring and basic language processing stuff is really nice and would allow for things like listing bugs within the same project before others quite easily.

We had a similar problem (tracking feature branches and triggering a herald rule when a commit shows up in master) and solved it by specifically looking for the merge commit in the master/production branch.
Depending on what your herald rule does it might be necessary to apply an action to every commit contained in a merge commit. I'm not sure what happens if you trigger an audit on the merge commit. It might include more commits than what you want.

• replaced some code with an already existing conduit call