Thanks for that confirmation. I like the look of your code comment: "Possibly we should white-list this kind of mail and deny all other types of mail." I'll create an enhancement request task related to that.

is Phab supposed to send email to users who are unverified?

Re scenario 1, I found another way to investigate it rather than digging up old logs. Question, though: is Phab supposed to send email to users who are unverified? Looking at mail logs, it seems one of our most recently added users was sent quite a lot of mail, perhaps all mail they would ordinarily get, prior to verification. If Phab is supposed to send mail to unverified users, then this would be an enhancement request which I could submit with a proper description of the use case. If not, I can investigate how to reproduce the behaviour.

Most of the work is disk/DB bound, so not having a high cpu load makes sense. That said, processing a lot of tasks will take a while.
When a task fails in an unexpected way, the daemon thread backs-off for a while, so it takes much longer to fail a task than to complete it; Unless most of the commits are failing, it should speed up at some point.
These tasks are also lower-priority than Herald and Publish tasks, which is why those get executed before these; Once these tasks are out of the queue, expect work to happen faster on the commit parsing tasks.

First, thank you for taking the time to answer...

Are the daemons doing anything? Are any tasks, at all, getting done? Are some tasks failing?

Sorry, I should have been more clear..

./bin/phd start will start all the right daemons. Look in the Daemons app in the web interface to see what's going on.

Nope, phd is definitely *not* making any progress on the PhabricatorRepositoryGitCommitMessageParserWorker queued tasks I'm stumped. When I run this:

Yes.. interesting, I had not considered that. I'll re-activate the repo

are the daemons running? You might need to activate the repository for the commits to be parsed.

One other thing that I should note, these commits are all in a repository that is no longer active. (if that matters)

OK, Here's the issue (from what I can tell)
PhabricatorRepositoryGitCommitMessageParserWorker has a count of 105,000 in the queue, but for some reason those commits are not getting parsed. I don't want to 're-queue' them by removing the --force-local, because (according to the help) that will duplicate the work that is already queued. How can I get phd to start working on those commits?

Will try.. Thank you.

I believe that if you drop --force-local, the reparse code will schedule the reparse to run using the daemons, which should fail to parse this commit and continue to the next one.

The task for parsing this commit should auto-cancel because it encountered a PhabricatorWorkerPermanentFailureException. In this case, you're manually re-parsing it, but it shouldn't be happening automatically.

What can I do next to get the commit parser to stop trying to parse this.

Related thought: I wonder how the sticky task order fares when a task is on multiple workboards; since the subpriority seems to be a task parameter, not a workboard card parameter, I guess moving it on one workboard may clobber its position on another; quite possibly not a problem in practice.

What's unclear?

Hmm yeah, we record where in the workboard card is in the list, sticky. We could probably ignore that transaction.

OK. The type of the other transaction is subpriority. From the database:

Initialisation doesn't seem to fit the pattern, as I would have expected an earlier column change to trigger it if that would. There are also over a dozen transactions on the timeline of this task before the section I pasted here, including comments (one deleted; I wonder if that's relevant), column changes, reassignments, and status changes.

There might be some crazy edge case of empty custom field that was previously hidden in a custom form and is being initialized for the first time, but not generating a visible transaction, so is getting lumped under "other". Hard to say, I can't really recreate how this might happen. Do you have "other" email sent to "on" but everything else "off" ? Knowing those would be important in a reproduction case as well.

Yes, both. I don't see how either of those could be involved with a column change, though, could it? Changes to our custom field do show on the timeline, too, but nothing is showing here.

Do you have custom fields or custom forms?

I wouldn't call this resolved. Invalid may be most relevant, as with no STR, we can't confirm or action this.

I believe this is now in reasonable shape, in the sense that the overwhelming majority of prose inputs I'm aware of produce reasonable, human-readable diffs. They aren't perfect in all cases and aren't necessarily the diffs that a human would produce, but they're useful and fairly sensible.

unit tests ftw

That seemed to just work? Slightly disconcerting.

Pretty sure this works as intended

Look at how much engagement we're generating!

comments

more

need

still

I would edit my comment to put a paragraph of text above the autoplay audio, but newsfeed caches the old comment. :3

I think we can safely conclude that autoplay audio in feed is an effective mechanism for disseminating important information.

Sure seems to be a lot of activity that generates feed stories on this task!

@epriestley FYI

{auto-playing-audio WilhelmScream.mp3}

Welcome to the wonderful, spooky world of Quicksand.

Yes I did have the persistent chat pane open. Also can't seem to repro without it.

Quicksand

We still do a pretty questionable job on the first case in T7643#180264, where a comma is removed from the middle of a sentence.

Do you have the persistent chat pane open?

I suspect this is a problem with the way you've installed phabricator.

I suppose an alternate path is to enable Dashboards on Project homepages, and you can embed your project wiki or text panel for a really cool long description and information.

Would you fill out at least, like, 5 of them?

Hovercards, Search, Browse, etc. It could get used.

We could call it "Search Description" so it feels clear that it only displays in some UIs.

Yeah, I think it's pretty reasonable if people would actually fill it out. We could surface it in a few other places too (e.g., /projects/) without fallback. It just seems like there's a decent chance no one will ever fill it out and it will sort of just be cluttering up the UI. But the cost on our side is pretty small, so if some people claim that they'd probably want to fill it out I'm happy to move forward with it.

Provide an alternate, text-only field for "Project summary in typehaead results." which users only enter text into, so we don't have to worry about remarkup.

For reference, here's the current state of "discovery" downstream:

I'm going to kick this out of the queue for now and we'll reconsider it alongside other internal priorities once the queue clears. I think it's probably secondary to Conpherence now, at least, product-roadmap-wise, maybe?

I'm going to kick this out of the queue for now and we can re-evaluate our internal priorities once the queue clears -- we got a decent iteration out and it looks like the next steps on NUX are probably a fair chunk of work.

The driving install dropped off the grid a while ago, so this is no longer prioritized.

I believe this is now resolved, in the sense that I expect all specific, reproducible examples I'm aware of function in a way consistent with user expectation. In particular:

The other errors that appear in DarkConsole:

I have some possible approaches for cheating on the "more than 100 results" case, but I think we ultimately need to put the prefix paging rule on the server side, so we return all prefix matches and then all content matches. That's kind of a big mess, but all the ways we can cheat will break down at some point (e.g., when you have 100 "wikidata" projects that are alphabetically earlier than "wikidata") so I think it's not avoidable.

• Prototype applications must be enabled.
• security.outbound-blacklist must not blacklist the target server (build servers may be on an internal network and on the default blacklist).
• The link protocol must be "http" or "https".
• You must not have exceeded your fetch quota for the last hour.
• You must have file storage configured.
• The image MIME type must be part of files.image-mime-types and files.viewable-mime-types.
• The Phabricator server must be able to make outbound requests on the network.

In T4190#199608, @epriestley wrote:

(But note that it's cached for a trillion years, so it won't work for build status.)

Does the {img } thing work yet? I've been trying to use it locally but have not been able to get it to actually show an image. This would be super useful for us to be able to show the build status on a remarkup page (like the README.MD of our repo, for example). Thanks.

Pausing this from our perspective since we're waiting on guidance to move forward.

Presuming this is resolved since it hasn't been updated in about a month, but let us know if it's still an issue after the update.