Let's just take another look at this during/after T10448? I think that's happening soon enough and this is minor enough that it's probably not worth fixing separately.

Thanks @pooyajavan for your update. But in your solution, the Blueprint is cloning the repository, which I don't want to do that just for running a simple cmd.

Hi I have found the solution, added in stackoverflow, forgot to post it here again.

I have exactly the same problem. Any update regarding this issue?

For phabricator.wikimedia.org, we decided to go with elasticsearch, partially because we already have a massive elasticsearch cluster and a lot of institutional elasticsearch knowledge / experience. My opinion is that we made the right choice. I believe this opinion is shared by most of the folks who use our phabricator on a daily basis. I've seen zero complaints about search since we made the switch, which is a huge improvement from what I saw with mysql FTS. Conclusion: Elasticsearch seems to perform well and the results are generally better (obviously this is subjective, but like I said, no complaints from users)

My install hit this today: we're trying to upgrade from http -> https, but don't want to break users with the old http:// address in their .arcconfig (either because they haven't pulled master recently, are working on a different branch, are bisecting to a commit prior to the .arcconfig being updated, etc).

thanks. I will ask there.

The link is fixed.

Inbound mail seems to be working now.

This is very minor, but blue-underneathie is still active even though repo012 is dead, which is causing the queue to spawn impossible backup jobs that sit there retrying forever. They should fix themselves once the instance is suspended. I'll likely pursue the remaining cleanup here shortly since we haven't seen any issues in a few weeks.

Inbound mail later bounced. I think the issue is that we have a *.phacility.com MX record, but an admin.phacility.com A record. I can't recite all the wildcard vs nonwildcard DNS rules offhand, but I think the A record is shadowing the MX record, e.g. dig mx admin.phacility.com does not produce the right results. I've added an admin.phacilty.com MX record which might sort things out once it propagates.

The new paid support application is now in closed beta. This will become publicly available in the relatively near future, then become then channel for SAAS support after that. It may later become the channel for some types of free/community support (most likely bug reports, per above) but this probably won't happen for a while.

Community and us can both use the normal task form to report bugs -- the "bug report" form just had extra instructions.

OK I made a basic bug form for Discourse. Will update docs later today.

I had assumed the forms here would still be open for Community, at least bug reporting or us.

That is, the flow would be one of these:

Let's just do Discourse for now? I don't plan to get anything up on admin for at least a while.

Using @mentions in issue comments hits a policy exception because of the slightly odd way pact policy filtering works.

This works now: PHI1

Test mail from the console now gets routed/accepted properly but real mail still doesn't. I'm going to write this off as possibly being a route propagation issue for a little bit and see if it fixes itself.

Email replies don't actually work since mail gets routed to the web tier (mail.phacililty.com) which can't hit admin. I think I can fix this with a mail route.

Some stuff I'm thinking of doing in the short term:

This is entering some kind of closed beta shortly, I'll follow up in T12904 for actual feature improvements to the Support application.

Thanks @avivey for your info.

Since the History is now dumbed down and separated from the Technical details on the Graph view, it might be more useful to filter out merge commits from the History view. Perhaps they can be replaced by a small "Merged to <branch>" link at the bottom of the box for the last non-merge commit that was part of the merge.

Ah, that makes sense. Everything else sounds entirely reasonable to me on the "VPC per customer" plan.

In T12847#229598, @epriestley wrote:

bastion host per subnet

Is "subnet" a typo or am I thinking of the wrong subnets? Not 4 bastions per customer, right? Don't the bastions have to be in the customer VPCs?

bastion host per subnet

So having done a bunch more reading/thinking about private cluster isolation, I'm starting to lean towards "one VPC per customer per region" instead of "one subnet per customer per region".

I think we end up more consistent overall if "Resign" and "Accept" have the same behavior in this case.

That sounds less surprising to me, in the sense that it's not an explicit reversal of having requested changes. I can imagine arguments in both directions for the resign case.

From reading the code, resigning (instead of accepting) probably also leaves the revision in "Needs Revision".

In T12896#228825, @jmeador wrote:

We"solved" the problem with a small python daemon that pings repository_statusmessage.epoch every minute and calculates the delta. When these deltas surpass Phabricator's max delta (21,600 sec) we page the oncall. For repositories that are mission critical, we use a smaller value. It obviously isn't a perfect solution, but it gets the job done and it's been extremely stable.

I've seen complaints about this on phabricator.wikimedia.org as well, so I can confirm that this is an issue.

O.O

In T12847#229548, @joshuaspence wrote:

AWS doesn't have a single device which can both listen for TCP on 22 (only ELB) and terminate SSL for websockets (only ALB).

That's not correct. We are routing our websocket traffic for Aphlict through an ELB that has been setup using SSL instead of TCP as the protocol. In Terraform, it looks like this:

It looks like the price difference is pretty large -- $45/month for a m4.large reserved instance vs $80/month for a m4.large dedicated -- plus ~$1,500/month/region for having at least one dedicated instance. It would be easy to start using dedicated hosts in the future, right? Just an issue of converting existing hosts if we run into issues?

We should also make a decision about whether or not we want to use "dedicated" AWS instances: https://aws.amazon.com/ec2/purchasing-options/dedicated-instances/

Another correction, Puppet doesn't //need/ a dedicated server. You can run Puppet in a standalone (agent) mode, if you are willing to somehow get your Puppet manifests onto each host that you wish to provision.

In T12856#229564, @joshuaspence wrote:

In T12856#227621, @amckinley wrote:

Terraform Review
Pros:

• Written in Python, which everyone knows is superior to Ruby in every way

Huh? Terraform is written in Golang, not Python.

In T12856#227621, @amckinley wrote:

Terraform Review
Pros:

• Written in Python, which everyone knows is superior to Ruby in every way

A customer is hitting an issue where pushes to staging areas do not push LFS objects. I'd guess this is an LFS upstream problem based on a hazy recollection of events here.

FWIW, I am actively working on a Puppet module for Phabricator, see https://forge.puppet.com/joshuaspence/phabricator.

AWS doesn't have a single device which can both listen for TCP on 22 (only ELB) and terminate SSL for websockets (only ALB).

Also, have you tried looking around the git community? I was always under the impression that SSH should be faster than HTTPS cloning, or possibly using the same protocol.