Can confirm that on latest stable my lfs-test repository now succeeds with both jars and jpegs. Huzzah!

T10751 and T4292 are reaching a steadier state now and have some indirect/soft dependencies on this (for example, the new panel for monitoring the cluster status of a particular repository is part of this new UI) so I expect to drive this forward at least somewhat in the next few days. I'm not sure how ambitious I'll feel about meddling with the URI code before the release cut, but there's a good chunk of basic work remaining before that becomes an issue.

I'm still not really thrilled with where the documentation situation on repositories is, but it's getting a little more manageable.

Thanks!
Yes, we will consider setting up alternate file domain

I think D15784 should fix the issue. It should land in master later today and be available in stable in about 24 hours. Let me know if you're still seeing issues once you update past that.

I think I know what's going on, although I'm unsure why the <2MB files would work, unless you happened to push those to a public ("Visible To: Public (No Login Required)") repository and pushed the large files to a private repository?

1. It's not configured
2. Yes
3. I'm not sure, these are my install versions (stable branch) :
   • phabricator f75b1cf562c0c3646324864851d693ef1069a068 (Mon, Apr 18)
   • arcanist 01b6fe8bb0239a4bce03f58d6288a3a52ad83a91 (Sat, Apr 16)
   • phutil b8c65df2a910a7028d39bec602d181428b6ce01f (Sat, Apr 16)

(edit, it has rP37b93f4)

• Is security.alternate-file-domain configured on your install?
• When you say "redirect to the correct file", do you mean you get taken to the file detail page /F1234?
• Also, do you have rP37b93f4 locally (April 7)? (You can grab the hashes in Config → Versions if you aren't immediately sure and I can figure it out from there.)

Ah, is the auth token (first [shortened]) same for every user? It is in my case

1. If I visit the URI while being logged in to the app, the browser redirects to the correct file, the download link works
2. The download URI is the same as the logged URI
3. Visiting the URI with private window returns normal 404 page

You shouldn't need to configure diffusion.allow-http-auth -- the auth token is basically the first [shortened] in the URI, so that appears to be working properly.

I'm using S3 as Files storage backend, phabricator runs behind nginx, and the diffusion is hosted over SSH (diffusion.allow-http-auth=false, HTTP is not configured anyway). I'm having an issue:

• pushing and pulling small files (<2MB) tracked as LFS -> works flawlessly
• pushing 11x 1 GB files works, but can't do git clone / git lfs fetch (got the same issue as @benjumanji, the lfs_trace log is similar, 404 errors when getting the lfs-* files)
• pushing a 11GB file, doesn't work. forgot the error message, will add it later if needed

Mixed progress. Repository clustering has been stable in production since the last update, and appears to actually work, which is good. We've hit a few minor snags (like a missing sync-before-read in diffusion.querycommits) but all pretty much expected stuff and nothing troubling.

Just more JavaScript.

Is adding a new project tag and "move on workboard" in the same set of stacked action a minor extension, or super hard?

secure is now running repositories in fully redundant multi-master mode. This appears to be working, although it has been live for about 20 minutes and survived about three pushes so far so I'm not yet brimming with confidence about it.

This got some progress over the weekend -- it dovetailed a bit with T4292, and I wanted to let the Phacility cluster sit on the new availability stuff for a little while. Upshot:

Just some cleanup today. New daemon console now shows which hosts daemons are running on:

A big chunk of this is finally in production and seems to be working, which I'm thrilled about.

For design reference, here's the GitHub version of this mail (each comment arrives in a separate mail):

I slightly reduced the amount of header magic, and the notification stuff is now cluster-ready (in theory, at least). I'm planning to get notifications, web nodes, and databases redundant in production today, and maybe repositories/daemons if nothing else comes up. No issues with repo bookkeeping so far but even if it's smooth sailing it needs a fair bit of UI work, and same with the daemons.

I've upgraded our instance now this has hit stable, and our team is thrilled. The interface may not be perfect, but it satisfies the vast, vast majority of our use cases. Thank you!

Given current progress and velocity, I'm aiming for a timeline roughly like this:

My example for this problem was with Maniphest tasks in T10790, where identical task titles is problematic in the search results.

I've started moving these changes into production (T10784) so I can begin gathering confidence that they actually work. The repository stuff definitely still doesn't work since several core pieces are currently implemented as // TODO comments, but it has the synchronization bookkeeping in place for all the locking/versioning: if that's stable, it will give me confidence that we're on relatively firm ground and can move forward.

Progress so far:

I'm probably done for the night, here's a quick update:

Hmm, odd. That definitely isn't the POST issue (although I suspect that may be an issue in some cases). The behavior will hopefully be more clear once master promotes, and definitely easier to debug. I'll dig into it a bit on my side in the meantime, I have a few ideas of things to try now. Thanks for your help!

Unfortunately, no dice. I removed the jar file, and added a jpeg. I have attached a trace of running GIT_TRACE=1 git lfs fetch origin.

Oh, I think I might know what's wrong. If you repeat the same process with a viewable file (like a JPEG or PNG) does that work?

Can confirm that navigating to the file in diffusion will let me download it, and it really does give me back the right file.

Hmm, I'm not immediately sure -- that all looks correct on your side to me. Let me add some kind of basic LFS administration page that shows Phabricator's view of LFS objects in the repository -- hopefully that will shed some light on where things are breaking down.

As an addendum, I can definitely find the zip file on the box...

I'm on latest stable (fea2389), and I just tried the following

$ git clone ssh://git@phabricator.example.com:222/diffusion/18/lfs-test.git                                                                     [10:54:36]
Cloning into 'lfs-test'...
warning: You appear to have cloned an empty repository.
Checking connectivity... done.                                                                                                                       
$ cd lfs-test/                                                                                                                                  [10:56:39]
$ git lfs install                                                                                                                               [10:56:42]
Updated pre-push hook.
Git LFS initialized.                                                                                                         
$ git lfs track "*.jar"                                                                                                                         [10:57:05]
Tracking *.jar                                                                                                  
$ cp ../some/path/antlr-3.2.jar .                                                                                                               [10:57:19]
$ ls                                                                                                                                            [10:57:52]
antlr-3.2.jar                                                                                                       
$ git add antlr-3.2.jar                                                                                                                         [10:57:54]
$ git commit -m "testing"                                                                                                                       [10:57:57]
[master (root-commit) 2690d02] testing
 1 file changed, 3 insertions(+)
 create mode 100644 antlr-3.2.jar
$ git push -u origin master                                                                                                                     [10:58:04]
Git LFS: (1 of 1 files) 1.84 MB / 1.84 MB                                                                                                                
Counting objects: 3, done.
Delta compression using up to 6 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 329 bytes | 0 bytes/s, done.
Total 3 (delta 0), reused 0 (delta 0)
To ssh://git@phabricator.example.com:222/diffusion/18/lfs-test.git
 * [new branch]      master -> master
Branch master set up to track remote branch master from origin.

so far so good. however...

$ cd ..                                                                                                                                         [10:58:18]
$ rm -rf lfs-test/                                                                                                                              [10:59:45]
$ git clone ssh://git@phabricator.example.com:222/diffusion/18/lfs-test.git                                                                     [10:59:50]
Cloning into 'lfs-test'...
remote: Counting objects: 3, done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (3/3), done.
Checking connectivity... done.                                                                                                                     
$ cd lfs-test/                                                                                                                                  [11:00:15]
$ ls                                                                                                                                            [11:00:19]
antlr-3.2.jar
$ cat antlr-3.2.jar                                                                                                                             [11:00:45]
version https://git-lfs.github.com/spec/v1
oid sha256:4c8737014e7ca0d2c85171edf37f5a26b2d8d8237c283357b81a3269b6848d38
size 1928009                                                                                         
$ git lfs fetch origin                                                                                                                          [11:05:19]
Fetching master
Git LFS: (2 of 1 files) 0 B / 1.84 MB                                                                                                                    
Repository or object not found: http://phabricator.example.com/file/data/w5mihitt7ijile3b7n44/PHID-FILE-kbhl75cnfm6nwks7ojmf/dosd5iqevvn3rune/lfs-4c8737014e7ca0d2c85171edf37f5a26b2d8d8237c283357b81a3269b6848d38
Check that it exists and that you have proper access to it
Warning: errors occurred                 

Alright, a v0 of this is live here now -- "Move on Workboard" in the "Actions" menu above the comment box. This is a rough cut and we may not keep it in this form, but the infrastructure it needs is built now, at least.