Page MenuHomePhabricator

Building OS packages and install scripts
Open, WishlistPublic

Assigned To
None
Authored By
epriestley
Dec 4 2013, 7:17 PM
Referenced Files
None
Tokens
"Mountain of Wealth" token, awarded by pouyana."Like" token, awarded by MatheusVieira."Like" token, awarded by vrusinov."Mountain of Wealth" token, awarded by allan.laal."Mountain of Wealth" token, awarded by btrahan.

Description

This is a general meta-task which discusses building OS-specific install scripts, VM images, AMIs, OS packages, etc.

Overview

Although we no longer get nearly as much feedback about Phabricator being difficult to install as we once did, it isn't easy to install. Consequently, we receive requests to provide packaging, like:

  • Make an install script for <some OS>
  • Get Phabricator into the package manager for <some distro>
  • Provide a VM/Vagrant/Container/AMI
  • Support brew/composer
  • Install man pages / init.d scripts / other hooks

We may pursue these things some day, but for now they're a very low priority, primarily because:

  • Phabricator runs on a wide variety of operating systems (e.g., Debian, RedHat, FreeBSD, OSX, Solaris, some Windows support), and a wide variety of webservers (apache, nginx, lighttpd), and multiple PHP runtimes (Zend PHP, HHVM). We can not realistically build an install script which supports all of these options without monumental effort. Users have generally expressed much more interest in being able to run on obscure platforms and with specific configurations than in having a more straightforward install process.
  • We're a small team, without expertise in packaging and distribution. None of us actually know how to do many of the things on this list.
  • We're a small team with finite bandwith. Even if we were packaging experts, committing to building and maintaining this stuff would limit the bandwidth available for feature development. Although there's some level of chicken-and-egg / selection bias, we see far more demand from users for feature development than configuration simplification (and even for feature development over configuration simplification, e.g., "that feature would be awesome, it's completely fine if it's super hard to configure"). Particularly, this stuff is an ongoing investment since we need to keep it up to date as every supported OS and dependency evolves. We currently weigh the cost of building features which will requiring substantial ongoing maintenance heavily.
  • Much of Phabricator's installation process involves interacting with dependencies. We can't just wipe out the Apache config or the MySQL config on a box, since they may be used for other things, and we can't always easily extend them either, since users can change their configuration and make it stop reading sites.enabled/ or whatever.
  • Much of Phabricator's installation process is install-specific. For example, some install steps include setting up accounts, hostnames, mail, etc. There's no way anything prebuilt can do this for you, and we could make it easier only by taking over the entire system.
  • Phabricator is a very fast-moving target, and we recommend installs update frequently. It would be difficult to keep some types of packages (like distro packages) anywhere near HEAD. Phabricator also has no regular versions/releases, and our current judgement is that a continuous release is a net benefit for both the upstream and users. For example, we are often able to fix issues that users report within hours, and having all users attached to HEAD means they can pull the fixes immediately after they're available. If we had to go through a package/distro system, it could take thousands of times longer to get fixes deployed.
  • We don't have the infrastructure in place to automatically rebuild or automatically test these scripts/packages. This will improve in the future, but even once this infrastructure exists it will require a significant investment to make sure these things aren't immediately obsoleted, irrelevant or broken (see T1049).
  • We're building a SAAS offering which will provide a zero-administration option (see T1315). For the set of users who would rather not deal with any administrative stuff, this is a preferable option to packaging, since it eliminates the problem rather than mitigating it.
  • Phabricator's dependencies are mostly standard packages and almost all of the setup steps can be navigated by Googling for how to perform them. For example, if you want to write a startup script for phd for your OS, you can just Google for how to do this. Provided your OS offers a reasonable mechanism, writing this script should not be especially difficult.

The Way Forward

We may eventually pursue these things, but they're very difficult to prioritize for now because of the large surface area, high initial cost, and high maintenance costs. Focusing attention on packaging would distract from feature and application development, face significant limits in possible impact, and generally slow things down. At a minimum, we are unlikely to invest significantly in this stuff until after SAAS is built.

If you have expertise with an OS and would like to contribute something, we're happy to provide links to it, but generally don't have the bandwidth or expertise to maintain this stuff in the upstream.

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Off the top of my head: arc upgrade currently depends on git. Should we install git when the user runs arc_installer.exe?

Yes, if it's not already on the PATH. (I don't think this is a must, since git for Windows is easy to install, but it's reasonable and a nice-to-have).

Should we disable arc upgrade? Should we replace arc upgrade with some command which downloads a newer copy of the installer?

arc upgrade should still work, just like Firefox has a Windows installer, but you can just go to Help->About Firefox and click "Check for updates"

arc runs in a variety of shells on Windows, including Git Bash, cygwin, cmd.exe, and I think something else. Which of these shells should we install?

msysgit, simply because it's used by http://git-scm.com/download/win (Windows installer on official git website) and thus is probably the best-supported, or one of the best (I have used this in the past).

Should we install SVN, Git and Mercurial?

Not sure. Depends how prevalent use of each is, and whether the non-git users think this would be useful.

Should we install PHP?

Yes, I think so, if it's not already installed.

What do we do if PHP is already installed but the version is too old? What do we do if PHP is installed and the version is fine, but the configuration is wrong?
What if the configuration is wrong and the install uses a nonstandard custom configuration?

Explaining the error and aborting the install seems reasonable. That edge case shouldn't block progress on the general case.

How do we deal with TortiseSVN and Git GUIs?

It seems they can just be ignored. SVN commits are atomic (so there is no shared state to worry about), and git GUIs should simply use the index in a standard way.

Which versions of Windows should we support?

I wouldn't let this be a blocker. Start with whatever is easiest to test on. It's an open source project, so people can later investigate how/whether to expand support.

I don't think any of us have ever built a Windows installer before. How do you do that? What do we need to know to make sure we can continue upgrading in the future?

Don't know, but I've heard of various tools like http://nsis.sourceforge.net/Main_Page . You could also ask other major projects with Windows installers/look at their code.

Every problem here is tractable, they're just almost certainly not the most valuable things we could be working on.

I understand you have to prioritize, but I do agree they're tractable (hence my comments above).

I see you merged my arcanist task into this one. That's fine, but note that arcanist is several orders of magnitude easier than a complete configuration of httpd/nginix, a database and all the other things you want to put in place for a full install.

Just installing arcanist as a macport is a dramatically simpler thing.

  1. Clone git
  2. build binaries/libraries
  3. install on PATH

Link to the Sandstorm Vagrant SPK documentation - https://docs.sandstorm.io/en/latest/vagrant-spk/packaging-tutorial/

Sandstorm apps can be in any language so long as it runs on Linux (PHP, Python, Node.js, Ruby, etc.). The app bundles its dependencies so it runs in a consistent environment. Sandstorm handles user management and mitigates 95% of security issues. App authors don't have to run servers since users run your app on their own servers. Sandstorm's "grain" model allows developers to rely on Sandstorm for supporting multiple instances/documents rather than writing that code in the app.

@MatheusVieira are you asking us to support it because you're a Phabricator user?

@chad yes, and want to help who don't know how to deploy Phabricator to use Phabricator and Sandstorm is perfect to that.

I noticed you posted the exact same message that you posted here to Drupal, with the same spelling error of "Oasis":

https://www.drupal.org/node/2751853

Do you have a relationship to the Sandstorm project?

Oh, you've just been spamming tons and tons and tons of projects with the same message:

https://github.com/MVprobr

I followed up on this in #sandstorm on Freenode but this isn't officially sanctioned as far as anyone there knew. They're not particularly happy about this either, but obviously there's little anyone can do about users going vigilante with evangelism.

@epriestley no i dont have any relationship with core Sandstorm, I'm a user and a fan of both apps. I'm just a student who wants to make life easier for the end user , and you create as much repercussion about it, I want to help phabricator and all other tools that I believe I send this text.

Sorry for bad english, I'm from brazil.

Commenting here since you merged T12429. I find surprising you put Arcanist in the same bucket as Phabricator. I think you don't even have to do all the work to make Arcanist users happy. I might be wrong, but I think there will be people adding support for Arcanist to their favorite package manager (I'm ignoring Windows), if only they'd have tags or marked releases to work with?

Once arc is installed, you can just run arc update. Given the plan to also distribute linters, etc, this would still be preferable for each individual installation vs. us.

There are debian and ubuntu packages for arcanist:

although popcon data (66) suggests that the debian one is not in wide use.

These packages use the git date as revision (as in 0~git20160726-3) which is ugly.

Most projects on github use the github "releases" to attach a X.Y.Z number to a tarball so that distributions can package it meaningfully.

Semantic versioning can be used to decide when to increment the X, Y and Z to signal when an upgrade breaks interfaces / APIs / contracts.

install_ubuntu.sh seems to be completely broken for modern Ubuntu. I don't mind updating it, but I think we discussed dumping it completely?

I'm happy to bring either change upstream if you want to do the legwork. We could also move them to a wiki page or something. No option here feels particularly good to me.

Hello,

It's been several year's since this task has been opened up, and it's not clear what the current progress on this. Is there a way that versioning please be added to GitHub, per the recommendations on https://github.com/Homebrew/homebrew-php/pull/3864 ? I'm not able to install arcanist currently using homebrew because there isn't a stable tagged version newer than one from 2012.

Are there set releases for arcanist that could be used [really quickly] for tagging purposes so that I could fix the homebrew formula??

No.

(Please use Discourse for this sort of discussion.)