Page MenuHomePhabricator

Plan the mid-term pathway for unit-test/linter bindings
Open, NormalPublic

Assigned To
Authored By
epriestley
Dec 22 2015, 12:44 PM
Referenced Files
None
Tokens
"Love" token, awarded by xiaogaozi."Burninate" token, awarded by nmcalabroso."Love" token, awarded by NameFILIP."Love" token, awarded by cspeckmim."Party Time" token, awarded by stevex."Party Time" token, awarded by chad."Like" token, awarded by featherless."Evil Spooky Haunted Tree" token, awarded by turadg.

Description

We have a number of open requests to bring various lint bindings upstream.

Past / Short Term: Currently, dealing with these is a very low priority for me. When I've spent time on them in the past it has almost always been unproductive (i.e., many change requests -- this is at least partially our fault, because writing lint bindings properly is currently too difficult and insufficiently documented), and when we've accepted linters they frequently bring substantial maintenance costs along with them (e.g., see all the broken linters in the table below, which we are largely now solely responsible for maintaining).

Future: The future is that we stop bringing these upstream and offer a packaging/distribution system instead, described in T5055. Additionally, we would:

  • fix the issues which make writing linters unnecessarily difficult, technically (e.g., severity/code handling, version handling);
  • provide guidance on how to write a proper linter (e.g., how to write tests);
  • probably offer some sort of advisory "yeah, we looked at this, it won't FTP all your stuff to foreign states" signing program.

However, this is likely far away. It is complex, it will take a lot of time to get right, and there isn't a great deal of user demand for it.

I'm not sure what we should do with these linters in the mid-term.

A pathway you can take today is to add your linter to Community Resources, and plan to bring it upstream as a real package once that is available. However, installing/managing community linters is difficult and cumbersome, and easier management is presumably the major benefit authors seek by upstreaming linters.


Architecture Problems

  • Lint codes / severities are difficult to get right (T3914, T8474).
  • Version requirements are difficult to get right (T9795).

Maintenance Issues

LinterStateDescription
cppcheckMinor IssuesT9734
cpplintBrokenT9957, T8404, T11719
xunitMinor IssuesT7653
phpcsMajor IssuesT9047, T7170,
cslintDeprecated/BrokenT8978, T8922, T4483
lessMinor IssuesT8506
pythonMinor IssuesT10237
composerBrokenhttps://discourse.phabricator-community.org/t/208
puppetlintBrokenT12327

New Linter Requests

LinterStateDescription
"Java"Requested / PatchT9886, D14632
ElixirPatchT9796, D14430
ESLintPatchT9785, D12198
clang-format-linterRequestedT9870
flowtypeRequestedT9262
PHPDCDRequestedT8495
PHPMDRequestedT8492
Perl::CriticRequestedT3185
FoodcriticRequestedT7780
ansible-lintPatchT7293, D14662
GolangPatchesT6867, D12620,
HackRequestedT4701
ScalastylePatch, No TaskD14145
XUnit.netPatches?D12067, D12156
ClosurePatches?D10690
ShellCheckPatchD11657

Revisions and Commits

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

It seems, this task/problem has truly no priority, as @epriestley mentioned in his task description already. After skimming through related T5055, that idea reminded me of https://aur.archlinux.org/.

So, I'll follow the route suggested now: Community Resources.

Thanks @avivey for sorting me in here.

@rscircus - this task (like I assume most tasks here) needs paid prioritization to move forward. see https://secure.phabricator.com/w/prioritization/

@rscircus - this task (like I assume most tasks here) needs paid prioritization to move forward. see https://secure.phabricator.com/w/prioritization/

Oh really? Nice to hear, that I'll get monetary appreciation for contributing. 😉

Do you see there being a "Blessed" foo-linter (maybe as a repo on this instance?), or will there be multiple foo-tributes fending themselves in the wilds of github?

This is all still pretty early, but I imagine this instance mostly just collecting packages into a directory for now. The actual source might be on GitHub, but you'd be able to search this instance to find all foo-linter extensions by authors who had bothered to register their packages.

In some cases, we'd likely pull linters out of arc and turn them into Phacility-Published extensions (secure.phabricator.com/phacility/foo-linter) which we'd sign. These would be "blessed", as long as you continue trusting our signing keys.

In some cases, I'd like to "soft-bless" extensions as "safe, but don't come to us for support" with some sort of "Community Badge of Quality" key. For example, you might publish a secure.phabricator.com/cburroughs/foo-linter) which we'd sign in an advisory way which mean "we looked at the code, it seems fine, but we aren't taking bug reports for it". The UI and arc install would show users that someone they trust also trusts this code.

In some cases, untamed wilds, but collected in one place and searchable, with signing and a consistent toolset.

If nothing is blessed, you may have to make a decision like this yourself:

$ arc install foo-lint
Query "foo-lint" matches multiple packages:

  secure.phabricator.com/alice/foo-lint
  secure.phabricator.com/bob/foo-lint

None of these packages or vendors are signed by anything you trust.

Which package do you want to install?

But I think that should be OK. And we can do some level of positive and negative curation (advisory signing on good packages, disabling bad packages)..

You'll also be able to have an internal package directory for your company's stuff, so you can just publish internal things to it and then everyone can use standard tools to manage extensions.

Some day far in the future I could imagine the package directory integrating with other applications (e.g., giving each package its own repositories and tasks, like GitHub) but that's definitely far in the future. I expect most extensions (like linter bindings) to be small so they wouldn't get much benefit (e.g., 100 lines of code updated a couple times a year) and large extensions (eventually, third-party applications) to probably want to self-host on their own Phabricator instances in most cases.

The immediate goal is just fixing the problem that distributing extensions is a huge pain and all solutions have major downsides, with "convince the upstream to accept the extension" being the best option for everyone but us. The new distribution mechanism would be:

$ arc lint
Okay, automatically installing all the extensions this project needs.

(...prompts if you need to trust new stuff...)

Done!
...
nice lint output

Which fixes distribution. In the long run, there's a lot we can do to build stronger ecosystem tools, but I'm less worried about tackling that stuff today than I am about moving distribution to a broadly reasonable mechanism.

I'm a fan of the way that Puppet has handled this on the Puppet Forge. Basically, there are [[https://forge.puppet.com/puppetlabs | puppetlabs]] modules, which are officially supported, and then there are also [[https://forge.puppet.com/puppet | puppet]] modules, which are maintained by the "Puppet Community" (although I'm not sure exactly who comprises this community). Typically, it seems that popular modules will be "taken over" (after consultation with the current maintainers of the module) by the community.

@epriestley I like the work going on to externalize the linters, and I understand why you would merge the tasks.

Will you accept a new differential fixing the current linters, or are they on hold until the code gets pulled out?

This is all frozen until we sort it out more generally.

avivey renamed this task from Plan the mid-term pathway for linter bindings to Plan the mid-term pathway for unit-test/linter bindings.Jun 1 2017, 7:48 PM