Page MenuHomePhabricator

Add Support For User Definable Macro Or Template Markup to Phriction Similar To WikiMedia Templates
Closed, DuplicatePublic

Description

The Problem

Phriction and Remarkup throughout Phabricator offers very good integration with bundled programs. You can shortcut links to differentials, task tickets, changesets and more. The linking to external resources is not as powerful or as feature rich as you are often limited to basic static external links.

This may in part be by design. Rich internal integration and lesser external encourages the centralised use of the tool. Rather than things splintering off into a dozen platforms for a dozen tools the Phabricator suit does a great job of providing a good single place to come to cooperate.

A commonly used external link might to an RFC definition. On day one you may wish to encourage users to always link to the text version of an RFC as hosted on the ieft.org's website. Presently the only way to make that link is to enter the text [[ https://tools.ietf.org/rfc/rfc123.txt | RFC123 ]] and update both the label and the link number every time a new RFC is used. What if in the future:

  • you want to include both the Text and HTML versions of RFCs, you must update every instance on every page separately.
  • The ietf.org might change their URL to https://rfc.ietf.org/rfc123.txt, you must update every instance on every page separately.

What if instead of writing a seperate link for an RFC on every page a user could define a new tag, say {{rfc|123}} that would automatically be translated into the [[ https://tools.ietf.org/rfc/rfc123.txt | RFC123 ]] markup when the page is rendered? If this new tag is defined in a single place and is avaiable throughout your instance it becomes trivial to edit it to point at a new URL or produce multiple links.

Other possibly common external link scenarios:

  • You move to Phabricator and have a legacy bug system full of old resolved bugs that you would like to link to simply using {{bugz|123}} instead of [[ https://bugzilla.acme.biz/show_bug.cgi?id=12345 | Legacy Bug 12345 ]]. Yes you could migrate them but is there value in that?
  • You have a database of test results that you commonly refer to.
  • You often do inter-wiki linking to something like Wikipedia

This same system could take no parameters and allow the generation of boiler plate. You might want the same "Introduction Of Product Widget 2000" section at the start of a summary page, how to write code page and hardware design page. If you put that into a Macro it could be included in all three pages and only edited in one place. That or you could transclude pages or sections of pages but that is a different kettle of fish.

The Current Solution

When I asked in chat about whether a template or macro like feature existed, @avivey responded with a plausible solution for those the can meddle about with the server side code. For a few simple cases this is a workable answer that I will in all likelihood use. It is much more limited than allowing users to generate their own macros as and when they find a need to though.


avivey 20:55
@TafThorne: You can write your own Remarkup syntax rule like this:

<?php

final class RFCRemarkupCodeRule extends PhabricatorRemarkupCustomInlineRule {

  public function apply($text) {
    return preg_replace_callback(
      '(RFC(\d*?)\b)s',
      array($this, 'markupInlineRFC'),
      $text);
  }

  public function markupInlineRFC($matches) {
    $engine = $this->getEngine();

    $id = $matches[1];

    $link = sprintf('[[https://tools.ietf.org/html/rfc%s | RFC %s]]', $id, $id);
    $link = phutil_tag(
      'a',
      array(
        'href' => sprintf('https://tools.ietf.org/html/rfc%s', $id),
        'target' => '_blank',
      ),
      sprintf('RFC %s', $id));

    return $engine->storeText($link);
  }

}

(Dump it in the src/extensions dir)


MediaWiki Template Behaviour

I used a MediaWiki for a number of years and came to really appreciate the power that their Templates feature. This makes generating some boilerplate (like a copyright notice or page footer with helpful links) as simple as creating a new article and adding a single line of markup to the target page.

Within MediaWiki the Templates feature allows you to insert boiler plate markup using a pseudonym such as {{foo}}. This is a user definable entry that causes instances of {{foo}} on a page to be replaced by markup text such as A simple example bit of _data_ to fill a space.

More complex templates can be created by nesting one template inside another. A call to {{foo}} might include a call to {{bar}} within it and so on. This allowed you to form complex bits of text from simple lumps.

The really interesting feature of templates is in using parameters with them. Ideally this is what I would like to use within Phriction, if possible. A template with a parameter is called by something like {{foo|var1}}. Now inside the text you can insert the 'var1' into the generated text to give you something such as var1 is a simple example bit of _data_ to fill a space.

What you really use this for is something like an RFC link template. You can add {{rfc|123}} to a page then have the template generate some markup like [[ https://tools.ietf.org/rfc/rfc123.txt | RFC123 ]]. What is the advantage of this? Well perhaps you decide next week that you want all your RFC's to reference the copies on https://www.rfc-editor.org/ instead. Or you decide that you want links to both the .txt on ietf.org and rfc-editor.org. By using the template you can create a single article to edit in the future to make a consistent view across your wiki.

Event Timeline

I do not think this is a duplicate. T3963 is about creating template Wiki pages that start off ready populated with some basic data. This request is about creating a Macro like tag (which is part of the Templates feature in WikiMedia markup) not about creating pre-canned pages.

@epriestley Please could you review my statement and re-open this request if you agree with my reading of the two requests.

There is basically zero chance we're going to implement parameterized templates which can call one another (easily dozens of hours of work) if the only real use case is making RFC5555 autolink (works today with InlineRule, takes a few minutes to make work).

This task just barely follows the guidance in Contributing Feature Requests, and specifically Describing Root Problems. The problems here are all about linking to objects in remote systems, but the bulk of the task focuses on a very specific solution with enormous complexity.

I've merged this into T3963 (which discusses both "templates as functions" and "templates as a way to make copy-paste easier"), but the short answer here is "no, we're not ever going to spend weeks of effort building a parameterized/functional templating system just to make it easier to link RFC5555".

See also Doorkeeper (and, e.g., T10538) for representing remote objects in local systems. Doing this formally (via Doorkeeper) rather than informally (e.g., via a templating system) when the remote object is a bug/issue provides a substantial amount of value.

See also Phurl for defining modifiable, named links with remarkup syntax like ((rfc5555)).

I gave a trivial example as that is better than a complicated one. In a huge wiki, like Wikipedia templates are used for all kinds of things besides making links simpler.

I thought I did a reasonable job of describing the root problem. You link to T10538 which basically says "we want more growth" as the root problem.

  • Extending the markup is hard or impossible
  • link syntax is long and complex for non-wiki items
  • users are forced to copy paste a lot which is error prone

the bulk of the task focuses on a very specific solution with enormous complexity.

Really? It is that complex to have a system that would allow {{RFC|123}} to map to a link with fex based on one parameter. I did not even request the ability to have {{RFC_Text|123}} and {{RFC_HTML|123}} be called by {{RFC|123}} so that users could compose sub-compents of their call. Yes that would be a nice to have but it seemed a harder job to allow for calls N templates deep.

Should I just have said "Please implement a feature exactly like Wikimedia Templates to solve..." then listed every time they are ever helpful?

I read the contributing feature requests page and attempted to comply with it. I knew there was no obligation for it to ever be implemented. I know people can sponsor features or possibly contribute towards them on GitHub. Your response comes across as rude is going to just guarantee I do neither of those. You are not an open organisation looking for contributors and maybe you don't care about small customers either but there is still such a thing as common courtesy.

Doorkeeper or Phurl might do for the RFC use case though. Thank you for suggestions of those.

I gave a trivial example as that is better than a complicated one.

Trivial examples aren't useful in arguing for features.

We prioritize new features mostly based on the value we think they'll provide by solving real problems that users encounter and saving real users time during day-to-day development activities. Trivial or theoretical examples don't support feature value. The only useful examples are concrete problems which you actually encounter day-to-day.

Should I just have said "Please implement a feature exactly like Wikimedia Templates to solve..." then listed every time they are ever helpful?

No. You should list the (actual, real, day-to-day, not theoretical) things you want to be able to do that you currently can not, or which you do at significant time or complexity cost through cumbersome workarounds. Optionally, you might also say "in other cases, we currently use Wikimedia templates to solve these". Suggesting a solution is the least important part of a feature request.

We can then suggest alternate approaches (like Phurl), contextualize the requests ("we'd prefer to use Doorkeeper for this in the long run, but here's a workaround until then"), consolidate similar use cases, and plan and eventually develop a solution.

If you do suggest solutions, you should weigh the complexity of what you're asking against the value it would provide to you and other installs. If you are asking us to spend weeks saving you a few seconds, you should expect that it will be very difficult for us to ever prioritize solving your problem.

map to a link with fex based on one parameter

The total lifetime cost of implementing a feature poorly is almost always much higher than the total lifetime cost of implementing it well. We bear the lifetime cost of everything we ship, and thus try to never ship bad implementations of anything. If we ship a bad implementation, we'll end up back here in a couple of months with some other user wanting two parameters, or variables, or template composition, or reporting a bug with the hacky mess we implemented as quickly as possible just to get some approximation of this feature out the door. Worse, we may need to migrate or provide legacy support for existing installs that used the bad version of the feature. In six months, we'll already have spent more time dealing with the mess we created than we would have spent just doing it right in the first place. We were less strict about this earlier in the lifetime of the product, and have paid for (and continue to pay for) these decisions many times over. We hold ourselves to these same standards; see e.g. T12270 and D17360#207247 for a long list of scalability and correctness issues we resolved before recently unprototyping the Badges application. If you ask us to implement a feature, you should weight the cost to implement, support, and maintain it over the coming decades, because this is what you're requesting.

possibly contribute towards them on GitHub

You can not; see Contributing Code.

Your response comes across as rude

Yes, this is common. Often, users would much rather we not respond than say "no" immediately. While I'm somewhat uncomfortable with changing our policy to "lie to users" (e.g., "Thanks for your thoughtful suggestion! We'll consider it.") which I think is the normal policy of larger companies, we do plan to reduce our responsiveness in the future to improve this. See T12134. See also T11597. Previously, see T9212.

You are not an open organisation looking for contributors

This is correct in some sense, and spelled out in more detail in Contributing Code.

maybe you don't care about small customers either

If you're a customer (that is, you pay us money for something), please shoot us an email with details at support@phacility.com. This request channel is intended for free users, not customers; customers enjoy dramatically broader latitude in getting support and help.

If you don't pay us for anything, please respect that you're asking us for our time, which is in very limited supply, and competing against thousands and thousands of other requests and installs in similar positions. If you want your requests addressed, they need to be easier and more valuable to address than the thousands of other similar requests.

No. You should list the (actual, real, day-to-day, not theoretical) things you want to be able to do that you currently can not, or which you do at significant time or complexity cost through cumbersome workarounds. Optionally, you might also say "in other cases, we currently use Wikimedia templates to solve these". Suggesting a solution is the least important part of a feature request.

I can go though these on this thread if you like but I will not if you feel it is just going to add more noise to a dead end. Unfortunately I will be working from three year old memory as the last system I was using is Trac which is rather limited in wiki features (Phabricator seems much better so far). I am knew to Phabricator and Phriction so trying to work out what it can or cannot do.

If you do suggest solutions, you should weigh the complexity of what you're asking against the value it would provide to you and other installs.

I cannot really weigh up the complexity of implementing something in your code base. I mentioned the mediawiki template interface as that is a tool I know solves a similar job. I needed something to make the clear point about short-handing sections of markup, rather than starting a new page with lots of markup in it as template is a fairly loose meaning.

If you ask us to implement a feature, you should weight the cost to implement, support, and maintain it over the coming decades, because this is what you're requesting.

Again, I cannot possibly do what you are asking. I can merely request a feature. You can easily assess it as quite big and dozens to hundreds of hours work.

Thank you for the links to Contributing Code. If I notice some bugs that I think I can fix I will consider doing that after I file a report. So far I think the only oddities I have noticed are config errors our end and some odd bit of mailto: link handling needed exact spaces (probably one of the earlier bits of implementation you menio, its an odd edge case).
Adding new features will probably have to wait until I have a much better understanding of the code base.

It isn't the "no" that sounded rude, I expected a no. You do not have to lie either, you could be quite frank and say that it is never likely to get done without sponsorship. You could just be nicer about explaining the deficiencies in the feature request, pointing out that they make specking out features very hard.

I realise I am not a customer yet, I am only an end user. How do people usually go from "What about adding feature F?" to how much do you want paid to do it? You mentioned dozens of hours work so thats 24~60 hours multiplied by $150/hour. My point was you are not likely to translate a feature request here into someone paying for it's development if you come across as rude.

Back to the useful bit. Would you like me to try and compile a more complete set of problem cases that I use to solve with templates and post it back here in a week or two or will it just add more noise and cost you more of your time for no benefit? If the problem is already relatively well understood and taken to be part of the request you marked as a duplicate do you need any further input from me to work out its value?

Here are a few of the use cases for templates (as in {{math 2 + 2}} results in 4) that I can remember from the past.

Potential Use Cases

On Page Load Templates

These are evaluated when the page is loaded. I do not think I have ever used them, they are slow and expensive. There are a couple of use cases though.

Todays Date

It is not often you want to generate a date stamp on a page but it is occasionally useful. The only times I have wished for this were when tracking some external link that included a date code like 2016-04-11 in to for "todays" item. Build status of daily builds being the case. Fortunately there has almost always been a latest or todays version of the link that I could use.

Where it is sometimes useful is if you generate meeting minutes or any other periodic pages in a wiki. If you have StatusMeetingMinutes/2017/04/11 you can use a template to have a "Create Todays Minutes" link that includes the current date in the title. Only a small usability simplification over a user typing the new URL by hand to generate the new page.

If you want examples of use on Wikipedia for a template you can go to the templates page, such as Template:Currentmonth and then follow the What Links Here item in the navigation menu. Some of them are not very good examples but it does show you how it is helpful as a way of keeping external links fresh.

On Page Save Templates

These are the type of templates I can remember using. They are evaluated either when the page that includes them is saved or when the page that defines them is saved. I would guess this is implemented by having a little cache result of the template that is included in the end page. If your system does not already allow for such a step I can see that adding this would be a huge change.

Almost Now

If you can automate the editing of articles (or templates) then you can easily script the updating of a {{CurrentDay}}, {{CurrentDayOfWeek}}, {{CurrentMonth}} and similar dynamic templates. Dates of course fall foul of Server Time vs. Viewer Time issues. Still a template that can be cached an only updated once a day at midnight(ish) server time is probably a lot simpler than one that updates ever page load.

The Latest Foo
In Wiki Page Text

In a previous life the current software was given a name and a number, like foo/1, foo/2, goo/1 and so forth. With a development staff getting a new starter every couple of weeks we wanted to keep our _Getting Started_ documents up to date, including copy paste instructions for cloning branches of repositories. There were numerous places in the wiki where it was helpful to include "the latest branch (foo/2)" or run the command bk clone foo/2. When the months rolled on we needed to update all these references to goo/1, having a single template page to edit made the update much simpler.

In the end a developer even scripted the update. Rather than having to grep and replace all mentions of foo/2 in our wiki he could update a single page.

In Links

Various resources referred to our branches by name. It was helpful to have "the latest" linked to for build statuses, test reports and webcode viewers. In git this is often master which is simple but on our large project we had 2 or 3 in development branches with most of the features going in the last one and lots of bug fixes going in the earlier two. As with the in text example, having a link that includes {{latestBranch}} that gets updated to 'foo/2' along with everything else is a time saver.

Simpler Links

When a resource is stored at a long and complicated link it can be helpful to simplify the markup down to something short, like {{bugsIn|foo/2}}, {{regressionTestResults|foo/2}} or {{buildStatus|foo/2}}. The template code then does the heavy lifting of generating the seach index links. If the URLs ever change there is one place to correct the links.

If you can nestle templates you can also start doing things like {{buildStatus|{{latestBranch}}}} that will always point to the latest branch.

Standardised Information Display

You want a status page for your branch that tells you what the build state is, gives you a link to the latest test results for it, what bugs are reported etc. You could generate seperate pages for each branch (or release) by copy and pasting the code between them and replacing all the mentions of <branch name> with the current topic. If you had a template you could write the information display once and use the branch name parameter to change all the text, all the links and ensure all the places that use the template are consistent. If you had seen the info page for the latest release it would be just as familiar to you as if you looked at the page for the release from 3 years ago.

Noticed a typo when editing the 7th release? With a template that typo is in one page that you correct and fix them all.

Useful Links By Topic

The breadcrumb navigation is very helpful when drilling down and up inside a category. Sometimes it is helpful to have a handful of key useful pages for a topic included in the footer of every page on that topic. With a template you can create and maintain a single list of links but have it appear on several pages.

For example if you have several products it might be nice to have the links to the Architecture, Source Code, Setting Up Build Environment & Software Build Status at the end of every page. These then lead readers view related information. Each product has a different list of key pages.


Mostly templates are about reducing duplicated typing. If you type a word, phrase, sentence, paragraph, table, link or section more than once, you could template it instead. Like with code reuse it is now guaranteed to be consistent everywhere. Fixing a bug in once instance fixes them all.

If I recall more use cases I will bring them up. I am sure Phriction or an extension to it will cover a lot of them. The only other big help that MediaWiki's templates add to that platform is to setup an API for template like things. You know that {{ something }} is not a simple link, it might take parameters and it should render to something on a page when you hit save (possibly not before). That means built in or scripted items can get added like {{today}} or the various hand rolled [[ looks like html link but isnt ]] work arounds do.