Page MenuHomePhabricator

Develop system for dumping/backing up Phriction documents
Closed, WontfixPublic

Description

We would like a system for viewing a recent (say, day- or week-old) backup of all Phriction documents in an external-to-Phabricator location, for reference when our Phabricator install is down.

For instance, if there were a bin/phriction dump script, then we could write a cron job to run this script and manage ourselves where we place the backups. I'm open to the specific format of the dump.

Event Timeline

jhurwitz raised the priority of this task from to Needs Triage.
jhurwitz updated the task description. (Show Details)
jhurwitz added a project: Restricted Project.
jhurwitz added subscribers: jhurwitz, angie.

This is a related wishlist-y thing but it would be really cool if:

  • The wiki was a repo
  • It could be mirrored in a minimal read-only way (which could mean a lot of things, maybe it renders remarkup but otherwise just links everything?) but sync with the 'main' instance somehow (master with a repo mirror?). Honestly just the raw markup in a repo would probably be good enough to make me switch.

This would give the wiki batch editing powers (sed!), nice editing with $FAVORITE_EDITOR, easy export for analysis [1], the ability to use differential with the wiki [2], atomic edits/renames of multiple files, and options for the 'where do I store runbooks about phabricator' problem.

[1] We use gitit today and have scripts that do things like 'find all broken links', 'draw a graph of all internal links', or 'guess at some old pages in need of love.

[2] We do this today too, but it's a bit rough without sentence-diff.

That sounds like another great solution option for this problem :)

I think Diviner is meant for these scenarios. Basically because

  • Diviner publishes a static html site
  • Diviner articles live in the source code

That said, Diviner isn't really ready yet, and perhaps the wiki should get some of this feature set too.

This is a fairly huge amount of complexity for a very niche use case, which makes me hesitant to pursue it in the upstream. I expect the number of installs which would use this feature is on the order of 1, and that it essentially amounts to custom development.

Some more practical approaches might be:

  • (T4571) Build Read-Only mode, then maintain a mirror of Phabricator in permanent read-only mode. This gives you access to everything else, but doesn't let you, e.g., print it off or whatever. This is an ongoing operational headache on your side.
  • (T7715) Modernize the API, use arc call-conduit phriction.query as your phriction dump, pipe it through some custom script to get output you're satisfied with.
  • I can spend ~30 minutes just writing you a script to do this at consulting rates. I don't anticipate it breaking anytime soon. This is almost certainly cheaper than building this feature for real.

On follow-ons here:

  • I'd be very surprised if Phriction is ever exposed as a Git interface. The repository would be different for each user (each page has policies, and each user's view of the wiki differs), so we'd basically need to build it on-demand for every user. This is just an enormous amount of effort for a feature I suspect very few users would actually use.
  • There's no reason we can't do find/replace (T5166), arc edit-wiki <page> for $EDITOR, and even arc export-wiki / arc import-wiki for dumping stuff to the filesystem, but these won't help in the disaster recovery case and are blocked by T7715 + T5873.

@jhurwitz I use conduit to fetch wiki pages and format them with conduit to get a html syntax. I do that to add Wiki-Pages to a Frontend-Styleguide. Then i just add the required phabricator css and everything looks fine.
So i think you can write your own simple script to do everything over conduit, right?

You have to take care about permissions by yourself for sure...

@epriestley , do you have an example of how one can use 'arc call-conduit phriction.query' ?

We are looking for a command line way to dump Phriction documents (primarily to be able to access them from a terminal when we are in a disaster recovery situation with no access to our html browsers).

epriestley claimed this task.

I'm going to roll this into T13077, which will provide full access to Phriction via the API. In combination with Webhooks, you could mirror everything into some external read-only backup system or just pipe every edit to lp and stick it in a binder or something.