Page MenuHomePhabricator

Support math (LaTeX) equation in Remarkup
Open, WishlistPublic

Assigned To
None
Authored By
calfzhou
Nov 21 2014, 7:55 AM
Tags
Referenced Files
F1121144: 屏幕快照 2016-02-21 上午11.14.54.png
Feb 21 2016, 3:25 AM
F1120525: 屏幕快照 2016-02-20 下午3.12.25.png
Feb 20 2016, 7:17 AM
F236683: pasted_file
Nov 21 2014, 7:55 AM
Tokens
"Love" token, awarded by arend.danielek."Like" token, awarded by franjesus."Like" token, awarded by thoughtpolice."Like" token, awarded by ablekh."Like" token, awarded by ProfFan."Like" token, awarded by hyOzd."Manufacturing Defect?" token, awarded by khuynh."Mountain of Wealth" token, awarded by partofthething."Like" token, awarded by alishir.

Description

Our algorithm team use mathematic equations frequently. Instead of copy&paste the image of the equation in to a Phriction wiki, it would be great if we could use some markup tag to present it.

The problem of copy&paste image is

  • I need find another web app or software help me render the equation
  • No one can get the original LaTeX code from the image
  • Anyone who want to modify the equation, need rewrite the full equation somewhere, and then paste the new image back

So I wish that we could write LaTeX code directly in the wiki, using some markup syntax, for example <math>P(E) = {n \choose k} p^k (1-p)^{ n-k}</math>. Then the code could be rendered by MathJax (a JavaScript display engine for mathematics) to visualised form such as:

pasted_file (55×196 px, 4 KB)

Event Timeline

calfzhou raised the priority of this task from to Needs Triage.
calfzhou updated the task description. (Show Details)
calfzhou added a project: Phriction.
calfzhou added a subscriber: calfzhou.

This would be cooler as a general remarkup feature instead of just Phriction.

chad triaged this task as Wishlist priority.Nov 21 2014, 2:54 PM
chad edited projects, added Remarkup; removed Phriction.

Hmmm... This could be handled like dot & cowsay! For starters: theres tex2im but it's not standard on many distros, there's also some forms of tex2png, also there's dvipng that You can use to convert compiled dvi file to png... there's more than one way to skin a cat... That's for starters.

Now once one got sorted on HOW to generate image - it's as simple as shown in D7332: Implement Graphviz, Figlet and Cowsay as Remarkup interpreter blocks - around 40 lines of code and You can have all the TeX/LaTeX goodness You want! :-)

Personally - I'm no fan of mathjax as it creates dependency on external JS code that upstream would have to handle and keep up to date. With using it like cowsay (or graphviz), rendering would only depend on commandline stuff that would be sever admin responsibility to deal with :)

_____________________________ < I say! commandline is good! > ----------------------------- \ ^__^ \ (oo)\_______ (__)\ )\/\ ||----w | || ||

Hi, everyone!

New kid on the Phabricator block here. Firstly, I just wanted to say that I thoroughly impressed by Phabricator and thank all of contributors you for producing such a nice piece of software. After rather comprehensive evaluation of existing tools (mostly open source), I have chosen Phabricator as a platform of choice for my initiatives on improvement of research and software engineering processes a materials science / materials informatics research group, where I currently work (Georgia Tech).

Secondly, I'd like to apologize for accidentally creating an event (already removed) and moving T9520: Allow Workboards to show a thumbnail image within each card around the workboard (see my profile for history). I haven't realized that I was logged in at phabricator.com instead of my own Phabricator server instance. Please restore the order (I don't remember, what board this task belongs to). Again, sorry about that.

Thirdly, and finally, regarding the LaTeX support in Phabricator. While I certainly understand @johnny-bit's point on introducing dependencies, when using MathJax, I think that it still might make sense to provide that option and allow site's admin to enable it on their server (with expectation of their keeping MathJax up-to-date). An IMHO interesting alternative to the MathJax (and some other less elegant approaches) might be KaTeX. It is much faster than MathJax and doesn't seem to have any external dependencies, plus, it supports server-side rendering of LaTeX code. Unfortunately, currently it supports only a subset of TeX functionality. Some of the relevant info sources and discussion that might be of interest: here and here. Would be quite interested in your opinions on this and perspectives on and chances of adding LaTeX support to Phriction and other relevant Phabricator applications. Thank you for your attention.

Best regards,
Aleksandr "Alex" Blekh

@chad: Got it, thank you for the reference!

After reading the Contributing Code guide, it seems that I would have quite a high barrier of entry, should I decide to contribute to the project on this feature (upstream; normal process). In regard to listed alternatives, local fork is likely not a good idea for me and developing an application is likely not relevant to this feature, as, if I understand correctly, it should be a part of general/core code (i.e, Remarkup blocks for wiki). Therefore, the only feasible option (for enabling this feature on local install) that I could think of (additional benefit: this would be general, enabling LaTeX for any Phabricator's Web output) is to move the LaTeX-to-HTML on-the-fly conversion outside of Phabricator's code, specifically, as part of Web server request processing. For example, it could be implemented as a Nginx module or directly - via Nginx config - pass the output through KaTeX's server-side processing ("Server side rendering: KaTeX produces the same output regardless of browser or environment, so you can pre-render expressions using Node.js and send them as plain HTML."). Does it make sense? Please let me know what you think about all this.

@ablekh: quite a lot has changed in this front since April - see T9408: Upgrading: `dot` (Graphviz) support removed, changes to `figlet` and `cowsay`; in particular, we no longer consider "executing arbitrary binaries" to be reasonable approach for this kind of situation.

That ticket also contains a patch you can apply by dumping it into src/extensions/ directory, which will allow you to to use the dot binary, without forking your install; You could hack it to use tex2im or tex2png or whatever; It's unsupported, but will keep you afloat until this feature is built.

I'm not sure what using Nginx module would actually mean; I'm guessing the Remarkup code would be "rendered" as some javascript code that executes another web-call? I'm not sure how much of Remarkup is actually prepared for this.

chad renamed this task from Support math (LaTeX) equation in Phriction wiki to Support math (LaTeX) equation in Remarkup.Nov 23 2015, 6:30 AM

@avivey: Thank you for clarification and references. I understand the rationale behind the changes, though, I still see that Phabricator still allows executing "trusted" binaries (whatever it means). Anyway, I could not find documentation on using extensions, so I'm not too sure about how to use tex2im, tex2png or similar programs in Phabricator's context. Perhaps, someone could point me to the relevant documentation and/or, better, share some examples for that approach.

Considering the security aspect of executing arbitrary binaries within Phabricator, there are two points I'd like to make: 1) since I was talking about making changes on my local instance, which is running behind firewall and using which requires a valid VPN connection, I think that the using "unsafe" extensions would be a rather safe alternative; 2) the Nginx approach that I proposed is somewhat similar in essence to the "cowsay" with the significant difference that a conversion would be done beyond Phabricator's scope, via server-side pre-rendering, by Nginx-CGI (something along these lines and/or these). Therefore, this approach would be independent from Phabricator's code and, thus, no changes in Remarkup or any other parts of code would be needed. This is just a broad idea that I find interesting enough.

I did a rough implementation of this functionality (full of dirty hacks) here at GitHub.

Currently it does well on multiline LaTeX but fails to parse complex inline LaTeX because of the { signs in LaTeX text.

Is it necessary to change the Remarkup parser to make inline LaTeX possible?

@ProfFan does this create images on the server side?

@hyOzd No, this utilizes KaTeX and renders LaTeX using HTML and CSS. It is completely combined into the normal rendering workflow and is cached properly.

Now updated the parsing mechanism to make it able to handle multiline inline LaTeX.

# This is currently a test

{nav Home}

The statistical model estimated the probability, {tex \pi_{i}}, of capturing dolphins on a tow, {tex i}. A year effect, {tex logit(\pi_i) = \lambda_{j[i]} + \sum_c \beta_c x_{ic}.} was estimated for each year, {tex j}, allowing for annual variation in the capture event rates that was unrelated to the covariates, {tex x}. The contribution of each covariate, indexed by {tex c}, was governed by a regression coefficient, {tex \beta_c}, that was estimated by the model. The logit transform of the capture event probability was defined as the sum of the year effect, {tex \lambda}, and the covariates:

katex {{{
logit(\pi_i) = \lambda_{j[i]} + \sum_c \beta_c x_{ic}.
}}}
determines the result.

# The following is more tests

katex {{{
f(x) = \int_{-\infty}^\infty
    \hat f(\xi)\,e^{2 \pi i \xi x}
    \,d\xi 
}}}

katex {{{
\begin{bmatrix}
    a & b \\
    c & a 
    \end{bmatrix}
}}}

katex {{{
\begin{bmatrix}
    a & c \\
    c & a 
    \end{bmatrix}
}}}

renders into

屏幕快照 2016-02-20 下午3.12.25.png (1×1 px, 188 KB)

This looks awesome @ProfFan !

Glad that you like it :) The codebase is still very rough so please feel free to contribute!

Just changed the code to utilize celerity and now the extension works on all pages and comments :)

I am still trying to find a way to integrate it seamlessly with Phabricator without patching...

屏幕快照 2016-02-21 上午11.14.54.png (666×1 px, 86 KB)

@ProfFan Great job! Thank you for your efforts. BTW, what is Celerity?

@ProfFan Great job! Thank you for your efforts. BTW, what is Celerity?

Thanks :) It's the static file manager of Phabricator. Please see https://secure.phabricator.com/book/phabcontrib/article/adding_new_css_and_js/

@ProfFan My pleasure! :-) Thank you for the link - will take a look.

Throwing a vote in that this feature would definitely be of great use to us as well.

I'll also add for the folks in the thread that are interested in having LaTeX equations available that the URL options available from the CodeCogs LaTeX equation editor are probably the best workaround solution that I've found without making modifications to Phabricator. This link gives you the equation from the original post:

http://www.codecogs.com/eqnedit.php?latex=P(E)&space;=&space;{n&space;\choose&space;k}&space;p^k&space;(1-p)^{&space;n-k}

eadler added a project: Restricted Project.Aug 7 2016, 8:12 PM