you can add libreoffice/soffice to your list. last time we had to do sth like this we used soffice --headless --convert-to pdf
not really better than running a browser though... in our use case we needed to use merchant uploaded doc/docx templates in combination with our data. so even less options.
but yeah most options around pdf generation are clunky.

Yeah, I'm really hoping to not require us to run an entire browser or depend on an external service to generate PDFs. The approaches I found in my research were:

Just my two cents here, but chrome has an headless mode that allows to "print to pdf" : https://developers.google.com/web/updates/2017/04/headless-chrome

When I apt-get install wkhtmltopdf (or something like that) on aux001, which is Ubuntu 14, I get this:

I've worked with wkhtmltopdf several times, and I don't remember it actually requiring any special setup - in all cases, I basically dropped the binary in the machine (Although it's possible all my machines had X11 built in for some reason? I've never intentionally installed X11).

Just some observations after popping open PDF invoices which have shown up in my inbox recently:

See PHI1367, which featured a specific case where git repack unambiguously did something good instead of being magic fairy dust that we sprinkle around to ward off demons.

This is also more "while I'm here", but there's no web UI way to void an invoice right now. It would be nice to have a "Void Invoice" button for staff instead of requiring database fiddling, since this comes up occasionally.

Per T13366, I'm just going to get rid of this behavior for now, with the expectation that we'll soon begin generating side-channel emails for "Billing Department Email Address" under the umbrella of T8389 and could put "Merchant Emails" on a similar channel if we want.

$info_object = new PhabricatorPDFInfoObject();

PDF files appear to consist of a series of objects that (mostly) look like this:

Adjusting log_warnings = 2 in production (to get connection aborts into the error log) is also possibly desirable, although the background level of connection abortions (general network flakiness, server restarts during deploy, wait_timeout on very long-running demons?) may be high enough that this is more noise than signal.

We could also consider these things:

So actual actionable stuff here is:

Bumping max_allowed_packet to 1G in the server config resolved things. The export process then spent a long time doing a bin/files migration (which could use a progress bar, maybe) and is now doing a dump (which could too, although I'm less sure of how we'd build one).

I adjusted innodb_log_file_size to 1GB and attempted the import again, but ran into the same issue.

The "age of the last checkpoint" error appears to primarily implicate innodb_log_file_size, which is currently set to the default value (5MB):

I'll also double check wait_timeout and interactive_timeout...

Aha! The MySQL error log actually appears to have something useful:

Run it with source ...;

Unzip the dump before running it.

Look at the unzipped dump and see if line 13935 is bad in some obvious way.

(Whatever the resolution is here might also motivate tailoring our restore/import instructions, since this error is pretty opaque and the next steps aren't obvious.)

(Internally, see also PHI1329.)

This is deployed everywhere now, and kicking secure004 over seemed to work correctly, at least.

This stuff is all deployed, now.

cd provides "at least once" delivery guarantees.

cd provides "at least once" delivery guarantees.

This incantation seems to work as an upstart job:

I'm going to install an upstart task on secure004 and kick it a few times, some stuff might be sketchy until I figure that out.

The stalled production backup process completed successfully after deploying the change.

I'm going to try to sneak this out to the db tier to resolve things before the west coast wakes up, at least.

PHP Fatal error: Out of memory (allocated 311164928) (tried to allocate 105988097 bytes) in /core/lib/libphutil/src/future/exec/ExecFuture.php on line 246

See one followup in T13326. The "import from disk" part seems to have worked properly in production.

I just ran into this for the first time:

I've also never seen anyone use kill -TERM `cat /path/to/pidfile` in real life over some flavor of pkill, which is basically the same thing as "pattern match the process titles".

provided I'm not missing some secret reason to retain PID files.

I have a patch for this which basically says "don't try to kill any process which doesn't look like a daemon process".

This is probably somewhat entangled with T13253 and/or T11037 and this whole thing is kind of a mess.

But wait! It says there are two ways! The other way is:

If you are just writing an upstart job that needs to start the service after the basic facilities are up, either of these will work:

start on (local-filesystems and net-device-up IFACE!=lo)