Page MenuHomePhabricator

Missing documentation for crafting raw Conduit API requests
Open, Needs TriagePublic

Description

I am attempting to update the build plugin for Phabricator->TeamCity. This involves needing to make some extra conduit calls to get the details for a diff. However, trying to figure out HOW to make the conduit call is proving to be an exercise in madness. I need to compose a raw HTTP request - but I'm having a terrible time figuring out what, exactly, needs to be in the POST body even when following the documentation on conduit/method/differential.querydiffs/ .

I can make a request on the documentation page fine, which is useful to show it works, but I'm having a terrible time translating the output into a POST to make off the documentation page.

It would be AMAZING if the conduit documentation in general could just show a raw HTTP request with the POST body formatted with however the JSON should actually look. Two of the three samples use arcanist, which isn't helpful, and the third one has a curl request with multiple -ds which also isn't particularly useful.

Event Timeline

Can you use a service like https://requestb.in with curl?

$ curl https://requestb.in/zjbxaszj -d api.token=api-token -d ids[0]=1

https://requestb.in/zjbxaszj?inspect

I've also found this tool pretty useful for inspecting HTTP requests:

https://www.charlesproxy.com

I think you can use curl --proxy localhost:8888 ... to get cURL to go through Charles.

We could provide a raw HTTP body too, but those approaches should work as-is without requiring you to update.

epriestley edited projects, added Conduit; removed Documentation.Mar 24 2017, 7:06 PM

(Kicking Documentation off because this would just be another tab next to PHP / cURL / arc in Conduit, not static documentation.)

I can definitely try either of those two options, but to my point, it wasn't obvious to me from the documentation:

that I should be doing -d ids[0]=1

I'll check out charlesproxy - for whatever reason, Fiddler doesn't want to capture traffic from arc even though I have it cracking other HTTPS requests.

As an aside, curl on Windows is a wrapper into Powershell's Invoke-WebRequest which also doesn't support multiple -d parameters for what it's worth.

The good news though is with that hint I seem to have it working without even needing to POST: https://phabricator.myurl.com/api/differential.querydiffs?api.token=api-TOKEN&ids[0]=5

Oh, maybe we could make this more clear. Here's how to use the examples to figure out how to make a call:

  • Make your call first, by typing JSON into the web form.
  • Submit the form.
  • Then, scroll down to the examples to see exactly how to represent it -- they'll update dynamically to reflect the call you just made with the same data you passed.

OH, that only happens if you do the human-readable option. I didn't notice that little box at the bottom had changed amidst the giant blob of json that the call returned.

Yeah, there's no real hint about that in the UI to suggest that the box will update, and it's easy to miss since it looks nearly identical after it updates. I'll add a piece of hint text to hopefully make it more clear.

To be clear though, I still haven't figured out how to actually put in a POST body. It's fine to use URL params for something like differential.querydiffs, but I can imagine the problems I'll run into with URL length when doing harbormaster.sendmessage (especially when reporting a bunch of unit test and linting results).

The page implies that all I should need to do is POST up, for example, the following:

{
  "buildTargetPHID": 12345,
  "type": "pass",
  "unit": [
    {
      "name": "PassingTest",
      "result": "pass"
    },
    {
      "name": "FailingTest",
      "result": "fail"
    }
  ]
}

But that definitely gets rejected by conduit.

error_info=API Method "harbormaster.sendmessage" does not define these parameters: '{ entire POST body }'

The page implies that all I should need to do is POST up, for example, the following:

Can you point me at where that's implied so I can make it more clear? That isn't currently expected to work.


When you run this command:

$ echo '{
  "buildTargetPHID": 12345,
  "type": "pass",
  "unit": [
    {
      "name": "PassingTest",
      "result": "pass"
    },
    {
      "name": "FailingTest",
      "result": "fail"
    }
  ]
}' | arc call-conduit --conduit-uri=http://local.phacility.com/ harbormaster.sendmessage

...here's what arc sends over the wire:

POST /api/harbormaster.sendmessage HTTP/1.0
Host: local.phacility.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 329

params=%7B%22buildTargetPHID%22%3A12345%2C%22type%22%3A%22pass%22%2C%22unit%22%3A%5B%7B%22name%22%3A%22PassingTest%22%2C%22result%22%3A%22pass%22%7D%2C%7B%22name%22%3A%22FailingTest%22%2C%22result%22%3A%22fail%22%7D%5D%2C%22__conduit__%22%3A%7B%22token%22%3A%22cli-oc7vfhywi3yqykld3dsputqemauu%22%7D%7D&output=json&__conduit__=1

Here's a different way to represent this request:

POST /api/harbormaster.sendmessage HTTP/1.0
Host: local.phacility.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 167

api.token=cli-oc7vfhywi3yqykld3dsputqemauu&buildTargetPHID=12345&type=pass&unit[0][name]=PassingTest&unit[0][result]=pass&unit[1][name]=FailingTest&unit[1][name]=fail

There are several other valid ways to represent the request, as well.

We do not currently support a blob of JSON as the post body, and the documentation shouldn't imply that we do (if it does, I'll fix it). This is perhaps something we should support, but see also T5955.

Maybe easier would be for you to tell me what language you want to use and I'll just write you a snippet which encodes a request?

But if you say "Haskell" you may have to wait a minute.

Raw HTTP is actually perfect for me! Thank you!

chad added a subscriber: chad.Mar 25 2017, 1:26 AM

But if you say "Haskell" you may have to wait a minute.

SphtKr added a subscriber: SphtKr.Jul 8 2017, 7:46 AM

The page implies that all I should need to do is POST up, for example, the following:

Can you point me at where that's implied so I can make it more clear? That isn't currently expected to work. ... We do not currently support a blob of JSON as the post body, and the documentation shouldn't imply that we do (if it does, I'll fix it).

Stumbled across this, and it happens I made the same assumption just two days ago. Just looking at the fact that arc and the PHP API module created a structure like a JSON array/hash--and being a developer who does a lot with REST APIs where you're posting JSON bodies around all over the place--I presumed that arc and the PHP example were sending JSON bodies to Conduit, and that the curl example that sent x-urlencoded-style form parameters was some kind of "legacy" way of sending data that was easier to do with curl (which was a silly assumption, I suppose, because it's perfectly easy to send JSON in a POST via curl).

I was using Ponduit (a Powershell library that does a little to make using Conduit easier from Powershell), and that incorrect assumption combined with poor/conflicting documentation of the Invoke-WebRequest -Body parameter drove me BATTY for half a day. I was trying to pass a hashtable+array structure to Pondit's cmdlets (which use Invoke-WebRequest) and couldn't figure out why on earth it wasn't working. Turned out I needed to pass a flattened hashtable with a bunch of parameters with field names like you would use in a form to submit to a PHP application (@{ "transactions[0][type]" = "statuses"; ... }...just like the curl example says.

So, I think the strong implication comes from a mindset of developers who do a lot of POST body passing of JSON, and assuming that arc does that looking at the example. In particular, I think it would have taken me a much longer time to disabuse myself of the bad assumption if I wasn't already familiar with PHP's mechanism for sending structured data fields...which others may not be.

pasik added a subscriber: pasik.May 12 2018, 1:47 PM