[Tutorial] Using cURL for REST and RPC API calls

Hello all

Due to the amount of posts on this forum about not getting REST / RPC calls to work when using parameters such as fields= and filters=, especially when using cURL, I’ve decided to give back a small contribution, making hopefully a HUGE difference in someone else’s life. Most of those posts remain unsolved, and so, with this tutorial I trust the opensource ERPNext community will go from strength to strength!

I assume you’ve set up token based authentication : user guide to do so
For further clarification on setting up token based authentication, please see the latter part of this thread, especially the posts by @MartinHBramwell

I’m testing the below cURL commands having created an API Access token on the Administrator user, which is probably not a good idea, but for testing purposes we’ll do that for now. Ideally you should create a user specifically for REST access.

I’m on Linux, so I’ll use Bash as my shell. Let’s assign your token to a shell variable, HT means Header Token.

HT='Authorization: token <your API key>:<your API secret>'

Let’s assign two more headers to shell variables, HA means Header Accept and HC means Header Content.

HA='Accept: application/json'
HC='Content-Type: application/json'

To test access to the HTTP API of your server, in your web browser’s URL bar type:

http://<url_or_ip_of_your_server>/api/method/frappe.handler.version

Note that your protocol could be https rather than http. The above frappe.handler.version is actually a RPC (not a REST) call, however it does not require authentication, so we can test whether our URL is correct. You should get back a JSON string such as

{"message":"12.15.0"}

Now let’s assign the URL to a shell variable. Note that the path to RPC endpoints is …/api/method/, but because we’re going to play around with REST calls first, the path is …/api/resource/ and we’ll call our shell variable UR meaning URL Resource.

UR='http://<url_or_ip_of_your_server>/api/resource/'

REST GET calls

  1. Let's test a REST call using cURL, which does require authentication

    curl -H "${HA}" -H "${HC}" -H "${HT}" -X GET ${UR}User

    You should get back a JSON string such as

    {"data":[{"name":"Administrator"},{...

  2. So far so good.

    However, cURL does offer something called URL globbing. From the man page we read that globbing uses the { } and characters. This means that our REST calls cannot use parameters such as fields=[“name”] without being interpreted as globbing characters. So to provide a URL containing these characters you need to disable globbing with the -g option.

    curl -H "${HA}" -H "${HC}" -H "${HT}" -X GET -g ${UR}'User?fields=["name","email"]'

    …and voila! A REST call with a fields= parameter

    {"data":[{"name":"Administrator","email":"admin@example.com"},{...

    Let’s add a filter= parameter

    curl -H "${HA}" -H "${HC}" -H "${HT}" -X GET -g ${UR}'User?fields=["name","email"]&filters=[["name","=","Administrator"]]'

    You should get only the Administrator as a result

    {"data":[{"name":"Administrator","email":"admin@example.com"}]}

    or alternatively try escaping the " characters within the URL string

    curl -H "${HA}" -H "${HC}" -H "${HT}" -X GET -g "${UR}User?fields=[\"name\",\"email\"]&filters=[[\"name\",\"=\",\"Administrator\"]]"

  3. But there's a better way...

    Instead of suffixing the parameters to the URL, we can supply the parameters as data using the -G and -d options. In this case we do not have to concern ourselves with the globbing conflict as the parameters are no longer part of the URL.

    curl -H "${HA}" -H "${HC}" -H "${HT}" -G "${UR}User" -d 'fields=["name","email"]' -d 'filters=[["name","=","Administrator"]]'

  4. As a final titbit, note that the field references can be either the field name (aka ID) or the label, and even mixed, eg Name and email. They can even be mixed within the fields= and filters= parameters, eg Name and name. I'm not sure how robust this is, and it's probably better to use either the name or label consistently.

    curl -H "${HA}" -H "${HC}" -H "${HT}" -G "${UR}User" -d 'fields=["Name","email"]' -d 'filters=[["name","=","Administrator"]]'

REST POST and PUT calls

  1. Create a new Lead record with a POST call

    curl -H "${HA}" -H "${HC}" -H "${HT}" -X POST ${UR}Lead -d '{"lead_name":"My-New-Lead"}'

    {"data":{"name":"CRM-LEAD-2021-00001","owner":"Administrator",...

  2. Update the contact_date of this new record with a PUT call. To avoid a

    frappe.exceptions.ValidationError: Next Contact Date cannot be in the past

    error, make sure the date is in the future.

    curl -H "${HA}" -H "${HC}" -H "${HT}" -X PUT ${UR}Lead/CRM-LEAD-2021-00001 -d '{"contact_date":"2021-12-31"}'

  3. View the newly created record along with the change you've made to the contact_date. Note that I pipe the output to jq

    jq homepage

    curl -H "${HA}" -H "${HC}" -H "${HT}" -X GET ${UR}Lead/CRM-LEAD-2021-00001 | jq -r .

RPC methods

There are a few idiosyncratic differences when it comes to how we need to structure our cURL RPC vs REST invocations. But first, let's create a new shell variable for the RPC endpoints.

UM='http://<url_or_ip_of_your_server>/api/method/'

  1. To invoke an RPC which takes an argument, add it as a query parameter, ie suffix it after a ?

    curl -H "${HA}" -H "${HC}" -H "${HT}" -X GET ${UM}frappe.client.get_list?doctype=User

  2. However, I could not supply multiple arguments to an RPC by separating them with an &, but I am successful when I specify them as data, using the -G and -d options. Let's add the fields as our second argument.

    curl -H "${HA}" -H "${HC}" -H "${HT}" -G ${UM}frappe.client.get_list -d 'doctype=User' -d 'fields=["name","full_name"]'

    Note that unlike for REST calls, field references in RPC methods only allow the name (aka ID) not also the label, ie “full_name” and not “Full Name”

Thanks to @revant_one for providing us with a convenient list of all whitelisted endpoints

That’s all from me :slight_smile:

14 Likes

Thanks…

I hope there is similar tutorial using requests. especially the syntax for writting multi params/data for POST and PUT.

Can you give an example of what you mean? What have you tried?

I just can’t find a comprehensive docs for the API using requests. There is basic docs about REST API using requests, but the example of manipulating document is using cURL (the one by @rmeyer).
It is not really the logic or how to use requests vs cURL but more on the writting of the syntax. Sometimes in frappe/erpnext the using of brackets need familiarity. And easily forgotten if not using it for some times :slight_smile:

Every time I realize I knew how to do something and then forgot, I:

  1. remind myself how I did it
  2. create a small, simple but complete, working stand alone example
  3. ask how to do it here in this forum
  4. immediately answer my own question with the working example.

I REALLY wish everyone would do this.

Expressed in general terms like that, I don’t really know how to help.

The best way to get help here is:

  1. describe what you wish to achieve
  2. create a small, simple but complete stand alone example of what you have done
  3. show the result you get
  4. explain why it is not what you expected to see
  5. ask what to change to get what you actually want

When you do the things I describe, you leave a valuable trail of clear documentation that will benefit many other people. Also, the real experts here in the forum will take the time to help you because:

  1. they can quickly see what you are asking
  2. they can quickly see an answer
  3. they can see that you have the intention to make it clear and easy
  4. they can see that you are paying back into the community the time it takes others to help you out