Custom apps for cloud users [extend ERPNext with microservices]

Traditional Custom App

  • It is part of the same server, database, site, environment, assets.
  • Upgrades along with other apps.
  • Migration and retirement of the app is complex and endangers whole system
  • Can compromise data integrity or system security
  • Privileges of app to access system and user resource cannot be restricted by user
  • Based on Frappe Framework only and communicates via Python and JS api

Microservice Custom App

  • It is on different server with own database, site, environment, assets etc.
  • Can have its own release cycle
  • Migration and retirement of the app is independent of production ERP
  • In case of compromise or failure, ERP Data and system is safe
  • User has the control to allow or disallow app from using ERP resource. App access can be revoked
  • Can be based on any framework including Frappe on any device and communicates via OAuth 2.0 and REST API

Why

  • Frappe ERPNext is fast changing
  • Not upgrading ERPNext as per the release schedule means missing out on lot of features and fixes
  • Custom app blocks upgrade and may have migration complexities
  • Custom app fails and takes down whole system
  • What is not worth going into core as a pull request may be unreviewed patch work or hack

How

  • Setup Social Login such that the primary ERPNext server is in your Frappe Server URL. e.g. SaaS Server URL
  • The microservice always signs in using primary ERPNext account
  • OAuth2Authenticator for android, ERPNext Connector or equivalent OAuth 2.0 Client is installed on Microservice app
  • If microservice is a frappe app, relevant Link fields must show upstream doctypes. ERPNext Connector is hacking this in with Connector DocTypes which query upstream server db for instances saved there instead of showing local data
  • Whenever user enters such link field the upstream docnames start showing and latest docnames are saved locally for saving data in link field
  • If there is any change on upstream server, Webhook can inform it to app
  • Even Android app can use login via Frappe. Check out MN Technique’s App Showcase for Android app using this approach

Note: Read Documentation Frappe > Integration for more information

Signed in to Primary ERPNext Server with nothing installed e.g. SaaS Server

Item List on ERPNext

Custom App login with Frappe

Signed into Custom App

Upstream Items in Custom App Query

Service Providers can now deliver customization / integration to cloud users and promote cloud usage.
Anyone interested can send PR to erpnext_connector and extend it as needed.

23 Likes

Thanks a lot @revant_one, you did an amazing job! :smiley:

I have a silly question: when I try to add the Frappe Server URL in my Primary Server, I get the following error:

Unable to make request to the Frappe Server URL

If my understanding is correct, this URL should be the URL of the Primary Server ?

When I test:
requests.get("http://primary-server-url/api/method/frappe.handler.version", timeout=5) in the console I correctly get {"message":"8.10.8"}

But when I add http://primary-server-url in the UI and try to save it doesn’t work.

I also configured the “host_name” in site_config.json, but with no success…

And when I add the URL of another server, it works fine, so the issue is only when it is it’s own URL.

Do you, by any chance, have any idea of what I have missed ?

1 Like

I tried it on trial account on erpnext.com and it is working for the subdomain configured there.
I was about to remove that validation, but it is working on erpnext cloud. So I kept it.

  1. What I found was, If I add the hostname to /etc/hosts it doesn’t timeout on validation.
  2. It works on appropriately configured domain/hostname I guess.
  3. frappe.db.set_value your way forward if you just need to test! e.g. https://github.com/frappe/frappe/blob/develop/frappe/tests/ui/test_oauth20.py#L18

The url is required in id_token. Decrypted id_token has the url of issuer. iss, Standard Fields

1 Like

Thanks a lot for your answer.

Indeed it works fine on erpnext.com so it must be a configuration issue on my remote development server.

The strange thing is that it works fine in the console but not from Frappé directly…

I haven’t been able to find it yet, but I will post the solution once I have it.

And thanks for the frappe.db.set_value tip :wink:

1 Like

Still haven’t found the solution to connect via Oauth on my dev server, but I have been able to make it work through erpnext.com account.

I have another question regarding erpnext_connector: how are we supposed to setup our doctype to get the document list from the master server.

I tried something like:

But it doesn’t work automatically.

I also saw in your code that there is a hook called “erpnext_connector_search_fields”. Can you please tell me what it is supposed to do ?

Thanks a lot for your help! :slight_smile:

check this code, It has 3 examples,

  1. All fields for ERPNext Connector doctypes have special query
  2. Specific Link field with query and filter
  3. Child table query example

Intended hook is for showing upstream searchfield for microservice app, because in the ERPNext Connector doctype there is only name field. Thus you cannot add search fields like normal doctype/customize form

"erpnext_connector_search_fields" = {
	"Contact": ["first_name", "last_name"],
	"DocType" ["field_1", "field_2"]
}

The connector is evolving, if I find a way to not use Conntector Doctypes at all then it will be ideal.

1 Like

Hi @revant_one,

I’ve managed to validate the “Social Login Keys” on the server. I just needed to increase the number of gunicorn workers (Bench is in Production Mode)…

If anyone is interested: https://discuss.frappe.io/t/how-to-increase-number-of-gunicorns-workers

Now I encounter another issue when trying to access the upstream document from my custom app (with an expired token):

Traceback (most recent call last):
  File "/home/frappe/frappe-bench/apps/frappe/frappe/app.py", line 62, in application
    response = frappe.api.handle()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/api.py", line 53, in handle
    return frappe.handler.handle()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/handler.py", line 22, in handle
    data = execute_cmd(cmd)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/handler.py", line 53, in execute_cmd
    return frappe.call(method, **frappe.form_dict)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/__init__.py", line 923, in call
    return fn(*args, **newargs)
  File "/home/frappe/frappe-bench/apps/erpnext_connector/erpnext_connector/api.py", line 14, in login_via_frappe
    info = get_info_via_oauth("frappe", code, json.loads)
  File "/home/frappe/frappe-bench/apps/erpnext_connector/erpnext_connector/account_manager.py", line 25, in get_info_via_oauth
    session = flow.get_auth_session(**args)
  File "/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/rauth/service.py", line 556, in get_auth_session
    session = self.get_session(self.get_access_token(method, **kwargs))
  File "/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/rauth/service.py", line 542, in get_access_token
    access_token, = process_token_request(r, decoder, key)
  File "/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/rauth/service.py", line 24, in process_token_request
    raise KeyError(PROCESS_TOKEN_ERROR.format(key=bad_key, raw=r.content))
KeyError: 'Decoder failed to handle access_token with data as returned by provider. A different decoder may be needed. Provider returned: {"error":"unsupported_grant_type"}'

Here is my app advanced setup:

So grant_type is identical to the one hard-coded in the connector:
image

And token setup in erpnext_connector:

Do you know if I’m missing something again ? :slight_smile:

1 Like

Try with query token Always. It is working on one of the server I’ve setup. In this case it validates access token everytime get_auth_token is calleby making a request to openid_profile endpoint.

Have you successfully logged in via Frappe?

I’ll check on expiry part.

Edit: both seem to work when I tested. Always and On Expiry.

Are you logging in with Administrator user? It won’t work with admin

Thanks @revant_one,

Actually that’s the strange thing: I was able to login successfully via Frappe on Friday, but when trying again yesterday I got the above issue.

I’m trying to login with a system manager like last Friday.

I am going to check with Postman to understand where it comes from.

1 Like

Increasing gunicorn workers to 2 solved the problem.

1 Like

I think there is an issue with the Oauth Bearer Token:

I have just deleted the existing Token and reconnected and now the connector is working fine.

The token registered on my primary server was expired, as expected, but maybe it caused an issue during the next token request.
I’ll let you know how it behaves after the next expiration.

Take a look at account_manager.py.

I’m overriding whitelisted function frappe.www.login.login_via_frappe

During override the bearer token recieved with rauth is stored in Connector User Data DocType

Custom Scripts on cloud server that call endpoints on custom server will result in Access-Control error

On custom app server, setup wide open cors or restrict it to user’s cloud domain
e.g
add_header 'Access-Control-Allow-Origin' '*';
or
add_header 'Access-Control-Allow-Origin' 'user.erpnext.com';

full gist

1 Like

Continue discussion here Custom apps for cloud users [extend ERPNext with microservices]