Let Us Make ERPNext a professional workflow alternative for real business use cases, your ideas count

IMHO, Frappe is actually an orchestrator that can instantly deploy modules (like microservices in the container world of docker and kubernetes) after the schema is defined under developer mode.

1 Like

Workflow across doctype is a need and frappe needs such feature. I think this should be part of core.

1 Like

@szufisher I am interested in your PR, Please reopen :slightly_smiling_face:

I also agree that ERPNext should have a more feature rich workflow capability.

The current workflow capability lacks things such as:

  1. Better modeling of real-world process flows, rather than the state transition tables currently in ERPNext
  2. Delegation and escalation of a specific ā€œtaskā€ to an individual user, rather than the current role based allocation rules

We use the Camunda Modeler to design our process flows : Camunda Modeler - Design Business Processes and Decision Models
This modeler is written in JS and is opensource, hence incorporating it into ERPNext should not be too difficult. Itā€™s a JS library and it can be customized to contain only the most basic modeling features :
bpmn-js walkthrough | Toolkits | bpmn.io and also GitHub - camunda/camunda-modeler: An integrated modeling solution for BPMN, DMN and Forms based on bpmn.io.

We also use Camunda as our process engine, with integration via the APIs in both Camunda and ERPNext. We do not integrate any other system other than ERPNext and find itā€™s unfortunate that we have to break out of ERPNext in order to manage rather simple process flows.

Camunda Modeler : Download The Camunda BPMN / DMN Process Modeler | Camunda
Camunda Engine : Download Camunda Platform 7 Community Edition

6 Likes

No not necessarily. If you have complex requirements you could drive state changes using the API from an external orchestrator. Likewise, code can run on workflow changes in frappe with the current system and do anything you like (such as call external software etc).

Right now Iā€™m mostly interested in some very BASIC things that would allow us (and by us I mean ME) to implement some simple non-automated workflows just in frappe through the UI. There are a handful of things that I think would make what is there now dramatically more useful. I attempted to reimplement some of our Jira based workflows (which are NOT complex) in ERPNext and couldnā€™t. Notably there doesnā€™t seem to be a way to communicate validation on state changes (ie to report to user ā€œfield X must be Yā€), or do any actions on state change other than setting a single field value. If those were improved it would at least meet MY requirements. Not attempting to claim MY requirements are everyoneā€™s!

If adding features also means providing a documented escape hatch out of those features amd into a rock solid workflow solution, that can be acceptable.

Since every code/feature is a legacy, not documenting and fulky supportinh the escape hatch might be a dangerous path. If a full blown integration wiith camunda/zeebe is documented (maybe even integrate the modeler as @EugenP suggested) it will become a lot easier for maintainers and the broader community to not fall into the falsehoods of feature creep.

I donā€™t really see this as an either/or question. Frappeā€™s design philosophy has always tried to maximize both internal power and external integration.

To that end @blaggacao, no escape hatch is needed. The way to bypass frappeā€™s workflow mechanism is to never set it up. I donā€™t know much about Zeebe implementation requirements, but given the scope of Frappeā€™s document API I have to imagine that linking an external orchestrator would be both possible now and relatively straight forward.

Precisely to avoid feature creep, the Frappe maintainers have lately been encouraging integrations as separate apps, maintained by the community. You seem passionate and knowledgable about Zeebe, which makes you the perfect person to get that work going!

2 Likes

Well, that sounds nice to fend off my activist vocalism, but it does not really solve the problem at hand:

Firstly, Iā€™m a singleton. So I would never dare to seriously offer to the public a half baked, unsupported dependency for them to build a business critical workflow engine on top.

We need a blessed and officially supported pathway to workflow managament.

This thread suggests mainly two lines of thought:

  • build on the shoulders of giants (nanos gigantium humeris insidentes)
  • build something ā€œquick and easyā€ within the communityā€™s comfort zone (= python/frappe)

Disclaimer: Iā€™m not only a fierce adversary of NIH syndrom, but I also donā€™t have much sympathy for comfort zones as they tend to produce suboptimal outcomes. :wink:

So in the case that upstream maintainers want to exploit the powers of workflow managament for their open source erp suite, and only in that case, they have to decide between those two lines of thought.

As I said, we have to acknowledge that every line of code written is a (huge) legacy. If we take a closer look this is true for the second option (home grown solution) in even more than one direction:

  1. Maintenance of the code itself
  2. Improve the thought model behind the implementation: a feature is always also a promise to the users.

Having far less dedicated resources than the camunda community, the second point is going to be a constant catch-up game for which the erpnext community probably lacks resources to spare.

Iā€™m actually pretty confident that the upstream maintainers will have the wisdom to make the right choice here (as they did by incoportating java based QZ Tray for host device access). As we can see from this thread, a demand for a solution seems to exist.

2 Likes

Iā€™m not deflecting your activism. Iā€™m suggesting that thisā€¦

ā€¦is a false and unproductive dichotomy.

Frappe is full of features that are implemented faster/better/stronger elsewhere. That doesnā€™t make them bad features. Frappeā€™s public-facing cms framework doesnā€™t come close to Drupal or Jamstack, for example, but it is nevertheless extremely useful to those whose needs are simple and who donā€™t want to manage a whole separate technology stack. For somebody who just wants to define a leave approval process, likewise, telling them that they need to load up a complex third-party orchestration platform is not a great answer.

Zeebe looks great. The next step, however, is not activism. The next step is a proof of concept. Frappe already has all the hooks you need, and I strongly suspect that little to nothing in the core code will need to change to make the integration possible. Thereā€™s no need to wait for a blessing.

3 Likes

Zeebe may look good. But have a look at the license:

Condition 2: Without limiting other conditions in this Agreement, the grant of rights under this Agreement does not include the right to provide a Commercial Process Automation Service.

A ā€œCommercial Process Automation Serviceā€ is a service intended for or directed towards commercial advantage or monetary compensation for the provider of the service enabling third parties (other than the providerā€™s employees or Contractors) to deploy and execute Custom Processes or to access the Software via its APIs.

By just quickly reading, I would understand that no commercial support may be provided to clients by Frappe Consultants based on Zeebe, since they would provide a ā€œservice directed towards ā€¦ monetary compensationā€ ā€œenabling third parties [the customers of the consultant] to deploy and execute Custom Processesā€¦ā€

Maybe I am wrong, but this should be considered firstā€¦

1 Like

Looks like somebody already started with n8n:

Their license says:

Without limiting other conditions in the License, the grant of rights under the License will not include, and the License does not grant to you, the right to Sell the Software.

For purposes of the foregoing, ā€œSellā€ means practicing any or all of the rights granted to you under the License to provide to third parties, for a fee or other consideration (including without limitation fees for hosting or consulting/ support services related to the Software), a product or service whose value derives, entirely or substantially, from the functionality of the Software. Any license notice or attribution required by the License must also include this Commons Clause License Condition notice.

In the case of Frappe/ERPnext I would suggest that the value of the product or service DOES NOT derive entirely or substantially, from the functionality of the Software (N8N).

1 Like

Both n8n and zeebe seem.to operate under the faircode.io philosophy. (btw thatā€™s a marketing invention of the creator of n8n)

However n8n seems quite a lot more restrictive than zeebe.

ā€œServiceā€ in ā€œCommercial Process Automation Serviceā€ coincides with kubernetesā€™s usage of the term, as long as it is commercial (either public or private). This clause is meant to fend off big money competition against their own SaaS offering.

n8n is quite a bit more restrictive, which generally prohibits making money with their software.

And if workflow management is part of the requirements, that is has an identifiable price tag, then it is definitively of substatial value to the product or service offering (of an independent integrator).


n8n does compete in simplicity, though, while zeebe competes on industry standards: BPMN 2.0 ā€” a slightly more concerted (much more than) marketing initative from the founders of camunda, among others.

It is hard to say who is the right go-to partner community for the ERPnext community.

Standards have certain appeal and conduct implicit long term guarantees, though.

1 Like

Since time constraints on my end are not a shameful circumstance, Iā€™m still actually looking into the option for somebody to step up. Sure, I would prefer to upstream the investment and put it at the source rather than a local and siloed reverberation of the cause with little long term upside potential.

Done that in the past. Itā€™s utterly frustrating to my visionā€™s aspirations.

1 Like

Have you had a look at this:

https://netflix.github.io/conductor/

Comming from Netflix under apache license.

1 Like

Yeah, conductor is the third elephant in the space. While there is also openintegrationhub.org which overlaps a little, but rather is in the data synching business.

What I find especially hard to beat with zeebe is the self documenting nature of litterally programming your workflows upon BPMN 2.0 graphically. In an organizational context an undocumented workflow is a non existing one (employee turnover, compliance, auditing, etc
).

I did not expose them initially, since Iā€™m (persobally) already past the technology evaluation phase. I should have mentioned them as well. There is also luigi and apach airflow which have been used in similar scenarios and surely a lt more.

1 Like

Elegant solution would be to capture/trigger the workflows based on erpnext events. An abstraction layer/adapters can provide integration to various workflow engines. And ability to provide a view into ErpNext as to what is happening with a given workflow. Is this possible?

Edit: Just want to thank everyone for this very informative discussion :smile:

1 Like

Thatā€™s great, and itā€™s a really positive way to move this forward. If I can help in any way, I absolutely will, because I unambiguously see the longterm value. Up to now, Iā€™ve done some workflow integration using Huginn/Zapier/n8n/myddleware, but none of it has ever felt like a great solution. It feels hacky and extremely prone to silent failure. At first glance, at least, Zeebe looks like a vastly more robust approach.

The challenge here, though, is familiar to open source. Frappe is complicated, and Zeebe is complicated. Both have relatively small expert user bases. Finding someone with deep knowledge of both going to be tough. Given that you are literally the only person who has ever posted the word ā€œZeebeā€ on this forum, I wonder if that person even exists right now.

Likewise, I donā€™t think itā€™s fair/accurate to accuse anyone in the Frappe community of Not Invented Here syndrome. Frappe is open and accessible from its very first design principles. The potential for interoperability with something like Zeebe (or Shopify, or Twilio, or Drupal, or Zapier, or PayPal, or microservicesā€¦) is fundamental to how Frappe works. It has taken a tremendous amount of work to make this possible.

The reality is much simpler: up to this point, nobody has put the time and energy into making a Zeebe integration happen. Nobody is opposed to it, but integration takes work and expertise that few have. Simple workflow management internal to Frappe has a legitimate place, and that functionality can coexist beautifully with integrations to external orchestrators. If both options exist, maintainers will be in a far better position to make good decisions about how much complexity Frappe wants to manage in-house.

To get that integration, the next steps are clear. Somebody with good technical knowledge of Zeebe needs to be able explain exactly what Frappe needs to be able to do to integrate. How do we get that specifications document?

1 Like

my idea is to support integrating ERPNext with existing mature workflow tool, after I finished my DDMRP project, I can join force to do those workflow project.

1 Like

Just a thoughtā€¦

Itā€™s absolutely imperative to decouple the modelling of processes from the execution of those processes.

Think of the modelling of processes as writing the business logic in a graphical manner, with icons and other modelling artifacts adhering to a well defined standard.

Think of the execution of processes as giving this logic (the model) to an interpreter for subsequent execution. Any execution engine which conforms to that modelling standard can then execute the models. This, in theory at least, affords you to swap either the modeler or the engine for a more suitable alternative at some later stage.

The de facto process modelling standard right now is BPMN 2.0, which is also an ISO/IEC 19510 standard. https://www.omg.org/bpmn/ What I like about the Camunda Modeler (https://bpmn.io/) is that you can reduce or limit the collection of modelling artifacts to suite your requirements, ie initially only avail a minimal set of modeling artifacts, which will allow ERPNext to grow into accommodating more advance modelling over time.

The choice of an engine is then for all intents and purposes immaterial, as it reduces into a ā€œblack boxā€ and only itā€™s API is of concern. Using the API, ERPNext can then schedule and invoke the execution of models, and receive status updates in return.

2 Likes

@EugeneP Please excuse my newbie-nessā€¦ What is the relationship between BPMN and Unified Modeling Language (UML)? Has one replaced the other? Or do they server different purposes? (I know very little about UML and less about BPMN but I am completely on-board with the need for modeling.

1 Like