Version 11 is a very rough release for us. We are constantly fixing bugs and other bugs are popping out from nowhere. Most of these bugs are related to UI interactions.
That’s why we have decided to bring back UI tests. We have tried writing UI tests before but they have failed for multiple reasons.
- Tests were directly coupled to Frappe API
- Tests would sometime fail (not deterministic)
- Tests took a long time to run
- Write tests that have no knowledge of how frappe works, we just give simple commands like visit a page, or click a button.
- Write tests that don’t depend on each other and can be tested in isolation. Also tests should not depend on explicit timeouts.
- Larger test suites will take a long time. This is inevitable. But if we implement the 2nd solution correctly, we can run tests in parallel.
Cypress makes it possible to achieve all of the above and also makes writing UI tests fun. Cypress has its own dashboard which records videos and screenshots of the tests that ran. You can take a look at it here.
Running UI tests in Cypress:
All of these tests run independently. None of them uses any frappe api, nor uses any timeouts for things to complete. These are the core principles of testing with Cypress, and I think they are important for robust test suites.
Since these tests do not depend on the frappe api, we will have to re-write any UI tests that we wrote for the previous system. We will be doing that over the course of the next 6 months.
If you are interested in helping us write UI tests, please check out the documentation: