Help with Writing Tests - Foundation and community invite to help with 'do nothing' pass tests

Continuing the discussion from Help with Writing Tests:

These two queries identify 100+ empty test cases - each one is a test class with a ‘do nothing’ test body. So they represent I think a false sense of confidence or quality assurance of code health.

frappe@erpnext:~/frappe-bench$ find . -name ‘test*.py’ | xargs grep ’ pass’ | grep erpnext
frappe@erpnext:~/frappe-bench$ find . -name ‘test*.py’ | xargs grep ’ pass’ | grep frappe

Some of the test classes are:

apps/frappe/frappe/integrations/doctype/gsuite_templates/test_gsuite_templates.py
apps/frappe/frappe/integrations/doctype/oauth_client/test_oauth_client.py
apps/frappe/frappe/frappeclient.py

apps/frappe/frappe/integrations/doctype/dropbox_settings/test_dropbox_settings.py
apps/frappe/frappe/core/doctype/domain/test_domain.py
apps/frappe/frappe/data_migration/doctype/data_migration_connector/test_data_migration_connector.py: pass
apps/frappe/frappe/data_migration/doctype/data_migration_plan/test_data_migration_plan.py: pass
apps/frappe/frappe/data_migration/doctype/data_migration_mapping/test_data_migration_mapping.py: pass
apps/frappe/frappe/core/doctype/error_snapshot/test_error_snapshot.py

apps/frappe/frappe/workflow/doctype/workflow_action/test_workflow_action.py
apps/frappe/frappe/workflow/doctype/workflow/test_workflow.py

apps/frappe/frappe/email/doctype/email_group/test_email_group.py: pass
apps/frappe/frappe/email/doctype/email_unsubscribe/test_email_unsubscribe.py: pass
apps/frappe/frappe/email/doctype/email_rule/test_email_rule.py: pass
apps/frappe/frappe/email/doctype/email_group_member/test_email_group_member.py: pass
apps/frappe/frappe/email/doctype/email_queue/test_email_queue.py: pass
apps/frappe/frappe/email/doctype/email_flag_queue/test_email_flag_queue.py: pass

apps/frappe/frappe/desk/doctype/kanban_board/test_kanban_board.py

The benefits to ERPNext from improved test coverage are immense, for a wide range of reasons not all of which are apparent.

For prospective developers (and veterans too), each test invites a clear path to quickly discover how that code applies, to support (test driven?) development.

To write the first test case is typically the biggest hump. After that, incremental extensions are less intimidating and more fun for users to extend an existing test.

For ERPNext to grow and flourish, the aim should be for a ‘test infected’ implementation.

However all of the empty tests represent a serious technical debt that should be resolved.

Thanks for listening

5 Likes

October 2017 Coverage was 39%.
Around Slide 18 there is a graph shared by
@achillesrasquinha

As a matter of fact, Frappe requires a coverage report in-built. Considering this!

2 Likes

Yes after all open source distinguishes itself with transparency visibility and ‘better together’ community sharing.

So arguably such a report would be good strategy, to challenge the status quo and inspire goals in terms of improved quality.

1 Like