Redisearch on frappe cloud

anandf · February 20, 2022, 8:52am

How to get redisearch on frappe cloud working?

adityahase · February 20, 2022, 11:32am

We observed some issues related to site creation when the redisearch module is enabled in redis-cache. Currently, we’ve disabled it for everyone on Frappe Cloud.

Maybe @marination can shed some light on this.

Please raise a ticket on frappecloud.com/support if you want to enable it for your bench anyway.

marination · April 4, 2022, 8:42am

github.com/frappe/erpnext

feat: Redisearch with consent (bp)

frappe:version-13-hotfix ← marination:redisearch-app-install-v13

opened 07:39AM - 01 Apr 22 UTC

marination

+121 -58

Port of https://github.com/frappe/erpnext/pull/30522 Closes https://github.com/…frappe/erpnext/issues/30559 > Can be tested on https://rs-install-1.frappe.cloud/all-products (v13) Documentation: https://docs.erpnext.com/docs/v13/user/manual/en/e_commerce/e_commerce_search ## Issues: <details><summary>App Install breakage</summary> App install would break due to stray methods in the `redisearch_utils.py` file being executed before migrate ```py Installing erpnext... Updating DocTypes for erpnext : [======================================= ] 99%Traceback (most recent call last): File "/home/frappe/frappe-bench/apps/frappe/frappe/commands/site.py", line 190, in install_app _install_app(app, verbose=context.verbose) File "/home/frappe/frappe-bench/apps/frappe/frappe/installer.py", line 162, in install_app sync_for(name, force=True, sync_everything=True, verbose=verbose, reset_permissions=True) File "/home/frappe/frappe-bench/apps/frappe/frappe/model/sync.py", line 70, in sync_for import_file_by_path(doc_path, force=force, ignore_version=True, reset_permissions=reset_permissions) File "/home/frappe/frappe-bench/apps/frappe/frappe/modules/import_file.py", line 137, in import_file_by_path path=path, File "/home/frappe/frappe-bench/apps/frappe/frappe/modules/import_file.py", line 257, in import_doc doc.insert() File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 268, in insert self.run_post_save_methods() File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 1003, in run_post_save_methods self.run_method("on_update") File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 867, in run_method out = Document.hook(fn)(self, *args, **kwargs) File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 1160, in composer return composed(self, method, *args, **kwargs) File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 1143, in runner add_to_return_value(self, fn(self, *args, **kwargs)) File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 861, in <lambda> fn = lambda self, *args, **kwargs: getattr(self, method)(*args, **kwargs) File "/home/frappe/frappe-bench/apps/frappe/frappe/core/doctype/doctype/doctype.py", line 326, in on_update self.run_module_method("on_doctype_update") File "/home/frappe/frappe-bench/apps/frappe/frappe/core/doctype/doctype/doctype.py", line 380, in run_module_method module = load_doctype_module(self.name, self.module) File "/home/frappe/frappe-bench/apps/frappe/frappe/modules/utils.py", line 205, in load_doctype_module doctype_python_modules[key] = frappe.get_module(module_name) File "/home/frappe/frappe-bench/apps/frappe/frappe/__init__.py", line 975, in get_module return importlib.import_module(modulename) File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1006, in _gcd_import File "<frozen importlib._bootstrap>", line 983, in _find_and_load File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 677, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 728, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/home/frappe/frappe-bench/apps/erpnext/erpnext/e_commerce/doctype/website_item/website_item.py", line 13, in <module> from erpnext.e_commerce.doctype.item_review.item_review import get_item_reviews File "/home/frappe/frappe-bench/apps/erpnext/erpnext/e_commerce/doctype/item_review/item_review.py", line 13, in <module> from erpnext.e_commerce.doctype.e_commerce_settings.e_commerce_settings import ( File "/home/frappe/frappe-bench/apps/erpnext/erpnext/e_commerce/doctype/e_commerce_settings/e_commerce_settings.py", line 9, in <module> from erpnext.e_commerce.redisearch_utils import ( File "/home/frappe/frappe-bench/apps/erpnext/erpnext/e_commerce/redisearch_utils.py", line 209, in <module> define_autocomplete_dictionary() File "/home/frappe/frappe-bench/apps/erpnext/erpnext/e_commerce/redisearch_utils.py", line 38, in wrapper func = function(*args, **kwargs) File "/home/frappe/frappe-bench/apps/erpnext/erpnext/e_commerce/redisearch_utils.py", line 152, in define_autocomplete_dictionary 'show_categories_in_search_autocomplete' File "/home/frappe/frappe-bench/apps/frappe/frappe/database/database.py", line 570, in get_single_value df = frappe.get_meta(doctype).get_field(fieldname) File "/home/frappe/frappe-bench/apps/frappe/frappe/__init__.py", line 917, in get_meta return frappe.model.meta.get_meta(doctype, cached=cached) File "/home/frappe/frappe-bench/apps/frappe/frappe/model/meta.py", line 37, in get_meta meta = Meta(doctype) File "/home/frappe/frappe-bench/apps/frappe/frappe/model/meta.py", line 83, in __init__ super(Meta, self).__init__("DocType", doctype) File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 113, in __init__ self.load_from_db() File "/home/frappe/frappe-bench/apps/frappe/frappe/model/meta.py", line 88, in load_from_db super(Meta, self).load_from_db() File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 156, in load_from_db frappe.throw(_("{0} {1} not found").format(_(self.doctype), self.name), frappe.DoesNotExistError) File "/home/frappe/frappe-bench/apps/frappe/frappe/__init__.py", line 439, in throw msgprint(msg, raise_exception=exc, title=title, indicator='red', is_minimizable=is_minimizable, wide=wide, as_list=as_list) File "/home/frappe/frappe-bench/apps/frappe/frappe/__init__.py", line 418, in msgprint _raise_exception() File "/home/frappe/frappe-bench/apps/frappe/frappe/__init__.py", line 372, in _raise_exception raise raise_exception(msg) frappe.exceptions.DoesNotExistError: DocType E Commerce Settings not found An error occurred while installing erpnext: DocType E Commerce Settings not found ``` </details> <details><summary>Use Redisearch with consent</summary> - A system could have had redisearch installed for other reasons (not for erpnext) - It is wrong to assume that if redisearch is just installed we must use it for searches - Also if something goes wrong with redisearch there should be a way to disable it temporarily </details> <details><summary>Product category results in Search was broken</summary> - The Product Category results in search had no redirect attached to it. - This is because the redisearch only returned the item group names, no metadata - Also, item group suggestion generation was redundantly done again and again </details> <details><summary>Useless bool return values from functions and absence of exception handling</summary> </details> ## Fix: - App install - Once Redisearch is installed, the **Item Search Settings** will not be read only. Users must then **enable Redisearch in the Settings**. This can be disabled too (normal db search will be used) <img width="1287" alt="Screenshot 2022-04-04 at 1 44 35 PM" src="https://user-images.githubusercontent.com/25857446/161502418-445b61d7-fc1a-47e7-8725-88dca7ca41bb.png"> - **Redisearch will be used if it is installed/loaded AND enabled in settings** - Item group suggestions in Search is functional: ![2022-04-04 13 48 15](https://user-images.githubusercontent.com/25857446/161503139-3e253e46-4736-483c-8ae3-17da93fa22ea.gif) - Added error logging for exceptions and left comments where we explicitly ignore the exception - Item Group suggestions are separately generated with payload(extra info) and weightage as well - Minor code cleanup > Skipping tests because cannot write tests as it needs a script to install RS. Will add during app separation

Merging this should get it working on FC and even otherwise. A handful of bug and design fixes.