ERPNext Foundation ERPNext Cloud User Manual Blog Discuss Frappé* Donate

Run ui test in headless chrome mode

tests

#1

I try to run QUnit ui tests with chrome --headless.
Replace chrome_options.add_argument('--start-maximized') with chrome_options.add_argument('--headless') in /frappe/utils/selenium_testdriver.py. chromedriver and chrome successfully start with bench run-ui-tests --app frappe but I get the error:

frappe@ubuntu:~/frappe-bench$ bench run-ui-tests --app frappe
Running frappe/tests/ui/test_number_format.js...
----------------------------------------
E
======================================================================
ERROR: test_test_runner (frappe.tests.ui.test_test_runner.TestTestRunner)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/frappe/frappe-bench/apps/frappe/frappe/tests/ui/test_test_runner.py", line 40, in test_test_runner
    console = driver.get_console()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/utils/selenium_testdriver.py", line 158, in get_console
    message = literal_eval(message)
  File "/usr/lib/python2.7/ast.py", line 49, in literal_eval
    node_or_string = parse(node_or_string, mode='eval')
  File "/usr/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 1
    "quill" "Overwriting formats/link with" class MyLink extends Link{static create(e){var t=super.create(e);return e=this.sanitize(e),t.setAttribute("href",e),(e.startsWith("/")||e.indexOf(window.location.host))&&t.removeAttribute("target"),t}}
                                                ^
SyntaxError: invalid syntax

----------------------------------------------------------------------
Ran 1 test in 19.627s

FAILED (errors=1)

Has anyone tried this run mode of ui tests? Perhaps this error is not related with the headless mode?


#2

I ran my own single test with this setup by add single line to app_name/tests/ui/tests.txt and run command bench run-ui-tests --app app_name. I seen that docs created by test appear at lists. At the end of test invoked TestDriver.get_console() in module frappe.untils.selenium_testdriver that raise error

Traceback (most recent call last):
  File "/home/frappe/frappe-bench/apps/frappe/frappe/tests/ui/test_test_runner.py", line 40, in test_test_runner
    console = driver.get_console()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/utils/selenium_testdriver.py", line 158, in get_console
    message = literal_eval(message)
  File "/usr/lib/python2.7/ast.py", line 49, in literal_eval
    node_or_string = parse(node_or_string, mode='eval')
  File "/usr/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 1
    "quill" "Overwriting formats/link with" class MyLink extends Link{static create(e){var t=super.create(e);return e=this.sanitize(e),t.setAttribute("href",e),(e.startsWith("/")||e.indexOf(window.location.host))&&t.removeAttribute("target"),t}}
                                                ^
SyntaxError: invalid syntax

I try patch frappe.utils.selenium_testdriver.TestDriver.get_console with

				#message = literal_eval(message)
				if not message.startswith('"quill" "Overwriting'):
					message = literal_eval(message)

and bench run-ui-tests --app app_name successful finished, but say “OK” while the same test failed in Test Runner page.
Can anyone offer a proper solution to this problem please. Maybe my environment just have non-updated dependencies or bench run-ui-tests have some bugs?


#3

Workaround with dirty patch, but i checked it with debugger and some test case :^)
frappe.utils.selenium_testdriver.TestDriver.get_console lines 150-158

			if message and message[0] in ('"', "'"):
				# message is a quoted/escaped string			
				#message = literal_eval(message)
				if not message.startswith('"quill" "Overwriting'):
					message = literal_eval(message)
					out.append(source + ' ' + line_no)
					out.append(message)
					out.append('-'*40)

		#out.append(source + ' ' + line_no)
		#out.append(message)
		#out.append('-'*40)
		return out

frappe.tests.ui.test_test_runner.TestTestRunner.test_test_runner lines 39-48

			finally:
				#console = driver.get_console()
				#passed  = 'Test Passed' in console
				if frappe.flags.tests_verbose or not passed:
					for line in console:
						print(line)
					print('-' * 40)					
				#else:
					#self.assertTrue(passed)
				self.assertTrue(passed)			
				time.sleep(1)