Too many keys specified; max 64 keys allowed

Frappe Framework App Development
1

I have made changes on Strudent Applicant. When I push to production server and do bench update, I get the error pymysql.err.InternalError: (1069, ‘Too many keys specified; max 64 keys allowed’)

When I check indexes on table tabStudent Applicant, I have too many duplicate indexes created like

student_email_id_9
student_email_id_10
student_email_id_11
student_email_id_12
etc
Delete the indexes and updating again doesnot help since they get recreated in the process. How could I solve this?

2

What are these changes ? Elaborate - did you go via the Customize Form route or Edit Doctype ?

Push to production server - means you committed these changes and then pulled them onto your server. Why running bench update ?

3

Changes are done through Customize Form. I just added 2 custom fields

I commited the changes in development server, saved the changes in a custom app and pushed to git repo. Then dit bench update to pull the changes to the production server, resulting into that error
Below is the full stack

 File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/utils/bench_helper.py", line 97, in <module>
    main()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/utils/bench_helper.py", line 18, in main
    click.Group(commands=commands)(prog_name='bench')
  File "/home/frappe/frappe-bench/env/lib/python3.5/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/frappe/frappe-bench/env/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/frappe/frappe-bench/env/lib/python3.5/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/frappe/frappe-bench/env/lib/python3.5/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/frappe/frappe-bench/env/lib/python3.5/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/frappe/frappe-bench/env/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/frappe/frappe-bench/env/lib/python3.5/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/commands/__init__.py", line 25, in _func
    ret = f(frappe._dict(ctx.obj), *args, **kwargs)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/commands/site.py", line 223, in migrate
    migrate(context.verbose, rebuild_website=rebuild_website)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/migrate.py", line 43, in migrate
    sync_fixtures()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/utils/fixtures.py", line 24, in sync_fixtures
    ignore_links=True, overwrite=True)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/core/doctype/data_import/data_import.py", line 54, in import_doc
    frappe.modules.import_file.import_file_by_path(f, data_import=True, force=True, pre_process=pre_process, reset_permissions=True)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/modules/import_file.py", line 65, in import_file_by_path
    ignore_version=ignore_version, reset_permissions=reset_permissions)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/modules/import_file.py", line 133, in import_doc
    doc.insert()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 249, in insert
    self.run_post_save_methods()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 907, in run_post_save_methods
    self.run_method("on_update")
  File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 774, in run_method
    out = Document.hook(fn)(self, *args, **kwargs)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 1050, in composer
    return composed(self, method, *args, **kwargs)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 1033, in runner
    add_to_return_value(self, fn(self, *args, **kwargs))
  File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 768, in <lambda>
    fn = lambda self, *args, **kwargs: getattr(self, method)(*args, **kwargs)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/custom/doctype/custom_field/custom_field.py", line 66, in on_update
    frappe.db.updatedb(self.dt)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/database/mariadb/database.py", line 278, in updatedb
    db_table.sync()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/database/schema.py", line 36, in sync
    self.alter()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/database/mariadb/schema.py", line 86, in alter
    raise e
  File "/home/frappe/frappe-bench/apps/frappe/frappe/database/mariadb/schema.py", line 75, in alter
    frappe.db.sql(query)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/database/database.py", line 165, in sql
    self._cursor.execute(query)
  File "/home/frappe/frappe-bench/env/lib/python3.5/site-packages/pymysql/cursors.py", line 170, in execute
    result = self._query(query)
  File "/home/frappe/frappe-bench/env/lib/python3.5/site-packages/pymysql/cursors.py", line 328, in _query
    conn.query(q)
  File "/home/frappe/frappe-bench/env/lib/python3.5/site-packages/pymysql/connections.py", line 516, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "/home/frappe/frappe-bench/env/lib/python3.5/site-packages/pymysql/connections.py", line 727, in _read_query_result
    result.read()
  File "/home/frappe/frappe-bench/env/lib/python3.5/site-packages/pymysql/connections.py", line 1066, in read
    first_packet = self.connection._read_packet()
  File "/home/frappe/frappe-bench/env/lib/python3.5/site-packages/pymysql/connections.py", line 683, in _read_packet
    packet.check_error()
  File "/home/frappe/frappe-bench/env/lib/python3.5/site-packages/pymysql/protocol.py", line 220, in check_error
    err.raise_mysql_exception(self._data)
  File "/home/frappe/frappe-bench/env/lib/python3.5/site-packages/pymysql/err.py", line 109, in raise_mysql_exception
    raise errorclass(errno, errval)
pymysql.err.InternalError: (1069, 'Too many keys specified; max 64 keys allowed')
4

DId you export fixtures ? Check if Custom Field is included in fixtures list in hooks.py of your custom app. Also while you are at it, also include Property Setter.

Also, bench update is not required for pulling changes. you can do a simple git pull in your custom app and you’ll get the changes. bench update will also update erpnext and frappe from the original repo.

Pull your changes via git and then do bench migrate to sync changes onto your production server and then bench restart

5

Yes, I had exported fixtures and Custom Field was already in. Just added Property Setter too

Thank you for the hint, it will save me time in future

When I check indexes on the Student Applicant, I get a lot of repeating indexes, as if there was some endless loop somewhere.

indexes
indexes868×582 98.3 KB

1 Like
6

Hi @RWEMA_Aimable,

If you have done changes using Edit Doctype, then it you should have changes in json file in doctype folder. If you have done using customize, then in those cases you will have to export fixtures and import them.

In Both cases, you should push the code from your local, and should pull on production.

Instead of $bench update
try

$bench migrate
$bench clear-cache
$bench build
$bench restart

The $bench update will pull code from upstream other things as well incuding bench, frappe and erpnext app. If you have customized code in ERPNext app, then it must be in your origin branch or your fork.

Hope this was helpful. Let me know if this was not what you were looking for :slight_smile:

Regards,

Parth

1 Like
7

bench migrate also leads to the same error.

8

I also made a new site with an old DB dump I knew was working and still the same problem

9

These fields mentioned payment_slip_number and email_id are these fields from Student Applicant. Maybe there could be some property that is doing this.

10

Yes, I added these fields using Custom Field method, not Custimize Form way. Now that I tink of it, maybe it is not a good idea to mix Custom Field and Customize Form? Let delete the custom field and confirm

11

yes, don’t use it directly. TO add custom fields use Customize Form.

And try it on your development server first.

12

I finally solved it by initialising a new bench. Thank you all for your support

13

The problem occuring again on a new installation

  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/frappe/uoh/apps/frappe/frappe/utils/bench_helper.py", line 97, in <module>
    main()
  File "/home/frappe/uoh/apps/frappe/frappe/utils/bench_helper.py", line 18, in main
    click.Group(commands=commands)(prog_name='bench')
  File "/home/frappe/uoh/env/local/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/frappe/uoh/env/local/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/frappe/uoh/env/local/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/frappe/uoh/env/local/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/frappe/uoh/env/local/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/frappe/uoh/env/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/frappe/uoh/env/local/lib/python2.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/frappe/uoh/apps/frappe/frappe/commands/__init__.py", line 25, in _func
    ret = f(frappe._dict(ctx.obj), *args, **kwargs)
  File "/home/frappe/uoh/apps/frappe/frappe/commands/site.py", line 223, in migrate
    migrate(context.verbose, rebuild_website=rebuild_website)
  File "/home/frappe/uoh/apps/frappe/frappe/migrate.py", line 43, in migrate
    sync_fixtures()
  File "/home/frappe/uoh/apps/frappe/frappe/utils/fixtures.py", line 24, in sync_fixtures
    ignore_links=True, overwrite=True)
  File "/home/frappe/uoh/apps/frappe/frappe/core/doctype/data_import/data_import.py", line 54, in import_doc
    frappe.modules.import_file.import_file_by_path(f, data_import=True, force=True, pre_process=pre_process, reset_permissions=True)
  File "/home/frappe/uoh/apps/frappe/frappe/modules/import_file.py", line 65, in import_file_by_path
    ignore_version=ignore_version, reset_permissions=reset_permissions)
  File "/home/frappe/uoh/apps/frappe/frappe/modules/import_file.py", line 133, in import_doc
    doc.insert()
  File "/home/frappe/uoh/apps/frappe/frappe/model/document.py", line 249, in insert
    self.run_post_save_methods()
  File "/home/frappe/uoh/apps/frappe/frappe/model/document.py", line 907, in run_post_save_methods
    self.run_method("on_update")
  File "/home/frappe/uoh/apps/frappe/frappe/model/document.py", line 774, in run_method
    out = Document.hook(fn)(self, *args, **kwargs)
  File "/home/frappe/uoh/apps/frappe/frappe/model/document.py", line 1050, in composer
    return composed(self, method, *args, **kwargs)
  File "/home/frappe/uoh/apps/frappe/frappe/model/document.py", line 1033, in runner
    add_to_return_value(self, fn(self, *args, **kwargs))
  File "/home/frappe/uoh/apps/frappe/frappe/model/document.py", line 768, in <lambda>
    fn = lambda self, *args, **kwargs: getattr(self, method)(*args, **kwargs)
  File "/home/frappe/uoh/apps/frappe/frappe/custom/doctype/custom_field/custom_field.py", line 66, in on_update
    frappe.db.updatedb(self.dt)
  File "/home/frappe/uoh/apps/frappe/frappe/database/mariadb/database.py", line 278, in updatedb
    db_table.sync()
  File "/home/frappe/uoh/apps/frappe/frappe/database/schema.py", line 36, in sync
    self.alter()
  File "/home/frappe/uoh/apps/frappe/frappe/database/mariadb/schema.py", line 86, in alter
    raise e
pymysql.err.InternalError: (1069, u'Too many keys specified; max 64 keys allowed')
14

Finally solved by creating a new site on a new bench

1 Like