Can't reach mysql server after mariadb upgrade in VM

Please advise:

I updated the VM .ova instance with
sudo apt-get update
sudo apt-get upgrade

the system upgraded the mariadb to v.10.

Now after reboot
run the command:
bench update
with the following result:

INFO:bench.utils:updating bench
INFO:bench.utils:git pull
Already up-to-date.
INFO:bench.utils:./env/bin/pip install Pillow
Requirement already satisfied: Pillow in ./env/lib/python2.7/site-packages
Requirement already satisfied: olefile in ./env/lib/python2.7/site-packages (from Pillow)
INFO:bench.app:pulling frappe
INFO:bench.utils:git pull upstream master
From https://github.com/frappe/frappe

  • branch master -> FETCH_HEAD
    Already up-to-date.
    INFO:bench.utils:find . -name “*.pyc” -delete
    INFO:bench.app:pulling erpnext
    INFO:bench.utils:git pull upstream master
    From https://github.com/frappe/erpnext
  • branch master -> FETCH_HEAD
    Already up-to-date.
    INFO:bench.utils:find . -name “*.pyc” -delete
    INFO:bench.app:pulling erpnext_demo
    INFO:bench.utils:git pull upstream develop
    From https://github.com/frappe/erpnext_demo
  • branch develop -> FETCH_HEAD
    Already up-to-date.
    INFO:bench.utils:find . -name “*.pyc” -delete
    Updating Python libraries…
    INFO:bench.utils:./env/bin/pip install --upgrade pip
    Requirement already up-to-date: pip in ./env/lib/python2.7/site-packages
    INFO:bench.utils:./env/bin/pip install -q -r /home/frappe/bench-repo/requirements.txt
    INFO:bench.utils:./env/bin/pip install -q -r ./apps/erpnext/requirements.txt
    INFO:bench.utils:./env/bin/pip install -q -r ./apps/erpnext_demo/requirements.txt
    INFO:bench.utils:./env/bin/pip install -q -r ./apps/frappe/requirements.txt
    Updating node libraries…
    INFO:bench.utils:npm install
    npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@^1.0.0 (node_modules/chokidar/node_modules/fsevents):
    npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.1.2: wanted {“os”:“darwin”,“arch”:“any”} (current: {“os”:“linux”,“arch”:“ia32”})
    npm WARN frappe@ No description
    Backing up sites…
    Traceback (most recent call last):
    File “/usr/lib/python2.7/runpy.py”, line 162, in _run_module_as_main
    main”, fname, loader, pkg_name)
    File “/usr/lib/python2.7/runpy.py”, line 72, in _run_code
    exec code in run_globals
    File “/home/frappe/frappe-bench/apps/frappe/frappe/utils/bench_helper.py”, line 94, in
    main()
    File “/home/frappe/frappe-bench/apps/frappe/frappe/utils/bench_helper.py”, line 18, in main
    click.Group(commands=commands)(prog_name=‘bench’)
    File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py”, line 722, in call
    return self.main(*args, **kwargs)
    File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py”, line 697, in main
    rv = self.invoke(ctx)
    File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py”, line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
    File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py”, line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
    File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py”, line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
    File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py”, line 535, in invoke
    return callback(*args, **kwargs)
    File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/click/decorators.py”, line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
    File “/home/frappe/frappe-bench/apps/frappe/frappe/commands/init.py”, line 24, in _func
    ret = f(frappe.dict(ctx.obj), *args, **kwargs)
    File “/home/frappe/frappe-bench/apps/frappe/frappe/commands/site.py”, line 289, in backup
    odb = scheduled_backup(ignore_files=not with_files, backup_path_db=backup_path_db, backup_path_files=backup_path_files, backup_path_private_files=backup_path_private_files, force=True)
    File “/home/frappe/frappe-bench/apps/frappe/frappe/utils/backups.py”, line 166, in scheduled_backup
    odb = new_backup(older_than, ignore_files, backup_path_db=backup_path_db, backup_path_files=backup_path_files, force=force)
    File “/home/frappe/frappe-bench/apps/frappe/frappe/utils/backups.py”, line 176, in new_backup
    odb.get_backup(older_than, ignore_files, force=force)
    File “/home/frappe/frappe-bench/apps/frappe/frappe/utils/backups.py”, line 48, in get_backup
    self.set_backup_file_name()
    File “/home/frappe/frappe-bench/apps/frappe/frappe/utils/backups.py”, line 63, in set_backup_file_name
    todays_date = now_datetime().strftime(’%Y%m%d
    %H%M%S’)
    File “/home/frappe/frappe-bench/apps/frappe/frappe/utils/data.py”, line 122, in now_datetime
    dt = convert_utc_to_user_timezone(datetime.datetime.utcnow())
    File “/home/frappe/frappe-bench/apps/frappe/frappe/utils/data.py”, line 145, in convert_utc_to_user_timezone
    return utcnow.astimezone(timezone(get_time_zone()))
    File “/home/frappe/frappe-bench/apps/frappe/frappe/utils/data.py”, line 139, in get_time_zone
    return frappe.cache().get_value(“time_zone”, _get_time_zone)
    File “/home/frappe/frappe-bench/apps/frappe/frappe/utils/redis_wrapper.py”, line 72, in get_value
    val = generator()
    File “/home/frappe/frappe-bench/apps/frappe/frappe/utils/data.py”, line 133, in _get_time_zone
    return frappe.db.get_system_setting(‘time_zone’) or ‘Asia/Kolkata’
    File “/home/frappe/frappe-bench/apps/frappe/frappe/database.py”, line 859, in get_system_setting
    return frappe.cache().get_value(“system_settings”, _load_system_settings).get(key)
    File “/home/frappe/frappe-bench/apps/frappe/frappe/utils/redis_wrapper.py”, line 72, in get_value
    val = generator()
    File “/home/frappe/frappe-bench/apps/frappe/frappe/database.py”, line 858, in _load_system_settings
    return self.get_singles_dict(“System Settings”)
    File “/home/frappe/frappe-bench/apps/frappe/frappe/database.py”, line 543, in get_singles_dict
    tabSingles where doctype=%s""", doctype))
    File “/home/frappe/frappe-bench/apps/frappe/frappe/database.py”, line 121, in sql
    self.connect()
    File “/home/frappe/frappe-bench/apps/frappe/frappe/database.py”, line 67, in connect
    use_unicode=True, charset=‘utf8mb4’)
    File “/home/frappe/frappe-bench/env/src/mysqlclient/MySQLdb/init.py”, line 86, in Connect
    return Connection(*args, **kwargs)
    File “/home/frappe/frappe-bench/env/src/mysqlclient/MySQLdb/connections.py”, line 204, in init
    super(Connection, self).init(*args, **kwargs2)
    _mysql_exceptions.OperationalError: (2002, ‘Can’t connect to local MySQL server through socket ‘/var/run/mysqld/mysqld.sock’ (2 “No such file or directory”)’)

Can’t even restart the database server.

sudo /etc/init.d/mysql start
fail

The only way is
bench reinstall
and
do a
restore from backup?

Please advise.

Well you know there is an issue with your MySql

First check the error logs of your MySQL in
/var/logs/

1 Like

Thank you for the fast response.

I checked again:
mysql.log - nothing 0 byte
mysql.err - nothin 0 byte
mysql.log.1.gz -nothing in it 20byte

my.cnf

[mysqld]

Misc Settings

-------------

datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock

skip-external-locking
table_cache=2048
thread_cache_size=16
back_log=100
max_connect_errors=10000
open-files-limit=20000
interactive_timeout=600
wait_timeout=600
max_connections=300

Disabling symbolic-links is recommended to prevent assorted security risks

symbolic-links=0

Global, Non Engine-Specific Buffers

-----------------------------------

max_allowed_packet=16M
tmp_table_size=64M
max_heap_table_size=64M

Generally, it is unwise to set the query cache to be

larger than 64-128M as this can decrease performance

since the penalty for flushing the cache can become

significant.

query_cache_size=32M

Per-Thread Buffers

------------------

sort_buffer_size=1M
read_buffer_size=1M
read_rnd_buffer_size=8M
join_buffer_size=1M

__________________________

#< Engine Specific Settings >

--------------------------

Set this to force MySQL to use a particular engine / table-type

for new tables. This setting can still be overridden by specifying

the engine explicitly in the CREATE TABLE statement.

default-storage-engine=InnoDB

MyISAM

------

Not sure what to set this to?

Try running a ‘du -sch /var/lib/mysql//.MYI’

This will give you a good estiamte on the size of all the MyISAM indexes.

(The buffer may not need to set that high, however)

key_buffer_size=64M

This setting controls the size of the buffer that is allocated when

sorting MyISAM indexes during a REPAIR TABLE or when creating indexes

with CREATE INDEX or ALTER TABLE.

myisam_sort_buffer_size=64M

InnoDB

------

Note: While most settings in MySQL can be set at run-time, InnoDB

variables require restarting MySQL to apply.

If the customer already has InnoDB tables and wants to change the

size of the InnoDB tablespace and InnoDB logs, then:

1. Run a full backup with mysqldump

2. Stop MySQL

3. Move current ibdata and ib_logfiles out of /var/lib/mysql

4. Uncomment the below innodb_data_file_path and innodb_log_file_size

5. Start MySQL (it will recreate new InnoDB files)

6. Restore data from backup

#innodb_data_file_path=ibdata1:2000M;ibdata2:10M:autoextend
#innodb_log_file_size=100M

InnoDB loves RAM. For customers using mostly InnoDB, consider setting

this variable somewhat liberally. Do make sure the server is 64-bit,

however, if you plan on setting it above 1.5-2GB.

innodb_buffer_pool_size=4G

#innodb_additional_mem_pool_size=20M

innodb_file_per_table can be useful in certain cases

but be aware that it does not scale well with a

large number of tables!

#innodb_file_per_table=1

Somewhat equivalent to table_cache for InnoDB

when using innodb_file_per_table.

#innodb_open_files=1024

If you are not sure what to set this two, the

following formula can offer up a rough idea:

(number of cpus * number of disks * 2)

#innodb_thread_concurrency=16

_____________________________

#< Replication/Backup Settings >

-----------------------------

server-id=16

It may be wise to include the server-name or an easy identifier

to the server in these logs, such as ‘log-bin=/var/lib/mysqllogs/db1-bin-log’

log-bin=/var/lib/mysqllogs/bin-log
log-bin-index=/var/lib/mysqllogs/bin-log.index

#relay-log=/var/lib/mysqllogs/relay-log
#relay-log-index=/var/lib/mysqllogs/relay-log.index

MySQL 5.0+ Only

expire_logs_days=60

This is usually only needed when setting up chained replication.

#log-slave-updates

Enable this to make replication more resilient against server

crashes and restarts, but can cause higher I/O on the server.

sync_binlog=1

MySQL 5.0+ Only

Uncomment the following when enabling multi-master replication

Do NOT uncomment these unless you know exactly what you are doing!

#auto_increment_increment=2
#auto_increment_offset=1

_________________

#< Script Settings >

-----------------

(You probably do not need to change these)

[mysql.server]
user=mysql
log-error=/var/log/mysqlserver.log
#basedir=/var/lib

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
open-files-limit=65535


Any new proposal?

There is no

mysql.sock
in
/var/lib/mysql

frappe@erpnext:/var/lib/mysql$ dir
aria_log.00000001 mysql-bin.000089 mysql-bin.000099 mysql-bin.000109
aria_log_control mysql-bin.000090 mysql-bin.000100 mysql-bin.000110
d56cb677eaab3383 mysql-bin.000091 mysql-bin.000101 mysql-bin.000111
debian-10.0.flag mysql-bin.000092 mysql-bin.000102 mysql-bin.000112
ibdata1 mysql-bin.000093 mysql-bin.000103 mysql-error.log
ib_logfile0 mysql-bin.000094 mysql-bin.000104 mysql-slow.log
ib_logfile1 mysql-bin.000095 mysql-bin.000105 mysql_upgrade_info
multi-master.info mysql-bin.000096 mysql-bin.000106 performance_schema
mysql mysql-bin.000097 mysql-bin.000107
mysql-bin.000088 mysql-bin.000098 mysql-bin.000108

eek TMI to grok krnkris

A search suggests this if you have no done so
https://stackoverflow.com/questions/5376427/cant-connect-to-local-mysql-server-through-socket-var-mysql-mysql-sock-38

@clarkej

Thanks to taking time for this.
I already done these, no success.

frappe@erpnext:/var/log$ mysqladmin -u root -p status
Enter password:
mysqladmin: connect to server at ‘localhost’ failed
error: ‘Can’t connect to local MySQL server through socket ‘/var/run/mysqld/mysqld.sock’ (2 “No such file or directory”)’
Check that mysqld is running and that the socket: ‘/var/run/mysqld/mysqld.sock’ exists!

After the mariadb update my
/var/run/mysqld/
is empty.

Do you have /var/run/mysqld/mysqld.sock
in your .ova VM instance?

“Do you have /var/run/mysqld/mysqld.sock
in your .ova VM instance?”

yes mine works

other ideas
reinstall your db or downgrade
compare cnf with another vm instance that works

@clarkej

Thank you, will try.

Product is working,
waiting for mariadb upgrade.

Will compare.

Takes some time.

Two ideas -

  1. NAT or bridged network config between guest VM and VB host is broken

  2. Your my.cnf does not list port or bind-address -

Mine here /etc/mysql/my.cnf lists those for eg

[client]
port = 3306
socket = /var/run/mysqld/mysqld.sock

Here is entries for some specific programs

The following values assume you have at least 32M ram

This was formally known as [safe_mysqld]. Both versions are currently parsed.

[mysqld_safe]
socket = /var/run/mysqld/mysqld.sock
nice = 0

[mysqld]

* Basic Settings

user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306

I found out
apt-get update Did not run fully

In this case I had to reinstall
Then restore.

Please clos this thread.

Thanks for the help.