Automatically stop the server

Hello,

I have one issue related to the bench start.
I have completed the setup process of frappe and erpnext in my system and when I enter the bench start command to start the server then immediately stops the server.

Can you please help me in this problem?

sudo service supervisor stop
sudo service nginx stop
bench start

Same issue arrives

share logs

ok sure

15:25:58 system           | worker_default.1 started (pid=10852)
15:25:58 system           | redis_socketio.1 started (pid=10837)
15:25:58 redis_socketio.1 | 10843:M 10 Sep 15:25:58.778 * Increased maximum number of open files to 10032 (it was originally set to 1024).
15:25:58 redis_socketio.1 |                 _._                                                  
15:25:58 redis_socketio.1 |            _.-``__ ''-._                                             
15:25:58 redis_socketio.1 |       _.-``    `.  `_.  ''-._           Redis 3.0.6 (00000000/0) 64 bit
15:25:58 redis_socketio.1 |   .-`` .-```.  ```\/    _.,_ ''-._                                   
15:25:58 redis_socketio.1 |  (    '      ,       .-`  | `,    )     Running in standalone mode
15:25:58 redis_socketio.1 |  |`-._`-...-` __...-.``-._|'` _.-'|     Port: 12000
15:25:58 redis_socketio.1 |  |    `-._   `._    /     _.-'    |     PID: 10843
15:25:58 redis_socketio.1 |   `-._    `-._  `-./  _.-'    _.-'                                   
15:25:58 redis_socketio.1 |  |`-._`-._    `-.__.-'    _.-'_.-'|                                  
15:25:58 redis_socketio.1 |  |    `-._`-._        _.-'_.-'    |           http://redis.io        
15:25:58 redis_socketio.1 |   `-._    `-._`-.__.-'_.-'    _.-'                                   
15:25:58 redis_socketio.1 |  |`-._`-._    `-.__.-'    _.-'_.-'|                                  
15:25:58 redis_socketio.1 |  |    `-._`-._        _.-'_.-'    |                                  
15:25:58 redis_socketio.1 |   `-._    `-._`-.__.-'_.-'    _.-'                                   
15:25:58 redis_socketio.1 |       `-._    `-.__.-'    _.-'                                       
15:25:58 redis_socketio.1 |           `-._        _.-'                                           
15:25:58 redis_socketio.1 |               `-.__.-'                                               
15:25:58 redis_socketio.1 | 
15:25:58 redis_socketio.1 | 10843:M 10 Sep 15:25:58.778 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
15:25:58 redis_socketio.1 | 10843:M 10 Sep 15:25:58.778 # Server started, Redis version 3.0.6
15:25:58 redis_socketio.1 | 10843:M 10 Sep 15:25:58.778 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
15:25:58 redis_socketio.1 | 10843:M 10 Sep 15:25:58.778 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
15:25:58 redis_socketio.1 | 10843:M 10 Sep 15:25:58.778 * The server is now ready to accept connections on port 12000
15:25:58 system           | worker_long.1 started (pid=10839)
15:25:58 system           | socketio.1 started (pid=10850)
15:25:58 system           | redis_queue.1 started (pid=10846)
15:25:58 redis_queue.1    | 10861:M 10 Sep 15:25:58.794 * Increased maximum number of open files to 10032 (it was originally set to 1024).
15:25:58 redis_queue.1    |                 _._                                                  
15:25:58 redis_queue.1    |            _.-``__ ''-._                                             
15:25:58 redis_queue.1    |       _.-``    `.  `_.  ''-._           Redis 3.0.6 (00000000/0) 64 bit
15:25:58 redis_queue.1    |   .-`` .-```.  ```\/    _.,_ ''-._                                   
15:25:58 redis_queue.1    |  (    '      ,       .-`  | `,    )     Running in standalone mode
15:25:58 redis_queue.1    |  |`-._`-...-` __...-.``-._|'` _.-'|     Port: 11000
15:25:58 redis_queue.1    |  |    `-._   `._    /     _.-'    |     PID: 10861
15:25:58 redis_queue.1    |   `-._    `-._  `-./  _.-'    _.-'                                   
15:25:58 redis_queue.1    |  |`-._`-._    `-.__.-'    _.-'_.-'|                                  
15:25:58 redis_queue.1    |  |    `-._`-._        _.-'_.-'    |           http://redis.io        
15:25:58 redis_queue.1    |   `-._    `-._`-.__.-'_.-'    _.-'                                   
15:25:58 redis_queue.1    |  |`-._`-._    `-.__.-'    _.-'_.-'|                                  
15:25:58 redis_queue.1    |  |    `-._`-._        _.-'_.-'    |                                  
15:25:58 redis_queue.1    |   `-._    `-._`-.__.-'_.-'    _.-'                                   
15:25:58 redis_queue.1    |       `-._    `-.__.-'    _.-'                                       
15:25:58 redis_queue.1    |           `-._        _.-'                                           
15:25:58 redis_queue.1    |               `-.__.-'                                               
15:25:58 redis_queue.1    | 
15:25:58 redis_queue.1    | 10861:M 10 Sep 15:25:58.794 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
15:25:58 redis_queue.1    | 10861:M 10 Sep 15:25:58.794 # Server started, Redis version 3.0.6
15:25:58 redis_queue.1    | 10861:M 10 Sep 15:25:58.795 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
15:25:58 redis_queue.1    | 10861:M 10 Sep 15:25:58.795 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
15:25:58 redis_queue.1    | 10861:M 10 Sep 15:25:58.795 * The server is now ready to accept connections on port 11000
15:25:58 system           | watch.1 started (pid=10840)
15:25:58 system           | worker_short.1 started (pid=10868)
15:25:58 system           | redis_cache.1 started (pid=10878)
15:25:58 system           | schedule.1 started (pid=10870)
15:25:58 redis_cache.1    | 10879:M 10 Sep 15:25:58.820 * Increased maximum number of open files to 10032 (it was originally set to 1024).
15:25:58 redis_cache.1    |                 _._                                                  
15:25:58 redis_cache.1    |            _.-``__ ''-._                                             
15:25:58 redis_cache.1    |       _.-``    `.  `_.  ''-._           Redis 3.0.6 (00000000/0) 64 bit
15:25:58 redis_cache.1    |   .-`` .-```.  ```\/    _.,_ ''-._                                   
15:25:58 redis_cache.1    |  (    '      ,       .-`  | `,    )     Running in standalone mode
15:25:58 redis_cache.1    |  |`-._`-...-` __...-.``-._|'` _.-'|     Port: 13000
15:25:58 redis_cache.1    |  |    `-._   `._    /     _.-'    |     PID: 10879
15:25:58 redis_cache.1    |   `-._    `-._  `-./  _.-'    _.-'                                   
15:25:58 redis_cache.1    |  |`-._`-._    `-.__.-'    _.-'_.-'|                                  
15:25:58 redis_cache.1    |  |    `-._`-._        _.-'_.-'    |           http://redis.io        
15:25:58 redis_cache.1    |   `-._    `-._`-.__.-'_.-'    _.-'                                   
15:25:58 redis_cache.1    |  |`-._`-._    `-.__.-'    _.-'_.-'|                                  
15:25:58 redis_cache.1    |  |    `-._`-._        _.-'_.-'    |                                  
15:25:58 redis_cache.1    |   `-._    `-._`-.__.-'_.-'    _.-'                                   
15:25:58 redis_cache.1    |       `-._    `-.__.-'    _.-'                                       
15:25:58 redis_cache.1    |           `-._        _.-'                                           
15:25:58 redis_cache.1    |               `-.__.-'                                               
15:25:58 redis_cache.1    | 
15:25:58 redis_cache.1    | 10879:M 10 Sep 15:25:58.820 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
15:25:58 redis_cache.1    | 10879:M 10 Sep 15:25:58.820 # Server started, Redis version 3.0.6
15:25:58 redis_cache.1    | 10879:M 10 Sep 15:25:58.820 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
15:25:58 redis_cache.1    | 10879:M 10 Sep 15:25:58.820 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
15:25:58 redis_cache.1    | 10879:M 10 Sep 15:25:58.820 * The server is now ready to accept connections on port 13000
15:25:58 system           | web.1 started (pid=10851)
15:25:59 socketio.1       | listening on *: 9000
15:26:01 web.1            |  * Running on http://0.0.0.0:8000/ (Press CTRL+C to quit)
15:26:02 web.1            |  * Restarting with inotify reloader
15:26:02 web.1            |  * Debugger is active!
15:26:02 web.1            |  * Debugger PIN: 173-715-838
15:26:02 watch.1          | SyntaxError: Unexpected end of JSON input
15:26:02 watch.1          |     at JSON.parse (<anonymous>)
15:26:02 watch.1          |     at make_build_map (/home/serpentcs/workspace/myprojects/erp_next/frappe-bench/apps/frappe/frappe/build.js:193:22)
15:26:02 watch.1          |     at Object.<anonymous> (/home/serpentcs/workspace/myprojects/erp_next/frappe-bench/apps/frappe/frappe/build.js:22:17)
15:26:02 watch.1          |     at Module._compile (module.js:652:30)
15:26:02 watch.1          |     at Object.Module._extensions..js (module.js:663:10)
15:26:02 watch.1          |     at Module.load (module.js:565:32)
15:26:02 watch.1          |     at tryModuleLoad (module.js:505:12)
15:26:02 watch.1          |     at Function.Module._load (module.js:497:3)
15:26:02 watch.1          |     at Function.Module.runMain (module.js:693:10)
15:26:02 watch.1          |     at startup (bootstrap_node.js:191:16)
15:26:02 watch.1          | SyntaxError: Unexpected end of JSON input
15:26:02 watch.1          |     at JSON.parse (<anonymous>)
15:26:02 watch.1          |     at make_build_map (/home/serpentcs/workspace/myprojects/erp_next/frappe-bench/apps/frappe/frappe/build.js:193:22)
15:26:02 watch.1          |     at Object.<anonymous> (/home/serpentcs/workspace/myprojects/erp_next/frappe-bench/apps/frappe/frappe/build.js:22:17)
15:26:02 watch.1          |     at Module._compile (module.js:652:30)
15:26:02 watch.1          |     at Object.Module._extensions..js (module.js:663:10)
15:26:02 watch.1          |     at Module.load (module.js:565:32)
15:26:02 watch.1          |     at tryModuleLoad (module.js:505:12)
15:26:02 watch.1          |     at Function.Module._load (module.js:497:3)
15:26:02 watch.1          |     at Function.Module.runMain (module.js:693:10)
15:26:02 watch.1          |     at startup (bootstrap_node.js:191:16)
15:26:02 watch.1          | compiling avatar.less
15:26:02 watch.1          | compiling calendar.less
15:26:02 watch.1          | compiling common.less
15:26:02 watch.1          | compiling controls.less
15:26:02 watch.1          | compiling desk.less
15:26:02 watch.1          | compiling desktop.less
15:26:02 watch.1          | compiling docs.less
15:26:02 watch.1          | compiling email.less
15:26:02 watch.1          | compiling form.less
15:26:02 watch.1          | compiling form_grid.less
15:26:02 watch.1          | compiling gantt.less
15:26:02 watch.1          | compiling indicator.less
15:26:02 watch.1          | compiling kanban.less
15:26:02 watch.1          | compiling list.less
15:26:02 watch.1          | compiling mixins.less
15:26:02 watch.1          | compiling mobile.less
15:26:02 watch.1          | compiling module.less
15:26:02 watch.1          | compiling navbar.less
15:26:02 watch.1          | compiling page.less
15:26:02 watch.1          | compiling report.less
15:26:02 watch.1          | compiling role_editor.less
15:26:02 watch.1          | compiling sidebar.less
15:26:02 watch.1          | compiling slickgrid.less
15:26:02 watch.1          | compiling tags.less
15:26:02 watch.1          | compiling tree.less
15:26:02 watch.1          | compiling website.less
15:26:02 watch.1          | compiling email.less
15:26:02 watch.1          | compiling erpnext.less
15:26:02 watch.1          | compiling hub.less
15:26:02 watch.1          | compiling pos.less
15:26:02 watch.1          | compiling website.less
15:26:02 watch.1          | file watching on *: 6787
15:26:03 watch.1          | Less files compiled
15:26:03 watch.1          | Wrote css/frappe-web.css - 65.56 KB
15:26:03 watch.1          | Wrote js/frappe-web.min.js - 135.75 KB
15:26:03 watch.1          | Wrote js/control.min.js - 82.73 KB
15:26:03 watch.1          | Wrote js/dialog.min.js - 125.5 KB
15:26:03 watch.1          | Wrote css/desk.min.css - 311.68 KB
15:26:03 watch.1          | Wrote css/frappe-rtl.css - 32.83 KB
15:26:03 watch.1          | Wrote js/libs.min.js - 1.13 MB
15:26:03 watch.1          | Wrote js/desk.min.js - 483.29 KB
15:26:03 watch.1          | Wrote css/module.min.css - 2.19 KB
15:26:03 watch.1          | Wrote css/form.min.css - 4.59 KB
15:26:03 watch.1          | Wrote js/form.min.js - 207.04 KB
15:26:03 watch.1          | Wrote css/list.min.css - 13.8 KB
15:26:03 watch.1          | Wrote js/list.min.js - 162.92 KB
15:26:03 watch.1          | Wrote css/report.min.css - 8.48 KB
15:26:03 watch.1          | Wrote js/report.min.js - 263.71 KB
15:26:03 watch.1          | Wrote js/web_form.min.js - 248.79 KB
15:26:03 watch.1          | Wrote css/web_form.css - 24.79 KB
15:26:03 watch.1          | Wrote js/print_format_v3.min.js - 25.97 KB
15:26:03 watch.1          | Wrote css/erpnext.css - 8.11 KB
15:26:03 watch.1          | Wrote js/erpnext-web.min.js - 4.29 KB
15:26:03 watch.1          | Wrote js/erpnext.min.js - 171.76 KB
15:26:03 watch.1          | Wrote js/item-dashboard.min.js - 8.58 KB
15:26:03 watch.1          | events.js:183
15:26:03 watch.1          |       throw er; // Unhandled 'error' event
15:26:03 watch.1          |       ^
15:26:03 watch.1          | 
15:26:03 watch.1          | Error: watch /home/serpentcs/workspace/myprojects/erp_next/frappe-bench/apps/frappe/frappe/public/less ENOSPC
15:26:03 watch.1          |     at _errnoException (util.js:992:11)
15:26:03 watch.1          |     at FSWatcher.start (fs.js:1382:19)
15:26:03 watch.1          |     at Object.fs.watch (fs.js:1408:11)
15:26:03 watch.1          |     at createFsWatchInstance (/home/serpentcs/workspace/myprojects/erp_next/frappe-bench/node_modules/chokidar/lib/nodefs-handler.js:37:15)
15:26:03 watch.1          |     at setFsWatchListener (/home/serpentcs/workspace/myprojects/erp_next/frappe-bench/node_modules/chokidar/lib/nodefs-handler.js:80:15)
15:26:03 watch.1          |     at FSWatcher.NodeFsHandler._watchWithNodeFs (/home/serpentcs/workspace/myprojects/erp_next/frappe-bench/node_modules/chokidar/lib/nodefs-handler.js:228:14)
15:26:03 watch.1          |     at FSWatcher.NodeFsHandler._handleDir (/home/serpentcs/workspace/myprojects/erp_next/frappe-bench/node_modules/chokidar/lib/nodefs-handler.js:407:19)
15:26:03 watch.1          |     at FSWatcher.<anonymous> (/home/serpentcs/workspace/myprojects/erp_next/frappe-bench/node_modules/chokidar/lib/nodefs-handler.js:455:19)
15:26:03 watch.1          |     at FSWatcher.<anonymous> (/home/serpentcs/workspace/myprojects/erp_next/frappe-bench/node_modules/chokidar/lib/nodefs-handler.js:460:16)
15:26:03 watch.1          |     at FSReqWrap.oncomplete (fs.js:153:5)
15:26:03 system           | watch.1 stopped (rc=0)
15:26:03 system           | sending SIGTERM to redis_socketio.1 (pid 10837)
15:26:03 system           | sending SIGTERM to worker_long.1 (pid 10839)
15:26:03 system           | sending SIGTERM to redis_queue.1 (pid 10846)
15:26:03 system           | sending SIGTERM to web.1 (pid 10851)
15:26:03 system           | sending SIGTERM to schedule.1 (pid 10870)
15:26:03 system           | sending SIGTERM to worker_default.1 (pid 10852)
15:26:03 system           | sending SIGTERM to socketio.1 (pid 10850)
15:26:03 system           | sending SIGTERM to redis_cache.1 (pid 10878)
15:26:03 system           | sending SIGTERM to worker_short.1 (pid 10868)
15:26:03 redis_socketio.1 | 10843:signal-handler (1536573363) Received SIGTERM scheduling shutdown...
15:26:03 worker_long.1    | 15:26:03 Warm shut down requested
15:26:03 redis_queue.1    | 10861:signal-handler (1536573363) Received SIGTERM scheduling shutdown...
15:26:03 worker_default.1 | 15:26:03 Warm shut down requested
15:26:03 worker_short.1   | 15:26:03 Warm shut down requested
15:26:03 system           | socketio.1 stopped (rc=-15)
15:26:03 redis_cache.1    | 10879:signal-handler (1536573363) Received SIGTERM scheduling shutdown...
15:26:03 system           | schedule.1 stopped (rc=-15)
15:26:03 redis_socketio.1 | 10843:M 10 Sep 15:26:03.491 # User requested shutdown...
15:26:03 redis_socketio.1 | 10843:M 10 Sep 15:26:03.491 # Redis is now ready to exit, bye bye...
15:26:03 system           | redis_socketio.1 stopped (rc=-15)
15:26:03 redis_queue.1    | 10861:M 10 Sep 15:26:03.511 # User requested shutdown...
15:26:03 redis_queue.1    | 10861:M 10 Sep 15:26:03.511 # Redis is now ready to exit, bye bye...
15:26:03 system           | redis_queue.1 stopped (rc=-15)
15:26:03 system           | worker_long.1 stopped (rc=-15)
15:26:03 system           | worker_default.1 stopped (rc=-15)
15:26:03 redis_cache.1    | 10879:M 10 Sep 15:26:03.538 # User requested shutdown...
15:26:03 system           | worker_short.1 stopped (rc=-15)
15:26:03 redis_cache.1    | 10879:M 10 Sep 15:26:03.539 # Redis is now ready to exit, bye bye...
15:26:03 system           | redis_cache.1 stopped (rc=-15)
15:26:03 system           | web.1 stopped (rc=-15)

ENOSPC is an error indicating insufficient disk space

then what can i do?

You should check if you have diskspace available.

df -h | grep -v “tmp|udev”

will show you what space is available on your system

run this command first then try to bench start
echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p

4 Likes

Thank You, It works for me.

I am aware that the preferred solution to ENOSPC problems is as you have indicated, and also know it often fixes the problem… (I have used it on my system too) :sunglasses:.

I have a question about the effect on RAM - if you are able to shed some light for me please.
On a machine with limited RAM (installs often have only 1-2GB available) - what would be the “acceptable” value/compromise for inotify settings. At the commonly chosen value, it is ~50% of RAM on a 1GB/64bit system. The OS would typically use ~40% of avail RAM for caching… which leaves, in theory, only 10% for the app(s) etc.

From here…

Can you increase the watches? By how much? Any tradeoffs?
Short answer: Sure, no sweat. Go to straight to a half-million (524288) if you want…the additional memory used should be negligible on a modern system with 4GB+ of memory.
• Each used inotify watch takes up 540 bytes (32-bit system), or 1 kB (double - on 64-bit) [sources: 1, 2]
• This comes out of kernel memory, which is unswappable.
• So, assuming you set the max at 524288, and all were used (improbable), you’d be using approx. 256MB/512MB of 32-bit/64-bit kernel memory
• Note that your application will also use additional memory to keep track of the inotify handles, file/directory paths, etc. – how much depends on its design.
• What’s the max value? I guess none, in theory, as long as you have enough RAM. In practice, 524288 has been officially recommended by apps, and people have been setting it to 2 million, with the accompanying memory usage, of course.

Thanks