Socket.error: [Errno 104] Connection reset by peer using ftplib

Hi there,

I’m getting this f**king error while retrieving directory list using ftplib client, but can’t understand the reason …

I’m on VM Ubuntu 16, Python 2.7.12 and frappe on develop:

frappe@ubuntu16-dev:~$ python -V
Python 2.7.12
frappe@ubuntu16-dev:~$ python
Python 2.7.12 (default, Jul  1 2016, 15:12:24)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ftplib
>>> ftp = ftplib.FTP('ftp.sunet.se', 'anonymous', 'anonymous@sunet.se')
>>> ftp.dir()
-rw-r--r--    1 ftp      ftp          1204 Oct 04 15:11 HEADER.html
lrwxrwxrwx    1 ftp      ftp             3 Mar 16  2010 Public -> pub
drwxr-xr-x    3 ftp      ftp            10 Nov 07 19:50 about
drwxr-xr-x   28 ftp      ftp            30 Nov 12 11:28 cdimage
drwxr-xr-x    2 ftp      ftp             3 Jun 14  2006 conspiracy
lrwxrwxrwx    1 ftp      ftp            22 Mar 16  2010 debian -> cdimage/.debian-mirror
drwxr-xr-x    5 ftp      ftp             7 Mar 13  2016 debian-backports
lrwxrwxrwx    1 ftp      ftp            16 Mar 16  2010 debian-cd -> cdimage/release/
-rw-r--r--    1 ftp      ftp           318 Mar 21  2005 favicon.ico
drwxr-xr-x   57 ftp      ftp            64 Oct 25 09:29 mirror
drwxr-xr-x    4 ftp      ftp            12 Feb 05  2016 pub
lrwxrwxrwx    1 ftp      ftp            23 Mar 16  2010 releases -> mirror/ubuntu-releases/
-rw-r--r--    1 ftp      ftp           965 Oct 12 21:30 robots.txt
lrwxrwxrwx    1 ftp      ftp            12 Aug 01 19:23 tails -> mirror/tails
lrwxrwxrwx    1 ftp      ftp            13 Mar 16  2010 ubuntu -> mirror/ubuntu
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/ftplib.py", line 534, in dir
    self.retrlines(cmd, func)
  File "/usr/lib/python2.7/ftplib.py", line 440, in retrlines
    line = fp.readline(self.maxline + 1)
  File "/usr/lib/python2.7/socket.py", line 480, in readline
    data = self._sock.recv(self._rbufsize)
socket.error: [Errno 104] Connection reset by peer

On MacOS and python 2.7.12 it’s working fine:

MacBook-Pro $ python -V
Python 2.7.12

MacBook-Pro $ python
Python 2.7.12 (default, Sep 17 2016, 18:01:33)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ftplib
>>> ftp = ftplib.FTP('ftp.sunet.se', 'anonymous', 'anonymous@sunet.se')
>>> fpt.dir()
-rw-r--r--    1 ftp      ftp          1204 Oct 04 15:11 HEADER.html
lrwxrwxrwx    1 ftp      ftp             3 Mar 16  2010 Public -> pub
drwxr-xr-x    3 ftp      ftp            10 Nov 07 19:50 about
drwxr-xr-x   28 ftp      ftp            30 Nov 12 11:28 cdimage
drwxr-xr-x    2 ftp      ftp             3 Jun 14  2006 conspiracy
lrwxrwxrwx    1 ftp      ftp            22 Mar 16  2010 debian -> cdimage/.debian-mirror
drwxr-xr-x    5 ftp      ftp             7 Mar 13  2016 debian-backports
lrwxrwxrwx    1 ftp      ftp            16 Mar 16  2010 debian-cd -> cdimage/release/
-rw-r--r--    1 ftp      ftp           318 Mar 21  2005 favicon.ico
drwxr-xr-x   57 ftp      ftp            64 Oct 25 09:29 mirror
drwxr-xr-x    4 ftp      ftp            12 Feb 05  2016 pub
lrwxrwxrwx    1 ftp      ftp            23 Mar 16  2010 releases -> mirror/ubuntu-releases/
-rw-r--r--    1 ftp      ftp           965 Oct 12 21:30 robots.txt
lrwxrwxrwx    1 ftp      ftp            12 Aug 01 19:23 tails -> mirror/tails
lrwxrwxrwx    1 ftp      ftp            13 Mar 16  2010 ubuntu -> mirror/ubuntu
>>>

As well as on VM Ubuntu 14, Python 2.7.6:

frappe@erpnext:~$ python -V
Python 2.7.6
frappe@erpnext:~$ python
Python 2.7.6 (default, Jun 22 2015, 18:00:18)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ftplib
>>> ftp = ftplib.FTP('ftp.sunet.se', 'anonymous', 'anonymous@sunet.se')
>>> ftp.dir()
-rw-r--r--    1 ftp      ftp          1204 Oct 04 15:11 HEADER.html
lrwxrwxrwx    1 ftp      ftp             3 Mar 16  2010 Public -> pub
drwxr-xr-x    3 ftp      ftp            10 Nov 07 19:50 about
drwxr-xr-x   28 ftp      ftp            30 Nov 12 11:28 cdimage
drwxr-xr-x    2 ftp      ftp             3 Jun 14  2006 conspiracy
lrwxrwxrwx    1 ftp      ftp            22 Mar 16  2010 debian -> cdimage/.debian-mirror
drwxr-xr-x    5 ftp      ftp             7 Mar 13  2016 debian-backports
lrwxrwxrwx    1 ftp      ftp            16 Mar 16  2010 debian-cd -> cdimage/release/
-rw-r--r--    1 ftp      ftp           318 Mar 21  2005 favicon.ico
drwxr-xr-x   57 ftp      ftp            64 Oct 25 09:29 mirror
drwxr-xr-x    4 ftp      ftp            12 Feb 05  2016 pub
lrwxrwxrwx    1 ftp      ftp            23 Mar 16  2010 releases -> mirror/ubuntu-releases/
-rw-r--r--    1 ftp      ftp           965 Oct 12 21:30 robots.txt
lrwxrwxrwx    1 ftp      ftp            12 Aug 01 19:23 tails -> mirror/tails
lrwxrwxrwx    1 ftp      ftp            13 Mar 16  2010 ubuntu -> mirror/ubuntu
>>>

I’m not really sure if it’s some of the installed packages from bench that differ from Ubuntu 14 or whatever …can someone help on this?

So for some reason the server abruptly quits the session once dir() completes!?

Why not just restart the ftp session to transfer files etc

This says http://stackoverflow.com/questions/20568216/python-handling-socket-error-errno-104-connection-reset-by-peer

your problem is on the server end and not your end…

@clarkej
thx for the hint, but I do not think it is a server fault ( the server i tested is ftp.sunet.se), otherwise I’d face the same error when connecting from macOS (host machine for the VMs) and from the other VM with Ubuntu 14.

I think it’s a problem on some package (socket?) installed by the frappe installer in Ubuntu 16.

@vjFaLk sorry to tag you :sweat:, do u have any idea on this?

@clarkej

Update: Created a new VM from scratch with Ubuntu 16 LTS, minimal install + openssh-server, python-minimal (aka python 2.7) and ftplib it’s working properly with no socket error:

frappe@ubuntu16-test:~$ python
Python 2.7.12 (default, Jul  1 2016, 15:12:24)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ftplib
>>> ftp = ftplib.FTP("ftp.debian.org")
>>> ftp.login()
'230 Login successful.'
>>> ftp.dir()
drwxr-xr-x    9 1176     1176         4096 Nov 17 08:57 debian
>>> ftp.cwd("/debian")
'250 Directory successfully changed.'
>>> ftp.dir()
-rw-rw-r--    1 1176     1176         1062 Sep 17 09:53 README
-rw-rw-r--    1 1176     1176         1290 Jun 26  2010 README.CD-manufacture
-rw-rw-r--    1 1176     1176         2590 Sep 17 09:53 README.html
-rw-r--r--    1 1176     1176       191243 Oct 31 07:52 README.mirrors.html
-rw-r--r--    1 1176     1176       103591 Oct 31 07:52 README.mirrors.txt
drwxr-sr-x   18 1176     1176         4096 Sep 17 09:55 dists
drwxr-sr-x    4 1176     1176         4096 Nov 17 07:52 doc
-rw-r--r--    1 1176     1176       148241 Nov 17 08:49 extrafiles
drwxr-sr-x    3 1176     1176         4096 Jul 27  2015 indices
-rw-r--r--    1 1176     1176     12165753 Nov 17 08:42 ls-lR.gz
drwxr-sr-x    5 1176     1176         4096 Dec 19  2000 pool
drwxr-sr-x    4 1176     1176         4096 Nov 17  2008 project
drwxr-xr-x    3 1176     1176         4096 Oct 10  2012 tools
drwxr-xr-x    6 1176     1176         4096 May 21 15:36 zzz-dists
>>>

Definitely that’s not a problem on remote server, i think it’s a problem of some package installed by bench install for Ubuntu 16, but i can’t figure out which one …i guess some packages socket related, but not sure which one.

@vjFaLk, again sorry for tagging you, can you please have a look?

Thanks a lot in advance

Wow this is quite odd. I can’t think of any package that would break ftplib :confused:

Are the ftplib versions the same for both systems?

Can you install frappe on the new VM that is working properly and try again? Maybe it’s unrelated.

@vjFaLk Yeah, that’s quite odd :disappointed: …i’ll try new frappe install on both Ubuntu 16 and CentOS 7 and will let you know.

@vjFaLk

Update:

i’ve created two new VM from scratch with CentOS 7 and Ubuntu 16 and installed bench and frappe using easy install:

sudo python install.py --develop

it’s working without any probs on CentOS, while on Ubuntu 16 i’m still getting the error:

frappe@ubuntu16-dev:~$ python -V
Python 2.7.12
frappe@ubuntu16-dev:~$ python
Python 2.7.12 (default, Jul  1 2016, 15:12:24)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ftplib
>>> ftp = ftplib.FTP("ftp.debian.org")
>>> ftp.login()
'230 Login successful.'
>>> ftp.dir()
drwxr-xr-x    9 1176     1176         4096 Nov 17 08:57 debian
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/ftplib.py", line 534, in dir
    self.retrlines(cmd, func)
  File "/usr/lib/python2.7/ftplib.py", line 440, in retrlines
    line = fp.readline(self.maxline + 1)
  File "/usr/lib/python2.7/socket.py", line 480, in readline
    data = self._sock.recv(self._rbufsize)
socket.error: [Errno 104] Connection reset by peer
>>>

:cry::sweat::confused:

@vjFaLk

Update 2: it seems to work on CentOS until i’ll install app to the site using:

bench --site [site-name] install-app [appname]

The app is a basic frappe app, the code:

def validate(self):
    print get_files_path(is_private="is_private")

    print self.ftp_server
    print self.ftp_user
    print self.get_password(fieldname="ftp_password")

    ftp = connectFTP(self.ftp_server, self.ftp_user,
                     self.get_password(fieldname="ftp_password"))

    listing = []
    ftp.dir(listing.append)



def getFile(ftp, file_path, filename):
   try:
       ftp.retrbinary("RETR " + filename,
                   open(file_path + "/" + filename, 'wb').write)
   except Exception, e:
       print("Error: " + str(e))


def connectFTP(server, user, password):
   ftp = ftplib.FTP(server)
   ftp.login(user, password)

   print ftp.getwelcome()

   return ftp

I’m really struggling with this, can’t understand what’s wrong with this :frowning: :sob:

It’s quite late in this part of the world, so lemme look at this tomorrow. This is such a weird bug :laughing:

@vjFaLk thx …take your time :grin:

@vjFaLk i think i got the problem :grin:

Error depends on VM network setting! If set as NAT, for some reason error [Errno 104] Connection reset by peer will be raised, not sure why …but that’s what happening :confused:

1 Like

:dizzy_face: :dizzy_face::dizzy_face::dizzy_face:

…exately :dizzy_face: :astonished: …at least i found out …u can close the thread …thx