~sschwarzer/ftputil#103: 
Error `open`ing new file-like objects when a connection is timed-out but not yet closed.

Here's the sample stack trace:

    File "build/bdist.linux-x86_64/egg/ftputil/host.py", line 195, in open
        
      File "build/bdist.linux-x86_64/egg/ftputil/host.py", line 166, in _available_child
        
      File "/usr/lib/python2.7/ftplib.py", line 583, in pwd
        resp = self.sendcmd('PWD')
      File "/usr/lib/python2.7/ftplib.py", line 249, in sendcmd
        return self.getresp()
      File "/usr/lib/python2.7/ftplib.py", line 215, in getresp
        resp = self.getmultiline()
      File "/usr/lib/python2.7/ftplib.py", line 201, in getmultiline
        line = self.getline()
      File "/usr/lib/python2.7/ftplib.py", line 191, in getline
        if not line: raise EOFError
    exceptions.EOFError: 

The reason might be that the server has already seen the connection as a timed-out one, but not yet closed it.

Related lines: line #164 to #172 of ftputil/host.py

Status
REPORTED
Submitter
ftputiluser (unverified)
Assigned to
No-one
Submitted
6 years ago
Updated
6 years ago
Labels
enhancement library

schwa (unverified) 6 years ago · edit

Thanks for entering this ticket. :-)

Replying to ftputiluser:

The reason might be that the server has already seen the connection as a timed-out one, but not yet closed it.

Assuming this is the reason for the stack trace:

  • I'd expect that the stack trace would be difficult to reproduce. How often have you seen it?

  • Probably the problem could happen in other situations when using ftputil. On the other hand, I assume the EOFError would happen most of the time in the lines you have in the stack trace.

After all, if client code closes a file-like object, the client code assumes that it's done with the file and wouldn't expect that any resources could still be in an "open" state. So a timeout when getting a file-like object is probably more surprising than a timeout during "normal operation" without using file-like objects. That could be a reason to fix the problem here, even if not catching EOFErrors in lots of other parts of the ftputil code.

I just did a web search and found this ​interesting discussion on StackOverflow. Together with the information in ​CPython's ticket 148036 that EOFError is contained in ftplib.all_errors it appears that ftputil is already implicitly catching EOFErrors in its context managers ftplib_error_to_ftp_os_error and ftplib_error_to_ftp_io_error in ftputil.error.

As it happens, _available_child uses a direct host._session.pwd() call (where _session usually is an ftplib.FTP instance) instead of the context managers.

I guess I'll use the ftplib_error_to_ftp_os_error context manager in _available_child and consider the child session unusable if the context manager gives me an FTPOSError. TemporaryError and PermanentError inherit from FTPOSError, so this would also implicitly catch the ftplib.error_temp I'm catching now in _available_child.

Register here or Log in to comment, or comment via email.