~sschwarzer/ftputil#62: 
FTP upload path not correct using ftp_sync.py under win32

The upload path is not correct under win32 when using ftp_sync.py For example:

dir/some.file

will be uploaded to FTP as 1 directory and 1 file named "dir\some.file"

dir/ dir\some.file

The bug is because of the default path seperator is "\" under win32. os.path.sep should be considered.

Status
RESOLVED FIXED
Submitter
ftputiluser (unverified)
Assigned to
No-one
Submitted
9 years ago
Updated
9 years ago
Labels
bug library

schwa (unverified) 9 years ago · edit

Thanks for the report.

If I understand correctly, you try to sync a file some.file in the directory dir from a Windows client machine to a remote machine. On the remote machine, that file shows up in the directory dir with the name dir\some.file. Is that right?

Can you possibly tell me what's the OS of the remote machine and possibly the FTP server there?

It would be great if you could attach a small runnable script which shows the problem.

ftputiluser (unverified) 9 years ago · edit

Replying to schwa:

Thanks for the report.

If I understand correctly, you try to sync a file some.file in the directory dir from a Windows client machine to a remote machine. On the remote machine, that file shows up in the directory dir with the name dir\some.file. Is that right?

Can you possibly tell me what's the OS of the remote machine and possibly the FTP server there?

It would be great if you could attach a small runnable script which shows the problem.

Thanks for the report. If I understand correctly, you try to sync a file some.file in the directory dir from a Windows client machine to a remote machine. On the remote machine, that file shows up in the directory dir with the name dir\some.file. Is that right? Can you possibly tell me what's the OS of the remote machine and possibly the FTP server there? It would be great if you could attach a small runnable script which shows the problem.

Thanks for looking into this. Below is the reproduce scenario:

  • Remote FTP Server: Mac OS X Lion
  • Local Client: Window 7
  • ftputil verison: current tip of hg repository
  • Local directory: dir/some.file

After ftp_sync, remote FTP Server's <curr dir> is:

  • <curr dir>/dir/
  • <curr dir>/dir\some.file

that is, a directory under <curr dir> "<curr dir>/dir/" and a file under <curr dir> directly "<curr dir>/dir\some.file"

the "\" is a valid file name char under Mac(*nix system), so ftp server just create the file "dir\some.file" directly under the <curr dir>

The right behavior should be upload "some.file" under "<curr dir>/dir/"

The script is as simple as:

with FTPHost(host, user, password) as remote:
    local = LocalHost()
    syncer = Syncer(local, remote)
    syncer.sync(local_directory, remote_path)

My quick fix is add following line to ftputil/ftp_file.py line 66:

    def _open(self, path, mode):
        """Open the remote file with given path name and mode."""
        # Check mode.
        path = path.replace(os.path.sep, '/') # Fix path error under win32

        if 'a' in mode:

schwa (unverified) 9 years ago · edit

I don't have a Windows system here, so I can't reproduce your problem. Before I'm guessing at a fix, I'd like to check if what I think happens actually happens. ;-)

Can you please uncomment the print statements in ftputil/ftp_sync.py (in _mkdir and _sync_file), re-run your script, and attach the output to this ticket?

Alternatively, you may put the output in the ticket comment, but in that case really make sure that you wrap the output in triple braces (see WikiFormatting under heading "Preformatted Text"). You can check the display with the "Preview" button before you submit your comment to the ticket system permanently.

ftputiluser (unverified) 9 years ago · edit

I uncomment the print statement and here's the re-run results:

Making /upload
Making /upload\dir
Syncing e:\a\dir\file.name -> /upload\dir\file.name

One more thing, when upload to filezilla FTP server on windows, the upload path is correct. however, upload to Mac(Unix) FTP, the upload file path is "dir\file.name"

I think to maxmize portability it's better to use portable file seperator "/"

schwa (unverified) 9 years ago · edit

Would you please replace your version of ftp_sync.py with the file I've attached? Ideally, the attached version should fix both tickets #62 and #63. Can you please put the resulting outputs of your test runs in the respective tickets?

Replying to ftputiluser:

I think to maxmize portability it's better to use portable file seperator "/"

I agree completely. I was only thinking about where to make this change. I had thought about using your fix of replacing the path separator in FTPHost.open, but eventually came to the conclusion that the user of ftputil should be responsible for passing valid arguments to the open method. Of course, ftp_sync is a different story, because a user of the module can only influence the "root" directories for source and target. So, in a sense, ftp_sync here is the FTPHost "user" which has to pass valid arguments to mkdir and open. ;-)

schwa (unverified) 9 years ago · edit

I revised the file to always change the separators from the source to the target system, not only for uploads to an FTP server.

Note: If you click on the file name link "ftp_sync.py", trac will show the file as an HTML page. To download the actual source file, use the link "Original Format" at the end of that page.

ftputiluser (unverified) 9 years ago · edit

The attached file fixed #62 but not #63. Here's the output for uploading from win32 to Mac FTP server:

Making /upload
Fixing inner_target_dir from /upload\.hg to /upload/.hg
Making /upload/.hg
Fixing target_file from /upload\.hg\file.name to /upload/.hg/file.name
Syncing e:\a\.hg\file.name -> /upload/.hg/file.name

The file and directory are both correctly created in the their right path.

It might be helpful to document that, the user is responsible for provide the portable path format to use the FTPHost function.

I have to change:

    with FTPHost(host, user, password) as remote:
        with remote.file(os.path.join('a', 'b'), 'wb') as f:
            f.write('1')

to:

    with FTPHost(host, user, password) as remote:
        with remote.file(remote.path.join('a', 'b'), 'wb') as f:
            f.write('1')

to create a correct file from Win32 to Mac FTP

schwa (unverified) 9 years ago · edit

Replying to ftputiluser:

The attached file fixed #62 but not #63.

Ok, that's something at least. :-) I've committed the fix for #62 as changeset [1863cfc0580091c225af4e5e68c37523a5feae1b](https://git.sr.ht/~sschwarzer/ftputil/commit/1863cfc0580091c225af4e5e68c37523a5feae1b "Fix path separators for nested paths on target system (ticket #62).").

It might be helpful to document that, the user is responsible for provide the portable path format to use the FTPHost function.

I've added a paragraph on this to the documentation in changeset 02c0e84406c417fb86a5346c08c9202605341c1a.

I have to change: [...] to create a correct file from Win32 to Mac FTP

Yes, I think that's the approach to take.

Register here or Log in to comment, or comment via email.