Skip to content

Create parent directories in get_file #1282

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 31, 2023

Conversation

ianthomas23
Copy link
Collaborator

This is a fix when calling get with a file to get it into a new (i.e. doesn't exist yet) directory we need to create the parent directory to put the file in. The issue was discovered as part of the new cp/get/put test harness (#1267). With this fix all memory filesystem tests pass whereas before this the 4 "get file/directory/glob into a new directory" tests all failed as the parent directory wasn't auto created.

@ianthomas23
Copy link
Collaborator Author

Windows test failing as a result of the usual different path separators. I will have to investigate further.

@martindurant
Copy link
Member

I will have to investigate further.

We should hold the PR, then?

Maybe we should replace os.makedirs with LocalFileSystem.makedirs, which should do path normalisation ?

@ianthomas23
Copy link
Collaborator Author

Yes, hold the PR until I have looked again.

@ianthomas23
Copy link
Collaborator Author

OK, this is good now using LocalFileSystem.makedirs rather than os.makedirs.

@martindurant martindurant merged commit 386a084 into fsspec:master May 31, 2023
@ianthomas23 ianthomas23 deleted the memory_get_mkdir branch May 31, 2023 13:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants