-
-
Notifications
You must be signed in to change notification settings - Fork 18.5k
BUG: convert_dtypes incorrectly converts byte strings to strings in 1.3+ #43199
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 32 commits
b534f56
23583fc
96338fa
ad4189f
e915bc0
6d0a497
5ab2d79
7921c6f
3e7a91f
b5b0e27
694fb7a
e12fd22
b774cef
f867f72
0cb3f08
9760857
2e18ebc
3193de9
20f8dda
1503730
c081d92
7654729
0a2e0a9
d4e086c
766a30f
52ac57a
be03f37
92677cc
07e1de9
5f2933d
bfb1242
37ee298
77ee435
e70f68b
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -1426,6 +1426,8 @@ def convert_dtypes( | |
if is_string_dtype(inferred_dtype): | ||
if not convert_string: | ||
return input_array.dtype | ||
elif inferred_dtype == "bytes": | ||
return pandas_dtype("object") | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @jbrockmendel is this the same as returning If we are not changing if not convert_string or inferred_dtype == "bytes":
return input_array.dtype
else:
... instead? I've still not looked at the source of the regression to know what the correct fix is. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Made the change. The incorrect fix to the string is stopped as the class continues to remain bytes so decoding will give string. It will not convert it to string anymore as the dtype will be object and not string. |
||
else: | ||
jreback marked this conversation as resolved.
Show resolved
Hide resolved
|
||
return pandas_dtype("string") | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you remove this whitespace.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done