Skip to content

ENH: Add dtype to read sql to be consistent with read_sql_query #50797

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jan 20, 2023
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v2.0.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,7 @@ Other enhancements
- Improved error message in :func:`to_datetime` for non-ISO8601 formats, informing users about the position of the first error (:issue:`50361`)
- Improved error message when trying to align :class:`DataFrame` objects (for example, in :func:`DataFrame.compare`) to clarify that "identically labelled" refers to both index and columns (:issue:`50083`)
- Added :meth:`DatetimeIndex.as_unit` and :meth:`TimedeltaIndex.as_unit` to convert to different resolutions; supported resolutions are "s", "ms", "us", and "ns" (:issue:`50616`)
- Added new argument ``dtype`` to :func:`read_sql` to be consistent with :func:`read_sql_query` (:issue:`50797`)
-

.. ---------------------------------------------------------------------------
Expand Down
10 changes: 10 additions & 0 deletions pandas/io/sql.py
Original file line number Diff line number Diff line change
Expand Up @@ -470,6 +470,7 @@ def read_sql(
columns: list[str] = ...,
chunksize: None = ...,
use_nullable_dtypes: bool = ...,
dtype: DtypeArg | None = None,
) -> DataFrame:
...

Expand All @@ -485,6 +486,7 @@ def read_sql(
columns: list[str] = ...,
chunksize: int = ...,
use_nullable_dtypes: bool = ...,
dtype: DtypeArg | None = None,
) -> Iterator[DataFrame]:
...

Expand All @@ -499,6 +501,7 @@ def read_sql(
columns: list[str] | None = None,
chunksize: int | None = None,
use_nullable_dtypes: bool = False,
dtype: DtypeArg | None = None,
) -> DataFrame | Iterator[DataFrame]:
"""
Read SQL query or database table into a DataFrame.
Expand Down Expand Up @@ -552,7 +555,12 @@ def read_sql(
implementation, even if no nulls are present.

.. versionadded:: 2.0
dtype : Type name or dict of columns
Data type for data or columns. E.g. np.float64 or
{‘a’: np.float64, ‘b’: np.int32, ‘c’: ‘Int64’}.
The argument is ignored if a table is passed instead of a query.

.. versionadded:: 2.0.0
Returns
-------
DataFrame or Iterator[DataFrame]
Expand Down Expand Up @@ -632,6 +640,7 @@ def read_sql(
parse_dates=parse_dates,
chunksize=chunksize,
use_nullable_dtypes=use_nullable_dtypes,
dtype=dtype,
)

try:
Expand Down Expand Up @@ -659,6 +668,7 @@ def read_sql(
parse_dates=parse_dates,
chunksize=chunksize,
use_nullable_dtypes=use_nullable_dtypes,
dtype=dtype,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this interact with use_nullable_dtypes when dtype is not a nullable type?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add tests for this :)

Right now we are casting to nullable and then apply astype (that's how it was implemented before as well). But I don't think that we have any tests covering this

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah would be good to add tests for this case. IIRC read_csv or a similar reader had a similar thing. This should hopefully operate similarly

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added

)


Expand Down
12 changes: 12 additions & 0 deletions pandas/tests/io/test_sql.py
Original file line number Diff line number Diff line change
Expand Up @@ -2394,6 +2394,18 @@ def test_chunksize_empty_dtypes(self):
):
tm.assert_frame_equal(result, expected)

def test_read_sql_dtype(self):
# GH#50797
table = "test"
df = DataFrame({"a": [1, 2, 3], "b": 5})
df.to_sql(table, self.conn, index=False, if_exists="replace")

result = pd.read_sql(
f"Select * from {table}", self.conn, dtype={"a": np.float64}
)
expected = DataFrame({"a": Series([1, 2, 3], dtype=np.float64), "b": 5})
tm.assert_frame_equal(result, expected)


class TestSQLiteAlchemy(_TestSQLAlchemy):
"""
Expand Down