Skip to content

Chunksize from json memory consumption as high as without chunksize #34548

Closed
@hasnain2808

Description

@hasnain2808
  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • (optional) I have confirmed this bug exists on the master branch of pandas.


While working on the PR (#33962) for adding nrows to read_json it was seen that maybe the chunksize parameter for read_json is not doing its task. ASV benchmarks were carried out but as this issue is not related to the PR we decided to create a new issue for it

Here is the link to the paste that has all the results
paste
The results are nicely added this to this spreadsheet
spreadsheet

Metadata

Metadata

Assignees

No one assigned

    Labels

    IO JSONread_json, to_json, json_normalizeRegressionFunctionality that used to work in a prior pandas version

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions