add a io module

[dacoex]

From:
#7 (comment)

Shall we add standard readers for typical data providers such as:

Input:

• Meteonorm
• SolarGIS, e.g. http://solargis.info/doc/_files/sample_data/SolarGIS-climData/SolarGIS-TMY-hourly-P50-PSA-ES.zip
• BSRN
• SoDa-IS

Output:

• epw: http://apps1.eere.energy.gov/buildings/energyplus/cfm/weather_data3.cfm
• SAM

Hints:
https://github.com/pydata/pandas/blob/master/pandas/io/date_converters.py

There was a similar effort at pandas some time ago which stalled:
pandas-dev/pandas#1180

This could be a meta issue for all input and output format existing in the (solar) energy meteo domain.

[wholmgren]

I would support renaming tmy.py to io.py and adding importers for the most useful data sets. We need to be careful about the scope, though. A comment on the pandas issue correctly points out the danger

It seems to me like this wiill either stagnate or grow into a melange of tailored
solutions to the 1001 weird data problems found in the wild that most pepole won't see.

It's also worth pointing out that pydata recently created a new repo pandas-datareader after having discussed this and related issues for a long time.

[bmu]

I would propose to call it data_io, because io is a module in the python standard library. Pandas uses io but if I remember correctly I had issues some time ago because of this name clash.

[wholmgren]

Do we want to go forward with this for 0.2? I'm still concerned about scope and future maintenance. I don't regularly use these kinds of datasets, so I'm not a good judge of the tradeoffs. Wondering what @Calama-Consulting thinks of this.

Does pvsystem.retrieve_sam belong here too?

Not sure about naming. Technically the module would be named pvlib.<<something>>, so shouldn't clashes should be fairly rare?

[jforbess]

Agreed that future maintenance of dataset importing is an unknowable
effort.

I think the scope to consider here would be how to frame it so that someone
could easily make a reader for a common dataset and add it to pvlib,
because I don't think it makes much sense to make the readers in advance of
need, but it would be great to get people to contribute a good reader if
they found themself using SolarGIS a lot. Personally, I anticipate using a
lot of data from individual plants, without a standard form, but a
well-defined reader template might help with that as well. Not sure.

Not sure about the EPW output. Isn't that an input? It's an input to
PVsyst.

On Wed, Apr 22, 2015 at 8:17 AM, Will Holmgren notifications@github.com
wrote:

Do we want to go forward with this for 0.2? I'm still concerned about
scope and future maintenance. I don't regularly use these kinds of
datasets, so I'm not a good judge of the tradeoffs. Wondering what
@Calama-Consulting https://github.com/Calama-Consulting thinks of this.

Does pvsystem.retrieve_sam belong here too?

Not sure about naming. Technically the module would be named
pvlib.<>, so shouldn't clashes should be fairly rare?

—
Reply to this email directly or view it on GitHub
#29 (comment).

[bmu]

One way for possible new readers would be to define a specific pvlib format for input and than every reader can read data and converts it to this format.

[wholmgren]

I'm still interested in this but will leave it the community to resolve. It gets a "Someday" milestone until somebody expresses interest in resolving it.

[dacoex]

@bmu Do you still think that this is needed?
Where do we get sample data for above data sources?
Which location is good for the test data?
Carpentras?
Or any where we get data for?

[bmu]

There is no need from my side at least ;-). So maybe Wills solution to let users add some readers is the best one.

Dont't think that it is a problem to geht some sample data. I could add Meteonorm data (and ask for permission) as well as SolarGis data.

Am 2. Dezember 2015 21:53:06 GMT-02:00, schrieb DaCoEx notifications@github.com:

@bmu Do you still think that this is needed?
Where do we get sample data for above data sources?
Which location is good?
Carpentras?
Or any where we get data for?

---

Reply to this email directly or view it on GitHub:
#29 (comment)