Reading SAS time variables

[sasutils]

Code Sample, a copy-pastable example if possible

Problem description

I tried to search to see if this issue was raised before, but could not find anything.

I notice that read_sas can convert SAS numbers formatted as DATE values (number of days since 1960) or DATETIME values (number of seconds since 1960) to datetime values supported by the datetime class in Python.

But it does not include any logic to convert numbers formatted as TIME values (number of seconds since midnight) even though it looks like the datetime class in Python does support time of day values.

Why does read_sas not also convert those numbers? Is it because nobody actually uses that Python data type? Nobody asked for that feature?

[TomAugspurger]

But it does not include any logic to convert numbers formatted as TIME values (number of seconds since midnight)

Is it actually seconds since midnight? If so that could be a timedelta. But if its supposed to represent the actual time of day, then no, pandas (nor numpy) have a type for that.

[sasutils]

Yes TIME is the number of seconds in 24 hours, ie the number of seconds since midnight.

Looks like the definition for datetime.time. https://docs.python.org/3/library/datetime.html

class datetime.time
An idealized time, independent of any particular day, assuming that every day has exactly 246060 seconds (there is no notion of “leap seconds” here). Attributes: hour, minute, second, microsecond, and tzinfo.

Perhaps I am misunderstanding the Python documentation?

[troels]

It could be converted to an ndarray of dtype object with time objects. It wouldn't be efficiently stored like datetime dtype=datetime, but it would be correct.

[TomAugspurger]

@sasutils yes, I was asking whether SAS's TIME matches Python's datetime.time (no leap seconds) or whether it was more of a datetime.timedelta (with leap seconds).

Loading it as a ndarray of time objects would be unusual for pandas. AFAIK, we only do this type of conversion for dtypes that are actually supported.

I'd recommend users do this themselves by converting the loaded data into an object-dtype column holding datetime.time values (and if someone really wants to they can make an extension array for this).

[troels]

Both e.g. the excel parser and the SQL module handle and do the conversion to and from time just fine, and as far as I know that is true for all other file formats supporting time as a type.
Why should sas-files be treated differently?

Doing the conversion themselves force the users to manually keep track of the types of the respective columns in a way that is unnecessary, as all the information required is right there in the sas-file.
Why would we force the user to do that?

[TomAugspurger]

I didn’t realize we did that for excel and sql.

[ofajardo]

pyreadstat converts SAS TIME to python datetime.time objects. You could add pyreadstat as a backend for read_sas (already used for read_spss) and solve this issue as well as others (#37088, #35545, #31243, #18198)