ENH: Categorical serialized

[jankatins]

Currently df which include a categorical can be written to a hdf store and other on disc storage formats. For formats which include factors/categoricals, also ensure that such data is read into a categorical.

• CSV WIP: categoricals as an internal CategoricalBlock GH5313 #7217, WIP/DO NOT MERGE: Categorical improvements #7444 (just testing, not a completely fungible format)
• HDF5 WIP: categoricals as an internal CategoricalBlock GH5313 #7217, WIP/DO NOT MERGE: Categorical improvements #7444 / ENH: serialization of categorical to HDF5 (GH7621) #8793
• msgpack pd.Catagorical breaks to_msgpack #8632 (closed by ENH: allow categoricals in msgpack #12573)
• stata ENH: categorical dataexport - graceful degradation  #8633 / ENH: Add categorical support for Stata export #8767

[jankatins]

For discussion on this, see #7217 and #7444 and also the tests in test_categorical (test_io_hdf, test_io_csv)

[fkaufer]

Stata
Categorical export could be implemented using value labels, see http://www.stata.com/help.cgi?dta. As the name already indicates Stata's value labels are not as generic as pandas' categoricals but limited to strings (similar to R).

Support for categoricals-to-value-labels-conversion would be fantastic, but I would already be happy, if category dtype is captured (pandas/io/stata.py#L1262) and coerced to string. Guess this requires also to either fix to_records for categoricals (pandas/io/stata.py#L1633, #8626) or to explicitely decode (#8628) on the fly.

[jreback]

cc @bashtage, interested?

[bashtage]

I'll have to take a look at how hard it will be - haven't used Categoricals, so extra hurdle.