Support Stata 14 format (format 118)

[makmanalp]

Hello, I'm just leaving this here, might try to pick this up as a weekend project soon.

The beautiful people at stata have released documentation for the format here: http://www.stata.com/help.cgi?dta

And the old format is here:
http://www.stata.com/help.cgi?dta_117

The docs say that it'll highlight changes between 118 and 117 but I couldn't find any such references. One thing I noticed is that they now have utf-8 support! Yay!

As a workaround for now, stata users who want to use stata 14 can use "saveold" which creates stata 13 files, which pandas supports.

[makmanalp]

Here's a diff of the docs. The word-diff looks way better than the standard one but I don't know a way to put that online, so I converted it to html and dumped it here:

https://gist.github.com/makmanalp/a53532d99c33e68a93a5

[bashtage]

Looks like UTF-8 Strings are the big change

Also seem to be some size changes (e.g. length of variable names)

[makmanalp]

That's what it looks like to me too. I have stata 14 so I can generate some random files to use as test cases if you like. Let me know what kinds of things you'd want - we could try invalid characters, etc.

[DanSmith]

@makmanalp An example stata 14 file would be great. Could you include some value labels and characteristics as well?

[makmanalp]

@DanSmith I created a sample file with a bunch of different data types, labels, unicode all over the place, etc.

Not really sure how to do characteristics. Docs say: " Characteristics are used to record information that is unique to Stata and has no equivalent in other data management packages." Do we care?

DTA file: https://drive.google.com/file/d/0B-_ZeAR_FZDGT0JndWltcldzazA/view?usp=sharing

[makmanalp]

Here is the Stata 13 version of the same file I got with saveold. It complained that some of the chars will show up wonky, which makes sense since unicode support comes with 14.

https://drive.google.com/file/d/0B-_ZeAR_FZDGa2ItNURBYjNWNDQ/view?usp=sharing