I extract some web data, parse it and save the output as a Pandas DataFrame to an HDF5 file. Right before I write DataFrameto the H5 file, I will add my own description line to annotate some metadata about where the data came from and whether something went wrong when I parsed it.
In [1]: my_data_frame.desc = "Some string about the data"
In [2]: my_data_frame.desc
Out[1]: "Some string about the data"
In [3]: print type(my_data_frame)
<class 'pandas.core.frame.DataFrame'>
However, after loading the same data with pandas.io.pytables.HDFStore()my added attribute, it is descmissing, and I get an error: AttributeError: 'DataFrame' object has no attribute 'desc'as if I never added this new attribute.
How can I get my metadata descriptions to be saved as an optional attribute of a DataFrame? (Or is there some existing recognized DataFrame attribute that I can capture metadata for my purposes?)