Pandas groupby histogram doesn't do transparant bars (alpha kwarg)

[egpbos]

I would expect the following code to produce a plot with transparent bars, but it doesn't.

import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
plt.style.use('ggplot')

N = 1000

df = pd.DataFrame({
  'a': np.random.randint(0, 2, N),
  'b': np.random.random(N)
})

#df.groupby('a').b.hist(alpha=0.5)
df.groupby('a').b.plot(kind="hist", alpha=0.5)
plt.legend()
plt.show()

Neither the commented out hist command, nor the plot command works.

Version information from pd.show_versions():

INSTALLED VERSIONS

commit: None
python: 2.7.6.final.0
python-bits: 64
OS: Linux
OS-release: 3.13.0-46-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8

pandas: 0.16.2
nose: 1.3.7
Cython: 0.20.1post0
numpy: 1.9.2
scipy: 0.13.3
statsmodels: 0.5.0
IPython: 3.0.0
sphinx: None
patsy: 0.2.1
dateutil: 2.4.2
pytz: 2015.4
bottleneck: None
tables: 3.1.1
numexpr: 2.2.2
matplotlib: 1.4.3
openpyxl: 1.7.0
xlrd: 0.9.2
xlwt: 0.7.5
xlsxwriter: None
lxml: None
bs4: 4.2.1
html5lib: 0.999
httplib2: 0.8
apiclient: None
sqlalchemy: None
pymysql: None
psycopg2: None

[TomAugspurger]

Thanks for the report. This works correctly for me in master. It seems like this was addressed in #8758 (which was pre 0.16.2, your version of pandas).

Could you try it out on master, and let me know if it's fixed for you? Otherwise, you can always iterate over the groupby object.

gr = df.groupby('a').b
for k, v in gr:
    v.plot(kind='hist', alpha=.5, label=k)

[egpbos]

It doesn't work in master either.

In [5]: pd.__version__
Out[5]: '0.16.2+385.g873bbc2'

[TomAugspurger]

(from master pandas).

It's hard to say what's going on.

[egpbos]

Did you create that by pasting my code example or by using your suggestion of iterating over the groupby object? I tried now on three different systems (Ubuntu 14.04, Scientific Linux 6 and OS X 10.9.5) and it doesn't work on either (with my example). It does, indeed, work by iterating over the groupby object as you suggest. Still, I don't understand why it doesn't work for my code.

[TomAugspurger]

That was using your code.

[LindsayVass]

I have a very similar issue using a newer version of Pandas.

import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
%matplotlib inline
plt.style.use('ggplot')

N = 1000

df = pd.DataFrame({
  'a': np.random.randint(0, 2, N),
  'b': np.random.random(N)
})

# alpha does not work
df.groupby('a').b.hist(alpha=0.5)

# plot renders incorrectly
df.groupby('a').b.plot(kind = 'hist', alpha = 0.5);

# alpha works
df.groupby('a').b.plot.hist(alpha=0.5);

INSTALLED VERSIONS

commit: None
python: 2.7.10.final.0
python-bits: 64
OS: Darwin
OS-release: 14.5.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8

pandas: 0.17.1
nose: 1.3.7
pip: 8.0.2
setuptools: 19.2
Cython: 0.22.1
numpy: 1.10.4
scipy: 0.17.0
statsmodels: 0.6.1
IPython: 4.0.3
sphinx: 1.3.1
patsy: 0.3.0
dateutil: 2.4.2
pytz: 2013b
blosc: None
bottleneck: 1.0.0
tables: 3.2.0
numexpr: 2.4.3
matplotlib: 1.4.3
openpyxl: 1.8.5
xlrd: 0.9.3
xlwt: 1.0.0
xlsxwriter: 0.7.3
lxml: 3.4.4
bs4: 4.3.2
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: 1.0.5
pymysql: None
psycopg2: 2.6.1 (dt dec pq3 ext lo64)
Jinja2: None

[TomAugspurger]

@LindsayVass strange I still can't reproduce the bug 😕

Can you post a gist showing these rendering incorrectly? e.g. https://gist.github.com/2c7d666fd1c5d4bbe327

[LindsayVass]

@TomAugspurger So strange! Thanks for looking into this.

https://gist.github.com/LindsayVass/ed4aa46cf3223a7999a2

[TomAugspurger]

Ha, I had you make a notebook because I assumed you made a copy-paste error on your second graph. But nope :)

A couple things. It looks like that second graph with the crazy lines should be fixed by upgrading to pandas 0.18.

For this bug, the alpha not taking, I've was able to reproduce it with python2 (matplotlib 1.4.3 or 1.50, pandas 0.17.2 or 0.18.0). I don't know offhand why that would matter. I won't have time to dig any more this week, but after that I'll follow up unless someone beats me to it.

[LindsayVass]

Haha, I would have assumed that too!

Confirm that upgrading to pandas 0.18 cleared up the problem with the crazy lines. New gist below.

https://gist.github.com/LindsayVass/0175d2f02e7e92b3013b

[egpbos]

On python3 alpha works for me as well.

However, I'm not sure if it's because of python3 or because I upgraded pandas to 0.18.0. I cannot exclude the latter, because I now get a segmentation fault (!) in python2 when I run df.groupby('a').b.plot(kind="hist", alpha=0.5). I guess that's a separate issue from this one, though.

[egpbos]

Ok, seems like the segfault was related to #11487, reinstalling matplotlib fixed that.

Also in python2 alpha now works for me! Here are the changes to my updated package versions. Quite a few changes compared to #10756 (comment), so probably hard to say what eventually did the trick...

-python: 2.7.6.final.0
+python: 2.7.10.final.0
-OS-release: 3.13.0-46-generic
+OS-release: 4.2.0-35-generic

-pandas: 0.16.2
+pandas: 0.18.0
-nose: 1.3.7
+nose: None
-Cython: 0.20.1post0
+Cython: None
-numpy: 1.9.2
+numpy: 1.11.0
-scipy: 0.13.3
+scipy: 0.17.0
-statsmodels: 0.5.0
+statsmodels: None
-IPython: 3.0.0
+IPython: 4.0.0

+pip: 1.5.6
+setuptools: 20.4
+xarray: None

-patsy: 0.2.1
+patsy: None
-dateutil: 2.4.2
+dateutil: 2.5.2
-pytz: 2015.4
+pytz: 2016.3

+blosc: None

-tables: 3.1.1
+tables: None
-numexpr: 2.2.2
+numexpr: None
-matplotlib: 1.4.3
+matplotlib: 1.5.1
-openpyxl: 1.7.0
+openpyxl: None
-xlrd: 0.9.2
+xlrd: None
-xlwt: 0.7.5
+xlwt: None
-bs4: 4.2.1
+bs4: None
-httplib2: 0.8
+httplib2: None

+jinja2: 2.8
+boto: None

And here the complete output:

INSTALLED VERSIONS
------------------
commit: None
python: 2.7.10.final.0
python-bits: 64
OS: Linux
OS-release: 4.2.0-35-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8

pandas: 0.18.0
nose: None
pip: 1.5.6
setuptools: 20.4
Cython: None
numpy: 1.11.0
scipy: 0.17.0
statsmodels: None
xarray: None
IPython: 4.0.0
sphinx: None
patsy: None
dateutil: 2.5.2
pytz: 2016.3
blosc: None
bottleneck: None
tables: None
numexpr: None
matplotlib: 1.5.1
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: 0.999
httplib2: None
apiclient: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.8
boto: None

[egpbos]

Actually, now I think about it, the problem might have been caused by using a virtualenv with --system-site-packages, I had problems with that before. I'm now using a clean virtualenv with --no-site-packages (as can be seen from the many None packages in the new list).