Training failed with ValueError: could not broadcast input array from shape (14444) into shape (43332)

[nwatab]

I followed Menpofit basic tutorial but it threw an error.

• Environment: Anaconda on pyenv with virtualenv
• OS: UBUNTU 16.04
• Python: Python 3.5.6 :: Anaconda, Inc.
• conda list:

#
# Name                    Version                   Build  Channel
appdirs                   1.4.3            py35h28b3542_0  
apptools                  4.4.0                    py35_0    menpo
asn1crypto                0.24.0                   py35_0  
attrs                     18.2.0           py35h28b3542_0  
automat                   0.7.0                    py35_0  
backcall                  0.1.0                    py35_0  
blas                      1.0                         mkl  
bleach                    2.1.4                    py35_0  
boost                     1.59.0                   py35_0    menpo
bzip2                     1.0.6                h14c3975_5  
ca-certificates           2018.03.07                    0  
certifi                   2018.8.24                py35_1  
cffi                      1.11.5           py35he75722e_1  
configobj                 5.0.6                    py35_1  
constantly                15.1.0           py35h28b3542_0  
cryptography              2.3.1            py35hc365091_0  
cycler                    0.10.0           py35hc4d5149_0  
cyffld2                   0.2.4                    py35_0    menpo
cypico                    0.2.7                    py35_0    menpo
cyrasterize               0.3.2                    py35_0    menpo
Cython                    0.23.5                    <pip>
cyvlfeat                  0.4.6                    py35_0    menpo
decorator                 4.3.0                    py35_0  
dlib                      18.18                    py35_2    menpo
docopt                    0.6.2                    py35_0  
entrypoints               0.2.3                    py35_2  
envisage                  4.5.1                    py35_0    menpo
ffmpeg                    2.7.0                         0    menpo
fftw                      3.3.4                         0    menpo
fontconfig                2.11.1                        6  
freetype                  2.5.5                         2  
glew                      2.0.0                         0    menpo
glfw3                     3.2.1                         0    menpo
gmp                       6.1.2                h6c8ec71_1  
hdf5                      1.10.2               hba1933b_1  
html5lib                  1.0.1                    py35_0  
hyperlink                 18.0.0                   py35_0  
icu                       58.2                 h9c2bf20_1  
idna                      2.7                      py35_0  
incremental               17.5.0                   py35_0  
intel-openmp              2019.0                      117  
ipykernel                 4.9.0                    py35_0  
ipython                   6.5.0                    py35_0  
ipython_genutils          0.2.0            py35hc9e07d0_0  
ipywidgets                6.0.0                    py35_0  
jedi                      0.12.1                   py35_0  
jinja2                    2.10                     py35_0  
jpeg                      9b                   h024ee3a_2  
jsonschema                2.6.0            py35h4395190_0  
jupyter                   1.0.0                    py35_4  
jupyter_client            5.2.3                    py35_0  
jupyter_console           5.2.0            py35h4044a63_1  
jupyter_core              4.4.0                    py35_0  
kiwisolver                1.0.1                     <pip>
libedit                   3.1.20170329         h6b74fdf_2  
libffi                    3.2.1                hd88cf55_4  
libgcc-ng                 8.2.0                hdf63c60_1  
libgfortran-ng            7.3.0                hdf63c60_0  
libpng                    1.6.34               hb9fc6fc_0  
libsodium                 1.0.16               h1bed415_0  
libstdcxx-ng              8.2.0                hdf63c60_1  
libtiff                   4.0.9                he85c1e1_2  
libxml2                   2.9.8                h26e45fe_1  
markupsafe                1.0              py35h14c3975_1  
matplotlib                1.5.0                     <pip>
matplotlib                1.5.1               np111py35_0  
mayavi                    4.5.0                    py35_0    menpo
menpo                     0.8.1                    py35_0    menpo
menpo                     0.8.1                     <pip>
menpo3d                   0.6.0                    py35_0    menpo
menpocli                  0.1.0                    py35_0    menpo
menpodetect               0.5.0                    py35_0    menpo
menpofit                  0.5.0                    py35_0    menpo
menpofit                  0.5.0                     <pip>
menpoproject              2.0                        py_0    menpo
menpowidgets              0.3.0                    py35_0    menpo
metis                     5.1.0                         0    menpo
mistune                   0.8.3            py35h14c3975_1  
mkl                       2019.0                      117  
mock                      2.0.0            py35h70ca42c_0  
nbconvert                 5.3.1                    py35_0  
nbformat                  4.4.0            py35h12e6e07_0  
ncurses                   6.1                  hf484d3e_0  
nose                      1.3.7                    py35_2  
notebook                  5.6.0                    py35_0  
numpy                     1.11.3          py35h1d66e8a_10  
numpy                     1.10.4                    <pip>
numpy-base                1.11.3          py35h81de0dd_10  
olefile                   0.46                     py35_0  
opencv3                   3.1.0                    py35_0    menpo
openssl                   1.0.2p               h14c3975_0  
pandas                    0.23.4           py35h04863e7_0  
pandoc                    2.2.3.2                       0  
pandocfilters             1.4.2                    py35_1  
parso                     0.3.1                    py35_0  
pbr                       4.2.0                    py35_0  
pexpect                   4.6.0                    py35_0  
pickleshare               0.7.4            py35hd57304d_0  
Pillow                    4.3.0                     <pip>
pillow                    4.2.1                    py35_0  
pip                       10.0.1                   py35_0  
pip                       18.0                      <pip>
prometheus_client         0.3.1            py35h28b3542_0  
prompt_toolkit            1.0.15           py35hc09de7a_0  
ptyprocess                0.6.0                    py35_0  
pyasn1                    0.4.4            py35h28b3542_0  
pyasn1-modules            0.2.2                    py35_0  
pycparser                 2.18                     py35_1  
pyface                    5.1.0                    py35_0    menpo
pygments                  2.2.0            py35h0f41973_0  
pyopenssl                 18.0.0                   py35_0  
pyparsing                 2.2.0                    py35_1  
pyqt                      4.11.4                   py35_4  
python                    3.5.6                hc3d631a_0  
python-dateutil           2.7.3                    py35_0  
pytz                      2018.5                   py35_0  
pyzmq                     17.1.2           py35h14c3975_0  
qt                        4.8.7                         3  
qtconsole                 4.3.1            py35h4626a06_0  
readline                  7.0                  h7b6447c_5  
scikit-learn              0.19.1           py35hbf1f462_0  
scikit-learn              0.17.1                    <pip>
scikit-sparse             0.3.1                    py35_0    menpo
scipy                     0.16.1                    <pip>
scipy                     0.19.1           py35ha8f041b_3  
send2trash                1.5.0                    py35_0  
service_identity          17.0.0           py35h28b3542_0  
setuptools                40.2.0                   py35_0  
simplegeneric             0.8.1                    py35_2  
sip                       4.18                     py35_0  
six                       1.11.0                   py35_1  
sqlite                    3.24.0               h84994c4_0  
suitesparse               4.4.1                         0    menpo
system                    5.8                           2  
terminado                 0.8.1                    py35_1  
testpath                  0.3.1            py35had42eaf_0  
tk                        8.6.8                hbc83047_0  
tornado                   5.1              py35h14c3975_0  
traitlets                 4.3.2            py35ha522a97_0  
traits                    4.5.0                    py35_0    menpo
traitsui                  5.1.0                    py35_0    menpo
twisted                   18.7.0           py35h14c3975_1  
vlfeat                    0.9.20                        1    menpo
vtk                       7.0.0                    py35_0    menpo
wcwidth                   0.1.7            py35hcd08066_0  
webencodings              0.5.1                    py35_1  
wheel                     0.31.1                   py35_0  
widgetsnbextension        3.4.1                    py35_0  
xz                        5.2.4                h14c3975_4  
zeromq                    4.2.5                hf484d3e_1  
zlib                      1.2.11               ha838bed_2  
zope                      1.0                      py35_1  
zope.interface            4.5.0            py35h14c3975_0  

I also tried following things but ended to the same result.

• without pyenv and virtualenv on Ubuntu,
• downgrade to menpofit==0.3.0
• upgrade numpy, scipy and matplotlib to the latest by pip
• change dataset to IBUG, XM2VTS found on IBUG website
• tried PatchAAM

Executed Jupyter notebook is below:

Input 1

import menpo.io as mio
path_to_images = '/home/myname/work/data/LFPW/trainset/'
training_images = mio.import_images(path_to_images, verbose=True)

Output 1
Found 811 assets, index the returned LazyList to import.

Input 2

from menpofit.aam import HolisticAAM
from menpofit.aam import PatchAAM
from menpo.feature import igo

aam = HolisticAAM(training_images, reference_shape=None,
                  diagonal=150, scales=(0.9, 1.0),
                  holistic_features=igo, verbose=True,
                  batch_size=16
                 )
print(aam)

Output 2

- Computing reference shape                                                     Computing batch 0
- Building modelsges size: [==========] 100% (16/16) - done.                    
  - Scale 0: Computing feature space: [          ] 6% (1/16) - 00:00:00 remaining

/home/myname/.pyenv/versions/anaconda3-5.2.0/envs/menpo/lib/python3.5/site-packages/menpofit/aam/base.py:164: MenpoFitBuilderWarning: No reference shape was provided. The mean of the first batch will be the reference shape. If the batch mean is not representative of the true mean, this may cause issues.
  MenpoFitBuilderWarning)

  - Scale 0: Doneding appearance model                                          ing
  - Scale 1: Building shape model                                               

/home/myname/.pyenv/versions/anaconda3-5.2.0/envs/menpo/lib/python3.5/site-packages/menpofit/builder.py:338: MenpoFitModelBuilderWarning: The reference shape passed is not a TriMesh or subclass and therefore the reference frame (mask) will be calculated via a Delaunay triangulation. This may cause small triangles and thus suboptimal warps.
  MenpoFitModelBuilderWarning)
/home/myname/.pyenv/versions/anaconda3-5.2.0/envs/menpo/lib/python3.5/site-packages/menpo/image/boolean.py:711: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
  copy.pixels[slices].flat = point_in_pointcloud(pointcloud, indices)

  - Scale 1: Doneding appearance model                                          
                                                              Computing batch 1
- Building modelsges size: [==========] 100% (16/16) - done.                    
  - Scale 0: Doneding appearance model                                          ning
  - Scale 1: Doneding appearance model                                          
                                                              Computing batch 2
- Building modelsges size: [==========] 100% (16/16) - done.                    
  - Scale 0: Doneding appearance model                                          ing
  - Scale 1: Doneding appearance model                                          
                                                              Computing batch 3
- Building modelsges size: [==========] 100% (16/16) - done.                    
  - Scale 0: Doneding appearance model                                          ing
  - Scale 1: Doneding appearance model                                          
                                                              Computing batch 4
- Building modelsges size: [==========] 100% (16/16) - done.                    
  - Scale 0: Doneding appearance model                                          ing
  - Scale 1: Doneding appearance model                                          
                                                              Computing batch 5
- Building modelsges size: [==========] 100% (16/16) - done.                    
  - Scale 0: Doneding appearance model                                          ning
  - Scale 1: Doneding appearance model                                          
                                                              Computing batch 6
- Building modelsges size: [==========] 100% (16/16) - done.                    
  - Scale 0: Doneding appearance model                                          ing
  - Scale 1: Doneding appearance model                                          
                                                              Computing batch 7
- Building modelsges size: [==========] 100% (16/16) - done.                    
  - Scale 0: Doneding appearance model                                          ing
  - Scale 1: Doneding appearance model                                          
                                                              Computing batch 8
- Building modelsges size: [==========] 100% (16/16) - done.                    
  - Scale 0: Doneding appearance model                                          ing
  - Scale 1: Doneding appearance model                                          
                                                              Computing batch 9
- Building modelsges size: [==========] 100% (16/16) - done.                    
  - Scale 0: Doneding appearance model                                          ing
  - Scale 1: Doneding appearance model                                          
                                                              Computing batch 10
- Building modelsges size: [==========] 100% (16/16) - done.                    
  - Scale 0: Building appearance model                                          ning

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-40-c5f48712ecae> in <module>()
      6                   diagonal=150, scales=(0.9, 1.0),
      7                   holistic_features=igo, verbose=True,
----> 8                   batch_size=16
      9                  )
     10 print(aam)

~/.pyenv/versions/anaconda3-5.2.0/envs/menpo/lib/python3.5/site-packages/menpofit/aam/base.py in __init__(self, images, group, holistic_features, reference_shape, diagonal, scales, transform, shape_model_cls, max_shape_components, max_appearance_components, verbose, batch_size)
    137         # Train AAM
    138         self._train(images, increment=False, group=group, verbose=verbose,
--> 139                     batch_size=batch_size)
    140 
    141     def _train(self, images, increment=False, group=None,

~/.pyenv/versions/anaconda3-5.2.0/envs/menpo/lib/python3.5/site-packages/menpofit/aam/base.py in _train(self, images, increment, group, shape_forgetting_factor, appearance_forgetting_factor, verbose, batch_size)
    181                 shape_forgetting_factor=shape_forgetting_factor,
    182                 appearance_forgetting_factor=appearance_forgetting_factor,
--> 183                 verbose=verbose)
    184 
    185     def _train_batch(self, image_batch, increment=False, group=None,

~/.pyenv/versions/anaconda3-5.2.0/envs/menpo/lib/python3.5/site-packages/menpofit/aam/base.py in _train_batch(self, image_batch, increment, group, verbose, shape_forgetting_factor, appearance_forgetting_factor)
    267                 self.appearance_models[j].increment(
    268                     warped_images,
--> 269                     forgetting_factor=appearance_forgetting_factor)
    270                 # trim appearance model if required
    271                 if self.max_appearance_components[j] is not None:

~/.pyenv/versions/anaconda3-5.2.0/envs/menpo/lib/python3.5/site-packages/menpo/model/pca.py in increment(self, samples, n_samples, forgetting_factor, verbose)
   1426         """
   1427         # build a data matrix from the new samples
-> 1428         data = as_matrix(samples, length=n_samples, verbose=verbose)
   1429         n_new_samples = data.shape[0]
   1430         PCAVectorModel.increment(self, data, n_samples=n_new_samples,

~/.pyenv/versions/anaconda3-5.2.0/envs/menpo/lib/python3.5/site-packages/menpo/math/linalg.py in as_matrix(vectorizables, length, return_template, verbose)
    151     i = 0
    152     for i, sample in enumerate(vectorizables, 1):
--> 153         data[i] = sample.as_vector()
    154 
    155     # we have exhausted the iterable, but did we get enough items?

ValueError: could not broadcast input array from shape (14444) into shape (43332)

PS.

visualize_images(training_images)

also failed with

AttributeError: module 'matplotlib.colors' has no attribute 'to_rgba'

, and it couldn't be addressed, which might also help to find the cause of this issue.

[patricksnape]

Is the mean of the first 16 images actually a sensible reference frame? Can you try creating the reference frame from all the shapes instead?