No setup.py to install repo error message when trying to install and insufficient guidance on preparing inputs for the codes...

[sjscotti]

Hi
I am very interested in trying out your repo, but I have had trouble installing it on my Windows PC.
I created an anaconda environment and installed all the dependencies in the requirements.txt file.
I cloned your repo using the command:

git clone https://github.com/CITlabRostock/citlab-article-separation-new.git

Changed directories using:

cd citlab-article-separation-new

But when I tried to install it in my environment, using:

pip install -e .

I got the error message:

ERROR: File "setup.py" not found. Directory cannot be installed in editable mode: D:\citlab-article-separation-new

Is the setup.py file missing, or is there another way the repo should be installed in my environment?
Thanks in advance!

[sjscotti]

I was able to run some of the python codes by putting the article_separation folder from the repo in the site-packages folder for the environment I set up. The run_net_post_processing.py for article separation seems to work well for my sample newspaper images. However, it was unclear how to prepare inputs to run the Text Block Detection python scripts. The only information in the readme.md for the repo was that a PAGE-XML file is needed, but there was no guidance on what specific contents are required for this file and how to obtain them. Is there more detailed guidance on how to go from an image of a newspaper page to obtaining text blocks and to progress on to the subsequent stages of the process?

[oforomar]

Hi @SJScott how did you setup the environment, did you end using a tensorflow-gpu?

[sjscotti]

Hi @SJScott how did you setup the environment, did you end using a tensorflow-gpu?

It has been quite awhile since I ran the article separation but I think these are the packages in the environment...

conda list
# packages in environment at C:\Users\Steve\anaconda3a\envs\citlab-article-separation-new:
#
# Name                    Version                   Build  Channel
abseil-cpp                20211102.0           h0e60522_0    conda-forge
absl-py                   1.2.0              pyhd8ed1ab_0    conda-forge
affine                    2.3.1                    pypi_0    pypi
astor                     0.8.1              pyh9f0ad1d_0    conda-forge
attrs                     22.1.0                   pypi_0    pypi
c-ares                    1.18.1               h8ffe710_0    conda-forge
ca-certificates           2022.6.15            h5b45459_0    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
certifi                   2022.6.15        py37h03978a9_0    conda-forge
click                     8.1.3                    pypi_0    pypi
click-plugins             1.1.1                    pypi_0    pypi
cligj                     0.7.2                    pypi_0    pypi
colorama                  0.4.5                    pypi_0    pypi
cssutils                  2.5.1                    pypi_0    pypi
cycler                    0.11.0                   pypi_0    pypi
et-xmlfile                1.1.0                    pypi_0    pypi
fonttools                 4.34.4                   pypi_0    pypi
freetype                  2.10.4               h546665d_1    conda-forge
gast                      0.5.3              pyhd8ed1ab_0    conda-forge
gdal                      3.0.4                    pypi_0    pypi
google-pasta              0.2.0              pyh8c360ce_0    conda-forge
grpc-cpp                  1.48.0               h95a78a2_0    conda-forge
grpcio                    1.48.0           py37h618d37b_0    conda-forge
h5py                      3.7.0           nompi_py37hebad4f9_100    conda-forge
hdf5                      1.12.1          nompi_h57737ce_104    conda-forge
importlib-metadata        4.11.4           py37h03978a9_0    conda-forge
intel-openmp              2022.1.0          h57928b3_3787    conda-forge
joblib                    1.1.0                    pypi_0    pypi
jpeg                      9e                   h8ffe710_2    conda-forge
jpype1                    1.4.0                    pypi_0    pypi
keras-applications        1.0.8                      py_1    conda-forge
keras-preprocessing       1.1.2              pyhd8ed1ab_0    conda-forge
kiwisolver                1.4.4                    pypi_0    pypi
krb5                      1.19.3               hc8ab02b_0    conda-forge
lcms2                     2.12                 h2a16943_0    conda-forge
lerc                      4.0.0                h63175ca_0    conda-forge
libblas                   3.9.0              15_win64_mkl    conda-forge
libcblas                  3.9.0              15_win64_mkl    conda-forge
libcurl                   7.83.1               h789b8ee_0    conda-forge
libdeflate                1.12                 h8ffe710_0    conda-forge
liblapack                 3.9.0              15_win64_mkl    conda-forge
libpng                    1.6.37               h1d00b33_3    conda-forge
libprotobuf               3.20.1               h7755175_0    conda-forge
libssh2                   1.10.0               h9a1e1f7_2    conda-forge
libtiff                   4.4.0                ha17eb64_2    conda-forge
libwebp                   1.2.3                h8ffe710_1    conda-forge
libwebp-base              1.2.3                h8ffe710_2    conda-forge
libxcb                    1.13              hcd874cb_1004    conda-forge
libzlib                   1.2.12               h8ffe710_2    conda-forge
lxml                      4.9.1                    pypi_0    pypi
lz4-c                     1.9.3                h8ffe710_1    conda-forge
m2w64-gcc-libgfortran     5.3.0                         6    conda-forge
m2w64-gcc-libs            5.3.0                         7    conda-forge
m2w64-gcc-libs-core       5.3.0                         7    conda-forge
m2w64-gmp                 6.1.0                         2    conda-forge
m2w64-libwinpthread-git   5.0.0.4634.697f757               2    conda-forge
markdown                  3.4.1              pyhd8ed1ab_0    conda-forge
markupsafe                2.1.1            py37hcc03f2d_1    conda-forge
matplotlib                3.5.2                    pypi_0    pypi
mkl                       2022.1.0           h6a75c08_874    conda-forge
msys2-conda-epoch         20160418                      1    conda-forge
numpy                     1.21.6           py37h2830a78_0    conda-forge
opencv-python             4.6.0.66                 pypi_0    pypi
openjpeg                  2.4.0                hb211442_1    conda-forge
openpyxl                  3.0.10                   pypi_0    pypi
openssl                   3.0.5                h8ffe710_1    conda-forge
packaging                 21.3               pyhd8ed1ab_0    conda-forge
pillow                    9.2.0            py37h8675073_0    conda-forge
pip                       22.2.1             pyhd8ed1ab_0    conda-forge
protobuf                  3.20.1           py37h03978a9_0    conda-forge
pthread-stubs             0.4               hcd874cb_1001    conda-forge
pymupdf                   1.20.1                   pypi_0    pypi
pyparsing                 3.0.9              pyhd8ed1ab_0    conda-forge
pytesseract               0.3.10             pyhd8ed1ab_0    conda-forge
python                    3.7.12          h900ac77_100_cpython    conda-forge
python-dateutil           2.8.2                    pypi_0    pypi
python_abi                3.7                     2_cp37m    conda-forge
rasterio                  1.1.4                    pypi_0    pypi
re2                       2022.06.01           h0e60522_0    conda-forge
scikit-learn              1.0.2                    pypi_0    pypi
scipy                     1.7.3            py37hb6553fb_0    conda-forge
setuptools                63.3.0           py37h03978a9_0    conda-forge
shapely                   1.8.2                    pypi_0    pypi
six                       1.16.0             pyh6c4a22f_0    conda-forge
snuggs                    1.4.7                    pypi_0    pypi
sqlite                    3.39.2               h8ffe710_0    conda-forge
tbb                       2021.5.0             h2d74725_1    conda-forge
tensorboard               1.14.0                   py37_0    conda-forge
tensorflow                1.14.0               h1f41ff6_0    conda-forge
tensorflow-base           1.14.0           py37hc8dfbb8_0    conda-forge
tensorflow-estimator      1.14.0           py37h5ca1d4c_0    conda-forge
termcolor                 1.1.0              pyhd8ed1ab_3    conda-forge
threadpoolctl             3.1.0                    pypi_0    pypi
tk                        8.6.12               h8ffe710_0    conda-forge
tqdm                      4.64.0                   pypi_0    pypi
typing_extensions         4.3.0              pyha770c72_0    conda-forge
ucrt                      10.0.20348.0         h57928b3_0    conda-forge
vc                        14.2                 hb210afc_6    conda-forge
vs2015_runtime            14.29.30037          h902a5da_6    conda-forge
werkzeug                  2.2.1              pyhd8ed1ab_0    conda-forge
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
wrapt                     1.14.1           py37hcc03f2d_0    conda-forge
xorg-libxau               1.0.9                hcd874cb_0    conda-forge
xorg-libxdmcp             1.1.3                hcd874cb_0    conda-forge
xz                        5.2.5                h62dcd97_1    conda-forge
zipp                      3.8.1              pyhd8ed1ab_0    conda-forge
zlib                      1.2.12               h8ffe710_2    conda-forge
zstd                      1.5.2                h6255e5f_2    conda-forge

pip list
Package              Version
-------------------- ---------
absl-py              1.2.0
affine               2.3.1
astor                0.8.1
attrs                22.1.0
cached-property      1.5.2
certifi              2022.6.15
click                8.1.3
click-plugins        1.1.1
cligj                0.7.2
colorama             0.4.5
cssutils             2.5.1
cycler               0.11.0
et-xmlfile           1.1.0
fonttools            4.34.4
gast                 0.5.3
GDAL                 3.0.4
google-pasta         0.2.0
grpcio               1.48.0
h5py                 3.7.0
importlib-metadata   4.11.4
joblib               1.1.0
JPype1               1.4.0
Keras-Applications   1.0.8
Keras-Preprocessing  1.1.2
kiwisolver           1.4.4
lxml                 4.9.1
Markdown             3.4.1
MarkupSafe           2.1.1
matplotlib           3.5.2
numpy                1.21.6
opencv-python        4.6.0.66
openpyxl             3.0.10
packaging            21.3
Pillow               9.2.0
pip                  22.2.1
protobuf             3.20.1
PyMuPDF              1.20.1
pyparsing            3.0.9
pytesseract          0.3.10
python-dateutil      2.8.2
rasterio             1.1.4
scikit-learn         1.0.2
scipy                1.7.3
setuptools           63.3.0
Shapely              1.8.2
six                  1.16.0
snuggs               1.4.7
tensorboard          1.14.0
tensorflow           1.14.0
tensorflow-estimator 1.14.0
termcolor            1.1.0
threadpoolctl        3.1.0
tqdm                 4.64.0
typing_extensions    4.3.0
Werkzeug             2.2.1
wheel                0.37.1
wrapt                1.14.1
zipp                 3.8.1

[oforomar]

here is my env too:
pip list absl-py 1.4.0 affine 2.4.0 astor 0.8.1 attrs 23.1.0 certifi 2022.12.7 click 8.1.3 click-plugins 1.1.1 cligj 0.7.2 cssutils 2.7.1 cycler 0.11.0 et-xmlfile 1.1.0 fonttools 4.38.0 gast 0.5.4 google-pasta 0.2.0 grpcio 1.54.2 h5py 3.8.0 importlib-metadata 6.7.0 joblib 1.2.0 JPype1 1.4.1 Keras-Applications 1.0.8 Keras-Preprocessing 1.1.2 kiwisolver 1.4.4 lxml 4.9.2 Markdown 3.4.3 MarkupSafe 2.1.3 matplotlib 3.5.3 numpy 1.21.6 opencv-python 4.7.0.72 openpyxl 3.0.10 packaging 23.1 Pillow 9.5.0 pip 22.3.1 protobuf 3.20.3 pyparsing 3.1.0 python-dateutil 2.8.2 rasterio 1.2.10 scikit-learn 1.0.2 scipy 1.7.3 setuptools 65.6.3 Shapely 1.7.0 six 1.16.0 snuggs 1.4.7 tensorboard 1.14.0 tensorflow 1.14.0 tensorflow-estimator 1.14.0 termcolor 2.3.0 threadpoolctl 3.1.0 tqdm 4.65.0 typing_extensions 4.6.3 Werkzeug 2.2.3 wheel 0.38.4 wrapt 1.15.0 zipp 3.15.0

I also copied the article segmentation dir to site-packages,
I try to run using :-
python article_separation/image_segmentation/net_post_processing/run_net_post_processing.py \ --path_to_image_list "list.txt" \ --path_to_pb "nets/separator_detection_net.pb" \ --mode "separator" --num_processes 1

list.txt file contiains only one line: -
/home/ai/ahram/citlab-article-separation-new/test/1997-01-07-0001.tif.png

but keep getting this error
`/home/ai/miniconda3/envs/news/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/ai/miniconda3/envs/news/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/ai/miniconda3/envs/news/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/ai/miniconda3/envs/news/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
WARNING:tensorflow:From /home/ai/miniconda3/envs/news/lib/python3.7/site-packages/article_separation/image_segmentation/net_post_processing/net_post_processing_helper.py:38: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.

WARNING:tensorflow:From /home/ai/miniconda3/envs/news/lib/python3.7/site-packages/article_separation/image_segmentation/net_post_processing/net_post_processing_helper.py:39: The name tf.GraphDef is deprecated. Please use tf.compat.v1.GraphDef instead.

WARNING:tensorflow:From /home/ai/miniconda3/envs/news/lib/python3.7/site-packages/article_separation/image_segmentation/net_post_processing/net_post_processing_helper.py:51: calling import_graph_def (from tensorflow.python.framework.importer) with op_dict is deprecated and will be removed in a future version.
Instructions for updating:
Please file an issue at https://github.com/tensorflow/tensorflow/issues if you depend on this feature.
WARNING:tensorflow:From /home/ai/miniconda3/envs/news/lib/python3.7/site-packages/article_separation/image_segmentation/net_post_processing/net_post_processing_helper.py:61: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

WARNING:tensorflow:From /home/ai/miniconda3/envs/news/lib/python3.7/site-packages/article_separation/image_segmentation/net_post_processing/net_post_processing_helper.py:68: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

2023-06-21 01:58:54.747478: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2023-06-21 01:58:54.767670: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3299990000 Hz
2023-06-21 01:58:54.768067: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x13946b0 executing computations on platform Host. Devices:
2023-06-21 01:58:54.768084: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): ,
2023-06-21 01:58:54.848930: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
`

Do you remember any other step you did before running?