✨ Extração e Download de Anexos

Adicionada funções principais e auxiliares para extração e parseamento dos anexos na àrvore de Issues. Caso passado uma Inspeção, os anexos são processados e baixados. Caso uma inspeção ou solicitação de inspeção, a árvore é varrida.

docs/constants.html

@@ -46,39 +46,6 @@ <h2 id="IDs-e-Constantes">IDs e Constantes<a class="anchor-link" href="#IDs-e-Co
 
 <div class="cell border-box-sizing code_cell rendered">
 
-</div>
-    {% endraw %}
-
-    {% raw %}
-
-<div class="cell border-box-sizing code_cell rendered">
-<div class="input">
-
-<div class="inner_cell">
-    <div class="input_area">
-<div class=" highlight hl-ipython3"><pre><span></span><span class="n">ID2FIELD</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
-</pre></div>
-
-    </div>
-</div>
-</div>
-
-<div class="output_wrapper">
-<div class="output">
-
-<div class="output_area">
-
-
-
-<div class="output_text output_subarea output_execute_result">
-<pre>dict_items([(2, &#39;Tipo_de_Inspecao&#39;), (5, &#39;Ano&#39;), (25, &#39;Fiscal_Responsavel&#39;), (26, &#39;Fiscais&#39;), (30, &#39;Entidade_da_Inspecao&#39;), (31, &#39;UF_Municipio&#39;), (57, &#39;Servicos_da_Inspecao&#39;), (69, &#39;Qnt_de_emissoes_na_faixa&#39;), (70, &#39;Emissoes_nao_autorizadas&#39;), (71, &#39;Procedimentos&#39;), (89, &#39;Classe_da_Inspecao&#39;), (91, &#39;Horas_de_Preparacao&#39;), (92, &#39;Horas_de_Deslocamento&#39;), (93, &#39;Horas_de_Execucao&#39;), (94, &#39;Horas_de_Conclusao&#39;), (111, &#39;SAV&#39;), (112, &#39;PCDP&#39;), (151, &#39;Uso_de_PF&#39;), (154, &#39;Acao_de_risco_a_vida_criada&#39;), (156, &#39;Frequencia_Inicial&#39;), (157, &#39;Unidade_da_Frequencia_Inicial&#39;), (158, &#39;Frequencia_Final&#39;), (159, &#39;Unidade_da_Frequencia_Final&#39;), (178, &#39;Coordenacao&#39;), (213, &#39;Agrupamento&#39;), (280, &#39;Utilizou_algum_instrumento&#39;), (422, &#39;Numero_Sei_do_Processo&#39;), (450, &#39;Impossibilidade_acesso_online&#39;), (463, &#39;AppFiscaliza&#39;), (541, &#39;Gerar_Relatorio&#39;), (544, &#39;Relatorio_de_Monitoramento&#39;), (543, &#39;Html&#39;), (596, &#39;Reservar_Instrumentos&#39;), (597, &#39;Reserva_de_Instrumentos&#39;), (598, &#39;Utilizou_algum_instrumento&#39;), (717, &#39;Coordenadas_Geograficas&#39;), (726, &#39;Qtd_Licenciadas&#39;), (727, &#39;Qtd_Identificadas&#39;)])</pre>
-</div>
-
-</div>
-
-</div>
-</div>
-
 </div>
     {% endraw %}
 

docs/info.html

@@ -81,7 +81,7 @@
 
 
 <div class="output_markdown rendered_html output_subarea ">
-<h4 id="insp2acao" class="doc_header"><code>insp2acao</code><a href="https://github.com/ronaldokun/fiscaliza/tree/master/fiscaliza/info.py#L29" class="source_link" style="float:right">[source]</a></h4><blockquote><p><code>insp2acao</code>(<strong><code>insp</code></strong>:<code>str</code>, <strong><code>fiscaliza</code></strong>:<code>Redmine</code>)</p>
+<h4 id="insp2acao" class="doc_header"><code>insp2acao</code><a href="https://github.com/ronaldokun/fiscaliza/tree/master/fiscaliza/info.py#L32" class="source_link" style="float:right">[source]</a></h4><blockquote><p><code>insp2acao</code>(<strong><code>insp</code></strong>:<code>str</code>, <strong><code>fiscaliza</code></strong>:<code>Redmine</code>)</p>
 </blockquote>
 <p>Recebe o objeto <code>fiscaliza</code> e a string referente à inspeção <code>insp</code> e retorna um dicionário resumo da Ação atrelada à inspeção</p>
 <p>Args:
@@ -118,7 +118,7 @@ <h4 id="insp2acao" class="doc_header"><code>insp2acao</code><a href="https://git
 
 
 <div class="output_markdown rendered_html output_subarea ">
-<h4 id="issue_type" class="doc_header"><code>issue_type</code><a href="https://github.com/ronaldokun/fiscaliza/tree/master/fiscaliza/info.py#L72" class="source_link" style="float:right">[source]</a></h4><blockquote><p><code>issue_type</code>(<strong><code>insp</code></strong>, <strong><code>fiscaliza</code></strong>)</p>
+<h4 id="issue_type" class="doc_header"><code>issue_type</code><a href="https://github.com/ronaldokun/fiscaliza/tree/master/fiscaliza/info.py#L75" class="source_link" style="float:right">[source]</a></h4><blockquote><p><code>issue_type</code>(<strong><code>insp</code></strong>, <strong><code>fiscaliza</code></strong>)</p>
 </blockquote>
 <p>Checa se a Issue de nº <code>insp</code> do Redmine é de um dos tipos válidos: <code>Inspeção | Ação</code></p>
 
@@ -150,7 +150,7 @@ <h4 id="issue_type" class="doc_header"><code>issue_type</code><a href="https://g
 
 
 <div class="output_markdown rendered_html output_subarea ">
-<h4 id="utf2ascii" class="doc_header"><code>utf2ascii</code><a href="https://github.com/ronaldokun/fiscaliza/tree/master/fiscaliza/info.py#L81" class="source_link" style="float:right">[source]</a></h4><blockquote><p><code>utf2ascii</code>(<strong><code>s</code></strong>)</p>
+<h4 id="utf2ascii" class="doc_header"><code>utf2ascii</code><a href="https://github.com/ronaldokun/fiscaliza/tree/master/fiscaliza/info.py#L84" class="source_link" style="float:right">[source]</a></h4><blockquote><p><code>utf2ascii</code>(<strong><code>s</code></strong>)</p>
 </blockquote>
 <p>Recebe uma string e retorna a mesma em formato ASCII</p>
 
@@ -175,7 +175,7 @@ <h4 id="utf2ascii" class="doc_header"><code>utf2ascii</code><a href="https://git
 
 
 <div class="output_markdown rendered_html output_subarea ">
-<h4 id="detalhar_issue" class="doc_header"><code>detalhar_issue</code><a href="https://github.com/ronaldokun/fiscaliza/tree/master/fiscaliza/info.py#L87" class="source_link" style="float:right">[source]</a></h4><blockquote><p><code>detalhar_issue</code>(<strong><code>issue</code></strong>:<code>str</code>, <strong><code>login</code></strong>:<code>str</code>=<em><code>None</code></em>, <strong><code>senha</code></strong>:<code>str</code>=<em><code>None</code></em>, <strong><code>api</code></strong>:<code>str</code>=<em><code>None</code></em>, <strong><code>fiscaliza</code></strong>:<code>Redmine</code>=<em><code>None</code></em>, <strong><code>teste</code></strong>:<code>bool</code>=<em><code>True</code></em>)</p>
+<h4 id="detalhar_issue" class="doc_header"><code>detalhar_issue</code><a href="https://github.com/ronaldokun/fiscaliza/tree/master/fiscaliza/info.py#L90" class="source_link" style="float:right">[source]</a></h4><blockquote><p><code>detalhar_issue</code>(<strong><code>issue</code></strong>:<code>str</code>, <strong><code>login</code></strong>:<code>str</code>=<em><code>None</code></em>, <strong><code>senha</code></strong>:<code>str</code>=<em><code>None</code></em>, <strong><code>api</code></strong>:<code>str</code>=<em><code>None</code></em>, <strong><code>fiscaliza</code></strong>:<code>Redmine</code>=<em><code>None</code></em>, <strong><code>teste</code></strong>:<code>bool</code>=<em><code>True</code></em>)</p>
 </blockquote>
 <p>Recebe número da inspeção <code>inspecao</code>, o login e senha ou opcionalmente objeto Redmine logado <code>fiscaliza</code>
 inspecao: str - Número da Inspeção a ser relatada
@@ -254,7 +254,7 @@ <h4 id="detalhar_issue" class="doc_header"><code>detalhar_issue</code><a href="h
 
 
 <div class="output_markdown rendered_html output_subarea ">
-<h4 id="download_attachments" class="doc_header"><code>download_attachments</code><a href="https://github.com/ronaldokun/fiscaliza/tree/master/fiscaliza/info.py#L207" class="source_link" style="float:right">[source]</a></h4><blockquote><p><code>download_attachments</code>(<strong><code>issue_obj</code></strong>:<code>Union</code>[<code>str</code>, <code>int</code>], <strong><code>savepath</code></strong>:<code>Union</code>[<code>str</code>, <code>Path</code>], <strong><code>login</code></strong>:<code>str</code>=<em><code>None</code></em>, <strong><code>senha</code></strong>:<code>str</code>=<em><code>None</code></em>, <strong><code>api</code></strong>:<code>str</code>=<em><code>None</code></em>, <strong><code>teste</code></strong>:<code>bool</code>=<em><code>True</code></em>)</p>
+<h4 id="download_attachments" class="doc_header"><code>download_attachments</code><a href="https://github.com/ronaldokun/fiscaliza/tree/master/fiscaliza/info.py#L217" class="source_link" style="float:right">[source]</a></h4><blockquote><p><code>download_attachments</code>(<strong><code>issue_obj</code></strong>:<code>Union</code>[<code>str</code>, <code>int</code>], <strong><code>savepath</code></strong>:<code>Union</code>[<code>str</code>, <code>Path</code>], <strong><code>login</code></strong>:<code>str</code>=<em><code>None</code></em>, <strong><code>senha</code></strong>:<code>str</code>=<em><code>None</code></em>, <strong><code>api</code></strong>:<code>str</code>=<em><code>None</code></em>, <strong><code>teste</code></strong>:<code>bool</code>=<em><code>True</code></em>)</p>
 </blockquote>
 
 </div>
@@ -278,7 +278,31 @@ <h4 id="download_attachments" class="doc_header"><code>download_attachments</cod
 
 
 <div class="output_markdown rendered_html output_subarea ">
-<h4 id="extract_attachments" class="doc_header"><code>extract_attachments</code><a href="https://github.com/ronaldokun/fiscaliza/tree/master/fiscaliza/info.py#L221" class="source_link" style="float:right">[source]</a></h4><blockquote><p><code>extract_attachments</code>(<strong><code>issue_obj</code></strong>:<code>Union</code>[<code>str</code>, <code>int</code>, <code>Issue</code>], <strong><code>login</code></strong>:<code>str</code>=<em><code>None</code></em>, <strong><code>senha</code></strong>:<code>str</code>=<em><code>None</code></em>, <strong><code>api</code></strong>:<code>str</code>=<em><code>None</code></em>, <strong><code>fiscaliza</code></strong>:<code>Redmine</code>=<em><code>None</code></em>, <strong><code>teste</code></strong>:<code>bool</code>=<em><code>True</code></em>, <strong><code>filter</code></strong>:<code>Iterable</code>[<code>str</code>]=<em><code>('relatada', 'conferida', 'aprovada')</code></em>)</p>
+<h4 id="extract_attachment" class="doc_header"><code>extract_attachment</code><a href="https://github.com/ronaldokun/fiscaliza/tree/master/fiscaliza/info.py#L231" class="source_link" style="float:right">[source]</a></h4><blockquote><p><code>extract_attachment</code>(<strong><code>issue_obj</code></strong>, <strong><code>filename</code></strong>:<code>str</code>=<em><code>'Info.json'</code></em>, <strong><code>only_last</code></strong>:<code>bool</code>=<em><code>True</code></em>)</p>
+</blockquote>
+
+</div>
+
+</div>
+
+</div>
+</div>
+
+</div>
+    {% endraw %}
+
+    {% raw %}
+
+<div class="cell border-box-sizing code_cell rendered">
+
+<div class="output_wrapper">
+<div class="output">
+
+<div class="output_area">
+
+
+<div class="output_markdown rendered_html output_subarea ">
+<h4 id="extract_attachments" class="doc_header"><code>extract_attachments</code><a href="https://github.com/ronaldokun/fiscaliza/tree/master/fiscaliza/info.py#L242" class="source_link" style="float:right">[source]</a></h4><blockquote><p><code>extract_attachments</code>(<strong><code>issue_obj</code></strong>:<code>Union</code>[<code>str</code>, <code>int</code>, <code>Issue</code>], <strong><code>login</code></strong>:<code>str</code>=<em><code>None</code></em>, <strong><code>senha</code></strong>:<code>str</code>=<em><code>None</code></em>, <strong><code>api</code></strong>:<code>str</code>=<em><code>None</code></em>, <strong><code>fiscaliza</code></strong>:<code>Redmine</code>=<em><code>None</code></em>, <strong><code>teste</code></strong>:<code>bool</code>=<em><code>True</code></em>, <strong><code>filter</code></strong>:<code>Iterable</code>[<code>str</code>]=<em><code>('relatada', 'conferida', 'aprovada')</code></em>, <strong><code>filename</code></strong>:<code>str</code>=<em><code>'Info.json'</code></em>, <strong><code>only_last</code></strong>:<code>bool</code>=<em><code>True</code></em>)</p>
 </blockquote>
 
 </div>

fiscaliza/_nbdev.py

@@ -33,6 +33,7 @@
          "utf2ascii": "info.ipynb",
          "detalhar_issue": "info.ipynb",
          "download_attachments": "info.ipynb",
+         "extract_attachment": "info.ipynb",
          "extract_attachments": "info.ipynb",
          "atualiza_fiscaliza": "update.ipynb",
          "del_attach": "update.ipynb",

fiscaliza/constants.py

@@ -285,6 +285,7 @@
     "Aguardando Execução": 11,
     "Em andamento": 13,
     "Relatando": 14,
+    "Relatada": 15,
 }
 
 CANCELADA = 19

fiscaliza/info.py

@@ -1,13 +1,14 @@
 # AUTOGENERATED! DO NOT EDIT! File to edit: nbs/info.ipynb (unless otherwise specified).
 
-__all__ = ['insp2acao', 'issue_type', 'utf2ascii', 'detalhar_issue', 'download_attachments', 'extract_attachments']
+__all__ = ['insp2acao', 'issue_type', 'utf2ascii', 'detalhar_issue', 'download_attachments', 'extract_attachment',
+           'extract_attachments']
 
 # Cell
 import re
+import json
 from unidecode import unidecode
 from datetime import datetime, timedelta
 import contextlib
-from pathlib import Path
 from typing import Union, Iterable
 
 from rich import print
@@ -17,8 +18,10 @@
 from redminelib.exceptions import ResourceAttrError
 
 from fastcore.foundation import L
-from fastcore.xtras import is_listy, listify
+from fastcore.xtras import is_listy, listify, Path
 from fastcore.script import Param, call_parse, bool_arg
+from fastcore.parallel import parallel
+from fastcore.basics import partialler
 
 
 from .constants import ACAO_DESCRIPTION, KWARGS, IDS, FIELDS, JSON_FIELDS
@@ -186,10 +189,17 @@ def _download_attachment(
 
     url = getattr(attach, "content_url").replace("http://", "https://")
     setattr(attach, "content_url", url)
+    d = dict(attach)
+    filename = f"{attach.Inspecao}_{attach.id}_{attach.filename}"
     attach.download(
         savepath=savepath,
-        filename=f"{attach.id}_{attach.filename}",
+        filename=filename,
     )
+    d['author'] = d['author']['name']
+    filename = Path(savepath / filename)
+    js = filename.read_json()
+    js.update(d)
+    json.dump(js, filename.open('w'))
 
 
 @call_parse  # Workaround the bug in recursion call in extract_attachments when using as a console script
@@ -215,8 +225,19 @@ def download_attachments(
     attachments = extract_attachments(issue_obj, login, senha, api, teste=teste)
     savepath = Path(f"{savepath}/{issue_obj}")
     savepath.mkdir(exist_ok=True, parents=True)
-    attachments.map(lambda attach: _download_attachment(attach, savepath))
-
+    func = partialler(_download_attachment, savepath=savepath)
+    parallel(func, attachments, threadpool=True, n_workers=min(len(attachments), 16))
+
+def extract_attachment(issue_obj, filename: str = 'Info.json', only_last: bool = True):
+    if attach := L(getattr(issue_obj, "attachments", [])):
+        if filename:
+            attach = attach.filter(lambda x: x.filename == filename)
+        if only_last:
+            ids = attach.attrgot('id')
+            last_id = ids.argwhere(lambda x: x == max(ids))
+            attach = attach[last_id]
+        attach.setattrs('Inspecao', issue_obj.id)
+    return attach
 
 def extract_attachments(
     issue_obj: Union[str, int, Issue],
@@ -226,32 +247,42 @@ def extract_attachments(
     fiscaliza: Redmine = None,
     teste: bool = True,
     filter: Iterable[str] = ('relatada', 'conferida', 'aprovada'),
+    filename: str = 'Info.json',
+    only_last: bool = True,
 ):  # sourcery skip: remove-unnecessary-cast
     attachments = L()
     fiscaliza = valida_fiscaliza(login, senha, api, fiscaliza, teste)
     if not isinstance(issue_obj, Issue) and isinstance(issue_obj, (str, int)):
         issue_obj = fiscaliza.issue.get(issue_obj, include=["relations", "attachments"])
 
+    prefix = issue_obj.id
+
     if (
         getattr(getattr(issue_obj, "tracker", None), "name", None)
         == "Atividade de Inspeção"
     ) and str(getattr(issue_obj, "status", '')).lower() in filter:
-        attachments.extend(getattr(issue_obj, "attachments", []))
+        attachments.extend(extract_attachment(issue_obj, filename, only_last))
         return attachments
 
+
     if relations := getattr(issue_obj, "relations", []):
         relations = (
             L(relations).map(dict).filter(lambda r: r["issue_to_id"] == issue_obj.id)
         )
-        for r in tqdm(relations, total=len(relations)):
+
+        if getattr(getattr(issue_obj, "tracker", None), "name", None) == 'Solicitação de Inspeção':
+            relations = tqdm(relations, total=len(relations))
+
+        for r in relations:
             issue_obj = fiscaliza.issue.get(
                 r["issue_id"], include=["relations", "attachments"]
             )
-
             if (
                 tipo := getattr(getattr(issue_obj, "tracker", None), "name", None)
-            ) == "Ação de Inspeção":
+            ) in {"Solicitação de Inspeção", "Ação de Inspeção"}:
                 attachments.extend(extract_attachments(issue_obj, fiscaliza=fiscaliza))
             elif tipo == "Atividade de Inspeção" and str(getattr(issue_obj, "status", '')).lower() in filter:
-                attachments.extend(getattr(issue_obj, "attachments", []))
+                    attach = extract_attachment(issue_obj, filename, only_last)
+                    attach.setattrs('Acao', prefix)
+                    attachments.extend(attach)
     return attachments

fiscaliza/update.py

@@ -90,7 +90,7 @@ def excluir_relatorio(
         }
         temp["Html"] = {"id": FIELD2ID["Html"], "value": ""}
         atualiza_fiscaliza(
-            inspecao, temp, fiscaliza, status=status_atual["status"].name
+            inspecao, temp, fiscaliza, status=status_atual["status"]
         )
         status_atual = detalhar_issue(inspecao, fiscaliza=fiscaliza, teste=teste)
         if status_atual.get("Relatorio_de_Monitoramento"):
@@ -177,7 +177,7 @@ def relatar_inspecao(
     with console.status("Resgatando Situação Atual da Inspeção...", spinner="pong"):
         status_atual = detalhar_issue(inspecao, fiscaliza=fiscaliza, teste=teste)
 
-    atual = getattr(status_atual["status"], "name")
+    atual = status_atual["status"]
 
     console.print(f":white_check_mark: [cyan]Estado Atual: [bold green]{atual}")
 

nbs/constants.ipynb

@@ -318,27 +318,6 @@
     "}"
    ]
   },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "2ad67bfd",
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "dict_items([(2, 'Tipo_de_Inspecao'), (5, 'Ano'), (25, 'Fiscal_Responsavel'), (26, 'Fiscais'), (30, 'Entidade_da_Inspecao'), (31, 'UF_Municipio'), (57, 'Servicos_da_Inspecao'), (69, 'Qnt_de_emissoes_na_faixa'), (70, 'Emissoes_nao_autorizadas'), (71, 'Procedimentos'), (89, 'Classe_da_Inspecao'), (91, 'Horas_de_Preparacao'), (92, 'Horas_de_Deslocamento'), (93, 'Horas_de_Execucao'), (94, 'Horas_de_Conclusao'), (111, 'SAV'), (112, 'PCDP'), (151, 'Uso_de_PF'), (154, 'Acao_de_risco_a_vida_criada'), (156, 'Frequencia_Inicial'), (157, 'Unidade_da_Frequencia_Inicial'), (158, 'Frequencia_Final'), (159, 'Unidade_da_Frequencia_Final'), (178, 'Coordenacao'), (213, 'Agrupamento'), (280, 'Utilizou_algum_instrumento'), (422, 'Numero_Sei_do_Processo'), (450, 'Impossibilidade_acesso_online'), (463, 'AppFiscaliza'), (541, 'Gerar_Relatorio'), (544, 'Relatorio_de_Monitoramento'), (543, 'Html'), (596, 'Reservar_Instrumentos'), (597, 'Reserva_de_Instrumentos'), (598, 'Utilizou_algum_instrumento'), (717, 'Coordenadas_Geograficas'), (726, 'Qtd_Licenciadas'), (727, 'Qtd_Identificadas')])"
-      ]
-     },
-     "execution_count": null,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "ID2FIELD.items()"
-   ]
-  },
   {
    "cell_type": "markdown",
    "id": "fd139cae",
@@ -360,6 +339,7 @@
     "    \"Aguardando Execução\": 11,\n",
     "    \"Em andamento\": 13,\n",
     "    \"Relatando\": 14,\n",
+    "    \"Relatada\": 15,\n",
     "}\n",
     "\n",
     "CANCELADA = 19\n",
@@ -6168,7 +6148,8 @@
       "Converted constants.ipynb.\n",
       "Converted format.ipynb.\n",
       "Converted index.ipynb.\n",
-      "Converted redmine.ipynb.\n",
+      "Converted info.ipynb.\n",
+      "Converted update.ipynb.\n",
       "Converted validation.ipynb.\n"
      ]
     }
@@ -6182,7 +6163,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python [conda env:fiscaliza]",
+   "display_name": "Python 3.9.12 ('teste')",
    "language": "python",
    "name": "python3"
   }