Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ajout d'une fn de surpression de dimensions #14

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions pdal_ign_macro/macro.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
import pdal
import numpy
import json

import pdaltools.las_info as li

"""
Some useful filters combinations for complete pdal pipeline
Expand Down Expand Up @@ -83,6 +87,40 @@ def keep_non_planar_pts(pipeline, condition, condition_out):
return pipeline


def remove_dimensions(input_las, dimensions, ouput_las):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pas sûre que ça a sa place dans le même fichier que les macros, parce que tu fais un pipeline à part ici
Potentiellement ça pourrait même aller dans ign-pdal-tools parce que je pense que ce ne sera pas utilisé que dans le contexte du plugin

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oui je suis d'accord avec toi, le mettre dans ign-pdal-tools serait plus judicieux. Je peux le faire. Mais il faut que l'on soit sur du concept en amont.

"""
export new las whitout some dimensions
"""
pipeline = pdal.Pipeline() | pdal.Reader.las(input_las)
pipeline.execute()
points = pipeline.arrays[0]

dim_remove = []
for dim in dimensions:
dim_remove.append(list(points.dtype.fields.keys()).index(dim))

new_points = []
for pt in points:
ptl = list(pt)
ptl = [ptl[i] for i in range(len(ptl)) if i not in dim_remove]
new_points.append(tuple(ptl))

new_dtype = {}
for dim in points.dtype.fields.keys():
if dim in dimensions:
continue
new_dtype[dim] = points.dtype.fields[dim]

new_las = numpy.array(new_points, dtype=new_dtype)
params = li.get_writer_parameters_from_reader_metadata(pipeline.metadata)

pipeline_end = pdal.Pipeline(arrays=[new_las], )
Comment on lines +95 to +117
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ca peut se simplifier en utilisant numpy :

Suggested change
pipeline.execute()
points = pipeline.arrays[0]
dim_remove = []
for dim in dimensions:
dim_remove.append(list(points.dtype.fields.keys()).index(dim))
new_points = []
for pt in points:
ptl = list(pt)
ptl = [ptl[i] for i in range(len(ptl)) if i not in dim_remove]
new_points.append(tuple(ptl))
new_dtype = {}
for dim in points.dtype.fields.keys():
if dim in dimensions:
continue
new_dtype[dim] = points.dtype.fields[dim]
new_las = numpy.array(new_points, dtype=new_dtype)
params = li.get_writer_parameters_from_reader_metadata(pipeline.metadata)
pipeline_end = pdal.Pipeline(arrays=[new_las], )
pipeline.execute()
points = pipeline.arrays[0]
input_dimensions = list(points.dtype.fields.keys())
output_dimensions = [dim for dim in input_dimensions if dim not in dimensions]
points_pruned = points[output_dimensions]
params = li.get_writer_parameters_from_reader_metadata(pipeline.metadata)
pipeline_end = pdal.Pipeline(
arrays=[points_pruned],
)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

vu comme ça... il faut tout de même ajouter le dtype au tableau.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pour moi c'est numpy qui s'en occupe tout seul, parce que la donnée est une structured array https://numpy.org/doc/stable/user/basics.rec.html , mais si t'as un doute ça vaut le coup de vérifier

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oui tu as raison. Je me suis vraiment complexifié la vie !! il va falloir que tu me donnes des cours de numpy

pipeline_end |= pdal.Writer.las(ouput_las, forward="all", **params )

pipeline_end.execute()



def build_condition(key, values):
"""
build 'key==values[0] || key==values[1] ...'
Expand Down
38 changes: 38 additions & 0 deletions test/test_remove_dimension.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
import tempfile
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tu peux mettre ce test dans test/pdal_ign_macro/test_macro.py stp ? (comme ça on a la même arborescence entre code et test, ce sera plus pratique quand on aura plus de code)

C'est valable que si la fonction testée reste là où elle est (à adapter sinon)

import pdal
import numpy

from pdal_ign_macro.macro import remove_dimensions


def test_remove_dimension():

ini_las = "test/data/4_6.las"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ça vaut peut-être le coup de tester avec des extra_dims qui ne sont pas supprimées en plus de ce test-là (pour être sûrs que ça supprime pas toutes les extra_dims d'unef façon ou d'une autre

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bien vu, je rajoute un test

added_dimensions = ["DIM_1", "DIM_2"]

tmp_las = tempfile.NamedTemporaryFile(suffix="_add.las").name
alavenant marked this conversation as resolved.
Show resolved Hide resolved

# recuperation des données ini
pipeline_read_ini = pdal.Pipeline() | pdal.Reader.las(ini_las)
pipeline_read_ini.execute()
points_ini = pipeline_read_ini.arrays[0]

# ajout de dimensions supplémentaires
pipeline = pdal.Pipeline()
pipeline |= pdal.Reader.las(ini_las)
pipeline |= pdal.Filter.ferry(dimensions="=>" + ", =>".join(added_dimensions))
pipeline |= pdal.Writer.las(tmp_las, extra_dims="all", forward="all",)
pipeline.execute()

# suppression des dimensions
remove_dimensions(tmp_las, added_dimensions, tmp_las)

# recuperation des données finales
pipeline_read_end = pdal.Pipeline() | pdal.Reader.las(tmp_las)
pipeline_read_end.execute()
points_end = pipeline_read_end.arrays[0]

# les données ini et finales doivent être identique
assert numpy.array_equal(points_ini, points_end)