Changing dedup logic to keep most recent entry

cityofcapetown · Dec 18, 2024 · 8e99aba · 8e99aba
1 parent 10cd4d4
commit 8e99aba
Showing 1 changed file with 2 additions and 1 deletion.
diff --git a/src/cct_connector/ServiceAlertFixer.py b/src/cct_connector/ServiceAlertFixer.py
@@ -153,7 +153,8 @@ def dedup_old_data(self):
         self.data = pd.concat((
             old_data, self.data
         )).drop_duplicates(
-            subset=[ID_COL, ]
+            subset=[ID_COL,],
+            keep='last'
         )
         logging.debug(f"    post dedup - {self.data.shape=}")