Skip to content

Latest commit

 

History

History
91 lines (60 loc) · 2.25 KB

README.md

File metadata and controls

91 lines (60 loc) · 2.25 KB
Raw

influxdb-csv-cleaner

Tool to clean CSV content exported from InfluxDB.

What it does

  • Remove repeated header lines, appearing in the middle of file.
  • Remove first column, which is just table (measurement) name.
  • Convert timestamp in "time" column to a readable format, like "2018-01-30 00:00:00".

Build

  1. For BeagleBone

    • Install gcc-6-arm-linux-gnueabihf, binutils-arm-linux-gnueabihf packages.
    • Install armv7-unknown-linux-gnueabihf target for Rust:
    rustup target install armv7-unknown-linux-gnueabihf
    • If you are cross-compiling from PC, need to add this content to .cargo/config file:
    [target.armv7-unknown-linux-gnueabihf]
linker = "arm-linux-gnueabihf-gcc"
    • Build with command:
    cargo build --target=armv7-unknown-linux-gnueabihf --release
    • Strip the compiled file to reduce file size:
    /usr/arm-linux-gnueabihf/bin/strip target/armv7-unknown-linux-gnueabihf/release/influxdb-csv-cleaner

  2. For PC

Too simple to tell.

Example

You export data from InfluxDB with this command:

influx -database myfarm -precision s -format csv -execute "SELECT temperature FROM condition LIMIT 100"

And save to a file sample.csv:

    name,time,temperature
    condition,1489544029,29.1
    condition,1489544039,29.2

Now, you want to remove the first column:

influxdb-csv-cleaner sample.csv -t Asia/Ho_Chi_Minh -o clean.csv

The ouput will be:

    time,temperature
    2017-03-15 09:13:49,29.1
    2017-03-15 09:13:59,29.2

The timestamp column is always convert to readable format. It also help avoid confusing when you use this CSV with graphing tool.

Note: The header line can apear many times in the InfluxDB export file, because influx client then makes chunked queries to handle big data. But influxdb-csv-cleaner tool will skip all of them, except the top line.

You can also use the tool in pipeline to clean on the stdin stream:

influx -database myfarm -precision s -format csv -execute "SELECT temperature FROM condition LIMIT 100" | influxdb-csv-cleaner - -t Asia/Ho_Chi_Minh

Please run

influxdb-csv-cleaner -h

to see more other usages.