[Task] Use apache arrow to store metrics data

[tomsun28]

Description

use apache arrow to store metrics data instead of protobuf Field ValueRow

    // monitoring collect metric field
    repeated Field fields = 9;
    // monitoring collect metric data, mapping with the fields
    repeated ValueRow values = 10;

Task List

No response

[Calvin979]

Is it a task now? I'm interested in it

[tomsun28]

yes welcome. 👍

[tomsun28]

maybe we can use

   // collect response data
    bytes data = 9;

to instead of

    // monitoring collect metric field
    repeated Field fields = 9;
    // monitoring collect metric data, mapping with the fields
    repeated ValueRow values = 10;

in collect_rep.proto

and then use the apache arrow to construct the metrics data bytes data = 9

[Calvin979]

maybe we can use

   // collect response data
    bytes data = 9;

to instead of

    // monitoring collect metric field
    repeated Field fields = 9;
    // monitoring collect metric data, mapping with the fields
    repeated ValueRow values = 10;

in collect_rep.proto

and then use the apache arrow to construct the metrics data bytes data = 9

Thanks for your suggestion, I need to learn Apache Arrow first.