How many ContextName can be define in dlt_logstorage.conf?

[Somesh55]

Is there any maximum limit to define CTID in dlt_logstorage.conf?

Issue:
This is dlt offline logstorage, I have defined multiple CTID but I could not able to capture the logs for all.
I have observed that logs from tag TG01 to TG20 are stored but after that logs are not stored.
My observation is till 20th CTID logs are stored. Please clarify, thanks you.

[FILTER1]
LogAppName=.*
ContextName=TG01,TG02,TG03,TG04,TG05,TG06,TG07,TG08,TG09,TG10,TG11,TG12,TG13,TG14,TG15,TG16,TG17,TG18,TG19,TG20,TG21,TG22,TG23,TG24,TG25
LogLevel=DLT_LOG_INFO
File=DLT_LOGS
FileWrtCtl=OFF
FileSize=524288
NOFiles=163

[Somesh55]

Hello All,

I have checked and below is my observation.

Dlt-daemon parses the dlt_logstorage.conf file, line by line.
And store the FILTER data as key and value pair, where key represents the data before delimiter "=" and value represents the data after delimiter "=".

Eg:
[FILTER1]
LogAppName=.*
ContextName=TG01,TG02,TG03,TG04,TG05,TG06,TG07,TG08,TG09,TG10,TG11,TG12,TG13,TG14,TG15,TG16,TG17,TG18,TG19,TG20,TG21,TG22,TG23,TG24,TG25
LogLevel=DLT_LOG_INFO
File=DLT_LOGS
FileWrtCtl=OFF
FileSize=524288
NOFiles=163

In above filter key and value can be,
Key1: LogAppName
Value1: .*

Key2: ContextName
Value2: TG01,TG02,TG03,TG04,TG05,TG06,TG07,TG08,TG09,TG10,TG11,TG12,TG13,TG14,TG15,TG16,TG17,TG18,TG19,TG20,TG21,TG22,TG23,TG24,TG25

and like wise.

Maximum 100 character can be parsed for value.
So, in above filter following CTID will be parsed and rest will not parsed.
TG01,TG02,TG03,TG04,TG05,TG06,TG07,TG08,TG09,TG10,TG11,TG12,TG13,TG14,TG15,TG16,TG17,TG18,TG19,TG20

Code:
https://github.com/COVESA/dlt-daemon/blob/master/src/shared/dlt_config_file_parser.h#L63
https://github.com/COVESA/dlt-daemon/blob/master/src/shared/dlt_config_file_parser.c#L302

If we increase the value of DLT_CONFIG_FILE_ENTRY_MAX_LEN then I can see the logs for beyond 20th CTID.
Please let me know if my observation is correct or not. If it's correct then can we increase the value?

Thank you!

[duvanan13]

Hello @Somesh55
I already verify your observation and suggest create a PR to make this as an option when config logstorage DLT
Hello @michael-methner @minminlittleshrimp
kindly check my point

[michael-methner]

Hello @Somesh55 , hello @duvanan13 ,
we could easily merge a PR which is making the DLT_CONFIG_FILE_ENTRY_MAX_LEN configurable. I guess also the other constants from dlt_config_file_parsher.h should be configurable.

Looking at the pattern from @Somesh55 config file, I think adding the support of regex would be great.

[minminlittleshrimp]

Hello all,
that is good observation. Thanks @Somesh55 a lots and so sorry I have no time and need to forward this topic to An @duvanan13

I already check the available dlt environment variables, maybe we could add this macro to the list?

Another option is to config via dlt-logstorage-ctrl. I prefer this option 😁
Any further ideas?

Hello @duvanan13
thanks for your support on this topic. Kindly cooperate with @Somesh55 on the patch for this feature.
Thank you all in advance

[Somesh55]

Hello @duvanan13 , @michael-methner, @minminlittleshrimp,
Thank you so much for the responses.
Are you going to share the patch for this issue?

[minminlittleshrimp]

Hello @Somesh55
@duvanan13 is working on this topic. I will ask him to create PR.

[minminlittleshrimp]

Hello all,
I have a deeper look into the source code today.
What I see is that this function is the function we are talking about, not the one on previous comment:

dlt-daemon/src/shared/dlt_config_file_parser.c

Line 518 in 35f599d

int dlt_config_file_get_value(const DltConfigFile *file,

This function is called whenever there is section detected (FILTER).
Thus, we should not change the limit of entry (100), it related to other security vulnerability (we need to control the limit size).
I suggest you should split your key into sections (many FILTERs).
Hash table key consists of "APID:CTID", so it is not a matter at all to config multi FILTER sections, e.g:

[FILTER1]
LogAppName=.*
ContextName=TG01,TG02,TG03,TG04,TG05,TG06,TG07,TG08,TG09,TG10,TG11,TG12,TG13,TG14
LogLevel=DLT_LOG_INFO
File=DLT_LOGS
FileWrtCtl=OFF
FileSize=524288
NOFiles=163

[FILTER2]
LogAppName=.*
ContextName=TG15,TG16,TG17,TG18,TG19,TG20,TG21,TG22,TG23,TG24,TG25
LogLevel=DLT_LOG_INFO
File=DLT_LOGS
FileWrtCtl=OFF
FileSize=524288
NOFiles=163

Hello @Somesh55 , @duvanan13
Could you kindly check this by reproducing the case?
Regards

[duvanan13]

Hello @minminlittleshrimp,
I already look into your observation and test it, and it runs exactly as you think, so I think we can close this discussion

[Somesh55]

Hello @minminlittleshrimp @michael-methner @duvanan13,
Thank you so much for working on this topic.
I understood, changing value might cause other vulnerability, so we have to use multiple filter.
Thank you.