Config - Cannot use hash (#) inside of inner single quotes within multi-line literal string

[joahwoodward-sfmc]

Relevant telegraf.conf

[[inputs.myplugin]]
value = '''This isn't a #comment.'''

Logs from Telegraf

error loading config file /etc/telegraf/inputs/myplugin.conf: error parsing data: line 3: invalid TOML syntax

System info

Telegraf 1.28.2

Docker

No response

Steps to reproduce

We are using a custom input plugin to execute sql queries read from config files, some of which operate on temp tables or have single quotes and/or #'s inside of SQL comments. Telegraf is truncating lines from these queries if it finds a # after a single quote inside of a multiline literal string and causing unexpected behavior and syntax errors.

Expected behavior

Valid TOML is supported and #'s inside multi-line literal strings are never treated as comments or stripped by the config preprocessor.

Actual behavior

Telegraf is treating the # as the beginning of a comment, resulting in invalid toml syntax or unexpected values.
From the example, the config preprocessor will turn it into:

[[inputs.myplugin]]
value = '''This isn't a 

Additional info

Seems to be a regression caused by 7c636b4

[powersj]

Hi,

Can you modify the following to give me an actual example:

[agent]
    debug = true
    omit_hostname = true

[[inputs.exec]]
    commands = ["echo influx value=42"]
    data_format = "influx"

[[processors.starlark]]
  source = '''
# this is a comment
def apply(metric):
  metric.tags["tag"] = "foo#bar"
  return metric
'''

[[outputs.file]]

As-is this returns the following and I don't seem to treat it as a comment:

influx,tag=foo#bar value=42 1698880266000000000

Additionally, do you know if a previous version worked and then a new version did?

[powersj]

Ah I think I missed this part:

if it finds a # after a single quote inside of a multiline literal string

I'll take a look more tomorrow.

[srebhan]

@joahwoodward-sfmc please test the binary in #14240 available as soon as CI finishes the tests. Let me know if this fixes the issue!

[Zegorax]

Hello everyone,

I'm getting the same issue with the SQL input when there is a # (hash, sign number) in the DSN. It's removing the end of the string after the hash because it considers it as a comment, but it should not.

Here is my config :

[[inputs.sql]]
  driver = "oracle"
  dsn = "oracle://usernamewith#inside:password@db:1521/service"

It will throw the following error :
2024-01-19T09:50:59Z E! [inputs.sql] unable to connect to database: unable to connect to database: address usernamewith: missing port in address

I'm not sure how I can fix it. I've tried with the unicode escape method as well (\uXXX), but it's not working as it will parse the rest as a comment.

[Zegorax]

Hello again,

I've investigated further, and it is not linked to this issue. The oracle plugin driver is using an URL parse. Therefore, replacing the # with %23 made it work.