Blocking kube container logs to be sent and some minor changes

[naiming-zededa]

• Not sending kube container side of logs, if needed, use collect-info to get the info or logs.
• kube log directory has changed, adjust various component logging to that

[eriknordmark]

If we are going to introduce a new /persist/kubelog directory we need to teach all parts of pillar about it (zedagent reports size of directories, onboot ensures there is sufficient space to boot EVE, etc).

Also, do we already manage the size of the existing /persist/newlog/kube using the limit for /persist/newlog? If so, to what value and what code will manage the space for /persist/kubelog?

[naiming-zededa]

If we are going to introduce a new /persist/kubelog directory we need to teach all parts of pillar about it (zedagent reports size of directories, onboot ensures there is sufficient space to boot EVE, etc).

Yeh, missed that. add to the patch for the /persist/kubelog in the 'ReportDirPaths'

Also, do we already manage the size of the existing /persist/newlog/kube using the limit for /persist/newlog? If so, to what value and what code will manage the space for /persist/kubelog?

This is handled in the cluster-init.sh

        check_log_file_size "k3s.log"
        check_log_file_size "multus.log"
        check_log_file_size "k3s-install.log"
        check_log_file_size "eve-bridge.log"
        check_log_file_size "containerd-user.log"

we rotate them and keep 3 copies with 5M max to each of them.
and currently the kubelog size range is about 30M-60M

[deitch]

Why are we moving it from /persist/newlog/kube to /persist/kubelog?

[naiming-zededa]

Why are we moving it from /persist/newlog/kube to /persist/kubelog?

we first put them inside the 'persist/newlog', since those kubernetes logs does not follow the newlog pattern, and if we want to use 'edgeview' to get the last 15 minutes log, it will get also tons of kuberntes logs. it breaks the way we can precisely getting the time range of newlogs. so we moved it out.

[deitch]

we first put them inside the 'persist/newlog', since those kubernetes logs does not follow the newlog pattern, and if we want to use 'edgeview' to get the last 15 minutes log, it will get also tons of kuberntes logs. it breaks the way we can precisely getting the time range of newlogs

I didn't understand why it does, but it doesn't matter. If you are saying that it breaks it, and we need to move them, good enough. Thanks for explaining.

[naiming-zededa]

Also added /persist/kubelog to 'collect-info.sh'

[eriknordmark]

If we are going to introduce a new /persist/kubelog directory we need to teach all parts of pillar about it (zedagent reports size of directories, onboot ensures there is sufficient space to boot EVE, etc).

Yeh, missed that. add to the patch for the /persist/kubelog in the 'ReportDirPaths'

There is also code in pkg/pillar/scripts/onboot.sh with PERSIST_CLEANUPS being a prioritized list of where to cleanup to make sure there is sufficient space for the system containerd (hence pillar, newlogd, etc) being able to start.

[naiming-zededa]

There is also code in pkg/pillar/scripts/onboot.sh with PERSIST_CLEANUPS being a prioritized list of where to cleanup to make sure there is sufficient space for the system containerd (hence pillar, newlogd, etc) being able to start.

Got it. updated on this file.

[eriknordmark]

LGTM