Store roles in database

[Al2Klimov]

TODO

• upgrading SQL
• Postgres
• ORM
• CRUD
  • pagination
  • search
• actual usage
  • audit ok?
• management via Ansible, Puppet, ... and via environment variables (Docker)
• migration
• installer?

[Al2Klimov]

@lippserd Was letzte Schema? :)

[lippserd]

Please show sample queries of how roles of a sample user would be loaded. Also note that CTEs can be used.

I'm unsure of the icingaweb_permission and icingaweb_restriction tables. They seem to have zero to no benefit and only make things more complicated. What was your idea behind these tables?

Is ROW_FORMAT=DYNAMIC really needed everywhere? I don't think so.

Please increase all name columns to 256. The SAM-Account-Name attribute can be that long.

[Al2Klimov]

Please show sample queries of how roles of a sample user would be loaded. Also note that CTEs can be used.

WITH RECURSIVE rl(id, parent_id, unrestricted) AS (
  SELECT id, parent_id, unrestricted FROM icingaweb_role
  WHERE all_users OR all_groups
-- I guess an index over (all_users, all_groups) would be nice, wouldn't it?
  OR id IN (SELECT role_id FROM icingaweb_role_user WHERE user_name=?)
-- Shall I swap icingaweb_role_user's PK's columns for this query by user_name?
  OR id IN (SELECT role_id FROM icingaweb_role_group WHERE group_name IN (?, ?, ?))
-- Shall I swap icingaweb_role_group's PK's columns for this query by group_name?
UNION
-- This even handles relationship cycles. 😎
  SELECT id, parent_id, unrestricted FROM icingaweb_role
  WHERE id IN (SELECT parent_id FROM rl)
)
SELECT id, unrestricted FROM rl;

SELECT name FROM icingaweb_permission WHERE id IN (
  SELECT permission_id FROM icingaweb_role_permission WHERE role_id IN (?, ?, ?)
)

SELECT (SELECT name FROM icingaweb_restriction WHERE id=irr.restriction_id) name, filter
FROM icingaweb_role_restriction irr WHERE role_id IN (?, ?, ?)

I'm unsure of the icingaweb_permission and icingaweb_restriction tables. They seem to have zero to no benefit and only make things more complicated. What was your idea behind these tables?

The same idea as behind Icinga DB's action_url and notes_url: deduplication of probably frequently used strings.

Is ROW_FORMAT=DYNAMIC really needed everywhere? I don't think so.

I thought this is our new guideline as Icinga DB uses this literally everywhere. (grep ENGINE=InnoDB schema/mysql/schema.sql |grep -v ROW_FORMAT=DYNAMIC)

Please increase all name columns to 256. The SAM-Account-Name attribute can be that long.

It's length is actually limited to 20:

• https://serverfault.com/a/627701
• https://learn.microsoft.com/en-us/windows/win32/adschema/a-samaccountname

And my length limit was just inspired from the existing username columns which are btw. 254 long to contain eMail addresses.

[Al2Klimov]

Btw.:

Distros supporting CTEs

via MariaDB 10.2.1+

• SUSE 12.5 👍
• Debian 10 👍
• Ubuntu 20.04 👍
• RHEL 8 👎 (RHEL 7 EOL: 2024-06-30)

[Al2Klimov]

Is ROW_FORMAT=DYNAMIC really needed everywhere? I don't think so.

Now as I see it: It IS literally everywhere, even the damn simple table icingaweb_user of this schema. Let's keep it for consistency.

[lippserd]

Let's keep it for consistency.

No. It is not needed. And consistency is no reason to add it. New code should reflect how it's done properly and not how it was done ages ago.

[lippserd]

Distros supporting CTEs

via MariaDB 10.2.1+

...

• RHEL 8 👎 (RHEL 7 EOL: 2024-06-30)

Red Hat states something else.

[Al2Klimov]

That's what I wanted to express, sorry for the misunderstanding. Btw. Ubuntu 20.04 also has MySQL 8 and RHEL 7 has PostgreSQL (almost forgot!) 9.2. v8.4 has CTEs.

[Al2Klimov]

Is ROW_FORMAT=DYNAMIC really needed everywhere? I don't think so.

It seems the default, so no – it's indeed not needed.

[Al2Klimov]

WITH RECURSIVE rl(id, parent_id, unrestricted) AS (
  SELECT id, parent_id, unrestricted FROM icingaweb_role
  WHERE id IN (SELECT role_id FROM icingaweb_role_user WHERE user_name IN ('*', ?))
  OR id IN (SELECT role_id FROM icingaweb_role_group WHERE group_name IN ('*', ?, ?, ?))
UNION -- This even handles relationship cycles. 😎
  SELECT id, parent_id, unrestricted FROM icingaweb_role
  WHERE id IN (SELECT parent_id FROM rl)
)
SELECT id, unrestricted FROM rl;

[Al2Klimov]

If this is an old schema,

CREATE TABLE `icingaweb_role`(
  `name`    varchar(64) COLLATE utf8_unicode_ci NOT NULL,
  `public`  tinyint(1) NOT NULL DEFAULT 0,
  `ctime`   timestamp NULL DEFAULT NULL,
  `mtime`   timestamp NULL DEFAULT NULL ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE `icingaweb_role_permission`(
  `role_name`   varchar(64) COLLATE utf8_unicode_ci NOT NULL,
  `name`        varchar(255) COLLATE utf8_unicode_ci NOT NULL,
  `ctime`       timestamp NULL DEFAULT NULL,
  `mtime`       timestamp NULL DEFAULT NULL ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (`role_name`,`name`),
  CONSTRAINT `fk_icingaweb_role_permission_icingaweb_role` FOREIGN KEY (`role_name`) REFERENCES `icingaweb_role` (`name`)
    ON UPDATE CASCADE ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE `icingaweb_role_restriction`(
  `role_name`   varchar(64) COLLATE utf8_unicode_ci NOT NULL,
  `name`        varchar(255) COLLATE utf8_unicode_ci NOT NULL,
  `restriction` text NOT NULL,
  `ctime`       timestamp NULL DEFAULT NULL,
  `mtime`       timestamp NULL DEFAULT NULL ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (`role_name`,`name`),
  CONSTRAINT `fk_icingaweb_role_restriction_icingaweb_role` FOREIGN KEY (`role_name`) REFERENCES `icingaweb_role` (`name`)
    ON UPDATE CASCADE ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE `icingaweb_user_role`(
  `username`  varchar(64) COLLATE utf8_unicode_ci NOT NULL,
  `role_name` varchar(64) COLLATE utf8_unicode_ci NOT NULL,
  `ctime`     timestamp NULL DEFAULT NULL,
  `mtime`     timestamp NULL DEFAULT NULL ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (`username`,`role_name`),
  CONSTRAINT `fk_icingaweb_user_role_icingaweb_role` FOREIGN KEY (`role_name`) REFERENCES `icingaweb_role` (`name`)
    ON UPDATE CASCADE ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE `icingaweb_group_role`(
  `group_name`  varchar(64) COLLATE utf8_unicode_ci NOT NULL,
  `role_name`   varchar(64) COLLATE utf8_unicode_ci NOT NULL,
  `ctime`       timestamp NULL DEFAULT NULL,
  `mtime`       timestamp NULL DEFAULT NULL ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (`group_name`,`role_name`),
  CONSTRAINT `fk_icingaweb_group_role_icingaweb_role` FOREIGN KEY (`role_name`) REFERENCES `icingaweb_role` (`name`)
    ON UPDATE CASCADE ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

we can

1. Make room for the new tables
   • alter table icingaweb_role rename to icingaweb_role1;
   • alter table icingaweb_role_permission rename to icingaweb_role_permission1;
   • alter table icingaweb_role_restriction rename to icingaweb_role_restriction1;
2. Upgrade the schema as usual
3. Migrate the data
   • insert into icingaweb_role(name, ctime, mtime) select name, unix_timestamp(ctime)*1000, unix_timestamp(mtime)*1000 from icingaweb_role1;
   • insert into icingaweb_role_user(role_id, user_name) select r.id, ur.username from icingaweb_user_role ur inner join icingaweb_role r on r.name = ur.role_name;
   • insert into icingaweb_role_group(role_id, group_name) select r.id , gr.group_name from icingaweb_group_role gr inner join icingaweb_role r on r.name = gr.role_name;
   • insert into icingaweb_role_permission(role_id, permission, allowed) select r.id, p1.name, 'y' from icingaweb_role_permission1 p1 inner join icingaweb_role r on r.name = p1.role_name;
   • insert into icingaweb_role_restriction(role_id, restriction, filter) select r.id, r1.name, r1.restriction from icingaweb_role_restriction1 r1 inner join icingaweb_role r on r.name = r1.role_name;

[Al2Klimov]

• management via Ansible, Puppet, ... and via environment variables (Docker)

Our Docker entry point already handles users in DB, so roles wouldn't be a problem. But the others already can't manage users or groups. Roles would be just yet another blind point (unless you let Ansible do INSERT). I think we need a more general strategy. Like a bunch of CLI commands (in the setup mod? or even a new mod "cfgmgmt"?) which manage stuff stored in the IW2 DB.

[Al2Klimov]

• management via Ansible, Puppet

Meeting results

The comrades already populate the admin user (and role), so that you can login at all, but only initially not to fiddle around in the db, so status quo here is OK.

Setup mod. CLI commands would be very nice, but -in our opinion- for all db stuff incl. user/groups. A separate (not small) project IMAO.

A cherry on the cake would be a per-row flag (again for all db stuff, separate construction area) for managed-externally, so that users can't change those Puppet-deployed stuff temporarily and get mad because it disappears again.

Question to @lippserd(?): when will the default roles backend change?