Implement permission policies in the API

[rijkvanzanten]

Scope

What's changed:

• Implements policy-system based permissions handling on the API
• Replaces AuthorizationService with new set of functions in the api/src/permissions folder
• Allows roles to be nested
• Adds new roles flag to accountability object. This is an ordered array of all the parent roles of the current user
• Cleans up get-ast-from-query by splitting it into multiple files
• Permissions are now injected into the AST through cases and whenCase. This allows us to dynamically generate the case/when SQL to have dynamic field output per item.
• Cleans up run-ast by splitting it up into smaller files

Potential Risks / Drawbacks

• The risks are very high. This replaces the full permissions system, and thus needs a lot of testing

Review Notes / Questions

• This PR now compiles, but doesn't run yet.
• There was some weird logic happening in the users controller for TFA enable/disable that I'm not sure we need to keep. Needs a bit more testing

Todos

• Add permissions processing for $CURRENT_USER etc flags
• Add permissions merging when you're accessing from a share
• Add caching to:
  • Fetching Policies
  • Fetching permissions
  • Fetching the roles tree
  • Fetching the field map
  • Fetching allowed fields
  • Fetching allowed collections
• Enable validation for admin users for wrong paths
• Figure out what we want to do with Presets
• Use whenCases in run-ast
• Use applyCases in Meta Service for permissions aware counts
• Handle admin-checks in roles and users services¹
• Add unit testing for clear method in memory/cache
• Make sure graphql gracefully handles optional fields when you have different fields for the same collection in multiple permission sets
• Add changeset
• Unbork /permissions endpoint
• Invalidate cache on permission changes

---

Closes #21778, closes #21765, closes #22163, closes #21769, closes #21768, closes #21767, closes #21766

Footnotes

1. Eg check to make sure there's still >=1 admin left after the mutation is done ↩

[hanneskuettner]

Any reason why the query building was changed from

const flatQuery = knex.select(fieldNodes.map(preProcess)).from(table);

Does the order where using to build the query matter in any way?

[rijkvanzanten]

Order shouldn't matter. In this case it has to go through the applyQuery function; that's the change. Don't remember if there was a reason to feed it through applyQuery without the field selection yet..

[hanneskuettner]

Just to verify, this will reuse existing aliases generated during the first applyFilter through the aliasMap and not result in additional, probably expensive, joins being introduced?

[rijkvanzanten]

That's the thinking, but we have to double check that 👍🏻

[hanneskuettner]

For performance this should probably be a set. Which does preserve insertion order¹, so roles.reverse() becomes const roleList [...roles].reversed().
Now we have the overhead of creating that array, but it still cuts down the includes/has down to O(1).

Footnotes

1. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Set#description ↩

[hanneskuettner]

fetchGlobalAccessForQuery should probably be refactored to use withCache

[hanneskuettner]

All these cases should be covered in the individual tests for extractFieldsFromChildren and extractFieldsFromQuery and we should just make sure that both of them are called with the same fieldMap and the correct parameters?

[hanneskuettner]

What does it mean if rule is null here? As far as I can tell it means that this permission does not have a permission filter, so it matches the globalWhenCase filter? So effectively another question is what is the difference between Permission.permissions == null and == {}.

Right now processChildren returns (Filter|null)[], which is an invalid type for AST.cases.

So we should only add it to cases, if rule != null and use an independently incrementing index instead of the loop index?

Suggested change

	for (const [index, { rule, fields }] of rules.entries()) {
	// If none of the fields in the current permissions rule overlap with the actually requested
	// fields in the AST, we can ignore this case altogether
	if (fields.has('*') === false && Array.from(fields).every((field) => requestedKeys.includes(field) === false)) {
	continue;
	}
	cases.push(rule);
	for (const field of fields) {
	caseMap[field] = [...(caseMap[field] ?? []), index];
	}
	}
	let index = 0;
	for (const { rule, fields } of rules) {
	// If none of the fields in the current permissions rule overlap with the actually requested
	// fields in the AST, we can ignore this case altogether
	if (fields.has('*') === false && Array.from(fields).every((field) => requestedKeys.includes(field) === false)) {
	continue;
	}
	if (rule === null) continue;
	cases.push(rule);
	for (const field of fields) {
	caseMap[field] = [...(caseMap[field] ?? []), index];
	}
	index++;
	}

[rijkvanzanten]

So effectively another question is what is the difference between Permission.permissions == null and == {}.

There's no difference between these two. null and {} should be treated the same in the case of the permission rules

[rijkvanzanten]

So we should only add it to cases, if rule != null and use an independently incrementing index instead of the loop index?

This is a good idea, as we can ignore any case that's null or {} anyways

[hanneskuettner]

You wanna do the honors, or should I?

[hanneskuettner]

Should this return null for role, the invalid role '456-789' or throw an error as it did before ?