Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix migration fail with encryption key #42617

Merged
merged 4 commits into from
May 14, 2024

Conversation

qnkhuat
Copy link
Contributor

@qnkhuat qnkhuat commented May 14, 2024

Fixes #42612 and #42611

The custom migration was getting raw db.details, but it didn't take into account cases where db-details are encrypted.

This PR fixes that and also:

  • Added a test so that future migrations that touch encrypted entities will be caught, this should prevent these types of bug from appearing
  • Removed the dependency for metabase.models.interface for custom migration; instead, it adds dependency on metabase.util.encryption but I think this ns is less likely to change and safe to use in migration.

@qnkhuat qnkhuat requested a review from camsaul as a code owner May 14, 2024 03:13
@metabase-bot metabase-bot bot added the .Team/BackendComponents also known as BEC label May 14, 2024
@qnkhuat qnkhuat requested a review from a team May 14, 2024 03:23
Copy link

replay-io bot commented May 14, 2024

Status Complete ↗︎
Commit d7d7384
Results
⚠️ 4 Flaky
2504 Passed


(defn- encrypted-json-out
[v]
(let [decrypted (encryption/maybe-decrypt v)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm surprised that this is outside the try, can you explain why that is? A comment in the code would be great.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh I copied the code without much thinking. Maybe it's like that because maybe-encrypt already handles exception graciously?

It's probably better to have it inside the try-catch imo.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't look like you updated this in the final version

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's fine to change this only here even if you don't want to update the app code (yet?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, just realized it's that way because decrypted is also used in the catch body.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh, I see it also does a try catch, and will just return the raw value if there was a failure. Seems like this could fail horribly if the encrypted string is all numeric characters then... 😬

I see there is already a warning logged if the secret key is not preset, or valid. Perhaps we should just raise that log level?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like these non-standard "maybe" semantics one bit 😄

Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
@qnkhuat qnkhuat added the backport Automatically create PR on current release branch on merge label May 14, 2024
@qnkhuat qnkhuat requested a review from a team May 14, 2024 09:00
(catch Throwable e
(if (or (encryption/possibly-encrypted-string? decrypted)
(encryption/possibly-encrypted-bytes? decrypted))
(log/error e "Could not decrypt encrypted field! Have you forgot to set MB_ENCRYPTION_SECRET_KEY?")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't it make more sense to just fetch and check the value of the secret key?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, we can probably get a better error message here by doing that.

But we have tests for it, and I don't want to change some existing behavior as part of an escalation fix.

@qnkhuat qnkhuat merged commit 11c9853 into master May 14, 2024
109 checks passed
@qnkhuat qnkhuat deleted the fix-migration-fail-with-encryption-key branch May 14, 2024 14:09
Copy link

@qnkhuat Did you forget to add a milestone to the issue for this PR? When and where should I add a milestone?

qnkhuat added a commit that referenced this pull request May 14, 2024
qnkhuat added a commit that referenced this pull request May 15, 2024
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport Automatically create PR on current release branch on merge .Team/BackendComponents also known as BEC
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Metabase can't decrypt the data when adding an encryption key after a failed migration
2 participants