Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inaccurate error message while using "unevaluatedProperties": False #1169

Open
MLL1228 opened this issue Sep 26, 2023 · 2 comments
Open

Inaccurate error message while using "unevaluatedProperties": False #1169

MLL1228 opened this issue Sep 26, 2023 · 2 comments
Labels
Enhancement Some new desired functionality Error Reporting Issues related to clearer or more robust validation error reporting

Comments

@MLL1228
Copy link

MLL1228 commented Sep 26, 2023

I am using jsonschema to validate JSON data against a schema using "unevaluatedProperties": False. When my data contains an unexpected property, the error message lists all the evaluated properties as unexpected, instead of the actual unevaluated property.

Here is a schema as an example:

{
  "allOf": [
    {
      "properties": {
        "street_address": { "type": "string" },
        "city":           { "type": "string" },
        "state":          { 
          "type": "object",
          "properties": {
            "name": { "type": "string" },
          },
          "additionalProperties": False
        }
      }
    },
    {
      "properties": {
        "type":  { "type": "string" },
      }
    }
  ],
  "type": "object",
  "unevaluatedProperties": False
}

And here is a json data:

{
  "street_address": "21 2nd Street",
  "city": "New York",
  "state": {"name": "NY", "cc": "Unexpected field"},
  "type": "home"
}

In this case, 'cc' under 'state' is unexpected. If I remove 'cc', the validation is successful. However, the error message reported by jsonschema is: jsonschema.exceptions.ValidationError: Unevaluated properties are not allowed ('city', 'state', 'street_address' were unexpected).

My questions are:

Why is the error message not mentioning 'cc', the actual cause of the error?
Are there any better ways to provide compatibility with the unevaluatedProperties keyword that could help us identify the specific unevaluated properties?
Please let me know if you need additional details.

Information about my current environment:

Python version: 3.7.16
jsonschema version: 4.16.0
Thanks in advance for your help.

@Julian
Copy link
Member

Julian commented Sep 28, 2023

I have to take a closer look but just off the bat I'd recommend comparing with #1132 or #1026 it may be that's relevant.

(Again not having played with your example) I suspect what's happening has to do with the spec's behavior saying that when a subschema fails, it does not produce annotations -- specifically what that means is that if your subschema is not considering your instance fully valid (the schema in your allOf), then indeed those properties were not evaluated, so they should be considered unevaluated -- there is no "partially" evaluated.

The fact that that error isn't the one you see though is #1132 which is why I point you there.

But I can have a closer look at some point if the above doesn't help / is incorrect on closer inspection.

@Julian Julian added Enhancement Some new desired functionality Error Reporting Issues related to clearer or more robust validation error reporting labels Oct 11, 2023
@ClementDelannoySlateDigital

Hey,

I think I found the same type of issue here.

The real error was
'44100' is not of type 'number'
(this property is defined in an 'allOf' section.
But jsonschema raises an 'Unevaluated properties' with all properties defined on the 'allOf' section.

This is an important issue for me because it leads to false leads that may complexify the debugging a lot.

That would be terrible if it was a design error in the standard...

Have you been able to investigate the issue? I can help looking for something wrong in the logic, please tell me what you've found on your side!

Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement Some new desired functionality Error Reporting Issues related to clearer or more robust validation error reporting
Projects
None yet
Development

No branches or pull requests

3 participants