#738 #1990 Route Metadata as custom properties

[vantm]

Closes #738 #1990

• Configuration for use by extensions, custom pipelines, or other extensions #738
• Proposal to add ExtraProps to the configuration #1990

Related to

• Extending Route configuration #651

Proposed Changes

• Adding metadata to the global configuration
• Adding metadata to the route configuration

Route metadata allows Ocelot users to add custom functions that solve ad-hoc requirements or maybe write their plugins/extensions.

The new configuration will look like:

{
  "Routes": [
      {
          "UpstreamHttpMethod": [ "GET" ],
          "UpstreamPathTemplate": "/posts/{postId}",
          "DownstreamPathTemplate": "/api/posts/{postId}",
          "DownstreamHostAndPorts": [
              { "Host": "localhost", "Port": 80 }
          ],
          "Metadata": {
              "api-id": "FindPost",
              "my-extension/param1": "overwritten-value",
              "other-extension/param1": "value1",
              "other-extension/param2": "value2",
              "tags": "tag1, tag2, area1, area2, feat1"
          }
      }
  ],
  "GlobalConfiguration": {
      "Metadata": {
          "server_name": "data-center-1-machine-1",
          "my-extension/param1": "default-value"
      }
  }
}

Now, the metadata is available in the DownstreamRoute object

var downstreamRoute = context.Items.DownstreamRoute();

if(downstreamRoute?.Metadata is {} metadata)
{
    var param1 = metadata.GetValueOrDefault("my-extension/param1") ?? throw new MyExtensionException("Param 1 is null");
    var param2 = metadata.GetValueOrDefault("my-extension/param2", "custom-value");

    // working with metadata
}

[raman-m]

Hi Van!
Thanks for PR submission!
Welcome to Ocelot world! 🐯

Please, Be patient! I need some time to read and understand linked issues...

[raman-m]

Extending route config by custom properties is interesting idea.
Please keep in mind we have #1183 which is similar.

[ggnaegi]

Extending route config by custom properties is interesting idea. Please keep in mind we have #1183 which is similar.

Are they really similar? The route metadata is a nice addition.

[raman-m]

@ggnaegi commented on Dec 10

Right, the technical designs are different.. But these both issues are about the same custom properties for route. Similarity in this thing only.
🙆‍♂️ I've put the Accepted label onto the linked issue #738

@RaynaldM Thanks for the review!
I have no idea which milestone we will include this PR in... Probable candidate for Jan'24 release if the author will be proactive...

[vantm]

All feedback has been resolved. Thank everybody for taking the time to work on my PR. Thank @RaynaldM for your reviews.

[raman-m]

• 1st concern, Why Metadata name?
  We have other names from linked issues: Extensions, see comment
  Internal code of Ocelot middlewares uses the HttpContext class Items dictionary to keep extra data of the route/request, see HttpContext.Items Property
  So, we use HttpContext.Items dictionary across all middlewares...

• 2nd concern is IDictionary<string, string> type of the Metadata property. But developer will use another types like int, double, even object... So, string data type looks like a restriction in data types we can define in JSON...

[vantm]

Hi @raman-m

1. • Why Use the Metadata Name? The linked issue lacks a detailed design; the term "Extensions" is merely a suggestion. In my opinion, "Metadata" carries a much broader meaning.
   • We use HttpContext.Items dictionary across all middlewares... As per my understands of Ocelot, it's a yes, but only when you want to access or manipulate configurations/states are stored here. Referencing these extension methods of HttpContext.Items here, it seems there is no restriction or issue of using them.
2. • I believe it's not advisable to manage entities beyond our scope. That means there is no strong-typed binding, metadata validation, complex metadata structure handling, etc. It is simply attaching extra information to a route. That is why I ended up with the approach of using key/pair data type (this idea is similar to the k8s metadata annotations). Enabling support for a wide array of data types would necessitate heavy design and implementation overhead.
   • Using string is a restriction in data types While string indeed restricts the data type, it doesn't restrict the data format. A string can express many formats such as numbers, plain text, JSON, XML, HTML, etc.

[raman-m]

@ggnaegi Could you please explain why you rebased this feature branch again? It appears the branch had already been rebased onto release/24. Why was this done?

Keep in mind that if rebasing occurs on the origin, you'll need to remove the branch in your local repository, fetch all changes from the origin, and then check out the branch again. If your local branch differs from the origin, you need not to force a rebase: just re-checkout!

It seems that the local release/24.0 was not synchronized... as there are now commits from develop. It appears you have rebased onto develop, which is incorrect.

⚠️ Important Notice

Parallel releases are currently in progress. The develop branch and the release/24.0 branch must not be synchronized or merged with one another!
Seems "March-April'24" will be released first. After that we will rebase onto, or just merge develop into release/24.0.

[raman-m]

@ggnaegi requested review from @raman-m, @RaynaldM and @ggnaegi on May 3

I'm unable to review due to redundant commits from the develop branch. Please create a backup branch to preserve the current state locally, then rebase onto ThreeMammals:release/24.0. If you require assistance, feel free to ask; I have a reliable local copy of the feature branch and can perform another force push if necessary.

[ggnaegi]

@ggnaegi requested review from @raman-m, @RaynaldM and @ggnaegi on May 3

I'm unable to review due to redundant commits from the develop branch. Please create a backup branch to preserve the current state locally, then rebase onto ThreeMammals:release/24.0. If you require assistance, feel free to ask; I have a reliable local copy of the feature branch and can perform another force push if necessary.

@raman-m Ok, then just do that, I don't have a reliable local copy anymore... Thanks. I probably forgot to delete the local copy and checkout.

[ggnaegi]

@raman-m all my commits are gone? 😅

[raman-m]

@ggnaegi Negative! Your commits are preserved!
The feature branch has been successfully rebased onto ThreeMammals:release/24.0❕ Done!
There are no redundant commits from develop.

If you want to contribute more, just delete branch locally, fetch all and checkout once again.

[raman-m]

@ggnaegi approved these changes on May 3

I apologize, but where can I find your approval?

I will conduct a review today as well, since it appears that most of the delivery artifacts, including documentation, are present.

@raman-m , the documentation might need to be completed though...

[raman-m]

Excellent work, team, @ggnaegi @vantm 💪
Here are my suggestions:

Major Issues:

• The feature functionality should be organized by software features (modules): in Dependency Injection, separate folders.
• The use of .NET code reflection technology in the library is fundamentally incorrect!

Sorry! We can't merge it for now! Let's work a bit more...

[ggnaegi]

Excellent work, team, @ggnaegi @vantm 💪
Here are my suggestions:

Major Issues:

• The feature functionality should be organized by software features (modules): in Dependency Injection, separate folders.
• The use of .NET code reflection technology in the library is fundamentally incorrect!

Sorry! We can't merge it for now! Let's work a bit more...

Why is code reflection a problem? Performance, maintainability?. I used the code reflection to avoid issues with functionalities that aren't supported in .NET 6 yet.

I don't agree with you, until we provide coding standards, this PR is good to be merged.

[raman-m]

@ggnaegi Gui,

Why is code reflection a problem? Performance, maintainability ?.

Code reflection can pose issues with performance and seems no issues with maintainability.
Indeed, you've got it right! In the context of Ocelot, if reflection occurs only at application startup, it typically doesn't present a problem. However, if reflection is executed for each request, it can lead to performance degradation by adding extra milliseconds to the request's processing time.
Finally, I'd say: reflection is a code smell, for high load systems it is real performance problem.

I used the code reflection to avoid issues with functionalities that aren't supported in .NET 6 yet.

Which functionality is unavailable in .NET 6? Is it the Parse method of a numeric type? 😃 You said: the exact issue addressed with the reflected "Parse" method was to bypass the absence of certain functionalities. But which functionality?
Why not invoke Parse directly?
Are you familiar with the IConvertible interface?

I don't agree with you, until we provide coding standards, this PR is good to be merged.

I concur that merging is a good step, but it's better to address the issues I've identified. If you're short on time, I can tackle the problems highlighted in my code review. However, my immediate focus is on the current monthly release. Once that's handled, I'll return to this pull request.

[ggnaegi]

I concur that merging is a good step, but it's better to address the issues I've identified. If you're short on time, I can tackle the problems highlighted in my code review. However, my immediate focus is on the current monthly release. Once that's handled, I'll return to this pull request.

@raman-m what monthly release? It wasn't planed like that!

[raman-m]

@ggnaegi

what monthly release? It wasn't planed like that!

This monthly release → March-April'24
And what we do now → Milestones

[ggnaegi]

@raman-m ready for re-review