Impact of Cost on Concurrency limit of a Partition

[udaysagar2177]

Hi, I would like to discuss a feature I need. I am not entirely sure if it fits well with concurrency limiting logic.

Problem: The cost (latency or resource usage) of the jobs should have high impact on the concurrency limit of the partition that issued them. The partition that is issuing costly jobs should receive lower concurrency limit when compared to other partitions with no change in their load pattern. The concurrency limit could get affected overall initially but as soon as the partition with costly jobs is penalized, the other partitions should be restored to normal limits.

To achieve this, do we just maintain separate SimpleLimiter for each of our partition type or is there a better way?

[digvijay7]

+1
@udaysagar2177 : Was there any update on this?

[udaysagar2177]

My opinion would be to use a separate limiter for each partition type