-
-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature: Eventbridge v2: add schedule executor #10817
Feature: Eventbridge v2: add schedule executor #10817
Conversation
424f308
to
ef1d553
Compare
LocalStack Community integration with Pro 2 files ± 0 2 suites ±0 1h 44m 29s ⏱️ + 2m 56s Results for commit 7a82594. ± Comparison against base commit 98dbcbc. This pull request removes 22 and adds 32 tests. Note that renamed tests count towards both.
♻️ This comment has been updated with latest results. |
ef1d553
to
3e52613
Compare
5946c25
to
e5aab49
Compare
af03fb2
to
933ed0a
Compare
response = aws_client.events.list_targets_by_rule(Rule=rule_name) | ||
snapshot.match("list-targets", response) | ||
|
||
time.sleep(60) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
required to wait until rate kicks in - minimum defineable rate duration is 1 minute
], | ||
) | ||
|
||
time.sleep(120) # required to wait for time delta 1 minute starting from next full minute |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
although cron tab fires in 1 minute, it is required to wait until start of next full minute to begin with due to cold start on AWS.
@@ -151,6 +154,12 @@ def __init__(self): | |||
self._rule_services_store: RuleServiceDict = {} | |||
self._target_sender_store: TargetSenderDict = {} | |||
|
|||
def on_before_start(self): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
required to start background thread of job scheduler
delay_secs = schedule.next( | ||
default_utc=True | ||
) # utc default time format for rule schedule cron | ||
# TODO fix execute on exact cron time |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the rework of the job scheduler in v2 needs to deal with executing a schedule cron at the exact specified time, not run the job if it should occur in less than 60 seconds from now.
del self.jobs[i] | ||
else: | ||
i += 1 | ||
self.jobs = [job for job in self.jobs if job.job_id != job_id] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cleaner
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
💡 We still need to make such modifications thread-safe using locks in the scheduler rework.
@pytest.mark.xfail( | ||
reason="This test is flaky is CI, might be race conditions" # FIXME: investigate and fix | ||
) | ||
def test_scheduled_rule_logs( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this test needs to be completely reworked, due to the unknown time requirement this will take, it is being deferred to a later stage since the test case is covered by other tests mostly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
makes sense 👍
Remember to track it in a backlog item, so we don't forget about it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM 👍
This PR integrates the same scheduler as EventBridge v1 and improves test coverage and parity (especially related to schedule conversions).
We will still need to tackle the scheduler rework (based on Thomas's PR #9298) in a follow-up, but it makes sense to tackle this separately as part of the "EventBridge Scheduler" roadmap item and focus on EventBridge provider v1 parity here. Sensible decision 👍
@pytest.mark.xfail( | ||
reason="This test is flaky is CI, might be race conditions" # FIXME: investigate and fix | ||
) | ||
def test_scheduled_rule_logs( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
makes sense 👍
Remember to track it in a backlog item, so we don't forget about it.
del self.jobs[i] | ||
else: | ||
i += 1 | ||
self.jobs = [job for job in self.jobs if job.job_id != job_id] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
💡 We still need to make such modifications thread-safe using locks in the scheduler rework.
tests/aws/services/events/scheduled_rules/test_events_scheduled_rules_logs.py
Show resolved
Hide resolved
b37f1b8
to
7a82594
Compare
Motivation
Rules can have a schedule that invokes the defined (max 5) targets for the rule, not based on incoming events that are matched via a pattern, but this deveined schedule.
https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-create-rule-schedule.html
The schedule can either be specified as a rate expression (https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-rate-expressions.html) or as a cron expression (https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-cron-expressions.html)
Changes
Add validation of schedule cron or rate.
Add schedule function that creates default event or uses custom event input and uses TargetSender to dispatch event.
Fix JobScheduleExecttor to use UTC time, since default cron schedule is in UTC.
Add logic to convert rate to cron.
Expand test suite.