You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
When migrate:import's --delete flag initiates a rollback, all skipped records in the ID Map are deleted as well. Example:
[notice] Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) in 1.2 seconds (0/min) - done with 'foo'
[notice] 2 items are missing from source and will be rolled back
5827/5827 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓] 100%
[notice] Rolled back 2721 items - done with 'foo'
Note how it says it will roll back 2 items, yet actually rolls back 2721.
That leaves the migration in an incomplete state with unprocessed items:
------------------ -------- ------- -------------- ------------- ---------------
Migration ID Status Total Imported Unprocessed Last Imported
------------------ -------- ------- -------------- ------------- ---------------
foo Idle 5827 3108 (53.3%) 2719
------------------ -------- ------- -------------- ------------- ---------------
If I run this migration a second time, the skipped rows are added back into the ID map and I'm once again at 100% processed:
------------------ -------- ------- -------------- ------------- ---------------------
Migration ID Status Total Imported Unprocessed Last Imported
------------------ -------- ------- -------------- ------------- ---------------------
foo Idle 5827 3108 (53.3%) 0 2022-02-03 17:07:31
------------------ -------- ------- -------------- ------------- ---------------------
Ideally only the records (and ID Map rows) now missing from the source would be deleted, leaving the unprocessed column at 0.
To Reproduce
Set up a migration that will skip some source rows. Import all source rows with it - note how many rows were skipped. Remove one or two rows from the source. Re-run this migration with the --delete flag. See that both the removed rows, as well as the previously skipped rows are rolled back. See that the migration now has unprocessed rows (via migrate:status).
Expected behavior
When migrate:import's --delete flag initiates a rollback, only records missing from the source are deleted. Previously-skipped, processed rows should be left alone unless they are actually missing from the source.
Actual behavior
Records missing from the source and any skipped, previously-processed, rows are rolled back.
Workaround
Run the migration twice. First run will leave it in an incomplete state with unprocessed rows. The second run will re-process the skipped rows, skipping them again but bringing them back into the map table. The migration will then have zero unprocessed rows and allow dependent migrations to run.
System Configuration
Q
A
Drush version?
11.0.4
Drupal version?
9.3.3
PHP version
7.4
OS?
Linux
Additional information
To get the delete flag working in 11.x I had to first apply the change in #5063.
The text was updated successfully, but these errors were encountered:
Describe the bug
When
migrate:import
's--delete
flag initiates a rollback, all skipped records in the ID Map are deleted as well. Example:Note how it says it will roll back 2 items, yet actually rolls back 2721.
That leaves the migration in an incomplete state with unprocessed items:
If I run this migration a second time, the skipped rows are added back into the ID map and I'm once again at 100% processed:
Ideally only the records (and ID Map rows) now missing from the source would be deleted, leaving the unprocessed column at 0.
To Reproduce
Set up a migration that will skip some source rows. Import all source rows with it - note how many rows were skipped. Remove one or two rows from the source. Re-run this migration with the
--delete
flag. See that both the removed rows, as well as the previously skipped rows are rolled back. See that the migration now has unprocessed rows (viamigrate:status
).Expected behavior
When
migrate:import
's--delete
flag initiates a rollback, only records missing from the source are deleted. Previously-skipped, processed rows should be left alone unless they are actually missing from the source.Actual behavior
Records missing from the source and any skipped, previously-processed, rows are rolled back.
Workaround
Run the migration twice. First run will leave it in an incomplete state with unprocessed rows. The second run will re-process the skipped rows, skipping them again but bringing them back into the map table. The migration will then have zero unprocessed rows and allow dependent migrations to run.
System Configuration
Additional information
To get the delete flag working in 11.x I had to first apply the change in #5063.
The text was updated successfully, but these errors were encountered: