Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Elasticsearch model import method conflicts with Rails 6.1 activerecord import method #1009

Open
robmathews opened this issue Sep 15, 2021 · 2 comments
Labels

Comments

@robmathews
Copy link

robmathews commented Sep 15, 2021

Hi there! Migrating some code from rails 5.2 to rails 6.1.4 and this code is failing:

objs.import(refresh: true)

with a traceback like this:

vendor/bundle/ruby/3.0.0/gems/activerecord-import-1.2.0/lib/activerecord-import/import.rb:648:in `import_helper': Invalid arguments! (ArgumentError)

which seems to be because rails has natively imported the bulk import feature.

I tried working around this by calling

objs.__elasticsearch__.import(refresh: true)

but instead that cause ActiveRecords import method to still be called, and that calls some serialization code, which ends badly with a "document too large' error, like this:

Elasticsearch::Transport::Transport::Errors::RequestEntityTooLarge

so it doesn't work.

@robmathews
Copy link
Author

robmathews commented Oct 14, 2021

Actually, the problem is more that the ActiveRelation integration is broken, so that instead of updating only the specified records, it is attempting to update all records, ignoring the conditions on the relation.

ie, if you do something like:

Article.where(id: 1).in_batches(of: 50) do |objs|
   objs.__elasticsearch__.import(refresh: true)
end

It will fetch all Article, not the specified one as the import.

I tried this without in_batches(of: 50) and got the same result. ie, this simplifies down to:

objs = Article.where(id: 1)
objs.__elasticsearch__.import(refresh: true)

@Aubermean
Copy link

This is still a problem, .import attempts to update all records and ignores the query or scope.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants