How to delete data from ElasticSearch through JavaAPI

Question

EDITED I'm trying to find out how to delete data from Elasticsearch according to a criteria. I know that older versions of ElasticSearch had Delete By Query feature, but it had really serious performance issues, so it was removed. I know also for that there is a Java plugin for delete by query:

org.elasticsearch.plugin:delete-by-query:2.2.0

But I don't know if it has a better implementation of delete which has a better performance or it's the same as the old one.

Also, someone suggested using scroll to remove data, but I know how to retrieve data scrolling, not how to use scroll to remove!

Does anyone have an idea (the amount of documents to remove in a call would be huge, over 50k documents.

Thanks in advance!

Finally used this guy's third option

I don't know, for instance, how to do something like this: curl -XDELETE localhost:9200/_search/scroll -d ' { "scroll_id" : ["c2Nhbjs2OzM0NDg1ODpzRlBLc0FXNlNyNm5JWUc1"] }' but with the API. — user3049941
– user3049941, Commented Dec 27, 2016 at 17:53

jhilden · Accepted Answer · 2016-12-28 04:06:11Z

1

You are correct that you want to use the scroll/scan. Here are the steps:

begin a new scroll/scan
Get next N records
Take the IDs from each record and do a BulkDelete of those IDs
go back to step 2

So you don't delete exactly using the scroll/scan, you just use that as a tool to get all the IDs for the records that you want to delete. In this way you're only deleting N records at a time and not all 50,000 in 1 chunk (which would cause you all kinds of problems).

answered Dec 28, 2016 at 4:06

jhilden

12.5k5 gold badges56 silver badges77 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

user3049941 Over a year ago

I did exactly that, thanks a lot! And sorry for delay replying!

Collectives™ on Stack Overflow

How to delete data from ElasticSearch through JavaAPI

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related