Elasticsearch 2.0: how to delete by query in Java

Question

I am trying to upgrade to ES 2.0. I have downloaed ES 2.0 and installed it on my Windows machine.

In my pom.xml, I have the following:

<dependency>
    <groupId>org.elasticsearch</groupId>
    <artifactId>elasticsearch</artifactId>
    <version>2.0.0-rc1</version>
</dependency>

<dependency>
    <groupId>org.elasticsearch.plugin</groupId>
    <artifactId>delete-by-query</artifactId>
    <version>2.0.0-rc1</version>
</dependency>

In my Java code, I did delete by query in the following way when using ES 1.7.3:

    StringBuilder b = new StringBuilder("");
    b.append("{");
    b.append("  \"query\": {");  
    b.append("      \"term\": {");
    b.append("          \"category\": " + category_value );
    b.append("      }");
    b.append("  }");
    b.append("}");

    client = getClient(); 

    DeleteByQueryResponse response = client.prepareDeleteByQuery("myindex")
                .setTypes("mydocytype")
                .setSource(b.toString())
                .execute()
                .actionGet();

I am hoping to replace this:

    DeleteByQueryResponse response = client.prepareDeleteByQuery("myindex")
                .setTypes("mydocytype")
                .setSource(b.toString())
                .execute()
                .actionGet();

with ES 2.0 way. Googled but failed to find an example for it. The online API documentation seems too abstract to me. How can I do it?

Another question: Do I have to install delete-by-query plugin in Elasticsearch server?

Thanks for any pointer!

UPDATE

I followed Max's suggestion, and here is what I have now:

First, when create the client, make settings look like the following:

Settings settings = Settings.settingsBuilder()
                        .put("cluster.name", "mycluster")
                        .put("plugin.types", DeleteByQueryPlugin.class.getName())
                        .build();

Second, at the place doing delete-by-query:

    DeleteByQueryResponse rsp = new DeleteByQueryRequestBuilder(client, DeleteByQueryAction.INSTANCE)
    .setIndices("myindex")
    .setTypes("mydoctype")
    .setSource(b.toString())
    .execute()
    .actionGet();

I also installed delete by query plugin by running the following in the root directory of ES:

bin\plugin install delete-by-query

I get errors if I do not install this plugin.

After all these steps, ES related parts work just fine.

Ümit · Accepted Answer · 2015-12-29 14:33:02Z

12

plugin.types have been deprecated in ES 2.1.0 (source). So the accepted solution will result in a NullPointerException.

The solution is to use the addPlugin method:

Client client = TransportClient.builder().settings(settings())
                .addPlugin(DeleteByQueryPlugin.class)
                .build()
                .addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("host",9300));

answered Dec 29, 2015 at 14:33

Ümit

17.5k7 gold badges58 silver badges75 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Swami PR Over a year ago

What is the purpose of "adding the plugin" on the client side as well?

Max · Accepted Answer · 2015-12-04 12:50:45Z

11

I believe you can use this:

     DeleteByQueryResponse rsp = new DeleteByQueryRequestBuilder(client, DeleteByQueryAction.INSTANCE)
            .setTypes("mydocytype")
            .setSource(b.toString())
            .execute()
            .actionGet();

You have to add plugin type to your settings:

     Settings settings = Settings.settingsBuilder()
                         .put("plugin.types", DeleteByQueryPlugin.class.getName())

If you have remote server you have to install the plugin.

edited Dec 4, 2015 at 12:50

answered Nov 13, 2015 at 13:02

Max

3213 silver badges16 bronze badges

5 Comments

curious1 Over a year ago

Hello Max, thanks for helping me out! What do you mean by "using local node"? I have everything on my notebook. The Java web app runs on port 8080 and ES runs on port 9300. I have to manually start the two one by one. Another thing: Your settings statement is in error. I updated my code, and always got null pointer error.

Max Over a year ago

Hello, with the local node I meant using embedded ES node (elastic.co/guide/en/elasticsearch/client/java-api/current/…). When you use this you don't have to start another elasticsearch client. I think you are connecting to your elasticsearch node using transport client(elastic.co/guide/en/elasticsearch/client/java-api/current/…). Where exactly do you get null pointer ?

Max Over a year ago

Ok I see that even if you don't use embedded node you have to put plugin.types. I am editing my comment for this.

curious1 Over a year ago

Max, thanks for follow-up and making your answer clearer. Really really really appreciate it! I got my code working now due to your help. Best.

Ümit Over a year ago

I get an NPE when I use the delete-by-query plugin with the TransportClient. Happens regardless if I add the plugin type to the settings or not.

Dani · Accepted Answer · 2017-01-07 22:17:09Z

6

From Elastic 5 in onwards...

final BulkIndexByScrollResponse response = DeleteByQueryAction.INSTANCE.newRequestBuilder(super.transportClient)
                    .filter(
                            QueryBuilders.boolQuery()
                                    .must(QueryBuilders.termQuery("_type", "MY_TYPE")) // Trick to define and ensure the type.
                                    .must(QueryBuilders.termQuery("...", "...")))
                    .source("MY_INDEX")
                    .get();

    return response.getDeleted() > 0;

Oficial documentation

answered Jan 7, 2017 at 22:17

Dani

4,2497 gold badges39 silver badges61 bronze badges

1 Comment

curious1 Over a year ago

Dani, thanks for the info. Good to know for upgrade. Cheers.

Yao Pan · Accepted Answer · 2016-07-26 10:23:37Z

2

firstly: add elasticsearch-2.3.3/plugins/delete-by-query/delete-by-query-2.3.3.jar to build path.

then:

Client client = TransportClient.builder().settings(settings)
                .addPlugin(DeleteByQueryPlugin.class)
                .build()
                .addTransportAddress(new InetSocketTransportAddress(
                        InetAddress.getByName("192.168.0.224"), 9300));

answered Jul 26, 2016 at 10:23

Yao Pan

52410 silver badges27 bronze badges

Collectives™ on Stack Overflow

Elasticsearch 2.0: how to delete by query in Java

4 Answers 4

1 Comment

5 Comments

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

1 Comment

5 Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related