Delete / add nested objects in Elastic search

Question

I cannot find examples in the Elastic manual on nested objects on how to modify fields and nested objects of documents using RESTful commands in Kibana Sense. I am looking for something similar to Solrs atomic updates here, which allow to update specific fields of documents.

How do RESTful commands in Kibana Sense look like that accomplish this? The only related info in the manual I can find is on Partial Updates to Documents, but I do not know how this can be applied for this use case.

For example, straight from the Elastic docs:

PUT my_index
{
"mappings": {
    "my_type": {
    "properties": {
        "user": {
        "type": "nested" 
        }
    }
    }
}
}

PUT my_index/my_type/1
{
"group" : "fans",
"user" : [
    {
    "first" : "John",
    "last" :  "Smith"
    },
    {
    "first" : "Alice",
    "last" :  "White"
    }
]
}

How can I delete an entry in the nested object, so that the document "1" looks like:

{
"group" : "fans",
"user" : [
    {
    "first" : "John",
    "last" :  "Smith"
    }
]
}

How can I add an entry in the nested object, so that the document "1" looks like:

{
"group" : "fans",
"user" : [
    {
    "first" : "John",
    "last" :  "Smith"
    },
    {
    "first" : "Alice",
    "last" :  "White"
    },
    {
    "first" : "Peter",
    "last" :  "Parker"
    }
]
}

Juha Vehnia · Accepted Answer · 2020-03-06 04:41:50Z

3

You will have to use scripted updates unless you want to fetch all nested objects then add / remove items and re-index them all which is the previous answer proposed. However if you have a lot of nested documents you should be doing partial updates / additions and deletes. It is much quicker from data transfer and indexing point of view.

Here is a good article how to do scripted updates in general:

https://iridakos.com/programming/2019/05/02/add-update-delete-elasticsearch-nested-objects

answered Mar 6, 2020 at 4:41

Juha Vehnia

1,42815 silver badges16 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Community · Accepted Answer · 2017-05-23 10:34:09Z

0

Unless I misunderstand your ask, you just post the updated document version to the same document id each time you want.

To delete a nested document (or any field):

PUT my_index/my_type/1
{
  "group" : "fans",
  "user" : [
    {
      "first" : "Alice",
      "last" :  "White"
    }
  ]
}

To add a user, add it to the list:

PUT my_index/my_type/1
{
  "group" : "fans",
  "user" : [
    {
      "first" : "Alice",
      "last" :  "White"
    },
    {
      "first" : "Peter",
      "last" :  "Parker"
    }
  ]
}

Note: Documents in elasticsearch are immutable. Making a change to a single field causes the entire document to be re-indexed. Nested documents are always re-indexed with the parent document so if you change a field in the parent the nested document is also re-indexed. This can be a performance issue if the nested documents are large and the parents have frequent changes.

edited May 23, 2017 at 10:34

CommunityBot

11 silver badge

answered Sep 18, 2016 at 0:37

Phil

1,28610 silver badges21 bronze badges

6 Comments

tkja Over a year ago

I want to add or delete within the nested "user" object, not re-index a new (potentially big) document again. I am looking for something similar to Solrs "atomic updates", I updated the question to clarify this.

Phil Over a year ago

What you want is not possible with elasticsearch. Documents are immutable. I adjusted my answer to include a note about this. You will probably want to un-nest the documents and link them through some application logic.

tkja Over a year ago

What about elastic.co/guide/en/elasticsearch/guide/master/… (Partial Updates to Documents)? Could this be used to accomplish a RESTful command to directly update as in my example?

Phil Over a year ago

You can certainly use an update script but I'm not sure that is much better. Either way the entire document is still re-indexed after the update: see elastic.co/guide/en/elasticsearch/guide/current/… & elastic.co/guide/en/elasticsearch/reference/current/… Let me know if it works the way you want.

Phil Over a year ago

This documentation should get you started: elastic.co/guide/en/elasticsearch/reference/master/…. You'll want to manipulate the _source.user field.

|

Reid · Accepted Answer · 2022-01-03 16:44:15Z

For this specific use case, you must use a scripted update. In javascript the call will look something like:

    const documentUpdateInstructions = {
      index: "index-name",
      id: "document-id",
      body: {
        script: {
          lang: "painless",
          source: `ctx._source.myNestedObject.removeIf(object -> object.username == params.username);`,
          params: {
            username: "my_username"
          },
        },
      },
    };
    await client.update(documentUpdateInstructions);

This takes a document in the form of

document._source = {
  ...
  "myNestedObject": [
    {
      "username": "my_username",
      ...
    },
    {
      "username": "not_my_username",
      ...
    }
  ]
}

and deletes the object inside myNestedObject who's username matches the username provided (in this case my_username). The resulting document will be:

document._source = {
  ...
  "myNestedObject": [
    {
      "username": "not_my_username",
      ...
    }
  ]
}

Collectives™ on Stack Overflow

Delete / add nested objects in Elastic search

3 Answers 3

Comments

6 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

6 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related