POC: Elasticsearch pod data replication¶
Imported from Confluence
Content may be outdated. Verify before following any procedures. View original | Last updated: May 2024
Here we will describe how eck-operator based eck-elasticsearch ensure data replication between pods. We tested the whole process in separate test namespace ect-testin dev cluster where eck-operator is already installed earlier.
We deployed eck-elasticsearch with single replica at first and create index, mapping and inserted data. We will describe the process in different phase to clarify pod replication.
First phase: The initial setup only have 1 pod to serve the functionality.
Now we will port-forward service and create index,mapping and ingest data here!
Now we will create index, map and insert data to index
curl --location --insecure --request PUT 'https://user:password@localhost:9200/developers' --header 'Content-Type: application/json'
Output:
{"acknowledged":true,"shards_acknowledged":true,"index":"developers"}
curl --location --insecure --request PUT 'https://user:password@localhost:9200/developers/_mapping' --header 'Content-Type: application/json' \
--data '{
"properties": {
"date_of_birth": {
"type": "date",
"format": "dd/MM/yyyy"
}
}
}'
Output:
{"acknowledged":true}
curl --location --insecure 'https://user:password@localhost:9200/_bulk' \
--header 'Content-Type: application/json' \
--data-raw '{ "index" : { "_index" : "developers", "_id" : "1" } }
{"id":1,"name":"Cristiano Ronaldo","email":"cronaldo@gmail.com","gender":"male","date_of_birth":"12/09/1989","club":"Real Madrid","position":"LW","experience":20,"country":"Portugal","phrase":"Both-footed generational talent","salary":15000}
{ "index" : { "_index" : "developers", "_id" : "2" } }
{"id":2,"name":"Lionel Messi","email":"lmessi@gmail.com","gender":"male","date_of_birth":"13/09/1990","club":"FC Barcelona","position":"F9","experience":20,"country":"Argentina","phrase":"Godly footballing heritage","salary":15000}
'
Output:
{"errors":false,"took":181,"items":[{"index":{"_index":"developers","_id":"1","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":2,"status":201}},{"index":{"_index":"developers","_id":"2","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":1,"_primary_term":2,"status":201}}]}
Check data by search api:
curl --location --insecure 'https://user:password@localhost:9200/developers/_search?pretty=true' \
--header 'Content-Type: application/json' \
--data '{
"query": {
"match": {
"phrase": {
"query" : "generational"
}
}
}
}'
Output:
{
"took" : 75,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 0.6548753,
"hits" : [
{
"_index" : "developers",
"_id" : "1",
"_score" : 0.6548753,
"_source" : {
"id" : 1,
"name" : "Cristiano Ronaldo",
"email" : "cronaldo@gmail.com",
"gender" : "male",
"date_of_birth" : "12/09/1989",
"club" : "Real Madrid",
"position" : "LW",
"experience" : 20,
"country" : "Portugal",
"phrase" : "Both-footed generational talent",
"salary" : 15000
}
}
]
}
}
Second Phase: Now we will scale up replica to 2 and insert data to one pod and query the data to both pod, which will gurrantee that data is replicated one pod to another pod.
Now we will insert data to pod elasticsearch-test-eck-elasticsearch-es-default-1and check the new record in both pod.
curl --location --insecure 'https://user:password@localhost:9200/_bulk' \
--header 'Content-Type: application/json' \
--data-raw '{ "index" : { "_index" : "developers", "_id" : "3" } }
{"id":3,"name":"Neymar JR","email":"jrneymar@gmail.com","gender":"male","date_of_birth":"12/09/1989","club":"Real Madrid","position":"LW","experience":20,"country":"Portugal","phrase":"Both-footed generational talent","salary":15000}
'
Output:
{"errors":false,"took":353,"items":[{"index":{"_index":"developers","_id":"3","_version":1,"result":"created","_shards":{"total":2,"successful":2,"failed":0},"_seq_no":2,"_primary_term":3,"status":201}}]}
Now we search the new record in pod elasticsearch-test-eck-elasticsearch-es-default-1
curl --location --insecure 'https://user:password@localhost:9200/developers/_search?pretty=true' \
--header 'Content-Type: application/json' \
--data '{
"query": {
"match": {
"phrase": {
"query" : "generational"
}
}
}
}'
Output:
{
"took" : 400,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 0.45315093,
"hits" : [
{
"_index" : "developers",
"_id" : "1",
"_score" : 0.45315093,
"_source" : {
"id" : 1,
"name" : "Cristiano Ronaldo",
"email" : "cronaldo@gmail.com",
"gender" : "male",
"date_of_birth" : "12/09/1989",
"club" : "Real Madrid",
"position" : "LW",
"experience" : 20,
"country" : "Portugal",
"phrase" : "Both-footed generational talent",
"salary" : 15000
}
},
{
"_index" : "developers",
"_id" : "3",
"_score" : 0.45315093,
"_source" : {
"id" : 3,
"name" : "Neymar JR",
"email" : "jrneymar@gmail.com",
"gender" : "male",
"date_of_birth" : "12/09/1989",
"club" : "Real Madrid",
"position" : "LW",
"experience" : 20,
"country" : "Portugal",
"phrase" : "Both-footed generational talent",
"salary" : 15000
}
}
]
}
}
########
Now we will port-forward pod elasticsearch-test-eck-elasticsearch-es-default-0 and check new record can be accessible from here as well
########
k port-forward pod/elasticsearch-test-eck-elasticsearch-es-default-0 9200:9200 -n eck-test
Forwarding from 127.0.0.1:9200 -> 9200
Forwarding from [::1]:9200 -> 9200
## Search query below
curl --location --insecure 'https://user:password@localhost:9200/developers/_search?pretty=true' \
--header 'Content-Type: application/json' \
--data '{
"query": {
"match": {
"phrase": {
"query" : "generational"
}
}
}
}'
Output:
{
"took" : 400,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 0.45315093,
"hits" : [
{
"_index" : "developers",
"_id" : "1",
"_score" : 0.45315093,
"_source" : {
"id" : 1,
"name" : "Cristiano Ronaldo",
"email" : "cronaldo@gmail.com",
"gender" : "male",
"date_of_birth" : "12/09/1989",
"club" : "Real Madrid",
"position" : "LW",
"experience" : 20,
"country" : "Portugal",
"phrase" : "Both-footed generational talent",
"salary" : 15000
}
},
{
"_index" : "developers",
"_id" : "3",
"_score" : 0.45315093,
"_source" : {
"id" : 3,
"name" : "Neymar JR",
"email" : "jrneymar@gmail.com",
"gender" : "male",
"date_of_birth" : "12/09/1989",
"club" : "Real Madrid",
"position" : "LW",
"experience" : 20,
"country" : "Portugal",
"phrase" : "Both-footed generational talent",
"salary" : 15000
}
}
]
}
}
Now if you increase the replica to 3 or more you will get same data from all pods as query above and also if you down scaled the replica, you will still get the the same response from all pods after downscaled.
Third phase: We will up scale replica to 3 to check data replicated to 3 pods or not.
k port-forward pod/elasticsearch-test-eck-elasticsearch-es-default-2 9200:9200 -n eck-test
Forwarding from 127.0.0.1:9200 -> 9200
Forwarding from [::1]:9200 -> 9200
Search in pod elasticsearch-test-eck-elasticsearch-es-default-2
curl --location --insecure 'https://user:password@localhost:9200/developers/_search?pretty=true' \
--header 'Content-Type: application/json' \
--data '{
"query": {
"match": {
"phrase": {
"query" : "generational"
}
}
}
}'
Output:
{
"took" : 345,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 0.45315093,
"hits" : [
{
"_index" : "developers",
"_id" : "1",
"_score" : 0.45315093,
"_source" : {
"id" : 1,
"name" : "Cristiano Ronaldo",
"email" : "cronaldo@gmail.com",
"gender" : "male",
"date_of_birth" : "12/09/1989",
"club" : "Real Madrid",
"position" : "LW",
"experience" : 20,
"country" : "Portugal",
"phrase" : "Both-footed generational talent",
"salary" : 15000
}
},
{
"_index" : "developers",
"_id" : "3",
"_score" : 0.45315093,
"_source" : {
"id" : 3,
"name" : "Neymar JR",
"email" : "jrneymar@gmail.com",
"gender" : "male",
"date_of_birth" : "12/09/1989",
"club" : "Real Madrid",
"position" : "LW",
"experience" : 20,
"country" : "Portugal",
"phrase" : "Both-footed generational talent",
"salary" : 15000
}
}
]
}
}
Now scaled down to single replica and check query result
curl --location --insecure 'https://user:password@localhost:9200/developers/_search?pretty=true' \
--header 'Content-Type: application/json' \
--data '{
"query": {
"match": {
"phrase": {
"query" : "generational"
}
}
}
}'
Output:
{
"took" : 10,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 0.45315093,
"hits" : [
{
"_index" : "developers",
"_id" : "1",
"_score" : 0.45315093,
"_source" : {
"id" : 1,
"name" : "Cristiano Ronaldo",
"email" : "cronaldo@gmail.com",
"gender" : "male",
"date_of_birth" : "12/09/1989",
"club" : "Real Madrid",
"position" : "LW",
"experience" : 20,
"country" : "Portugal",
"phrase" : "Both-footed generational talent",
"salary" : 15000
}
},
{
"_index" : "developers",
"_id" : "3",
"_score" : 0.45315093,
"_source" : {
"id" : 3,
"name" : "Neymar JR",
"email" : "jrneymar@gmail.com",
"gender" : "male",
"date_of_birth" : "12/09/1989",
"club" : "Real Madrid",
"position" : "LW",
"experience" : 20,
"country" : "Portugal",
"phrase" : "Both-footed generational talent",
"salary" : 15000
}
}
]
}
}
So we can see now in all way data is replicated per pod and is also consistent.