ES索引修改mappings与重建reindex详解之修改字段类型

文章目录

- 概要
- 一、创建索引
- - - 1.1、获取mappings
- 二、新增字段修改mappings
- 三、修改OR删除mappings已有字段
- - - 3.1、创建新索引，将要改字段加进去
    - 3.2、同步数据
    - 3.3、删除原索引并对新索引重命名
    - 3.4、同步数据的技巧
- 四、参考文献

概要

elasticsearch一直在使用，这里总结一下mappings的修改方法，分为两种情况：

增加新的字段，这种很简单；
修改已有的字段类型，这种就比较麻烦了，需要reindex,对索引进行迁移重建。

一、创建索引

curl -XPUT 'http://127.0.0.1:9200/test?pretty' -H  'Content-Type: application/json' -d '{"settings":{},"mappings":{}}'

{"settings": {"index": {"number_of_shards": 2,"number_of_replicas": 3,"analysis": {"analyzer": {"char_analyzer": {"filter": ["lowercase"],"type": "custom","tokenizer": "char_split"}},"tokenizer": {"char_split": {"token_chars": ["letter", "digit", "punctuation", "symbol"],"min_gram": "1","type": "nGram","max_gram": "1"}}}}},"mappings": {"doc": {"properties": {"id": {"type": "long"},"pd_name": {"type": "text","fields": {"keyword": {"type": "keyword","ignore_above": 256}},"analyzer": "char_analyzer"},"pd_uname": {"type": "text","fields": {"keyword": {"type": "keyword","ignore_above": 256}}},"product_id": {"type": "long"}}}}
}

1.1、获取mappings

curl -XGET 'http://127.0.0.1:9200/test/_mappings?pretty'
{"test" : {"mappings" : {"doc" : {"properties" : {"id" : {"type" : "long"},"pd_name" : {"type" : "text","fields" : {"keyword" : {"type" : "keyword","ignore_above" : 256}},"analyzer" : "char_analyzer"},"pd_uname" : {"type" : "text","fields" : {"keyword" : {"type" : "keyword","ignore_above" : 256}}},"product_id" : {"type" : "long"}}}}}
}

二、新增字段修改mappings

增加一个 new_stocks 字段，如下：

curl -XPUT 'http://127.0.0.1:9200/test/doc/_mapping?pretty' -H  'Content-Type: application/json' -d '{"properties":{"new_stocks":{"type":"nested","properties":{"value":{"type":"long"},"conversion":{"type":"long"}}}}}'
{"acknowledged" : true
}

再查一下：

curl -XGET 'http://127.0.0.1:9200/test/_mappings?pretty'                                                                                                          {"test" : {"mappings" : {"doc" : {"properties" : {"id" : {"type" : "long"},"new_stocks" : {"type" : "nested","properties" : {"conversion" : {"type" : "long"},"value" : {"type" : "long"}}},"pd_name" : {"type" : "text","fields" : {"keyword" : {"type" : "keyword","ignore_above" : 256}},"analyzer" : "char_analyzer"},"pd_uname" : {"type" : "text","fields" : {"keyword" : {"type" : "keyword","ignore_above" : 256}}},"product_id" : {"type" : "long"}}}}}
}

可以看到new_stocks字段已经加上去了。

三、修改OR删除mappings已有字段

如果想把product_id字段类型由long改成text，并删除id字段呢？此时用第二章节的方案就不行了，需要借用reindex命令重做索引。

3.1、创建新索引，将要改字段加进去

curl -XPUT 'http://127.0.0.1:9200/new_test?pretty' -H  'Content-Type: application/json' -d '{"settings":{},"mappings":{}}'

{"settings": {"index": {"number_of_shards": 2,"number_of_replicas": 3,"analysis": {"analyzer": {"char_analyzer": {"filter": ["lowercase"],"type": "custom","tokenizer": "char_split"}},"tokenizer": {"char_split": {"token_chars": ["letter", "digit", "punctuation", "symbol"],"min_gram": "1","type": "nGram","max_gram": "1"}}}}},"mappings": {"doc": {"properties": {"id": {"type": "long"},"pd_name": {"type": "text","fields": {"keyword": {"type": "keyword","ignore_above": 256}},"analyzer": "char_analyzer"},"pd_uname": {"type": "text","fields": {"keyword": {"type": "keyword","ignore_above": 256}}},"product_id": {"type": "text","fields": {"keyword": {"type": "keyword","ignore_above": 256}}}}}}
}

3.2、同步数据

curl -XPOST 'http://127.0.0.1:9200/_reindex?pretty' -H  'Content-Type: application/json' -d '{"source":{"index":"test"},"dest":{"index":"new_test"}}'

3.3、删除原索引并对新索引重命名

#删除
curl -XDELETE 'http://127.0.0.1:9200/test?pretty'
#设置别名
curl -XPOST 'http://127.0.0.1:9200/_aliases?pretty' -H  'Content-Type: application/json' -d '{"actions":[{"add":{"index":"new_test","alias":"test"}}]}'

3.4、同步数据的技巧

同步部分字段

#从源头过滤, _source 参数，只同步 id，pd_name两个字段
curl -XPOST 'http://127.0.0.1:9200/_reindex?pretty' -H  'Content-Type: application/json' -d '{"source":{"index":"test","_source":["id","pd_name"]},"dest":{"index":"new_test"}}'
#从目的地移除 脚本控制，移除id字段
curl -XPOST 'http://127.0.0.1:9200/_reindex?pretty' -H  'Content-Type: application/json' -d '{"source":{"index":"test"},"dest":{"index":"new_test"},"script":{"lang":"groovy","inline":"ctx._source.remove(\"id\")"}}'

提高同步速率
调整参数size大小，默认1000每批

curl -XPOST 'http://127.0.0.1:9200/_reindex?pretty' -H  'Content-Type: application/json' -d '{"source":{"index":"test","size":5000},"dest":{"index":"new_test"}}'