es6 dsl与sql对比

【README】

1.本文总结了 dsl 与 sql的对比写法；

2.es采用 7.2.1 版本；

【1】创建es索引

1）新建一个数据库事务执行日志索引

put localhost:9200/txlog

{   "mappings"  :{      "properties":{"APPNAME":{"type":"keyword"}            , "ITFNAME":{"type":"keyword"}, "DBNAME":{"type":"keyword"}, "EXE_COST":{"type":"integer"}, "EXE_TIMES":{"type":"integer"}			, "CREATE_TIME":{"type":"date", "format":"yyyy-MM-dd HH:mm:ss.SSS||yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"} }}       
}

字段含义：

序号	字段名	描述
1	APPNAME	应用名
2	ITFNAME	接口名
3	DBNAME	数据库名
4	EXE_COST	执行耗时
5	EXE_TIMES	执行次数
6	CREATE_TIME	创建时间

字段类型为 keyword，即不分词，支持精确查找，不支持模糊查找；

补充：如要分词，使用 text，即支持模糊查找；

【2】插入10条数据

采用 bulk 批量插入api；如下：

post localhost:9200/txlog/_doc/_bulk

{"index":{}}
{"APPNAME":"trcd1", "ITFNAME":"TRLOGQRY11", "DBNAME":"TRDB11", "EXE_COST":"11", "EXE_TIMES":"2", "CREATE_TIME":"2022-04-13 20:09:00"}
{"index":{}}
{"APPNAME":"trcd1", "ITFNAME":"TRLOGQRY11", "DBNAME":"TRDB11", "EXE_COST":"12", "EXE_TIMES":"3", "CREATE_TIME":"2022-04-14 20:09:00"}
{"index":{}}
{"APPNAME":"trcd1", "ITFNAME":"TRLOGQRY11", "DBNAME":"TRDB11", "EXE_COST":"13", "EXE_TIMES":"4", "CREATE_TIME":"2022-04-15 20:09:00"}
{"index":{}}
{"APPNAME":"trcd1", "ITFNAME":"TRLOGQRY11", "DBNAME":"TRDB11", "EXE_COST":"14", "EXE_TIMES":"5", "CREATE_TIME":"2022-04-16 20:09:00"}
{"index":{}}
{"APPNAME":"trcd2", "ITFNAME":"TRLOGQRY21", "DBNAME":"TRDB21", "EXE_COST":"15", "EXE_TIMES":"6", "CREATE_TIME":"2022-04-17 20:09:00"}
{"index":{}}
{"APPNAME":"trcd2", "ITFNAME":"TRLOGQRY21", "DBNAME":"TRDB21", "EXE_COST":"16", "EXE_TIMES":"7", "CREATE_TIME":"2022-04-17 20:09:00"}
{"index":{}}
{"APPNAME":"trcd2", "ITFNAME":"TRLOGQRY22", "DBNAME":"TRDB22", "EXE_COST":"17", "EXE_TIMES":"8", "CREATE_TIME":"2022-04-19 20:09:00"}
{"index":{}}
{"APPNAME":"trcd2", "ITFNAME":"TRLOGQRY23", "DBNAME":"TRDB01", "EXE_COST":"18", "EXE_TIMES":"9", "CREATE_TIME":"2022-04-20 20:09:00"}
{"index":{}}
{"APPNAME":"trcd2", "ITFNAME":"TRLOGQRY23", "DBNAME":"TRDB01", "EXE_COST":"19", "EXE_TIMES":"12", "CREATE_TIME":"2022-04-21 20:09:00"}
{"index":{}}
{"APPNAME":"trcd2", "ITFNAME":"TRLOGQRY23", "DBNAME":"TRDB01", "EXE_COST":"20", "EXE_TIMES":"22", "CREATE_TIME":"2022-04-22 20:09:00"}

【3】dsl与sql对比

【3.0】普通查询

sql：

select col1, col2 from table1 where col = '' ... limit 10

dsl如下：

{"_source":["APPNAME", "ITFNAME", "DBNAME", "EXE_COST"],"query":{"bool":{"filter":{"bool":{"must":[{"range":{"CREATE_TIME":{"gte":"2022-04-13 00:00:00", "lte":"2022-04-19 00:00:00"}}}]}}}}, "size":10    
}

【3.1】范围查询

sql：

where  create_time > '' and create_time < ''

dsl如下：把 bool 封装在filter里面，不会计算分数，提高查询性能；

{"query":{"bool":{"filter":{"bool":{"must":[{"range":{"CREATE_TIME":{"gte":"2022-04-13 00:00:00", "lte":"2022-04-15 00:00:00"}}}]}}}}
}

【3.2】聚合查询

1）根据appname 分组求均值，求和；

sql如下：

select sum(exe_times) as sum_times, avg(exe_cost) as avg_cost 
from table 
where create_time > '#' and create_time < '#'
group by appname

dsl如下：（先分组，后求均值，求和）

{"query":{"bool":{"filter":{"bool":{"must":[{"range":{"CREATE_TIME":{"gte":"2022-04-13 00:00:00", "lte":"2022-04-19 00:00:00"}}}]}}}}, "size":0, "aggs":{"group_by_app":{"terms":{"field":"APPNAME"}, "aggs":{"sum_times":{"sum":{"field":"EXE_TIMES"}}, "avg_cost":{"avg":{"field":"EXE_COST"}}}}}
}

查询结果：

"aggregations": {"group_by_app": {"doc_count_error_upper_bound": 0,"sum_other_doc_count": 0,"buckets": [{"key": "trcd1","doc_count": 4,"avg_cost": {"value": 12.5},"sum_times": {"value": 14.0}},{"key": "trcd2","doc_count": 2,"avg_cost": {"value": 15.5},"sum_times": {"value": 13.0}}]}}

2）根据多个字段分组，如应用名，数据库名

sql：

select * from 
(select sum(exe_times) as sum_times, avg(exe_cost) as avg_cost from table where create_time > '#' and create_time < '#'group by appname,dbname
) a 
order by sum_times desc 
limit 3

dsl：使用 painless 脚本根据多个字段分组；

{"query":{"bool":{"filter":{"bool":{"must":[{"range":{"CREATE_TIME":{"gte":"2022-04-13 00:00:00", "lte":"2022-04-19 00:00:00"}}}]}}}}, "size":0, "aggs":{"group_by_app":{"terms":{"script":{"source":"doc['APPNAME'] + '#SPLIT#' + doc['DBNAME']", "lang":"painless"}, "size":"3", "min_doc_count":"1", "order":{"sum_times":"desc"}}, "aggs":{"sum_times":{"sum":{"field":"EXE_TIMES"}}, "avg_cost":{"avg":{"field":"EXE_COST"}}}}}
}

步骤：