There are more than 600 pieces of data in my log, but only more than 300 pieces are written to elasticsearch.
Does anyone know the reason for this?
This is my configuration
input {
file {
path => ["/usr/local/20170730.log"]
type => "log_test_events"
tags => ["log_tes_events"]
start_position => "beginning"
sincedb_path => "/data/logstash/sincedb/test.sincedb"
codec => "json"
close_older => "86400"
#1 day
ignore_older => "86400"
}
beats{port => 5044}
}
filter {
urldecode {
all_fields => true
}
}
output{
elasticsearch {
hosts => "localhost:9200"
index => "logstash_%{event_date}"
}
stdout { codec => json }
}
因为读日志的时候es的模版根据数据的格式自动创建数据类型 比如字段a的值为int和字符串 他创建的索引第一个读取到是数字就是int类型 索引
修改配置 做映射
output {
stat_day.json模版格式
"order" : 1,
"template" : "test1",
"mappings" : {
}
}