在数字化时代,日志分析已经成为IT运维和开发人员日常工作中不可或缺的一部分。ELK堆栈(Elasticsearch、Logstash、Kibana)因其强大的日志处理和分析能力,被广泛应用于各个领域。本文将全面解析ELK堆栈的实用命令,帮助您轻松掌握日志分析与搜索技巧。
一、Elasticsearch命令
1. 查询命令
1.1 基本查询
GET /_search
{
"query": {
"match_all": {}
}
}
1.2 精确查询
GET /_search
{
"query": {
"term": {
"field": "value"
}
}
}
1.3 范围查询
GET /_search
{
"query": {
"range": {
"field": {
"gte": "2021-01-01",
"lte": "2021-01-31"
}
}
}
}
1.4 高级查询
1.4.1 组合查询
GET /_search
{
"query": {
"bool": {
"must": [
{ "match": { "field": "value" } }
],
"filter": [
{ "range": { "field": { "gte": "2021-01-01", "lte": "2021-01-31" } } }
]
}
}
}
1.4.2 排序查询
GET /_search
{
"sort": [
{ "field": "field", "order": "asc" }
]
}
二、Logstash命令
2.1 输入插件
2.1.1 File输入
input {
file {
path => "/path/to/logfile.log"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
2.1.2 Stdin输入
input {
stdin { }
}
2.2 过滤插件
2.2.1 Grok过滤
filter {
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{IP:clientip} %{WORD:method} %{URIPATH:uri} HTTP/%{NUMBER:version} (%{WORD:status})" }
}
}
2.3 输出插件
2.3.1 Elasticsearch输出
output {
elasticsearch {
hosts => ["localhost:9200"]
index => "logstash-%{+YYYY.MM.dd}"
}
}
三、Kibana命令
3.1 创建索引
curl -X POST "localhost:9200/index" -H 'Content-Type: application/json' -d'
{
"settings": {
"number_of_shards": 1,
"number_of_replicas": 0
},
"mappings": {
"properties": {
"timestamp": {
"type": "date"
},
"clientip": {
"type": "ip"
},
"method": {
"type": "keyword"
},
"uri": {
"type": "text"
},
"version": {
"type": "keyword"
},
"status": {
"type": "keyword"
}
}
}
}'
3.2 创建仪表板
curl -X POST "localhost:5601/api/saved_objects/dashboard" -H 'Content-Type: application/json' -d'
{
"attributes": {
"title": "My Dashboard",
"description": "A dashboard for my log data",
" panelsJSON": [
{
"type": "timeseries",
"title": "Log Data",
"yaxis": {
"title": "Requests"
},
"data": [
{
"query": {
"match_all": {}
},
"size": 0
}
],
"gridPos": {
"h": 7,
"w": 12,
"x": 0,
"y": 0
}
}
],
"version": 1
}
}'
四、总结
通过本文的全面解析,相信您已经掌握了ELK堆栈的实用命令和日志分析与搜索技巧。在实际应用中,不断实践和总结,您将能够更好地发挥ELK堆栈在日志分析方面的强大功能。