nginx - Logstash Grokked Data Does Not Reach ElasticSearch -

-

i have simple logstash 1.4.1 setup , data seems correct in stdout, in es/kibana "message" field not overwritten.

logstash.conf

input {   file {     path => "/var/log/nginx/access.log"     type => "nginx_access"   } }  filter {   if [type] == "nginx_access" {     grok {       match => [ "message", "%{nginxaccess}" ]       overwrite => [ "message" ]     }   } }  output {   elasticsearch {     protocol => "http"   }   stdout { codec => rubydebug } }

nginx pattern

ngusername [a-za-z\.\@\-\+_%]+ nguser %{ngusername} nginxaccess %{iporhost:clientip} %{nguser:ident} %{nguser:auth} \[%{httpdate:timestamp}\] "%{word:verb} %{uripathparam:request} http/%{number:httpversion}" %{number:response} (?:%{number:bytes}|-) (?:"(?:%{uri:referrer}|-)"|%{qs:referrer}) %{qs:agent}

stdout

{         "message" => "127.0.0.1 - - [27/nov/2014:13:43:53 +0100] \"get /favicon.ico http/1.1\" 502 574 \"-\" \"mozilla/5.0 (x11; linux x86_64) applewebkit/537.36 (khtml, gecko) chrome/38.0.2125.122 safari/537.36\"",        "@version" => "1",      "@timestamp" => "2014-11-27t12:43:54.230z",            "type" => "nginx_access",            "host" => "laptop",            "path" => "/var/log/nginx/access.log",        "clientip" => "127.0.0.1",           "ident" => "-",            "auth" => "-",       "timestamp" => "27/nov/2014:13:43:53 +0100",            "verb" => "get",         "request" => "/favicon.ico",     "httpversion" => "1.1",        "response" => "502",           "bytes" => "574",           "agent" => "\"mozilla/5.0 (x11; linux x86_64) applewebkit/537.36 (khtml, gecko) chrome/38.0.2125.122 safari/537.36\"" }

what doing wrong? how can debug?

edit

get api showing fields in _source, not indexed. do?

{      "_index":"logstash-2014.11.27",    "_type":"nginx_access4",    "_id":"aunxr34z5dzktb-2teug",    "_version":1,    "found":true,    "_source":{         "message":"127.0.0.1 - - [27/nov/2014:13:43:53 +0100] \"get / http/1.1\" 502 574 \"-\" \"mozilla/5.0 (x11; linux x86_64) applewebkit/537.36 (khtml, gecko) chrome/38.0.2125.122 safari/537.36\"",       "@version":"1",       "@timestamp":"2014-11-27t12:43:54.230z",       "type":"nginx_access",       "host":"laptop",       "path":"/var/log/nginx/access.log",       "clientip":"127.0.0.1",       "ident":"-",       "auth":"-",       "timestamp":"27/nov/2014:13:43:53 +0100",       "verb":"get",       "request":"/",       "httpversion":"1.1",       "response":"502",       "bytes":"574",       "agent":"\"mozilla/5.0 (x11; linux x86_64) applewebkit/537.36 (khtml, gecko) chrome/38.0.2125.122 safari/537.36\""    } }

reloading index pattern's field list helped. created 1 before logging data.


Comments

Post a Comment

Popular posts from this blog

javascript - Any ideas when Firefox is likely to implement lengthAdjust and textLength? -

-
any ideas when firefox implement lengthadjust andtextlength?..... working svg application. lengthadjust , textlength working fine in chrome not working in fire fox . <svg viewbox = "0 0 500 300" version = "1.1"> <defs> <path id="s3" d="m 10,90 q 100,15 200,70" /> </defs> <g> <text font-size = "20"> <textpath xlink:href="#s3" textlength="205"> short text</textpath> </text> <use xlink:href="#s3" fill="none" stroke="black" stroke-width="1"/> </g> </svg> this implemented in firefox text elements (not textpath or tspan) in case that's ok though can move attribute text element. <svg viewbox = "0 0 500 300" version = "1.1"> <defs>
Read more

matlab - "Contour not rendered for non-finite ZData" -

-
i'm trying plot frequency characteristic equation using ezplot, matlab gives following warning, "contour not rendered non-finite zdata". have used command plot frequency equations warning , plot display empty , not change axis range well. can please help. appreciated. here's code i'm using. % transfer matrix case-i, thin rotor clear all; clc; ei = 1626; l = 0.15; m = 0.44108; = 2.178*10^-4; i_p = 2.205*10^-5; itr = 0.24; i_pr = 0.479; syms p n; f = [1 l*1i l^2/(2*ei)*1i l^3/(6*ei); 0 1 l/ei -l^2/(2*ei)*1i; 0 0 1 -l*1i; 0 0 0 l]; p = [ 1 0 0 0; 0 1 0 0; 0 -it*p^2+i_p*n*p 1 0; -m*p^2 0 0 1]; p_r = [1 0 0 0; 0 1 0 0; 0 -itr*p^2+i_pr*n*p 1 0; -m*p^2 0 0 1]; = f*p*f*p*f*p*f; b = p_r*f*p*f*p*f; r = a(1,2)/a(1,4); a12_p = 0; a22_p = a(2,2)-r*a(2,4); a32_p = a(3,2)-r*a(3,4); a42_p = a(4,2)-r*a(4,4); ap(2,2) = a22_p; ap(3,2) = a32_p; ap(4,2) = a42_p; ap(4,4) = 1; c = b*ap; m = [c(3,2) c(3,4); c(4,2) c(4,
Read more

delphi - Indy UDP Read Contents of Adata -

-
i using indy udp server read string of data. however, not know how work adata parameter. can around converting binary using bytetobin function below , convert hex using bintohex1 . feel stupid , works slow. know how can directly results without 2 conversions? thanks!! here codes: procedure tform1.idudpserver1udpread(athread: tidudplistenerthread; const adata: tidbytes; abinding: tidsockethandle); var buf: tidbytes; buffer: string; data_received: string; begin if bytestostring(adata) <> 'hello' begin buffer := hextostring('00'); setlength(buf, length(buffer)); copytidstring(buffer, buf, 0); abinding.sendto(abinding.peerip, abinding.peerport, buf); data_received := bintohex1(bytetobin(adata[1]) + bytetobin(adata[2]) + bytetobin(adata[3]) + bytetobin(adata[4]) + bytetobin(adata[5]) + bytetobin(adata[6]) + bytetobin(adata[7]) + bytetobin(adata[8]) + bytetobin(adata[9]) + bytetobin(adata[10]) + bytetobin(adata[11
Read more