Regex in config for dynamic columns in logstash -

-

i have log file of have pasted 2 rows below:

nov 26 14:20:32 172.16.0.1 date=2014-11-26 time=14:18:37 devname=xxxxcccfffff devid=xxxccvvgffdd logid=3454363464 type=traffic subtype=forward level=notice vd=root srcip=172.16.1.251 srcport=62032 srcintf="combo_lan" dstip=x.x.x.x dstport=x dstintf="wan2" sessionid=16172588 status=close user="x.x" group="open group" policyid=2 dstcountry="united states" srccountry="reserved" trandisp=snat transip=x.x.x.x transport=x service=http proto=6 applist="block_applications" duration=11 sentbyte=2377 rcvdbyte=784 sentpkt=6 rcvdpkt=7 identidx=5 utmaction=passthrough utmevent=webfilter utmsubtype=ftgd-cat urlcnt=1 hostname="tacoda.at.atwola.com" catdesc="advertising"

nov 26 14:20:32 172.16.0.1 date=2014-11-26 time=14:18:37 devname=xxxxcccfffff devid=xxxccvvgffdd logid=3454363464 type=utm subtype=webfilter eventtype=ftgd_allow level=notice vd="root" policyid=2 identidx=5 sessionid=15536743 user="x.x" srcip=x.x.x.x srcport=x srcintf="combo_lan" dstip=x.x.x.x dstport=80 dstintf="wan2" service="http" hostname="streaming.sbismart.com" profiletype="webfilter_profile" profile="open group_policy" status="passthrough" reqtype="direct" url="/diffusion/" sentbyte=984 rcvdbyte=202 msg="url belongs allowed category in policy" method=domain class=0 cat=18 catdesc="brokerage , trading"

my question can parse data if number of columns , order fixed.

but, how parse dynamic columns in config file don't _grokparsefailure?

ruby plugin can you.

here configuration:

input {     stdin{     } }  filter {     ruby {         code => '             msg = event["message"]             msgindex = msg.index("date=")             msginsert = msg[msgindex..-1]             msgmap = msginsert.scan(/(\w+)=("(.*?)"|([^ ]+))/).map { |(first, second)| [first, second] }             x in msgmap                 key = x[0]                 value = x[1]                 event[key] = value             end         '     } }  output {     stdout{         codec => rubydebug     } }
  1. first, key=value pair index start value date=
  2. then map key,value string array.
  3. use for loop insert value.

i have try logs , can create correspond field value. hope can you


Comments

Post a Comment

Popular posts from this blog

matlab - "Contour not rendered for non-finite ZData" -

-
i'm trying plot frequency characteristic equation using ezplot, matlab gives following warning, "contour not rendered non-finite zdata". have used command plot frequency equations warning , plot display empty , not change axis range well. can please help. appreciated. here's code i'm using. % transfer matrix case-i, thin rotor clear all; clc; ei = 1626; l = 0.15; m = 0.44108; = 2.178*10^-4; i_p = 2.205*10^-5; itr = 0.24; i_pr = 0.479; syms p n; f = [1 l*1i l^2/(2*ei)*1i l^3/(6*ei); 0 1 l/ei -l^2/(2*ei)*1i; 0 0 1 -l*1i; 0 0 0 l]; p = [ 1 0 0 0; 0 1 0 0; 0 -it*p^2+i_p*n*p 1 0; -m*p^2 0 0 1]; p_r = [1 0 0 0; 0 1 0 0; 0 -itr*p^2+i_pr*n*p 1 0; -m*p^2 0 0 1]; = f*p*f*p*f*p*f; b = p_r*f*p*f*p*f; r = a(1,2)/a(1,4); a12_p = 0; a22_p = a(2,2)-r*a(2,4); a32_p = a(3,2)-r*a(3,4); a42_p = a(4,2)-r*a(4,4); ap(2,2) = a22_p; ap(3,2) = a32_p; ap(4,2) = a42_p; ap(4,4) = 1; c = b*ap; m = [c(3,2) c(3,4); c(4,2) c(4,...
Read more

delphi - Indy UDP Read Contents of Adata -

-
i using indy udp server read string of data. however, not know how work adata parameter. can around converting binary using bytetobin function below , convert hex using bintohex1 . feel stupid , works slow. know how can directly results without 2 conversions? thanks!! here codes: procedure tform1.idudpserver1udpread(athread: tidudplistenerthread; const adata: tidbytes; abinding: tidsockethandle); var buf: tidbytes; buffer: string; data_received: string; begin if bytestostring(adata) <> 'hello' begin buffer := hextostring('00'); setlength(buf, length(buffer)); copytidstring(buffer, buf, 0); abinding.sendto(abinding.peerip, abinding.peerport, buf); data_received := bintohex1(bytetobin(adata[1]) + bytetobin(adata[2]) + bytetobin(adata[3]) + bytetobin(adata[4]) + bytetobin(adata[5]) + bytetobin(adata[6]) + bytetobin(adata[7]) + bytetobin(adata[8]) + bytetobin(adata[9]) + bytetobin(adata[10]) + bytetobin(adata[11...
Read more

qt - How to embed QML toolbar and menubar into QMainWindow -

-
i using qwt library plot data. seems not possible embed qwidget qml quick 2 . so, decided create qmainwindow main window , create toolbar , menubar using quick controls . how should embed qml toolbar , menubar qmainwindow ? you should create qml applicationwindow qml menubar , toolbar main.qml applicationwindow { visible: false menubar: menubar { menu { title: "edit" menuitem { text: "cut" } } } toolbar: toolbar { row { anchors.fill: parent toolbutton { iconsource: "1.png" } } } } main.cpp qapplication app(argc, argv); qqmlapplicationengine engine; engine.load(qurl(qstringliteral("qrc:///main.qml"))); then pointer applicationwindow qwindow *qmlwindow = qobject_cast<qwindow*>(engine.rootobjects().at(0)); create window container, using qwidget::createwindowco...
Read more