python - Merging Pandas DataFrames on categorical series -

-

i'm trying understand if pandas supports merging dataframes on columns of categorical data (i.e. dtype="category").

i of data work in r, trying more work in python/pandas. in r, merging on factors (analogous categorical dtype) induces type coercion, typically character. allows 1 data frame have by-variable (join column) specified factor (categorical) , other have by-variable string. pandas perform similar coercion of categorical data string prior merging/joining? should expect merging on categoricals robust? can find documentation on (automatic) type coercion in pandas?

simple example:

+++ error test categorical vector equality against non-categorical/non-scalar vector:

in [52]: import pandas pd  = pd.series(['a','b','c'],dtype="category") b = pd.series(['a','b','c'],dtype="object") c = pd.series(['a','b','cc'],dtype="object")  in [54]:  a==b  --------------------------------------------------------------------------- typeerror                                 traceback (most recent call last) ... typeerror: cannot compare categorical op <built-in function eq> type <class         'numpy.ndarray'>. if want compare values, use 'series <op> np.asarray(cat)'.

+++ merging dataframe on columns of different type--one categorical, 1 string--does not throw error (at least in simple case). type of coercion must occur: 

in [59]: = pd.dataframe({'a':a,'b':[1,2,3]}) b = pd.dataframe({'a':b,'c':[4,5,6]}) print(a.merge(b,on='a'))     b  c 0   1  4 1  b  2  5 2  c  3  6

so in short, in 0.15.1 merging behavior changed (fixed really) allow merging of categoricals had same categories. further if object array merged in allowed, resulting character of returned merge object (iirc). don't recall if try infer categorical or not.

i created issue here discussion on this.

the equality shown above, e.g. not allowing comparisons of categoricals vs object dtypes done first, while merging behavior expanded allow merging of like-categoricals , objects dtypes (assuming merged categoricals share same categories).

so think allowing equality work api not catching up. address in 0.16.0, pls provide comments on issue.

pr here

this in upcoming 0.15.2 release (slated week of december 7, 2014)


Comments

Post a Comment

Popular posts from this blog

matlab - "Contour not rendered for non-finite ZData" -

-
i'm trying plot frequency characteristic equation using ezplot, matlab gives following warning, "contour not rendered non-finite zdata". have used command plot frequency equations warning , plot display empty , not change axis range well. can please help. appreciated. here's code i'm using. % transfer matrix case-i, thin rotor clear all; clc; ei = 1626; l = 0.15; m = 0.44108; = 2.178*10^-4; i_p = 2.205*10^-5; itr = 0.24; i_pr = 0.479; syms p n; f = [1 l*1i l^2/(2*ei)*1i l^3/(6*ei); 0 1 l/ei -l^2/(2*ei)*1i; 0 0 1 -l*1i; 0 0 0 l]; p = [ 1 0 0 0; 0 1 0 0; 0 -it*p^2+i_p*n*p 1 0; -m*p^2 0 0 1]; p_r = [1 0 0 0; 0 1 0 0; 0 -itr*p^2+i_pr*n*p 1 0; -m*p^2 0 0 1]; = f*p*f*p*f*p*f; b = p_r*f*p*f*p*f; r = a(1,2)/a(1,4); a12_p = 0; a22_p = a(2,2)-r*a(2,4); a32_p = a(3,2)-r*a(3,4); a42_p = a(4,2)-r*a(4,4); ap(2,2) = a22_p; ap(3,2) = a32_p; ap(4,2) = a42_p; ap(4,4) = 1; c = b*ap; m = [c(3,2) c(3,4); c(4,2) c(4,...
Read more

delphi - Indy UDP Read Contents of Adata -

-
i using indy udp server read string of data. however, not know how work adata parameter. can around converting binary using bytetobin function below , convert hex using bintohex1 . feel stupid , works slow. know how can directly results without 2 conversions? thanks!! here codes: procedure tform1.idudpserver1udpread(athread: tidudplistenerthread; const adata: tidbytes; abinding: tidsockethandle); var buf: tidbytes; buffer: string; data_received: string; begin if bytestostring(adata) <> 'hello' begin buffer := hextostring('00'); setlength(buf, length(buffer)); copytidstring(buffer, buf, 0); abinding.sendto(abinding.peerip, abinding.peerport, buf); data_received := bintohex1(bytetobin(adata[1]) + bytetobin(adata[2]) + bytetobin(adata[3]) + bytetobin(adata[4]) + bytetobin(adata[5]) + bytetobin(adata[6]) + bytetobin(adata[7]) + bytetobin(adata[8]) + bytetobin(adata[9]) + bytetobin(adata[10]) + bytetobin(adata[11...
Read more

javascript - Any ideas when Firefox is likely to implement lengthAdjust and textLength? -

-
any ideas when firefox implement lengthadjust andtextlength?..... working svg application. lengthadjust , textlength working fine in chrome not working in fire fox . <svg viewbox = "0 0 500 300" version = "1.1"> <defs> <path id="s3" d="m 10,90 q 100,15 200,70" /> </defs> <g> <text font-size = "20"> <textpath xlink:href="#s3" textlength="205"> short text</textpath> </text> <use xlink:href="#s3" fill="none" stroke="black" stroke-width="1"/> </g> </svg> this implemented in firefox text elements (not textpath or tspan) in case that's ok though can move attribute text element. <svg viewbox = "0 0 500 300" version = "1.1"> <defs> ...
Read more