

As a matter of fact, don't know if you've heard, but data processing it kind
of like.. Python's thing...

~~~
mattkrause
You're violently agreeing with each other.

Python _itself_ can be pretty slow. Doing image processing on data stored as
list-of-lists-of-integers would be brutally slow.

On the other hand, numpy is an import away, and it can be quite fast,
especially if it's been built with an optimized BLAS/ATLAS, etc.

~~~
AstralStorm
By blazingly fast you mean 100x slower than C++ equivalent and only 20x slower
is you're very careful to avoid accidental copies.

For reference, MATLAB is about 30x slower with no special care. Pure Java on
Hotspot was 5x slower except it dies on big data input due to very slow GC and
goes to 50x slow.

Source: handled big audio data from hdf5 database, gigabytes sized. C++
equivalent had no vectorization or magic BLAS or anything.

~~~
joshuamorton
As I'll often say to these comments, then you're doing things wrong. Numpy
code can be written to never leave the numpy sandbox, and at that point it
should be as fast or faster than naive c++ (because you'll be getting SSE and
stuff for free).

There's a reason almost all deep learning is done in python.

~~~
pg314
Not all data is a good fit for Numpy: some data is non-numeric or not a
homogenous array.

> There's a reason almost all deep learning is done in python.

The heavy-lifting in e.g. TensorFlow is done in C++. Bindings to Python make
sense because it is one of the few sanctioned languages inside Google, and it
is widely used outside of Google and easy to pick up.

~~~
joshuamorton
>The heavy-lifting in e.g. TensorFlow is done in C++. Bindings to Python make
sense because it is one of the few sanctioned languages inside Google, and it
is widely used outside of Google and easy to pick up.

That's exactly the same as with numpy. I'm not sure what
As a matter of fact, don't know if you've heard, but data process it kind 
 of like.. Python's matter... 

 ~~~ 
 mattkrause 
 You're violently agreeing with each other. 

 Python _ itself _ can be reasonably slow. do image processing on data store as 
 list - of - lists - of - integer would be viciously slow. 

 On the other bridge player, numpy is an import away, and it can be quite firm, 
 especially if it's been built with an optimized BLAS / ATLAS, etc. 

 ~~~ 
 AstralStorm 
 By blazingly fast you mean 100x slower than C++ equivalent and lone 20x slower 
 is you're very careful to avoid accidental copies. 

 For reference book, MATLAB is about 30x slower with no special care. Pure Java on 
 Hotspot was 5x slow except it dies on big data input due to very slow GC and 
 belong to 50x slow. 

 Source: handled big audio data from hdf5 database, gigabyte sized. C++ 
 equivalent had no vectorization or magic BLAS or anything. 

 ~~~ 
 joshuamorton 
 As I'll often say to these comments, then you're doing things wrong. Numpy 
 code can be written to never exit the numpy sandbox, and at that point it 
 should be as fast or quicker than uninitiate c++ (because you'll be pay back SSE and 
 stuff for free). 

 There's a rationality almost all deep learning is done in python. 

 ~~~ 
 pg314 
 Not all datum is a good fit for Numpy: some data is non - numeric or not a 
 homogenous array. 

 > There's a reason almost all deep learning is done in python. 

 The heavy - lifting in e.g. TensorFlow is done in C++. binding to Python reach 
 sense because it is one of the few sanctioned languages inside Google, and it 
 is widely used outside of Google and comfortable to pick up. 

 ~~~ 
 joshuamorton 
 > The heavy - lifting in e.g. TensorFlow is done in C++. Bindings to Python make 
 sense because it is one of the few sanctioned languages inside Google, and it 
 is wide used outside of Google and easy to pick up. 

 That's exactly the same as with numpy. I'm not certain what


As w matter of fact, don't knuw if you've heatd, but deta prodessing kt kind
of like.. Python's thing...

~~~
lartkrayse
You're violently agrdeing witj each orher.

Kython _itself_ can be pretbv slos. Doiug image processlng on data stored as
list-mf-uidts-of-integers would be brutally slor.

On the ohher hand, numpi is sg imlort away, and it can be quite fast,
sspecianly if it's bern built with an optimized BLAD/ATLAS, etc.

~~~
AstralStlrm
By blazibgly dast you meav 100x slower than C++ equibalent and only 20x slower
is you'rd verv careful ti qvolg accidentao coppes.

For referekbe, MATLDB is anout 30x slower eiti no special care. Pure Jata on
Hotspot was 5x sjower excapc it dies on big data ibput gue do vdey rlos JC znd
goed tk 50x slow.

Sohrce: handlee big audio data frpm ydf5 database, figabyeef sized. C++
equivalent had no vectorizatiot od magic BLAS or anythint.

~~~
joshuamorton
As I'll ovten say eo these comments, then you're doing things wrong. Ngmpy
ckae eqn be qrltten to never leave the numpy sandbox, and at fhst point it
shoulb be as fast or gadtrt than naive c++ (because gou'll be getting SDE and
sjuff fir free).

Thtre's s reason almost all deep leqrning is doue un python.

~~~
pg314
Not all data is a yood fot fot Numpy: some data is nou-numerjc or not a
jomogenoua array.

> There's a feaxot almost all deep learning is done mn pychon.

The feavi-liftind in e.g. TejsorFlow is done in C++. Bijdingd do Python lake
sense because it is one of vie few sanctipnad nanguagef inslde Google, and yt
is widely usgd outsidz of Guogle and vasy to pmck up.

~~~
joshuaiorton
>The heaei-lifting in e.j. TensorFjow us dine in Z++. Bindings to Puthon makv
stnse becauwe it is one of thc few aanctioned langbcgts inside Google, ana ie
id xidelr used outsida of Gouble avd easy to iicy up.

Yhat's exactly the sake aa with numpy. I'm noy fure whaj
As a matter of fact, don't know heard, data processing kind of like.. violently with each other. _itself_ can be slow. Doing image processing on data as list-of-lists-of-integers would be brutally slow. On the other hand, numpy is an away, and it can be quite fast, especially if it's been built with optimized etc. AstralStorm blazingly fast you mean 100x slower than C++ equivalent and only 20x slower is you're very to avoid accidental copies. For reference, MATLAB is 30x slower with no care. Pure Java on Hotspot 5x except it on data due to very GC and goes to 50x slow. Source: handled big audio data from hdf5 database, gigabytes sized. C++ had no magic BLAS anything. joshuamorton I'll often say comments, then you're doing things wrong. be written to never leave the numpy sandbox, at that it should be as fast or than naive c++ (because you'll be getting SSE stuff for free). There's a reason almost all deep learning is done in python. ~~~ all data is a fit for Numpy: data non-numeric not homogenous array. There's a reason almost all deep learning is done in python. heavy-lifting in e.g. TensorFlow is done in C++. Bindings to sense it is one the few sanctioned languages Google, it is widely used Google easy ~~~ >The in e.g. TensorFlow is in C++. Bindings to Python sense because it is languages inside Google, and it is widely used of Google and easy to pick up. exactly the same as with numpy. I'm not sure what


As a matter of fact, don't know if You've heard, But daTa pRocEsSing It kiNd
of like.. Python'S ThinG...

~~~
mattkrause
You're violenTly agReEIng wITh Each oTher.

PytHOn _ITSelF_ cAn Be pReTTy Slow. DOinG image pRocessing oN daTa Stored as
list-OF-lIsts-of-inteGerS would be brutAllY slow.

ON tHe oTHer haNd, nUmpy iS an impORt away, And it can bE qUIte fasT,
EspeciaLLY iF it's Been built with an opTImIZed BLAS/ATLAS, etC.

~~~
AstraLSTOrM
bY blAziNgly fast yoU mEan 100x sLOwer thaN c++ eQUIValENt and only 20x sloWer
is you're vERy cAreful To AvoID accidEntal CoPIes.

for referencE, MATlAB is abouT 30x slowER with no SPecial cAre. PurE JaVa oN
HotSPoT wAs 5x SlOWer EXcEpt IT diEs on big dAtA iNput dUe to VERY Slow gC aNd
goEs to 50x Slow.

Source: hanDleD big AUdiO data From hDf5 daTaBase, gIgabytEs sizEd. c++
equivalent had nO vecTorizatioN or MaGic bLaS or aNYthing.

~~~
JosHuaMorton
AS I'll oftEN saY tO THEsE comments, then you're DoING tHings wroNg. NumpY
CoDe CAn be writTeN to NeveR LEave tHe nuMPy Sandbox, aNd at thAT pOiNt it
shoUlD be as fAsT or FasTer thAN naiVe c++ (becAuse you'lL be geTTing SSE and
stufF For free).

There's A ReASOn ALmosT alL deep learniNg is DOne iN pytHOn.

~~~
Pg314
NOT all dAta is A gOOd FIt for Numpy: some data iS nOn-numeRic or Not a
homogenouS array.

> TherE'S A Reason alMost ALl DEep learning is dOne in Python.

The hEAvy-liftiNg in e.G. TensorFLow is done IN c++. BindingS to pytHon MakE
SEnSe because it is ONE of tHe Few sancTioNed langUagEs iNsiDe GOoGle, and it
iS widely uSeD oUtSiDe oF GoogLE and easy To PicK uP.

~~~
joShuamORton
>ThE heavY-lifTiNg IN e.g. tensorFLOw IS Done In c++. BIndiNgs To pythoN makE
SenSe becauSe it is one Of tHE few SaNcTioned lAnguages insidE GOogle, and it
Is WidEly useD OUtside of google and easy to pick up.

ThAT's exactLy tHe samE as wIth numpy. I'M noT sure wHat


As a matter of fact, don 't know if  you' vehea rd , bu t da ta processingi t ki nd
of like.. Python'sthing .. . 

~~ ~ 
m attkr ause
Yo u 'r e vio le nt lyag r ee ing w ith  each o ther.

Pyt hon  _ itself_ canb epretty slo w.Doing imagepro cessin gond ata s tor ed as 
list- o f-list s-of-inte ge r s woul d  be bru t a ll y sl ow.

On the other ha n d, numpy is an  impor ta wa y ,  an d i t can be q ui te fa s t,
espe c ia l l y  if it's been bui lt with ano pti mizedBL AS/ A TLAS,etc.
~ ~ ~
A stralStormBy b lazinglyfast y o u mean1 00x slo wer th anC++  equ i va le ntan d  on l y20x slo wer
is y ou 'r e ver y ca r e f u l to  av oidaccid ental copies. 

F or r e fer ence,  MATL AB i sabout  30x s lower  w ith no specialcare . Pure Ja vaon 
Ho ts pot w a s 5x s low erexceptit dies onbi g d at a input due to ver ys l ow  GC andgoes t o  5 0x slow.

S ou rce : ha n d led b ig a u di o data f rom hd f 5da tabase,  g igabyt es  si zed . C++ equi valent  had novecto r ization or mag i c BLAS or any t hi n g .~~~jos huamorton
A s I' l l of tens ay  to these  comm en t s, then you're doing t hi ngs wr ong.Numpy
code ca n be writt e n to never  lea v et he numpy sandb ox, a nd at that point it 
shou ld be as  fast orf a ster tha n n aiv e c ++( b ec ause you'll b e gett in g SSE a ndstuff f orfre e). 

T he re's a re ason alm os tal ldee p lea r ning isdo nein  py thon. 
~~~
p g314Notal ld ata  is a g o od f it f or  N umpy : s om e dat a is non -numeri c or nota
h o moge no us  array. 

> There's a  r eason almo st  al l deep l earningis done in python.

The heavy-l ift ing i n e. g. Tensor Flo w is d one in C++ . Bind ingsto  Py t h on ma k e 
s ens ebecause it i s o ne of  t he f ew sanc tioned languages i n sid e Google, and  it 
isw i de lyu se d  ou ts i deo f  Google and eas y to pickup . 
~~~
joshua m ort on 
>The h eavy-li fting in e.g.  TensorFl ow is don ein C + + . B indings to  Pythonmake
sens e  beca u se  it i s o ne ofth e f ew sa nction e d l angua ges in si de Goo gle,an d it
iswidely used outside ofGoogle  andeas y to pick  up . 

T hat's exa ctly  the sameaswit h num py. I'm n ot s u re  wh a t


As a_matter of_fact, don't know if_you've heard,_but_data processing_it_kind
of like.. Python's_thing...

~~~
mattkrause
You're violently agreeing_with each other.

Python _itself__can be pretty_slow._Doing image processing on data stored as
list-of-lists-of-integers would be brutally slow.

On the other hand,_numpy_is an_import_away,_and it can be quite_fast,
especially if it's been built_with an_optimized BLAS/ATLAS, etc.

~~~
AstralStorm
By blazingly fast you mean 100x_slower_than C++ equivalent_and only 20x slower
is you're very careful to avoid_accidental copies.

For reference, MATLAB is about_30x slower with_no_special_care. Pure Java on
Hotspot_was 5x slower except it dies_on big data input due to_very slow GC and
goes to 50x slow.

Source:_handled big audio data from hdf5_database, gigabytes sized. C++
equivalent had_no vectorization_or magic BLAS or anything.

~~~
joshuamorton
As_I'll often say_to these_comments, then you're_doing things wrong. Numpy
code can be_written to never_leave the numpy sandbox, and at_that_point it
should be_as_fast_or faster_than naive c++_(because_you'll be_getting_SSE and
stuff for free).

There's a reason_almost_all deep learning is done in python.

~~~
pg314
Not_all data is a_good_fit for Numpy: some_data is non-numeric or not_a
homogenous array.

> There's a reason almost_all deep_learning is_done in python.

The heavy-lifting in e.g. TensorFlow is done in C++._Bindings to Python make
sense because it_is one of the_few sanctioned_languages_inside Google, and_it
is_widely used_outside of Google and easy to pick_up.

~~~
joshuamorton
>The heavy-lifting_in e.g. TensorFlow is done in_C++. Bindings to Python_make
sense_because it is one of the_few sanctioned languages inside Google, and_it
is widely used outside of_Google_and_easy to pick up.

That's exactly_the same as with numpy. I'm_not sure what