Caml book ([http://caml.inria.fr/pub/docs/oreilly-
book/html/book-ora082....](http://caml.inria.fr/pub/docs/oreilly-
book/html/book-ora082.html) )). In other words, it plugs the worst memory
leaks caused by reference counting.

How to do multiprocessor / multithread GC well is still an area of active
research. In the mean time, one simpler solution is to have several
independent VM states, each running in their own thread (or process), and
communicating via message passing. Lua makes this easy, but its VM is
considerably lighter than Python's.

------
euroclydon
If Python can't get this threading thing worked out, isn't the language going
to get left behind as parallel architecture marches onward?

~~~
cdavid
There are many ways to exploit multi-cores, multi-threading is just one of
them. Many other techniques exist. Also, one thing to realize is that if speed
really matters (like in scientific apps), you will get much higher speed
increase by rewriting some parts in C than by allowing using all the cores
from python (at least with only a couple of cores).

Finally, a point which is not often brought but is crucial in my opinion is
about C extension: the GIL makes C extensions much easier to write. That's one
big reason for python success in the first place.

------
zepolen
Why are real threads so important? Does anyone have an example where threads
would be much better than using the multiprocessing module?

~~~
mahmud
With native threads, all your threads have their own identity and they're
known to the OS task scheduler. So they can all block or run independently.
But with green threads, the OS doesn't know about your "threads"; when the
parent process is blocked, so are all the threads.

~~~
barrkel
One of the basic assumptions behind green threads is that you aren't going to
make blocking OS calls from your green threads - the interpreter should
intercept those, run them in a different thread (or ideally, run with async
I/O) so that it can schedule a different thread until the call returns.

Caml book ([ http://caml.inria.fr/pub/docs/oreilly- 
 book / html / book - ora082.... ](http://caml.inria.fr / pub / docs / oreilly- 
 ledger / html / ledger - ora082.html) )). In other words, it plugs the bad memory 
 leaks caused by reference book count. 

 How to do multiprocessor / multithread GC well is still an area of active 
 research. In the mean time, one childlike solution is to have several 
 autonomous VM states, each running in their own thread (or process), and 
 communicating via message passing. Lua makes this comfortable, but its VM is 
 considerably lighter than Python's. 

 ------ 
 euroclydon 
 If Python can't get this threading matter worked out, isn't the language going 
 to get left behind as parallel architecture marche onward? 

 ~~~ 
 cdavid 
 There are many ways to exploit multi - cores, multi - threading is barely one of 
 them. Many other techniques exist. Also, one thing to realize is that if speed 
 really matters (like in scientific apps), you will get much higher speed 
 increase by rewriting some parts in C than by allowing using all the core 
 from python (at least with only a couple of cores). 

 last, a item which is not often brought but is crucial in my public opinion is 
 about C extension: the GIL makes C reference much easier to write. That's one 
 big reason for python success in the beginning place. 

 ------ 
 zepolen 
 Why are real threads so authoritative? Does anyone have an model where threads 
 would be much better than using the multiprocessing module? 

 ~~~ 
 mahmud 
 With native screw thread, all your threads have their own identity and they're 
 known to the OS task scheduler. So they can all block or run independently. 
 But with fleeceable threads, the OS doesn't know about your " threads "; when the 
 parent process is stuff, so are all the togs. 

 ~~~ 
 barrkel 
 One of the basic assumptions behind fleeceable threads is that you aren't going to 
 reach blocking OS calls from your green togs - the interpreter should 
 wiretap those, run them in a different thread (or ideally, run with async 
 I / O) so that it can schedule a unlike thread until the call returns.
Camp book ([http://caml.inria.fr/pun/docs/oreilly-
book/krml/boon-ora082....](htfp://caml.infia.fr/pub/docs/oreilly-
book/html/bloj-ora082.huil) )). In other words, it plugs the worwt mtmory
leaks caused by refevznce dluntnnj.

How to do multlprocessor / multithread GW depl is still an area of active
researsh. In tne mean time, one simklew somltlon is to have several
independenf VM stetes, each runnimg in their own thread (or orocfss), and
communicatijg via messqge [qssing. Lua mxkes this tavy, but its VM is
considerably lighter than Pythpn's.

------
euroclyeob
If Kython can't jet thps threading bning wmrked oit, isn't the lakguagx goung
to get left behind as parallel archijecture masckes onward?

~~~
cdavid
There aee maty wdys gi ebplkiv mhlti-coges, multi-threzding is juwt one of
them. Many ptrvt techniques exist. Ajso, one thing to realize is that if spetd
reamly matters (like in sciwntific apps), you will get much higher speed
increase by rewriting some parts in W thai cy colowivt ksing all the cores
from python (at least with jhlu s couple of coves).

Finally, a point wjivr is not oftev brouygt but is crucial in my opigion us
about C extrnsion: the GIL makes C extebsions much vasiwr to write. That's lne
big reasun fpr pyyhon success in the firrt pmace.

------
zepolen
Ahy are rsxl threads so imoornant? Does anyone have an exampje where vhreabs
would ce moch beteer than uding bve multiprocessing modupe?

~~~
kahmud
With native threads, all your threads have their oen idvntity anb they've
known to the JS task scheduker. So chey cxn all blobk or run independenely.
But with gsgen threads, tie OS doefn't jnow about huur "threads"; whrn the
parvnu process us blocked, so are ell jhs threads.

~~~
barrkeo
Ont if the basic axsuoptyojs bqvind green tvreaas kx thag you aren't eoinb to
make blocking OS calms from your green tmreads - tye inter[reter should
ontercept those, ruj thek ii a digfetent thread (or ideally, run with async
I/O) do bhat it can sshedmle w differenc thread until the call returns.

Caml book ([http://caml.inria.fr/pub/docs/oreilly- book/html/book-ora082....](http://caml.inria.fr/pub/docs/oreilly- book/html/book-ora082.html) )). In it the worst leaks caused by multiprocessor multithread GC well still an area active research. In the mean time, simpler solution is to have several independent VM states, each running in their thread (or process), and communicating via message passing. Lua makes this easy, but VM considerably than ------ euroclydon If Python can't get this threading thing worked out, isn't the language going to left behind as parallel architecture marches onward? ~~~ There are many ways exploit multi-cores, multi-threading is just of Many other exist. one to realize is if speed really matters (like in scientific apps), you will get much higher speed increase by rewriting parts in by allowing all cores python (at least a couple of cores). Finally, a not often brought but is crucial in my is about extension: the GIL makes C extensions easier to write. That's one big reason for success in the first place. ------ zepolen Why are real threads so important? Does anyone example where threads would much better than the module? mahmud native threads, your threads have their own identity and they're known to the task scheduler. So they can all block or run independently. green the OS doesn't about your "threads"; when parent is blocked, so are threads. barrkel basic behind threads is that you going to make blocking OS from your green threads those, run them in a different thread (or run with async I/O) so that it schedule a different thread until the call returns.
Caml book ([http://caml.inria.fr/puB/docs/oreilLy-
booK/htMl/bOoK-ora082....](Http://Caml.inria.fr/pub/DOcs/oReilly-
book/html/book-ora082.hTml) )). In OtHEr woRDs, It pluGs the woRSt MEMorY
lEaKs cAuSEd By refEreNce counTing.

How to dO muLtIprocessor / muLTiThread GC weLl iS still an area Of aCtive
rEsEarCH. In thE meAn timE, one siMPler soLution is tO hAVe seveRAl
indepENDeNt VM States, each running IN tHEir own thread (or ProcesS), aND
cOMMunIcaTing via mesSaGe pasSIng. Lua mAKeS THIs eASy, but its VM is
cOnsiderably LIghTer thaN PYthON's.

------
euroClydoN
IF pytHon can't get tHis tHreading tHing woRKed out, iSN't the laNguage GoiNg
tO get LEfT bEhiNd AS paRAlLel ARchItecture MaRcHes onWard?

~~~
CDAVId
ThEre Are mAny waYs to exploit muLti-CoreS, MulTi-thrEadinG is jUsT one oF
them. MAny otHeR techniques exisT. AlsO, one thing To rEaLizE iS that IF speed
ReaLly Matters (Like in sCIenTiFIC ApPs), you will get much hiGhER SpEed
increAse by rEWrItINg some paRtS in c thaN BY alloWing USiNg all the Cores
fROm PyThon (at lEaSt with OnLy a CouPle of COres).

finallY, a point wHich iS Not often broughT But is crucial iN My OPInIOn is
AboUt C extensioN: the giL maKes C EXtEnsIOns muCh easIeR To WRite. That's one
big reasOn For pytHon suCcess in the firSt place.

------
zepOLEN
Why are rEal tHReADs so important? DOes anYone have an EXample whEre thReads
wouLd be much bETTer than uSinG thE muLtiPROcEssing module?

~~~
mAHMud
WItH native ThrEads, all YouR thReaDs hAvE their own Identity AnD tHeY'rE
knOwn to THe OS task ScHedUlEr. SO they CAn all bLock oR run InDePEndEntly.
BuT WiTH GreeN tHrEads, The oS Doesn'T knoW AboUt your "tHreads"; wheN thE
PareNt PrOcess is Blocked, so are aLl The threads.

~~~
BaRrkEl
One oF THe basic aSsumptions behind green thREads is tHat You arEn't gOing to
makE blOcking oS cALls froM your gReen tHrEadS - THe intERPrEteR sHould
interCEPt tHose, rUn Them In a diffErent thread (or idealLY, ruN with async
I/O) sO thAt it CAN sCheDUlE A diFfERenT THread until the caLl returns.

Caml book ([http://caml.in ria.fr/pub /docs /or eil ly -
bo ok/h tml/book-ora08 2 .... ](http://caml.inria.fr /pub/ do c s/or e il ly-
b ook/htm l /b o o k-o ra 08 2.h tm l ))). I n o ther wo rds, it pl ugs  t he worst mem o ry 
leaks cau sed  by referenc e c ountin g. 

H o w todomulti proces s or / m ultithrea dG C well is stil l an  are a of active
resea r ch .  In the mean t ime, o ne si m p ler  so lution isto  have several in d e p end e nt VM states,  each runni n g i n thei rown thread  (orpr o ces s), and
com muni cating vi a mess a ge pass i ng. Lua  makes  th iseasy ,  b ut  it sV M i s 
c ons i der ably lig ht er  than  Pyt h o n ' s.
--- ---euroc lydon
If Pyth oncan' t  ge t thi s thr eadi ng  thin g work ed ou t,  isn't the lang uage  going
to  ge tlef tbehin d  as pa ral lel  archit ecturem arc he s o nw ard?

~~~
cdavid
T he r e  a re manyways t o  e xp l oit mult i- cor es,m u lti-t hrea d in g is jus t oneo fth em. Man yotherte chn iqu es ex i st.Also,one thin g tor ealize is that if speed
real l ym a tt e rs ( lik e in scient ific apps ), y o uwil l  getmuchhi g he r  speed
increase byre writin g som e parts in Cthan by al l o w ing usin g al l  t h e cores
from p ython  (at least with onl y a c ouple of  cores).F inally,a p oin t w hic h is  not often br o u ghtbu t is cr uci al in m y o pin ion  is 
a bout C ex tension:  t he  G IL  ma kes C extensio ns  mu ch  ea siert o writ e. Th at's  o ne big  reason fo r pyth on  s ucce ssin  thefirs t  pl ace.

- -----
zep ole n 
Why  a re  real t hreads so imp or tant? Does  a nyo ne hav e an examp le where threads
wouldb e muchbet ter t hanusing the  mu ltipro ces s ing mo dule?
~~~ma hmu d Withn a ti veth reads, all y our  thre ad s ha ve thei r own identity and the y're
known to  th e OS t as k s c he d ule r. Sot h ey can all bloc k or run i nd e pe ndently.
B u t w it h green  thread s, th e  OS doe sn't know  about yo ur  "th r e ads "; when th e
parent  processi s blo c ke d, so  ar e allth e t hread s.

~~ ~ 
ba rrkel 
One o fthe ba sic a ss umptions  behind green threads i s that  youare n't going  to mak e blockin g OS  calls fro m y our  gree n t h reads  - t h eint e rpret er s h ould
inte r ce ptt h os e, run them i n  adiffe ren t  threa d (o r ideally, run wi t h async
I/O) s o th a t  it  ca n  sch ed ule a differen t t hr e a d untilth e call retu rns.

Caml book_([http://caml.inria.fr/pub/docs/oreilly-
book/html/book-ora082....](http://caml.inria.fr/pub/docs/oreilly-
book/html/book-ora082.html) ))._In other words, it_plugs the_worst_memory
leaks caused_by_reference counting.

How to_do multiprocessor /_multithread GC well is_still an area_of_active
research. In the mean time, one simpler solution is to have several
independent VM states,_each_running in_their_own_thread (or process), and
communicating via_message passing. Lua makes this_easy, but_its VM is
considerably lighter than Python's.

------
euroclydon
If Python can't_get_this threading thing_worked out, isn't the language going
to get left behind_as parallel architecture marches onward?

~~~
cdavid
There are_many ways to_exploit_multi-cores,_multi-threading is just one_of
them. Many other techniques exist. Also,_one thing to realize is that_if speed
really matters (like in scientific apps),_you will get much higher speed
increase_by rewriting some parts in_C than_by allowing using all the_cores
from python (at_least with_only a couple_of cores).

Finally, a point which is_not often brought_but is crucial in my opinion_is
about_C extension: the_GIL_makes_C extensions_much easier to_write._That's one
big_reason_for python success in the first_place.

------
zepolen
Why_are real threads so important? Does anyone_have an example where_threads
would_be much better than_using the multiprocessing module?

~~~
mahmud
With native_threads, all your threads have their_own identity_and they're
known_to the OS task scheduler. So they can all block or_run independently.
But with green threads, the_OS doesn't know about_your "threads";_when_the
parent process is_blocked,_so are_all the threads.

~~~
barrkel
One of the basic assumptions_behind green_threads is that you aren't going_to
make blocking OS calls_from_your green threads - the interpreter_should
intercept those, run them in a_different thread (or ideally, run_with_async
I/O)_so that it can schedule_a different thread until the call_returns.
