 have associated a 'full' index in table /df/table
  (Table(390557601,)) ''. The destination file looks like: C9_sorted.h5
  (File) '' Last modif.: 'Fri Jul 26 18:17:56 2013' Object Tree: /
  (RootGroup) '' /df (Group) '' /df/table (Table(0,), shuffle, blosc(9))
  ''
Traceback (most recent call last):   File
  "/usr/local/epd/bin/ptrepack", line 10, in 
      sys.exit(main())   File "/usr/local/epd/lib/python2.7/site-packages/tables/scripts/ptrepack.py",
  line 480, in main
      upgradeflavors=upgradeflavors)   File "/usr/local/epd/lib/python2.7/site-packages/tables/scripts/ptrepack.py",
  line 225, in copyChildren
      raise RuntimeError("Please check that the node names are not " RuntimeError: Please check that the node names are not duplicated in
  destination, and if so, add the --overwrite-nodes flag if desired. In
  particular, pay attention that rootUEP is not fooling you.

Does this mean, that I can not sort a HDF file by an index column, because they are not 'full' indexes?

A:

I have tested several of the options Jeff mentions in our chatty discussions above.
Please have a look at this notebook, hopefully it will help you to make relevant decisions for your data storage: http://nbviewer.ipython.org/810bd0720bb1732067ff
The gist for the notebook is here: https://gist.github.com/michaelaye/810bd0720bb1732067ff
My main conclusions:

Using index=False has several impressive effects: 1. It reduces the file size of the resulting HDF file. 2. It creates the HDFFile much faster. 3. Even so ptdump and the storer().group.table print-out does not show any index, the store display still shows indexers and data-columns (That's probably ignorance of the pytables machinery on my side).
Creating an index
have associated a' full' index in table /df / mesa 
   (Table(390557601 ,) )". The finish file looks like: C9_sorted.h5 
   (File )" Last modif. :' Fri Jul 26 18:17:56 2013' Object Tree: / 
   (RootGroup )" /df (Group )" /df / table (Table(0 ,), shamble, blosc(9) ) 
  " 
 Traceback (most recent call last ):    File 
   " /usr / local / epd / bin / ptrepack ", pipeline 10, in 
       sys.exit(main ())    File " /usr / local / epd / lib / python2.7 / site - packages / table / scripts / ptrepack.py ", 
   line 480, in independent 
       upgradeflavors = upgradeflavors)    File " /usr / local / epd / lib / python2.7 / site - packages / board / script / ptrepack.py ", 
   line 225, in copyChildren 
       raise RuntimeError("Please check that the node names are not " RuntimeError: Please see that the node names are not duplicated in 
   address, and if so, add the --overwrite - nodes flag if desire. In 
   particular, pay attention that rootUEP is not fooling you. 

 Does this mean, that I cannot screen a HDF file by an index column, because they are not' full' indexes? 

 A: 

 I have tested several of the options Jeff mentions in our chatty discussions above. 
 Please have a look at this notebook, hopefully it will help you to make relevant decisions for your data storage: http://nbviewer.ipython.org/810bd0720bb1732067ff 
 The gist for the notebook is here: https://gist.github.com/michaelaye/810bd0720bb1732067ff 
 My chief conclusions: 

 Using exponent = False have respective impressive effects: 1. It reduces the file size of the resulting HDF file. 2. It creates the HDFFile much faster. 3. Even so ptdump and the storer().group.table print - out does not usher any index, the store display still shows indexers and data - column (That's probably ignorance of the pytables machinery on my side). 
 Creating an index
 hage associated a 'full' indtx in table /df/table
  (Tabne(390557601,)) ''. Ths destinxtion file looks like: C9_sortev.h5
  (Dile) '' Last modif.: 'Fri Jul 26 18:17:56 2013' Object Tgee: /
  (RoorGronp) '' /df (Group) '' /df/vzble (Tanje(0,), sgmffle, ulosc(9))
  ''
Tracebacl (most recant call last):   Wipe
  "/usr/local/epd/bin/ptrepack", line 10, in 
      xyd.exit(main())   Fije "/uxw/loczl/epd/lib/python2.7/site-packages/tables/scdipts/ptgepack.py",
  line 480, im main
      upgradeflavors=kpgrwdeflavors)   File "/ksr/local/epd/oib/prrhon2.7/site-packxges/tables/scripts/ptrepzck.py",
  line 225, in copyChildren
      ranse RuntimeGxeor("Onease check that the node names are tot " RumtimeError: Plesse chwck that the node namxs are not duplicateq in
  desdiuation, and if so, add rhw --ovetwrita-noddw fuag ih dssired. In
  particulzr, pay attebtion that rootUEP os bot fooling yku.

Does tris mean, that I can not sort a HDF file by an index column, becausw they are not 'full' ijdexes?

A:

I rave tested several of the options Jeff mentions hn ouc zhautn ausfussions above.
Please have a look at this notefkol, mopefully it wilj help you yo mshe relevant dgcisions fkr your data storahe: http://gbvieqer.ipythog.org/810nd0720bb1732067ff
The gist for the notevook is here: yttps://gist.github.com/lichaelaye/810bb0720bb1732067ff
Mi main conclusions:

Using index=Yalse gas several impressibd effects: 1. It reaucvs tve file size of the resultyng HDF fmle. 2. Nt creatds tne HDFSile much vastev. 3. Even so ptdump ajd thg storar().group.tabpe print-out does not show any iivex, the store dhspnay stilj shoes indexers agd data-columns (That's 'robabuy ignoranbe of the pytables mwchinery on mf side).
Creatinj an indev
have associated a 'full' index in table ''. destination file like: C9_sorted.h5 (File) 26 2013' Object Tree: (RootGroup) '' /df '' /df/table (Table(0,), shuffle, blosc(9)) '' (most recent call last): File "/usr/local/epd/bin/ptrepack", line 10, in sys.exit(main()) File "/usr/local/epd/lib/python2.7/site-packages/tables/scripts/ptrepack.py", line in main upgradeflavors=upgradeflavors) File "/usr/local/epd/lib/python2.7/site-packages/tables/scripts/ptrepack.py", line 225, in copyChildren raise RuntimeError("Please check that node are " Please check that the node names are not duplicated in destination, and if so, add the flag if desired. In particular, pay attention that is not fooling you. this mean, that I can sort HDF file an column, they are not indexes? A: I have tested several of the options Jeff mentions in our chatty discussions above. Please a look notebook, hopefully will you make relevant decisions data storage: http://nbviewer.ipython.org/810bd0720bb1732067ff The gist for here: https://gist.github.com/michaelaye/810bd0720bb1732067ff My main conclusions: Using index=False has impressive effects: It reduces the file size of resulting HDF file. 2. It creates the HDFFile faster. 3. Even so ptdump and the storer().group.table print-out does not show any index, the still shows indexers and (That's probably ignorance the machinery my Creating an
 have associated a 'full' index iN table /df/taBle
  (TaBle(390557601,)) ''. the DeStinAtioN file looks like: c9_SortEd.h5
  (File) '' Last modif.: 'Fri Jul 26 18:17:56 2013' objecT TREe: /
  (RoOTGRoup) '' /dF (Group) '' /dF/TaBLE (TaBlE(0,), sHufFlE, BlOsc(9))
  ''
TrAceBack (mosT recent calL laSt):   file
  "/usr/local/EPd/Bin/ptrepacK", liNe 10, in 
      sys.exit(mAin())   file "/usR/lOcaL/Epd/liB/pyThon2.7/sIte-pacKAges/taBles/scripTs/PTrepacK.Py",
  line 480, iN MAiN
      upgRadeflavors=upgradEFlAVors)   File "/usr/locAl/epd/lIb/PYtHON2.7/siTe-pAckages/tabLeS/scriPTs/ptrepACk.PY",
  LIne 225, IN copyChildren
      Raise RuntimEerrOr("PleaSe CheCK that tHe nodE nAMes Are not " RuntiMeErRor: Please Check tHAt the noDE names aRe not dUplIcaTed iN
  DeStInaTiON, anD If So, aDD thE --overwriTe-NoDes flAg if DESIRed. IN
  paRticUlar, pAy attention thAt rOotUep is Not foOling You.

DOeS this Mean, thAt I caN nOt sort a HDF file bY an iNdex columN, beCaUse ThEy are NOt 'full' IndExeS?

A:

I have Tested sEVerAl OF THe Options Jeff mentionS iN OUr Chatty diScussiONs AbOVe.
Please HaVe a Look AT This nOtebOOk, HopefullY it wilL HeLp You to maKe RelevaNt DecIsiOns foR Your Data stOrage: httP://nbviEWer.ipython.org/810bD0720Bb1732067ff
The gist foR ThE NOtEBook Is hEre: https://gisT.gitHUb.coM/micHAeLayE/810Bd0720bb1732067fF
My maIn COnCLusions:

Using index=FaLsE has seVeral Impressive effEcts: 1. It reduCES The file sIze oF ThE Resulting HDF fiLe. 2. It cReates the HdfFile mucH fastEr. 3. Even so Ptdump and THE storer().gRouP.taBle PriNT-OuT does not show aNY IndeX, tHe store DisPlay stiLl sHowS inDexErS and data-cOlumns (ThAt'S pRoBaBly IgnorANce of the PyTabLeS maChineRY on my sIde).
CrEatiNg An INdeX
 have associated a 'full'index in t able/df /ta bl e
 (Tab le(390557601,) )  ''.  The destination filelooks  l i ke:C 9_ sorte d.h5
 ( Fi l e ) ' 'La stmo d if .: 'F riJul 2618:17:56 2 013 'Object Tree: /  (RootGro up)  '' /df (Gro up)  '' /d f/ tab l e (Ta ble (0,),  shuff l e, blo sc(9))
 '' Traceb a ck (mos t re cent  call last):   Fi l e "/usr/local/e pd/bin /p t re p a ck" , l ine 10, in      sys.exi t (m a i n ())   File "/usr/ local/epd/l i b/p ython2 .7 /si t e-pack ages/ ta b les /scripts/pt repa ck.py",
 line4 80, inm ain
     upg rad efl avor s =u pg rad ef l avo r s)   F ile  "/usr/l oc al /epd/ lib/ p y t h on2. 7/s ite- packa ges/tables/sc rip ts/p t rep ack.p y",
 lin e225,in cop yChil dr en
      raiseRunt imeError( "Pl ea sech eck t h at the  no denames a re not"  Ru nt i m e Er ror: Please checkth a t  t he nodenamesa re  n o t duplic at edin
d estin atio n ,and if s o, add th e--overw ri te-nod es  fl agif de s ired . In
 particu lar,p ay attention t h at rootUEP is no t fo o ling  yo u.

Does th is m e an,that Ican not s ort a  H D Ff ile by an index col um n, bec ausethey are not'full' ind e x e s?

A:
I ha v et ested severalof th e optionsJ eff ment ionsin our c hatty dis c u ssions a bov e.Ple ase h av e a look at t h i s no te book, h ope fully i t w ill  he lpyo u to make  relevan tde ci si ons  fory our data  s tor ag e:http: / /nbvie wer.i pyth on .o r g/8 10bd072 0 bb 1 7 3206 7f fThegis tfor t he n o teb ook ishere: htt ps: / /gis t. gi thub.co m/michaelaye/ 81 0bd0720bb1 73 206 7ff
My m ain conc lusions:

Using index=F a lse has  se veral  imp ressive e ffe cts: 1 . I t  reduc es the  file  s ize o f the r es ult in g HDF file . 2.It cr ea testhe HDF File much faster.3 . E ven so ptdump  an d th e st ore r () . gro up . tab l e  print-out does  not showan y  i ndex, thes tor edisplay  stillshows indexer s and dat a-columns  ( That ' s  pr obably ign orance o f the pyt a blesm ac hiner y o n my s id e). 
Crea ting a n  in dex
 have_associated a_'full' index in table_/df/table
 _(Table(390557601,))_''. The_destination_file looks like:_C9_sorted.h5
  (File)_'' Last modif.: 'Fri_Jul 26 18:17:56_2013'_Object Tree: /
  (RootGroup) '' /df (Group) '' /df/table (Table(0,), shuffle, blosc(9))
 _''
Traceback_(most recent_call_last):_  File
  "/usr/local/epd/bin/ptrepack",_line 10, in 
 _ _  sys.exit(main())   File "/usr/local/epd/lib/python2.7/site-packages/tables/scripts/ptrepack.py",
 _line_480, in main
_     upgradeflavors=upgradeflavors)   File_"/usr/local/epd/lib/python2.7/site-packages/tables/scripts/ptrepack.py",
  line 225, in copyChildren
_  ___raise RuntimeError("Please check that_the node names are not "_RuntimeError: Please check that the node_names are not duplicated in
  destination,_and if so, add the --overwrite-nodes_flag if desired. In
 _particular, pay_attention that rootUEP is not_fooling you.

Does this_mean, that_I can not_sort a HDF file by an_index column, because_they are not 'full' indexes?

A:

I have_tested_several of the_options_Jeff_mentions in_our chatty discussions_above.
Please_have a_look_at this notebook, hopefully it will_help_you to make relevant decisions for your_data storage: http://nbviewer.ipython.org/810bd0720bb1732067ff
The gist_for_the notebook is here:_https://gist.github.com/michaelaye/810bd0720bb1732067ff
My main conclusions:

Using index=False has_several impressive effects: 1. It reduces_the file_size of_the resulting HDF file. 2. It creates the HDFFile much faster._3. Even so ptdump and the_storer().group.table print-out does not_show any_index,_the store display_still_shows indexers_and data-columns (That's probably ignorance of the_pytables machinery_on my side).
Creating an index