title: Removing a "row" from a structure array
body: <p>This is similar to a question I asked before, but is slightly different:</p>
  
  <p>So I have a very large structure array in matlab. Suppose, for argument's sake, to simplify the situation, suppose I have something like:</p>
  
  <p>structure(1).name, structure(2).name, structure(3).name structure(1).returns, structure(2).returns, structure(3).returns (in my real program I have 647 structures)</p>
  
  <p>Suppose further that structure(i).returns is a vector (very large vector, approximately 2,000,000 entries) and that a condition comes along where I want to delete the jth entry from structure(i).returns for all i.  How do you do this?  or rather, how do you do this reasonably fast?  I have tried some things, but they are all insanely slow (I will show them in a second) so I was wondering if the community knew of faster ways to do this.</p>
  
  <p>I have parsed my data two different ways; the first way had everything saved as cell arrays, but because things hadn't been working well for me I parsed the data again and placed everything as vectors.</p>
  
  <p>What I'm actually doing is trying to delete NaN data, as well as all data in the same corresponding row of my data file, and then doing the very same thing after applying the Hampel filter.  The relevant part of my code in this attempt is:</p>
  
  <pre><code>for i=numStock+1:-1:1
      for j=length(stock(i).return):-1:1
          if(isnan(stock(i).return(j)))
              for k=numStock+1:-1:1
                  stock(k).return(j) = [];
              end
          end
      end
      stock(i).return = sort(stock(i).return);
      stock(i).returnLength = length(stock(i).return);
      stock(i).medianReturn = median(stock(i).return);
      stock(i).madReturn = mad(stock(i).return,1);
  end;
  
  for i=numStock:-1:1
      for j = length(stock(i+1).volume):-1:1
          if(isnan(stock(i+1).volume(j)))
              for k=numStock:-1:1
                 stock(k+1).volume(j) = [];
              end
          end
      end
      stock(i+1).volume = sort(stock(i+1).volume);
      stock(i+1).volumeLength = length(stock(i+1).volume);
      stock(i+1).medianVolume = median(stock(i+1).volume);
      stock(i+1).madVolume = mad(stock(i+1).volume,1);
  end;
  
  
  
  for i=numStock+1:-1:1
      for j=stock(i).returnLength:-1:1
          if (abs(stock(i).return(j) - stock(i).medianReturn) &gt; 3*stock(i).madReturn)
              for k=numStock+1:-1:1
                  stock(k).return(j) = [];
              end
          end;
      end;
  end;
  
  for i=numStock:-1:1
      for j=stock(i+1).volumeLength:-1:1
          if (abs(stock(i+1).volume(j) - stock(i+1).medianVolume) &gt; 3*stock(i+1).madVolume)
              for k=numStock:-1:1
                  stock(k+1).volume(j) = [];
              end
          end;
      end;
  end;
  </code></pre>
  
  <p>However, this returns an error:</p>
  
  <p>"Matrix index is out of range for deletion.</p>
  
  <p>Error in Failure (line 110)
                  stock(k).return(j) = [];"</p>
  
  <p>So instead I tried by parsing everything in as vectors.  Then I decided to try and delete the appropriate entries in the vectors prior to building the structure array.  This isn't returning an error, but it is very slow:</p>
  
  <pre><code>%% Delete bad data, Hampel Filter
  
  % Delete bad entries
  id=strcmp(returns,'');
  returns(id)=[];
  volume(id)=[];
  date(id)=[];
  ticker(id)=[];
  name(id)=[];
  permno(id)=[];
  sp500(id) = [];
  
  id=strcmp(returns,'C');
  returns(id)=[];
  volume(id)=[];
  date(id)=[];
  ticker(id)=[];
  name(id)=[];
  permno(id)=[];
  sp500(id) = [];
  
  % Convert returns from string to double
  returns=cellfun(@str2double,returns);
  sp500=cellfun(@str2double,sp500);
  
  % Delete all data for which a return is not a number
  nanid=isnan(returns);
  returns(nanid)=[];
  volume(nanid)=[];
  date(nanid)=[];
  ticker(nanid)=[];
  name(nanid)=[];
  permno(nanid)=[];
  
  % Delete all data for which a volume is not a number
  nanid=isnan(volume);
  returns(nanid)=[];
  volume(nanid)=[];
  date(nanid)=[];
  ticker(nanid)=[];
  name(nanid)=[];
  permno(nanid)=[];
  
  % Apply the Hampel filter, and delete all data corresponding to
  % observations deleted by the filter.
  
  medianReturn = median(returns);
  madReturn = mad(returns,1);
  
  for i=length(returns):-1:1
      if (abs(returns(i) - medianReturn) &gt; 3*madReturn)
          returns(i) = [];
          volume(i)=[];
          date(i)=[];
          ticker(i)=[];
          name(i)=[];
          permno(i)=[];
      end;
  end
  
  medianVolume = median(volume);
  madVolume = mad(volume,1);
  
  for i=length(volume):-1:1
      if (abs(volume(i) - medianVolume) &gt; 3*madVolume)
          returns(i) = [];
          volume(i)=[];
          date(i)=[];
          ticker(i)=[];
          name(i)=[];
          permno(i)=[];
      end;
  end
  </code></pre>
  
  <p>As I said, this is very slow, probably because I'm using a for loop on a very large data set; however, I'm not sure how else one would do this.  Sorry for the gigantic post, but does anyone have a suggestion as to how I might go about doing what I'm asking in a reasonable way?</p>
  
  <p>EDIT: I should add that getting the vector method to work is probably preferable, since my aim is to put all of the return vectors into a matrix and get all of the volume vectors into a matrix and perform PCA on them, and I'm not sure how I would do that using cell arrays (or even if princomp would work on cell arrays).</p>
  
  <p>EDIT2: I have altered the code to match your suggestion (although I did decide to give up speed and keep with the for-loops to keep with the structure array, since reparsing this data will be way worse time-wise).  The new code snipet is:</p>
  
  <pre><code>stock_return = zeros(numStock+1,length(stock(1).return));
  
  for i=1:numStock+1
      for j=1:length(stock(i).return)
          stock_return(i,j) = stock(i).return(j);
      end
  end
  
  stock_return = stock_return(~any(isnan(stock_return)), : );
  </code></pre>
  
  <p>This returns an Index exceeds matrix dimensions error, and I'm not sure why.  Any suggestions?</p>