How to group li element via using Drupal.behaviors?
I am not completely sure I understand what you are trying to do - I am assuming the html is some kind of Drupal menu you are trying to add some kind of dropdown/flyout action to?

I am not sure whether you want to be doing all this DOM manipulation on mouseover/mouseout either? At least I think that's what you are doing from looking at that code snippet.

Taking your html example at face value, this bit of javascript should do the job of splitting it out into 'columns':

var $list = $('ul'),
    $items = $('li', $list),
    column_counter = 1,
    $current_list,
    items_to_column=6;

$items.each(function(index,item){
  if (index%items_to_column==0){
    var new_column = $("&lt;li&gt;&lt;div class='column"+column_counter+"'&gt;&lt;ul&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/li&gt;").appendTo($list);
    $current_list = $('ul', new_column);        
    column_counter ++;
  }
  $current_list.append( $(item).detach());
});