Cleaning up code for Conway's Game of Life
Ruby convention is to use CamelCase for Classnames (GameOfLife)
Since "size" and "str" are mutably exclusive parameters, using a hash parameter here makes the API confusing
Using flags like "@test" to define object behavior is bad style (antipattern)
Ruby doesn't have tail recursion, so you shouldn't do recursive calls without end conditions


To make this more OOP like and self documenting you can define a base class for the basic initialization:

class GameOfLife
  def initialize(length)
    @length = length # Why not call this "size"?
    @board = []
  end

  def check_board
    # Implemented as above without the "if" statement
  end

  def choose_neighbour_set(index)
    # As above
  end
end


Then you can use subclasses for specific implementations

class RandomGameOfLife &lt; GameOfLife
  def initialize(size)
    super(size)
    (size*size).times do
      @board &lt;&lt; [0,1].sample
    end
  end

  def iterate_and_output_board
    while true # Iterate till ^C?
      check_board
      sleep 0.5
      puts "\e[H\e[2J"
      @board.each_slice(@length) {|row| p row }
    end
  end
end

class PredefinedGameOfLife &lt; GameOfLife # Or use a better name here
  def initialize(str)
    super(Math.sqrt(str.length))
    str.each_char {|c| @board &lt;&lt; c.to_i}
  end

  def check_board
    # This is just an example how to overwrite method behavior
    # Defining another method using "check_board" might be better here
    super
    @board.join("")
  end
end


Somebody else might cover the efficiency aspect. I don't want to intermix the aspects here and its better to focus on readability first.