Extremal Optimization is a nature-inspired optimization method which features small computational and memory complexity. Due to these features it can be efficiently used as an engine for processor load balancing. The paper presents how improved Extremal Optimization algorithms can be applied to processor load balancing. Extremal Optimization detects the best strategy of tasks migration leading to balanced application execution and reduction in execution time. The proposed algorithm improvements cover several aspects. One is algorithms parallelization in a multithreaded environment. The second one is adding some problem knowledge to improve the convergence of the algorithms. The third aspect is the enrichment of the parallel algorithms by inclusion of some elements of genetic algorithms - namely the crossover operation. The load balancing based on improved Extremal Optimization aim at better convergence of the algorithm, smaller number of task migrations to be done and reduced execution time of applications. The quality of the proposed algorithms is assessed by experiments with simulated parallelized load balancing of distributed program graphs.