1. A system operative to achieve load balancing among a plurality of compute elements accessing a shared memory pool, comprising: a shared memory pool configured to store a plurality of data sets comprising a first data set and a second data set; a first data interface configured to extract and serve any of said plurality of data sets from said shared memory pool, and comprising an internal registry configured to keep track of the data sets extracted and served; and a plurality of compute elements comprising at least a first compute element and a second compute element, in which said plurality of compute elements are communicatively connected with said first data interface, and said plurality of compute elements are configured to execute distributively a first task associated with said plurality of data sets; wherein: the first compute element is configured to send a first data request to the first data interface after deciding that said first compute element is currently available or will soon become available to start or to continue contributing to said execution, and the first data interface is configured to: conclude, according to the internal registry, that the first data set is next for processing; extract the first data set from the shared memory pool; serve the first data set extracted to the first compute element, thereby enabling said first compute element to perform said contribution; and update the internal registry to reflect said serving of the first data set; and the second compute element is configured to send a second data request to the first data interface after deciding that said second compute element is currently available or will soon become available to start or to continue contributing to said execution, and the first data interface is configured to: conclude, according to the internal registry reflecting that the first data set has already been served, that the second data set is next for processing; extract the second data set from the shared memory pool; serve the second data set extracted to the second compute element, thereby enabling said second compute element to perform said contribution; and update the internal registry to reflect said serving of the second data set, such that said decisions regarding said availabilities facilitate said load balancing in conjunction with said executing distributively of said first task, without the plurality of compute elements being aware of the order in which said plurality of data sets are extracted and served.