{'fraction_non_alphanumeric': 0.02209645874079314, 'fraction_numerical': 0.00853064791311223, 'mean_word_length': 4.723267444629673, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
MESING – a new method of organizing the joint work of neural networks and its metrology

A description of a new method for the collective solution of local problems – the method of evolutionary coordination of solutions based on the original use of genetic algorithms – is given. Interaction rules, developed on their basis, coordinate the work of intelligent agents (actors). Based on the Rasch model, an absolute scale for measuring the intellectual power of actors and the costs of intellectual labor when solving local problems with a given probability of their correct solution is introduced. The unit of measurement for these values is introduced and justified – 1 INT. A number of theorems are presented that make it possible to substantiate a new procedure for obtaining a collective solution (mesing) both to increase the intellectual power of a committee of neural networks by 150 times in comparison with a single neural network as part of a committee, and to reduce the probability of an erroneous decision to zero under certain conditions. As a result of the committee’s work, either the correct decision is formed, or the answer “no solution has been found” with a low probability of an erroneous decision.


Introduction
Currently, there is a significant restructuring of the production method. The old industrial methods are being replaced by modern ones related to digital technologies. Due to the fact that the problems being solved are becoming more complicated and the available resources are not always enough to maintain the required rates of development [1,2], the problem arose of finding and theoretically substantiating new technologies for solving complex intellectual problems arising in science, technology and industry and supporting these technologies appropriate intelligent systems. There is a need for a guaranteed solution of such problems by existing specialists and their groups, as well as specially created intelligent systems [3,4]. Gradually, the idea of collective intelligence was formed as the ability of a collective of intelligent agents to find solutions to problems that are more effective than the best individual solution in this collective. One of the trends in the modern development of artificial intelligence technologies is the organization of joint work of the committees of deep learning neural networks to reduce the number of erroneous decisions in various fields of application.
On the other hand, for the class of problems of the developing digital economy related to decisionmaking and solving mass creative problems, such basic quantities as the intellectual complexity of the task and the intellectual power of the actor have not been determined. The lack of an objective system for measuring these quantities does not allow comparing various systems of collective and individual intelligence and assessing the changes made to the procedures for the functioning of these systems to improve their efficiency.
The current situation impedes the development of methods of group work, which will guarantee the solution of difficult problems put forward by practice and create appropriate software products to achieve these goals.

Overview of problem solving methods
As shown by the experimental and theoretical studies carried out by us, tools with the required characteristics can be created on the basis of the theory of collective intelligence systems developed in the course of our research, based on the construction of interaction rules suitable for these systems, as well as the corresponding measurement tools [4]. In these works, the concept of an actor was introduced as an intelligent agent acting in two roles -as a generator of solutions to some local problems and as an expert -evaluator of other people's solutions.
In [4][5][6][7], a description of the evolutionary decision matching method based on genetic algorithms acting as a coordinator of the joint work of actors is given. The essence of this method lies in the fact that at the generation stage the actors propose the first variants of partial solutions to the problem posed to them, and at the stages of evolutionary agreement they evaluate other people's solutions (slots), cross them, make some changes -mutations, exchange updated slots until most of the experts have matching solutions.
The new method has shown its effectiveness in drawing up a subjective portrait by a team of witnesses [8]. Later, it was repeatedly tested on other problems and received the name "the method of evolutionary coordination of solutions"(MES).
The system for measuring the intellectual complexity of the problems being solved and the intellectual strength of actors is based on the Rasch model [9], which links the probability of getting the correct answer with the level of difficulty of test questions and the degree of preparedness of the actors .
The preparedness of the actor and the difficulty of the task are latent variables. It is believed that they cannot be directly measured, but can be obtained only as a result of using mathematical models operating with observable variables, called indicator variables. The indicators in the Rasch model are the results of testing a group of actors using test tasks of various difficulty levels. To determine the dependence of the probability of a wrong decision on the difficulty of the task, we first define the dependence of the probability of filling a slot with an actor using the Rasch model: where is the preparedness of the actor to fill the slots with both correct and incorrect answers. The probability of an actor making a wrong decision will be Similar expressions are obtained for the probabilities of correct and erroneous examination: Dependency graphs for (1 -5) are shown in Fig. 1. The graph also shows the dependence of the probability of a wrong decision 2 = 1 − in the case when binary logic is used to assess the correctness of decisions, and the actor seeks to give an answer, even if he does not know it and tries to guess. It is seen that with increasing difficulty of the problem, this probability significantly exceeds the probability . The decrease in the probability of making a wrong decision with increasing difficulty of the problem is due to the fact that the actor gives the answer "no solution was found" in difficult cases for him.
The tendency of an actor to generate an erroneous answer can be characterized by the value = − , an actor's propensity for erroneous examination is = − . For an ideal actor, these values are considered equal to zero and his preparedness is further denoted by the value θ. Figure 1. Graphs of dependences G R (β), G S (β) и G N (β) on the difficulty of the problem β In [10], a theorem was proved, which is of fundamental importance for the issues considered in this paper, since it opens up the possibility of the probability of making erroneous decisions by a group actor tending to zero under certain conditions. The theorem is formulated as follows.
Let a group actor consist of M single actors with preparedness , , , and these values belong to the interval [ , ]. Moreover − = 2 , where δ is the half-width of the interval.
Let there be a probability of an incorrect solution of a problem of arbitrary difficulty by a group actor.
Then → 0 for → ∞ and < ln 2. In the case of using neural networks as MES actors, the value of δ can be reduced if it is greater than ln2 by setting the corresponding values of the tuning parameters when training neural networks. You can also increase their number, which will allow you to get an arbitrarily small value of the probability of making erroneous decisions, since neural networks are trained in such a way that in difficult cases they will give the answer "no solution found".
Consequently, the committee of neural networks will give either the correct solution or the answer "no solution found" with an arbitrarily small value of the probability of an erroneous decision. In the latter case, the task can be transferred to a committee with greater intellectual power, or the number of neural networks can be increased.
In [11] the metrology of collective work is considered. The unit of measurement of the basic values of collective intelligence systems is substantiated -the intellectual complexity of the problem S (β), which determines the labor costs of actors for its correct solution with a predetermined probability q and the intellectual strength of the actors Z (θ).
The unit of measurement was introduced on the basis of the principle of equal pay for equal work and the fulfillment of the assumption that if at least one of the actors in the group solved the problem correctly, then it is considered that the group actor also solved it and the probability q, therefore, can be determined by the formula: It is shown in [11] that the use of MES allows one to obtain such a probability of a correct solution. Labor costs aimed at solving the problem with a given probability q are measured in units of intellectual labor costs INT, introduced and substantiated in the theory developed by the authors. The following definition of this unit has been introduced: 1 INT is the cost of intellectual labor of an ideal actor with readiness θ = 0 when solving the problem of difficulty β = 0 with probability 0.5.
In [11], theorems are proved connecting the basic values of collective intelligence systems with the latent values θ and β introduced by Rasch. The intellectual complexity of the problem of difficulty β, solved with a given probability q is where 0 is the number of actors who correctly solve the problem of difficulty β = 0 with probability q, = 1 2 2 = 2.05720346 …, 0 is determined from the expression The intellectual strength of an actor with preparedness θ is equal to Let us estimate, based on (1), the level of preparedness of an actor who solves the problem of difficulty β = 0 with a probability of 0.999. Substituting the found value θ = 6.99067 in (9) we get the value of the intellectual power of the actor Z≈145.6 INT. From (8) it follows that for q = 0.999, actors with preparedness θ = 0, or, according to (9), with intellectual strength 1 INT each, will cope with this task. Since 10 actors with an intellectual power of 1 INT solved the problem solved by an actor with an intellectual power of 145.6 INT, we can say that a group of 10 actors using MES increased their intellectual power by about 150 times.
In [10], the definition of a second-rank group actor is given, consisting of M actors divided in to 1 groups, each of which consists of 2 actors, so that = 1 2 . At 1 = 1 we get a group actor of the first rank, consisting of M actors.
The development of mathematical models of group actors and the calculation of their characteristics in a wide range of changes in input parameters made it possible to pose an important question for practical applications: Is it possible, in principle, to create such a system of collective intelligence, which is guaranteed to reduce the probability of making erroneous solutions to problems of arbitrary difficulty to values acceptable for practice? At the same time, the main property of these systems should be preservedobtaining the correct solution with a probability higher than that of the best of the group's actors.
The absence of a system for measuring the difficulty of tasks and the levels of preparedness of actors and the absence of a theory of systems of collective intelligence did not allow previously to pose a very important question for practice about the predictability of the results of group problem solving. When conducting research using test items, such a forecast is not needed, since the correct answers are known a priori. In the case when the problem is solved for the first time, it is currently impossible to assess the probability of its correct or erroneous solution.
In this regard, the question arises -if it is impossible to assess these probabilities, then is it possible to organize the work of the actors in such a way that, using some cross-checking group procedures, the group could obtain either the correct solution, with a small, close to zero probability of an erroneous decision, or the answer "no solution found"? Obviously, a single, even the most prepared actor, cannot solve problems with high difficulty for him with a low probability of error, since there is always a problem on the verge of this actor's capabilities, when he cannot determine whether he has solved it correctly. To check the correctness of the solution, it is necessary to use the work of the second actor, or a group of actors, with greater preparedness than that of the actor who solved the problem with an uncertain outcome.
The problem of minimizing the probability of erroneous decisions is also relevant for image recognition using artificial intelligent agents -deep learning neural networks. In medical diagnostics, the solution to this problem will help to significantly reduce the number of false diagnoses, and in the case of mass recognition of people in streams, eliminate tens of thousands of false recognitions.

Methodology and results of experiments on computer models
Let us apply for our purposes an analogue of the principle of duplication of identical devices operating in parallel, known in the art, when the control action is generated when most of the output signals coincide. This technique significantly reduces the likelihood of failures and failures.
Let us calculate the probabilities of making an erroneous solution to a certain problem when using an actor of the 2nd rank. Let this actor consist of three actors of the 1st rank, which consist of five actors each, that is, a total of 15 actors will be used. In this case, the condition of minimizing the cost of solving the problem must be satisfied. We will assume that in our case the problem has one correct answer and two types of incorrect answer.
The scheme of work of a group actor of the 2nd rank using evolutionary decision reconciliation and the mesing procedure based on this method, so named by analogy with the well-known bagging and boosting procedures of deep learning of neural networks, is shown in Fig. 2. As will be seen below, the new procedure is fundamentally better than the ones mentioned above, since it gives guarantees of obtaining a correct solution or an answer "no solution found" with a low probability of error even in difficult cases.
At the stage of generating solutions, the actors fill in the slots in accordance with their preparedness. In the figure, the black rectangle is a correctly filled slot, gray and dark gray represent one of the two incorrect answers, the white rectangle imitates the answer "no solution found". In accordance with the scheme of work at the stage of evaluation of decisions, lists of possible solutions are drawn up, which are a combination of all completed solutions. Actors, analyzing these options, choose the correct ones from them in their opinion and fill them in their previously empty slots. Further, the results of the work of the 1st rank actors are formed. In the decision, equally filled slots are presented if their number is more than half of the number of actors, or the answer "no solution was found" if none of the types of slots received a majority of votes. If three out of three slots match, then the content of these slots forms the result of the work of the 2nd rank actor, otherwise the slot is filled with the answer "no solution found". In accordance with this scheme, a computer model was compiled and statistical calculations of the required probabilities were carried out using the Monte Carlo method for various values of the actors' preparedness within a wide range of changes in the difficulties of tasks. The averaging of the results was carried out on a million tests of each calculation option, which provided sufficient accuracy. Figure 3 shows In all experiments, it was assumed that the number of types of incorrect answers does not exceed four.
Four groups of experiments were carried out. In the first group, the curves of dependences of probabilities on the difficulty of the problem of a group actor of the 1st rank were calculated for M = 51. The probability of an erroneous decision at the maximum was 0.0023 in this case.
In the second group of experiments at M = 15, the maximum error was about 0.1, which is unacceptable in a number of practical applications. The results of the first and second groups of experiments are shown in Fig. 3a).
In the third group of experiments, a group actor of the 2nd rank was created, consisting of three actors of the 1st rank at 2 =5. We considered the case of coincidence of three results from three equally filled slots. In the figure, the corresponding curves are indicated by the symbol (III). The maximum error in this case was 0.00053. On a large interval of problem difficulties, the probability of a correct solution exceeds the values of the probability of a correct solution of a single actor. The results of the third group of experiments are shown in Fig. 3 b). In the figure, the corresponding curves are indicated by the symbol (III). The maximum error in this case was 0.00053. On a large interval of problem difficulties, the probability of a correct solution exceeds the values of the probability of a correct solution of a single actor. The results of the third group of experiments are shown in Fig. 3b).  Calculations were also carried out for the case when the result of the work of the 2nd rank actor was formed by the coincidence of two out of three slots. The corresponding curves are marked with an index (II). It can be seen that the probability of a correct solution increased in comparison with case (III), but the value at the maximum of the curve ( ) in this case increased in comparison with ( ) by 20 times. For practical application, therefore, the method of matching three results out of three must be adopted.
In the fourth group of experiments, interesting results were obtained when modeling the work of a group actor of the 2nd rank, consisting of actors striving to introduce an erroneous solution into the system. They had high levels of preparedness for the generation and examination of solutions, and knowing the correct solution, they gave deliberately incorrect answers.
As a result, for a group actor of the 2nd rank, the value of the probability of making a correct decision in a large area of change in the difficulty of tasks turned out to be significantly lower than for a single actor.
At the same time, increased insignificantly compared to the group actor of the 2nd rank, consisting of actors striving to find the correct solution, and amounted to = 0.0025, which is comparable to the results of the group actor of the 1st rank, consisting of 51 actors (51) = 0.0023.
We also experimented with actors who randomly choose solutions. In this case, for a group actor of the 2nd rank, the probability of a correct choice was 0.00020, an erroneous one, 0.00079, and the probability of the answer "no solution was found" was 0.99901.
Studies have also been conducted on the effect of group size and number on the likelihood of erroneous decision making. It turned out that with a linear increase in the number of actors, there is an exponential decrease in the probability of making an erroneous decision. This result can be explained by the fact that the probability of a composite event is determined by the product of the probabilities of single events.
In [12], when solving problems of neural network recognition, the messing procedure was first used. The problem of making decisions with a minimum probability of erroneous recognition of objects was posed and solved. Theorems on the conditions for the existence of such a solution are formulated and proved.
The method proposed in [12] was tested in object recognition by a collective of neural networks. A single-layer neural network formed according to the classical scheme for 256 inputs was trained on a series of 100 photographs of four objects from different angles, shot in different lighting conditions. By the method of back propagation of the error during training, the components of the vectors of the weight coefficients of neural networks were obtained on different training samples presented in a random way from a hundred initial ones for 11 computers and 4 objects.
The averaged values were also obtained for the corresponding samples. Further, ten photographs of each of the 4 objects were obtained, which were subjected to a special procedure of varying degrees of "noise".
The results of solving the problem of object recognition by a collective of neural networks are shown in Table 1. Index V corresponds to the answer "no solution found". The following conclusions were drawn from the test results.
-The experiments have fully confirmed the theory that a collective of neural networks can give close to zero probabilities of erroneous recognition even in difficult cases.
-Application of single neural networks does not guarantee correct recognition, and in difficult cases the probability of an erroneous decision can be close to 0.5.
-Using neural networks for evaluating the results of object recognition by other neural networks when using the messaging procedure can significantly increase the probability of correct recognition in simple cases and reduce to almost zero the probability of erroneous recognition.

Discussion
As a result of the creation of the foundations of the theory of collective intelligence systems, described in [10,11] and concisely in this work, and the conducted computer experiments, the following conclusions were made: -A system and an absolute scale for measuring the intellectual power of people in their areas of activity, as well as AI systems, both single and their teams and committees, has been developed. Introduced and mathematically substantiated unit of measurement of intellectual strength -1 INT. Calculations show that differences in the magnitude of the intellectual power of actors differ by tens of thousands of times.
-A technology has been developed for the evolutionary coordination of solving local problems by both carriers of natural and artificial intelligence and their symbiosis. Distinctive features of these systems are: a) the proven effect of increasing intellectual power in a group of 10 actors by about 150 times compared to the average bearer of intelligence in a group; b) for the first time in the world, intelligent systems have been obtained that allow finding practically error-free solutions. These systems enable the group to give either the correct answer, or the answer "no solution was found," with an error probability close to zero, even if the difficulty of the problem exceeds the capabilities of the carriers of intelligence. A single carrier, on the other hand, is always prone to errors in difficult cases that neither he himself nor an outside expert with the same intellectual power can determine.
The property described in point b) is especially relevant when using AI systems. AI systems are still banned, or used with great care and mistrust, in areas where mistakes cannot be made.
The construction of collective intelligence systems, assembled from single neural networks with deep learning and working using the messaging procedure, makes it possible to get guaranteed correct solutions, or, in the case of the answer "no solution found", transfer tasks to other systems with greater intellectual power or consisting of a larger number of actors .
Therefore, using mesing, AI can be extended to almost all areas of application, including medical diagnostics by images, nuclear industry, defense, finance, streaming human recognition, etc.
To reduce the likelihood of a wrong decision, it is necessary to increase the number of groups and the number of actors in the group. With a linear increase in the number of actors, there is an exponential decrease in the probability of an erroneous decision.