Question: Represent the text to find a very short summary (1-2 sentences max)

Question: With a growing number of available services, each having slightly different parameters, preconditions and effects, automated planning on general semantic services become highly relevant. However, most exiting planners only consider PDDL, or if they claim to use OWL-S, they usually translate it to PDDL, losing much of the semantics on the way. In this paper, we propose a new domain-independent heuristic based on a semantic distance that can be used by generic planning algorithms such as A* for automated planning of semantic services described with OWL-S. For the heuristic to include more relevant information we calculate the heuristic at runtime. Using this heuristic, we are able to produce better (fewer expanded states) in less time than with established techniques. We motivate our work by the need of a heuristic for AI planning. Since the search space of domain-independent planners for large problems becomes computationally intractable BID9 we need heuristics to guide our search through the state space. For domain-specific planners that have a special purpose (e.g., finding a route from one place to another for a GPS traffic guidance systems), a heuristic can easily be provided e.g. the Manhattan-Distance or the Euclidean distance. But for an agent which has the capability of creating general plans, these heuristics are not sufficient. This means it is impossible for our general purpose planner to create a problem specific heuristic at design time. Even reusing old ones like it is done for meta-heuristics or learning parameters of hyper-heuristics have only been successfully applied to simple problems BID20. Meta-heuristics or hyper-heuristics have an additional drawback: they need a learning phase to gather information about the problem to be solved. The calculation of the heuristic during runtime is motivated by the additional information available like the grounding information which could consist of concrete in-dividuals to abstract classes describing e.g. the parameters of a service. The creation of heuristics during runtime can lead to the encounter of new concepts used in an interface definition like a service description, which then lead us back to a fundamental question in AI research: How can AI make sense of new concepts? For heuristics this means interpreting the new concepts and adding information to classical heuristic approaches. A function H: state → R + is called heuristic (Russel and Norvig 2002, p. 92) and estimates the distance from a state to a given goal. We extend this definition of heuristic to H: service × state × goal → R + making the heuristic more dynamic since now it is able to adapt with changing goals and services. With that, the heuristic determines the usefulness of the given service in the current state regarding a current goal. This is done because if an alone state would be the information source for the heuristic, information like the service description would be lost. The interested reader is referred to (Pearl 1985) for a formal description of heuristics and their properties. During our analysis of this topic, we have found that understanding the described functionality of a service is an AI-hard task BID26. This is because interpretation of what a description creator might have understood the service to be, might not be entirely reflected in the description. Furthermore, the service can have multiple interpretations in different contexts. Here the context we defined is the additional information relevant to our problem. As an example strategy for problem-solving using a heuristic, we have selected planning. This means our context consists of the start and goal state which include a domain description. Starting from this setup, we need to evaluate if a capability is useful in the endeavour of finding a plan solving our problem. The approach presented in in Figure 1 is a goal-oriented heuristic at runtime utilizing the semantics of the goal and capability description. The heuristic is thought for a oneshop-planning problem, where it is expensive for the agent to try out services since we are looking at possible worldaltering capabilities, which means a learning phase should be kept as short as possible. This is done at runtime so that we know the goal we want to fulfill and can create a heuris- Figure 1: Abstract approach to a greedy heuristic tic reflecting the given problem. We do so by looking at the goal and the capability description we encounter to calculate the usefulness of a capability. Here the idea of the heuristic is to find useful capabilities to try in our search to the goal state, and reprobate others. The heuristic additionally estimates how much of the precondition is fulfilled to see which capabilities are more likely to be executed. These two evaluations of the service are then combined to create our heuristic. In this section we will first look at the state of the art of heuristics in Section 2. Then we create our goal oriented heuristic in Section 3, select a data set to test this heuristic on in Section 4 and discuss the in Section 5. We conclude this experiment in Section 6. The state of the art in general heuristics for the planning problem is limited. The main conference on AI Planning and heuristic search is the International Conference on Automated Planning and Scheduling (ICAPS) / Conference on Artificial Intelligence Planning Systems (AIPS) (ICA 2016). Here, starting from 1990, the last 28 years the community of AI planning has discussed the different approaches on problem solving.1 During that research effort, multiple specializations of the general planning domain have been identified. Most of them use some kind of translation of the semantic domain to classical STRIPS planning in e.g. PDDL, like OWLS-Xplain BID13 BID13 or Simplanner BID15, or agent-based approaches as proposed in BID0. A comprehensive overview can be found in the work of Markou and Refanidis BID18. Service composition approaches focus on Quality of Service, as in BID16, or use e.g. model checking 1 See www.icaps-conference.org for the proceedings.as in BID3.The different nature of a STRIPS-like planning problem without semantic and service planning address the problem of a search for a plan in different ways. Classical planners are highly optimized to solve problems like the 15-puzzle (Rat 1986) or the four-peg towers of Hanoi problem BID14. For semantic general-purpose planning, more general heuristics are needed. There are heuristics like the minimal step count to the goal, called a uniform action cost (Pearl 1985). These heuristics is equal to the step count if an action cost is equal to 1 BID12. If the uniform cost function of 1, the heuristic is admissible since it predicts that the action to be executed is always only one step form the goal. This is sometimes also called an "optimistic" heuristic. Greedy heuristics are those which count the overlap to the goal, thus measuring the usefulness of a service in how much of the goal it archives. This is a quite simple heuristic which performs well regarding its simplicity. Haslum and Geffner BID6 describe a greedy heuristic which is derived from the STRIPS planning problems, executing the services with the highest overlap of their add-list and the goal first. This heuristic is admissible, it can be used for all planning problems, and is the basis of the most successful planners according to the International Planning Competition (Helmert, Röger, and Karpas 2011). However, it only works for the relaxed problem when the service effect does not have a delete-list, and only the add-list is added to the current state to create a new state. The approach of BID6 has been extended by an optimization with an abstraction of the effects called Patterns BID7. Those patterns are then subtracted from each effect and the start and goal states, creating abstract states, which are then mapped to the state space through these patterns. The patterns represent subproblem of the original planning problem, which is already solved, to get the patterns. A pattern is a set of variable assignments which reoccurs in different states, e.g. start and the goal state, creating homomorphism abstractions. The drawback of those Pattern Database Heuristics (PDB) is the that they do not scale up to real-world problems BID11.The approach of BID11 again optimizes the of BID7 ) by adding a Causal Graph structure and the Domain transition graph to the PDB heuristics, ing in "causal graph structural patterns", which approximate the original problem but are more abstract. This abstraction in done with the SAS + formalization of the planning problems BID1, which has an additional restriction for the domain descriptions. The simplification SAS + has the effect that abstractions like those done by Katz et al. are possible. Here e.g. "Post-uniqueness" means that an effect is only given by at most one action. Additionally, the "Binariness" restriction demands that all state variables have exactly two possible values. With an open world assumption and a distributed service development we cannot fulfill those restrictions, thus these cannot be applied to our problem. Learning the domain structure by observing plans is still subject to research. Gregory and Lindsay BID5 propose a model of action cost, which is learned through the observation of plan traces. Even though the ing cost function can be used as a heuristic, its creation, the observation of executed plans, puts this heuristic into the runtime. This interleaving of the plan-and runtime is out of scope for this work because we want to study the understanding of services and measure the degree of understanding by their use in a plan, not the other way around. Despite this being a valid approach, the idea is a trial and error mechanisms of learning the usefulness of services. For certain services, this might be appropriate, but we restrict our domain to intellectual problems where deterministic actions are analyzed. The same argument can be applied with approaches leaning for other planning properties to plan optimality(Nedunuri, Cook, and Smith 2011). Other research on heuristic creation in uncertainty like BID17 is concentrated on numerical effects, which we neglect here. Another approach of guiding the search through the state space is called Landmarking BID10. Landmarks are facts which must hold true on the way towards reaching a goal. The two sub-problems of landmarks are: How to find landmarks BID21, and how to use the information given by a landmark to create a heuristic BID27. These landmarks then can be used to decompose the search problem and use a local search to iteratively search from landmark to landmark BID10.Using landmarks for creating a heuristic is done in the LAMA planner from Richter and Westphal, which performed well in the IPC 2008 BID10, counting fulfilled landmarks in contrast to unfulfilled ones. In BID23 they combine this greedy heuristic search with landmarks with preferred operators that take into account the usefulness of services by keeping them in a "preferred-operator queue". Those preferred operators are in consequence always tried first. Deciding which service is preferred is part of the heuristic. Again, the problem is formalized as a SAS + problem, which lets Richter et al. decide which service is a landmark (because its effect is unique). This is not given in our planning problem, thus this kind of heuristic needs adaption to be able to function with, e.g., the open world assumption. As a , the state of the art in generating heuristics is mainly based on the relaxation of an original problem by abstraction. Some of them use the domain description to structure the search space, others analyze services to identify landmarks which help to break down the search problem. But no heuristic found so far has used the semantics of the planning problem. Thus, those heuristics are often not applicable to general planning in real-world problems. In this work, we propose a new heuristic for service planning using the Semantic Distance between states. Since we are not considering services with different costs, the here looked at planning problem is a constraint satisfaction problem. As a planning algorithm, we used a variation of an A * algorithm, due to its theoretical properties (see (Pearl 1985) ). The variation from the basic A * is that the function g from the cost function to be minimized f = g + h, where h is the used heuristic and g is supposed to be the cost of the path so far, is selected as g = n−1 i=0 h(s i), where s 0 is the start state, s n is the current state, and all states s i are on the path to the current state. In order to determine the semantic distance between two states, we first have to perform a Semantic Decomposition of those states. We then use a Marker Passing algorithm to determine the semantic distance. At first, we will have a look a the semantic decomposition algorithm. The decomposition takes a word and looks up definitions and relations of this work in data sources like Wikipedia, WordNet or other dictionaries. The words related to the original word are then decomposed recursively until a predefined decomposition depth is reached. With that the connectionist interpretation of meaning is represented as a ing graph. The functions AddRelation and AddConcept are convenience methods for adding the relation and concepts into the semantic graph. The functions AddConcept(concept, decomposition) and AddRelation(relation, GetTargets(relation), decomposition) add the concepts or relations to the graph which represents our decomposition. AddConcept adds the given concept to the graph nodes and AddRelation adds the relation between the concept its targets to the relations of the graph. We now have a look at how such decomposition can be created and how automatisms might help. We identified the steps for a decomposition as described in the recursive Algorithm 1. The algorithm takes as input the concept that is return decomposition subject to the decomposition. As a successful decomposition will always build a graph, the semantic primes are the termination criterion for the recursion. The Algorithm 1 reads as follows:Line 1 initializes the semantic graph which we will build up during this algorithm and which represents the at the end. Line 2 to 28 represents the recursive function which is called on all decomposed concepts. This function adds the decomposition to the semantic graph initialized in Line 1. Which is called until the decomposition depth is reached or all concepts have been decomposed into semantic primes. We will build a hierarchical structure made up of concepts also referred to as lexical units. Those concepts include a lexical representation, the textual representation of a lexeme and a decomposition. Line 3 checks if the concepts have been already decomposed or if the decomposition depth is reached. The decomposition depth is a parameter of the decomposition, which restricts the decomposition to an amount of relations to which the decomposition extends. The second part stops the decomposition of decomposing the same concepts over and over again. Additionally, the decomposition stops here, if a synonym of the concept has been decomposed previously. This is because if a synonym has been decomposed previously, its synonyms are added to the decomposition as well. Thus this synonym, which is supposed to get decomposed now, is already part of the decomposition and is not decomposed again. Line 4 takes the concepts to decompose and normalizes them. Here the inflection is removed, revealing the stem of a concept. Furthermore a concept includes all its inflections (all concepts which can be created by applying grammatical forms to a concept like 'eating', 'ate', 'eaten'), all lexical paradigms for this concept (all concepts rooting from the same word stem like to 'dine', 'dinner') and all sub-categorization frames (like the valence which is the amount of parameters like 'ask', 'ask X', 'ask X for Y'). We remove this kind of inflection because we are interested in the concepts described by a word, not its relation to other words. We can integrate syntactic information into the graph by adding syntax relations and nodes. For this reduction, we use the linguistic process of Lemmatization.2 The function Normalization in Algorithm 1 Line 4 hides this normalization of a concept. Line 5 gets all the relations of the concept from the used dictionaries. This means we are looking through all our dictionaries and look up all the semantic relations we can find and remember them for later processing. Line 6 likewise looks up the definitions of the concept in all available dictionaries. Lines 8 to 10 check whether the concept itself is a semantic prime. If this is the case, the prime is added to the decomposition, the decomposition is finished for this concept and returned. This hides technical optimizations like that we check for synonyms of primes as well to make the search a bit broader. At the same time, we simplified the stop word removal here. Stop words represent words which can be ignored, taken from natural language processing theory BID25. These are mostly words with little semantic meaning like, e.g.,'a','an' or'the'. Those nodes are removed and are not further decomposed. Lines 11 to 18 handle the relation of the concept we are decomposing. Here all relations are added to the decomposition as a relation between concepts. Then all concepts which are connected by those relations are recursively decomposed. Lines 19 to 25 decompose the definitions. Each definition is a list of concepts which get decomposed again. The definition is connected to the definiendum via a "definition" relations. This Marker Passing algorithm is a generalization of the algorithm described by F. Crestani (Crestani 1997, Figure 5, p. 461). Crestani describes the Marker Passing in four steps: Pre-adjustment, spreading, post-adjustment and termination condition evaluation. This is quite general and can in inaccurate interpretations of the algorithm. Consequently, we introduce a more precise description of the algorithm by breaking the activation down into multiple steps without losing generality. Crestani's algorithm is based on the following principle: Starting from a start activation, a concept has a threshold (seen as an upper limit of activation in a node to decide if the node is activated), with each incoming activation the activation level of the node builds up. If the threshold is reached, the node is selected as activated and is spreading in the next spreading step. This means that the node passes all its markers on to its neighbors. This step is repeated until a termination condition is reached BID2 M ← afterSend(M, srcC) 14: end for 15: return M Algorithm 2 describes our extension of the spreading activation algorithm of Crestani BID2.The algorithm defines two maps pulse out and pulse in, which hold the markers passed during a pulse. The function " ∪ ←" describes the insertion of the remaining tuple into the appropriate set of, e.g., all markers of the current pulse (in contrast to replacing them). In line 3 of Algorithm 2 we add the of the outF unction in the form of (Relation × Edge × M arkers) * to the map pulse out, where for each edge the markers are sorted. We separate the Algorithm 2 into four blocks each consisting of one loop: Lines 2 -4: All passing concepts activate their outfunction and the to the current pulse stored in the variable pulse out. This is the input for the edge functions of the appropriate relations of the next step. Lines 5 -9: Each marker passed by the current pulse is given to the appropriate relation it is passed to, and this relation activates its edge-function. The of the edgefunction is added to the pulse which is used as input for the in-functions of the targets of this relations. Lines 10 -12: Concepts that are targets of the relations passing markers are given the markers passed to them and activate their in-function. Lines 13 -15: The after-send-function is activated to fix the markers on the source concepts if needed. With this marker passing, we then can set start markers e.g. onto the start state and the goal state and analyze how the pass to services. As depicted in Figure 1 our goal-oriented heuristic is composed of two parts: The closeness to the start state and the closeness to the goal state. DISPLAYFORM0 Here the set S denotes all services, S 0 denotes the start state and G describes the goal state. U F is the usefulness of the given state w.r.t. the goal, and E its executability, and w 1,2 are their weights. The marker information and the detailed parameters, like termination condition or weight configurations of the marker passing algorithms, can be found in BID4.We selected these two measurements for our heuristic, because if we leave out one of the aspects two effects happen:Goal overcommitment If we only look at the usefulness the services fulfilling subgoals will be tried first. Even though the probability of them being at the end of the plan is higher. This means if we are not talking about a planning problem which is trivial because all service in the plan is independent, that one or more service needs to be executed to enable this useful service. By only looking at the usefulness the search will always try those service first. Low hanging fruits If we only look the executability, services which are executable are always tried first. This is good at the beginning of the planning process because reaching the goal is less probable at this point. But the more service is executed, the more service preconditions become enabled and all of them are tried first. To reach the goal then becomes like a breadth-first-search, where all service possible are tried before we get closer to the goal. This argumentation leads us to introduce two weights w 1,2 which can be adapted depending on how far the search has progressed towards the goal. In the beginning, the executability should, in consequence, be highly weighted and become less important the closer to the goal the search progresses. This is inverse for the usefulness. Both parts use the same kind of mechanism to check whether a fact is fulfilled (in the precondition or effect of a service) which is given by the goal or start state. To check this fulfillment we extract the predicates from the service precondition (effects) and the start (goal) axioms and their arguments and compare them. The comparison is done in two ways: first, for the predicates, we separate the word included in the predicate e.g. "IsBookedFor(Flight x, Customer c)" become the predicate "is booked for". Since this resembles a sentence, the sentence similarity measure d sen is used to compare predicates with a sentence similarity measure based on the semantic similarity presented in BID4. Second we compare the arguments with the same semantic similarity measure d sem. The of those both similarity is then aggregated. The aggregation is done in the following way:The main difference of the sentence similarity measure d sen to the semantic distance measure d sem is that the marker carries the information from which sentence they started out from. This information is used in the interpretation of the markers in the way described in Equation 3. DISPLAYFORM1 Where s 1 and s 2 are two lists of concepts (the sentences) and Ξ represents the activation of the set of concepts in both sentences. StartMarkers is the set of initially placed markers. The is the marked graph after the marker passing has finished passing markers. |s 1 | + |s 2 | − |s 1 ∪ s 2 | |s 1 | + |s 2 | The AvgActivation gets the average of the activation of all markers of all concepts that are activated by both sentences. In Equation 3 we calculate the concepts present in both sentences plus the average activation of the concepts activated by markers of both sentences normalized by the total activation present after the initial marking. The ing similarity is then again normalized to the interval from 0 to 1.With Equation 3 we calculate the similarity of two sentences by calculating the ratio of equivalent words in both sentences in Ξ. This means that if the two sentences are equivalent, then Ξ becomes 1. To this ratio, we add the normalized average activation of all concepts activated by markers of both sentences. This captures that if concepts are semantically closer together, then more markers of both sentences, carrying more activation exist. In extreme cases, this value can become larger than one, which makes a normalization to the interval of zero to one of the necessary. These measures, d sen and d sem, are used to calculate the Distance between two states, which in turn is used to calculate the usefulness UF and executability E. DISPLAYFORM0 Here we sum up the maximal weighted name and arguments match over the set of subgoals. Thus our service gets a usefulness of 1 if it fulfills all subgoals. The argument matching follows the same structure: The arguments of the goal predicate are matched to the arguments of the effect predicate. The maximal match is then summed up over all predicates of the goal. This means we are collecting all predicates of the goal which are semantically close to the effect of the services. The semantic closeness is calculated in parts: with the predicate name and its arguments. These two parts are weighted to define their influence on the overall heuristic . This is done because we want to maximize the argument matches and maximize the number of effects the service can fulfill for the given goal. The d sen and the argument matches are then weighted with weights w prd, w arg determining how much influence the different similarities have in the overall . For the comparison of predicates two kinds of similarity measures are used:Predicate comparison is done with the sentence similarity measure d sen based on the semantic similarity measure proposed in BID4. This is because a predicate mostly describes verbs and their form, direction and if they are passive or active. Our example "is booked for" is a typical use case for a predicate in ontologies. Argument comparison is done with the semantic similarity measure d sem proposed in see BID4. Here the arguments are compared and the maximum is summed up. This makes the argument comparison independent of argument order. The is then normalized w.r.t. the number of predicates in the goal. Here we can see that H(S, Z) becomes 1.0 for a service fulfilling all predicates of the goal, and 0.0 when none of the effects fulfill anything from the goal. The executability (E) then is calculated with a similar measure then the usefulness: DISPLAYFORM1 The same measure as for the usefulness can be applied for the evaluation of services to the extent of their fulfilled preconditions, meaning that service with unsatisfied precondition will be avoided. The effect of this heuristic is that the search algorithm of our planner now will try all services fulfilling parts of the goal first if the weight on the usefulness is high, and try executable service first if the weight of executability is higher. Now we need an example problem to test our heuristic upon. This will be discussed in the Section 4 next. There is one dataset which uses semantic service descriptions we could find, called the Secure Agent-Based Pervasive Computing (Scallop) domain 3. It has a collection of 21 services in the domain of health, air travel, and medical transport, and includes a set of 24 ontologies building up the domain model. Here, the scenario in focus is the medical transport of victims to a medical facility mostly by airplane and some ground transport. For technical compatibility we have translated the services and the domain to OWL-S 1.2.The problem to be solved is to transport a patient from one place to a hospital. This includes finding the nearest airport from which one can fly to an airport that is close to a hospital. In addition, transport from and to the airports has to be organized. To book a flight, a flight account has to be created, a fitting flight has to be found, and the flight needs to be booked. After the flight a ground transport to the nearest hospital needs to be organized. Having done this, the goal of our example problem is reached. We created a start and end state of this domain in which a victim has to be transported to a medical destination. The goal state consist of 64 axiomatic facts which need to be fulfilled to reach the goal state. Since the start and goal states have a large overlap, FIG0 shows both states combined. Here the red states are states from the start state, and blue nodes are from the goal state; the gray nodes are found in both states. 4 For readability, the top-most "owl:Thing" class is omitted, and subclass relations are shown as dashed lines and individuals with dotted lines. The overall domain is modeled in an ontology, describing the individuals and their relations. This ontology is too big to be displayed as a whole here, but the interested reader can download it from the Scallop project website. The initial state has declares multiple facts about the domain, e.g. the transports available in our domain, where we see that Vehicle Transports and Flights are the two possible transports. We want to be at a certain location at a certain time. All the modeling around those facts is necessary because we have to make sure all individual are available so that we can evaluate a potential execution of a service and with that reason upon the effect of this service. The goal state consist of the information we want to see fulfilled. We start out by declaring the individuals we need in FIG0 with the blue nodes. This models our'patient zero' who needs the "RequiredTreatmenat" that can be provided at a certain hospital. As part of the goal we state that we wish the flight to be booked for "Patient 0", who owns the credit card the flight can be booked with and that the flight arrives at our target destination at the desired time. Since we need a valid account to book a flight, we want that "Patient 0" has a personal account to transport our patient and to book the relevant flights and transports from and to the airports. Next we model the individuals that are needed to reach our destination, as well as the departure and arrival airport. When starting to plan, the start and goal state already overlap with 57 out of 64 axioms in the goal state, thus the plan to be made has to fulfill seven more axioms to reach 3 http://www.dfki.de/scallops/ 4 A node not appearing in the goal does not have to mean that that node is to be deleted, but just that it is not relevant for the goal. the goal state. The optimal plan for this problem includes 4 steps: two requests for flight information for departure and arrival time, creating a flight account, and booking the flights. The planning problem, with its 21 services, which have continuous inputs, like dates, has an infinite search space. Since the goal state is specified as a set of OWL axioms, there is a multitude of states which subsume our goal state. All of them are considered a success. The next section will elaborate on the of our planner with the different heuristics. The evaluation is run using the data set described in Section 4. To measure the performance, we count the extended nodes during the search for the goal state. The'gold standard' (described in Section 2) for general purpose heuristics is still a Greedy heuristic (Pearl 1985), which checks the overlap of the effects of a service with the facts wanted in the goal state, always selecting the service with the highest overlap. Additionally we compare our with the Uniform Cost distribution, where each execution of a service is calculated to cost 1 abstract cost measure. With that admissible heuristic, A * finds an optimal solution. The Random heuristic has been added to the comparison as a baseline. TAB3 shows the average of ten runs. The four columns of TAB3 describe the mean µ and standard derivation σ of the experiment of the steps and time. A step here is an extended state during the search for the goal state. Thus less looked at states means a more directed search and less effort. Column µ steps indicates how many states the search had to extend on average to find the goal state, and column µ time describes how much time one search has taken on average in seconds. The average has been created over 10 runs.5 All heuristics are tested on the same start and goal state, thus there is no difference in the planning problem state space the search had to traverse. The variation of the performance is rooted in the random selection of services if two services are ranked with the same usefulness. The greedy heuristic is seen as the'gold standard' for general purpose planning. It is used in most best-first search problems. As the name suggest, the services with the most overlap with the goal (the "best" ones) are tried first. This can lead to the that in each state of the state space, the same "best" services are tried over and over again. The evaluation of the goal overlap does take about as much time as the semantic heuristic, as can be seen by comparing how many steps on average are looked at, and how much time is spent. Here we have an average step calculation time of 6.0 seconds for greedy, which is close to the 5.2 seconds the marker passing heuristic spends on each state. The standard derivation from the mean can be explained with the random selection of services with equal usefulness. The Uniform Cost function with the same usefulness for all services creates a breadth-first search in an A * algorithm. Here we can see that the standard derivation of the steps reduces to 2.6. This is because we are looking at all service in one state before progressing to the next; it does not matter in which order we look at the services. The Random cost heuristic is the baseline to beat: If we are worse then this, our heuristic creates more confusion then it guides the search. In addition, this is used to ground the overall speed of our heuristics. This can be done, because the creation of a random number does almost consume as few resources as the Uniform Cost heuristic and does not give any information to the search. Here the standard derivation of steps is less then one, because the random heuristic does not speed up the search. As expected, the random heuristic has to look at the most state in the search space (see TAB3) which is at a mean of 53.9. Of course, the relation between looking at more state and taking more time correlates. Thus, while the evaluation of the precondition and the instantiation of the effect take time, a uniform cost heuristic (which has no cost to calculate the heuristic) is still inefficient because each explored service takes some time. The use of semantics in the marker passing heuristic reduces the standard derivation from the mean, which means that we gather more useful information then by just comparing the overlap with the goal. The standard derivation from the mean is due to the random selection of services with the same heuristic value, which means that the heuristic is still not precise enough. This might change if grounded actions would be analyzed. The problems used in academia are mostly formalized in PDDL with few semantics, e.g. input and output parameter type class hierarchies. Additionally, the problems are made'hard' by scaling the problem up, e.g. by extending the 15-puzzle to an n-puzzle. In those toy domains, the services available are domain specific and are mostly necessary to solve the problem. Thus, the planning task is not a task of using the right services but rather to bring them in the right order. In contrast, in the general planning problem, the right services have to be selected to establish a domain. This domain includes the relevant services that help the agent to reach its goal. This concludes our experiments. We will now take a step back and look at our with some mental distance. Since the heuristic always returns a value between zero and one, it seems still admissible because until the goal is reached, at least one facts remains unsatisfied and thus at least one service need to be executed. In addition the optimal selection of the weights w 1 and w 2 has to be analyzed further in future work. The heuristic seems to benefit of using semantic information, but it remains to be shown that the effort describing the services with additional semantic description is worth the effort. Further, we plan to use the executability as cost and adopt the goal by removing fulfilled subgoals for the calculation of the heuristic. Also, we are currently implementing a fastforward planner by removing the reasoning for consistent states during the search.

represent this text to find a much much longer article about it with say hundreds-thousands of words

Describing a semantic heuristics which builds upon an OWL-S service description and uses word and sentence distance measures to evaluate the usefulness of services for a given goal.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

Using saturated cost partitioning to select patterns is preferable to all existing pattern selection algorithms.


Question: Represent the text to find a very short summary (1-2 sentences max)

Question: In colored graphs, node classes are often associated with either their neighbors class or with information not incorporated in the graph associated with each node. We here propose that node classes are also associated with topological features of the nodes. We use this association to improve Graph machine learning in general and specifically, Graph Convolutional Networks (GCN). First, we show that even in the absence of any external information on nodes, a good accuracy can be obtained on the prediction of the node class using either topological features, or using the neighbors class as an input to a GCN. This accuracy is slightly less than the one that can be obtained using content based GCN. Secondly, we show that explicitly adding the topology as an input to the GCN does not improve the accuracy when combined with external information on nodes. However, adding an additional adjacency matrix with edges between distant nodes with similar topology to the GCN does significantly improve its accuracy, leading to better than all state of the art methods in multiple datasets. One of the central assumptions in node classification tasks is that neighboring nodes have similar classes (; ; b;). This has been extensively used in node classification tasks (; a). Such approaches are now often denoted as graph neural networks (i.e. machine learning where the input is a graph/network) (; ;). Four main approaches have been proposed to take advantage of a graph in machine learning: • Regularize the output requiring that neighboring nodes either in the graph or in its projection have similar classes. • Use the graph to propagate labels and learn the best propagation. • Use the graph to project the nodes to real valued vectors and use those for supervised or unsupervised learning. • Use Graph Convolutional Networks (GCN) for convolutions on the input of a node and its neighbors. Regularization or graph partitioning methods include among others partitioning the graphs based on the eigenvalues of the Laplacian (assuming that nodes with the same partition have similar classes). The Laplacian of a graph is: L = D −A, where D is a diagonal matrix, with the sum of each row in the diagonal and A is the adjacency matrix. This Laplacian is often weighted by multiplying it on the left and the right by D to normalize for the degree . Other works have used variants of this idea, each using smoothness and graph distance differently . An alternative approach is to use quadratic penalty with fixed labels for seed nodes (; a). Multiple diffusion and information propagation models have also been proposed either through explicit diffusion, or through the projection of nodes into real valued vectors . For example, DeepWalk , where a truncated random walk is performed on nodes. It then uses these sentences as an input to skipgram to compute a projection of each word into R N, maximizing the sentence probability. also uses random walks combined with negative sampling. uses a translation of subgraphs to hash functions for a similar task in the context of molecule classifications. A very similar approach was presented by (Node2Vec) by projecting nodes minimizing the distance of neighbored nodes in a truncated random walk. The DNGR model uses random walk to compute the mutual information between points (the PPMI-positive pointwise mutual information), and then a SVD decomposition to project into space. PPMI was used for word representations in and is a sparse high dimensional representation. Another possible approach is the projection of the graph (often using the Laplacian eigenvectors), and the usage of the projection for classification (and not only for a smoothness based regularization), where either the graph itself is used (in such a case, the eigenvectors themselves are used) or an input to the graph is used. In such a case, a convolution with these eigenvectors was used . A Multi-Dimensional-Scaling (MDS) projection of the points in the graphs was also used for a similar goal . Alternative approaches were inspired again by word embedding methods such as word2vec. These methods use the graph to define a context in relation to which the node embedding is constructed. When the data includes only the graph, the embeddings are used as features and fed into existing predictors . These methods can be thought of as propagating features rather than labels. defines local features to translate each node to a features vector and use those to predict classes. Recently, Kipfs and collaborators, in a seminal work, proposed a simplification of spectral based convolutions , and instead use a two-layer approach, which can be summarized as: whereÃ is a normalized adjacency matrix: They test their work on multiple graphs with labeled nodes including CiteSeer, Cora, Pubmed, and Nell. Convolution approaches can also be used with the graph as a filter on the input. Most such convolutions are spectral (use the Laplacian eigenvectors). However, recent methods are based on random filters. Those include among others: which defines predetermined convolutions with powers of the adjacency matrix and then combines these powers using learned weights to maximize the classification precision of either the full graph or the classification of nodes. provide a multi-level graph convolution with pooling, where at each stage nodes are merged into clusters using agglomerative clustering methods, and combine it with a pooling method to represent the different resolution of images. This has been extended (; to different convolutional kernels (mainly spectral, but also diffusion-based kernels) and the classification of images, using ImageNet (see for a detailed review of all convolution methods). Vandergheynst and collaborators mainly use polynomial convolution in the spectral domain. Similar formalisms were used to study not only single snapshots, but also with recurrent networks time series of graphs, mainly again in image analysis . Over the last 3 years, over 1,500 extensions and applications of GCN have been published in combination with many other learning methods, including among many others combinations of GCN with recurrent neural networks , with GANs and with active learning . GCNs capture dependencies of nodes' features. However, current techniques consider only local neighborhoods. Thus, long-range dependencies can only be captured when these operations are applied repeatedly, propagating signals progressively through the data. To catch long-range dependencies, proposed stacking multiple layers of GCN. While this is possible in theory, it has never been successfully applied. In practice, GCN models work the best with 2-3 layers (; ; Veličković et al., 2017; ;). proposed using NGCN train multiple instances of GCNs over different distances regions. While this led to good performance, it is highly inefficient and does not scale to long distances (as the number of models scales linearly with the desired length). However, long range correlations can be obtained from a different direction. Recently, a correlation has been shown between the topological attributes (e.g. degree, centrality, clustering coefficient...) of nodes and their class (; ; ; ;). Inspired by the improvement of non-local operations in a variety of tasks in the field of computer vision , we propose a novel non-local operation for GCN, based on the topology of the graph. Our operation is generic and can be implemented with every GCN to capture long-range dependencies, allowing information propagation to distant nodes. There are several advantages of using non-local operations: (a) In contrast to the standard local convolution layer, non-local operations capture long-range dependencies directly by computing interactions between any two nodes, regardless of their positional distance; (b) As we show in experiments, non-local operations are efficient and achieve their best even with only a few layers; (c) Finally, our non-local convolution can be easily combined with other graph convolution techniques (e.g. GCN, GAT). We here propose the following contributions of nodes topology to graph-based machine learning. First, we show that in the absence of external information, node topology can be used to predict the class of nodes using a feed-forward network. The topology of a node is represented by a vector of attributes of each node, including among others, its degree, the frequency of different sub-graphs around it and its centrality. We then show that this can be translated to GCN through an input representing the number of first and second neighbors belonging to each class in the training set. Finally, we show in the context of GCN, that it is better to add an additional adjacency matrix representing the similarity between node topologies to the GCN than actually adding the topology of the nodes as an input. GCN and Graph Attention Networks (GAT) with this additional adjacency matrix produce accuracies better than all state of the art methods on the Cora, Pubmed, and CiteSeer Datasets. , we used four well-known citation network datasets: PubMed, CiteSeer and CORA , as well as the extended version of CORA from Bojchevski & Günnemann, denoted as CORA-Full, and two co-authorship networks: Coauthor CS, Coauthor Physics. Descriptions of these datasets, as well as statistics, can be found in Appendix A.1. We used the standard GCN model developed by or the GAT (Veličković et al., 2017). Each GCN layer is defined as in Eq. 1, whereÃ is defined above, X n is the input from the previous layer, and W n are the weights of the current layer. In GAT, each layer may contain multiple heads. A GAT head is a linear combination of nodes' features followed by a non linear function: where h j is a set of features for node j, W is a weight matrix, σ is a non linear function, and α i,j are the normalized attention coefficients. Attention coefficients are calculated for each pair of connected nodes to be: where a is a single layer feed forward network, and W is weight matrix. The extensions we propose to the model come from either changing the input of the model or from alteringÃ. The following modifications were considered: • Topology based GCN (T-GCN). We extend graph convolution operation to propagate information through distant neighbors with similar topological features. We construct a dual graph with the same nodes as the original graph, but different edges representing the topological similarity of nodes. Nodes with similar topology are connected with an undirected edge. There are many ways to construct those topological edges. Here we chose to present each node as a R N vector of topological attributes and connect each node to its k most similar nodes (see Appendix A.3 for the full description of attributes used to define the topology of a node). The T-GCN includes two GCN layers performed simultaneously on the input (external features). One GCN uses the regular adjacency matrix. The second uses the dual graph. These two outputs are then concatenated to serve as an input for the next layer (typically, a standard GCN on the original graph). The network's structure is illustrated at Fig 1. • Topology based GAT (T-GAT). The same as T-GCN, with GAT layers instead of GCN layers (Eq. 2 instead of Eq.1). We have also tested the following two alternative methods to use the topology. However, both produce lower accuracies than the standard GCN. • Asymmetric GCN (A-GCN). We incorporate the direction of directed networks by taking the adjacency matrix (asymmetric in directed graph) and concatenate its transpose to it creating a 2n × n matrix: The dimension of the output of each layer is:, which in turn is passed to the next layer following a rearrangement of the output by splitting and concatenating it to change dimensions from -2N × O n to N × 2O n. For more details about the parameters see Appendix A.2. Multiple inputs were tested in these configurations as detailed in Appendix A.3. • Combined GCN (C-GCN): This model includes two input types: a topology features matrix and an external features matrix (in the Cora and Citeseer case, the bag-of-words features). First, we pass the data matrix through a GCN layer, which leads to a 2n × L 1 output. The two inputs (topology and external features) are then concatenated following a rearrangement of the processed data matrix by splitting in dimension 0 and concatenating in the dimension 12n × L 1 → n × 2L 1. Following the concatenation, an n × (2L 1 + T) matrix is obtained, which is passed forward to the A-GCN layers. The following layers are as above in the A-GCN. For more details about the parameters see Appendix A.2. Multiple inputs were tested in these configurations as detailed in appendix A.3. As proposed by and Veličković et al., we used one set of hyper-parameters for Cora, and used the hyper-parameters optimized for Pubmed for all other networks: For the Cora dataset we used the following parameters. In the T-GCN, we used 1 hidden layer of size 32 for each graph (original and dual). For the T-GAT we chose 16 internal nodes for the regular and 8 internal nodes for the dual graph. We also chose 8 heads (8 independent attention mechanisms, see Veličković et al. for more details) for both operations at the first layer, and 1 head for the last layer (same as the original GAT). For the other datasets, we used the optimal parameters found for PubMed: 1 hidden layer with a size of 64 + 16 for T-GCN, and 16 + 16 features for T-GAT. We used 16 heads on the original operation, 8 heads at the dual operation at the first layer, and 8 heads at the last layer (same as the original GAT). The first layer activation function was ReLU for T-GCN, and TanH for T-GAT (except for T-GAT for Cora which we used also ReLU). SoftMax was performed on the last layer of all models. Note that for T-GAT, the external features were normalized and GAT heads were concatenated on the first layer, and averaged on the last layer (same as the original GAT). See a summarized table of all parameters in Appendix A.2. To test that neighbor class and the node self-topology (as shall be further defined) are correlated with the node class, we performed two tests. We first computed the relative frequency of classes in neighbors, given the class of the node: p(neighbor has class i | current node has class j) (Fig 2 lower plots). In the absence of correlations, one would expect a flat value, while an absolute correlation would produce an identity matrix. In the Cora or Citeseer networks, the mass of the diagonal is 60 % of the mass (compared with an expected 15 %). To test for the relation between node topology and class, we computed the average value of multiple topological features (Appendix A.3) for nodes with a given class (in the current context manuscripts belonging to a certain field). Except for the betweenness centrality, the only topological features correlated with class were 3 and 4 small scale motif frequencies. To test for that, a Kruskal Wallis non-parametric test was performed to test for the relation between the node class (manuscript field) and the distribution of features, Over sixty different small scale motifs are associated with the node class (Fig 2 upper plots). To test that topology and information propagation can be used to classify node classes, we introduced the topological features above, and the number of neighbors belonging to the training set with a given class as input to a Feed Forward-Network (see Appendix A.4). These two types of information by themselves can be used to classify the class of nodes quite precisely (see Appendix 4). Upper left plot Average of each topological feature for nodes belonging to a given class. All values were stacked and normalized to 1. An equal distribution would produce equally divided columns. The upper group are 4 node subgraph frequencies, and the lower group (separated by empty row) are 3 node motifs. Except for the centrality, no other node feature had a a significant KW p value after multiple measurements correction. Lower plots -Correlation of CiteSeer and Cora manuscript class with the neighboring manuscript class. The color in each row represents the fraction of nodes neighboring a given class that belong to all possible classes. One can clearly see the dominating diagonal representing the fact that neighboring nodes tend to have the same color. Since the main topological factors correlated with the class are small scale motif, we propose an alternative method to test their contribution to the classification. In order to avoid the explicit computation of sub graph frequencies, which can be computationally expensive, an indirect computation of the topology can be proposed. A simple way to describe such local features is though operations on products of the adjacency matrix. For example, the number of triangles i → j, i → k and j → k, are the and combination of A and A * A (Fig 4). Thus, instead of explicitly computing such features, one can use as input to the FFN combinations of these products on a one hot representation of the training set class. Formally, let us define for node i, the vector v i, where v j i is the number of neighbors of node i that are in the training set and are of class j, and V is the matrix of all vectors v i. To this, we add a last constant value to the vector, as shall be explained. We then use different combinations of A × V, A T × V, A × A T × V etc. as inputs to an FFN (see Methods). When these products are applied to the last row (a constant value), they simply count sub-graphs. However, when multiplied by the other component, the sub-graphs composed of a specific class are counted (Fig 4 B). The accuracy obtained for such products outperforms the only explicit topological measures, or information propagation (Fig 4 upper plot). Given the correlation between the node's topology and class, we tested whether adding the topology by itself or as an additional input to the BOW would increase the prediction accuracy of the node class for the Cora and Citeseer networks. We have tested both symmetric and asymmetric GCN (see description above), and either topological input by itself or combined with the BOW. Within the topological input, we tested three alternatives: the number of first and second neighbors belonging to each class in the training set, the topological features of each node or their combination. Under review as a conference paper at ICLR 2020 Figure 3: Average accuracy obtained with 50 random splits as a function of training size (starting from 5%). Validation and test sets were split evenly. Upper plots are for Cora and lower plots for CiteSeer. In the right plots, we compared our asymmetric model with the standard GCN in the absence of external information. Different types of topological features were tested as input (see Results). One can clearly see that the neighbors feature is the best input (performed almost as good as external information), and the standard GCN is better than the A-GCN. In the left plots, we compared the standard GCN with T-GCN and C-GCN (with different types of topological features), where the input is BOW. The C-GCN does not perform well with all three types, and the T-GCN is always equal to or better than the standard GCN. As expected, over all tested training set fractions, the models with the BOW outperform the ones without it. Within the models without BOW, ignoring the edge direction, and using only the number of neighbors in each class is better than any other combination. Moreover, combining BOW with topology as input only reduces the accuracy (Fig 3). Still, it is interesting to note that the accuracy without the BOW is not so far from the accuracy with the BOW at high training set fraction (Fig 3). For example, the accuracy for Cora with train size of 55% is 87.9% with BOW, and 85% with only neighbors features as input. Since adding topology as an input did not improve the accuracy, we tested whether using the topology to propagate information between distant nodes based on the similarity of topological attributes helps. We compared our topology-based convolution (T-GCN) to state of the art models on multiple standard real-world networks (see Models and Data). In Cora, CiteSeer, and PubMed networks, we used the previously published split of train-test , and for Cora-Full and the co-authorship networks we took 20 labeled nodes per class as the training set, 30 nodes per class as the validation set, and the rest as the test set (same as). We repeated the experiment 100 times and report the average accuracy over all trials. In each trial, we split the data randomly (except for the standard splits where the train is fixed). Parameters for all models can be found in Appendix A.2. All baselines were copied from the related paper. Furthermore, we report the of GCN and GAT (Veličković et al., 2017) using our own implementation, written using pytorch (which produce slightly lower than the published for the same architecture). to fairly evaluate GAT, we used 500 epochs for training. These are the base implementation used for T-GCN and T-GAT. The summary of the is presented in table 1 One can clearly see that the T-GCN and T-GAT outperform all other models in Cora, Pubmed, and Physics (Table 1). Moreover, the current comparison was performed using the original split in Table 1: Results -average accuracy over 100 trials. For Cora, CiteSeer, and PubMed we used the standard splits as. For Cora-Full, Physics, and CS we used 20 × #Classes random nodes as train, and 30 × #Classes for validation . For Cora-Full and Physics we reported the T-GAT of only 20 and 10 trials accordingly since they were tested on the CPU Method CiteSeer Cora PubMed Physics CS Cora Full DCNN 71.1 81.3 ----Planetoid 64.7 75.7 77.2 ---ChebNet 69 CiteSeer, Cora, and Pubmed. We have tested random splits to check the performance of the T-GCN. Indeed, in the random split, the T-GCN always has a higher accuracy than the GCN (Fig 3) even in the CiteSeer dataset, with a difference that can reach up to 3.3%. Convolution methods to aggregate information from multiple distances are among the leading image classification methods. In images, most of these convolutions are symmetric and sometimes isotropic around each point. However, in contrast with images that are typically overlaid on a 2D lattice, graphs have a complex topology. This topology is highly informative of the properties of nodes and edges , and can thus be used to classify their classes. This complex topology can be combined with convolutional networks to improve their accuracy. In undirected graphs, the topology can often be captured by a distance maintaining projection into R N, using unsupervised methods, such as the classical MDS , or supervised methods to minimize the distance between nodes with similar classes in the training set . In directed graphs, a more complex topology emerges from the asymmetry between incoming and outgoing edges (i.e., the distance between node i and node j differs from the distance between node j and node i), creating a distribution of subgraphs around each node often denoted sub-graph motifs . Such motifs have been reported to be associated with both single node/edge attributes as well as whole-graph attributes . We have here shown that in a manuscript assignment task, the topology around each node is indeed associated with the manuscript class. In order to combine topological information with information propagation, we proposed a novel GCN where the fraction of second neighbors belonging to each class is used as an input, and the class of the node is compared to the softmax output of the node. This method can indeed produce a high classification accuracy, but less than the one obtained using a BOW input. Moreover, explicitly combining the topology as an input with the BOW reduces the accuracy. However, using the topology to add new edges between nodes with similar topological features actually significantly improves performance in most studied datasets. This suggests that the topology is better used to correlate between the class of distant nodes than to be actually used as an input. The presented here are a combination of information propagation and topology-based classification. While each of these two elements was previously reported, their combination into a single coherent GCN based classifier provides a novel content independent method to classify nodes. With the current ever-increasing concerns about privacy, new content independent methods for node classification become essential. The citation networks contain scientific papers divided into classes by their research field. Edges describe citations in the data set. BOW is also available to describe each publication in the dataset. BOW can be either a 1/0 vector or a TF/IDF weighted word vector for PubMed. Coauthor CS and Coauthor Physics are co-authorship graphs based on the Microsoft Academic Graph from the KDD Cup 2016 challenge 3. Here, nodes are authors, that are connected by an edge if they co-authored a paper, node features represent paper keywords for each authors papers, and class labels indicate the most active fields of study for each author. Here are the parameters used for each of the models. For T-GCN and T-GAT the parameters were optimized for PubMed (as observed by and Veličković et al. ) except for Cora data for set which we used slightly different parameters (denotes as T-GCN Cora, and T-GAT Cora). The parameters are summarized in Table 3. In all models, the activation function of the last layer is Softmax. The activation function of the first layer is presented In Table 3. Hidden size X+Y means size of X for the original GCN operator and Y for the GCN on the dual graph. The two outputs are concatenated to a total of X+Y size. GAT heads X,Y,Z means X heads for the original GAT operator, and Y heads for the GAT on the dual graph. Z is the number of heads in the last layer. See Models And Data for more details. Our goal is to use the graph structure to classify node colors. Hence, we compute features that are only based on the graph structure, ignoring any external content associated with each node. Those features are used to convert nodes into the appropriate network attribute vector (NAV) . Following is a list of attributes used. Note that other attributes may have been used with probably similar . • Degree -number of in and out (in case of directed graphs) edges. •. Betweenness is a centrality measure of a vertex. It is defined by the numbers of shortest paths from all vertices that pass through the vertex. • Closeness Centrality. Closeness is a centrality measure of a vertex. It is defined as the average length of the shortest path between the vertex and all other vertices in the graph. • Distance distribution. We compute the distribution of distances from each node to all other nodes using a Djekstra algorithm , and then use the first and second moments of this distribution. • Flow . We define the flow measure of a node as the ratio between the undirected and directed distances between the node and all other nodes. • Attraction . Attraction Basin hierarchy is the comparison between the weighted fraction of the network that can be reached from each vertex with the weighted fraction of the network from which the vertex can be reached. • Motifs Network motifs are small connected sub-graphs. We use an extension of the algorithm to calculate motifs. For each node, we compute the frequency of each motif where this node participates. • K-cores . A K-core is a maximal subgraph that contains vertices of degree k or more. Equivalently, it is the subgraph of G formed by repeatedly deleting all nodes of degree less than k. • Louvain community detection algorithm. The Louvain algorithm is a community detection algorithm. The algorithm works by optimization of modularity, which has a scale value between -1 to 1. Neighbors Feature. We also used a feature of the training set labels. We summed for each node the number of neighbors belonging to each class in the training set. The sum was represented as a vector of sums (e.g. if a node has 10 neighbors, only three of which are in the training set, with two belonging to the first class, and one belonging to the third class, the vector would be [2, 0, 1, ..]). The sum was performed on first and second neighbors producing a vector of twice the number of classes. In a directed graph we calculated two features, one for In neighbors and the second for Out neighbors. The in Figure 2 are produced through a feed-forward network with two internal layers of sizes 300 and 100 internal nodes and an output layer with the number of possible classifications (7 and 6 in CiteSeer and Cora, respectively). The nonlinearities were Relus in the internal layers and a linear function in the output layer. An L2 regularization of 0.2 was used for all layers and a 10% drop in our rate. The loss function was a categorical cross-entropy as implemented in Keras with a TensorFlow backend., when each node is classified using the distribution of classes in its neighbors (dashed), or when products of the neighbors and different combinations of the adjacency matrix are used (full line). Lower plot. Example of sub-graph frequency through adjacency matrix products. Given the graph plotted on the left, the appropriate adjacency matrix (A) and a division of the nodes into green and blue nodes, one can count the number of feed-forward motifs (x → y and

Represent this text to find a much much longer article about it with say hundreds-thousands of words

Topology-Based Graph Convolutional Network (GCN)

Represent the TLDR for finding the corresponding scientific article

We propose an efficient, provable and data independent method for network compression via neural pruning using coresets of neurons -- a novel construction proposed in this paper.


Question: represent the text to find a very short summary (1-2 sentences max)

Question: Flies and mice are species separated by 600 million years of evolution, yet have evolved olfactory systems that share many similarities in their anatomic and functional organization. What functions do these shared anatomical and functional features serve, and are they optimal for odor sensing? In this study, we address the optimality of evolutionary design in olfactory circuits by studying artificial neural networks trained to sense odors. We found that artificial neural networks quantitatively recapitulate structures inherent in the olfactory system, including the formation of glomeruli onto a compression layer and sparse and random connectivity onto an expansion layer. Finally, we offer theoretical justifications for each . Our work offers a framework to explain the evolutionary convergence of olfactory circuits, and gives insight and logic into the anatomic and functional structure of the olfactory system. Over the last two decades, both the anatomic and functional organization of the fly and mouse olfactory systems have been mapped to excruciating detail, affording knowledge of how odors are processed along the entirety of olfactory pathway. In both model organisms, the layout of the olfactory system is two layers deep and comprises of a compression layer and an expansion layer. Olfactory perception is initiated by the recognition of odorants by a large repertoire of receptors in the sensory epithelium . In fruit flies, individual olfactory receptor neurons (ORNs) express only one of 50 different olfactory receptors (ORs), and all neurons (10 on average) that express the same receptor converge with precision onto a unique set of 2-4 projection neurons (PNs) through a specialized structure known as an olfactory glomerulus . This layout establishes a one-to-one mapping between ORs and PNs. Information is then conveyed to an expansion layer of 2,500 Kenyon Cells (KCs) through sparse and random connectivity to support a high dimensional representation of odor information before it is classified by 20 read-out neurons, the mushroom body output neurons (MBONs). Experiments reveal that synaptic plasticity at the KC-MBON synapse is necessary and causal in odor learning. The only major differences between the circuits of mice and flies appear to be numerical. Whereas the fly olfactory system consists of 50 ORs, 50 glomeruli, and 2500 KCs, the mouse olfactory system consists of 1500 ORs, 1500 glomeruli, and 1 million piriform neurons. The fact that evolution has evolved to hardwire the same architecture in flies, mice, and multiple other organisms suggests that such an architecture is optimal for the general task of odor sensing. Although we have a detailed anatomy of the olfactory system in both flies and mice, it is unclear why certain features are optimal for odor sensing. In particular, 1) why does every ORN express a single OR, 2) why is information preserved through a one-to-one mapping between ORs and PNs, and 3) why is connectivity onto the expansion layer sparse and random ? To study optimal circuit design, we use a goal-driven approach to train an artificial neural network to classify odors and then analyze the anatomical and functional structures that emerge after training. This approach has recently been used to study the functional profiles of the ventral stream in visual object processing . The simplicity of the fly olfactory circuit and the exhaustive knowledge that we have of its anatomy provides constraints that can be used to gain insight into evolutionary design. We use a simple odor classification task that maps odors to classes (100 total, 4 shown, Figure 1a). To generate this dataset, we first generated 100 odor prototypes. Each prototype activates 50 olfactory receptors, and the activation of each receptor is sampled independently from a uniform distribution drawn between 0 and 1. For all odors, the ground-truth class is set to be its closest odor prototype as measured by Euclidean distance in the ORN space. The training set consists of 1 million odors, and the validation consists of 8192 odors, and each odor is sampled the same way as the odor prototypes. Such a task mimics the evolutionary drive for organisms to distinguish between dissimilar odors and to generalize between similar odors. The networks connections are modified during training to classify odors according to this exact mapping (Figure 1b). We used standard training techniques based on stochastic gradient descent. This form of training can be thought of as evolving a circuit architecture in silico. We modeled the olfactory system as a layered feed-forward network with each layer corresponding to 500 ORNs, 50 PNs, 2500 KCs, and 100 class neurons, in this order. Connections between each layer represent synaptic strengths, and the activities of neurons in our network represent firing rates. The 500 ORNs are subdivided into 10 ORN duplicates that ex- press the same OR for all 50 unique ORs. MBONs, the readout neurons of the olfactory system, are simplified into class neurons, and each outcome is represented by the activation of a single class neuron. For simplicity, this architecture omits several biological structures, including interneurons, a realistic readout, and an additional pathway. Connections between neurons in each layer have no initial structure. After training, the network performs odor classification with 83% accuracy. We analyzed the network structure and observed that connections between ORNs and PNs appear to resemble the convergence of ORNs expressing the same OR onto glomeruli in the olfactory system. All ORNs that express the same OR project onto a unique PN, and PNs sample from a single type of ORN (Figure 1c). We quantify the extent that PNs pool from a single type of ORNs using a simple but stringent metric called Glomeruli Score (GloScore). A maximal GloScore of 1 means that PNs sample exclusively from ORNs expressing the same OR, whereas a score of 0 means that PNs sample from multiple ORs with the same connection weight. During training, the GloScore of the model ORN-PN connectivity quickly approaches 1. Every KC is initially connected to all 50 PNs. While glomeruli are forming, we observe that these connections sparsen (Figure 1d). We measured the KC input degree (the number of strong PN connections for each KC neuron) after training and observed an average input degree of 7. This is striking as it matches the input degree derived from exhaustive anatomical tracing studies (Figure 1e,f). Theories suggest that random connectivity support highdimensional representations that enhance the ability of downstream readout neurons to learn associations, much as in theories of cerebellar cortex . We also observe that KCs evolved to randomly sample from PNs after training. We thus calculated the average correlation between the input connections of every pair of KCs, and found that correlations quickly decrease to approach that of randomly shuffled connectivity during training. A more stringent analysis revealed that KCs sample uniformly from all PNs and KCs in our network are not connected to any preferential pairs of PNs, similar to what has observed in the wiring diagram of fruit flies . To ensure the robustness of these , we performed an extensive hyperparameters sweep, exploring the impact of learning rate, the number of KCs, dropout rate, batch normalization, and input noise. The quantitative are robust to all but one hyperparameter. Decreasing the learning rate eventually leads to a non-separation of weak and strong weights. Therefore, we used the highest learning rate that allows for classification accuracy to exceed 50% (chance is 1%). Moreover, these were also robust to the addition of biologically realistic network motifs, such as normalization in the PN and KC layers. In a network with exclusively excitatory connections, glomeruli emerge (Figure 1c). In a network with both excitatory and inhibitory connections, PNs mix from multiple ORs (Figure 2a). Irrespective of mixing, accuracy is maintained (Figure 2b). Moreover, the average Pearson's correlation between the activities of different PNs with and without mixing were close to zero, suggesting that connections to PNs evolved to preserve odor information. However, by minimizing correlation the network loses out on an opportunity to increase the dimensionality of its odor representation. We hypothesize that a network must preserve information if information can be expanded downstream. Conversely, a network with far less than 2500 KCs cannot expand and should bias the PN layer to mix ORs and increase dimensionality. We thus trained networks with variable numbers of KCs while keeping the numbers of ORNs and PNs fixed at 500 and 50, respectively. Indeed, as we decreased the number of KCs, GloScore decreases as PNs begin to sample from multiple ORs (Figure 2c). We further note that there is only a marginal benefit in performance with more than 2500 KCs (Figure 2d), which is the number of KCs within each hemisphere of the mushroom body. We further predict that PNs will mix if given the resources to do so. We varied the number of PNs while keeping the numbers of ORNs and KCs fixed at 500 and 2500, respectively. Indeed, when the number of PNs exceeds the number of unique OR types, we observe that excess PNs receive mixed OR input (Figure 2e). Moreover, classification accuracy saturates, implying that having more than 50 PNs does not aid task performance (Figure 2f). When there are less than 50 PNs, information flow is bottlenecked, and mixing occurs to ensure that all ORs are represented (Figure 2e). Together, these argue that the glomeruli representation is only optimal when expansion occurs downstream. We further added input noise sampled from a normal distribution onto ORNs, and found that the formation of glomeruli is minimally dependent on input noise. When trained on the standard classification task, the input degree of the expansion layer (KCs) settles at K ≈ 7 (Figure 1f). To predict the connectivity of arbitrarily-sized olfactory systems, we trained networks with various numbers of ORs, ranging from 50 to 400. For each network, we quantified the input degree of the expansion layer after training (Figure 3a, plus signs). The can be well fitted by a power law function, K ∼ N 0.71, where K is the optimal expansion layer input degree for a system of N unique ORs. This prediction is consistent with experimental estimates from anatomical studies in mouse, K ≈ 40 − 100, N ≈ 1, 000 and fruit fly K ≈ 7, N ≈ 50 , (Figure 3a, x signs). (Figure 3a, gray), and more specifically, K ≈ 8 for N = 1, 000, which is far lower than expected. We hypothesized that an input degree ofK ∼ N 0.71 did not emerge solely to minimize classification loss. Instead, this power-law maximizes robustness to perturbations in connection weights. This hypothesis is inspired by findings that stochastic gradient descent, due to its stochastic nature, not only maximizes classification performance but also finds flat minima where the loss changes more gradually when connection weights are perturbed . This robustness can be interpreted both as robustness against variability in synaptic transmission and also against variability between individuals of the same species. To quantify robustness, we measured the average angle between odor representations in the expansion layer before (y) and after perturbation (y + ∆) to connection weights (Figure 3b). The perturbation angle is proportional to the amount of mis-classification that happens as a of weight perturbation, so a smaller angle corresponds to a more robust network. When N=50 and K explicitly varied from 1 to 30, we found that the perturbation angle is indeed minimized at around K = 7 (Figure 3c), in agreement with our hypothesis. For a network with N unique ORs, we can numerically search for the optimal K that maximizes robustness (minimizes perturbation angle). We found that the optimal K derived from maximal robustness (Figure 3d, red line) matches closely with from direct training of networks of various N (Figure 3d, plus signs). While the We trained artificial neural networks using stochastic gradient descent to classify odors. We found that glomeruli emerged in PN layer, and sparse random connectivity emerges in the PN to KC connections. We then explored the sufficient conditions that enabled these features to emerge. We found that the formation of glomeruli did not depend on input noise but rather on the existence of an expansion layer downstream. In addition, we found that an expansion layer with a synaptic degree of 7 endows the olfactory system with robustness by allowing for large tolerances in synaptic efficacies without affecting task performance. Our work offers a framework to explain the Optimal K predicted by maximal weight robustness (red) and direct training (plus signs). The line is a power-law fit of the red dots.

Represent the TLDR for finding the corresponding scientific article

Artificial neural networks evolved the same structures present in the olfactory systems of flies and mice after being trained to classify odors

Represent the short summary for finding the corresponding scientific article (long)

A theoretical analysis of a new class of RNNs, trained on neuroscience tasks, allows us to identify the role of dynamical dimensionality and cell classes in neural computations.