Question: Represent the scientific article to find a short TLDR.

Question: Current classical planners are very successful in finding (non-optimal) plans, even for large planning instances. To do so, most planners rely on a preprocessing stage that computes a grounded representation of the task. Whenever the grounded task is too big to be generated (i.e., whenever this preprocess fails) the instance cannot even be tackled by the actual planner. To address this issue, we introduce a partial grounding approach that grounds only a projection of the task, when complete grounding is not feasible. We propose a guiding mechanism that, for a given domain, identifies the parts of a task that are relevant to find a plan by using off-the-shelf machine learning methods. Our empirical evaluation attests that the approach is capable of solving planning instances that are too big to be fully grounded. Given a model of the environment, classical planning attempts to find a sequence of actions that lead from an initial state to a state that satisfies a set of goals. Planning models are typically described in the Planning Domain Definition Language (PDDL) BID16 ) in terms of predicates and action schemas with arguments that can be instantiated with a set of objects. However, most planners work on a grounded representation without free variables, like STRIPS BID4 or FDR BID1. Grounding is the process of translating a task in the lifted (PDDL) representation to a grounded representation. It requires to compute all valid instantiations that assign objects to the arguments of predicates and action parameters, even though only a small fraction of these instantiations might be necessary to solve the task. The size of the grounded task is exponential in the number of arguments in predicates and action schemas. Although this is constant for all tasks of a given domain, and grounding can be done in polynomial time, it may still be prohibitive when the number of objects is large and/or some predicates or actions have many parameters. The success of planners like FF BID9 or LAMA BID24 in finding plans for large planning tasks is undeniable. However, since most planners rely on grounding for solving a task, they fail without even starting the search for a plan whenever an instance cannot be grounded, making grounding a bottleneck for the success of satisficing planners. Grounding is particularly challenging in open multi-task environments, where the planning task is automatically generated with all available objects even if only a few of them are relevant to achieve the goals. For example, in robotics, the planning task may contain all objects with which the robot may interact even if they are not needed BID13 ). In network-security environments, like the one modeled in the Caldera domain BID17, the planning task may contain all details about the network. However, to the best of our knowledge, no method exists that attempts to focus the grounding on relevant parts of the task. We propose partial grounding, where, instead of instantiating the full planning task, we focus on the parts that are required to find a plan. The approach is sound -if a plan is found for the partially grounded task then it is a valid plan for the original task -but incomplete -the partially grounded task will only be solvable if the operators in at least one plan have been grounded. To do so, we give priority to operators that we deem more relevant to achieve the goal. Inspired by relational learning approaches to domain control knowledge (e.g., BID31, BID3, BID11), we use machine learning methods to predict the probability that a given operator belongs to a plan. We learn from small training instances, and generalize to larger ones by using relational features in standard classification and regression algorithms (e.g., BID12). As an alternative model, we also experiment with relational trees to learn the probabilities BID18.Empirical show that our learning models can predict which operators are relevant with high accuracy in several domains, leading to a very strong reduction of task size when grounding and solving huge tasks. Throughout the paper, we assume for simplicity that tasks are specified in the STRIPS subset of PDDL BID4. Our algorithms and implementation, however, are directly applicable to a larger subset of PDDL containing ADL expressions BID20.A lifted (PDDL) task Π PDDL is a tuple (P, A, Σ C, Σ O, I, G) where P is a set of (first-order) atomic predicates, A is a set of action schemas, DISPLAYFORM0 is a non-empty set of objects consisting of constants Σ C, and non-constant objects Σ O, I is the initial state, and G is the goal. Predicates and action schemas have parameters. We denote individual parameters with x, y, z and sets of parameters with X, Y, Z. An action schema a[X] is a triple (pre(a), add(a), del(a)), consisting of preconditions, add list, and delete list, all of which are subsets of P, possibly pre-instantiated with objects from Σ C, such that X is the set of variables that appear in pre(a) ∪ add(a) ∪ del(a). I and G are subsets of P, instantiated with objects from Σ.A lifted task Π PDDL can be divided into two parts: the domain specification (P, A, Σ C) which is common to all instances of the domain, and the problem specification DISPLAYFORM1 DISPLAYFORM2 The plan is optimal if its length is minimal among all plans for Π.We define the delete-relaxation of a task Π as the task Π DISPLAYFORM3, we can compute the corresponding STRIPS task Π by instantiating the predicates and action schemas with the objects in Σ. Then, F contains a fact for each possible assignment of objects in Σ to the arguments of each predicate P [X] ∈ P, and O contains an operator for each possible assignment of objects in Σ to each action schema a[X] ∈ A. In practice, we do not enumerate all possible assignments of objects in Σ to the arguments in facts and action schemas. Instead, only those facts and operators are instantiated that are delete-relaxed reachable from the initial state BID7 ). We base our method on the grounding algorithm of Fast Downward BID6. To ground a planning task, this algorithm performs a fix-point computation similar to the computation of relaxed planning graphs BID2, where a queue is initialized with the facts in the initial state and in each iteration one element of the queue is popped and processed. If the element is a fact, then those operators of which all preconditions have already been processed (are reached) are added to the queue. If the element is an operator, all its add effects are pushed to the queue. The algorithm terminates when the queue is empty. Then, all processed facts and operators are delete-relaxed reachable from the initial state. For simplicity, the algorithm we describe here considers only STRIPS tasks but it can be adapted to support other PDDL features like negative preconditions or Algorithm 1: Partial Grounding. DISPLAYFORM0 conditional effects as it is done by BID7.Algorithm 1 shows details of our approach. The main difference with respect to the approach by BID7 is that the algorithm can stop before the queue is empty, and operators are instantiated in a particular order. For these two choice points we suggest an approach that aims at minimizing the size of the partially grounded task, while keeping it solvable. That said, our main focus is the operator ordering, and we only consider a simple stopping condition. Stopping condition. Typical grounding approaches terminate only when the queue is empty, meaning that all (deleterelaxed) reachable facts and operators have been grounded. In partial grounding, we allow the algorithm to stop earlier. Intuitively, this is a good idea because most planning tasks have short plans, usually in the order of at most a few hundred operators, compared to possibly millions of grounded operators. Hence, if the correct operators are selected, partial grounding can potentially stop much sooner than complete grounding. The key issue is how to decide when the probability of finding a plan using the so-far grounded operators is sufficient. Consider the following claims: 1. The grounded task is delete-relaxed solvable iff G ⊆ F. Item 1 provides a necessary condition for the task to be relaxed-solvable, so grounding should continue at least until G ⊆ F. But this is not sufficient, as it does not guarantee that a plan can be found for the non-relaxed task. Item 2 provides an obvious, but difficult to predict, condition for success. In this work, we consider only a simple stopping condition. To maximize the probability of the task being solvable, it is desirable to ground as many operators as possible. The main constraint on the number of operators to ground are the resources (time and memory) that can be spent on grounding. For that reason, one may want to continue grounding while these resources are not compromised 1. We provide a constant N op as a parameter, an estimate on the number of operators that can be grounded given the available resources, and let the algorithm continue as long as |O| ≤ N op.If not all actions are grounded, the ing grounded task is a partial representation of the PDDL input and the overall planning process of grounding and finding a plan for the grounded task is incomplete. We implemented a loop around the overall process that incrementally grounds more actions, when finding the partially grounded task unsolvable. This converges to full grounding, ing in a complete planner. Queue order. Standard grounding algorithms extract elements from the queue in an arbitrary order -since all operators are grounded, order does not matter. Our algorithm always grounds all facts that have been added to the queue, giving them preference over operators. This ensures that the effects of all grounded operators are part of the grounded task. After all facts in the queue have been processed, our algorithm picks an operator according to a heuristic criterion, which we will call the priority function. Some simple priority functions include FIFO, LIFO, or random. Since our aim is to ground all operators of a plan, the priority queue should sort operators by their probability of belonging to a plan. To estimate these probabilities, we use machine learning techniques as detailed in the next section. Additionally, one may want to increase the diversity of selected operators to avoid being misguided by a bias in the estimated probabilities. We consider a simple round robin (RR) criterion, which classifies all operators in the queue by the action schema they belong to, and chooses an operator from a different action schema in each iteration. RR works in combination with a priority function that is used to select which instantiation of a given action schema should be grounded next. We define a novelty criterion as a non-trivial priority function that is not based on learning, inspired by novelty pruning techniques that have successfully been applied in classical planning BID14. During search, the novelty of a state is defined as the minimum number m for which the state contains a set of facts of size m, that is not part of any previously generated state. This can be used to prune states with a novelty < k. We adapt the definition of novelty to operators in the grounding process as follows. Let Σ be the set of objects, a[X] an action schema, and O the set of already grounded operators corresponding to all instantiations of a[X]. Let σ = {(x 1, σ 1),..., (x k, σ k)} be an assignment of objects in Σ to parameters X instantiating an operator o, such that o ∈ O. Then, the novelty of o is defined as the number of assignments (x i, σ i) such that there does not exist an operator o ∈ O where x i got assigned σ i. In the grounding we will prioritize operators with a higher novelty, which are likely to generate facts that have not been grounded yet.1 While search can benefit from grounding less operators, an orthogonal pruning method that uses full information of the grounded task, can be employed at that stage (e. g. BID8). To guide the grounding process towards operators that are relevant to solve the task, we use a priority queue that gives preference to more promising operators. We use a priority function f: O → that estimates whether operators are useful or not. Ideally, we want to assign 1 to operators in an optimal plan and 0 to the rest, so that the number of grounded operators is minimal. We approximate this by assigning to each operator a number between 0 and 1 that estimates the probability that the operator belongs to an optimal plan for the task. This is challenging, however, due to lack of knowledge about the fully grounded task. We use a learning approach, training a model on small instances of a domain and using it to guide grounding in larger instances. Our training instances need to be small enough to compute the set of operators that belong to any optimal plan for the task. We do this by solving the tasks with a symbolic bidirectional breadth-first search BID29 ) and extracting all operators that belong to an optimal solution. Before grounding, the only information that we have available is the lifted task Π PDDL = (P, A, Σ C, Σ O, I, G). Our training data uses this information, consisting of tuples (I, G, Σ O, o, {0, 1}) for each operator o in a training instance, where o is assigned a value of 1 if it belongs to an optimal solution and 0 otherwise. We formulate our priority functions as a classification task, where we want to order the operators according to our confidence that they belong to the 1 class. To learn a model from this data, we need to characterize the tuple (I, G, Σ O, o) with a set of features. Since training and testing problems have different objects, these features cannot refer to specific objects in Σ O, so learning has to be done at the lifted level. We propose relational rules that connect the objects that have instantiated the action schema to the training sample (I, G, Σ O) to capture meaningful properties of an operator. Because different action schemas have different (numbers of) arguments, the features that characterize them will necessarily be different. Therefore, we train a separate model for each action schema a[X] ∈ A. All these models, however, predict the probability of an operator being in an optimal plan, so the values from two different models are still comparable. We considered two approaches to conduct the learning: inductive relational trees and classification/regression with relational features. Inductive Relational Learning Trees. Inductive Logic Programming (ILP) BID18 ) is a wellknown machine learning approach suitable when the training instances are described in relational logic. ILP has been used, e.g., to learn domain control knowledge for planning (de la BID3 BID11 . We use the Aleph tool BID27 to learn a tree where each inner node is a predicate connecting the parameters of a to-be-grounded operator to the facts in the initial state or goal, to objects referred to in a higher node in the tree, or to a constant. The nodes are evaluated by checking if there exists a predicate instantiated with the given objects in the initial state or goal. A wildcard symbol (" ") indicates that we do not require a particular object, but that any object instantiating the predicate at this position is fine. In FIG0, the left child corresponds to this check evaluating to false and the right child to true. For a given action, the tree is evaluated by checking if there exists an assignment to the free variables in a path from the root to a leaf node, such that all nodes on the path evaluate to the correct truth value. We then take the real value in the leaf node as an estimate of the probability that the operator is part of an optimal plan. This evaluation is akin to a CSP problem, so we need to keep the depth of the trees at bay to have an efficient priority function. FIG0 shows the tree learned for the turn-to action schema in Satellite. In this domain, the goal is to take pictures in different modes. Several satellites are available, each with several instruments that support some of the modes. The actions are switching the instruments on and off, calibrating them, turning the satellite into different directions, and taking images. The turn-to action changes the direction satellite? s is looking at. In this case, the learned tree considers that the operators turning to and from relevant directions are more likely part of an optimal plan than turning away from the goal direction. More concretely, if for a tobe-instantiated operator turn-to(s, from, to) with objects s, from, and to, there is no goal have-image(to,), i. e., taking an image in direction to using any mode, then the operator is deemed not useful by the trained model, it only has a probability of 1% of belonging to an optimal plan. In the opposite case, and if there is a have-image goal in the from direction, but no goal pointing(, from), then the operator is expected to be most useful, with a probability of 38% of being in an optimal plan. This is relevant information to predict the usefulness of turn-to. However, there is some margin of improvement since the initial state is ignored. An alternative is to use relational rules as features for standard classification and regression algorithms BID12. Our features are relational rules where the head is an action schema, and the body consists of a set of goal or initial-state predicates, partially instantiated with the arguments of the action schema in the head of the rule, constant objects in Σ C, or free variables used in other predicates in the rule. This is very similar to a path from root to leaf in the aforementioned relational trees. We generate rules by considering all possible predicates and parameter instantiations with two restrictions. First, to guarantee that the rule takes different values for different instantiations of the action schema, one of the arguments in the first predicate in the body of the rule must be bound to a parameter of the action schema. Second, at least one argument of each predicate after the first one, must be bound to a free variable used in a previously used predicate. This aims at reducing the number of features by avoiding redundant rules that can be described as a conjunction of simpler rules. We assume that, if the conjunction of two rules is relevant for the classification task, the machine learning algorithms will be able to infer this. Most of the generated rules do not provide useful information to predict whether an operator will be part of an optimal plan or not. This is because we brute-force generate all possible rules, including many that do not capture any useful properties. Therefore, it is important to select a subset of relevant features. We do this filtering in two steps. First, we remove all rules that evaluate to the same value in all training instances (e.g., rules that contain goal:predicate in the body will never evaluate to true if predicate is never part of the goal description in that domain). Then, we use attribute selection techniques in order to filter out those features that are not helpful to predict whether the operator is part of an optimal plan. As an example, the most relevant rule generated for the turn-to schema is: This can be read as: "do we have to take images in directions? to and?from in modes that are supported by one of the instruments on board?". This rule surprisingly accurately describes a scenario where turn-to is relevant (and can be complemented with other rules to capture different cases).Given a planning task and an operator, a rule is evaluated by replacing the arguments in the head of the rule by the objects that are used to instantiate the operator and checking if there exists an assignment to the free variables such that the corresponding facts are present in the initial state and goal of the task. Doing so, we generate a feature vector for each grounded action from the training instances with a binary feature for every rule indicating whether the rule evaluates to true for that operator or not. This in a training set where for each operator we get a vector of boolean features (one feature per rule), together with a to-be-predicted class that is 1 if the operator is part of an optimal plan for the task, and 0 if not. On this training set, we can use either classification or regression methods to map each operator to a real number. With classification methods we use the confidence that the model has in the operator belonging to the positive class. In regression, the model directly tries to minimize the error by assigning values to 1 for operators in an optimal plan and 0 to others. It is important to note that it is possible that there are two training examples with the same feature vector, but with different values in the target. In these cases, we merge all training examples with the same feature vector and replace them with a single one that belongs to the 1 class if any of the examples did 2. During grounding, for every operator that is inserted in the queue, we evaluate all rules and call the model to get its priority estimate. To speed-up rule evaluation, we precompute, before grounding, all possible assignments to the arguments of the action schema that satisfy the rule. The computational cost of doing this is exponential in the number of free variables but it was typically negligible for the rules used by our models. We evaluate the relational trees in a similar way. For the evaluation of our partial grounding approach, we adapted the implementation of the "translator" component of the Fast Downward planning system (FD) BID6. The translator parses the given input PDDL files and outputs a fully grounded task in finite-domain representation (FDR) BID1 BID7 ) that corresponds to the PDDL input. Our changes are minimally invasive, only changing the ordering in which actions are handled and the termination condition, as indicated in Algorithm 1. Therefore, none of the changes affect the correctness of the translator, i. e., the generated grounded planning task will always be a proper FDR task. The changes do not affect the performance too much either, except when using a computationally expensive priority function. Experimental Setup. For the evaluation of our technique, we require domains for which instance generators are available to generate a set of diverse instances small enough for training, and the size of the grounded instances grows at least cubically with respect to the parameters of the generator so that we have large instances that are hard to fully ground, for evaluation. We picked four domains that were part of the learning track of the international planning competition (IPC) 2011 (Blocksworld, Depots, Satellite, and TPP), as well as two domains of the deterministic track of IPC'18 (Agricola and Caldera). For all domains, we used the deterministic track IPC instances and a set of 25 large instances that we generated ourselves for the experiments. For the training of the models, we used between 40 and 250 small instances, to get enough training data for each action schema. Since the number of grounded actions per schema varies significantly across domains, we individually adapted the number of training instances. To generate the large instances, we started at roughly the same size as the largest IPC instances, scaling the parameters of the generator linearly when going beyond that. As an example, in Satellite, the biggest IPC instance has around 10 satellites and 20 instruments, which is the size of our smallest instances. In the largest instances that we generated, there are up to 15 satellites and 60 instruments. In Blocksworld, where IPC instances only scale up to 17 blocks, we scale in a different way, starting at 75 blocks and going up to 100, which can still easily be solved by our techniques. Regarding the domains, we used the typed domain encoding of Satellite from the learning track, which simplifies rule generation, but does not semantically change the domain. In Blocksworld, we use the "no-arm" encoding, which age but this ed in slightly worse in most cases. shows a cubic explosion in the grounding, in contrast to the "arm" encoding, where the size of the grounded task is only quadratic in the PDDL description. Beside the static queue orderings, FIFO, LIFO, and novelty-based methods, we experiment with learning-based approaches using classification and regression models. While the former exemplify what is possible without learning, the latter methods aim at grounding only those actions that belong to a plan for a given task. In all cases, we combine the methods with the round robin queue setup (RR).Learning Framework. We report for a logistic regression classifier (LOGR), kernel ridge regression (KRN), linear regression (LINR), and a support vector machine regressor (SVR). While LINR and LOGR learn linear functions to combine the features, differing mostly in the loss function and underlying model that is used, KRN and SVR are capable of learning non-linear functions. We expect nonlinear functions to be useful to combine features, which cannot be done with linear functions. We also report the of the decision trees learned by Aleph. To implement the machine learning algorithms, we use the scikit Python package BID21, which nicely connects to the translator in FD.For feature selection, i. e., to select which rules are useful to predict the probability of an operator to be in a plan, we used the properties included in the trained models. For each feature (rule) contained in the feature vector, the model returns a weight according to its relevance to discriminate the target vector. After experimenting with multiple different models, we decided to use a decision tree regressor to predict rule usefulness, for all trained models. We evaluated our models in isolation on a set of validation instances that are distinct from both our training and testing set, and small enough to compute the set of operators that are part of any optimal plan. FIG2 shows the outcome of the priority function learned by LOGR in Blocksworld. The Table 1: Number of instances solved by the baseline with full grounding (Base), and incremental grounding with static action orderings (FIFO, LIFO, random (RND)), novelty-based ordering, and several learning models (see text). "RR" indicates that we use a separate priority queue for each action schema, taking turns over the schemas. Best coverage highlighted in bold face.bars indicate the number of operators across all validation instances that got a priority in a given interval, highlighting operators from optimal plans in a different color. The plots nicely illustrate that the priority function works very well for the action schemas move-t-to-b and move-b-to-t, where it is able to distinguish "optimal" from "non-optimal" operators. The distinction works not so well for move-b-to-b, but in general gives a significantly lower priority to this action schema. Another important observation is that the total number of grounded move-b-to-b actions is much higher than that of the other two action schemas. Projecting these observations to the grounding process, we expect the model to work well when used in a single priority queue, since it will prioritize move-t-to-b and move-b-to-t (which are the only ones needed to solve any Blocksworld instance) over move-b-to-b (which is only needed to optimally solve a task). On the validation set, grounding all operators with a priority of roughly > 0.6 suffices to solve the tasks, pruning all move-b-to-b operators and most non-optimal ones of the other schemas. RR in contrast will ground an equal number of all action schemas, including many unnecessary operators. These conjectures are well-supported by the plots in Figure 3.When working with machine learning techniques, there is always the risk of overfitting. In our case the on the training set are very similar to those on the validation set shown in FIG2, suggesting that overfitting is not an issue in our setup. The in other domains are similar. Incremental Grounding. We use the incremental approach, where the first iteration grounds a given task until the goal is found to be relaxed-reachable. The left half of Figure 3 shows detailed information on how many operators need to be grounded until this is achieved for different priority functions. We discuss details later. In case this first iteration fails, i. e., the partial task is not solvable, we set a minimum number of operators to be grounded in the next iteration by using an increment of 10 000 operators. This strategy does not aim to maximize coverage but rather to find out what is the minimum number of operators that need to be grounded to solve a task for each priority function (with a granularity of 10 000 operators). The number of operators that were necessary to actually solve a given instance is illustrated in the right half of Figure 3.For all configurations, after grounding, we run the first iteration of the LAMA planner BID23, a good standard configuration for satisficing planning that is well integrated in FD. We also use LAMA's first iteration as a baseline on a fully grounded task, with runtime and memory limits for the entire process of 30 minutes and 4GB. All other methods perform incremental grounding using their respective priority function. We allowed for a total of 5 hours and 4GB for the incremental grounding, while restricting the search part to only 10 minutes per iteration to keep the overall runtime of the experiments manageable. We show coverage, i. e., number of instances solved, in Table 1, with the time and memory limits mentioned in the previous subsection. The left part of the table considers instances as solved when the overall incremental grounding process (including finding a plan) finished within 30 min. In the right part, we approximate the that could be achieved with a perfect stopping condition by considering an instance as solved if the last iteration, i. e., the successful grounding and search, finished within 30 min. The baseline (Base) can still fully ground most instances except in Caldera and TPP, but fails to solve most of the large instances with up to 9 million operators. We scaled instances in this way so that a comparison of the number of grounded operators to the baseline is possible; further scaling would make full grounding impossible. The table nicely shows that the incremental grounding approach, where several iterations of partial grounding and search are performed (remember that we only allow 10min for the search), significantly outperforms the baseline, even when considering an overall time limit of 30min. In fact, all instances in Blocksworld can be solved in less than 10s by LOGR. This illustrates the power of our approach when the learned model captures the important features of a domain. The static orderings typically perform worse than the baseline, only the novelty-based ordering can solve more instances in Blocksworld, and in Caldera when using RR.The plots in Figure 3 shed further light on the number of operators when (leftmost two columns) the goal is relaxed reachable in the first iteration and (rightmost two columns) the number of operators needed to actually solve the task. Each data point corresponds to a planning instance, with the number of ground actions of a fully grounded task on the xaxis. The y-axis shows the number of grounded actions for several priority functions, including FIFO (LIFO in TPP), novelty, the learned model that has the highest reduction on the number of grounded actions, and Aleph. In general, the models capture the features of most domains quite accurately, leading to a substantial reduction in the size of the grounded task, and still being able to find a solution. The plots show that our models obtain a very strong reduction of the number of operators in the partially grounded task in Agricola, Blocksworld, and Caldera; some reduction (one order of magnitude) in Depots, and Satellite, and a small reduction in TPP. In terms of the size of the partially grounded tasks, different learning models perform best in different domains, and there is not a clear winner. In comparison, the baselines FIFO, LIFO, and Random do not significantly reduce the size of the grounded task in most cases, with a few exceptions like LIFO in TPP and FIFO in Caldera. The novelty criterion is often the best method among those without learning. Grounding a delete-relaxed reachable task with less operators is often beneficial, but may be detrimental for the coverage if the task is unsolvable, as happens for the Novelty method in Agricola or the LIFO method in TPP. This also explains why the learning models with highest reductions in some domains (e.g. LOGR in Agricola) are not always the same as the ones with highest coverage. The RR queue mechanism often grounds more operators before reaching the delete-relaxed goal but this makes the first iteration solvable more often leading to more stable . The exception is Aleph, where RR has the opposite effect, making the partially grounded tasks unsolvable. Some approaches in the literature try to alleviate the grounding problem, e. g. by avoiding grounding facts and operators unreachable from the initial state BID7 ), reformulating the PDDL description by splitting action schemas with many parameters BID0, or using symmetries to avoid redundant work during the grounding process BID26.Lifted planning approaches that skip grounding entirely BID22 have lost popularity due to the advantages of grounding to speed-up the search and allow for more informative heuristics which are not easy to compute in a lifted level. BID25 adapted the delete-relaxation heuristic BID9 to the lifted level. This is related to our partial grounding approach since their relaxed plan extraction mechanism can be used to obtain a grounded task where the goal is relaxed reachable, and it could be used to enhance the novelty and learning priority functions that we use here. There are many approaches to eliminate irrelevant facts and operators from grounded tasks BID19 BID10 BID5 BID28. The closest to our approach is under-approximation refinement BID8, which also performs search with a subset of operators. However, all these techniques use information from the fully grounded representation to decide on the subset of relevant operators, so are not directly applicable in our setting. The of our learning models (see FIG2 show that applying learning to identify irrelevant operators is a promising avenue for future research. Recently, BID30 introduced a machine learning approach to learn heuristic functions for specific domains. This is similar to our work, in the sense that a heuristic estimate has been learned, though for states in the search, not actions in the grounding. Furthermore, the authors used neural networks instead of our, more classical, models. In this paper, we proposed an approach to partial grounding of planning tasks, to deal with tasks that cannot be fully grounded under the available time and memory resources. Our algorithm heuristically guides the grounding process giving preference to operators that are deemed most relevant for solving the task. To determine which operators are relevant, we train different machine learning models using optimal plans from small instances of the same domain. We consider two approaches, a direct application of relational decision trees, and using relational features with standard classification and regression algorithms. The empirical show the effectiveness of the approach. In most domains, the learned models are able to identify which operators are relevant with high accuracy, helping to reduce the number of grounded operators by several orders of magnitude, and greatly increasing coverage in large instances. Figure 3 : The scatter plots show the number of operators of a fully grounded task on the x-axis. The y-axis shows the number of operators that are needed to make the goal reachable in the grounding (leftmost two columns), and the number of operators that are needed to solve the task (rightmost two columns), for several priority functions.

Represent the short summary for finding the corresponding scientific article (long)

This paper introduces partial grounding to tackle the problem that arises when the full grounding process, i.e., the translation of a PDDL input task into a ground representation like STRIPS, is infeasible due to memory or time constraints.

This work aims to provide quantitative answers to the relative importance of concepts of interest via concept activation vectors (CAV). In particular, this framework enables non-machine learning experts to express concepts of interest and test hypotheses using examples (e.g., a set of pictures that illustrate  the concept). We show that CAV can be learned given a relatively small set of examples. Hypothesis testing with CAV can answer whether a particular concept (e.g., gender) is more important in predicting a given class (e.g., doctor) than other sets of concepts. Interpreting networks with CAV does not require any retraining or modification of the network.


Question: Represent the text to find a very short summary (1-2 sentences max)

Question: In this paper, we introduce a novel method to interpret recurrent neural networks (RNNs), particularly long short-term memory networks (LSTMs) at the cellular level. We propose a systematic pipeline for interpreting individual hidden state dynamics within the network using response characterization methods. The ranked contribution of individual cells to the network's output is computed by analyzing a set of interpretable metrics of their decoupled step and sinusoidal responses. As a , our method is able to uniquely identify neurons with insightful dynamics, quantify relationships between dynamical properties and test accuracy through ablation analysis, and interpret the impact of network capacity on a network's dynamical distribution. Finally, we demonstrate generalizability and scalability of our method by evaluating a series of different benchmark sequential datasets. expose them to defined input signals such as step and sinusoid functions. Through evaluation of output attributes, such as response settling time, phase-shift, and amplitude, we demonstrate that it is possible to predict sub-regions of the network dynamics, rank cells based on their relative contribution to network output, and thus produce reproducible metrics of network interpretability. For example, step response settling time delineates cells with fast and slow response dynamics. In addition, by considering the steady-state value of the cellular step response and the amplitude of the sinusoid response, we are able to identify cells that significantly contribute to a network's decision. We evaluate our methodology on a range of sequential datasets and demonstrate that our algorithms scale to large LSTM networks with millions of parameters. The key contributions of this paper can be summarized as follows:1. Design and implementation of a novel and lightweight algorithm for systematic LSTM interpretation based on response characterization;2. Evaluation of our interpretation method on four sequential datasets including classification and regression tasks; and 3. Detailed interpretation of our trained LSTMs on the single cell scale via distribution and ablation analysis as well as on the network scale via network capacity analysis. First, we discuss related work in Sec. 2 and then introduce the notion of RNNs as dynamic systems in Sec. 3. Sec. 4 presents our algorithm for response characterization and defines the extracted interpretable definitions. Finally, we discuss the interpretations enabled by this analysis in Sec. 5 through an series of experiments, and provide final of this paper in Sec. 6. Deep Neural Networks Interpretability -A number of impactful attempts have been proposed for interpretation of deep networks through feature visualization BID9 BID37 BID36 BID20 BID34 BID6. Feature maps can be empirically interpreted at various scales using neural activation analysis BID27, where the activations of hidden neurons or the hidden-state of these neurons is computed and visualized. Additional approaches try to understand feature maps by evaluating attributions BID32 BID10 BID21 BID35. Feature attribution is commonly performed by computing saliency maps (a linear/non-linear heatmap that quantifies the contribution of every input feature to the final output decision). The contributions of hidden neurons, depending on the desired level of interpretability, can be highlighted at various scales ranging from individual cell level, to the channel and spatial filter space, or even to arbitrary groups of specific neurons BID27. A dimensionality reduction method can also be used to abstract from high dimensional feature maps into a low dimensional latent space representation to qualitatively interpret the most important underlying features BID25 BID5. However, these methods often come with the cost of decreasing cell-level auditability. Richer infrastructures have been recently developed to reason about the network's intrinsic kinetics. LSTMVis BID34, relates the hidden state dynamics patterns of the LSTM networks to similar patterns observed in larger networks to explain an individual cell's functionality. A systematic framework has also been introduced that combines interpretability methodologies across multiple network scales BID27. This enables exploration over various levels of interpretability for deep NNs; however, there is still space to incorporate more techniques, such as robust statistics BID22, information theory approaches BID31, gradients in correlation-domain BID14 and response characterization methods which we address in this paper. Recurrent Neural Networks Interpretability -Visualization of the hidden-state of a fixed-structure RNNs on text and linguistic datasets identifies interpretable cells which have learned to detect certain language syntaxes and semantics BID20 BID34. RNNs have also been shown to learn input-sensitive grammatical functions when their hidden activation patterns were visualized BID18 BID19. Moreover, gradient-based attribution evaluation methods were used to understand the RNN functionality in localizing key words in the text. While these techniques provide rich insight into the dynamics of learned linguistics networks, the interpretation of the network often requires detailed prior knowledge about the data content. Therefore, such methods may face difficulties in terms of generalization to other forms of sequential data such as time-series which we focus on in our study. Another way to build interpretability for RNNs is using the attention mechanism where the network architecture is constrained to attend to a particular parts of the input space. RNNs equipped with an attention mechanism have been successfully applied in image captioning, the fine-alignments in machine translation, and text extraction from documents BID15. Hidden-state visualization is a frequently shared property of all of these approaches in order to effectively understand the internals of the network. Hudson et al. BID17 also introduced Memory, Attention, and Composition (MAC) cells which can be used to design interpretable machine reasoning engines in an end-to-end fashion. MAC is able to perform highly accurate reasoning, iteratively directly from the data. However, application of these modification to arbitrary network architectures is not always possible, and in the case of LSTM specifically, the extension is not possible in the current scope of MAC.Recurrent Neural Networks Dynamics-Rigorous studies of the dynamical systems properties of RNNs, such as their activation function's independence property (IP) BID4, state distinguishability BID2, and observability BID11 BID12 date back to more than two decades. Thorough analyses of how the long term dynamics are learned by the LSTM networks has been conducted in BID16. Gate ablation analysis on the LSTM networks has been performed to understand cell's dynamics BID13 BID8. We introduce the response characterization method, as a novel building block to understand and reason about LSTM hidden state dynamics. In this section, we briefly we recap kinetics of RNNs. We denote the global dynamics of the hidden state values as h l t, with t ∈ {1..T} denoting the time, and l ∈ {1..L} representing the layers of the neural network. A vanilla recurrent neural network (RNN) can be formulated as BID29 BID20: DISPLAYFORM0 where DISPLAYFORM1 shows the weight matrix. h 0 t retains an input vector x t and h L t holds a vector at the last hidden layer, L, that is mapped to an output vector y t which is ultimately the function of all input sequence {x 1, . . ., x T}.RNNs are formulated as control dynamical systems in the form of the following differential equation (For the sake of notation simplicity, we omit the time argument, t): DISPLAYFORM2 where h denotes its internal state (˙ illustrates time-shift or time derivative for the discrete and continuous-time systems, respectively), x stands for the input, and R [n×n], W [n×m] and C [p×n] are real matrices representing recurrent weights, input weights and the output gains, respectively. σ: R → R indicates the activation function. In the continuous setting, σ should be locally Lipschitz (see BID3 for a more detailed discussion). Long short term Memory (LSTM) BID16, are gated-recurrent neural networks architectures specifically designed to tackle the training challenges of RNNs. In addition to memorizing the state representation, they realize three gating mechanisms to read from input (i), write to output (o) and forget what the cell has stored (f). Activity of the cell can be formulated as follows BID13: rise to distinct outputs for two different initial states at which the system is started BID33. Observable systems realize unique internal parameter settings BID4. One can then reason about that parameter setting to interpret the network for a particular input profile. Information flow in LSTM networks carries on by the composition of static and time-varying dynamical behavior. This interleaving of building blocks makes a complex partially-dependent sets of nonlinear dynamics that are hard to analytically formulate and to verify their observability properties As an alternative, in this paper we propose a technique for finding sub-regions of hidden observable dynamics within the network with a quantitative and systematic approach by using response characterization. DISPLAYFORM0 DISPLAYFORM1 In this section, we explore how response characterization techniques can be utilized to perform systematic, quantitative, and interpretable understanding of LSTM networks on both a macro-network and micro-cell scale. By observing the output of the system when fed various baseline inputs, we enable a computational pipeline for reasoning about the dynamics of these hidden units. Figure 1 provides a schematic for our response characterization pipeline. From a trained LSTM network, comprising of M LSTM units, we isolate individual LSTM cells, and characterize their output responses based on a series of interpretable response metrics. We formalize the method as follows:Definition 1 Let G, be a trained LSTM network with M hidden LSTM units. Given the dynamics of the training dataset (number of input/output channels, the main frequency components, the amplitude range of the inputs), we design specific step and sinusoidal inputs to the network, and get the following insights about the dynamics of the network at multi-scale resolutions:• the relative strength or contribution of components within the network;• the reactiveness of components to sudden changes in input; and• the phase alignment of the hidden outputs with respect to the input. Specifically, we analyze the responses of the step input and the sinusoidal input. We use the classic formulations for each of these input signals wherein step: Across a network of LSTM units we can approximate sub-regions of the dynamics of a single cell, u, by extracting the input and recurrent weights corresponding to that individual cell. We then define a sub-system consisting of just that single cell and subsequently feed one of our baseline input signals, x t ∀ t∈{1..T} to observe the corresponding output response, y t. In the following, we define the interpretable response metrics for the given basis input used in this study: DISPLAYFORM0 Definition 2 The initial and final response of the step response signal is the starting and steady state responses of the system respectively, while the response output change represents their relative difference. Response output change or the delta response for short determines the strength of the LSTM unit with a particular parameter setting, in terms of output amplitude. This metric can presumably detect significant contributor units to the network's decision. The settling time of the step response is elapsed time from the instantaneous input change to when the output lies within a 90% threshold window of its final response. Computing the settling time for individual LSTM units enables us to discover "fast units" and "slow units". This leads to the prediction of active cells when responding to a particular input profile. The amplitude and frequency of a cyclic response signal is the difference in output and rate at which the response output periodically cycles. The response frequency,f, is computed by evaluating the highest energy component of the power spectral density:f = arg max S yy (f).The amplitude metric enables us to rank LSTM cells in terms of significant contributions to the output. This criteria is specifically effective in case of trained RNNs on datasets with a cyclic nature. Given a sinusoidal input, phase-shifts and phase variations expressed at the unit's output, can be captured by evaluating the frequency attribute. The correlation of the output response with respect to the input signal is the dot product between the unbiased signals: DISPLAYFORM0 The correlation metric correspondes to the phase-alignments between input and output of the LSTM unit. Systematic computation of each of the above responses metrics for a given LSTM dynamics, enables reasoning on the internal kinetics of that system. Specifically, a given LSTM network can be decomposed into its individual cell components, thus creating many smaller dynamical systems, which can be analyzed according to their individual response characterization metrics. Repeating this process for each of the cells in the entire network creates two scales of dynamic interpretability. Firstly, on the individual cell level within the network to identify those which are inherently exhibiting fast vs slow responses to their input, quantify their relative contribution towards the system as a whole, and even interpret their underlying phase-shift and alignment properties. Secondly, in addition to characterizing responses on the cell level we also analyze the effect of network capacity on the dynamics of the network as a whole. Interpreting hidden model dynamics is not only interesting as a deployment tool but also as a debugging tool to pinpoint possible sources of undesired dynamics within the network. While one can use these response characterization techniques to interpret individual cell dynamics, this analysis can also be done on the aggregate network scale. After computing our response metrics for all decoupled cells independently we then build full distributions over the set of all individual pieces of the network to gain understanding of the dynamics of the network as a whole. This study of the response metric distributions presents another rich representation for reasoning about the dynamics, no longer at a local cellular scale, but now, on the global network scale. In the following section, we provide concrete of our system in practice to interpret the dynamics of trained LSTMs for various sequence modeling tasks. We present our computed metric response characteristics both on the decoupled cellular level as well as the network scale, and provide detailed and interpretable reasoning for these observed dynamics. We chose four benchmark sequential datasets and trained on various sized LSTM networks ranging from 32 to 320 LSTM cell networks. The and analysis presented in this section demonstrate applicability of our algorithms to a wide range of temporal sequence problems and scalability towards deeper network structures. We start by reasoning how our response characterization method can explain the hidden-state dynamics of learned LSTM networks for a sequential MNIST dataset and extend our findings to three additional datasets. We perform an ablation analysis and demonstrate how some of our metrics find cells with significant contributions to the network's decision, across all datasets. We start by training an LSTM network with 64 hidden LSTM cells to classify a sequential MNIST dataset. Inputs to the cells are sequences of length 784 generated by stacking the pixels of the 28 × 28 hand-writing digits, row-wise (cf. FIG1) and the output is the digit classification. Individual LSTM cells were then isolated and their step and sine-response were computed for the attributes defined formerly (cf. Fig. 4). FIG1 ). This interpretation allows us to indicate fast-activated/deactivated neurons at fast and slow phases of a particular input sequence. This is validated in FIG1, where the output state of individual LSTM cells are visually demonstrated when the network receives a sequence of the digit 6. The figure is sorted in respect to the predicted settling time distribution. We observe that fast-cells react to fast-input dynamics almost immediately while slow-cells act in a slightly later phase. This effect becomes clear as you move down the heatmap in FIG1 and observe the time difference from the original activation. The distribution of the delta-response, indicates inhibitory and excitatory dynamics expressed by a 50% ratio (see FIG1). This is confirmed by the input-output correlation criteria, where almost half of the neurons express antagonistic behavior to their respective sine-wave input FIG1. The sine-frequency distribution depicts that almost 90% of the LSTM cells kept the phase, nearly aligned to their respective sine-input, which indicates existence of a linear transformation. A few cells learned to establish a faster frequencies than their inputs, thereby realizing phase-shifting dynamics FIG1. The sine-amplitude distribution in FIG1 demonstrates that the learned LSTM cells realized various amplitudes that are almost linearly increasing. The ones with a high amplitude can be interpreted as those maximally contributing to the network's decision. In the following sections, we investigate the generalization of these effects to other datasets. We trained LSTM networks with 128 hidden cells, for four different temporal datasets: sequential MNIST BID23, S&P 500 stock prices BID1 and CO 2 concentration for the Mauna Laua volcano BID0 forecasting, and classification of protein secondary structure BID30. Learned networks for each dataset are denoted seq-MNIST, Stock-Net, CO 2 -Net and Protein-Net. all five metrics with the network size of 128. It represents the average cell response metric attributes for various datasets and demonstrates the global speed and amplitude of the activity of network in terms of dynamical properties of the response characterization metrics. Fig 3A- E, represents the distributions for the metrics sorted by the value of their specific attribute across all datasets. Cells in Protein-Net realized the fastest dynamics (i.e. smallest settling time) compared to the other networks, while realizing a similar trend to the seq-MNIST (Fig. 3A). The settling time distribution for the LSTM units of CO 2 and Stock-Net depicts cell-groups with similar speed profiles. For instance neurons 52 to 70 in Stock-Net, share the same settling time (Fig. 3A). Sine frequency stays constant for all networks except from some outliers which tend to modify their input-frequency (Fig. 3D). The delta response and the correlation metrics (Fig. 3B and Fig. 3E) both indicate the distribution of the inhibitory and excitatory behavior of individual cells within the network. Except from the Seq-MNIST net, neurons in all networks approximately keep a rate of 44% excitatory and 56% inhibitory dynamics. The high absolute amplitude neurons (at the two tails of Fig. 3C), are foreseen as the significant contributors to the output's decision. We validate this with an ablation analysis subsequently. Moreover, most neurons realize a low absolute delta-response value, for all datasets except for MNIST (Fig. 3B). This is an indication for cells with an equivalent influence on the output accuracy. Sine-amplitude stays invariant for most neurons in Stock and CO 2 -Nets (Fig. 3C). For the seq-MNIST net and Protein-net, this distribution has a gradually increasing trend with weak values. This predicts that individual cells are globally equivalently contributing to the output. To assess the quality of the predictions and interpretations of the provided response characterization metrics, we performed individual cell-ablation analysis on LSTM networks and evaluated the cellimpact on the output accuracy (misclassification rate), for the classification problems and on the output performance (mean absolute error), for the regression problems. We knocked out neurons from trained LSTM networks with 128 neurons. Fig. 4A -H illustrate the performance of the network for individual cell ablations for all four datasets. The gray solid line in each subplot, stands for the predictions of the response metrics. For CO 2 -Net, this confirms that neurons with higher sine amplitude tend to disrupt the output more (Fig 4D). For the same network, the delta response predicted that neurons with high negative or positive value, are more significant in output's prediction. This is clearly illustrated in Fig. 4C. For seq-MNIST-Net, the same held true; neurons with high absolute value of delta response or sine-amplitude reduce the accuracy at the output dramatically (Fig. 4A-B). By analyzing the sine-amplitude and delta-response of Protein-Net, we observe that neurons are equivalently valued and tend to contribute equivalently to the output accuracy. This is verified in the ablation analysis, shown in Fig. 4G and 4H, where the mean-misclassification error rate stays constant for all neural ablations. The absolute value for Stock-Net was also weak in terms of these two metrics, though there were some outliers at the tails of their distribution that predicted dominant neurons. This is clearly notable when comparing the neurons 120 to 128 of Fig. 4F to their prediction (gray line) where the amplitude of the response is maximal. In Fig. 4E ablation experiments for neurons 1 to 40 and 100 to 128 impose higher impact on the overall output. This was also observed in the delta response prediction shown in 4B, since neurons with stronger output response were present at the two tails of the distribution. While we analyzed the response characterization distributions on a cellular level above, in this subsection we focus on the effect of network capacity on observed hidden dynamics of the system on a global scale. Reasoning on this scale allows us to draw on how increasing the expressive capacity of LSTM networks trained on the same dataset can in vastly different learned dynamics. We experimentally vary the capacity by simply adding hidden LSTM cells to our network and retraining on the respective dataset from scratch. The relationship between each response characteristic metric and the network capacity is visualized in FIG4 -E. The trends across datasets are visualized in a single subplot to compare respective trends. One especially interesting of this analysis is the capacity relationship with response amplitude (cf. FIG4). Here we can see that the amplitude response decays roughly proportionally to 1 N, for all datasets, where N is the number of LSTM cells. In other words, we get the intuitive finding that as we increase the number of LSTM cells, the magnitude of each cell's relative contribution needed to make a prediction will subsequently decrease. Yet another key finding of this analysis is that the distribution of settling time is relatively constant across network capacity (cf. FIG4). Intuitively, this means that the network is able to learn the underlying time delay constants represented in the dataset irrespective of the network capacity. One particularly interesting point comes for Protein-Net which exhibits vastly different behavior for both settling time FIG4 ) than the remainder of the datasets. Upon closer inspection, we found that Protein-Net was heavily overfitting with increased capacity. This can be seen as an explanation for the rapid decay in its settling time as the addition of LSTM cells would increase specificity of particular cells and exhibit dynamical properties aligning with effectively memorizing pieces of the training set. In this paper, we proposed a method for response characterization for LSTM networks to predict cell-contributions to the overall decision of a learned network on both the cell and network-level resolution. We further verified and validated our predictions by performing an ablation analysis to identify cell's which contribution heavily to the network's output decision with our simple response characterization method. The ing method establishes a novel building block for interpreting LSTM networks. The LSTM network's dynamic-space is broad and cannot be fully captured by fundamental input sequences. However, our methodology demonstrates that practical sub-regions of dynamics are reachable by response metrics which we use to build a systematic testbench for LSTM interpretability. We have open-sourced our algorithm to encourage other researchers to further explore dynamics of LSTM cells and interpret the kinetics of their sequential models. In the future, we aim to extend our approach to even more data modalities and analyze the training phase of LSTMs to interpret the learning of the converged dynamics presented in this work.7 Acknowledgment

represent this text to find a much much longer article about it with say hundreds-thousands of words

Examples:
Provided: "A state-of-the-art model based on global reasoning for image super-resolution" Match: "Recent image super-resolution(SR) studies leverage very deep convolutional neural networks and the rich hierarchical features they offered, which leads to better reconstruction performance than conventional methods. However, the small receptive fields in the up-sampling and reconstruction process of those models stop them to take full advantage of global contextual information. This causes problems for further performance improvement. In this paper, inspired by image reconstruction principles of human visual system, we propose an image super-resolution global reasoning network (SRGRN) to effectively learn the correlations between different regions of an image, through global reasoning. Specifically, we propose global reasoning up-sampling module (GRUM) and global reasoning reconstruction block (GRRB). They construct a graph model to perform relation reasoning on regions of low resolution (LR) images. They aim to reason the interactions between different regions in the up-sampling and reconstruction process and thus leverage more contextual information to generate accurate details. Our proposed SRGRN are more robust and can handle low resolution images that are corrupted by multiple types of degradation. Extensive experiments on different benchmark data-sets show that our model outperforms other state-of-the-art methods. Also our model is lightweight and consumes less computing power, which makes it very suitable for real life deployment. Image Super-Resolution (SR) aims to reconstruct an accurate high-resolution (HR) image given its low-resolution (LR) counterpart. It is a typical ill-posed problem, since the LR to HR mapping is highly uncertain. In order to solve this problem, a large number of methods have been proposed, including interpolation-based (Zhang & Wu., 2006), reconstruction-based , and learning-based methods (; Peleg & Elad., 2014; ; ; ; ; a;). In recent years, deep learning based methods have achieved outstanding performance in superresolution reconstruction. Some effective residual or dense blocks b;;; Ahn et al.; ) have been proposed to make the network wider and deeper and achieved better . However, they only pay close attention to improving the feature extraction module, ignoring that the upsampling process with smaller receptive fields does not make full use of those extracted features. Small convolution receptive field means that the upsampling process can only perform super-resolution reconstruction based on local feature relationships in LR. As we all know, different features interact with each other, and features which are in different regions have corresponding effects on upsampling and reconstruction of a certain region. That is to say that a lot of information is lost in the process of upsampling and reconstruction due to the limitation of the receptive field, although the network extracts a large number of hierarchical features which are from low frequency to high frequency. Chariker et al. (2016; show that the brain generates the images we see based on a small amount of information observed by the human eye, ranther than acquiring the complete data from the point-by-point scan of the retina. This process of generating an image is similar to a SR process. According to their thought, we add global information in SR reconstruction and propose to use relational reasoning to implement the process that the human visual system reconstructs images with observed global information. In general, extracting global information requires a large receptive field. A large convolution receptive field usually requires stacking a large number of convolutional layers, but this method does not work in the upsampling and reconstruction process. Because this will produce a huge number of parameters. Based on the above analysis, we propose an image super-resolution global reasoning network (SR-GRN) which introduces the global reasoning mechanism to the upsampling module and the reconstruction layer. The model can capture the relationship between disjoint features of the image with a small respective field, thereby fully exploits global information as a reference for upsampling and reconstruction. We mainly propose global reasoning upsampling module (GRUM) and global reasoning reconstruction block (GRRB) as the core structure of the network. GRUM and GRRB first convert the LR feature map into N nodes, each of which not only represents a feature region in the LR image, but also contains the influence of pixels in other regions on this feature. Then they learn the relationship between the nodes and fuse the information of each node in a global scope. After that, GRUM learns the relationship between the channels in each node and amplifies the number of channels for the upsampling process. And then they convert N nodes into pixels with global reasoning information. Finally, GRUM and GRRB complete the upsampling and reconstruction process respectively. In general, our work mainly has the following three contributions: • We propose an image super-resolution global reasoning network (SRGRN) which draws on the idea of image reconstruction principles of human visual system. We mainly focus on the upsampling module and the reconstruction module. The model reconstructs SR images based on relational reasoning in a global scope. • We propose a global reasoning upsampling module (GRUM) and global reasoning reconstruction block (GRRB), which construct a graph model to implement the relational reasoning among the feature regions in an image via 1D and 2D convolution, and finally adds the information obtained by global reasoning to each pixel. It can provide more contextual information to help generate more accurate details. • Our proposed GRUM and GRRB are lightweight, which makes it suitable for real life deployment. More importantly, GRUM and GRRB balance the number of parameters and the reconstruction performance well. They can be easily inserted into other models. Deep CNN for SR and upsampling methods. Deep learning has achieved excellent performance in image super-resolution tasks. For the first time, Dong et al. applied convolutional neural networks to image SR. After this, Kim et al. proposed VDSR and DRCN which introduced residual learning to make the network depth reach 20 layers achieved significant improvement. And then more and more researchers are starting to pay attention to the improvement of the network feature extraction part. proposed EDSR and MDSR, which introduce residual scaling and remove unnecessary modules from the residual block. Concerned that the previous models only adopt the feather of the last CNN, Zhang et al. (2018b) proposed residual dense network to make full use of hierarchical features from each Conv layer. The above and most of the subsequent networks implement the upsampling based on either transposed convolution (; Zeiler & Fergus., 2014) or sub-pixel convolution . Although these models have achieved good , there exists a problem that these upsampling methods have only a small receptive field. This means that upsampling can only take advantage of contextual information within a small area. Recently, researchers propose some new super-resolution upsampling process. LapSRN allows low-resolution images to be directly input into the network for step-by-step amplification. exploit iterative up-and-down sampling layers and propose DBPN. further explore the application of feedback mechanism (weight sharing) in SR and propose the SRFBN. These models have achieved a better reconstruction performance. However, the Conv layers in these upsampling modules still have only a small receptive field. Global reasoning machansim. Recently, graph-based deep learning methods have begun to be widely used to solve relation reasoning. Santoro et al. propose Relation Networks (RN) to solve problems that depend on relational reasoning. propose SIN, which implement a object detection using a graph model for structure inference. Furthermore, model a Global Reasoning unit that consists of five convolutions for image classification, semantic segmentation and video action recognition task. Considering that the human visual system generates images based on the observed global information is also a reasoning process. Moreover, correlation between feature regions can be obtained through relational reasoning, which makes each pixel in the generated SR image jointly determined by the information in a global scope. Therefore, we propose a global reasoning network for SR. We will detail our SRGRN in next section. According to Chariker et al. (2016;, there is only little information transmitted from the retina to the visual cortex, and then the brain will reconstruct the real-world images based on the information received. We regard it as a reasoning process in a global scope. For image SR, the upsampling module constructs SR images base on features in LR images, which is substantially similar to detecting the category of each pixel of a SR image and generating these pixels based on contextual information of corresponding LR image. Due to the limitation of the convolution receptive field, only a small amount of contextual information can be utilized to generate HR images in most other models. This leads that many details in the HR image are not fine. Similarly, the above problem also exists in the reconstruction process. To solve these problems, we simulate the reasoning process that exists in human visual system, and then propose SRGRN to make full use of the contextual information to recover accurate details, which is achieved by constructing graph model and reasoning the relationship between these regions in an image. Figure 1, our SRGRN includes feature extraction part, global reasoning upsample module(GRUM) and global reasoning reconstruction block(GRRB). Let's denote I LR and I SR as the input and output of SRGRN. The feature extraction part can use the relevant architecture of most other models. Here we introduce the feature extraction part of the RDN (b) as an example. where H F EX (·) denotes a series of operations of feature extraction part. As with the previous work , the number of GRUM depends on the scaling factor, The GRUM receives F L as input. F S represents the output of the GRUM. GRUM can be expressed by the following mathematical formula: where H GRU M (·)denotes a series of operations of GRUM. More details about GRUM will be given in Section 3.2. We further conduct global reasoning reconstruction block(GRRB) to utilize the global contextual information to generate the output image. GRRB can be expressed by the following mathematical formula: where H GRRB (·) denotes a series of operations of GRRB. More details about GRRB will be given in Section 3.3. After the above operations, we get the corresponding SR image. In this section, we present details about our proposed global reasoning upsample module(GRUM) in Figure 2. In order to help achieve relation reasoning, we map each image to a graph model We first need a function to construct N nodes in the oriented graph, each of which represents a region in the image. In GRUM, we obtain relationship weights between these pixels through a 2D 1 × 1 convolution, and then convert the input F L into N nodes via element-wise product. The benefits of this approach are mainly reflected in the following aspects: It can not only aggregates a feature region of the input F L into a node, but also dig out the influence of other pixels in the image on this region. This is equivalent to adding global guidance of the image to each node. Using convolution means that these relationship weights are trainable. This process can be expressed by the following mathematical formula: refers to N nodes with C channels. After that, we use the 1D Conv -Leaky ReLu -1D Conv (CLC) structure to implement reasoning and interaction between N nodes in the graph. The parameters in CLC refer to the adjacency matrix of the weighted oriented complete graph, which store the correlations between the nodes. CLC can learn and reason the complex nonlinear relationship between nodes better than only one 1D Conv. We use the following formula to describe the reasoning process between nodes: where Conv(·) and LRelu(·) denote 1D convolution along node-wise and Leaky ReLU operation respectively. And then we use the bottleneck to achieve channel amplification. The bottleneck receives Y N ∈ R N ×C as input and redistributes these channels by modeling the relationship between the channels of each node, amplifying the number of channel to C × r 2, where r is the upscaling factor. The first convolution in bottleneck makes channel C drop toC = C/α, where α represents reduction ratio. Then the second convolution makes the channel dimensionC grow to C × r 2. The bottleneck not only fits the complex relationships between channels better and redistributes channels more accurately, but also greatly reduces the number of parameters compared to the method of utilizing a single convolution. We use the following formula to describe channel amplification: where Y N C ∈ R r 2 C×N refers to the output tensor. In order to expand the resolution by pixelshuffle like ESPCN , we need to retransform the N nodes (r 2 C × N) which have implemented the relational reasoning into a space whose shape is C × H × W. As above, we still learn a function to get a weight matrix whose shape is N ×HW through a 1 × 1 2D convolution, and then normalize the weight matrix along the column with softmax. Finally, Y rC and W P can be obtained through: where C×H×W is a feature map where each pixel is associated with N nodes. W P ∈ R N ×HW is the normalized weight matrix. The value of these weights ranges from 0 to 1. This means that the reconstruction of each pixel is affected by N nodes to varying degrees. Each pixel in the feature map contains information which is generated by global reasoning. After pixelshuffle, the output is multiplied by a parameter γ 1 and added to the upsampling without global reasoning. The initial value of γ 1 is set to 0. As the global reasoning module trains, the network will gradually learn to assign values to the γ 1, thereby fully exploiting global reasoning. This process can be expressed by the following formula: where H P S (·) denotes the operations of pixel shuffle and H U P (·) denotes the operations of subpixel convolution. Finally, F S can be obtained by: Figure 3: Global reasoning reconstruction block (GRRB) architecture As shown in Figure 3, the specific details are similar to GRUM. We also construct a graph model for reconstruction block. In GRRB, we first obtain the relationship weights W RN ∈ R N ×rHrW between pixels of F S by 2D 1x1 convolution, and then aggregate the regions in F S into N nodes by element-wise product. The output of this process Y RW ∈ R N ×C can be formulated as: After that, we use CLC to achieve the relationship reasoning between nodes. Then we exploit the weight matrix W RP ∈ R N ×rHrW = Sof tmax(V d (Conv(F S))) obtained by learning to redistribute the information of N nodes to the pixels. The output of this process Y RrC ∈ R C×rH×rW can be obtained by: where F CLC refers to the operations of CLC. In addition, we apply the idea of residual connection in GRRB, which multiplys the information generated via global reasoning by a parameter γ and then add it to the input feature map. The output is given by: The initial value of γ is set to 0. As the training progresses, the network assigns more weight to γ. Finally, we input the feature map with global reasoning into the two Convs for reconstruction. We can get the final output through: where H RL (·) denotes the operations of two Convs. In our proposed SRGRN, like the previous method , the number of GRUM depends on the scaling factor. For Conv layers with kernel size 3 × 3, we pad zeros to keep size fixed. We set the reduction ratio in bottleneck as α. The number of nodes in the graph model is set to N. We utilize Leaky ReLu with a negative slope of 0.2 as non-linear activation function. The feature extraction part of the network are the same as the RDN (b) settings. The final Conv layer has 1 or 3 output channels, as we output gray or color HR images. 4.1 SETTINGS Datasets and Metrics. We train all our models using 800 training images in the DIV2K (Agustsson & Timofte., 2017) dataset, which contains high-quality 2K images that can be used for image superresolution task. And We use five standard benchmark datasets to evaluate PSNR and SSIM metrics: Set5 , Set14 , B100 , Urban100 and Manga109 (Y.). The SR are evaluated on Y channel of transformed YCbCr space. Degradation Models. In order to make a fair comparison with existing models, bicubic dowmsampling(denoted as BI) is regarded as a standard degradation model. We use it to generate LR images with scaling factor ×2, ×3, and ×4 from ground truth HR images. To fully demonstrate the effectiveness of our model, we also use two other degradation models and conduct special experiments for them. Our second model, we defined it as BD, which blurs HR images with a Gaussian kernel of size 7 × 7 and a standard deviation of 1.6, and then downsamples the image with scaling factor ×3. In addition to BI and BD, we also built the DN model, which first performs bicubic downsampling with scaling factor ×3 and then adds Gaussian noise with a noise level of 30. Training Setting. In each training batch, 16 LR RGB patches of size 48 × 48 are extracted as inputs. We perform data enhancement on the training images, which are randomly rotated by 90 •, 180 •, 270 • and flipped horizontally. We use the Adam optimizer to update the parameters of the network with β 1 = 0.9, β 2 = 0.999, and ϵ = 10 −8. For all layers in the network, the initial learning rate is set to 0.0001, and then the learning rate is halved every 200 epochs. We use the Pytorch framework to implement our model with Tesla P100. Global reasoning upsampling module. In order to verify the importance of the GRUM, we remove the GRUM from the network, leaving only the GRRB in the network for relation reasoning. As shown in Table 1, after removing GRUM, the performance of the network drops from 32.45 dB to 32.40 dB. When the Case Index is equal to 1, the corresponding model is the baseline model. We can observe that after GRUM is added to the baseline model, the network performance is improved from 32.31 dB to 32.42 dB. It can be seen that although our baseline model has achieved quite good , GRUM can still improve the performance by relation reasoning in upsample module. This also indicates that relation reasoning can indeed in better prformance. These comparisons fairly demonstrate the effectiveness of the GRUM for SR tasks. Global reasoning reconstruction block. Then, we continue to study the effectiveness of GRRB for the network. After we add the GRRB to the baseline model, GRRB improves the performance of the model from 32.31 dB to 32.40 dB. Furthermore, the model with GRUM has achieved good performance. And it is difficult to obtain further improvements. But when we add the GRRB to it, the network performance shows a significant improvement, and the PSNR value on Set5 increases from 32.42 dB to 32.45 dB. These indicates that it is very essential for our network. Basic parameters. Moreover, we also study the effects of two basic parameters N and α on the performance of the model. As shown in Table 1, we observe that larger N and smaller α would lead to higher performance. Considering that larger N and smaller α will also bring more computation, we set 10 and 8 as the value of N and α respectively. Figure 4. Although our SRGRN has less parameter number than that of EDSR, MDSR and D-DBPN, our SRGRN and SRGRN+ achieve higher performance, having a better tradeoff between model size and performance. This demonstrates our method can well balance the number of parameters and the reconstruction performance. For BI degradation model, we compare our proposed SRGRN and SRGRN+ with other seven stateof-the-art image SR methods in quantitative terms. Following the previous works (; b;), we also introduced a self-ensemble strategy to further improve the performance. We denote the self-ensemble method as SRGRN+. A quantitative for ×2, ×3, and ×4 is shown in Table 2. We compare our models with other state-of-the-art methods on PSNR and SSIM. It can be seen that our proposed SRGRN outperforms other methods on all datasets without adding self-ensemble. After adopting self-ensemble, the performance further improves on the basis of SRGRN, and it achieved the best on all datasets. It is worth mentioning that SRFBN uses DIV2K+Flickr2K as their training set, which employs more training images than us. Previous research has come to a that more data in training set leads to a better . However, their are still not comparable to ours. Although RDN (b) is a state-of-the-art method, our SRGRN can achieve better performance in all datasets through relational reasoning in upsampling and reconstruction parts. The quantitative indicate that our GRUM and GRRB play a vital role in improving network performance. (Dong et al.) and VDSR for BD and DN degradation model because of mismatched degradation model. For BD and DN, there is no doubt that reconstruction has become more difficult. As shown in Table 3 and Table 4, in the case of images with a lot of artifacts and noise, our SRGRN can get a excellent performance. This shows that SRGRN can effectively denoise and alleviate blurring artifacts. And when added to self-ensemble, SRGRN+ can achieve a better improvement. To prove that our SRGRN can be widely used in the real world and performs robustly, we also conduct SR experiments on representative real-world images. We reconstruct some low resolution images in the real world that lack a lot of high frequency information. Moreover, in this case, the original HR images are not available and the degradation model is unknown either. Experiments show our SRGRN can recover finer and more faithful real-world images than other state-of-the-art methods under this bad condition. This further reflects the superiority of relation reasoning. In this paper, inspired by the process of reconstructing images from the human visual system, we propose an super-resolution global reasoning network (SRGRN) for image SR, which aims at completing the reconstruction of SR images through global reasoning. We mainly propose global reasoning upsampling module (GRUM) and global reasoning reconstruction block (GRRB) as the core of the network. The GRUM can give the upsampling module the ability to perform relational reasoning in a global scope, which allows this process to overcome the limitations of the receptive field and recover more faithful details by analyzing more contextual information. The GRRB also enables the reconstruction block to make full use of the interaction between the regions and pixels to reconstruct SR images. We exploit SRGRN not only to handle low resolution images that are corrupted by three degradation model, but also to handle real-world images. Extensive benchmark evaluations demonstrate the importance of GRUM and GRRB. It also indicates that our SRGRN achieves superiority over state-of-the-art methods through global reasoning. Visual comparison with BI degradtion model. As shown in Figure 5, we show a visual comparison on 4× SR. For image "img_078" from Urban100, we observe that most methods, even RDN and SRFBN, cannot recover these lattices and suffer from extremely severe blurring artifacts. Only our SRGRN can alleviate these blurring artifacts, recovers sharper and clearer edges and finer texture. For image "MukoukizuNoChonbo" from Manga109, There are heavy blurrings artifacts in all comparison methods, and the outline of some letters are broken. However, our proposed SRGRN can accurately recover these outlines, more faithful to the ground truth. The above comparison are mainly due to the fact that SRGRN can enable upsampling and reconstruction modules to utilize more contextual information through relation reasoning. Visual comparison with BD and DN degradtion model. In Figure 6, we show the comparison of SRGRN with other models in visual . For image "img_014", we use bicubic upsampling to recover these images whose HR images are blurred with a Gaussian kernel before bicubic downsampling, then we obtain SR images with a lot of noticeable blurring artifacts. We have also observed that most methods, including RDN and SRFBN, do not clearly recover the lines around the window. Only our SRGRN can suppress blurring artifacts and recover these clear enough lines close to the ground truth by relation reasoning. For image "img_002", a large amount of noise corrupt the LR image and make it loss some detail. It can be seen that when using bicubic for upsampling, the obtained image not only has a large number of blurring artifacts but also a large amount of noise. However, we find that our SRGRN has great potential for removing noise efficiently and recover more detail. This fully demonstrates the effectiveness and robustness of our SRGRN for BD and DN degradation models. Visual comparison on Real-World Images. In figure 7, the resolution of these images is so small that there is a lot of high frequency information missing from them. Moreover, in this case, the original HR images are not available and the degradation model is unknown either. For image "window"(with 200 × 160 pixels), only our SRGRN is able to recover sharper window edges and produce clearer SR image. For image "flower"(with 256 × 200 pixels), most other methods recover images whose upper left corner produces the edge of the pistil that looks unreal. And their edges of the petals in the whole image are very blurry. Our SRGRN can recovers sharper edges and finer details than other state-of-the-art methods. The above analysis indicate our model perform robustly unknown degradation models. This further reflects the superiority of relation reasoning."

Introducing the response charactrization method for interpreting cell dynamics in learned long short-term memory (LSTM) networks.