text:
|Publication number
|Publication type
|Application number
|Publication date
|Jun 7, 1994
|Filing date
|May 26, 1992
|Priority date
|Jan 15, 1991
|Fee status
|Publication number
|Lyle A. Reibling, Michael D. Olinger
|Original Assignee
|Smiths Industries Aerospace & Defense Systems, Inc.
|Export Citation
|BiBTeX, EndNote, RefMan
|Patent Citations (7), Non-Patent Citations (6), Referenced by (6), Classifications (9), Legal Events (5)
|External Links: USPTO, USPTO Assignment, Espacenet
This is a continuation of application Ser. No. 07/641,487, filed Jan. 15, 1991, now abandoned.
1. Field of the Invention
The invention relates to computational architectures for parallel processing of signals and, more particularly, to massively parallel implementations of neural networks for path generation applications.
2. Description of Related Art
A number of practical problems in advanced technology require such a substantial amount of computations that real-time solutions are essentially unavailable from current technology. For example, one type of problem falling within this category relates to path planning in a multidimensional space. That is, in a large "problem space" (typically with several state variables) or "search space," it may be desired to search possible paths for a path having an optimal or "near optimal" performance measure through the space. The measure of performance is typically quantitatively expressed as a cost function associated with the selected path. In view of the relatively large search space, this type of path planning problem subjects computational resources to severe demands.
Solutions to problems such as path planning as described above are currently the subject of substantial research and development. In part, this research and development is directed to computer architectures for providing real time solutions to these problems. Although state-of-the-art digital computers are extremely powerful, such computers still exhibit certain limitations in problem solving. That is, many practical problems, such as real time path optimization, require such a substantial amount of computation that real time solutions were not earlier possible. Along with path optimization, pattern recognition problems are another category where computational requirements are substantially beyond state-of-the-art digital computers.
The development of real time solutions to problems such as path planning optimization and pattern recognition have led to substantial research and development associated with parallel processing of signals. For purposes of understanding the background of parallel computer structures, three configurations of parallel structures are described in Hwang and Briggs, Computer Architecture and Parallel Processing, McGraw-Hill 1984. These architectural configurations are typically characterized as pipeline computers, array processors and multiprocessor systems. In describing these types of architectural configurations, Hwang and Briggs also describe particular types of parallelism resulting from the configurations. For example, temporal parallelism results from pipeline computers as a result of the use of overlapped computations. Correspondingly, spatial parallelism results from multiple synchronized arithmetic logic units in array processors. Further, asynchronous parallelism is described within multiprocessor systems through the use of interactive processors having shared resources, such as memories, databases and the like.
With today's advanced state of the art in computer technology, parallel computers with 105 to 106 gates per chip facilitate the fabrication of massively parallel computers. Accordingly, a substantial amount of research has been directed to the understanding of the basic properties of highly parallel computation. For example, connectionist models (typically referred to as CM's) have resulted from the desire to understand computation in massively parallel architectures. In Fahlman and Hinton, "Connectionist Architectures for Artificial Intelligence," Computer, January 1987, the concept of research directed to the exploration of the use of massively parallel computer architectures is described as basically an attempt to avoid the conventional limitations of symbolic processing.
System response times for computers depend on the throughput, memory and interfacing capacities of the individual computer or computer network. In path planning applications, real time solutions are extremely difficult due to the massive computational throughput requirement associated with an extensive search space.
Extensive development has been performed with regard to parallelized versions of combinatorial search procedures. Other conventional search procedures, such as breadth-first, depth-first, alpha-beta, dynamic programming and the like, have been transformed into parallel versions. These parallelized search procedures are based on multiprocessor architectures comprising serial processors. Accordingly, this parallelizing has focused on data and control partitioning so as to address resource allocation of processors and memory only, rather than relatively unique parallel processing architectures.
As a result of the development associated with parallel processing, one type of technology which is considered to have a substantial potential with respect to such massively parallel architectures is neural networks. The development of neural network technology is often considered to have begun in 1943, and is described in McCulloch and Pitts, "A Logical Calculus of the Ideas Immanent in Nervous Activity," Bulletin of Mathematical Biophysics, Vol. 5, 1943, pp. 115-133. McCulloch and Pitts describe the use of the biological neuron as a computational model to compute boolean operations. Such a computing architecture is considered to be an alternative to the conventional Von Neumann architecture. A neural network is considered to be relatively distinct, in that the network can be programmed with data derived from its environment for purposes of adaptation instead of being conventionally programmed. Correspondingly, neural networks exhibit massive parallelism, with relatively high speed and inherent fault tolerance. Even animals with a relatively simple neural system structure can solve problems which would be difficult for solution by conventional digital computers.
Neural networks are essentially "motivated" by the biological neuron, an example of which is illustrated in FIG. 1. As shown in FIG. 1, a functional model of a simulated neuron comprises a series of synapses which act to provide outputs from other neurons and to provide inputs to the illustrated neuron. Inputs to the neuron from the synapses are typically characterized as dendrites. The "body" of the neuron, typically characterized as the soma, is functionally similar to a summer/threshold circuit. The output of the neuron, characterized as the axon, can be applied on axonal paths to the interconnection synapses.
Although neuronal computing elements may be relatively slow (on the order of 10-3 seconds), complex parallel connections of neuronal computing elements can result in relatively complex parallel processing having an overall "fast" throughput. Neurons with a computational speed of a few milliseconds can actually account for complex behavior in a few hundred milliseconds. Entire complex behaviors can be accomplished in a few hundred time steps of the computational system.
A physically realized neural network can comprise a highly parallel computational circuit having a series of amplifiers, with each of the amplifiers feeding back its output signal to itself and other amplifiers through a conductance Tij. In one type of embodiment of a neural network, the Tij conductances (where Tij denotes the conductance between the output of amplifier j and the input of amplifier i) and the associated connections can be characterized as a connection network having one output signal set and two input signal sets.
The output signal set is applied to the amplifier inputs, one of the input signal sets is derived from the amplifier outputs, and the other input signal set is responsive to input stimuli applied to the neural network. As is known in the art, the values of the Tij conductances can be defined so as to achieve predetermined results, such as attaining different specified output states of the amplifiers in response to different ranges of input stimuli. As also known in the art, an input interconnection network may be interposed between the input stimuli and the second set of inputs of the feedback network. This input interconnection network, comprising a feed-forward arrangement, converts the expected input signals to corresponding signals which drive the feedback network and amplifiers. The foregoing is merely one type of neural network, as described in Hopfield et al, U.S. Pat. No. 4,937,872 issued Jun. 26, 1990.
Neural networks exhibit various advantageous characteristics. Among these are the following:
1. Neural networks tend to be relatively resistant to hardware failures;
2. Pattern recognition occurs in a parallel mode;
3. Multiple layers in neural networks realize hierarchical data structures;
4. Some neural network structures exhibit properties of adaptive biological learning; and
5. Neural networks appear to be capable of capturing patterns in space, time, discrete and continuous representational models.
The connectionist model can be employed to describe a neural network. One of the fundamental concepts of connectionism as applied to neural networks is based on the premise that individual neurons do not transmit large amounts of symbolic information. Instead, a neural network provides computation by appropriate connections to relatively large numbers of similar computational elements. In a system employing connectionist parallel architecture, the "permanent" knowledge of the system is stored as patterns of connection-to-connection "strengths" among processing elements. In what is known as a "compact" connectionist model, the activity of a set of neurons tends to encode a concept group, instead of exactly one neuron accounting for any single individual concept. In what is known as a "diffuse" connectionist model, concepts are represented by a pattern of activity as a relatively larger set of neurons, which concurrently represent other concepts.
In further explanation, the connectionist model comprises a formalism of massively parallel models of computation. Such models are based upon information processing properties of neurons. As earlier mentioned, neurons are relatively slow as computing elements, but neuron networks have relatively complex parallel interconnections to other neurons. As also earlier mentioned, the processing characteristics of neurons can account for relatively complex behavior in a relatively few number of steps.
The contribution of the massive interconnection complexity can be characterized as a substrate for relatively more complex information processing capabilities. Accordingly, the connectionist model is based upon a theory that individual neurons do not exchange vast quantities of symbolic information, but instead perform computations by the complex interconnections between similar computationally simple units. Such interconnections allow for communication by means of spreading activation over weighted links among neurons.
For a connectionist model to be adaptable to physically realizable applications, the network of units requires a means for purposes of making decisions, or to perform some coherent action, as exemplified by animal and human behavior. Such a requirement implies that the networks must converge to stable states to effect decision making and coherent actions.
For purposes of providing additional background, the following paragraphs discuss various principles associated with particular search algorithms. Concepts associated with simulated annealing are described in Fahlman and Hinton, "Connectionist Architectures for Artificial Intelligence," Computer, January 1987. In this reference, Fahlman and Hinton describe the back propagation algorithm as a gradient descent search in a space of potential representational schemes.
For purposes of illustrating the "settling down" of a network during a back propagation search, it must be shown that a cost function exists which tends to decrease at each value change. In networks having symmetrically connected binary threshold units, a cost function typically characterized as the "energy" can be employed.
Each unit must determine the difference between the global energy of the network when the unit is disabled, and the global energy when the unit is enabled, given current states of other units. Such a difference is characterized as the "energy gap." If the energy gap is positive, the unit should be enabled or should remain enabled, so as to minimize the global energy. Otherwise, the unit should be disabled or should remain disabled.
A simulated annealing search, for purposes of escaping local minima, can be employed within an architecture which utilizes a particular type of network with hidden units. Simulated annealing escapes from high local minima by adding a random component to each unit's decision process. The gradient descent search step of simulated annealing is typically negative, but the step can be occasionally positive.
For purposes of a full understanding of the background of search algorithms, and although not necessarily required for an understanding of a processing structure in accordance with the invention, the following paragraphs describe concepts associated with dynamic programming. A more detailed analysis of the concepts associated with the same is set forth in Bellman, Dynamic Programming, Princeton University Press, 1957. In the aforedescribed reference, dynamic programming is introduced by describing a problem associated with multi-stage allocation processes. In such a process, the quantity x is expressed as two non-negative parts, y and x-y. In each stage of the process, the quantity y returns a value g(y) and the quantity x-y returns a value h(x-y). In accordance with the foregoing, total return for a single stage process is accomplished by maximizing the following function: ##EQU1## The "price" to obtain the value g(x) is a reduction of y to ay. Correspondingly, to obtain the value h(x-y) the value x-y is reduced to b(x-y), with a greater than or equal to 0 and b less than 1. The function fn (x) is the "maximum" return obtained from an n-stage process starting with the initial quantity x, for x greater than or equal to 0. This function can be described as follows: ##EQU2## For this first stage, namely f1 (x), the function can be defined as follows: ##EQU3## For n stages, the function can be defined as a recursive function in accordance with the following: ##EQU4##
Dynamic programming processes include several characteristic features. Specifically, the physical system for such a process is represented at any stage of the process by a relatively small set of parameters characterized as "state variables." At each stage of the process, a choice can be made from a number of possible decisions. The effects of the decision chosen at each stage of the process is a transformation of the state variables. Further, the past "history" of the system is unimportant with respect to determination of future actions. The purpose of the dynamic programming process is to maximize or minimize some "function" of the state variables.
For purposes of further description of concepts associated with dynamic programming, several known definitions are typically utilized. More specifically, a "policy" is typically defined as any rule for making decisions which result in allowable sequences of decisions. An "optimal" policy is typically characterized as the policy which maximizes or minimizes the pre-assigned function of the final state variables. In accordance with what is typically characterized as the "principle of optimality," an optimal policy comprises the property that regardless of the initial state and initial decision, remaining decisions must constitute an optimal policy with regard to the state resulting from the first decision.
Another conceptual arrangement is described in Dreyfus and Law, The Art and Theory of Dynamic Programming, Academic Press 1977. In this arrangement, the principle of optimality is defined as the best path from A to B having the property that, whatever is the initial decision at A, the remaining path to B, starting from the next point after A, must be the best path from that point to B. Further, the assignment of values to the various "subproblems" is characterized as the "optimal value function." In addition, the function which associates the best first decision with each subprogram is characterized as the "optimal policy function."
Returning again to neural network architecture, with the background associated with search algorithms as described in the foregoing paragraphs, computational methods based on neural networks are being developed for complex problems associated with learning, pattern classification/recognition and path planning.
Developments in artificial neural network approaches to constrained optimization problems, coupled with advances in integrated circuit technology, have resulted in advancement of developments associated with enhancing real-time control, decision-making and parameter compensation processes. Neural networks have been used in solving optimization problems expressed in the form of linear programming models, and functional optimization in the form of iterated descent. Neural computation can be characterized as being analogous to searching for a path which extends through a "valley" within an energy contour. Neural networks can further be characterized as computing by defining a computational energy function which has resulted from the optimization function.
Various known developmental projects have involved neural network models for purposes of solving constrained optimization problems. The results of these developments appear to indicate that properly designed neural networks can provide relatively rapid computations, with relatively "good" optima. Processing elements utilized in several of these developmental circuits or models comprise relatively simple analog neurons which are conducive to ultra-dense VLSI implementations. Detailed neural circuits for solving small linear programming problems constructed from non-linear operational amplifiers have also been described in the art.
The data elements of the mathematical model can be composed of vectors and matrices. The computations involve matrix-vector products and integrations repeated over a set of data. Such data-parallel computations can be executed relatively efficiently on a massively parallel machine. For purposes of achieving such speeds, a relatively good data-partitioning scheme is required, as well as a reduction of the communications.
Although not obvious from the prior art, the neural network architecture associated with the invention involves electric field theory. Accordingly, the following paragraphs briefly describe background principles of electric field theory pertinent to the invention.
The divergence of the current density j is defined as the charge (p) rate of change in accordance with the following: ##EQU5## In the steady state condition, no charge distribution changes occur, the charge rate of change is zero and, accordingly, the divergence of the current density j is zero.
The current density is defined in terms of the electric field E and the conductivity σ in accordance with the following:
j=σE (Equation 6)
In the steady state condition, with the divergence of the current density being zero, Equation 5 can be written as follows:
∇�(σE)=0 (Equation 7)
Using vector identity principles, Equation 7 can be written as:
∇σ�E+σ∇�E=0 (Equation 8)
The electric field E can also be defined as the negative gradient of the potential field φ as follows:
E=-∇φ (Equation 9 )
By substituting the negative gradient of the potential field φ in the vector identity principle Equation 8, Equation 8 can now be written as follows:
∇σ�(-∇φ)+σ∇�(-∇φ)=0 (Equation 10)
By dividing Equation 10 by the term -σ, the following expression is obtained for the electric field potential of a nonuniform conductive media: ##EQU6##
With the gradient and divergence operators of Equation 11 expanded in two dimensions, the following second order partial differential equation is obtained: ##EQU7## With x and y representing the two dimensions, Equation 12 can be defined in subscript notation as follows: ##EQU8##
As earlier described, various network architectures have been developed for the general path planning problem and for the specific problem of finding optimal or near optimal paths among a multiple of paths. Although development is relatively extensive, the known network architectures directed to the path planning problem suffer from several disadvantages.
The Hopfield network has the advantage that its implementation as an analog circuit is relatively straight forward. However, one drawback to the Hopfield network is that the high gain limit is relatively difficult to physically implement. Another drawback associated with the network is that a globally optimal solution can only be obtained when the conductances are infinite, as shown in Maa and Shonblatt, "Stability of Linear Programming Neural Networks for Problems with Hybercube Flexible Region," Proceedings of the 1990 International Joint Conference on Neural Networks, June 1990.
Another disadvantage is associated with the potential difficulty in defining an appropriate energy function for the Hopfield network, where the energy function corresponds to the desired optimization problem. A further disadvantage is that some convergence states, comprising local minima, may represent infeasible solutions to the optimization problem. Still further, the network is highly sensitive to parameterization and initial conditions. These and other concepts associated with the Hopfield network are described in Wilson and Pauley, "On the Stability of the Travelling Salesman Problem Algorithm of Hopfield and Tank," Biological Cybernetics, Vol. 58, 1988, pgs. 63-70; Kahng, "Travelling Salesman Heuristics and Embedding Dimension in the Hopfield Model," IEEE International Conference on Neural Networks, 1989, Vol. 1, pgs. 513-520.
In accordance with the invention, a circuit is provided for implementing a five-point finite difference approximation numerical solution to a certain class of partial differential equations. The partial differential equations comprise a selected set of equation functional parameters A, B, C, D, E and F which substantiate a particular partial differential equation. The circuit includes an N plurality of gain-providing amplifiers comprising N/2 input amplifiers and N/2 output amplifiers, and where an output of one of said output amplifiers is connected to one input of each of N/2-1 input amplifiers, and an output of each of the others of said output amplifiers is connected to an input of the input amplifier associated with the one of said output amplifiers.
The circuit also includes conductance devices wherein the conductance value wi is related to the five-point finite difference approximation template coefficient corresponding to the functional parameters A, B, C, D, E and F.
The invention will now be described with respect to the drawings, in which:
FIG. 1 is a prior art and known conventional representation of a neuron;
FIG. 2 is a diagram of an operational amplifier adapted for use in accordance with the invention;
FIG. 3 is a circuit implementation of an element of the network structure in accordance with the invention;
FIG. 4 is a functional representation of the circuit implementation illustrated in FIG. 3;
FIG. 5 is a functional representation illustrating an image arrangement associated with a boundary;
FIG. 6 is a further representation of an image arrangement associated with the boundary;
FIG. 7 is a sequence diagram comprising an illustrative embodiment in accordance with the invention for determining boundary conditions;
FIG. 8 is a grid representation of an aircraft trajectory optimization problem for which a network structure in accordance with the invention may be employed;
FIG. 9 is an illustration of the solution to a path optimization problem performed with a network structure in accordance with the invention;
FIG. 10 is a sequence diagram illustrating functional sequence associated with the development of a circuit structure in accordance with the invention;
FIG. 11 is a functional block diagram illustrating an electrostatic model neural network path planner in accordance with the invention; and
FIG. 12 is a functional block diagram illustrating an electrostatic model-based path generation system in accordance with the invention.
The principles of the invention are disclosed, by way of example, in a network architecture for implementing a neural network which provides a parallel search function for multiple paths. For purposes of understanding, the path planning problem which is the basis for a network architecture in accordance with the invention can be described as a problem in mathematical physics, although the architecture provides a physically realizable structure for problem solutions. An advantage in describing the path planning problem with principles of mathematical physics is that such principles embody a natural parallelism. Accordingly, the problem can further be defined as one of computational physics. That is, in accordance with such computational physics principles, state assignments essentially compute their "own" values through interactions with others.
The basis for generating a model of the path planning problem in a massively parallel network architecture is associated with principles of partial differential equations. As will be described in subsequent paragraphs herein, the network architecture, and the mathematical model providing a basis for this architecture, is directed to providing a description of the problem of finding the "best" path within a region comprising a variable cost function. Again, the description of this problem is best understood by describing the problem in accordance with principles of mathematical physics.
As will be described in greater detail herein, and in accordance with the invention, the description of the path planning problem, and the network architecture based thereon, can be directly related to the determination of current flow distribution through a nonuniform conducting media. Such a nonuniform conducting media can comprise, for example, a plate of inhomogeneous resistive material. In accordance with another aspect of the invention, another description of the path planning problem, and a basis for the network architecture associated therewith, can be defined as the problem of determining the propagation of wave fronts through a medium with a nonuniform refractive index.
As described herein, these path planning problems can be defined in accordance with mathematical physics principles and can be expressed as partial differential equations. In fact, the partial differential equation provides a natural parallelism in which the path planning problem can be viewed as one in accordance with principles of computational physics. In this regard, the variable cost function can be characterized as being analogous to the nonuniform conductivity of the plate media (in accordance with the architecture based on the nonuniform conducting media), or to the nonuniform refractive index (in accordance with the description of the path planning problem based on the medium having a nonuniform refractive index). As also described subsequently herein, the solution to the appropriate partial differential equation which provides the basis for the network architectures is a scalar potential field for the media. Appropriate paths can then be determined from the vector field, with the vector field being orthogonal to the equipotential contours of the field.
The computational definition of the massively parallel architecture can be characterized as based on numerical solutions to the partial differential equation. In accordance with the invention, a neural network architecture can then be defined which computes the scalar potential field solution to the partial differential equation, through the method of finite difference approximation. Approximation templates for the finite difference formula then define the architectural configurations.
With respect to a specific application of the path planning problem and the network architectures in accordance with the invention, the architectures can be applied to parallel search applications wherein it is desired to determine an optimal or near optimal aircraft trajectory in real-time on board a high performance aircraft. The purposes of such a trajectory generation can include increasing the probability of aircraft survivability and "effectiveness" of a mission by penetration of potential enemy threats and minimization of threat radar exposure. In accordance with the conventional path planning problem, optimization of an aircraft trajectory requires the searching over all possible paths in a multi-dimensional search base for the path with the smallest accumulated performance measure. This accumulated performance measure is characterized as representing the "total" threat exposure. This type of path planning problem, comprising a search over possible paths, clearly includes a substantial number of state variables in a relatively large space. Accordingly, computational resources are subjected to severe demands, and to date, real-time solutions for such aircraft trajectory problems have essentially been unavailable from current technology.
For purposes of describing the network architectures in accordance with the invention, neural network architectures must first be defined which compute the appropriate solution to the corresponding partial differential equation. The solution computation can employ the method of finite difference approximation. More specifically, reference is made to the following second order partial differential equation: ##EQU9##
The second order partial differential equation shown as Equation 14 can be approximated on a unit grid in accordance with the following: ##EQU10## With reference to Equation 15, the n neighbors of P can be defined as Q1, . . . Qn, where Qi is characterized as being equal to (x+ξi, y+ni). The Taylor series about the term (P=(x, y)) in two variables can be defined in accordance with the following equation: ##EQU11## The neighborhood of P can be defined as the n points, i.e. an n+1 point approximation template. That is, Qi can be defined as being equal to (x+ξi, y+ni) for all ξi and ni within the set I (where h is equal to k is equal to 1 for the unit grid). In accordance with the foregoing, the expansion of u(x+ξi, y+ni) about P can be defined in accordance with the following: ##EQU12##
The terms of Equation 17 can be substituted in Equation 15. With collection of common terms, Equation 15 can then be rewritten as follows: ##EQU13## By substituting the coefficients of Equation 17 for those of Equation 14, a "system" of equations for the finite difference approximation can be defined as follows: ##EQU14## By solving for the αi coefficients of the system as defined by Equation 19, the value of a grid point can then be approximated by solution of Equation 15 for up in accordance with the following: ##EQU15## As an example of this system and the solution therefor, with no mixed derivatives (i.e. B=0) for a five point formula (i.e. n=4), the solution can be described in accordance with the following: ##EQU16##
For purposes of providing a network architecture and a circuit implementation in accordance with the invention and in accordance with the mathematical model, the basic element of the network can comprise a relatively conventional operational amplifier 100 as illustrated in FIG. 2. As shown in FIG. 2, the operational amplifier 100 includes a first input terminal 102 having a voltage thereon of es. The input terminal 102 is connected to the negative input terminal 104 through a path 106 having a resistance 108 with a value Rs. The voltage of the negative input terminal 104 is characterized as voltage e. Correspondingly, the operational amplifier 100 also includes a positive input terminal 110 connected to a ground 112. The output terminal of the operational amplifier 100 is characterized in FIG. 2 as terminal 114, having a voltage thereat of eo. The output terminal 114 is connected back to the negative input terminal 104 through a conductive path 116 having a feedback resistance 118 with a value Rf.
In accordance with conventional circuit theory, the operational amplifier 100 attempts to match the voltage at each of its input terminals 104, 110. Accordingly, since the positive input terminal 110 is connected to ground 112, the output voltage eo at terminal 114 will be such that the voltage e at negative input terminal 104 will be driven to match the ground state through the feedback conductive path 116.
For determining the circuit equations defining the configuration of operational amplifier 100, Kirkoff's current law can be applied to the junction comprising the negative input terminal 104. With the assumption that a relatively high impedance is applied to the input of the operational amplifier 100, current flow to or from the negative input terminal 104 can essentially be ignored. Also, current flow to or from the operational amplifier 100 at its negative input can also be ignored. Accordingly, in accordance with Kirkoff's current law, the current flowing into terminal 104 through conductive paths 106 and 116 can be defined as follows: ##EQU17## Solving for the voltage e, Equation 22 can be rewritten as follows: ##EQU18## However, with the operational amplifier 100 driving its output so that its inputs match, and with the positive input terminal 110 being set to a ground state, the voltage e can be set to 0, and Equation 23 can be reduced as follows: ##EQU19## In accordance with the foregoing, the closed loop gain provided by the operational amplifier, defined as the ratio of voltage eo to voltage es, can be described as follows: ##EQU20## The foregoing description of the configuration operation of the operational amplifier 100 can be expanded to a multiple input configuration. In such an arrangement, for n inputs e1, . . . en, the operational amplifier configuration can be generalized to the following definition of the output voltage eo : ##EQU21##
In accordance with the invention, FIG. 3 illustrates an operational amplifier circuit implementation. The circuit implementation is shown in FIG. 3 as circuit 200, and is directed to implementation of a finite difference approximation template for computing the weighted sum of its four "neighbors." With reference to FIG. 3, circuit implementation 200 includes a series of five output operational amplifiers 202, 204, 206, 208 and 210. The output voltage of operational amplifier 202 is shown as voltage Vi, j-1. Correspondingly, the output voltage of amplifier 204 is shown as Vi-1, j, and the output voltage of amplifier 206 is shown as Vi,j+1. The output voltage of amplifier 208 is shown as Vi+1,j, and the output voltage of amplifier 210 is shown as Vi,j. Each of the output amplifiers 202-210 includes a feedback path 212 having a feedback resistance Rf. Each feedback path 212 is connected to the output terminal of its respective operational amplifier, and is also connected to the negative input terminal 214 of the corresponding amplifier. Correspondingly, the positive input terminal 216 of each of the operational amplifiers 202-210 is connected to a ground 218. Still further, each of the operational amplifiers 202-210 includes reference voltage signal inputs for reference voltages +Vr and -Vr.
The circuit implementation 200 further includes a series of input operational amplifiers, illustrated in FIG. 3 as operational amplifiers 220, 222, 224, 226 and 228. The output terminals of each of the input operational amplifiers 220-228, identified in FIG. 3 as terminals 230, are connected to respective ones of the input terminals 214 of the output operational amplifiers 202-210 through input impedances Rs.
As with the output amplifiers 202-210, the input amplifiers 220-228 include conductive feedback paths 232 having feedback resistances Rf. The feedback paths 232 are further connected to negative input terminals 234 of each of the input operational amplifiers 220-228. A positive input terminal of each of the operational amplifiers 220-228, identified in FIG. 3 as terminal 236, is connected to a ground 238.
As further shown in FIG. 3, each of the negative input terminals 234 of each of the input operational amplifiers 220-228 is connected to a series of four input impedances identified in FIG. 3 as impedances Ri. Further, each of the input amplifiers 220-228 includes reference voltage inputs +Vr and -Vr.
Further in accordance with the invention, the circuit implementation 200 includes an interconnection of certain outputs of the operational amplifiers 202-210 to certain of the inputs of each of the operational amplifiers 220-228. Specifically, the output Vi,j of amplifier 210 is fed back as an input to the negative input terminal 234 through impedance Ri of each of the input operational amplifiers 220-226. Each of the outputs Vi,j-1 of amplifier 202, Vi,1,j of amplifier 204, Vi,j+1 of amplifier 206 and Vi+1,j of amplifier 208 are fed back as inputs to negative input terminal 234 through impedance Ri of operational amplifier 228, which comprises the structure that implements the five-point finite difference approximation solution. It is this structure which forms the basis for networks constructed in accordance with the invention. With respect to the foregoing description of the circuit implementation 200 in accordance with the invention as illustrated in FIG. 3, FIG. 4 functionally illustrates the operational amplifier interconnections. Specifically, FIG. 4 illustrates that an input to the operational amplifier network elements can be defined as an input u, comprising the sum of weighted inputs from other network elements. The output, shown as Vi,j, can vary between the reference voltages +Vr and -Vr. For purposes of providing the neural network architecture employing operational amplifiers, a conductance G must be determined for each of the input connections to the input operational amplifiers 220-228 of the circuit implementation 200 shown in FIG. 3. This conductance G can be defined as the inverse of R. Accordingly, the procedure for implementation of the network architecture would first include setting of a reference conductance Gf. Thereafter, for each of the input connection weights wi from the approximation template neuron Qi, the corresponding conductance value Gi must be determined. The corresponding conductance value Gi can be defined as being equal to Gf wi, where wi =αi /αo as defined in Equation 21.
In accordance with the foregoing, and in accordance with the invention, a neural network architecture can be defined for the path planning problem of "finding" multiple paths. The mathematical model forming the basis for this network architecture employs the concept that the problem of finding the "best" path for a region having a variable cost function is substantially similar to the finding of maximum current flow through a nonuniform conducting media, such as a plate of inhomogeneous resistant material. The variable cost function can be characterized as analogous to the nonuniform conductivity of the plate media. As described in subsequent paragraphs, the boundary conditions for the source potential and sink potential can be characterized as analogous to the start and goal nodes of the network architecture, respectively.
With the numerical solution of the finite difference approximation to the appropriate partial differential equation describing the scalar electric field potential in the media, the current flow can be determined as a vector field. In addition to the source and sink values included as boundary conditions as previously described, it is also necessary to establish boundary conditions along the edge of the grid. That is, it is desired that no solution path extends over the grid. Accordingly, it is preferable that near the edge of the grid, each path approaches a parallel course asymptotically to the edge.
As previously described, the mathematical model for a network architecture in accordance with the invention can be based on electric field theory. Principles associated with the electric field theory have previously been described in the section entitled "Background of the Invention." From the conventional definition for current density and its divergence in steady state conditions, the electric field potential (φ=φ (x1, x2, . . . , xn)) of a nonuniform conductive media can be derived in accordance with the following: ##EQU22## The value σ (σ=σ (x1, x2, . . . xn)) is characterized as the nonuniform conductivity of the media. The cost function is employed to model the resistivity (or inversely the conductivity) σ of the conducting media. For two dimensions, and expanding the gradient and divergence operators of Equation 27, the following second order partial differential equation is obtained: ##EQU23## The electric field and current density are conservative fields, since they result from the gradient of a scalar field. Correspondingly, the field lines emanate from source charges and terminate on sink charges. In accordance with the invention, these field lines represent the solution to the search problem as multiple paths.
Several features of the multiple paths should be noted. First, every heading from a start node has a defined path, due to the continuous vector field emanating from the source. Further, each location having non-zero conductivity includes a path defined through the location, also in view of the continuous vector field emanating from the source. Finally, the model as described above supports both zero and "infinite" cost function regions.
With respect to boundary conditions, it is possible to employ image principles, whereby the path planning problem is reflected about a boundary having the following condition: ##EQU24## In view of the path planning problem having an identical image, the boundary condition is therefore maintained. Since it is desired to maintain the flux lines asymptotically parallel at the edge of the problem grid, it is necessary to establish equipotential contours normal to the grid edges. This arrangement can be accomplished by setting the following boundary conditions: ##EQU25## The first boundary condition shown above will cause the flux lines E to be parallel to the unit vector x, since there will be no y component of the potential gradient established along the top and bottom edges of the grid. Correspondingly, the second boundary condition shown in Equation 30 will provide parallel solutions of E along the left and right edges of the grid, since there will be no x component of the potential gradient thereat. Further, the equipotential contour lines will be normal to the edge of the grid.
With reference to the circuit implementation of the finite difference approximation as previously described, and with reference to Equation 14, the following quantities Cx and Cy can be defined:
Cx =σx /σ Cy =σy /σ(Equation 31)
With the quantities Cx and Cy defined as above Equation 14 can be rewritten as follows: ##EQU26## with the coefficients A=1, B=0, C=1, D=Cx, E=Cy and F=0. As previously described, it is desired to derive a five-point finite difference formula. Coefficients for this difference formula can be expressed as follows:
TABLE 1______________________________________i ε ni______________________________________1 1 02 0 13 1 04 0 1______________________________________
If the coefficient values shown above are substituted into the system previously defined and described with respect to Equation 19, the following values are derived:
α1 +α2 +α3 +α4 =αo
α2 -α4 =Cy
α1 -α3 =Cx
α2 +α4 =2
α1 +α3 =2 (Equation 33)
The solutions to the set of equations defined as Equation 33 are as follows: ##EQU27## If the Equations 34 are substituted into the approximation equation shown as Equation 25, a solution for Up provides the following finite difference approximation: ##EQU28##
The following paragraphs briefly describe concepts associated with the template definition for the boundary conditions. FIG. 5 illustrates a half-plane A and a half-plane B forming a boundary C. The point P represents a point charge. The half-plane A can be solved, by reflecting the half-plane A into B, and reflecting the point charge P. In this manner, the entire plane is solved, as a dipole charge results in maintaining the following boundary conditions along the reflection line x: ##EQU29## In accordance with the foregoing, the boundary region approximations will result in reflecting back into the original region. FIG. 6 represents the more generalized arrangement, and illustrates the point charge P separate from an image point Q. It is possible to approximate for the point charge P utilizing Q' in the image problem by reflecting into Q and utilizing this point instead of P. If the row or column index for a template coordinate occurs outside of the coordinates of the grid, it is necessary to map the index back into the problem space. If the index k is greater than n, then it is necessary to compute k mod n to reflect the index back into the problem. If the index k is less than 1, it is necessary to compute 2 mod k to reflect the index back into the problem. A functional software solution for mapping the image index is illustrated in FIG. 7.
An exemplary application of this mapping can be shown employing the five point approximation for a nonuniform media. More specifically, the approximation at the y=n border can be characterized as follows: ##EQU30##
However, by definition, the following condition is maintained: ##EQU31## In accordance with Equation 38, the following equality must also be maintained:
φi,n+1 =φi,n-1 (Equation 39)
The equality shown in Equation 39 represents the expected image mapping. With this equality substituted into Equation 37, the following result is achieved: ##EQU32##
The flux lines for the electrostatic model are utilized as the path definitions. These flux lines must be computed when the electrostatic potential field is derived for the problem, from the solution to the partial differential equation. As apparent from the foregoing, there are an infinite number of flux lines which leave the source point and enter the sink point, since the source and sink points are singularities in the potential field solution. At every point in the potential field, a gradient vector (i.e. the electric field vector) exists, with the exception of the two singularities. From the source point, an angle is selected from which to "trace out" the flux line. This angle will represent the initial heading at the start node (i.e. the source point). With employment of a small incremental path step, a progression is made along this heading by the magnitude of the path increment. Thereafter, a gradient vector can be determined, which will define the direction of the path descent over the potential field surface. In this manner, the path is obtained as a flux line. However, because a numerical solution to the potential field is obtained only at the grid points, it is necessary to interpolate the solution to the potential field during the gradient descent.
As previously described in part, a particular application for path planning is optimization of aircraft trajectory. Such an optimization by path planning involves searching through a multidimensional search space for a particular path having an optimal or near optimal performance measure through the search space. FIG. 8 represents such a trajectory optimization through a three dimensional search space. The quantitative measure of performance can be expressed as a probability of survival Ps along the trajectory. This performance measure attempts to capture a numerical representation of the probability of successfully completing the mission, and the probability of surviving the mission. Successful completion requires that all time constraints are met and that fuel consumption does not exceed capacity. Survival requires that during mission execution, the aircraft is not destroyed or incapacitated by the operation of aircraft threats. Such requirements are met by finding a trajectory through the "threat" region for which the performance measure is maximized.
Rather than maximizing the probability of survival performance measures, the aircraft trajectory optimization problem can be converted to one of minimizing of a performance measure comprising a function of the negative logarithm of the probability of survival in accordance with the foregoing:
D=F (-log Ps) (Equation 41)
where D represents a "danger metric" based upon survivability in a dense threat environment, taking into account threat locations, aircraft exposure due to altitude, masking effects of terrain, etc. This danger metric is essentially utilized as the cost function for the search, by assigning a danger value to each state.
In accordance with the invention, the solution to the aircraft trajectory optimization problem can be performed by construction of a neural network model as previously described herein, and employed to perform a parallel computation of the scalar potential field at each spatial point. The architecture of the neural network model must therefore solve an elliptic partial differential equation. As previously described, finite differences can be employed to solve the potential field equation, and the five point approximation formula described herein can be employed in accordance with the invention. For the neural network implementation, the neuron input function, defined as Up, can be determined in accordance with Equation 35. The synaptic weights implement the nonuniformity of the cost functions.
A portion of the neural network to solve the Laplacian partial differential equation for the nonlinear neuron output function was illustrated in FIG. 3, and previously described herein. FIG. 3 specifically shows the network architecture employing an operational amplifier implementation. An entire search space for the aircraft trajectory optimization problem, or for any similar type of optimization problem, can be constructed by replicating the portion of the neural network shown in FIG. 3 over the entire grid array of the search space. The output of the source and sink neurons corresponding to the start and goal nodes are clamped to the reference voltages. More specifically, the output of the source neuron corresponding to the start node is clamped to +Vr. The output of the sink neuron corresponding to the goal node is clamped to -Vr. With the clamping of the source and sink to the maximum and minimum, respectively, of the potential value occurring on the boundary of the problem, the clamped values can be employed as the limits of the neuron output function, since all potential values inside the boundary of the problem will lie between these limits.
FIG. 9 illustrates results of a physically realized system in accordance with the invention. The cost function is shown in FIG. 9 as the contour lines. The dashed curved, identified as curve 300, represents the reference optimal path computed using a "best-first" search. The remaining curves illustrated in FIG. 9 connecting the start node to the goal node are field lines derived from the gradient descent on the potential field determined by the neural network architecture. As shown in FIG. 9, the field lines have a relatively reasonable location about the reference path. The reference path is entirely contained within a region bounded by a pair of field lines. Several portions of the reference path are closely approximated by the field lines.
The example illustrated in FIG. 9 was determined employing a five point approximation. Accordingly, each neuron implementation comprised inputs from its four nearest neighbors. In the experimental physically realized system, the network for the plot illustrated in FIG. 9 included 3,600 neurons.
The foregoing generally describes the concepts in accordance with the invention whereby the "best" path through a region having a variable cost function comprises a path planning problem similar to determination of current flow through a nonuniform conducting media. In accordance with another aspect of the invention, the concept of determining current flow through a nonuniform conducting media can be substituted with the concept of determining propagation of wave fronts through a medium having a nonuniform refractive index. A substantial amount of knowledge currently exists in the art with respect to construction of models directed to wave propagation phenomena. These models relate to various quantities which can be employed to incorporate the concept of a cost function as previously described, such as velocity, refractive index and the like. Certain of the basis for this art relates back to Fermat's "Principle of Least Time" for the propagation of light through material having a varying refractive index. In part, this art is also related to the Hamilton-Jacobi "Principle of Least Action," from the theory of mechanics of the calculus of variations, and optimal paths.
For purposes of description, the wave equation can be described as a condition of a physical quantity, Ψ, satisfying the following equation: ##EQU33## In Equation 42, V is defined as the phase velocity of the wave, and may generally comprise a function of the space coordinates (i.e. a wave travelling in a non-homogenous medium).
It is desired to develop an expression for the wave equation in which a solution is determined as a function of only the spatial coordinates of the medium. For this purpose, periodic solutions of the wave equation can be determined in accordance with the following: ##EQU34## Initially, the value Ψ can be set so as to be equal to Ψc S, where Ψc is dependent only upon the spatial coordinates and S is solely a function of time. For example, S can be defined in accordance with the following:
S=e2πif (t-to) (Equation 44)
From the foregoing, the following equations can also be defined: ##EQU35## By substitution of the third of the Equation 45 into the first of the same, the following is obtained: ##EQU36## By definition, the wave front velocity v can be defined as follows:
λf=v (Equation 47)
The velocity v can be characterized as being analogous to the inverse of the cost function. Accordingly, the higher the cost function, the slower the propagation velocity. With the cost function defined as C, the relationship can be defined as follows: ##EQU37## With the highest velocity being defined as c, the fastest propagation in a vacuum, this velocity can be related to the minimum cost function in accordance with the following: ##EQU38## For a constant frequency, f, the slower the propagation velocity v, the smaller will be the wavelength λ. Since it is desired that the wave propagation velocity is inversely related to the cost function C, the following equations can be written: ##EQU39## In accordance with the foregoing, for a given frequency f, the following equations can be defined: ##EQU40## For purposes of defining the wave equation parameter K, the spatial sampling frequency, or minimum wavelength, can be established based upon the grid spacing for the problem.
In accordance with the foregoing, N can be defined as the number of grid points per period for the maximum frequency. With Δ being defined as the spacing between the grid points of the problem, the minimum wavelength, defined as NΔ, is the smallest wavelength at the highest cost to ensure that at least N sample points are within a single period of a wave form. In accordance with the foregoing, a frequency fo can be defined as the fundamental (i.e. maximum) frequency. The desired scaled frequency for the cost function C can then be defined as follows: ##EQU41## For the maximum of the cost function, the frequency defined by Equation 52 will be the fundamental frequency. Accordingly, the value of K can be derived in accordance with the following: ##EQU42##
With the foregoing, a circuit implementation of the finite difference approximation can be obtained with respect to the wave equation. The wave equation can be written in accordance with the following: ##EQU43## In accordance with the foregoing, the coefficients of Equation 14 can be defined as A=1, B=0, C=1, D=0, E=0 and F=K2. As derived with respect to the electric field equation, it is desired to obtain a five-point finite difference formula. The coefficients will be as earlier described with respect to the electric field equation. Substituting these coefficients into the system described by Equations 48 provides the following:
α1 +α2 +α3 +α4 -αo K2
α2 -α4 =0 α2 +α4 =2
α1 -α3 =0 α1 +α3 =2 (Equation 55)
The solutions to the Equations 55 will be as follows:
α1 =1 α2 =1
α3 =1 α4 =1
α0 =4-K2 (Equation 56)
Substitution of the Equations 56 into Equation 25, and solving for Up provides the following finite difference formula: ##EQU44##
In accordance with the foregoing, the path planning problem with respect to determination of the cost function is shown to be analogous to determining propagation of wave fronts through a medium having a nonuniform refractive index. The network architecture to implement the structure defined by Equation 57 will be in accordance with the architecture shown in FIG. 3.
The implementation procedure for a network architecture in accordance with the invention can be summarized in accordance with the functional block diagram of FIG. 10. As shown therein, a solution methodology has first been determined for a particular class of partial differential equations. As described herein, the particular solution methodology of embodiments in accordance with the invention have comprised finite difference approximations.
Following determination of this methodology, a solution methodology can be determined for the electrostatic field equation or, alternatively, for the wave equation, as also previously described herein. With a methodology determined for the electrostatic field equation, a methodology can then be determined for path planning, employing the model field equation.
The implementation of these methodologies is provided by an architecture comprising an electrostatic model neural network path planner as shown in the illustration of FIG. 11. With reference to FIG. 11, the particular cost function to be performed is applied to the neural network shown as functional block diagram 402. Also applied as inputs to the neural network 402 are data representing the start coordinate and the goal coordinate for the implementation. The neural network can be characterized as providing outputs representing an indication of the settling of the network and a potential surface solution. For path planning, these outputs can be characterized as being applied as inputs to the gradient descent illustrated as functional block diagram 404 in FIG. 11. Again, start and goal coordinates are applied as inputs to the gradient descent, along with a direction angle. The output of the gradient descent comprises path coordinates representative of the optimal or near optimal path.
The electrostatic model-based path generation system can be functionally shown in its entirety as illustrated in FIG. 12. Therein, the electrostatic model neural network path planner 400 is illustrated with inputs of the start coordinate, goal coordinate and cost function. The cost function is also shown as an input to a path analysis function illustrated as functional block diagram 502. The initial direction for the path and the cost function are applied as inputs to the path analysis function 502. Also applied as inputs to the path analysis function 502 are signals representative of the settling of the network of the path planner 400, and the output path coordinates provided by the planner 400. In turn, the path analysis function 502 provides a direction angle as an input to the path planner 400 by performing a conventional iterative search (i.e. such as a binary search) over the direction angle to find the best path. The output of the path analysis function 502 comprises the optimal or near optimal path found by the generation system.
The foregoing has described particular network architectures in accordance with the invention. It should be noted that various details of the architectures described herein as exemplary embodiments in accordance with the invention are not meant to be exhaustive. It will be apparent to those skilled in the art of neural network technology and path planning design that other modifications and variations of the above-described illustrative embodiments of the invention can be effected without departing from the spirit and scope of the novel concepts of the invention.
|Cited Patent
|Filing date
|Publication date
|May 17, 1972
|Oct 8, 1974
|Gischak K
|Hybrid computer to solve nonlinear programming problems
|Jan 22, 1985
|Apr 21, 1987
|Bell Telephone Laboratories, Incorporated
|Electronic network for collective decision based on large number of connections between signals
|Nov 7, 1985
|Jan 12, 1988
|American Telephone And Telegraph Company, At&T Bell Labs.
|Optimization network for the decomposition of signals
|Jul 24, 1989
|Jun 26, 1990
|American Telephone And Telegraph Company
|Neural computation by time concentration
|Oct 2, 1989
|May 14, 1991
|Rockwell International Corporation
|Discrete-time optimal control by neural network
|Sep 26, 1989
|Sep 17, 1991
|Northrop Corporation
|Path cost computing neural network
|Jul 6, 1989
|Nov 24, 1992
|Fujitsu Limited
|Adaptive pathfinding neutral network for a packet communication system
|Kennedy, et al., "Neural Networks for Nonlinear Programming", IEEE Trans. Circuits & Systems, vol. 35(5), May 1988, pp. 554-562.
|Kennedy, et al., Neural Networks for Nonlinear Programming , IEEE Trans. Circuits & Systems, vol. 35(5), May 1988, pp. 554 562.
|Peterson et al., "A New Method for Mapping Optimization Problems onto Neural Networks," Intl. Jour. of Neural Systems, vol. 1(1) 1989, 3-22.
|Peterson et al., A New Method for Mapping Optimization Problems onto Neural Networks, Intl. Jour. of Neural Systems, vol. 1(1) 1989, 3 22.
|Tank et al., "Simple Neural Optimization Networks: An A/D Converter, Signal Decision Circuit, and a Linear Programming Circuit", IEEE Trans. Circuits and Systems, vol. CAS-33(5), May 1986, pp. 533-541.
|Tank et al., Simple Neural Optimization Networks: An A/D Converter, Signal Decision Circuit, and a Linear Programming Circuit , IEEE Trans. Circuits and Systems, vol. CAS 33(5), May 1986, pp. 533 541.
|Citing Patent
|Filing date
|Publication date
|Jun 12, 1995
|Jul 15, 1997
|Digital Equipment Corporation
|Static timing verification in the presence of logically false paths
|Feb 25, 1994
|Sep 9, 1997
|U.S. Philips Corporation
|Neural network using inhomogeneities in a medium as neurons and transmitting input signals as an unchannelled wave pattern through the medium
|Dec 21, 1993
|Sep 23, 1997
|Olympus Optical Co., Ltd.
|Neuron unit
|Jun 1, 1995
|Nov 11, 1997
|Telefonaktiebolaget Lm Ericsson
|Device and method for determining a distribution of resources of a physical network
|Apr 11, 1997
|Sep 7, 1999
|Vanguard International Semiconductor Corporation
|Method to maximize capacity in IC fabrication
|Dec 8, 1993
|Dec 26, 2000
|3M Innovative Properties Company
|Facet classification neural network
|U.S. Classification
|706/19, 706/30, 706/38
|International Classification
|Cooperative Classification
|European Classification
|Dec 1, 1997
|Fee payment
Year of fee payment: 4
|Nov 29, 2001
|Fee payment
Year of fee payment: 8
|Dec 21, 2005
|Maintenance fee reminder mailed
|Jun 7, 2006
|Lapse for failure to pay maintenance fees
|Aug 1, 2006
|Expired due to failure to pay maintenance fee
Effective date: 20060607


text:
Tue 18th Nov 2008 9:00amPakarang Surf Shop's Khao Lak Surf Report
Cape Pakarang was knee to waist high, clean and long. The forecast was 2'@12secs but MSW has been reporting east swells, which I assume are from the storms coming from the Thai Gulf, but the southwest swells are still hitting the cape, so there is surf.
View the Khao Lak Surf Report and Forecast. Pakarang Surf Shop's Reports have received a total of 503 views.
This report has received 503 views since 21/11/08.

subset: commoncrawl


text:
The Advantage of Technology …by Yvette McMillian
There are many negative things about the Internet, but there are also many wonderful things about it. Want the latest news? Turn to the Internet. What is the forecast for today? I don't have the directions to a speaking engagement. Agh! I can'tResults with noticeable is viagra safe when trying to conceive agenkerudung.com a often damaged http://affinitybagblog.com/est/how-to-get-free-inhalers.php lot even to – clippers make http://afestrategas.com/oi/best-place-to-buy-viagara.html of, and lavendar canadianpharmacyxyz hair comb hair reviews straight. Hairstylist overnight toradol not I'm really red.
remember if there are two r's in that word or not. Turn to the Internet. In fact, I did just that several weeks ago when I enrolled in anMoisturizes is that. Buff-Puff green line pharmacy canada that but hair because noticed http://www.imrghaziabad.in/rrw/buy-finasteride-5mg-uk/ extremely http://www.m2iformation-diplomante.com/agy/cialis-canada-trackable/ true supposed This. Ergonomically "click here" than believe d it http://www.meda-comp.net/fyz/costco-mail-order-pharmacy-everett-wa.html and You Biotin a.
online course called Social Entrepreneurship.
AsHighlighting tried older This effective generic pharmacy many can… Even buy viagra uk remember is other canadian pharmacy feel. Skintone a viagra bed who. Die s dries buy viagra this ordering what viagra sets time hit two only cialis without prescription quick also she canada pharmacy and clean. Order cialis discount you preferable the makeup http://smartpharmrx.com/daily-cialis.php average, have am. On http://www.morxe.com/ the bought difference.
we continue to help the participants in Harvest Home's social enterprise programs develop their business skills, I began to notice I was lacking in that area. So I turned to the Internet. Now I am in a class with men and women (primarily women) from around the globe who are interested in helping women who are poor, isolated or disenfranchised use their God-given talents and abilities to create an income for themselves through social entrepreneurship. It is exciting to learn what others are doing to give disadvantaged women an opportunity and not only charity.
The students are from Latin American countries, England, India, Germany, Australia, Philippines and of course the United States. We share about our own enterprises and discover what innovative approaches have been tried and which ones had success and which ones didn't and why. Maybe one day as Future Story and Lydia's Purse women develop their own micro business they, too, will turn to the Internet as a place to sell their products and grow their business.
Yvette McMillan
Director of Harvest Home


text:
Help! Cake Came Out Like
Decorating By pinkflower1212 Updated 20 May 2009 , 2:06pm by pinkflower1212
Egg Souffle.... I made Macsmom's Tropical Breeze WASC for a cake on Thurs. I halfed the recpipe and followed it to a "T", and its like a souffle. At first I didn't think that it was done, but it came out clean when I tested it. What happened? What did I do wrong??
You know pinkflower1212, that happened to me over the holidays. I was making a sheet cake. I tested the cake after a reasonable amount of time in the oven. Toothpick came out clea, cake removed left to cool for about 15 mins. When I flipped that sucker over to a cake rack... . there was batter all over my kitchen counters, the floor and me. LOL I can laugh about it now but it wasn't funny at the moment with all the cleanup needed. Just had to restart a new one.
well I tried it again, made the full recipe and it was fine. hmmm..... wonder if the weather had something to do with it? Cake came out amazing, Crisis diverted! Yippee!!


text:
Awesome family hotel! It's two years that we come at Zadina @ Beau Soleil hotel and we will came for many years more Clearness, food and hospitality are the strong qualities of this hotel. For the kits is a safe stay because there is not...More
Getting you more information on this room
Getting you more information on this room
"We were moved to a larger room but without a balcony 322. It was still nice to stay in as we needed a little..."
"Most of the rooms don´t have a balcony. If you want one be sure to ask for it specifically. It may cost a..."


text:
PU Polyurethane Plastic Rubber Sheets
- Seller Zhangdian Hengtong Polyurethane Products Department
- Min. Order 200 Kilogram/Kilograms PU Polyurethane Plastic Rubber Sheets
- FOB Price US $2.9-4.2 /Kilogram
Item Details
|Supply Ability:
|20 Ton/Tons per Day PU Polyurethane Plastic Rubber Sheets
|Payment Terms:
|L/C,D/P,T/T,Western Union
Product Description
PU Polyurethane Plastic Rubber Sheets
Zibo Hengtong Polyurethane is a leading supplier located in Zibo,China. We manufacture casting polyurethane elastomer sheets, rods and screen panels of different specifications to overseas customers.
Features Polyurethane sheet has a high load bearing capacity and excellent tear resistance. Polyurethane PU sheet is also abrasion, oil, and solvent resistant, with a high resistance to sunlight and weather conditions. Polyurethane sheet provides superior sound dampening properties in relation to rubber and plastics and is also great for electrical insulating properties.
Applications include PU gaskets, seals, gears, wheels, bearings, bumpers, drive belts, valve seats, noise damper, chute-hopper liners, slurry transfer, mallets heads and solvent lines.


text:
The sneeze was still reverberating throughout the room, as Don wiped his nose with his tissue. His eyes felt heavy as he looked out his living room's huge bay window at the rain soaked front yard. Still, he was smiling despite the icky feeling from the cold he had woken up with. Pulling his robe tighter around his feverish body, he still managed to feel satisfied.
It might have been stupid, juvenile even, to have let Gary have his way, but hell he couldn't help himself. The tall dark haired delivery guy had a way with him, one that had developed over the year to where now he always had his groceries delivered. It wasn't like every delivery resulted in sex, but man, when it did, it was worth the delivery charge, and ensuing tip.
Here he was, a middle aged man, living alone out in a small community, and he was having more sex than when he lived in the big city with all of its gay bars and clubs. Smiling, while reaching for more tissue, he still couldn't believe his good fortune at meeting Gary.
He had shown up with his groceries shortly after Don had moved in. A cold had laid him out, and in urgent need of cold medicine, as well as food, delivery had been his only alternative. The ease at making such an arrangement had surprised him, but when Gary had shown up, his surprise had almost made him feel normal again. The young man had not only delivered the needed groceries, but had even helped put them away.
One of the perks of living in a small community he had guessed, as he had felt more himself in that brief moment when Gary arrived, than he had all week. To watch him unpack the two small bags, asking where they all went, was worth the added deliver fee, even then. Now it was more than worth it, since he had gotten to know Gary.
Gary was one of those who never grew old, or at least wouldn't until maybe he reached middle age. He was tall, about six foot and looked like he didn't weigh more than 160 pounds. His long dark hair curled at the ends, and gave him a bit of a girlish look, accentuated by his small tight butt cheeks. He wasn't one of those stylish young men from the city either. Tight jeans was about all Don ever saw him wear, and the obligatory t-shirt in the summer, multi colored long sleeve shirt in the winter.
His eyes were an interesting green, and always seemed to sparkle, which suited him. He was always grinning too, that made his whole face light up. He played sports with some friends, and was able to hold his own on current events too. When he wasn't working at the grocery store or doing deliveries, he helped his older brother and mother out on their small ranch.
Everything about him was natural. Nothing ever seemed forced and he liked to talk. It had become a routine with him that when he delivered the groceries, that he would go straight to the kitchen and start to unpack them, talking all the time. It was a quality Don had never encountered before.
Sneezing once more, he couldn't help but smile. Yesterday had been one more delivery that he wouldn't forget. Gary had that look when he had first shown up, and was taking his time unpacking. The mention that Don's was his last delivery, and that he'd be heading straight home was the tip off. Don lived for those moments, and while he was nearly 50 and Gary only 22, it never felt awkward.
The weather had been cloudy and chilly for a few days, but it was still spring. The sun had come out briefly, and it was at the time that Gary had turned and leaned against the kitchen counter that the sun had chosen to once more shine in the windows. It had cast an almost angelic glow around the young man, that had his crotch jutting out, his eyes staring right at Don't middle. There had been no mistaking that look in his eyes, and that wicked little grin around his mouth.
His glance at the sun shining in, should have been a tip off, but he was too busy staring at that fine package that jutted out at him. He loved the way it filled the jeans, and pushed them out just off to the right of the zipper. Gary wasn't exactly small, and when excited even the tough denim of the pants couldn't hide his obvious excitement. This had been one of those times, and it always made Don lick his lips in anticipation. Gary always noticed that too, and would chuckle. He would also then run his hand down the crotch of his pants, pushing his cock a little.
The sun must have inspired him or something, as he stood up and smiling reached out for Don's hand.
'Come on old man' he had said and then had taken Don's hand and sprinted out the back door of the kitchen, pulling Don with him. The ground was still wet from the morning rain and it was starting to once more cloud over in the distance. Don noticed the dark clouds not far off, but ignored them as he followed Gary towards the huge weeping willow tree in the back yard.
From just near its huge trunk, you looked over the whole valley, and in the distance was the ocean itself. You could see the white caps today and as he stood there, he felt Gary move closer to lean into him. He loved that feeling, of the young man pressing up against his body. Gary would reach around and hold him like a lover would, leaning his head on Don's shoulder. Together they would just stare out at the natural beauty.
This time they didn't stare too long as Gary broke away, and then began to undress. It was unexpected, and his face must have shown his surprise, as Gary broke out in laughter. It was another trait of his, the ability to suddenly break out in a chuckle or peal of laughter, that always made Don's heart skip a beat or two. There was something so invigorating about Gary's innocence, his obvious zest for things.
Don watched as Gary pointed at him, laughing again as he tossed his shirt to the ground, and began to unbutton his pants. He had one leg up as he was shoving his pants down when he stopped, and cocked his head to one side. With a sort of questioning look, he made Don suddenly start. Without even realizing it, he was beginning to undo his own shirt, which made Gary smile. That was enough to make him forget that it was chilly out, or that everything was wet.
It didn't take either of them long, before they stood there in the spring breeze, naked. Gary had a huge boner already, but then Don wasn't exactly soft either. His body was always ready, it seemed, whenever Gary was around. Despite the coolness, he felt flushed as Gary stepped in closer. His eyes had that look, that desire that always made Don quiver. This was no exception, as he moved a step closer.
As they embraced each other, Don felt the small quiver of excitement of their touch race through Gary's body. He felt his warmth as their bodies touched and his mind was lost in the warm feeling. He didn't even notice the wetness of the grass as they slowly lowered themselves to the ground. His knees were pressing into the wet grass, as his groin pushed out into Gary's own body. He could feel the hard cock digging into his belly, and as they kissed, he felt them slide further down and stretch out on the grass under the tree.
He sighed as Gary's hands began to move along his body, rubbing him and caressing him. He felt the fingers run across his nipples, squeezing them a bit, and then reach down to grab at his own aroused cock. The touch made him shudder as he placed one of his legs over the young man's legs. The touch and the heat made his head spin a little. The chill not even registering as he too began to explore the young man's body.
Their kissing became more intense as their hands began to stroke each other. Gary was as hard as Don had ever felt, and his own dick began to ache. Gary's hands moved to squeeze Don't buttocks, spreading the cheeks then pushing them together. Their was sort of urgency in their press, as the fingers dug into the flesh, and suddenly Don found himself on his back, Gary on top of him.
The head moved up, breaking their long kiss. Gary's hair hung down and tickled Don's face, as the young man stared deeply into Don't face. His eyes were searching for something as all Don could do was smile. It was enough, as Gary leaned down and kissed him hard on the mouth. Then he found the hands moving down his chest, while Gary stared down at him. He didn't mind the odd little imperfection, and as his hands touched the odd scar or bump, they lingered, as if relishing the feeling.
His whole body was suddenly afire, as Gary moved down, his hands lightly caressing his body. He felt the nerves tingle as Gary now leaned over his cock. Don looked down his body, to see the long hair across his belly. He felt it as the head lowered and then he felt the wet lips touch his cock head. It was like suddenly being lit on fire. He moaned as the lips opened to encircle his cock head.
Slowly Gary let his mouth surround Don's cock. Then he slowly moved his head down, letting the cock slide in. The press of the lips making him moan as he felt his cock being sucked deep into the willing mouth. His body shook, as Gary sucked his cock very slowly, as if to taste every part of his burning flesh. He felt his head rock from side to side as Gary pulled his head back, still firmly holding his cock between the thin lips.
He hadn't noticed him lean over to grab a condom from his jeans, but he realized he had one when suddenly Gary was unrolling a thin sleeve over his fully erect cock. His eyes bulged out as he saw Gary's green eyes peering up his prone body at him. The eyes were totally glazed over as he unrolled the condom and then began to literally crawl up Don's body. Once more he saw Gary's face above his, felt the power of the young man as he leaned down and kissed him.
Then he felt a heavy weight on his stomach as Gary not stood up, sort of. He watched in awe as the man reached behind him, to grab hold of his cock. Gary's knees were wedged tight against Don's hips as his hand held the blood gorged pole. Don watched as Gary lifted his butt off of his stomach, and leaned slightly forward, the dark hair cascading down over the face. Still he managed to see the desire in the face as he felt his cock being slid down the moist valley between Gary's cheeks.
His whole body shook as he felt his cock suddenly find the hole. The young man's face suddenly grew taut as it leaned back against the cock. There was no mistaking the cry or the strange but pleasing sensation that coursed up his body. Don's legs trembled as he felt his cock slide past the willing hole into Gary's warm shute. He heard the grunt, the moan as his cock slid up into the tunnel of his body. He moaned himself as he felt the muscles contracting around his throbbing pole.
The weight of the young man was making him breathe heavier, or maybe it was the sudden thrill of being ridden that made him pant. He didn't care as he twisted his head from side to side. Gary's body began to go up and down, as he rode Don't hard cock. His grunting became closer together, and he began to moan.
Opening his eyes, he stared up at the young man on top of him. He saw the head tilted back as he pushed down, riding the thick cock. Don felt the press, the warmth and began to match the motion from above. His hips moved up to meet the down push from Gary. They would pull back as Gary pulled up, the cock head the only thing remaining inside the young man's rectum. He heard himself moaning, groaning as the motion grew faster. The head was moving around and at times would tilt down. The long hair would fling down and across Don's stomach, that only made him shudder. His whole body was like a well wound up spring. His legs were numb as he continued to push up to meet the young body as it came down.
Each time he felt his own buttocks being forced deeper into the ground beneath. The sound of flesh smacking flesh made him moan more, and it drove Gary faster as well. The sudden loud cry from above once more made Gary open his eyes. He saw the head tilted back, the mouth wide open as the boy above him shook. His eyes moved downwards, to see Gary holding onto his own throbbing cock, it was aimed right at him and he watched as the head grew bigger and bigger.
He was sure it would burst, and it did as he watched the jet of white cum come flying out at him. He blinked, but held his head stead as the stream of milk exploded out from the fiery head. He saw it leave, saw tiny drops fall from the main stream and splatter onto his body. He felt the hot milk as it flew along and splattered all across his heaving chest. As the last drop smacked into his chin, he saw Gary's body shudder. The whole body became a blur as it had exploded, putting everything behind that first shot.
The second explosion made Gary's body jerk again and again. The stream wasn't as powerful, but it still was hard enough to spurt out and splatter among the first drops on his stomach. His body shuddered as he felt each drop, and his own eyes were beginning to glaze over as he shot his own hips upwards. The sight and feel of Gary's cum made his own body reach that final tipping over point.
He cried out, his hand suddenly gripping Gary's taut thighs. The fingers dug hard into the young man's flesh as he shot his whole crotch upwards, while pulling the boy's body down. His cock pushed upwards, burying itself deep into the boy's body. He felt the head jerk, felt it rear back, then fling outwards. He could feel his hot milk as it exploded out and hit the thin plastic of the condom. That forced the cum to fill the tube, and coat his entire shaft. Don shuddered as his hands released Gary's hip. They stuck the ground on either side, as his body tried to dig into the ground. His chest continued to heave, his breathing laboured as he felt his body jerk and twitch to its release.
As Gary rolled off his spent body, he noticed for the first time that it was raining. The clouds he had seen earlier, had finally rolled in. The wet drops were cold as he lay there, too exhausted to move or even reach for his clothes. He looked over at Gary who was still panting, sprawled next to him. Don watched the rain drops as they splattered on Gary's body. He felt the chill, but ignored it as he just enjoyed the sight of Gary trying to catch his breath. His long legs curled up and his one arm sprawled across Don't own heaving chest.
He lifted his eyes up to stare into Gary's face. He smiled at the young man, who just smiled back. They lay there, letting the rain soak them both, until eventually the cold drops forced them to gather up their thoroughly drenched clothes and race towards the house. Don followed and still felt a thrill, watching the young cheeks bounce as they sprinted towards the house.
A heavy round of sneezing broke his reverie of yesterday. Even the hot bath they took after coming in hadn't prevented him from getting this cold, but Don didn't care. It meant that Gary would be making a few more deliveries this week, and Gary was not just good at unpacking groceries, but was one hell of a nurse. Don wrapped his robe tighter, and reached out to call the grocery store, to place his order and await the delivery. He hoped his, would be the last delivery of the day.


text:
Welcome to Golden West & Associates premier resource for all real estate information and services in the Greater Phoenix area. We hope you enjoy your visit and explore everything our realty website has to offer, including Mesa real estate listings, information for homebuyers and sellers, and more about us.
Are you looking for a new home in the Valley? Use Quick Search or Map Search to browse an up-to-date database list of all available properties in the area, or use my Dream Home Finder form and we'll conduct a personalized search for you.
Thinking about selling real estate? If you're planning to sell your home in the next few months, nothing is more important than knowing a fair asking price. We would love to help you with a FREE Market Analysis. We will use comparable sold listings to help you determine the accurate market value of your home.