text:
Integrating design, deployment, and management phases for a system in accordance with certain aspects includes using a system definition model to design a system. The system definition model is subsequently used to deploy the system on one or more computing devices and, after deployment of the system, the system definition model is used to manage the system deployed on the one or more computing devices.
1. A method comprising:
using a system definition model to design a system;
subsequently using the system definition model to deploy the system on one or more computing devices; and
after deployment of the system, using the system definition model to manage the system deployed on the one or more computing devices.
2. A method as recited in
3. A method as recited in
4. A method as recited in
using knowledge obtained during management of the system to design a subsequent version of the system.
5. A method as recited in
6. A method as recited in
7. A method as recited in
8. A method as recited in
9. A method as recited in
10. A method as recited in
11. A method as recited in
during management of the system, using a flow to automatically propagate a configuration change to the system.
12. A method as recited in
using another system definition model to design the environment;
subsequently using the other system definition model to deploy the environment on the one or more computing devices; and
after deployment of the environment, using the other system definition model to manage the environment deployed on the one or more computing devices.
13. A method as recited in
14. A method as recited in
15. A method as recited in
using a plurality of different system definition models to design each of the plurality of environments, wherein each of the plurality of environments is associated with one of the plurality of different system definition models;
using, for each environment, the associated one of the plurality of different system definition models to deploy the environment; and
after deployment, using, for each environment, the associated one of the plurality of different system definition models to manage the environment.
16. A method as recited in
17. One or more computer readable media having stored thereon a plurality of instructions that implement a schema, wherein the plurality of instructions, when executed by a processor, cause the processor to:
facilitate design of a system;
facilitate deployment of the system; and
facilitate management of the system.
18. One or more computer readable media as recited in
19. One or more computer readable media as recited in
20. One or more computer readable media as recited in
21. One or more computer readable media as recited in
22. One or more computer readable media as recited in
23. One or more computer readable media as recited in
24. One or more computer readable media as recited in
25. An apparatus that implements a schema comprising:
means for facilitating design of a system;
means for facilitating deployment of the system; and
means for facilitating management of the system.
26. An apparatus as recited in
27. An apparatus as recited in
28. An apparatus as recited in
29. An apparatus as recited in
30. An apparatus as recited in
31. A system comprising:
a system definition model applicable across a lifecycle of an application, wherein the lifecycle of the application includes design of the application, deployment of the application, and management of the application; and
a schema to dictate how functional operations within the system definition model are to be specified.
32. A system as recited in
33. A system as recited in
34. A system as recited in
35. A system as recited in
36. A system as recited in
37. A system as recited in
38. A system as recited in
another system definition model applicable across a lifecycle of an environment, wherein the lifecycle of the environment includes design of the environment, deployment of the environment, and management of the environment; and
wherein the schema is further to dictate how functional operations within the other system definition model are to be specified.
39. A system as recited in
40. A system as recited in
41. A system as recited in
an additional system definition model applicable across a lifecycle of an additional environment, wherein the lifecycle of the additional environment includes design of the additional environment, deployment of the additional environment, and management of the additional environment;
wherein the additional environment is layered below the environment; and
wherein the schema is further to dictate how functional operations within the additional system definition model are to be specified.
42. A method comprising:
defining an instance of a system definition model that is used during design of a system, as well as in conjunction with a system definition model runtime during deployment and management of the system.
43. A method as recited in
44. A method as recited in
45. A method as recited in
46. A method as recited in
47. A method as recited in
[0001] This application claims the benefit of U.S. Provisional Application No. 60/452,736, filed Mar. 6, 2003, entitled "Architecture for Distributed Computing System and Automated Design, Deployment, and Management of Distributed Applications", which is hereby incorporated herein by reference.
[0002] This application is related to the following US patent applications (all of which are incorporated by reference herein):
[0003] U.S. patent application Ser. No. 10/382,942, filed on Mar. 6, 2003, titled "Virtual Network Topology Generation", which is hereby incorporated herein by reference.
[0004] U.S. patent application Ser. No. 09/695,812, filed on Oct. 24, 2000, titled "System and Method for Distributed Management of Shared Computers", which is hereby incorporated herein by reference.
[0005] U.S. patent application Ser. No. 09/695,813, filed on Oct. 24, 2000, titled "System and Method for Logical Modeling of Distributed Computer Systems", which is hereby incorporated herein by reference.
[0006] U.S. patent application Ser. No. 09/695,820, filed on Oct. 24, 2000, titled "System and Method for Restricting Data Transfers and Managing Software Components of Distributed Computers", which is hereby incorporated herein by reference.
[0007] U.S. patent application Ser. No. 09/695,821, filed on Oct. 24, 2000, titled "Using Packet Filters and Network Virtualization to Restrict Network Communications", which is hereby incorporated herein by reference.
[0008] U.S. patent application Ser. No. 09/696,707, filed on Oct. 24, 2000, titled "System and Method for Designing a Logical Model of Distributed Computer System and Deploying Physical Resources According to the Logical Model", which is hereby incorporated herein by reference.
[0009] U.S. patent application Ser. No. 09/696,752, filed on Oct. 24, 2000, titled "System and Method Providing Automatic Policy Enforcement in a Multi-Computer Service Application", which is hereby incorporated herein by reference.
[0010] The invention relates to an architecture for a distributed computing system.
[0011] Internet usage has exploded over the past several years and continues to grow. People have become very comfortable with many services offered on the World Wide Web (or simply "Web"), such as electronic mail, online shopping, gathering news and information, listening to music, viewing video clips, looking for jobs, and so forth. To keep pace with the growing demand for Internet-based services, there has been tremendous growth in the computer systems dedicated to hosting Websites, providing backend services for those sites, and storing data associated with the sites.
[0012] One type of distributed computer system is a data center (such as an Internet data center (IDC) or an Enterprise Data Center (EDC)), which is a specifically designed complex that houses many computers for hosting network-based services. Data centers, which may also go by the names of "Webfarms" or "server farms", typically house hundreds to thousands of computers in climate-controlled, physically secure buildings. Data centers typically provide reliable Internet access, reliable power supplies, and a secure operating environment.
[0013] Today, large data centers are complex and often called upon to host multiple applications. For instance, some websites may operate several thousand computers, and host many distributed applications. These distributed applications often have complex networking requirements that require operators to physically connect computers to certain network switches, as well as manually arrange the wiring configurations within the data center to support the complex applications. As a result, this task of building physical network topologies to conform to the application requirements can be a cumbersome, time consuming process that is prone to human error. Accordingly, there is a need for improved techniques for designing and deploying distributed applications onto the physical computing system.
[0014] Integrating design, deployment, and management phases for systems is described herein.
[0015] In accordance with certain aspects, a system definition model is used to design a system. The system definition model is subsequently used to deploy the system on one or more computing devices. After deployment of the system, the system definition model is used to manage the system deployed on the one or more computing devices.
[0016] The same numbers are used throughout the drawings to reference like features.
[0017]FIG. 1 illustrates an example network setting.
[0018]FIG. 2 is a block diagram illustrating an example architecture using the SDM definition model.
[0019]FIG. 3 illustrates an example layered setting.
[0020]FIG. 4 is a flowchart illustrating an example process for using the system definition model (SDM) across the entire lifecycle of a system.
[0021]FIG. 5 illustrates an example architecture using an SDM runtime.
[0022]FIG. 6 illustrates an example SDM document.
[0023]FIG. 7 illustrates an base definition and members.
[0024]FIG. 8 illustrates an example member.
[0025]FIG. 9 illustrates example setting values and value lists.
[0026]FIG. 10 illustrates an example lifecycle of an SDM application in accordance with certain embodiments.
[0027]FIG. 11 shows an example mapping of a web application to a web server host.
[0028]FIG. 12 illustrates an example built-in datatype hierarchy.
[0029]FIG. 13 illustrates an example of implicit extension of an abstract object definition.
[0030]FIG. 14 illustrates an example of implicit extension of an abstract relationships.
[0031]FIG. 15 illustrates an example of a change request.
[0032]FIG. 16 illustrates an example process of loading new definitions into the runtime.
[0033]FIG. 17 illustrates an example of carrying out change requests.
[0034]FIG. 18 illustrates examples of connected members.
[0035]FIG. 19 illustrates example structures with regard to connections.
[0036]FIG. 20 illustrates an example UML diagram that provides an overview of the instance space.
[0037]FIG. 21 illustrates a general computer environment which can be used to implement the techniques described herein.
[0038] The following disclosure describes a number of aspects pertaining to an architecture for designing and implementing a distributed computing system with large-scale application services. The disclosure includes discussion of a system definition model (SDM), which may also be referred to as a service definition model (SDM), and an SDM runtime environment. The SDM provides tools and a context for an application architect to design distributed computer applications and data centers in an abstract manner. The model defines a set of elements that represent functional units of the applications that will eventually be implemented is by physical computer resources and software. Associated with the model elements is a schema that dictates how functional operations represented by the components are to be specified.
[0039] As used herein, the term "wire" may also be referred to as "connections", "communication", or "communication relationship". Also, the term "system" may be referred to as "module" and the term "resource space" may be referred to as "resources". Additionally, the term "application space" may also be referred to as "applications", and the term "instance space" may also be referred to as "instances". Further, the term "class" may also be referred to as "abstract definition", the term "port" may also be referred to as "endpoint", and the term "type" may also be referred to as "definition".
[0040]FIG. 1 illustrates an example network 100. In setting 100, multiple (x) computing devices 102(1), 102(2), . . . , 102(x) are coupled to a network 106. Network 106 is intended to represent any of a variety of conventional network topologies and types (including wire and/or wireless networks), employing any of a variety of conventional network protocols (including public and/or proprietary protocols). Network 106 may include, for example, a local area network (LAN), a wide area network (WAN), portions of the Internet, and so forth. Setting 100 represents any of a wide variety of settings, including, for example, data centers (e.g., Internet data centers (IDCs)), office or business settings, home settings, educational or research facilities, retail or sales settings, data storage settings, and so forth.
[0041] Computing devices 102 can be any of a variety of conventional computing devices, including desktop PCs, workstations, mainframe computers, server computers, Internet appliances, gaming consoles, handheld computers, cellular telephones, personal digital assistants (PDAs), etc. One or more of devices 102 can be the same types of devices, or alternatively different types of devices. Additionally, even if multiple devices are the same types of devices, the multiple devices may still be configured differently (e.g., two devices 102 may be server computers, but may have different hardware configurations, such as different processors, different amounts of RAM, different sizes of hard disk drives, and so forth).
[0042] One or more computing devices 102 may also be re-configured after being added to setting 100. For example, a particular computing device 102 may operate for a period of time (e.g., on the order of minutes, hours, days, months, etc.) performing one function, and then an administrator may decide that a different function is desirable (e.g., change from being a server computer to a workstation computer, from a web server to a local file server, etc.).
[0043]FIG. 2 is a block diagram illustrating an example architecture 200 using the system definition model. The SDM is designed to be used across the entire lifecycle of a system. A system is a set of related software and/or hardware resources that can work together to accomplish a common function. One example of such a system is an application, which refers to a set of instructions that can be run or executed by a computing device to perform various functionality. Examples of applications include entertainment applications such as games, productivity applications such as word processors, reference applications such as electronic encyclopedias, distributed applications such as may be used for web services or financial analysis, and so forth. Another example of such a system is an environment on which an application (or another environment) can be deployed. An environment refers to the software and/or hardware resources on which an application (or another environment) is deployed. Such environments can be layered, as discussed in more detail below.
[0044] The lifecycle of a system typically includes three primary phases (also referred to as stages): a design or development phase, followed by a deployment or installation phase, followed by an operations or management phase. As the model applies to all three phases of the lifecycle of a system, the model can thus be seen as an integration point for the various phases in the lifecycle of a system, and facilitates each of these phases. Additionally, by using the model knowledge can be transferred between these phases, such as: knowledge regarding management of the system (e.g., being fed back to the design and development team, allowing the design and development team to modify the system, such as for future versions or to improve the performance of the current version); knowledge of the structure, deployment requirements and operational behavior of the system; knowledge of the operational environment from the desktop to the data center; knowledge of the service level as observed by the end user; and so forth.
[0045] Generally, during the design phase, development tools leveraging the SDM are used to define a system comprised of communicating software and hardware components. A system definition contains all information necessary to deploy and operate a distributed system, including required resources, configuration, operational features, policies, etc. During the deployment phase, the system definition is used to automatically deploy the system and dynamically allocate and configure the software and hardware (e.g., server, storage and networking) resources required. The same system definition can be used for deployments to different host environments and to different scales. During the management phase, an SDM Service in the operating system provides a system-level view for managing the system. This enables new management tools to drive resource allocation, configuration management, upgrades, and process automation from the perspective of a system.
[0046] The architecture 200 employs the SDM definition model as well as a schema that defines functional operations within the SDM definition model. The definition model includes various different kinds of data structures which are collectively referred to as "definitions". Functionality of the SDM is exposed through one or more platform services, such as application program interfaces (APIs).
[0047] During the design phase for a system, a development system 202 generates a document that contains the system definition, such as an SDM document 204. Development system 202 can be any of a variety of development systems, such as the Visual Studio� development system available from Microsoft� Corporation of Redmond, Wash. SDM document 204 defines all information (also referred to herein as knowledge) related to the deployment and management of the system. Any knowledge necessary for or used when deploying the system or managing the system is included in SDM document 204. Although described herein as a single document, it is to be appreciated that the knowledge could alternatively be spread out and maintained in multiple documents.
[0048] A system definition defines a system in terms of one or more of resources, endpoints, relationships and sub-systems. A system definition is declared in an SDM document (e.g., an XML document). Resources may be hardware resources or software resources. Endpoints represent communications across systems. Relationships define associations between systems, resources and endpoints. Sub-systems can be treated as complete systems and are typically part of a larger system.
[0049] A system definition captures the basic structure of a dynamic system. It can be viewed as the skeleton on which all other information is added. This structure is typically specified during the development process, by architects and developers, and typically does not change frequently. In addition to the structure, the SDM can contain deployment information, installation processes, schemas for configuration, events and instrumentation, automation tasks, health models, operational policies, etc. Other information can be added by the operations staff, by vendors, and/or by management systems across the lifetime of a distributed system.
[0050] SDM document 204 includes one or more constraints (also referred to as requirements) of the system that an environment in which the system is to be deployed and/or run must satisfy. The environment itself is also described using an SDM document. Such environments can be single computing devices, or alternatively collections of computing devices (e.g., data centers), application hosts, etc. Different systems can be installed to different environments. For example, a data center may include fifty computing devices, and one system may be deployed to five of those computing devices, while another system may be deployed to thirty five of those computing devices. These requirements can take a variety of forms, such as: hardware requirements regarding the computing device(s) on which the system is to be deployed (e.g., a minimum processor speed, a minimum amount of memory, a minimum amount of free hard drive space, a minimum amount of network bandwidth available, particular security mechanisms available, and so forth), software requirements regarding the computing device(s) on which the system is to be deployed (e.g., a particular operating system, one or more other applications that also must be installed, specifications regarding how a particular system and/or the operating system is to be configured, a particular type of security or encryption in use, and so forth), other requirements regarding the computing device(s) on which the system is to be deployed (e.g., particular security keys available, data center policies that must be enforced, authentication that is used, environment topology, etc.).
[0051] Requirements can also go in the other direction—that is, the environment can have constraints or requirements on the configuration of the system that is to be installed (e.g., to implement the standards or policies of the environment). These can be "explicit" requirements that are created by the operator of the environment, such as particular settings or configurations the system must have, particular functionality the system must provide or support, particular security mechanisms the system must support, and so forth. These can also be "implicit" requirements that that arise because of a particular configuration of the environment. For example, if a host computing device in the environment is using a particular type of file system then it may not be possible for some actions to be performed using that file system (although it may be possible for those same actions to be performed using another file system).
[0052] During the design and development phase of the system, SDM document 204 can be used to validate the system for one or more particular environment(s). This is a two-way validation: the system is validated for the environment and the environment is validated for the system. The environment can be validated for the system by comparing the requirements identified in the SDM document 204 with the environment and determining whether all of the requirements are satisfied by the environment. The system can be validated for the environment by comparing the requirements identified in an SDM document for the environment with the system and determining whether all of the requirements are satisfied by the system. If all of the requirements are satisfied by the environment and the system, then the designer or developer knows that the system can be deployed in and will run in the environment. However, if all of the requirements are not satisfied by the environment and/or the system, then the designer or developer is optionally informed of the requirements that were not satisfied, thereby informing the designer or developer of what changes should be made to the SDM document 204 (and correspondingly to the system) and/or to the environment in order for the system to be deployed and run in that environment.
[0053] The knowledge regarding deployment of the system that is included in the SDM document 204 describes how the system is to be deployed in one or more environments. The SDM document 204 is made available to a controller 206, which includes a deployment module 208 and a management module 210. In certain embodiment, the SDM document 204 as well as all of the files of the system (e.g., binaries, data, libraries, etc.) needed to install the system are packaged together into a single container (e.g., a single file) referred to as an SDU (System Definition Unit). Controller 206 can be one or more of computing devices 102 of FIG. 1. For example, a single device 102 of FIG. 1 may be the controller for a particular data center, or alternatively the controller responsibilities may be distributed across multiple devices 102.
[0054] Deployment module 208 includes services that are used to deploy the system in the environment(s). In FIG. 2, the environment in which the system is deployed is (or is deployed on) one or more target devices 212. Systems may also be deployed to controller 206. These services of deployment module 208 include one or more functions that can be called or invoked to install or deploy one or more systems in the environment.
[0055] Different knowledge for deployment in different environments may be included in the SDM document 204. This deployment knowledge describes any changes that need to be made to the in the environment (e.g., changes to a system registry; folders, directories, or files that need to be created; other setting or configuration parameters of the computing device that need to be set to particular values; and so forth), as well as what files (e.g., program and/or data files) that need to be copied to the computing device(s) in the environment and any operations that need to be performed on those files (e.g., some files may need to be decompressed and/or decrypted). In many implementations, the deployment knowledge in the SDM document 204 includes, for example, information analogous to that presently found in typical setup or installation programs for systems.
[0056] During the deployment process, controller 206 generates a record or store of the software and hardware resources involved in the deployment as well as the relationships between them. This record or store can subsequently be used by controller 206 during the management phase.
[0057] Management module 210 includes services that are used to manage the system once it is installed in the environment(s). These services of management module 210 include one or more functions that can be called or invoked to manage the systems in the environment. The knowledge regarding management of the system that is included in the SDM document 204 describes how the system is to be managed in one or more environments.
[0058] Different knowledge for managing a system in different environments may be included in the SDM document 204. The management knowledge includes any knowledge used in the management or operation of the system. Management involves, for example, configuration (and optionally subsequent reconfiguration), patching and upgrading, maintenance tasks (e.g., backup), health or performance monitoring, and so forth.
[0059] Changes to deployed systems are made through management module 210. The services of management module 210 include one or more functions that can be called or invoked to make changes to one or more systems deployed in the environment. By making such changes through the management module 210, several benefits can be realized. One such benefit is that controller 206 can maintain a record of the changes that have been made. Controller 206 may maintain a copy of the SDM document 204 for the system and record in the SDM document 204 any changes that are made to the system. Alternatively, controller 206 may maintain a separate record of the changes made to the system.
[0060] This record of changes maintained by controller 206 can simplify subsequent operations, such as solving problems with the system and/or environment, or when having to reinstall the system due to a hardware failure (allowing the system to be reinstalled and returned to running with the same parameters/settings as it had at the time of failure). By having such changes made through controller 206 and by having controller 206 maintain the record, some human error can be removed from the environment (e.g., if the administrator making the change is supposed to log the change in a book but forgets to do so there would be no record of the change—this problem is solved by having controller 206 maintain the record).
[0061] Furthermore, by making changes to systems through controller 206, as well as deploying systems through controller 206, controller 206 can serve as the repository of knowledge about the environment, the systems deployed in the environment, and interactions between them. Knowledge regarding the environment and/or systems deployed in the environment can be readily obtained from controller 206. This knowledge can be used to ensure the consistency of the controlled environment by validating that the controlled devices in the environment reflect the state stored in the central controller 206.
[0062] It should be noted that in some situations changes may be made to a system and/or environment but are not made through controller 206. For example, a computing device may be accidentally turned off or may fail. In these situations, attempts are made to reflect such changes in controller 206. These changes may be reflected in controller 206 automatically (e.g., a system may run that attempts to detect device failures and use the services of management module 210 to notify controller 206 of such failures) or may be reflected in controller 206 manually (e.g., an administrator may use the services of management module 210 to notify controller 206 of such changes). Alternatively, the changes that were made could be reversed to bring the system and/or portion of the environment back into line with the desired state of the system as recorded by controller 206.
[0063] The SDM document 204 can thus be viewed as a "live" document—it can be constantly changing based on changes to the environment and/or changes to the system throughout the lifecycle of the system.
[0064] The SDM enables the functional composition of systems across a horizontal and vertical axis. Composition along the horizontal axis is done with systems and subsystems. Composition along the vertical axis is done with "layers". Applications, services, network topologies, and hardware fulfill a role in a distributed system, but are typically defined independently and owned by different teams or organizations. Layering is accomplished by components defining a set of constraints on a host and vice versa.
[0065]FIG. 3 illustrates an example layered setting. Four layers are illustrated in FIG. 3: layer 302, layer 304, layer 306, and layer 308. Although four layers are shown in FIG. 3, the actual number of layers can vary, and can be greater or less than four. Additionally, the content of different layers can vary in different embodiments. As can be seen in FIG. 3, the different layers are situated above and/or below other layers (e.g., layer 306 is above layer 304 but below layer 308).
[0066] Different systems and subsystems within a layer can interact with one another, and also can interact with systems and subsystems of different layers. For example, a subsystem 310 in layer 308 can interact with a subsystem 312 in layer 308, as well as a subsystem 314 in layer 306. Additionally, each layer can be viewed as the environment for the next higher layer. For example layer 306 is the environment for systems and subsystems in layer 308, while layer 304 is the environment for systems and subsystems in layer 306. Each layer 302, 304, 306, and 308 has its own associated SDM document.
[0067] The different layers 302, 304, 306, and 306 can represent different content. In certain embodiments, layer 302 is a hardware layer, layer 304, is a network topology and operating systems layer, layer 306 is an application hosts layer, and layer 308 is an applications layer. The hardware layer represents the physical devices (e.g., computing devices) on which the layered system is built (e.g., devices 102 of FIG. 1). The network topology and operating systems layer represents the network topology of the computing devices (e.g., network setting 100 of FIG. 1) as well as the operating systems installed on those computing devices. The application hosts layer represents applications installed on the computing devices that can host other applications (e.g., SQL Server, IIS, and so forth). The application layer represents applications that are installed on the computing devices that do not host other applications (e.g., entertainment applications such as games, productivity applications such as word processors, reference applications such as electronic encyclopedias, distributed applications such as may be used for web services or financial analysis, and so forth).
[0068]FIG. 4 is a flowchart illustrating an example process 400 for using the SDM across the entire lifecycle of a system. The various acts of process 400 of FIG. 4 can be implemented in software, firmware, hardware, or combinations thereof.
[0069] Initially, the system is designed based on the SDM (act 402). The system is designed to include requirements that an environment(s) must satisfy in order for the system to be deployed and run in the environment, as well as additional knowledge that is used for the deployment and management of the system. This knowledge is included in an SDM document associated with the system. Once designed, the system can optionally be validated using the SDM (act 404). This validation allows the designer or developer to verify that the system will be able to be deployed and run in the environment being validated against. As discussed above, not only is the system validated against the environment in which it is to be deployed, but that environment is also validated against the system. If the validation fails for a particular environment, then additional design steps can be taken to alter the system so that the system can be run in that environment (or alternatively steps can be taken to alter the environment).
[0070] Once validated, the system can be deployed using the SDM (act 406). When deploying the system in an environment, the system is installed in that environment so that it can be subsequently run in the environment. The knowledge used to install the system is included in the SDM document associated with the system. Once deployed, the system is monitored and/or managed using the SDM (act 408). Knowledge in the SDM document associated with the system identifies how the system is to be monitored and/or managed, and the system is monitored and/or managed within the environment in accordance with this knowledge.
[0071] Process 400 of FIG. 4 can be used with systems that are applications as well as systems that are environments. For example, an application can be validated against the SDM of its environment (e.g., an application in layer 308 of FIG. 3 is validated against the environment of layer 306 of FIG. 3). By way of another example, an operator or system architect can design environments (e.g., layer 306 or layer 304 of FIG. 3) and validate those environments against the environments they are to be deployed in (e.g., layers 304 and 302, respectively).
[0072] The constraints on a system and/or the environment can also be used during runtime (while the system is being monitored and/or managed) to validate changes to the system and/or the environment during runtime. Such runtime validation allows, for example, an operator of an environment to determine how changes to the environment may affect a running system, or a system designer to determine how changes to the system may affect its running in the environment.
[0073] In the discussions to follow, reference is made to flow and setting flow with respect to the runtime. Flow is used to pass configuration information between parts of a distributed system (e.g., allowing a developer to specify the configuration information in one place or allowing an operator to only provide a single entry). Flow is also used to determine the impact of changes to configuration by following the flow of setting data between parts of the system.
[0074]FIG. 5 illustrates an example architecture 500 using an SDM runtime. Architecture 500 is an example of architecture 200 of FIG. 2 using an SDM runtime 510 as well as the example implementation of the SDM discussed below in the section "Example SDM Implementation". The SDM runtime 510 contains a set of components and processes for accepting and validating SDM files, loading SDUs (System Definition Units—which are packages of one or more SDM files and their related files), creating and executing SDM Change Requests and deploying SDM based systems into target environments. The runtime functionality allows systems described using the SDM to be defined and validated, deployed to a set of computing devices, and managed.
[0075] The SDM, which is discussed in more detail below in the section "Example SDM Implementation" is designed to support description of the configuration, interaction and changes to the components in a distributed system (the modeled system). SDM is based on an object-relational model. "Definitions" describe entities that exist in a system and "relationships" identify the links between the various entities. Definitions and relationships are further defined to capture semantic information relevant to the SDM. In particular, definitions are divided into components, endpoints and resources. Relationships are divided into the following: connections (also referred to as communication), containment, hosting, delegation and reference. Further details regarding definitions and relationships are provided below.
[0076] The SDM includes "abstract definitions" that provide a common categorization of system parts, provide tool support for a wide range of systems and provide the basis for definition checking at design time. A set of abstract definitions provide a comprehensive basis for service design. "Concrete definitions" represent parts of an actual system or data center design. A concrete definition is generated by selecting an abstract definition and providing an implementation that defines the concrete definition's members and setting values for its properties. Distributed applications are generated using collections of these concrete definitions.
[0077] The SDM also includes "constraints" that model restrictions based on the allowed set of relationships in which an instance of a relationship can participate. Constraints are useful in describing requirements that depend on the configuration of objects involved in a relationship. For example, a constraint may be used to determine whether participants on each end of a communication protocol are using compatible security settings.
[0078] In order to effect change on a target system, SDM uses a declarative description of the required changes called a "change request" or CR. SDM defines the process that is used to expand, validate and execute a change request as part of the "SDM execution model".
[0079] The "instance space" captures both the desired and current state of the managed application. Changes in the instance space are tracked and associated with the change request that initiated the change. The instance space is stored in an SDM runtime and reflects the current state of the modeled system. The runtime contains a complete record of the instances that have been created and the relationships between these instances. Each instance has an associated version history where each version is linked to a change request. The process of creating new instances is initiated by a change request. The change request defines a set of create, update and delete requests for definitions and relationships associated with specific members of an existing instance.
[0080] The following is a brief, functional discussion of how the components in FIG. 5 work together. An operator or administrator is able to describe an environment into which applications can be deployed, such as the topology of a data center. The operator or administrator produces an SDM file describing the environment, the file being referred to as the "logical infrastructure" (LIM) 502, or as a data center description or data center model. This SDM file can be generated using any of a variety of development systems, such as the Visual Studios development system available from Microsoft� Corporation of Redmond, Wash.
[0081] Additionally, an application developer is able to design and develop their application using any of a variety of development systems, such as the Visual Studio� development system. As the developer defines components of the application and how these components relate to one another, the developer is able to validate the application description against the datacenter description 502. This is also referred to as "Design Time Validation".
[0082] Once the application is complete, the developer saves the description in an SDM and requests that the application be packaged for deployment as an SDU 504. The SDU includes the application SDM as well as the application binaries and other referenced files used to install the application.
[0083] The LIM 502 and SDU 504 are fed to deployment tool 506 of a controller device 520 for deployment. Deployment tool 506 includes a user interface (UI) to enable an operator to load the desired SDU 504. Deployment tool 506 works with create CR module 530 to install the application associated with the SDU 504 in accordance with the information in the SDM within SDU 504. Additionally, SDM definitions and instances from SDU 504 are populated in a store 508 of the SDM runtime 510. SDUs are managed in SDM runtime 510 by SDU management module 540, which makes the appropriate portions of the SDUs available to other components of runtime 510 and target(s) 522.
[0084] The operator can also specify what actions he or she wants to take on the targets 522 (e.g., target computing devices) on which the application is being deployed. The operator can do this via a deployment file, which is also referred to herein as a Change Request (CR). The CR is run through one or more engines 512, 514, 516, and 518. Generally, expand CR engine 512 expands the CR to identify all associated components as well as their connections and actions, flow values engine 514 flows values for the components (such as connection strings), check constraints engine 516 checks constraints between the environment and the application, and order actions engine 518 specifies the order for all of the necessary actions for the CR.
[0085] To initiate change to the system (including deploying an application) or validation of a model, an operator or process submits a CR. The CR contains a set of actions that the operator wants performed over the instances in the runtime 510. These actions can be, for example, create actions, update actions, and/or delete actions.
[0086] In addition to user or operator initiated change requests, there may also be expansion/automatically generated change requests that are generated as part of the expansion process, discussed in more detail below. Regardless of their source, the change requests, once fully expanded and checked, are executed by sending actions to the targets 522, such as: discover, install, uninstall and change a target instance.
[0087] The CR is treated as an atomic set of actions that complete or fail as a group. This allows, for example, the constraint checking engine 516 to consider all actions when testing validity.
[0088] In design time validation, the CR will be created by the SDM Compiler 528 and will contain one or the minimum of each SDM component in the SDM file. This CR of create instance commands will flow through the expansion engine 512, the flow values engine 514, and the constraint checking engine 516. Errors found in these three phases will be returned to the user via the development system he or she is using.
[0089] In deployment, the operator will create a CR with the UI presented by deployment tool 506. The CR will flow through all the engines 512, 514, 516, and 518 in the SDM runtime 510, and the appropriate actions and information will be sent by CR module 532 to the appropriate target(s) 522, where the request is executed (e.g., the application is installed). The appropriate target(s) 522 for a particular installation are typically those target(s) on which the application is to be installed.
[0090] When beginning to process a CR, in a definition resolution phase, create CR module 530 resolves all definitions and members that are referenced in the change request. The change request will assume that these are already loaded by the runtime 510; create CR module 530 initiates a load/compile action if they do not exist. Create CR module 530 also implements a path resolution phase where references to existing instances and instances defined by create actions within the change request are resolved.
[0091] The expansion performed by expansion engine 512 is a process where, given a change request, all the remaining actions required to execute the request are populated. In general, these actions are construction and destruction actions for definition and relationship instances. The operator could optionally provide details for all the actions required to construct or destroy an instance, or alternatively portions of the process can be automated: e.g., the operator provides key information about the changes he or she wants by identifying actions on members (e.g., byReference members), and the remainder of the actions are filled in on nested members (e.g., byReference and byValue members) and relationships. By way of another example, automated expansion can also refer to external resource managers that may make deployment decisions based on choosing devices with available resources, locating the application close to the data it requires, and so forth.
[0092] Expansion engine 512 also performs "auto writing". During auto writing, engine 512 analyzes the scale invariant grouping of components and compound components specified in the SDM and determines how the components should be grouped and interconnected when scaled to the requested level.
[0093] Expansion engine 512 also performs value member expansion, reference member expansion, and relationship expansion.
[0094] Value member expansion refers to identification of all of the non-reference definition members. The cardinality of these members are noted and, since all the required parameters are known, for each member create requests are added to the change request for those members whose parent is being created. If the change request contains destruction operations, then destruction operations are added for all their contained instances.
[0095] Reference member expansion refers to reference members (as opposed to non-reference definition members). The cardinality of reference members is often undefined and they can have deployment time settings that require values in order for the instance to be constructed. So the process of expanding a reference member (e.g., a byReference member) can require more information about the instance than the runtime is in a position to provide.
[0096] Related to reference member expansion is a process referred to as discovery, which is a process used to find instances that have already been deployed. Discovery is an action typically initiated by an operator of the environment. For example, during an install request, expansion engine 512 determines if the instance already exists, if so determines what exists and if not then creates it. An instance manager (IM) 534 on the controller 520 communicates with the instance managers 526 on the target device 522 to initiate a discovery process. The discovery process returns data regarding the instance from the target device 522 to the controller 520.
[0097] The process of discovery populates reference definition members as part of a construction or update action. Typically, only reference members with object managers (instance managers that also do discovery) that support discovery participate in this process.
[0098] When a new instance is discovered a check is made that the instance does not already exist in the SDM database using instance specific key values. Once it is known that it is a new instance, the instance is classified according to the definitions of the members being discovered. If the instance does not match a member or there is an ambiguous match then the member reference is left blank and the instance is marked as offline and incomplete.
[0099] Relationship expansion refers to, once all the definition instances that will be constructed are known, creating relationship instances that bind the definition instances together. If definition instances are being destroyed, all relationship instances that reference the definition instances are removed.
[0100] To create the relationships the member space is used to identify the configurations of the relationships that should exist between the instances. Where the definition members have cardinality greater than one the topology of the relationships is inferred from the base relationship definition. For example, for communication relationship an "auto wiring" can be done, and for host relationships a host is picked based on the algorithm associated with the hosting relationship.
[0101] During a flow stage, flow values engine 514 evaluates flow across all the relationship instances. Flow values engine 514 may add update requests to the change request for instances that were affected by any altered parameter flow. Engine 514 evaluates flow by determining the set of instances that have updated settings as a result of the change request. For each of these, any outgoing settings flows that depend on the modified settings are evaluated and the target nodes added to the set of changed instances. The process continues until the set is empty or the set contains a cycle.
[0102] After the flow stage, a process of duplicate detection is performed. The duplicate detection may be performed by one of the engines illustrated in FIG. 5 (e.g., flow values engine 514 or check constraints engine 516), or alternatively by another engine not shown in FIG. 5 (e.g. a duplicate detection engine may be included in SDM runtime 510). The process of duplicate detection matches expanded instances against instances that already exist in the SDM data store. For example, the process detects if another application has installed a shared file. When an instance that already exists is detected, one of several actions can be taken depending on the version of the existing instance: the install can be failed; the instance can be reference counted; the instance can be upgraded; or the installation can be performed side-by-side.
[0103] Check constraints engine 516 implements a constraint evaluation phase in which all the constraints in the model are checked to see if they will still be valid after the change request has been processed.
[0104] After check constraints engine 516 finishes the constraint evaluation phase, a complete list of actions is available. So, order actions engine 518 can use the relationships between components to determine a valid change ordering. Any of a variety of algorithms can be used to make this determination.
[0105] Once order actions engine 518 is finished determining the ordering, deployment can be carried out by distributing subsets of the ordered set of actions that are machine specific. Once the actions have been ordered and grouped by machine, the actions as well as a copy of the necessary portion of the SDM runtime store 508 with instance information are sent to a target computing device 522. The SDM can be stored temporarily at the target device in a store cache 538.
[0106] The target computing device includes a target portion 536 of the SDM runtime that communicates with SDM runtime 510. The target computing device 522 also includes an agent that contains an execution engine 524 and can communicate with the appropriate instance managers (IMs) 526 on the target device to make changes on the target, such as create, update, and delete actions. Each action is sent as an atomic call to the instance manager 526 and the instance manager 526 returns a status message and for some actions, also returns data (e.g., for discovery). Once all the actions are completed on target 522, the target's agent returns any errors and status to the controller 520. The controller 510 then uses this information to update the SDM runtime store 508.
[0107] As discussed above, change is carried out by breaking the change requests down into distributable parts based on the relationships that are affected. Once all the parts are completed (or after one or more has failed) the results are collated in the runtime 510 and a summary returned to the operator. In the event of a failure, all the actions can be "rolled back" and the system returned to the state it was in before the change was initiated.
[0108] In certain embodiments, during design time validation discussed above, an SDM Compiler 528 receives an SDM file, creates a test CR, runs the test CR through the expand, flow values and check constraints engines of the SDM runtime, and returns any errors to the development system. This process provides SDM validation for deployment during design time for the developer.
[0109] The public interface to SDM runtime 510 and/or controller 520 is through an object model (APIs) library. The library is a managed code object model and allows the following to be performed:
[0110] Manage the SDMs in the runtime—SDM files can be loaded into the runtime. SDMs are immutable and are loaded one at a time (i.e., an SDM file can be loaded rather than only parts of the file (e.g., individual ones of the individual definitions, classes or mappings from the SDM file)). SDMs can be deleted from the runtime and an XML document for an SDM in the runtime can be produced.
[0111] Manage the SDUs known by the runtime.
[0112] Manage SDM definitions—find and reflect on SDM elements (from an SDM loaded in the runtime). There is no public API provided for authoring a new SDM (i.e., this is a read only object model over the immutable elements of the SDM). This includes SDMs, SDUs, identities, versions, classes, definitions, binding/mappings and versioning policy.
[0113] Manage SDM instances—find and reflect on instances of components, endpoints, resources and relationships. In the instance space each instance can be identified by a GUID, a stable path or an array based path. The paths are strings and can be relative. These identifiers, including relative paths allows instances to be found and referenced in documents such as the change request document.
[0114] Manipulate instances—make changes to SDM instances, including creating, changing topology, upgrading, changing settings and deleting. Instance changes are made within the bounds of a change request which provides an atomic unit of update so that any errors or constraint violations will result in the entire request failing. Instance requests also allow for instances to exist temporarily without a binding to a host, as an instance must have a host when the request is committed. It also allows for many operations that will affect a single component's installation or settings to be performed and have the installation or settings update deferred until commit so that a single update occurs on the component. The SDM model checking is performed prior to or at change request commit time and the commit will fail on any model or constraint violations.
[0115] Load a change request—a change request is a document, for example an XML file, that represents a set of instance space operations. This document can take advantage of relative paths to be a reusable 'script' for creating or deleting application instances.
[0116] Find and reflect on change requests—including getting the installation/update tasks and all error information, and retrying the installation/update of components affected by the request.
[0117] Generate a change request document from a change request in the database. Such documents are somewhat portable.
[0118] Subscribe to events on change request tasks, such as progress, log or status updated. The lifetime of these event subscriptions limited by the lifetime of the process that loaded the client library (i.e. these are regular CLR events).
[0119] The SDM runtime engine performs the reasoning on the SDM model and the functions surfaced by the APIs. The library communicates to the runtime engine as a web service with fairly coarse calls such as load SDM, create component instance and get entire SDM (for reflecting on SDM entities). The format of many of the parameters for this web service is XML with the same schema for SDM files. The engine may also perform checks on permissions.
[0120] The controller 520 can make use of Instance Managers (IMs), which can be associated with any definition or relationship in the model. IMs may perform one or more of the following roles:
[0121] Support deployment of the instance.
[0122] Support validation of the instance once it has been deployed (auditing).
[0123] Support discovery of already deployed instances that were not deployed through the runtime.
[0124] Support flow of setting values.
[0125] Support evaluation of constraints.
[0126] Support expansion of a change request.
[0127] Support presentation of the instance to a user as a CLR class through the API.
[0128] For deployment, an instance manager (IM) plug-in on controller 520 is associated with a class host relation and is separate from the plug-in used in the development system that provides the design experience for the classes and produces the associated binaries in the SDU 504 and the settings schema. Instance managers are supplied to the SDM runtime 510 as CLR classes (e.g., in a dII assembly) that implement an instance manager interface or inherit from abstract class. An SDM Instance Manager, also referred to as an Instance Manager (IM) plug-in, provides the following functions to the controller 520:
[0129] Generates the files and commands (tasks) to install, uninstall or reinstall component instances on their hosts—When a change request results in a new component instance, removal of a component instance or a change to a component that requires an uninstall and reinstall, it is the instance manager that takes the settings for the instance, the host instance, the definitions associated with the component and the binaries associated with those definitions in the SDU 204 and produces the files and commands needed to perform the install or uninstall on a target server ready for either manual execution or dispatch via the deployment engine.
[0130] Generates the files and commands (e.g., tasks) to update a component instance when its settings change or when the view from one of its endpoints changes (e.g., due to communication relationship topology changes or a visible endpoint has settings change)
[0131] Maps the endpoint instances visible on a component instance's endpoints to settings on component instance—In the SDM a component instance has endpoint instances that, as a result of some communication relationship topology, can see other endpoint instances. The details of the other endpoint instances are mapped to settings that the component instance can fetch at runtime, usually so that it can bind to it. For example, a web site may have a database client endpoint instance so a communication relationship can be established with a database. When correctly established its database client endpoint is able to see a single database server endpoint instance and the settings on that server endpoint. This information is used by the instance manager to place a connection string for the server in a configuration file under the name of the client endpoint. The end result is that code simply reads the connection string for the database from its configuration settings.
[0132] Generates the files and commands (tasks) to audit a component instance—Auditing confirms existence, correct settings. This may apply to host instance settings also.
[0133] For any task will report status—The IM will translate the output captured, either partial or complete, and provide the status of the task as success, failure or incomplete and optionally offer progress on incomplete (% or last response), details on failure (error message) and a human readable log on any status. By going back to the instance manager to interpret the output of a task, the instance manager is free to have its tasks log structured information (for example, as XML or even SOAP) rather than trying to have to produce sufficient logging for diagnosis while keeping it human readable.
[0134] The instance managers may also provide code that does the constraint checking between hosts and their guests. Installers may use a common constraint language, for example based on XML, XPath and XQuery.
[0135] Example SDM Implementation
[0136] The following discussion describes an embodiment of the schema that defines the elements of the SDM.
[0137] Example Computer Environment
[0138]FIG. 21 illustrates a general computer environment 600, which can be used to implement the techniques described herein. The computer environment 600 is only one example of a computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the computer and network architectures. Neither should the computer environment 600 be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary computer environment 600.
[0139] Computer environment 600 includes a general-purpose computing device in the form of a computer 602. Computer 602 can be, for example, a computing device 102 of FIG. 1, or implement development system 202 or be a controller 206 of FIG. 2, or be a target device 212 of FIG. 2, or be a controller 520 or target 522 of FIG. 5. The components of computer 602 can include, but are not limited to, one or more processors or processing units 604, a system memory 606, and a system bus 608 that couples various system components including the processor 604 to the system memory 606.
[0140] The system bus 608 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, such architectures can include an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnects (PCI) bus also known as a Mezzanine bus.
[0141] Computer 602 typically includes a variety of computer readable media. Such media can be any available media that is accessible by computer 602 and includes both volatile and non-volatile media, removable and non-removable media.
[0142] The system memory 606 includes computer readable media in the form of volatile memory, such as random access memory (RAM) 610, and/or non-volatile memory, such as read only memory (ROM) 612. A basic input/output system (BIOS) 614, containing the basic routines that help to transfer information between elements within computer 602, such as during start-up, is stored in ROM 612. RAM 610 typically contains data and/or program modules that are immediately accessible to and/or presently operated on by the processing unit 604.
[0143] Computer 602 may also include other removable/non-removable, volatile/non-volatile computer storage media. By way of example, FIG. 21 illustrates a hard disk drive 616 for reading from and writing to a non-removable, non-volatile magnetic media (not shown), a magnetic disk drive 618 for reading from and writing to a removable, non-volatile magnetic disk 620 (e.g., a "floppy disk"), and an optical disk drive 622 for reading from and/or writing to a removable, non-volatile optical disk 624 such as a CD-ROM, DVD-ROM, or other optical media. The hard disk drive 616, magnetic disk drive 618, and optical disk drive 622 are each connected to the system bus 608 by one or more data media interfaces 626. Alternatively, the hard disk drive 616, magnetic disk drive 618, and optical disk drive 622 can be connected to the system bus 608 by one or more interfaces (not shown).
[0144] The disk drives and their associated computer-readable media provide non-volatile storage of computer readable instructions, data structures, program modules, and other data for computer 602. Although the example illustrates a hard disk 616, a removable magnetic disk 620, and a removable optical disk 624, it is to be appreciated that other types of computer readable media which can store data that is accessible by a computer, such as magnetic cassettes or other magnetic storage devices, flash memory cards, CD-ROM, digital versatile disks (DVD) or other optical storage, random access memories (RAM), read only memories (ROM), electrically erasable programmable read-only memory (EEPROM), and the like, can also be utilized to implement the exemplary computing system and environment.
[0145] Any number of program modules can be stored on the hard disk 616, magnetic disk 620, optical disk 624, ROM 612, and/or RAM 610, including by way of example, an operating system 626, one or more application programs 628, other program modules 630, and program data 632. Each of such operating system 626, one or more application programs 628, other program modules 630, and program data 632 (or some combination thereof) may implement all or part of the resident components that support the distributed file system.
[0146] A user can enter commands and information into computer 602 via input devices such as a keyboard 634 and a pointing device 636 (e.g., a "mouse"). Other input devices 638 (not shown specifically) may include a microphone, joystick, game pad, satellite dish, serial port, scanner, and/or the like. These and other input devices are connected to the processing unit 604 via input/output interfaces 640 that are coupled to the system bus 608, but may be connected by other interface and bus structures, such as a parallel port, game port, or a universal serial bus (USB).
[0147] A monitor 642 or other type of display device can also be connected to the system bus 608 via an interface, such as a video adapter 644. In addition to the monitor 642, other output peripheral devices can include components such as speakers (not shown) and a printer 646 which can be connected to computer 602 via the input/output interfaces 640.
[0148] Computer 602 can operate in a networked environment using logical connections to one or more remote computers, such as a remote computing device 648. By way of example, the remote computing device 648 can be a personal computer, portable computer, a server, a router, a network computer, a peer device or other common network node, and the like. The remote computing device 648 is illustrated as a portable computer that can include many or all of the elements and features described herein relative to computer 602.
[0149] Logical connections between computer 602 and the remote computer 648 are depicted as a local area network (LAN) 650 and a general wide area network (WAN) 652. Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets, and the Internet.
[0150] When implemented in a LAN networking environment, the computer 602 is connected to a local network 650 via a network interface or adapter 654. When implemented in a WAN networking environment, the computer 602 typically includes a modem 656 or other means for establishing communications over the wide network 652. The modem 656, which can be internal or external to computer 602, can be connected to the system bus 608 via the input/output interfaces 640 or other appropriate mechanisms. It is to be appreciated that the illustrated network connections are exemplary and that other means of establishing communication link(s) between the computers 602 and 648 can be employed.
[0151] In a networked environment, such as that illustrated with computing environment 600, program modules depicted relative to the computer 602, or portions thereof, may be stored in a remote memory storage device. By way of example, remote application programs 658 reside on a memory device of remote computer 648. For purposes of illustration, application programs and other executable program components such as the operating system are illustrated herein as discrete blocks, although it is recognized that such programs and components reside at various times in different storage components of the computing device 602, and are executed by the data processor(s) of the computer.
[0152] Various modules and techniques may be described herein in the general context of computer-executable instructions, such as program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments.
[0153] An implementation of these modules and techniques may be stored on or transmitted across some form of computer readable media. Computer readable media can be any available media that can be accessed by a computer. By way of example, and not limitation, computer readable media may comprise "computer storage media" and "communications media." "Computer storage media" includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer.
[0154] "Communication media" typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as carrier wave or other transport mechanism. Communication media also includes any information delivery media. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media. Combinations of any of the above are also included within the scope of computer readable media.
[0155] Alternatively, portions of the framework may be implemented in hardware or a combination of hardware, software, and/or firmware. For example, one or more application specific integrated circuits (ASICs) or programmable logic devices (PLDs) could be designed or programmed to implement one or more portions of the framework.
[0156] Conclusion
[0157] Although the invention has been described in language specific to structural features and/or methodological acts, it is to be understood that the invention defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as exemplary forms of implementing the claimed invention.
Referenced by
Legal Events
Original Image


text:
The Securities and Exchange Commission voted 5-0 to propose a rule amendment that would prohibit flash-order trading. The SEC will seek public comment on the proposal for 60 days. Nasdaq OMX Group's Nasdaq Stock Market and privately held BATS Exchange recently ended the practice of flashing marketable orders.
View Full Article in:Reuters · Wall Street Journal, The
Published in Brief:

subset: commoncrawl


text:
And the award goes to… Sprint for the "Sustainability Leadership Award." For the third consecutive year, the International Electronics Recycling Conference and Expo (IERCE) has recognized Sprint for its commitment to sustainability and recycling.
Conference leaders presented the "Sustainability Leadership Award" to Sprint during their event on May 30 in Las Vegas.
"Sprint sets the bar high when it comes to handling electronic waste and continues to push the industry forward in best practices in this space," said Ismail Oyekan, publisher of Electronic Waste Journal and program director of IERCE.
Sprint is committed to being a leader in driving more sustainable practices. This award recognizes the company for its outstanding electronic waste, or e-waste, management program, which focuses on the full lifecycle of the electronics it buys and sells.
Sprint's Electronic Stewardship Policy makes it the only wireless carrier to outline specific commitments that address e-waste holistically. E-waste management includes:
• Designing and procuring eco-friendly electronics
• Maximizing equipment's useful life
• Boosting collection of equipment
• Maximizing reuse of electronics
• Responsibly recycling electronic scrap
• Using electronically and socially responsible recycling vendors
This is the latest example of how Sprint is gaining momentum with its efforts to "Go Green" and focus on being more environmentally sustainable.
Other notable recent achievements include the highest ranking among all U.S. telecom companies on Newsweek's 2011 Rankings of America's Greenest Companies at number three (up from number six in 2010), as well as recognition from the Environmental Protection Agency (EPA) for our commitment to phone recycling.
More information:
Want to learn more about Sprint programs that protect the environment? Go to http://www.sprint.com/responsibility/ or follow @SprintGreenNews on Twitter.


text:
Metrolyrics is featuring James Durbin's new lyric video for 'Santa Claus is Back in Town'! Check it out for yourself...
Metrolyrics is featuring James Durbin's new lyric video for 'Santa Claus is Back in Town'! Check it out for yourself...
Civil Twilight recently sat down with Baeble Music to talk about theirlatest music video for "River". Check out the feature & interview they posted today!
Young Guns sits down with Last.fm to talk about their album, "Bones".
The Virginmarys' Ally Dickaty recently talked shop with Alternative Addition...read the interview HERE!
A brand new video of an acoustic version of The Virginmarys' "Dead Man's Shoes", shot in New York by Bubba Thrash, is now being exclusively premiered by AOL/Spinner.
The Virginmarys are currently featured on iTunes as part of their 'Rock's Best Power Trios'
Evanescence's track, "Call Me When You're Sober" is featured on iTunes as part of their "00's Hits" spotlight for $.69 - grab it while you can!
We're happy to announce Amazon has selected The Darkness' latest album, "Hot Cakes" as part of their "Best Music of 2012" feature!
Check out the feature here and buy "Hot Cakes" on Amazon HERE!
Just enter your email address below, and Jillette Johnson's song, "Pauvre Coeur" is all yours, for FREE!
Want to know what tunes have become hits with the Wind-up staff lately? Take a listen to our Spotify playlist and get familiar with some of our favorite songs and albums from 2012!
During Aranda's last trip to NYC, they spent some time doing a new photo-shoot. During the process, Loudwire got some behind-the-scenes footage...and here it is!
We have some rockin' playlists featured on eMusic and Spotify to celebrate our 15th Anniversary and give you guys a taste of some of the newest and best sounds on the Wind-up roster!
Head to eMusic or Spotify to give these awesome playlists a listen!


text:
The venue for social activist Anna Hazare's fast and dharna here was finally decided as Azad Maidan, volunteers of 'India Against Corruption' said here on Thursday, even as they informed that they had moved the Bombay High Court seeking "right to hold reasonable protest".
"The protest will be held at Azad Maidan since the Chief Minister has refused to grant any concession to us. Even Anna ji refused to allow any of the volunteers to pay for the charges of the ground," Preeti Sharma-Menon, a volunteer of IAC told The Hindu.
Team Anna had earlier taken permission to hold the protest at the MMRDA [Mumbai Metropolitan Region Development Authority] ground at Bandra-Kurla Complex, but they would have had to pay charges of nearly Rs 19 lakh for the three-day dharna protest starting from December 27.
They had sought concession from the State government for the same, but they said it was refused as they are not a registered entity.
"We have filed a writ petition in the court against MMRDA and the State government requesting the court to intervene as the State government wants to usurp prohibitive costs for the protest. Our Constitutional right to protest is being affected due to it. They are levying commercial rates on us. If my organisation is not registered, am I not entitled to reasonable protest?" Praful Vora, volunteer of IAC, said.
The petition is expected to come up for hearing on Friday.
Mr. Vora said that the police had given them permission for Anna Hazare's fast and dharna at Azad Maidan here. But none of the senior police officials confirmed that the protest had been granted permission.
Despite repeated attempts, no senior police official clarified if any permission was granted to the protest and what preconditions were laid for it. Highly placed sources in the Mumbai Police however told The Hindu that the protesters had not been granted permission.
"Mumbai police have agreed to give permission. IAC will meet all the required compliances," a note issued by IAC stated.
Keywords: Lokpal Bill, India against corruption, Team Anna


text:
Career Jam
Are you a middle school or high school student thinking about your career options? Then come to Career Jam on Thursday, October 30 from 4pm to 7pm at the Morris Lawrence Building at Washtenaw Community College. Discover the exciting career options and job opportunities here in Washtenaw County. There will be plenty of hands-on demonstrations to give you a taste of what your future could hold. Door prizes and a limited number of t-shirts will be available for youth who participate. For more information, contact the Teen Center Without Walls or ETCS.


text:
Affluent Lifestyle Guide
You might like Best Runway Models >
Here is something to squeak about. If this drug works on humans the same as it did on mice, automatic weight loss … with no exercising … would be possible.
In a study on weight loss at the University of Michigan, mice that were injected with amlexanox, a drug used to treat mouth sores, kept eating the same amount of food and lost weight anyway. The weight returned to the mice when the injections stopped.
CNN reports that studies on human beings will begin later this year. Although many drugs that work on mice do not work on humans, the doctors at the university remain hopeful.
"It is a tour de force and offers a new and potentially exciting opening for developments of new anti-obesity drugs -- something which is badly needed," says Dr. George Bray, chief of the Division of Clinical Obesity and Metabolism at Pennington Biomedical Research Center at Louisiana State University.
Four Tech Trends to Watch for in 2014Eco-Friendly & Luxury Cars Drive 2014 San Diego International Auto ShowJustLuxe Presents Vacation Fitness: Anywhere, Anytime. No Gear RequiredEuro Currency 2013's Best-Performing Currency per FOREXHow Elon Musk Knows if Someone Is Lying During a Job InterviewZappos to Zap All Job Titles in 2014


text:
To embed, copy and paste the code into your website or blog:
The United States Supreme Court held Monday in PPL Corporation v. Commissioner (No. 12-43) that a U.S. taxpayer was entitled to claim a foreign tax credit on its share of a "windfall tax" imposed on the taxpayer's United Kingdom (U.K.) subsidiary. Applying a "commonsense approach that considers the substantive form of the tax," the Court found the windfall tax creditable because its "predominant character" "is that of an excess profits tax, a category of income tax in the U.S. sense." In so doing, the Court reaffirmed the principle that substance, not form, should determine the characterization of a given tax transaction, and provided some welcome clarity for taxpayers and practitioners. Sutherland lawyers along with other amici filed briefs urging the Court to review the case and address the applicability of the substance over form doctrine.
The PPL opinion conclusively resolves the creditability of the windfall tax at issue and reaffirms the substance over form doctrine, but the impact it will have on the broader creditability question is not clear. While the majority opinion reaffirms the role of substance, not form, in determining the predominant character of a tax, Justice Sotomayor's concurring opinion raises the question of whether the "predominant character" test requires that it have the same character for all taxpayers in order to satisfy the test. Whether this prompts the Treasury Department to consider clarifying amendments to its regulations on this point remains to be seen. The decision also raises the possibility that Treasury will promulgate new regulations specifically addressing excess profits taxes as a subset of income taxes.
Please see full alert below for more information.
Firefox recommends the PDF Plugin for Mac OS X for viewing PDF documents in your browser.
We can also show you Legal Updates using the Google Viewer; however, you will need to be logged into Google Docs to view them.
Please choose one of the above to proceed!
LOADING PDF: If there are any problems, click here to download the file.
DISCLAIMER: Because of the generality of this update, the information provided herein may not be applicable in all situations and should not be acted upon without specific legal advice based on particular situations.
© Sutherland Asbill & Brennan LLP
Read fresh new writing on compliance, cybersecurity, Dodd-Frank, whistleblowers, social media, hiring & firing, patent reform, the NLRB, Obamacare, the SEC…
…or whatever matters the most to you. Follow authors, firms, and topics on JD Supra.
Create your news brief now - it's free and easy »
Sutherland is an international legal service provider composed of associated legal practices that...
View Profile »
See more »
Sign up for your custom alerts now, using LinkedIn ›
Back to Top
Copyright ©


text:
The Franconia Custom Fleece Cooler provides your horse with protection from chills. Made of durable, anti-pill, double-sided polyester fleece, this horse cooler has a contoured neck with hook-and-loop closures. Secures with hidden surcingle and tail cord. Allow 3-4 weeks for delivery; allow an additional 1-2 weeks for a monogram.
The Franconia Custom Fleece Dress Sheet is both a fitted cooler and travel sheet. Constructed of premium quality 380 gram fleece with a pill-resistant finish, this dress sheet wicks moisture to protect your wet horse from chills. Choose body and trim colors to reflect your personalized style. Allow 3-4 weeks for delivery; 5-6 weeks if you add a monogram.
Weatherbeeta Freestyle 1200D Detach-A-Neck Heavyweight Turnout. This turnout blanket helps you cover your horse from poll to tail for tough protection against winter weather. It's cut for a snug yet comfortable fit and the neck can be removed on mild winter days. 360 grams of poly fill provide insulation.
With tough, waterproof and breathable fabric and Freestyle gussets and shoulder darts, the WeatherBeeta Freestyle 1200D High Neck Medium Weight Turnout offers exceptional comfort for your horse. The Freestyle design features allow greater freedom of movement, and a full-wrap tail flap provides extra protection. This turnout is insulated with 220 grams of poly fill and lined with 210 denier oxford cloth.
The TuffRider® Hippo Heavyweight Turnout Blanket offers every horse a custom fit with its innovative front closure. A waterproof, durable 1200D outer and 350 GSM poly fill insulation protect your horse from the elements. Chest area of this turnout blanket has heavy elastic at the top to facilitate grazing posture. Special buckles at the sides of the neck and hook-and-loop closures along the front piece allow for a customized fit. Elastic shoulder gussets allow freedom of movement.
Upland's Honeycomb Chill Chaser is a multi-use sheet that's handy all year. Use as an anti-sweat, a cooler, a liner or show sheet.
Restore hook-and-loop fasteners to top condition by removing dirt, shavings and debris! The Hook-and-Loop brush has gentle, wire-like bristles that handle the task effectively and easily.
Repair torn horse blankets or riding apparel with the Horse Blanket Repair Kit Plus! This easy-to-use kit eliminates the need for needles and thread. It dries fast and the repairs are long-lasting.
The Horseware® Ireland Blanket Repair Kit contains everything you need to make quick, permanent repairs to horse blankets: coated ballistic nylon patches, super-strong and flexible Rambo® Stormsure glue and instructions.
The Rambo® Optimo Stable Sheet from Horseware® Ireland is updated for the season! Constructed with Rambo split technology, this sheet provides your horse with exceptional comfort and allows full freedom of movement. It has a tough 1000D ripstop polyester outer, a coat-smoothing polyester lining and wither padding for added comfort.
The Rider's International Hug Fly Sheet prevents shoulder rubs and constriction while it protects against coat-fading UV rays, insects and dust. This unique mesh fly sheet features overlapping chest panels that move freely as your horse moves. It's easy to fit on developed horses-no more slipping or pulling.
The Stretchies® Hood from Flex Rider in lightweight, four-way stretch fabric keeps manes and mane braids protected. This hood is also helpful for preventing shoulder rubs under blankets.
Protect your horse's shoulders from blanket rubs with the Stretchies® Shoulder Guard from Flex Rider. Made of lightweight, 4-way stretch fabric, this shoulder guard is fully adjustable for a snug yet non-restrictive fit.
Made with mildew-resistant Textilene® mesh, the Kensington™ Fly Mask with Fleece Trim is both comfortable for your horse and durable. The plaid design makes for a great look.


text:
Sorry if this has been posted, but I didn't notice it.
I was just wondering about the Type A and Type B free agents. What kind of stats does a player have to put up to be considered a Type A or lower?
Also, what is the number of draft picks received for each type?


text:
Night Of The Headless Horseman
Post your Comments or Review
This page has been viewed 72 times this month, and 2157 times total.
Night Of The Headless HorsemanNight Of The Headless Horseman (1999) - Television Cartoon Episode Guide
This page has been viewed 72 times this month, and 2157 times total.
- Television Cartoon Episode Guide
- Fox Television Studios, Cinematek Productions, Computed Animation Technology, Locomotion Studios
- Distributed by: 20th Century Fox Home Entertainment
- Cartoon Characters: Ichabod Crane, Brom Bones, Katrina Van Tassel, Hessian Trooper, Van Ripper.
- Originally Released in 1999.
- Color
- Running Time: 60 minutes.
Cartoon Summary:
A soldier who was beheaded during a battle haunts the road to Sleepy Hollow. The local schoolmaster uses the road to visit his sweetheart Katrina Van Tassel. His rival, town bully Brom Bones, tries to terrify him with the story of a soldier's ghost who rides through the hollow in search of his missing head.
Submit Additional Information:
Do you have anything to add to this page? Have we made any mistakes... or do you ahve any additional information about Night Of The Headless Horseman? If so, we would love to hear from you. Please send us a quick note with your additions or corrections to this page, and we will make the corrections as soon as possible!Report Errors / Submit Additional Information
Cartoon Comments:
Do you love Night Of The Headless Horseman, or do you think it is the worst cartoon ever? Let us know what you think! Surely you have an opinion... so share what you think. Take a minute and post your own comments about this cartoon here.


text:
Siert Bruins, a former member of the Nazi Waffen SS, was charged with the 1944 murder of a Dutch Resistance fighter.
A court in the German town of Hagen is determining if the case can go to trial, according to The Associated Press. Bruins, 91, is a Netherlands native but is living in and is a citizen of Germany.
He is accused of killing Aldert Klaas Dijkema, a Dutch Resistance leader captured and shot to death near the German border with the Netherlands.
Bruins, who was convicted in 1980 and served eight years in prison for the wartime murder of two Dutch Jews, also was sentenced to death in absentia in the Netherlands in 1949.
He volunteered for the Waffen SS in 1941 and later served in the Nazi internal intelligence agency and the Security Police with a unit looking for Resistance fighters and Jews.
Efraim Zuroff, chief Nazi hunter with the Simon Wiesenthal Center, told the AP that the apprehension of Bruins was "wonderful."
"It again reinforces that it is still possible to bring Nazi war criminals to justice," Zuroff said.
We welcome your feedback.
Privacy Policy
Your information will not be shared or sold without your consent. Get all the details.
Terms of Service
JewishJournal.com has rules for its commenting community.Get all the details.
JewishJournal.com reserves the right to use your comment in our weekly print publication.


text:
Presidential Candidates' Tax Records
Marty Sullivan talked about Republican 2012 presidential candidate Mitt Romney'ss tax records from 2010 and 2011 as well as other… read more
Marty Sullivan talked about Republican 2012 presidential candidate Mitt Romney'ss tax records from 2010 and 2011 as well as other former Speaker Gingrich's tax records. He responded to telephone calls and electronic communications.
C-SPAN Radio's Nancy Calo read news headlines at the end of the program. close
*The transcript for this program was compiled from uncorrected Closed Captioning.
People in this video
- Nancy Calo Producer C-SPAN->Radio
- Steven Scully Host C-SPAN
- Martin Sullivan Contributing Editor Tax.com
Hosting Organization
More Videos From
Washington Journal
- Representative Allen West on President Obama's State of the Union Address
- Senator Mark Pryor on President Obama's State of the Union Address
Related Video
Open Phones
Telephone lines were open for comments on the question, "Should presidential candidates release their tax…
2012 Presidential Candidates on the Economy
Matt Trumball talked about the presidential candidates' economic plans. He also responded to telephone calls and…
2012 Presidential Race
Political blogger Craig Crawford talked about the latest in the presidential race following Rick Santorum's exit…
Phone Interview with Daniel Stauss
Daniel Strauss spoke about the 2012 presidential election and a disagreement over past tax payments between Mitt Romney…


text:
ASUS P5N32-SLI Premium/WiFi-AP Edition nForce 590 Review
View Single Post
Join Date: Jul 2004
Location: MKE
Posts: 13,629
Re: ASUS P5N32-SLI Premium/WiFi-AP Edition nForce 590 Review
Originally Posted by
Mr Bigman
You have alot of opprotunity for your age. 22 is still considered todays teen. but you must get payed for this or do you live with your parents and their T1 line?
Some of you youngsters have it made these days.
Why do you think that my age has anything to do with where I am financially?
I have a full-time job that pays me a lot because I'm awesome at what I do.
I have my own apartment that I pay the rent and all the bills for out of my pocket.
I pay for my own cell phone, my car insurance and the gas used to fuel the truck I bought with money I earned.
I even paid the last two years' worth of tuitition (after my scholarships and loans were applied) to Marquette out of my own pocket. And that wasn't cheap, even with aid.
My parents cut me off a long time ago and I'm thankful for it. As you can see, I'm far above where most 25-28 year olds are, and I'm doing this all myself age the "barely-out-of-my-diapers" age of 22. I have opportunities because I've done what I've needed to in order to have them. This didn't all just happen because of luck.
View Public Profile
Visit superklye's homepage!
Find More Posts by superklye


text:
After uploading many of my paper log QSO's to LOTW, I have received Eight (8) Band LOTW, WAS awards, I am still having a great time on the air and along with LOTW things are very quick and easy for QSO confirmations. I have just a few states remaining on 160 meters for that WAS award. Namely, AK, CA, and HI, still searching and will get them done one day.
Got the bug while serving in the USN in 1984 to 1990, licensed as soon as time permitted and have enjoyed this wonderful hobby ever since. Lately, I have been uploading all QSO's to LOTW, I also use EQSL as well as I started with it first and am getting close to WAS on many bands, lacking 160M, 6M and 2M, just a few States left for awards on 160 and 6, many left on 2M. With family and work, I let alot of good radio experiences slip away but attempting to make up for it now, HI HI.
CW is difficult for me, but I am still practicing!
Home QTH was changed in 1998, so just getting set up with antenna systems again, I have very minimal antenna systems and station as to not adversely impact neighborhood asthetics, HI HI. Mostly wires and have installed a Mosley Mini Beam. QTH elevation is 966 feet msl.
FLEX Radio Systems 3000 or ICOM 736 for HF and 6 meter operation, 80 meter loop and or windom at 48 feet (N to S), Alpha Delta sloper(NE to SW), Butternut vertical (HF9V-X), Mosley Mini beam on a new mast at 35 feet, homebrew 20 meter monobander about 32 feet in the air, 3 elements, long wire for 160 meters configured as a loop(but too short and too low) to be really effective. Long wire for SWL enjoy that also. Astroplane for 10 meter when it opens, base is 38 feet on mast. Diamond CP22E for 2 meter operation.
Kenwood 751A for 2 meter operation, SSB is my favorite here. JT modes are very convenient and it is exciting what the transmitter and a bit of wire can do with lower power levels, 5 to 15 watts. I added a Yaesu 897D for field day portable operations and find it a great little radio.
Active in RACES when get a chance. I love field day activities, usually from the lake or my deck in the backyard and operating QRP with a generator.
Have now installed a tower with a 6 meter beam. Also have installed a ground plane for 10 meters, am enjoying the AM mode as well now.
Hope to make your acquaintance on the air soon, very active on the digital JT modes. I always enjoy this exciting hobby and the pleasure of a contact(new one or previously worked) makes no difference to me. 73 and thanks for reading this bio. Cary.
1949426 Last modified: 2015-05-08 22:39:16, 2615 bytes
You must be logged in to file a report on this page
Currently updating logbook display.


text:
News from WTOP
News from the AP
Mike Causey
Shows & Panels
Shows & Panels
- Agency of the Month
- Business of Government Hour
- The CFO Agenda: Delivering value from financial transparency and audit efforts in the public and private sector
- Cloud-Driven Data Insights in Government
- CXO Surveys
- Disaster Resilience for America
- Federal News Countdown
- Government Perspectives on Mobility and the Cloud
- Reducing Risk in the Cloud
- Reimagining the Next Generation of Government
Shows & Panels
- AFCEA Answers
- American Readiness: Renewable Power and Efficiency Technologies
- Ask the CIO
- The Business Case for Cloud Computing in Government
- Cloud-Driven Data Insights in Government
- Connecting the Dots: Digital Authentication in Government
- Data-Centric Approaches to Government in the Cloud
- Delivering the Digital Government Mission
- Federal Executive Forum
- Federal News Radio's National Cyber Security Awareness Month Special Panel Discussion
- Federal Tech Talk
- The Future of Government Data Centers
- Government Perspectives on Mobility and the Cloud
- Industrialized, Interoperable and Insight-driven Federal Health Panel
- Innovation and Tech Superiority through Acquisition Reform
- Mitigating Insider Threats in Virtual & Cloud Environments
- Moving Agency Applications to the Cloud with PaaS
- Reducing Risk in the Cloud
- Security in the Age of Targeted Attacks
- Air Traffic Management Transformation Report
- The Business of Cloud
- Cloud First Report
- Data Driven Cloud Series
- Disaster Resilience for America
- General Dynamics IT Enterprise Center
- Gov Cloud Minute
- Government in Technology Series
- Homeland Security Cybersecurity Market Report
- The Intersection: Where Technology Meets Transformation
- Modern Mission Critical Series
- National Cybersecurity Awareness Month
- The New IP for Federal IT
- The Social Platform Solution for the Department of Veterans Affairs
- Technology Insights
- The Cyber Security Report
- The Next Generation Cyber Security Experts
Shows & Panels
- Cloud-Driven Data Insights in Government
- Disaster Resilience for America
- F-35 Lightning II Program
- Government Perspectives on Mobility and the Cloud
- On DoD
- Pentagon Solutions
- Reducing Risk in the Cloud
- Reimagining the Next Generation of Government
- Transforming Business Operations at the Defense Department
Search Result
Search Tags: Connectivity
What will your agency's communications networks look like in five years?
What will your agency's communications networks look like in five years? Now is the time to decide. In the coming months, federal agencies will be making the move from FTS2001 to the Networx Program. Will your agency transition to Networx with a like-for-like approach or will it seize the opportunity and transform its networks to 21st century capabilities? Hughes offers agencies a cost-effective migration path to the future network today-- with resilient, path-diverse satellite solutions that are critical for emergency preparedness and COOP planning; ensuring broadband connectivity everywhere, even in remote locations; enabling secure, managed networks; and much more.
Tags: Technology , Satcom Report , Hughes , Networx , communication , COOP , networks
DHS ups connectivity on the northern border
A new project will outfit CBP border agents, check point officers and agency aircraft with secure voice and data communications connectivity along 1,200 miles of continuous U.S. border and more than 20,000 square miles of operating area. Details from Motorola's Mark McNulty.
Tags: Federal Drive , DHS , Contracting , Mark McNulty , Motorola , Suzanne Kubota