text:
Enter your email address:
Delivered by FeedBurner
Dion's Speaking Calendar:
Web 2.0 Architectures
by Duane Nickull, Dion Hinchcliffe, and James Governor
Agile Methods, Enterprise Architecture, 2.0 Services, and Web Development
Monday, December 14, 2009 #
With the service-oriented architecture (SOA) community recently issuing its SOA Manifesto (based closely it seems on the largely excellent example of the Agile Manifesto), I have been thinking recently about whether we need such a document for Web-oriented architecture, also known as WOA. The manifesto format can be a useful tool for creating critical mass around and focus on an important new idea or approach. I don't believe, however, that we need one for everything that comes out; only when important new advancements are still in need of clear explanation. So while the reception was decidedly mixed for the SOA Manifesto, by-and-large I do think such efforts can create useful statements of principles that get people thinking about the fundamentals and underlying truths of their work and help in creating a shared understanding.
The challenge with broad declarative statements like a manifesto is that technical communities are by definition consist of technical people. Such people are typically deeply experienced with formal intellectual appraisal and are famously independent thinkers who in my experience generally respect open debate and discussion. But they will almost automatically resist when ideas appear to be predigested for them and then presented on a platter. This is particularly the case if a hidden agenda or conflict of interest is perceived. Thus, the manifesto approach is a useful one most often when it's created through a collaborative effort and with concepts that have previous been subject to widespread discussion. It is less effective when it's used to distribute propaganda or tied in with motives that are unclear. In other words, it shouldn't be done lightly.
So I debated at length over the last few months whether WOA actually needs a manifesto. In the end, for reasons that I explain below, I finally concluded that it does, just not in the traditional way. I'm calling the output of this new approach an "unmanifesto". You'll find the 17 principles of the WOA Unmanifesto listed at the bottom of this post, each of which will be explored in more detail in upcoming posts.
Skip straight to the principles of the WOA Unmanifesto
Personally, I've been fortunate to spill quite a bit of fairly popular digital ink on the subject of WOA over the last few years. I did this because I felt information availability on the topic was fairly poor despite the central importance of the concept to most efforts involving software architecture these days, either on the Internet or in the enterprise.
Of course, Gartner's Nick Gall is the man most directly responsible (to paraphrase a line from a great sci-fi film) for creating the initial concept of WOA. I was fortunate to meet him present personally at an enterprise architecture summit many years ago now and see him present the concepts. I was captivated at the time and I remain captivated by WOA today. Since then, I've been fortunate to help carry the banner and put my own spin on what I think WOA is and where it's going. I've not had as much time to sit down and compare notes as much as I'd like with Nick and other great REST/WOA thinkers like Roy Fielding, Sam Ruby, Tim Bray, and literally hundreds of others that I must leave out for brevity. This is an attempt to start that discussion in earnest since WOA is clearly a growing influence on a great many aspects of software architecture today.
But ensuring we have a broad understanding about what WOA is part of the problem. My premise from the beginning has been that we can use the example of the Web itself to extract enormously important lessons about Web architecture if we are willing to learn what it has to teach us. WOA at its core is based firmly on REST principles, but that's only the beginning since the Web continues to evolve rapidly and we've seen important patterns and additions emerging such as mashups as well as OpenID and OAuth, to name but three major developments. But as I've covered in the past, while REST is the canonical basis for building Web-based software when it comes to moving data around the network, we still need a bigger conceptual umbrella to capture the higher order architectural advancements in Web software (i.e. those building on REST) . This broader set of essential patterns and concepts is one that has co-evolved with REST and the Web (including distribution, composition, security approaches, data models, data portability, and so on) over the years. WOA is that umbrella.
Though the underpinning is REST, WOA is therefore a superset of resource-oriented architecture (ROA) and contains all the moving parts we'd find in an architecture: 1) the basic elements of resource-based architecture, 2) the glue the binds them (HTTP transfer is the core one, but there are refinements now, e.g. ATOM and JSON feeds between browser widgets for example), 3) consumption models (how browsers and other clients fit in, mashups, recognizing OAuth as a first-class citizen in Web architecture, etc.), and the patterns that connect them all together.
First, though, to get perspective on all this and to help us decide what's really in WOA and what's not -- something that I've tried to do for several years now -- we have to define some generally agreed-upon principles to give us a set of objective discriminators. Let's not forget that the Web is one of the most effective systems for connecting people, information, and software yet created. It has profound and fundamental properties that are enormously powerful when understood and used effectively. These can be fully unleashed when architectural principles are followed that are aligned with the Web's fundamental nature. Fortunately for us, many of these principles have been discovered or developed over the last 16 years or so of the Web. Tragically, they are as not well known as they should be. Part of the reason for this is that the Web itself is an emergent, bottom-up phenomenon and it's not always easy to see what's happening on a wide scale. This bottom-up state of affairs has held back the development of the Web to the extent that most people don't realize that WOA, by definition, is the dominant software architectural model in the world today. It's what's being used today and is what works most effectively, most naturally, and most easily when it comes to the overall design of Web-based systems.
But "effective", "natural", and "easily" are subjective terms. How can we get more specific about defining WOA? Can we reach a genuinely accepted common understanding that would allow us to decide if a software system that we encountered did or did not use WOA concepts? Most importantly, how do we look at new innovations in Web architecture and decide if they are part of WOA or not? There needs to be a dividing line. Something like a manifesto might help.
To create this dividing line, I am issuing something I'm calling here an "unmanifesto." A manifesto developed by a single person is by nature of limited value, since it should instead reflect some wider conversation that represents a broader, community-validated understanding. Like an unconference, where the contents of the agenda are arrived at by a collaborative process and the attendees are also presenters, an unmanifesto is a distributed one in which there is more than one voice and the statements are made in public, online via social media. I call it an "un-manifesto" because it's not a manifesto, it's just the start of an important conversation by making some specific statements. An "unmanifesto" is an open, online discussion where it's assumed that there are multiple points of view. Anyone can join in and no one is excluded. Sound confusing? It is a bit messier than a traditional manifesto but it's also far richer and more open. If you care about modern Web architecture, I hope you will join me in this effort.
Thus, the principles below represent my WOA unmanifesto. There may be other WOA unmanifestos. This one will evolve over time as we learn more. Of course, anyone may issue their own WOA unmanifesto and anyone may comment and critique the ones that are created. The best ideas will be debated, survive, and be replicated. The ideas that aren't good just won't last. Eventually, after a distributed and open discussion, clarity will be achieved. Of course, this will only work if a community exists that is interested in the topic and is willing to actually listen to each other. I hope there is and that we will.
The Web-Oriented Architecture (WOA) Unmanifesto
The principles are broken down into four major groups: Resource-Orientation, Performance and Consumption, Interaction and Composition, and Openness and Security.
Principle #1: The fundamental artifact and unit of value in WOA is data stored as uniquely identified Web resources on the network with a uniform interface based on REST.
Principle #2: Web resources are maintained in a hyperlinked fabric using hypermedia (content with non-linearly navigable embedded links). In general, URIs should favor granularity and depth in linkage instead of being monolithic.
Principle #3: Resources and their network addresses (URIs) are self-descriptive so that WOA clients can determine how to consume them. In general, a URI should indicate what data format is being used and indicate nested elements with URL segmentation.
Principle #4: All application data is therefore accessible as resources in the manner presented here. Resources are visible on the network to authorized WOA clients, where sharing to all is the default should be the default whenever possible. Note: The major implication of this, particularly for legacy environments (enterprises) is that all existing databases and other datasets should be mapped to the resource model described here via REST. Even servers, storage, and network infrastructure are considered resources as they increasingly are in cloud computing APIs today (see OCCI).
Principle #5: Thus, resource-orientation is the best way known to 1) create a loosely-coupled, participative and shared global architecture that 2) can scale almost infinitely, 3) be used by virtually any platform or client, 4) directly facilitates information discovery (by analyzing link structure), 5) be easily recombined, and last (not least) 6) builds value non-linearly in the form of network effects.
Principle #6: Web resources must be stateless, layered, and cacheable, as per REST.
Principle #7: Network consumption of resources (transfers) is inversely proportional to how complex a Web resource is; i.e. the simpler the structure and data of a resource is, the more likely it will be used, recombined, and perform well. See the Services Continuum discussion for more background.
Principle #8: Resources should organized to optimize their discovery and reuse. This principle has many more implications that it sounds like at first. Not only should resources be SEO-friendly but it must be recognized that they are also APIs that support applications elsewhere on the network. Optimization for the latter can mean licensing agreements, SLAs, and customer support and all the trappings and requirements of open APIs.
Principle #9: A resource is recognized as having a higher realized value (network effects) the more that other parts of the network link to it.
Principle #10: Copying of resources is discouraged: The network effect of Web resources is increased when they are linked to from other Web resources instead of replicated. Caching is encouraged however, as long as it's at the same URI.
Principle #11: WOA clients provide value by connecting to the network to view, interact with, operate on, and recombine resources from multiple sources using REST. Note: The browser is the canonical Web client. There are many other types of clients.
Principle #12: Observable value is increased when resources from simultaneous multiple domains are accessed by WOA clients (mashups) which are themselves resources (self-contained applications).
Principle #13: Self-distributing applications create additional network effects. Native WOA clients can be distributed as resources (code-on-demand) that are self-contained applications (such as AJAX, Flash applications, widgets, etc.) to be embedded in elements of other WOA clients. This increases in turn the value that the resources the WOA clients use.
Principle #14: WOA applications inherently encourage link propagation, both in the resource fabric and out-of-band, through syndication, widgets, badges, mashable components, open APIs, and any other method that invokes Jakob's Law.
Principle #15: Resources are in formats that are based on open, published descriptions and standards. Resources must be readily consumable from the canonical WOA client.
Principle #16: Resources can be secured with HTTPS, WebDAV, OpenID, digital signatures, etc. Locally defined user ID and passwords are permitted but discouraged as are proprietary (i.e. not open standards-based) security approaches.
Principle #17: Resources that are secured must be exposed to WOA clients by a safe 3rd party authorization method. OAuth is preferred.
These 17 principles represent a significant update to my original definition, which remains widely cited. WOA has evolved and our description of it must evolve with it. Please note that I have pushed some details down into the description of the principles, which will be forthcoming soon.
I hope you see how there is more to WOA than just REST and that there are clearly higher order principles around composition and distribution architectures that need more definition. On the composition side, for example, incorporating Michael Ogrinz's mashup patterns may be a good way to access best practices and eliminate duplication. I'll be exploring that as well as other sources of high value inputs as this unmanifesto evolves. Keeping the principles clean, well-organized, and at the same level of abstraction will important and I'll do my best to achieve that. Also, I'll be clear that I still firmly believe that SOA itself is largely headed in an WOA direction, it'll just take a while given the momentum and vendor history in that space. So the WOA Unmanifesto is just as relevant and applicable to the SOA world as the Web world.
Finally, there are many essential details and implications for each of these principles, which were not included here lightly; each and every one is a key aspect of Web-oriented architecture. I also realize that for these principles to be broadly understood and themselves consumable by most Web architecture, each and everyone one of these principles needs a detailed breakdown with examples. This is one of the big problems with WOA being an emergent architectural style and not a standard in itself.
Over the coming months, as often as I am able, I will be breaking down each of these components as we begin discussing them here. Input and feedback is very welcome and encouraged both in comments below and posts (these are preferred) and by e-mail (mine is firstname.lastname@example.com) only if necessary. Also, I'm reserving the right to tweak this for a few more days to make sure the wording on the principles reflects my intent. I'll put a change log below to make sure nothing is lost.
Please join in the WOA Unmanifesto discussion below. I will respond as often as I can.
Further Reading:
What Is WOA? It's The Future of Service-Oriented Architecture (SOA)
Where Is The Future of SOA Headed? Where The Web Goes...
Unboxing Web-Oriented Architecture: The 6 Aspects Of An Emergent Architectural Style
Web 2.0 success stories driving WOA and informing SOA
The SOA world begins considering Web-Oriented Architecture (WOA) in earnest
Thursday, August 06, 2009 #
Recently InfoQ did a good summary of the debates around the apparent (to some) limitations of REST when it comes to creating good Web services. At issue is that REST APIs seem to expose "CRUDy" services that fly in the face of years of good services design, particularly when they are just read/write interfaces instead of the richer, full REST architecture (more on what this is later.) The discussion was spurred by Arnon Rotem-Gal-Oz's assertion recently that CRUD is bad for REST, which in my opinion is close but not quite right.
In my view there appears to be two primary ways that REST are used today. One is via the aforementioned CRUD approach (where CRUD = Create, Read, Update, Delete) and HTTP's verbs (which in the same order as CRUD are POST, GET, PUT, DELETE) are used to manipulate a data resource on a Web server. This is the most basic, most common way REST is used by application designers (sometime even using these to replicate traditional RPC) and is the foundational request/response model in REST services. Unfortunately, it's also probably the least interesting or powerful way to apply the full REST architectural style.
For one thing, using only CRUD to manipulate data treats resources as loose, unintelligent bags of bits, seems too fine grained, and generally feels like client/server in a cheap disguise. Sure, it works extremely well with the protocol that runs the Web, HTTP, but no architect would consider this story so far good enough reason by itself to bend an entire application design around it. Things get much more interesting, however, when URIs start getting added to the mix. This happens in one of two ways and, depending on how URIs are applied, it's the key to the whole REST hypertext approach (aka HATEOAS, or "hypermedia as the engine of application state") which has worked so well on the Web for content. And now, with REST, we're doing it with data.
First of all, like the Web itself, REST works better when the data itself links to URIs of other resources that contain a more complete representation of a given piece of data (a line item summary might link to the full line item resource, for example). Deep linking, and not maintaining local copies of a representation is essential to prevent a variety of issues including data duplication and out-of-date copies.
Second, the very essence of REST is to make the states of an application explicit and addressable by URIs. The current state of the application state machine is represented by the URI that the client just operated on and the state representation that was returned. The client can change state simply by operating on the URI of the state it wants to move into, making that the new current state. A given state's full representation includes the links (arcs in the URI graph) to the other valid states that that can be reached from the present state. So in the visual below, we show a simple but complete interaction model that includes a single valid state transition. In short, as even Roy Fielding himself recently emphasized last year, REST APIs must be hypertext driven.
Click To Enlarge
The scenario above shows a complete and I hope canonical (albeit simple) interaction model that exhibits the full RESTful properties. The individual numbered steps of this scenario are described below:
Step 1: To use a well-designed REST API, only a single well-known URI needs to be published by the creator and referred to by the client. From this top level resource, all the resources and state transitions for the entire API and all its services can be found. A REST API is self-descriptive and therefore self-contained, with only minimal "out-of-band" documentation required. A simple HTTP GET at this URI will retrieve the API entry point.
Step 2: The API is examined and a service known as FooService is found, along with the URI for the entry point to the FooService API.
Step 3: The resource list for FooService is retrieved via GET. The URI came from the top-level API description examined in the previous step. In this example, a list of fooitems is retrieved, including possible state transitions.
Step 4: There is only one state transition, "reverse". This operation is chosen to be used in this scenario.
Step 5: A PUT (which means the resource will be modified) is issued to the URI described by the original resource. A back-end component (business logic) then reverses the name element of the resource (changing the state of the resource). A copy of the modified resource is also returned by the operation, making the operation efficient as well.
Step 6: Although a copy of the resource was returned from the state transition, the change is durable but we do another GET, this time just on the resource to see what the current state is.
Step 7: The state of the resource is examined and found to be changed by the state transition operation.
So, I think one of the biggest problems with REST is that there is no canonical description for how to use it. This is also one of its core strengths. There are many ways to apply REST to the problem at hand, and if all you need is CRUD, you are still doing basic REST and meeting your needs. No single description could do justice to the power and flexibility inherent in the model and I agree strongly with Sam Ruby that "REST isn't an all or nothing proposition. One can get significant value from partial adoption."
But this discussion does highlight some very specific lessons learned that many REST practitioners don't often follow but are important to consider carefully in order to build services that are resilient, adaptable, and maintainable over the long term. Many of the benefits of REST aren't felt until later when you want to fix or change things. That you can do so easily is a testament to the potency of architecture of the Web, if only we can understand it clearly enough.
Here are my takeaways on what most REST adopters can and should do to get the most from their use of this increasingly popular architectural style:
I've frequently described REST as a foundational element of Web-Oriented Architecture for both enterprise and Web applications and open data in particular. It's been thrilling to see REST become so popular in recent years, even if we have a long way to go yet to reach a wide understanding of what it is and how to use it. I hope these ideas are helpful and I'd love to hear your additions below.
Saturday, June 06, 2009 #
So you've been reading the many recent articles on the Web about REST, how it's the future of Internet architecture, and you've decided to begin applying some of it in your work. But when you start digging into the subject, REST by itself seems to be about moving data resources from one place to the other over the Internet, not building complete applications. It's unclear (and certainly poorly documented) to you how to situate it properly in context; a fully realized and well-designed piece of software.
You start wondering how REST is used specifically (and strategically) in an application or overall software architecture. In other words, what are the moving parts and rules for applying them. You also wonder what else you should aware of that can be used in conjunction with or to complement REST and make it better. You might even want to know when to break the rules and use other related approaches. This is where something known as WOA becomes useful.
This is where a deeper discussion on Internet applications and specifically, Web-Oriented Architecture (WOA) comes into the picture. WOA creates a more sophisticated and up-to-date vision for modern Web applications that aligns gracefully with the grain of the Internet. It also is generally (but of course, as with any approach, not always) true that WOA applications are easier to build, connect to other systems, and maintain for the Internet (and yes, Internet-type networks, like your enterprise intranet.)
Why is WOA generally so much better than traditional service-based architectures? Because WOA is an integrated, emergent architecture that is born out of countless lessons learned about what works and what doesn't when designing software for the Web. It's not created by a big software company, it's not a commercial piece of software, it's not a giant set of standards from a standards body.
Instead, WOA is just what developers are doing lately and more and more of it is showing up in our favorite Web application frameworks like Ruby on Rails, Django, and the nascent cloud computing platforms. In other words, it's a set of best practices for designing Web applications. And it's not a grab bag of approaches and technologies, it's actually a fairly elegant model that works: It's secure, it scales to the Web, it performs, it's all those other things that we like when we are idealized lazy programmers. WOA is also generally simple to work with, easy to implement, and usually is intuitive once you understand the Web. Note: One exception to this is REST being "the engine of hypermedia state", that certainly takes some genuine noodling over to understand. I'll see if I can tackle that in an explainable way in the near future, since it's one of the best and most important parts of REST. In the meantime, a walkthrough of Sun's RESTful Cloud API perfectly illustrates this concept.
So, in short, WOA is much more than REST and REST is the foundational architectural style for WOA.
You can see all the key elements in the diagram in the upper right of this post. It's a good start at understand what's essential about WOA (which at its core is about RESTful approaches to software) and what else you can do with it. Chances are good you'll end up using things like JSON and ATOM. Perhaps to a lesser extend things like OpenID or OAuth, though you should definitely put those on your to do list.
A more exhaustive list of things that are WOA is below. Many of them can be used in non-WOA ways (in other words, breaking RESTful principles). Don't use them that way. Instead, think about resources, links, and hypermedia and how to compose, distribute, and consume them.
If you on the enterprise-side of the story and wondering how this will affect you, please read my recent examination of WOA and SOA.
Update, June 8th, 2009: Dave West over on InfoQ did a good summary of this post today in REST is a style -- WOA is the architecture.
Wednesday, April 15, 2009 #
Are you creating a new Web site and developing a user registration system that requires new visitors to sign-up and create a user ID and password? Stop now and read this. There are now more effective approaches for dealing with Web accounts which are more powerful and are better for you and your users.
The concept of Web identity has recently undergone significant evolution that all Web developers and architects, both consumer and enterprise, should be readily familiar with today. These new identity options, specifically OpenID, OAuth, and Facebook Connect, when individually used can:
This post explores these three new ways to provide user identity that will make your Web applications work substantially better and make your users more secure and satisfied. There are also some important caveats and issues to be aware of and which we'll also discuss.
One of the parts of the classic Web that's unfortunately still with us is the trusty old user ID and password combination, one per Web site. That's because the Internet itself has no intrinsic identity system built-in; anonymous access is the default and anything more sophisticated has to be cobbled together one-off for each site. This means most Web sites today implement user identity uniquely with different sign-up processes, user ID requirements (sometimes you get to pick, sometimes you have to use your e-mail address, and very often your favorite ID is already chosen, etc.) Then there are varying password length requirements if you are lucky, variable recovery processes if the password is lost, and so on. This is the challenge of an open, decentralized system (the Web) that has very little top-down design: How to provide users control and consistency in an online world where there are very few widely accepted user-centric open standards.
Interestingly, this is the one of the rare instances where enterprises got it right long before the Web and is now a relatively well-solved problem -- at least inside the firewall -- with something known as single sign-on. Often referred to as SSO and embodied by technologies such as Kerberos, smart cards like RSA's SecurID, and Windows Integrated Authentication, single-sign on ensures that users have just one single, consistent identity and sign-in credentials. Single sign-on works then seamlessly to enable them access to the various applications that they use on the network. There is only one user ID and password to remember and it can be reset, administered, or even shutdown centrally whenever required.
Note I say "easy to implement" is a requirement for Web identity systems and I should note that making user identity truly secure is one of the harder problems in software. This can mean that OpenID and OAuth require a fair amount of work to implement successfully. The good news: All three new identity approaches presented here are now fairly easy to use right out of the box since much of the hard work has been done to create libraries for the most common programming languages and environments. I will observe that it's still up to you, the designer of the Web application, to ensure these libraries are doing the right thing by your users. This is yet another reason why Web identity is still so often reimplemented over and over again; lack of trust of 3rd party code and not-invented here are still two powerful forces in software development. However, the value proposition has grown to the extent that these do-it-all-yourself positions are growing increasingly untenable.
That's not to say considerable care shouldn't be used. In general, the more important the user data your application handles, the more auditing and regular code reviews you should conduct of your security and Web identity libraries.
Each of the identity options presented here does something slightly different in terms of providing users with either their choice of login and Web identity or safe access to their data from elsewhere on the Web. As such, you will likely be faced not with choosing one of these three identity options but actually all three of them to give your users the options they what and will increasingly expect. I've highlighted in the past the number of core competencies that modern Web developers and architects have to master these days, and it's a tall order. These options add to that burden but as you'll see, provide considerable value in return.
The best and most authoritative explanation of OpenID can be found here at OpenID Explained. It's quite simple: Let your users use the identity provider of their choice to login to your application instead of filling out an online form and capturing the information in a private, local account that users are not likely to trust nor long remember. Chances are nearly 100% that a user already has a valid OpenID from the many popular services that already allow their IDs to be used this way today. OpenID use is also growing: Recent reports show that sites offering OpenID logins are currently reporting that 10-15% of users will login this way and the average is climbing. Venture Beat also had a decidedly bullish outlook on OpenID today.
Many site have their business model and valuations wrapped around the number of registered users that they maintain. OpenID does NOT make this model go away, it merely provides an easier way to begin a relationship with new users. It even makes it easier for the user to interact with a new site. The full set of benefits for offering support for OpenID is:
To actually support OpenID, you'll need to implement the current specification or use one of the many available OpenID libraries that already does it for you. Support exists for all the common languages: C#, Java, Perl, PHP, Ruby, Python, and even Haskell. The bottom line, you'll be learning about digital signatures such as HMAC-SHA1 and extensible resource identifiers, or XRIs.
It should also be noted that the getting the user experience of OpenID right is essential for being successfully with it. Because most visitors will still be unfamiliar with it, a lot has been learned about how to offer the sign-up and sign-in experience in a way that is accessible and understandable. Early OpenID adopters often made the process too complicated or inexplicable to the user. A lot has been learned about how to do this correctly to enable the double digit usage rates we're seeing today on sites that are successful with OpenID. Probably the best presentation about effective OpenID user experiences is Brian Ellin's deck from this year's OpenID UX Summit, I encourage you to study it.
Using a site through its user experience is now only one way in which the Web will interact with your service. If you're only offering a visual user experience and don't currently have an application programming interface, or API, you're behind the times. Open APIs are one of the most powerful new models for delivering services on the Web and they enable 3rd parties to integrate with and build on top of your product, creating all new services and mashups. This leverages something I refer to as Jakob's Law, which says that most of your traffic will ultimately be driven by external activities on other sites. Thus, more usage will often come in through an API than through the user interface. For example, Twitter famously has 10 times more usage through its API than through its Web experience, which I recently reconfirmed is still the case with Twitter's Alex Payne at Web 2.0 Expo earlier this month. It's also no accident that Twitter recently added support for OAuth.
However, the challenge with APIs is the same as with user experiences: How to let users access their data remotely in a safe way without having to give the 3rd party application their user ID and password for your site? Users might even feel comfortable giving their credentials for a Twitter application like Twitterific, but they certainly aren't going to do that for critical applications like e-mail, banking, or e-commerce. Enter OAuth, a method for allowing 3rd party applications to gain access to user data with their permission, but without having to give the credentials to that application. In other words, users can "authorize" 3rd party applications to access their data in your site, without giving that 3rd party application the user ID and credentials.
The benefits of securing your API with OAuth include:
You can find the OAuth specification here but most will opt to use one of the many pre-built libraries that is already available and which greatly simplify implementation. The language options are much the same as with OpenID which is good since OAuth is seeing broader and broader adoption as its benefits become clear and it gains critical mass.
Now, those paying close attention to this discussion might notice that OpenID and OAuth seem in direct opposition. OpenID allows users to employ remote 3rd party logins and OAuth tries hard to make the local ID safely usable externally, at least when it comes to access through the API. In reality, OpenID and OAuth are an excellent combined strategy and plenty of work is underway to make it possible to use OAuth with OpenID. Sites that support this combination will use the user's preferred login for both the user experience (UX) of the site as well as for the API, creating a seamless, consistent experience. The Step 2 project has begun to "combine the OpenID authentication and the OAuth authorization protocols" and has involvement with many of the major players in the space including Joe Smarr of Plaxo and David Recordon of Six Apart.
OpenID and OAuth represent major, scalable Web identity authentication and authorization systems that work with both a user experience as increasingly popular API access methods. They are both open standards and have the backing of many major players, particularly OpenID which has the backing (though only partial implementation) from Yahoo!, Microsoft, IBM, Google, Facebook, and other influential organizations.
Open Web advocates are probably wondering why I've included the proprietary Facebook Connect in this discussion of Web identity. That's because there is as of yet no major social identity system that is open and has broad support. Consequently, I will warn you that using Facebook identity is something that should be done carefully with eyes wide open for the lock-in issues and dependencies that can result. However, the value proposition is considerable. For example, Facebook claims that for many sites "2 out of 3 new registrations come via Facebook Connect, and those users have about 50% more engagement on sites". This alone will be worth it for a great many services, and certainly thousands of services have agreed.
By social identity I'm referring to not just a user's individual identity but also their social graph, or list of connections they've established online, as well as the communication that takes place between those contacts, typically an activity stream.
Social identity is fast becoming essential context for for online applications; it's not just enough to have you as a registered user since today's Web applications also need to know who your connections are in order to provide the services they offer for communication, collaboration, etc. Social identity systems allow you to login and then be able to refer to and access your social connections and communicate with them. This is far preferable to the old-school equivalent: Importing your contact lists and then inviting all of them to the new application you're using. The days when that was acceptable are over, if they ever were. Now you can just use your social identity and they can see what you're doing in their activity stream and decide whether or not to respond or participate.
For example, I use the video sharing site Vimeo quite often but many of my friends don't and don't even have an account there. But they can see what I'm doing on Vimeo with my Facebook Connect link that shows my videos in their Facebook activity streams when I post them. They can then decide if they want to view the video or otherwise interact with Vimeo.
While many organizations remain uncomfortable delegating any aspect of user identity to external services, the reality is that online services are invariable stronger, more robust, and more attractive to users when they do. The effort is higher however, and there are more dependencies to manage but Web identity has definitively entered a near era. Proactive companies that take advantage of what is possible today will be poised to enjoy additional growth, higher user retention, and better customer relationships.
Tuesday, March 17, 2009 #
In the last year or so, after quite a lull, the software architecture business has gotten rather exciting again. We're finally seeing major new topics emerging into the early mainstream that are potential game-changers, while at the same time a few innovations that have been hovering in the margins of the industry are starting to break out in a big way.
The big changes: The hegemony of traditional 3 and 4-tier application models, heavyweight run-time platforms, and classical service-oriented architecture that has dominated for about a decade is now literally being torn asunder by a raft of new approaches for designing and architecting applications.
These might sound like incautious words but major changes are in the air and architects are reaching out for new solutions as they encounter novel new challenges in the field. As a consequence, these new advances either address increasingly well-understood shortcomings of existing approaches or add new capabilities that we haven't generally focused on before but are becoming increasingly important. A few examples of the latter include creating reusable platforms out of applications from the outset (the open API story) or cost-effectively creating architectures that can instantly support global distribution, hundreds of terabytes of data, and tens of millions of users. There are others that we'll explore throughout this post.
These innovations are hallmarks particularly of the largest systems being built today (which are running into unique challenges due to scale, performance, or feature set) though these software advances are also moving across the spectrum of software from everyday corporate systems and Internet applications to new mobile devices and beyond, such as the emerging space of social networking applications.
Mainstays of application architecture such as the relational database model, monolithic run-times, and even deterministic behavior are being challenged by non-relational systems, cloud computing, and new pull-based systems where consistency and even data integrity sometimes take a backseat to uptime and performance.
To try to get a handle on what's happening and to explore these emerging topics, I've been doing conference talks lately about the transformation of software architecture that we're beginning to see in so many quarters these days and generally finding consensus that the exciting days of architecture are back, if they ever left. Now it's up to us to begin the lengthy process of taking many of these ideas into our organizations and integrating them into our thought processes and architectural frameworks and bringing them to bear to solve problems and provide value. As one software architect came up and asked me recently, "How do I get my organization to understand what's happening out there?" This is an attempt at addressing that question.
Here's a list of the most important new areas that software architects should be conversant in and looking at in 2009:
Any list of what is new and important in software architecture must be personal perspective so I invite you to add your own below in comments.
Tuesday, November 18, 2008 #
I recently had the privilege of being on David Linthicum's excellent Real World SOA podcast show on Infoworld to talk about Web-Oriented Architecture (WOA), a topic that readers here know I've been exploring for a while now. David's one of the most respected names in enterprise architecture and SOA and so I enjoyed the opportunity to discuss with him what's happening to SOA as it meets the Web and begins to evolve in new and interesting directions. We had a lively conversation that covered the gamut from innovation and enterprise mashups to open Web APIs and cloud computing.
A full transcript with links is provided lower down in this post. You can also listen to it interactively with the Web widget below, or you can download the the entire mp3 file of the Real World SOA Podcast episode on WOA.
Note: I created this transcript from the mp3 file using the terrific CastingWords service which is a WOA application that offers its transcription service by building on top of Amazon's innovative Mechanical Turk API.
Powered by Podbean.com
Monday, September 08, 2008 #
It's been an interesting few months in the enterprise architecture space as we look at where service-oriented architecture (SOA) is headed this year. I wrote a detailed exploration of this on ZDNet last April and the discussion since then has only become deeper and more urgent. Getting general consensus on a new acronym is always a difficult thing to get widespread traction on, much less a new architectural approach. I'm not personally caught up on what we call this next generation of lightweight, Web-aligned SOA either, but WOA is the best name I've seen so far.
With the WOA discussion it's also been clear that the SOA industry -- vendors and practitioners alike -- are protecting their turf and looking a little skeptically at something that has the potential to change the center of gravity in the SOA business. I think there is actually little threat here; most of the top-down activities that SOA initiatives have been putting in place, such as governance and cross-functional business architecture alignment, are just as appropriate -- if not more so -- when it comes to making WOA successful. We'll talk a little bit more later about SOA products.
And let's be clear here: WOA is a really a sub-style of SOA that is actually highly complimentary. I personally believe we've collectively discovered that we've been spending the last few years on a course that just needs a healthy and appropriate re-adjustment, with the concepts in WOA helping us find a better way.
WOA clearly offers a number of advantages to those doing traditional SOA today. This includes considerably improved service consumption models that are less expensive and time consuming to use as well as unleashing the tremendous power of link architecture to drive information discovery, leverage, and analysis. The profound business implications of open APIs and cloud computing are growing clearer by the week. WOA also provides an excellent on-ramp to access the many powerful new online product distribution models currently available today. Distribution of SOA is a woefully neglected topic and WOA can bring an extensive set of techniques, from syndication to extraordinarily low-impedance Web services.
For its own part, the term WOA itself goes well beyond the simple reconciliation of Web services technology from a SOAP-based model to one based on REST. For this very important reason, WOA is not synonymous with REST. WOA encompasses all the architectural issues that are drive by the design of the World-Wide Web, an architecture, I will point out that has continued to refine itself including the rise of open Web APIs), prepackaged service consumption mini-applications (aka widgets or gadgets in the Web world, the advent of JSON, browser-based mashups, the recent resurgence of the Semantic Web, and much more. So, hopefully to put the REST/ROA vs. WOA debate to bed; REST remains an absolutely core architectural element, but WOA by definition encompasses the full architecture of the Web today.
The Web also clearly includes the browser and it's the browser itself that has driven many of recent innovations and trends in WOA, particularly mashups and the interest in JSON. And when it comes to the enterprise space, the reason that WOA has become such a significant topic is for the reason I gave in my most recent sum-up of WOA:
"It's important to remember that no small system can sustain contact with a large system for very long without being fundamentally changed by it. This is what is happening with businesses (the small system, no matter how large) and the Web today (the big system.) The intrinsic nature of the Web is driving major changes in how we create network-based products and services and is inexorably turning us into Web-oriented businesses. Businesses that want to be successful on this network without understanding its fundamental nature and capabilities are only delaying the time it takes to reach the full potential the Web offers."
Unfortunately, technology always moves faster than businesses can adapt, and the situation hasn't gotten any better, quite the opposite. Fortunately, the on-ramp to WOA for most organizations should be far less traumatic that many systemic shifts in the past, such as the one from structured systems to object-orientation. Even more helpful for enterprises looking to start down the WOA route, many SOA products already offer support for core WOA capabilities -- especially around REST -- even though it's a far cry from what the potential could be. The REST support in Java EE 5 and Windows Communication Server (aka Indigo) are good examples.
Here are just some the things possible for companies that open up their data and functionality in a WOA model:
I'll be the first to admit that most IT organizations would find many of the examples above somewhat distressing. The shift in control, the increase in openness, the different way of thinking about architecture, the countless security issues and governance concerns will likely prevent movement to WOA at a rapid pace for many businesses. However, I believe these scenarios offer too much competitive advantage for them not to be a major goal for IT organizations over the next 18-24 months. Worker productivity, innovation, growth, and even marketshare are going to be directly driven soon by whether a company has an open API or not, whether WOA services are in place to unleash data to workers, and so on.
Here are some other key statements from SOA practitioners and experts alike over the last few weeks:
ZDNet's Joe McKendrick had this to say in "WOA wins hands down in a SOA popularity contest":
"SOA may benefit from WOA (and Web 2.0 in general) because it enables business end users to see and experience online services via composite mashups and cloud computing. SOA could be sold as an internal cloud that provides online services inside the walls of the enterprise. In this regard, WOA makes SOA real to perplexed business users. Plus, enterprise SOA implementations may function as islands of integration that will eventually be assimilated into a larger WOA, while still retaining boundaries."
Also be sure to catch Joe's new "Everyone Loves Web-Oriented Architecture" for an irreverent and informative look at WOA. His comments from Steve Bjorg that WOA came before SOA are priceless.
Joe also cited the Information Week article on WOA by Roger Smith titled "A Simpler Approach to SOA", who reported:
"A growing number of companies are finding that lower-visibility Web-oriented architecture (WOA) developments, spawned through grassroots movements, are a better route to the service-oriented architecture. WOA, like SOA, is an architectural approach to system design, though WOA is resource-oriented rather than service-oriented. What's the difference? While the core SOA design unit is a reusable service that fulfills a distinct business function, resource-oriented services are more limited and data-focused."
Well-known SOA expert David Linthicum recently weighed in on Infoworld with:
"What's attractive about WOA is the fact that it's just sexier and easier to understand than SOA. Moreover, it incorporates many new other cool buzzwords such as cloud computing and mashups. I think what's most attractive is that it represents the movement of critical and core business processes from the datacenter to the cloud. This trend will continue, but it's going to be a slow migration over time, with some very visible short term successes, typically around outsourced infrastructure such as the new array of infrastructure services offered by Amazon."
David and I discussed this latter point last night in an upcoming SOA Expert Podcast episode and we agree: The biggest story for WOA will probably be its cross-firewall implications for API divisions that monetize SOA and provide scalable, self-service trade partnering capabilities, though the inside of the business will be a big story too. I'll update this post with a link as soon as that's out.
So we'll see how WOA fares over the next few months, but regardless of the success of the term (I suspect it will stick), the ideas behind WOA are going to remake IT and business over the next few years and I'll continue to cover it here. Please share your WOA thoughts in comments below.
Please read my popular What is WOA? article and 12 Things You Need To Know About REST and WOA for a deeper examination of what WOA is and why it's often a better way to do SOA.
Tuesday, April 08, 2008 #
There have been a number of interesting tracts written lately about that increasingly popular topic in the world of SOA and Web services: REST. In particular, the one that is circulating around social bookmarking sites and SOA blogs the most in the last month is Stefan Tilkov's excellent Addressing Doubts about REST. The article tackles the continuing skepticism that SOA practioners have had about the integration approach that has become the dominant one on the Web and is now making significant inroads in the enterprise (more on my findings on enterprise adoption of REST here soon).
Key SOA Trend: As of March 2008, leading industry analysts -- such as Anne Thomas Manes -- are concluding that "SOA is not working in most organizations."
Compellingly, Stefan goes far beyond the simple and often misleading SOAP vs. REST debates and makes a number of excellent points about the REST approach ranging from encapsulation and transaction boundaries to documentation and tool support. But what struck me most is that these largely technical concerns, while very important, still don't strike directly to the heart of what makes REST, and Web-Oriented Architecture in general, so significant to the practice of effective large scale software integration and composition. Specifically, the more I look at working examples of large scale SOA on the Web, the more I'm aware of the fundamentally different mindset and approach that are used by those that have an urgent business need to achieve deep levels of integration between many customers and trading partners.
So what are the big differences between traditional SOAP-based, top-down SOA and lightweight, bottom-up WOA? In the end, it's as much architectural and philosophical as it is technical. I'll also be clear and note that while successful large-scale SOA on the Web tends to favor REST, REST drives many of the concepts described below, rather than promoting them explicitly. In other words, REST resides at the core of Web-Oriented Architecture, which in turn describes a set of related approaches for creating a robust and bustling network ecosystem of loosely cooperating entities that typically compete for consumption via "architecture of the fittest." Here are some of the key things we've learned over the last half-decade that REST has been used widely to build WOAs:
There are literally dozens of models for building services to connect systems together. However, HTTP is at the core of many of the more promising ones, including REST. Here's an overview of the most common service models for SOA today.
However, many readers of this article are doubtless still wondering if REST and WOA are really the end-all, be-all for service-oriented architectures. For now, we're seeing it as one of the best available options despite a great deal of work yet to figure out how to apply it fully to the world of the enterprise. Is it ideal for every single type of application and scenario? Of course not. Your mileage will vary entirely depending on your requirements and your understanding of how REST deeply informs system architecture. However, it's increasingly emerging on the short list for those integrating systems of even the very largest size and complexity as well as down to the simplest and most nimble application. We've also learned a lot about the strengths and its weaknesses of this Web services model, however, as a fundamental part of the Web (since REST is nothing more than HTTP applied to data), REST along with WOA is the model that underpins many of the largest and most successful networks (and SOAs) in history.
Finally, I'll be delivering a complimentary Webinar on this subject matter next week, on Thursday, April 17th, to present the full scope of WOA and how it can be used to drive adoption, better business outcomes, and make SOA work in the enterprise. I do hope you attend.
What are your concerns? What do you think REST is capable or incapable of doing for your applications. Please share your story below in comments.
Wednesday, February 27, 2008 #
The need for businesses to open up their silos of information and internal capabilities to their internal customers has become an increasingly pressing issue as organizations strive to increase operational efficiencies and innovate more effectively with existing resources in the business and technical climate of early 2008. And in the last couple of years, as exposing uniquely powerful sets of data to online business partners has moved into the mainstream in the form of open Web APIs, opening up our IT systems across the Internet has become a competitive imperative as well. Unfortunately, despite two decades of experiments in heavyweight software engineering (the alphabet soup of EAI, SOA, ESB) for solving these types integration problems, we've seen relatively marginal improvements for most implementors despite heavy investments by businesses large and small. In short, integration between the systems running our business still isn't happening at the levels we need. However in the last several years, promising developments from the Web are pointing a way to a better model that seems to overcome many of the adoption and effectiveness issues of traditional SOA and is gaining wider adoption yearly (see sidebar below, right).
Most of us would agree that we still can't easily get access to the data and the systems we need to in order to get our daily work done. Workers still spend a great deal of time copying and pasting data between their various applications, data is batched and then exported and imported between IT systems around the world millions of times per day, and information just isn't getting to the places that we want it to without unacceptable amounts of manual labor. Even though Service-Oriented Architecture (SOA) initiatives around the world have the right goals, most efforts have fallen profoundly short of our desired levels of integration and improved business agility.
However, the news isn't all bad, the fascinating story is that there is a place today where the deep integration of our systems and information on a large scale has largely been solved and is a foregone conclusion in most cases. And that place is at the leading-edge of the World-Wide Web, sometimes referred to as Web 2.0. This success story has taken a while however and it's also managed to fly under the radar of most enterprise architects and IT vendors in the process. The left-hand turn that Web services took early on in the Internet story (circa 1999-2000) with SOAP, WSDL, UDDI, and WS-I Basic Profile turned out to be definitely not the right answer for the vast majority of integration scenarios (we'll see why below), despite the continued prevalence of these approaches today in most enterprises.
In contrast, the vast living laboratory of the Web has provided a singularly different answer, than has a fundamentally different focus though it remains a close cousin to traditional SOA. This much more Web-oriented approach is something that many have called Web-Oriented Architecture (WOA) and is based on the immense tensile strength of the World Wide Web itself and its underlying architectural fundamentals. And it's based on the basic concepts and rich outcomes that have made the Web far and away the largest open network on the planet as well as the largest SOA presently in existence. At the leading end of this is the Web mashups story with enterprise mashups being one of the major improvements to the IT landscape that WOA is heralding.
So to loosely paraphrase a famous line from history, I come to praise SOA, not to bury it. In the process I hope to explain WOA as simply as I can. This is still important because WOA just isn't standard fare yet for discussion in many IT circles while it's something that folks that build online services out on the globally scalable consumer Web increasingly take for granted. There are few vendors (though growing) that have lined up behind this pragmatic, effective, efficient, and highly popular approach, further limiting the body of formal knowledge and support available to practitioners who want to transplant this profoundly useful and simple way of connecting our systems together.
You may have noticed I've left "easy" out of my lists of adjectives describing WOA. And that's because I don't believe it's easy to set aside the last decade of evolving mindsets, habits, investments, tools, and skill sets for a better alternative, no matter how compelling or promising. And WOA is more than just a way of building Web services, it's also an ecosystem mindset and as such can at times be less accessible to non-system thinkers, particularly if said alternative is not well documented in the industry. And though significant headway has been made recently, particularly with an excellent spate of books such as Sum Ruby's superb RESTful Web Services, the grassroots, emergent nature of WOA has not lent itself well to extensive formal documentation. Despite the improvements in the available literature, I still find that a succinct, direct, and complete explanation of WOA is lacking and here's my attempt to boil it down to the essential principles. I then contrast it with traditional SOA so that the differences and similarities are clearly highlighted. Here's what I've come up with:
The basic tenets above paints a picture of a galaxy of nearly infinite granular information resources integrated into a deeply interconnected set of dynamic connections that can be processed individually for a given task, in part (for integrated applications), or as a whole (such as enabling a comprehensive directory or search engine of all data and metadata.) In other words, the Web model provides a single, open, and unified information architecture that is consistent, easily consumed, extremely scalable, securable, very reusable, resilient, and highly federated. The Web itself is the single largest example of this and increasingly, enterprises are adapting their existing IT systems and legacy silos to this model, discovering the advantages of this ecosystem model for information resources and enterprise architecture.
But the basic tenets previous list are at the architectural level. How does an SOA practitioner ensure they are implementing a WOA model as they build WOA Web services and resource-enable existing services and IT systems. Here are the key guidelines to be aware of:
I should close by emphasizing that I enjoy and use traditional SOA technologies like SOAP, WSDL, and XSD frequently. But as more and more of the consumer Web moves to a more Web-oriented model, the evidence continues to mount that approaches based on WOA are easier to implement, scale better, and result in much greater uptake and usage scenarios. Traditional SOA is facing a crises of identity at this point, particularly given fairly lackluster results for most, and WOA may just be the prescription we need to make SOA deliver the robust outcomes that we were formerly expecting of it. Especially read the article I wrote last year (below in the reading section on eleven new ideas for SOA architects) to show the promise of and a new vision for user-controlled SOA and other aspects that WOA enables and that traditional SOA tends to constrain.
Other vital reading on the convergence and evolution of SOA, WOA, and Web 2.0:
Note: I reserve the next few days to clean up and revise this article. In the meantime, please share your SOA vs. WOA stories and opinions below.
Update: ZDNet's Joe McKendrick provides some coverage of these ideas in a new post.
Thursday, January 10, 2008 #
In my recent 2008 predictions for the future of Web services and open APIs for enterprise applications, I said that we'd finally see a large scale movement to newer, lightweight Web-based models for opening up our software systems and integrating them together. In other words, heavyweight SOA has finally fallen out of favor and lightweight SOA -- sometimes known as Web-Oriented Architcture (WOA), is in.
However, this sea change has long since taken place on the Web and this year will see best practices in this area take another major step forward as we'll examine below. The recent convergence of the Web, SaaS, SOA, and other approaches has also made the boundaries between our architectures and systems increasingly intertwine and blur. As part of this evolution, we have also watched the gains that successful firms like Amazon and Facebook have made by opening up their products on the Web. And strategically, as an industry, we've begun to find it a lot smarter to think in terms of reusable, interconnected open platforms instead of single-play software applications. Along this journey, we have begun a major return to the roots of the globally linked structure of the Web.
The rest of this post consists of two sections. One conceptual and one technical:
The daily reinforcement and continuous growth in the fundamental power of HTTP and URL link structure, which is directly driving the Web's overall network effect, has starting giving rise to a new generation of software architects and product designers. This generation has grown up deeply influenced by it and they tend to think about the creation of software in novel, new, highly Web-oriented ways. Though the classical software industry has a long and proud heritage of its own around methodologies, architectural approaches, and design patterns -- proven in the crucible of real-world implementations of years past -- in this decade the Web has managed to exert its own unique, irresistible, and pervasive influence on virtually all aspects of producing software. For example, agile processes have been pushed to the limit and beyond by the forces imposed by the realities of the Perpetual Beta. And the scale of even average sized applications on the Web are now the largest we've ever seen. The absolute necessity of cost-effective operations and the marketplace requirements of embracing the new business models for Web 2.0 applications -- including advertising, user generated content, and rich user experiences -- have also changed the fundamental technical and commercial ground rules for success. As a whole, these changes have been driving a need for new software platforms that are explicitly designed to help us efficiently produce scalable, compelling online applications while also addressing the reality of modern-day Web apps.
Many of us who have to create the next generation of Web applications have been taking a hard look at the new platforms that have been created for the modern era of very large-scale networked software applications. And I'll be very clear here: While a great many of the old ideas and techniques in software development are as applicable today as they were ten years ago, there are also an whole new set of constraints and enablers for which we have to be very good at optimizing. As the Web begins its 2nd major wave of maturity -- and depending on who you listen to -- there is a considerably less tolerance for older, inefficient methods for developing Web applications; vigorous online competition for marketshare and increasingly online-savvy businesses have a much better sense of what is possible and how much it should cost and when it should be delivered.
These factors as a whole have pushed us into a new era of productivity-oriented platforms that started years ago with languages like Tcl and Perl and quickly moved on to Python, PHP, Ruby. Ultimately we ended up where we are today, with advanced, highly-efficient frameworks for these languages such as Ruby on Rails and CakePHP. These tools now let us create Web applications literally 10 to 20 times more efficiently than the general purpose language platforms of the 20th century, and with both traditional software engineering as well as new Web 2.0 best practices already built in. These improvements have only spurred what can only be called a "radical" movement in the software business, which started with open source software (the peer production of software) starting in the 90s, and where we have arrived, with a dramatic departure from the way we used to look at software languages and platforms in terms of how vertical a software development platform could be before it lost general appeal.
These new efficiency gains and vertical focus, however, are almost exclusively aimed at the twins goals of developer productivity and good design. These are both admirable and important goals since programmer time always been one of the leading costs in producing software. Software applications also spend most of their lifetimes in maintenance mode and clean application architectures from the outset can greatly facilitate updates and revisions. However, over the same time frame, the run-time efficiency of our programming environments, partially obscured by a little help from Moore's Law and Nielsen's Law, has been in a major decline. This has been largely intentional, when it comes to supporting improved developer productivity, or entirely unfortunate, such as the general failure of the software industry to figure out how to help software designers fully leverage the now ubiquitous generation of multi-core processors.
Out of all this there has grown a distinct and growing tension between the need to rapidly and inexpensively produce quality software and the requirement for it to scale cost-effectively to millions of users. The simple fact is, which you can readily see in the Hard Metrics diagram to the right, is that the previous generation of programming languages and platforms is up to 40x faster than what many of would prefer to use today to develop Web applications. Yet the more you go to the left on both diagrams, the more that programming platform becomes extremely expensive and time-consuming to develop with. Why is this? There are two primary reasons.
One is that the more popular, older programming languages tend to be relatively low level and general purpose and were designed for a different, older set of constraints. This has given us baggage that is often not very applicable to the modern Web-based world. Second, we've become very good at understanding the idioms and "syntactic sugar" that makes developers more productive for Web development and we've put that into the latest generation of programming languages and Web frameworks. Unfortunately, the combined newness of these new Web development platforms and their preference for coding time efficiency in favor of run-time efficiency has conspired to make the results they produce relatively slow and resource inefficient compared to what is potentially possible. Newness in this case is also a kind of performance tax since we just haven't had enough time learning how to make these new platforms perform well at run-time, similar to early versions of Java before the advent of the Just-In-Time (JIT) compiler. Fortunately, efforts like Ruby .NET have made some notable headway in this space recently, but are not commonplace yet.
The intent of the rest of this article is to explore the new release of Ruby on Rails 2.0 and examine it in the context of the trends above. The ultimate 10 million dollar question in the Web development platform arena is: Are the developer productivity benefits, including the embodiment of many current Web application best practices, that are conferred by new generation Web development platforms like Rails worth their cost in terms of operational efficiency? Increasingly, whether you're a corporate IT executive or a programmer at an Internet startup, you're going to be facing this difficult decision when you choose your target platform. Questions like "is it worth 5x-10x the programmer time to get run-time efficient software?", or "should I just increase the investment in more processor cores and bandwidth in the data center?" will keep you up at night. Making the wrong choice has potentially serious long-term consequences in terms of what it will ultimately take to maintain and operate your application. A programming platform's implications for operations are particularly pronounced since Web apps require more operational resources on the server-side the larger they grow, unlike traditional, installed stand-alone applications.
One way of thinking about the problem is that it's almost never a good idea to bet against significant improvements in computing and network bandwidth. So far we've not yet seen much to indicate that large, regular improvements won't continue for the foreseeable future. Another is assuming that a platform should be used in a slavishly monolithic fashion for an entire application. In fact, as an insightful interview with Alex Payne, a lead developer of one of most well-known Rails success stories, Twitter, shows it often makes sense to move the slowest parts of the app into something faster. This is such a common situation in software development that it's long been codified as the Alternate Hard/Soft Layers pattern. And while these two considerations alone will go a long way towards helping one decide which direction to take, one must also look to where the industry is going as a whole. The new productivity-oriented platforms are here to stay and adopting strategies to use their strengths effectively while being proactive in addressing their weaknesses, is the best route to success with 21st century Web applications.
Where's the interface? REST doesn't have a contract description language and essentially uses duck typing. Read about best practices for WOA/Client development.
So all of these issues form the lens through which we must look at the modern Web 2.0 applications arena. But let's take an actual look at what we're talking about here. How efficient can these new development platforms really be? And do they are actually encourage us down the right paths in term of modern best practices in the Web 2.0 era? Let's validate this by actually building, hands-on, an entire Web application using one of these new productivity-oriented programming platforms, specifically using the newly released Ruby on Rails 2.0. Those following along will need a little bit of technical skill, but you'll see that these new platforms are tremendously efficient from a developer perspective. In fact, we'll have an application up and running in literally a few minutes after you get your Rails 2.0 environment installed.
In contrast to object-oriented models for software, or the procedural models used by traditional Web services such as SOAP, REST only uses four methods, those built into HTTP itself: GET, POST, PUT, and DELETE, which themselves operate on data resources located at URI endpoints located on Web servers holding the data (typically a relational database under the covers). Consequently, REST applications tend to have a much larger (and transparent) set of surface area dependencies directly on sets of addressable Web data instead of on bundles of procedural methods through which XML schema instances are passed.
Since platforms like Rails embody many of our latest ideas about how best to develop for the Web, it should come as little surprise that the principle creator of Rails, David Heinemeier Hansson, recently observed that the latest release of Rails consists mainly of "a slew of improvements to the RESTful lifestyle." One of the most remarkable things about Rails is how it pays more than lip service to this essential resource-oriented view of the Web. As you shall see, since open APIs are one of the hot topics in the Web applications business at the moment, it's nice to know that every Rails app automatically gets its very own RESTful API. So, let's see this for ourselves...
To explore developing a Rails 2.0 app and creating/using its open Web API, you'll need to install four pieces of software on your test computer.
gem install rails -y
rails --version
Rails 2.0.2
rails railsapp
cd railsapp
ruby script/server
Now we're ready to start development and testing of our Rails 2.0 application. Keep WEBrick running in one command line window so you can see its debug output (it will show you all the HTTP requests that go back and forth), and have another command line in the railsapp directory ready to invoke various Rails commands.
For the purposes of this demonstrator, we're going to build a very simple employee tracking application in Rails. We're going to use the newer syntax in Rails for easy creation of a full Web application with employee record creation, viewing, updating, and deletion. Rails will even create the database tables for us, all the user interface screens (albeit they will be unstyled), unit tests, and even a complete RESTful API, our final end goal.
Astonishingly, we're going to create all of this using only two short commands at the command-line. You'll see why Rails is one of the most productive Web development platforms available and this step in particular shows some of the radical ease of use that Rails proponents (myself included) are consistently impressed with.
Step 1: Create the basic employee tracking application
Rails use a well-designed Model-View-Controller architecture and in our first command, were going to ask it to create all three items for us for our employee tracking database as well as matching unit tests and cross-platform database scripts. To keep it simple, we're only going to track two fields for the employee: their name and their extension. You are welcome to add additional fields but will have to deal with the additional fields in the steps below.
In the railsapp directory, type the following command:
ruby script/generate scaffold emp name:string extension:integer
You will see a lot of output showing the files that the Rails framework creates to handle the employee data we've just specified. The command itself invoked the generate facility of Rails to create a new scaffold for a model called 'emp' which has two fields, an employee name typed as a text string and an extension typed as an integer. A scaffold is an initial, working application skeleton with basic functionality including database persistence and a matching set of HTML forms for the CRUD.
Now, believe it or not, the employee tracking application is now mostly finished, the only thing we need to do it to update the database so that it has the schema for the employee records. You can use Rails' rake facility to get this done. This will require that you have correctly set up your database.yml file, and you will have to debug any connection issues to get this step to work. To migrate our employee model, named 'emp' to the database, type the following in the railsapp directory:
rake db:migrate
Now, you can run the application by going to the emps directory on the WEBrick instance. Note that emp as been pluralized automatically by rails, so our applications is located at the emps endpoint. To access our new Rails 2.0 Web app, point your browser to:
You should see the listing screen of the employee tracking database that looks like the one below:
Click on New emp to create a new employee, and enter the data as seen below (aside: Roy Fielding is co-inventor of HTTP and the person who created the original vision around REST), and click on Create.
The data entered is then transmitted from the browser to the server and stored in the back-end database. You can then view it, destroy it, or add more employees with the user interface that was generated for us by Rails.
We've now completed a simple but fully functional Rails application from beginning to end. But what we've come here to see is the fully RESTful open Web API that was created for us along the way. For this we'll need to use cURL to issue the API calls via HTTP to simulate another online program integrating live with our Web application.
Step 2a: Invoke the REST API to GET the employee resources
Now we're going to exercise all the HTTP verbs on our open Web API to see how it works. The diagram below shows the overall lifecycle of a REST-based resource using our emp example. The good news is that Rails automatically offers URL addressable resources for all the data in a Rails Web application. This access can be controlled and channeled as needed but it's open by default for whichever views already have visual access via HTML forms. This means Rails developers get a RESTful API for their applications simultaneously as they develop their user interface.
Let's go ahead use the REST API to pull the data for the employee that we added above. We'll use the handy HTTP utility, cURL, to interact with the Rails application via HTTP. Note that the URL we'll use now has the '.xml' extension added to it. This tells Rails that we're trying to access the XML representation of the resource instead of using the HTML user interface (in other words, we're playing the role of a program instead of a human user.)
curl http://localhost:3000/emps.xml
Enter the text above in a local command-line or shell with the cURL binary in the execution path. You should see output that looks similar to the following below. It's an XML representation of the employee data in list format, pulled fresh from the server via the REST API, with the emps tag as the enclosing list structure holding individual emp instances.
<?xml version="1.0" encoding="UTF-8"?> <emps type="array"> <emp> <created-at type="datetime">2008-01-11T01:02:53+01:00</created-at> <extension type="integer">1234</extension> <id type="integer">1</id> <name>Roy Fielding</name> <updated-at type="datetime">2008-01-11T01:02:53+01:00</updated-at> </emp> </emps>
It's idiomatic in Rails to use the id attribute as the primarily key for application data. In fact, this convention is required for a lot of the magic in Rails to happen automatically and the rake migration way back in Step 1 already took care of adding this column to the database for us. That means we can use the id as the final addition of our employee resource URIs for updating, getting, and deleting individual employee resources.
Step 2b: Update an employee resource through the REST API via PUT
Let's go ahead and update Roy Fielding's phone extension through the REST API. Since we can tell from the employee list above that Roy's id is '1', we can use that to let the API know which record we'd like to update. You only have to send two parts of the resource in the API call, the id and the attributes we'd like to update.
Create a file called put.xml with the following contents:
<?xml version="1.0" encoding="UTF-8"?> <emp> <extension type="integer">5678</extension> <id type="integer">1</id> <name>roy fielding</name> </emp>
Invoke cURL with the following parameters to actually update the phone extension in the resource on the server (and consequently in the database.) The -H parameter sets the header so that Rails knows that an XML representation of the resource is being sent to it. -T makes the HTTP invocation a PUT operation, and the URL of the resource is http://localhost:3000/emps/1.xml where the number 1 corresponds to the id of the resource:
curl -v -H "Content-Type: application/xml; charset=utf-8" -T put.xml http://localhost:3000/emps/1.xml
Step 2c: Add an employee resource through the REST API via POST
Now we'll add a new employee to our application over the network using the REST API. This employee will be Tim Berners-Lee, so we'll create another XML file called post.xml that looks like the following:
<?xml version="1.0" encoding="UTF-8"?> <emp> <extension type="integer">1212</extension> <name>Tim Berners-Lee</name> </emp>
To send this via a POST operation through the REST API using cURL, issue the following on the command line. The --data-ascii parameter identifies the file to send via HTTP to our REST API. Because the resource does not yet exist, the URL is the base of the resource type, http://localhost:3000/emps. Rails conveniently returns the XML representation of the added resource so the id generated on the server for the newly added record can be obtained in the client without a second call to the server. Add Tim Berners-Lee to our employee tracking application via the API:
curl -v -H "Content-Type: application/xml; charset=utf-8" --data-ascii @post.xml http://localhost:3000/emps.xml
A browse of the employees list via cURL or the employee tracking apps Web forms will see that Tim Berners-Lee has now been added to the application, including the database, via the REST interface.
Step 2d: Delete an employee resource through the REST API via DELETE
Now we'll go ahead and remove Roy Fielding from the database using our REST API. This process is straightforward and uses the HTTP verb DELETE. You can issue this via cURL using the following command:
curl --request DELETE http://localhost:3000/emps/1.xml
You can now verify through the employee tracking Web forms that Roy Fielding's employee record has been permanently removed from the database.
We've seen how Rails 2.0 makes it enormously simple to create a database-driven Web application, expose it via a REST API, and manipulate it via a REST-capable client in a clean, no-nonsense manner. Developing similar capability in C++, Java, or .NET environments is currently much more difficult. What you see above however, is only the beginning; Rails 2.0 has added a lot of other support for more sophisticated uses of REST and HTTP. I'll cover these in one of my upcoming posts as soon as I am able. The key point here is that the next generation of Web application platforms puts almost staggering amounts of power in the hands of the average Web developer while providing powerful capabilities like properly formed REST APIs automatically. This further puts the latest best practices for Web apps into places it wouldn't otherwise happen. Open APIs will help power the next generation of online success stories and for this and other reasons, Rails should be on the short list for those considering new Web development efforts. That is, only if they are prepared to do what's necessary to address Ruby's and Rails' shortcomings in run-time performance.
Still trying to exactly understand why Rails is such a compelling option? Read an analysis of why platforms like Rails are a major improvement over previous generations of Web application platforms.
If you have any trouble getting the code to work, please contact me at email@example.com.
Powered by: .Text and ASP.NET - All text and images on this blog are licensed by Hinchcliffe & Company, 2005-2009. This work by http://hinchcliffe.org is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License.For more information, please contact email@example.com. - Dion Hinchcliffe


text:
Hi my name is diane. On april on 2011 I started finding out stuff about common variable immunodeficiency. I have always had something medical going on with me I was born a month early. And failured to thrive when I was first born. I was...
A Common Variable Immunodeficiency anonymous support group with information on diagnosis, treatment, symptoms, along with personal stories and experiences with Common Variable Immunodeficiency. You're not alone.
Other experience groups that have been combined into this group include: None yet.
This is an experience group where everyone says: I Have Common Variable Immunodeficiency. It features support, personal stories and experiences, advice, chat, talk, forums, videos, pictures and resources from real people, just like you.
It's free to join, and the people in this group would love to meet you! If you've had this experience, or plan on having it in the future, simply click 'Me Too' or 'Plan To' and add it to your experiences. As you add more experiences, we're better able to
recommend interesting people to you that share many of your personal experiences.
Login or Register to get started in seconds.
Login or Register to get started in seconds.

subset: commoncrawl