While some will dispute what mainstream is defined as exactly -- with my own personal favorite being when my grandparents and their grandchildren both are doing whatever is under discussion -- the rise of consumer-powered media platforms has all the hallmarks of being something that's not only here to stay, but something that's increasingly pushing everything else off the stage.  Yes, I'm talking about blogs, but also wikis and every other kind of two-way, user controlled participation tool that is currently proliferating on the Internet in every country and almost all demographics.

Now before I present my case for the mainstreaming of shared, collaborative media, we should more carefully define the term that captures this best: social media.  Wikipedia of course has the most easily accessible definition of social media, describing it as "online tools and platforms that people use to share opinions, insights, experiences, and perspectives with each other. Social media can take many different forms, including text, images, audio, and video. Popular social mediums include blogs, message boards, podcasts, wikis, and vlogs."  The key here is that people are the ones that use and control these tools and platforms instead of organizations and large institutions.  Further, I would add to this that social media platforms tend to work best in networked environments , particularly on the Web, but also behind firewalls though to a lesser degree.  Why is the networked aspect so important?  Primarily because it's a powerful democratizing force due to its pervasive, low cost nature; anyone can get in the conversation with only a small investment of their personal time and access to a network.  And since communication is essentially free over computer networks today, combining an architecture of participation powered by network effects makes social media platforms almost certainly the most powerful form of media yet created.

These todays anyone posting anything on a simple blog lets them automatically reach the 1.1 billion users on the Web today.  And with syndication, social media content is picked up and spread throughout Internet via feed engines and the entire syndication ecosystem and can be found by anyone looking for information via Technorati, Google Blog Search, TechMeme or dozens of other innovative discovery mechanisms. At long last, hundreds of years after the invention of the printig press, anyone can truly reach a global audience by spending a couple of minutes of their time creating a blog on one of the hundreds of free blog sites.  I've highlighted in the past how social media has been used in both emergent and deliberate fashion to do everything from locating the survivors of natural disasters to motivating end-users en masse to create online video advertisements for a major corporation.

Of course, any effective technique or phenomenon has those who attempt to co-opt it or copy it, the latter which is the most sincerest form of flattery.  The recent Public Relations 2.0 flap, which ostensibly boiled down to whether or not traditional organizations can even conceive of how these new freeform platforms work, was a good example of how institutions firmly grounded in the 20th century struggle to understand the power shift under way.  Because these platforms are no longer under anyone's control for the very reason that the Web is a system without an owner, except all of us together.

Bounding the Social Media phenomenon 

But how significant is this really?  What are the compelling datapoints that tell use that social media is changing the landscape of communication, collaboration, and personal interaction?  David Sifry's quarterly State of the Blogosphere, most recently updated in October, is an excellent place to start. Taking a look at this, we can tracking over 57 million blogs, with over 900,000 blog posts a day on just about any conceivable subject.  3 million new non-spam blogs were created in just the most recent 3 months of tracking.  But blogs are primarily text and there are many other forms of social media and so it's worth looking at podcasting and video, two important types of social media that are growing rapidly with the spread of high quality, fast Internet connections.  Fortunately or unfortunately, unlike blogs, podcasts or video sharing do not have their own syndication system and for the most part they just ride inside the existing RSS/ATOM feed systems.  This makes it hard to discern what is really happening and so we can only pull on some individual data points such as Google Trends data showing the rapid rise of podcasting as a search term.

The video side of social media is a bit easier, which Hitwise and YouTube providing enough hard data on the most recent version of the YouTube Fact Sheet to get a general though unscientific impression of what's happening there.  According to this, YouTube has 60% of all online video viewers with up to 70 million viewers in an evening and over 65,000 videos uploaded every day, making it both the #1 online video site and #1 social video sharing site online.  This implies that most video consumption on the Web is already based on social media, and that there are over 115 million online viewers of video overall.  At least for video, social media is not an edge case and is they dominant model overall. Note: Yes, one can quibble about whether YouTube is truly a social media site and certainly it skirts the concept but in my book it makes the list. 

Why is YouTube considered Social Media though?  What aspects does it -- any many of the most successful media sites -- have that make it social and non-coincidentally so popular?  To understand this best, it's worth creating a list of what exactly must an aspiring social media platform actually have in order to be considered such.  Here is my take, culling the capabilities and features of the most popular social media sites as well as the consensus of leading thinkiners in this space such as Stowe Boyd, Tina Sharkey, and others.

Defining Social Media: Some Ground Rules
(as we understand them circa January 2007)

1. Communication in the form of conversation, not monologue.  This implies that social media must facilitate two-way discussion, discourse, and debate with little or no moderation or censorship.  In other words, the increasingly ubiquitious comments section of your local blog or media sharing site is NOT optional and must be open to everyone.
2. Participants in social media are people, not organizations.  Third-person voice is discouraged and the source of ideas and participation is clearly identified and associated with the individuals that contributed them.  Anonymity is also discouraged but permissible in some very limited situations.
   
3. Honesty and transparency are core values.  Spin and attempting to control, manipulate, or even spam the conversation are thoroughly discouraged.  Social media is an often painfully candid forum and traditional organizations -- which aren't part of the conversation other than through their people -- will often have a hard time adjusting to this.
4. It's all about pull, not push.  Like John Hagel and John Seely Brown observed in the McKinsey Quarterly a year ago or so, push-based systems, of which one-way marketing and advertising and command-and-control management are typical examples are nowhere near as efficient as pull systems.  Pull systems let people bring to them the content and relationships that they want, instead of having it forced upon them by an external entity.  Far from being a management theory, much of what we see in Web 2.0 shows the power of pull-based systems with extremely large audiences.  As you shape a social media community, understanding how to make embrace pull instead of push is one of the core techniques.  In social media, people are in control of their conversations, not the pushers.
5. Distribution instead of centralization.  One often overlooked aspect of social media is the fact that the interlocutors are so many and varied.  Gone are the biases that inevitably creep into information when only a few organizations control the creation and distribution of information.  Social media is highly distributed and made up of tens of millions of voices making it far more textured, rich, and heterogeneous than old media could ever be (or want to be).  Encouraging conversations on the vast edges of our networks, rather than in the middle, is what this point is all about.

The rise of social media platforms within businesses, often dubbed Enterprise 2.0 , will place a significant challenge on organizations as they try to grapple with the ground rules above.  That's because not following them will tend to reduce the long-term success and effectiveness of social media in business.  Also, increasingly, as more and more time and world-wide attention is given to social media, who really owns the discussions online will become a bigger and bigger deal.  YouTube recently announced they will begin paying their users for their video contributions (which are the seeds for often virulent conversation on that site), but they still place far too many restrictions on the content that is uploaded including making it belong to YouTube.

 Both of these trends show that when users are in control via the highly democratizing tools of the Web, the fundamental ground rules change.  Understand them, follow them, and embrace them, this is the pre-eminent media model for the 21st century.

These aren't the only rules for social software however, just social media in particular.  Be sure to check out my Notes on Making Good Social Software for more good ideas.

What else did I miss? What makes social media uniquely what it is? 

While the window on using the "2.0" suffix is probably closing, I thought it would be worthwhile to explore an especially significant trend in 2006 that will likely see much more widespread uptake in 2007.  Specifically, I'm talking about building highly competitive online products by turning over non-essential control to users directly via the Web.  For now, I'm calling this online business trend "Product Development 2.0", a concept that embodies the use of Web 2.0 concepts such as harnessing collective intelligence, users as co-creators, and turning applications into platforms, three of the most powerful techniques in the Web 2.0 arsenal.

What is Product Development 2.0 exactly?  It's an informal term I'm applying to something that online startups and traditional businesses both are increasingly doing: leveraging of mass user contributions, providing open architectures for others to build on as they like, and even handing control over key product decisions directly to users.  The reasoning behind doing this is simple:  Satisfied customers have always been essential to having the most successful business, both online and offline.  But how best can you ensure that they get exactly what they want from you, as customized and quickly as possible?  This is where the scale, new tools, and business models of Web 2.0 have stepped in, giving us the potential to provide our customers with better, rich products, much more quickly, and with more of what they want.  Taken as a whole, it's increasingly clear that there are new business models afoot that are just now being well understood.

Given that any business typically is vastly outnumbered by its customers and potential customers, and that putting a bureaucratic, centralized product development team into the critical path of product creation and ongoing maintenance highlights how little we can actually serve them, especially in an individualized way. And with everyone online, it's increasingly obvious where the biggest source of talent, engagement, innovation, agility, and worker bandwidth really lies: with your customers.  Using the techniques and technologies that have emerged in just the last few years, you can now finally give them the tools and motivation to tweak, tune, refine, and contribute to your products and services.  And increasingly, they'll probably do it.  YouTube is still currently one of the best examples of user co-development of a world-class product in its pure form (65,000+ videos uploaded by users per day), but sites like eBay, Slashdot, and many others have been leveraging their users in product development for a long time now.  And as it turns out, Product Development 2.0 is not a small topic and starts off at collecting explicit user contributions, leveraging the Database of Intentions, and putting in automated real-time feedback loops to identify the best or most popular new content or capabilities for other users that come along later.

It's important to note that it's a fundamental shift for a business to turn over a large part of its product development to its users, becoming more of a mediator and facilitator than a product creator or owner.  This is the shift of control from institutions to individuals that the apparently relentlessly democratizing force of the Web has begun exerting on the business models of organizations of every description around the world.  As more organizations figure out how to apply Product Development 2.0 to their individual offerings, they will reap significant competitive advantage over those not harnessing the Web to directly connect to customers and begin a rapid and never-ending innovation cycle.  This is another aspect of the perpetual beta concept that reflects the fact that increasingly, products and services online are never finished, and indeed, can't ever be finished as changes and additions seamlessly pour in over thousands of millions of Internet connections.

But enough about the possibilities.  Let's talk some examples, both in terms of what older style product development did vs. what this new style is doing.  Finally, let's talk about some companies actually doing this successfully.  Note: Incidentally, though I normally write about services in terms of Software as a Service (SaaS) or Web Services, for the purposes of this discussion I'm talking about non-physical business processes for sale, such as car or medical insurance, tax preparation, etc. and not software.

Like the recently discussed Programming 2.0 concept -- a set of software development tools, techniques, and attitudes that is, not incidentally, enabling much of this -- and the original Web 2.0 definition, it is examples in lieu of principles that's one of the best ways to paint a picture of what appears to be happening in the evolution of product development:

Looking back over 2006 it's clear that we've experienced one of the most remarkable growth surges in Web application history.  Literally hundreds of Web sites and applications were launched this year and brought to our attention via the popular review sites like Michael Arrington's TechCrunch, Pete Cashmore's Mashable , and Emily Chang's eHub.  And our very popular list of last year's Best Web 2.0 Software of 2005 was ultimately read by hundreds of thousands of readers in over a dozen languages.  This makes it clear that not only is the ongoing supply of capable, online software flowing freely but that there is high-demand from the general Web populace as well. 

The overall trend: We have begun moving all our software, data, and even our social activities onto the Web en masse and the demand for high-quality online sites and applications that support this shift in primary focus from the PC to the Internet is there in vast numbers (there are now 1 billion users on the Web today).  The net result is that 2006 brought us some of the best online applications ever created and you can see the results for yourself below.

Last year's Web 2.0 software list we had a variety of categories ranging from Image Storing and Sharing to Web-Based Word Processing.  Since then, the scope of Web applications has broadened considerably as has the definition of Web 2.0 itself, which has formalized and settled a bit as well.  This reflects the real diversity in online applications from every kind of social media site to online productivity apps.  Thus this year's categories have been consolidated and new categories added.  Most notably I've added a Office 2.0 Suite category to cover the growing lists of ensemble software sites such as Zoho's Office Suite that are increasingly treading squarely on the integrated feature set that traditional productivity suites like Microsoft Office and Open Office.   We've also added an Honorable Mentions section to reflect the fact that some of the new Web applications are so innovative that they nearly defy description but clearly deserve to be highlighted.

So I hope you enjoy touring the applications on this list.  Finally, this list is entirely subjective and any errors or omissions are mine alone.  You may not agree with some of the software I've listed but this isn't a one-way web; I definitely encourage you to list anything you feel we missed or got wrong below in the comments (and last year we received hundreds of submissions via comments).  Please use the wiki link syntax ([url text_desc]) in the comments to make sure you embed plenty of good links.

Note: The site did not have to launch in 2006 to make this list, it just had to provide the best offering in a given category during the calendar year.

The Best Web 2.0 Software of 2006

Category: Social Network

Best Offering: MySpace

Description: The pre-eminent granddaddy of all social networking sites, MySpace needs no introduction.  And while it's taken hit in numerous quarters for the quality of its users and content, for its lack of attractiveness, and even garnered a reputation for being a somewhat unsafe place to socialize online, it's still far and away the leader in terms of users.  Easy to use and very customizable, MySpace offers some of the best user experience as well.   And users apparently agree.  If you take a look at the (unscientific and off-the-cuff) Alexa traffic chart of the other top social networking sites, you'll see that MySpace remains an absolute juggernaut.  And it's because of this very fact MySpace remains the place where the most users are and there are now those fully taking advantage of that fact.  A key trend: Businesses are increasingly opening up storefronts on MySpace. Why?  It's easy to do and a great draw for customers in a younger demographic.  Hollywood has taken to promoting its films with MySpace profiles and countless companies are using it has a marketing and advertising platform such as HBO with their Entourage car giveaway earlier this year.  And while commercialism of just about any new media is inevitable, it reaffirms that MySpace remains the king.  That's not to say that plenty of other good social networking sites don't exist.  But they simply can't come close to matching in terms of scale, which means MySpace remains the most significant online community today.

Runners Up:  

Category:  Start Pages

Best Offering: Netvibes

Description:  Netvibes won this category in last year's list and also gets the #1 spot this year.  The start page pheneomenon has been an interesting online Web app trend that got underway in 2005 with the release of numerous different products in this space.  In short, start pages provide a roaming desktop that can host all of a user's most common Web information such as news, weather, e-mail, RSS feeds, and more, all in a single user-controlled Web page.  My overview of these earlier this year on ZDNet was Slashdotted, just another indicator of the apparent popularity of these personalized Web desktops, usually powered by Ajax but often by Flash as well.  During 2006 however, not many of these products saw serious growth and their visitor traffic growth has been slow.  Except for Netvibes that is, which has been growing month by month by offering things like an extremely polished look and feel, localization in many different languages, and open API.  The last piece is critical for allowing others to add to and build upon the Netvibes platform (turning applications into platforms being a key Web 2.0 technique) and result of this shows clearly in the Netvibes product.  The Netvibes developer ecosystem is vibrant and growing with over 500 different add-on modules from Comic of the Day to a module that will quickly turn any document into a PDF file.  While Live.com has much more overall traffic than Netvibes, it's likely due to Microsoft's own mega-ecosystem since personalization has moved to the back burner of the front page of Live.com and has been upstaged by Microsoft's search engine.  Click here for a more complete list of existing start pages.

Runners Up:   

Category: Social Bookmarking

Best Offering: StumbleUpon

Description: StumbleUpon has unseated last year's winner, del.icio.us.  Search engines like Google can help you find the material you're looking for using keywords, but social bookmarking sites can let you directly harness the collective intelligence of other users on the Web the directly share personal interests with you.  Theoretically, this can help you find what you're looking for better, but what it really ends up doing is helping you find things that you never knew existing, but wished you did.  StumbeUpon installs a toolbar in your browser and lets you collaborative rate content.  This improves the recommendations for other users and behavior matching is used to find users like you and pages that you haven't seen before, on-demand.  One indicator I use for the popularity of a social bookmarking site is how much inbound traffic I get from it, and I've seen a clear switch during the year from del.icio.us bookmarks to StumbleUpon referrers.  StumbleUpon reports that it has over 1.7 million registered users and growing.  Bottom Line: Del.icio.us is still my favorite bookmarking service, but for true content discovery, StumbleUpon now makes it much easier to find new content than del.icio.us does.  StumbleUpon is a winner by a nose for taking content discovery to the next step.

Runners Up:  

 Category: Peer Production News

Best Offering: Netscape.com

Description: In a decision that likely won't be agreed with by the users of last year's winner in this category, Netscape has been selected as the best all around peer production news site.  Though Digg is more popular in terms of traffic than the next three most popular peer production news sites in this category combined (though only barely), Digg remains primarily a technology news site, with actual general purpose news seeping in occasionally around the edges.  In contrast, Netscape consistently delivers news on its front page that is genuinely newsworthy and geared towards a broad audience, combined with a mature community that frequently engages in genuine civil discourse in the comments.  This highlights the demographics of the site of course since peer production sites have the news stories delivered by their users and the top stories selected by other users.  Thus Netscape currently provides the best overall mix of news content and community and wins this year's peer production news category.

Runners Up:  

Category:  Social Media Sharing

Best Offering: YouTube

Description:  The rise of YouTube this year has been one of the most phenomenal rises of an online property in Internet history.  With up to 100 million viewers in a given day and averaging 65,000 videos uploaded per day, YouTube has successfully leveraged network effects for growth and viral adoption with a success that few have ever equaled.  Last year image sharing was the hot social media sharing play, but 2006 is clearly the year of video.  You can find a video on just about anything you can think of on YouTube and its radical ease of use, innovative tagging infrastrcture, and drop-dead easy to host YouTube badge (with the Javascript snippet it for it right next to each and every video) sets the standard for the rest of the industry.  The selection for this category was easy and YouTube was the clear choice. 

Runners Up:    

Category: Online Storage

Best Offering: Amazon's S3 with JungleDisk

  with 

Description: I did a round-up earlier this year of most of the leading online storage products (and there are many), but the one that I have ended up using the most by far and ultimately selecting as my permanent online storage solution is Amazon's terrific S3 storage Web services API combined with Jungle Disk for Windows Explorer integration.  S3 stands for Simple Storage Service and that's exactly what it is.  There's no limit to how much data you can store with S3, how much data you can transfer to and from your home or work PC from S3, and S3 is very fast, reliable, secure, and cheap.  I now host hundreds of gigabytes of data in my S3 account for a few dollars a month and I can access it from anywhere I travel without having to worry about backups or otherwise maintaining my data to make sure it's not lost (Amazon does it all for you).  While there are other good online storage solutions, nothing comes close to the freedom and security of using S3 since Amazon is one of the leading Internet companies and will likely be around for a long time.

Runners Up:   

Category: Office 2.0 Suite

Best Offering: Zoho Office Suite

Description: The Office 2.0 phenomenon become a true reality this year as just about any kind of business application could be found in a purely browser version.  Zoho has been diligently releasing product and product this year and now has entire online productivity suite that has a word processor, spreadsheet, wiki, project management, presentation, contact management, and much more.  While you can find the individual pieces from various other Web apps, Zoho provides a nice integrated, one-stop package that is very reminiscent of Microsoft Office. Microsoft and Google have been slow to get fully into this space and it may very well end up that smaller players establish dominance in an area that most expected the Big Two would dominate in this space.  And an important space it is too: Online apps ultimately will be where our software and data is for most users, and establishing leadership in this product space with the Web as the only major new software paltform on the horizon is a major open opportunity.

Note: Last year I broke the individual categories of Office 2.0 out, and with the overall quality of such tools now being fairly consistent, I'm now highlighting the suite aspect as an important trend trend in 2006. 

Runners Up:    

Category: Honorable Mentions

Blog Filters:  Like last year, Gabe Rivera's brilliant meme engine for the blogosphere still reigns supreme as far as taking the pulse of the conversation on the Web right now.  And its permalinking structure with history support is just about the best example of Web design and content addressibility that I've seen.  If you aren't using TechMeme daily to see what's going on, you don't know what you're missing.

Social Music: Online music doesn't get easier than Pandora, which has now become my favorite way to discover new music.  Just a single Web page written in OpenLaszo, Pandora creates a custom radio station for every visitor in seconds based on the names of artists or songs you know, and then continously plays new music related to what you suggested.  Now with social features, Pandora serves The Long Tail of music demand very nicely and is very easy to use and it shows: Pandora reportedly has over 2 million users.

Professional Social Network: 2006 was the year that having a LinkedIn profile was almost mandatory if you were in business, particularly now that all profiles have a URL.  Almost everyone has received a LinkedIn invitation at some time or other, and LinkedIn really made it on the radar this year.  While lacking robust social networking features such as blogging, LinkedIn's core functionality of maintaining a network of contacts that is automatically updated as people move around from job to job is just about the best out there.

Consumer Generated Advertising: The Chevy Apprentice campaign was just about the best example of a true Web 2.0 phenomenon as GM opened up the doors in early 2006 of a competition for anyone to create online videos about the Chevy Tahoe SUV, tends of thousands which were ultimately created and submitted.  GM even left the negative ads up and sparked a real conversation about how much control of their marketing message should companies hand over to their customers.  Since the original competition site is no longer online, click on the picture above or here to see the YouTube hosted copies of the ads that were created, some of which are very creative and are just as often negative as they are positive.

Online File Conversion: There is a growing list of online file conversion sites, but Zamzar has an impressive list of support file formats for documents (including MS Office docs), images, audio, and video including WMV, AVI, and many more.  More importantly, the site is incredibly easy to use and very handy when you need to do an urgent file conversion while on the road or want to avoid the hassle of the numerous freeware downloads.

Web Application Stack:  Ruby on Rails took a front seat this year as it become one of the most popular new ways to develop online database-driven software, Web 2.0-style (collective intelligence apps) or otherwise.  I wrote up a more detailed story about Ruby on Rails for ZDNet that's worth reading if you want more details but the big take away is that Ruby on Rails is optimized for ease-of-development, extremely rapid results with little effort (10-20 times more productive that previous platforms like J2EE and .NET).  I suspect that in 2007 the majority of new Web apps will be developed in Rails or PHP, they're just that much better.

Mashup Tool:  While next year will see the release of a flood of end-user mashup tools, a few good ones hit this year, but DataMashups.com gets the credit for getting there first and with a surprisingly robust product.  I recently wrote up the state of mashups for 2006 as well as a round-up of mashup tools , and while it's still an product space that is in its very early stages, the promise is impressive for users to soon be able to assemble the software solutions they need onthe fly.  Expect the mashup tool market to start growing rapidly in 2007.

And that's it for now.  And since this is a Web 2.0 blog, please do contribute your own mentions and nominations below and I'll do an update a few times with some of the best suggestions so we can make this the best Web software list of 2006.

The end of 2006 is nigh upon us and this blogger for one had a terrific time covering Web 2.0 for those of you that are interested in following the topic.  Love or hate buzzwords, there's little question that subjects related to Web 2.0, from its convergence with SOA , to the rise of rich user experiences including Ajax, to a flood of exciting new largely user-powered online applications both inside and outside the firewall and much more, were all very popular with our readers and covered here in as much detail as possible.

2006 was filled with significant events for us with regards to the next generation of the Web.  During the year we participated in Microsoft's SPARK event, helped organize The New New Internet conference with great appearances by Michael Arrington and Andrew McAfee, launched AjaxWorld magazine in its print edition as editor-in-chief , and delivered numerous talks around the country on RIAs and Web 2.0 design patterns and business models for conferences including Interop, AjaxWorld, Office 2.0, and many others. 

A quick look at the trends tell us that 2007 is shaping up to be even bigger than last year as an even larger, more general audience continues to develop interest in the possibilities of applying Web 2.0 patterns and best practices deeply into the core of their products and services both existing and new.  Harnessing collective intelligence via network effects and feedback loops became generally understood as the dominant design element of the Web 2.0 by most accounts.  This was palpably reinforced by new and old companies alike including YouTube and MySpace gaining market dominance over industry leaders in just a score of months while Google and Amazon continued to use their years old network effect advantage to maintain leadership in their sectors.

But much of this entire story was driven directly by the increasing scale, size, speed and interconnectedness of the Web, making it easier than ever to reach out to tens of millions of potential users practically overnight via the 1 billion+ users that reside there in the biggest single marketplace in history.  Continued performance improvements in a number of metrics has also made much of the Ajax and RIA phenomenon possible.  This includes not just the speed of the Internet itself but the speed of the computers that the average user has as well.  Thus, the dramatic performance improvements in the overall physics of the computing experience will just continue to push the envelope of what's possible on the Web in an essentially continuous fashion.  Hopefully early adopters of the Internet such as the United States will continue investment in Internet infrastructure improvements and not let this trend languish.

While I'll save the predictions for where all this will lead in 2007 for another upcoming post, it seems clear that users, businesses, and other organizations that deeply embrace the fundamental nature of the Web as a communications-oriented platform without any single owner except all of us, will be the only ones able to fully exploit the possibilities of online applications.  Because until now actionable ideas and techniques that directly explain what the most successful ways of building online software weren't well understood or easily accessible to most.  The continually evolving model of what works and what doesn't in online applications design is currently labelled Web 2.0.  And our tools and techniques finally started to adapt to these models this year and the rise of simplicity and optimization for Web-oriented systems as exemplified by the new applications stacks like Ruby on Rails, the growing adoption of lightweight protocols like RSS, ATOM, JSON, and REST, and network effect-powered business models including building hard-to-recreate sources of data and fully leveraging The Long Tail will become the norm.  For now, the early adopters will be able to use techniques potentially heads and shoulders above their competition.  What this will mean for those that fail to embrace this is something I'll cover in a my 2007 predictions post.

With a hat tip to Rod Boothby's idea of the same, here is a summary of our most popular material on Web 2.0 this year as judged by our readers.  These are the top read posts of 2006 on this blog site with over 10,000 page views.  I do hope you enjoy:

Top Web 2.0 Blog Entries for 2006 

11. Thinking Beyond Web 2.0: Social Computing and the Internet Singularity (10,131 page views)

10. All We Got Was Web 1.0, When Tim Berners-Lee Actually Gave Us Web 2.0 (10,203 page views)

9. Notes on Making Good Social Software (10,485 page views)

8. The Ajax Spectrum (10,544 page views)

7. Why Ajax Is So Disruptive (11,320 page views)

6. Seven Things Every Software Project Needs to Know About Ajax (11,346 page views)

5. Web 2.0 Predictions for 2006 (16,531 page views)

4. Ten Ways To Take Advantage of Web 2.0 (21,666 page views)

3. Ruby on Rails 1.1: Web 2.0 on Rocket Fuel (29,204 page views)

2. The Most Promising Web 2.0 Software of 2006 (44,125 page views)

1. The State of Web 2.0 (50,147 page views) 

Stay tuned for Web 2.0 Predictions for 2007 and The Best Web 2.0 Software of 2006, coming next week. 

Despite being considered so ten minutes ago in some corners of the the Internet, Time Magazine has selected Web 2.0 -- in particular those people that are directly shaping it -- as its esteemed Person of the Year.  Specifically, Time Magazine has singled out you for recognition in this achievement and as the actual source of the exciting things happening on the Internet and in society today.  Yes, that's you, reading this right now.  At least if you've been contributing to the Web in some way using the increasingly ubiquitious tools and technologies ranging from the basic blog or wiki all the way up to video sharing platforms and social bookmarking sites. 

But the truth of the matter is that just about any interaction with the Web at all generates new content of use to someone else (the so-called Database of Intentions ) and so that means frankly, if you're currently using the Web today even just to surf, you've become an integral part of this.  "This" being a new generation of openness, sharing, and community powered by the Web that some think may be recognized in hindsight as breaking down important cultural barriers and institutions in a very similar fashion as what happened in the 1960's.  True, it often doesn't seem like a revolution to us that see it growing bit and bit every day, but taken as a whole, there's now little doubt that the Web has become the most powerful, egalatarian, and knowledge rich platform in human history.  Rapid evolution appears to have accelerated into a sort of revolution.

The Person of the Year cover story appears with the tagline that "in 2006, the World Wide Web became a tool for bringing together the small contributions of millions of people and making them matter. The cover story's lead author Lev Grossman then starts off with some fairly inspired prose after noting that there are still serious problems in the word which aregrowing in conjunction with this apparent technological Utopia, writing:

But look at 2006 through a different lens and you'll see another story, one that isn't about conflict or great men. It's a story about community and collaboration on a scale never seen before. It's about the cosmic compendium of knowledge Wikipedia and the million-channel people's network YouTube and the online metropolis MySpace. It's about the many wresting power from the few and helping one another for nothing and how that will not only change the world, but also change the way the world changes.

The cynical among us will find some of Lev's analysis to be starry-eyed and excessively optimistic, but calling out Web 2.0 by name, the Person of the Year cover story makes careful note that the mass participation we're witnessing on a grand scale on the Internet cuts both ways:

Sure, it's a mistake to romanticize all this any more than is strictly necessary. Web 2.0 harnesses the stupidity of crowds as well as its wisdom. Some of the comments on YouTube make you weep for the future of humanity just for the spelling alone, never mind the obscenity and the naked hatred.

But the lead story is just the beginning and Time has prepared an extravaganza of supporting material and documention in the form of fourteen separate stories that range across the Web 2.0 terrain covering subjects from online virtual worlds such as Second Life to an article titled in near purple prose fashion: "The Beast With a Billion Eyes -  On the Web, anyone with a digital camera has the power to change history."

None of this however is likely to please most of us who have lived through the year of Web 2.0, as 2006 undoubtedly was its big break with the term making the covers of major media properties like Newsweek and The Economist .  In terms of the blogosphere, the self-appointed contributors that are making some of this this happen, the commentary on Time's choice covered the spectrum: Jeff Jarvis agreed with most of what they wrote, just requested that they turn down the volume a bit.  Nick Carr took it surprisingly easy on the article, though he's long since posted his opinions of the Web 2.0 phenomenon.  Paul Kedrosky came in as one of harshest critics of the story series and accused it of being a blatent cop-out, what with more important issues existing elsewhere in the world needing to be highlighted. With this altter bit I would suggest that the printing press didn't get much credit at the time but it's impact was practically profound and beneficial when looking back several hundred years.

In reality, the Web as it exists today with sites like MySpace and YouTube which eagerly offer anyone who wants it an essentially permanent, scalable "channel" of their very own on the Internet, makes it possible for anyone with great -- or at least interesting -- ideas to reach the over 1 billion users that presently comprise the Web.  Never before in history has access to the largest audience of users in the world been essentially free other than the personal time it takes to contribute.  The long-term of effects of this will no doubt be as unpredictable as they will be significant as the control over information and content becomes relentlessly decentralized.  The Web is essentially a system without an owner, a platform that is under no one's control, though anyone is free to built a new platform on top of that.  Companies have had varying success in doing just that but the design patterns and business models for making the Web work best are at least beginning to be understood (aka Web 2.0).  But in the end, control is shifting to the edge of the Internet instead of the center and it's not likely to shift direction without extremely potent motivation.

The aftershocks of all this (the shift of control, pervasive ability of anyone to trigger inflection points, etc) have sometimes been called Social Computing, and it will be long in unfolding.  Companies and organizations that continually hand over more non-essential control to their employees, customers, and suppliers will almost certain be the big winners here.  We have plenty of examples to cite already.  The sudden pervasiveness of the two-way, participatory sites and tools powered by network effects and feedback loops have quickly remade the online landscape and Time has decided it is as big an event at least as it famously did in the 1980s by making the personal computer Person of the Year.  I would wager however, the Web 2.0 is probably a more significant event by a good margin that even the PC was.  Although the precise definition of Web 2.0 continues to evolve, the fundamental effect, the harnessing of collective intelligence is the one that has the genuine potential to fundamentally remake our cultures, societies, businesses and even, as Lev Grossman states, "change the way we change the world.

In any case, as usual, like the term or not, the Web is putting you in charge of just about anything that you can imagine.  I recently spoke to a major fashion industry CEO who said he would expect to have product lines that were designed entirely by user contribution and the best resulting submissions selected by their customers to be that year's product line.  The lesson: The consumers have become the producers.  The same with just about any line of business; turning over non-essential control can result in enormous gains in economic efficiency as tens of thousands or even millions of customers creative output is harnessed in a mutually beneficial way.  Organizations that fail to embrace the Web's natural communication-oriented strengths will fail when put in competition that those that do.  Thus, a fascinating chain of events is forming as people around the world begin to realize the true significance of what the Web 2.0 era can truly offer.  What will you do?

What do you think?  Is Web 2.0 evolution or revolution?  Why?

One of the things I'm doing this week is preparing for a presentation at Web Builder 2.0 on how to monetize mashups in Las Vegas next week.  Consequently, I've been pulling together notes, talking to mashup creators, and studying real-world examples of how companies are applying innovative ways of generating revenue with Web 2.0 applications and open APIs.  Though there are all sorts of interesting emerging stories, such as the new Second Life millionaire, product developers are increasingly trying to explore the options beyond the obvious: namely big value acquisitions ala YouTube or the often fickle, if mostly workable, online advertising route.   But the biggest question that comes up is that if you let your users generate most of your content and then expose it all up via an API, how can a profitable business be made from this?

This has been the question from the outset, and though you can build enormously successful sites in terms of numbers of users and amounts of content using Web 2.0 techniques, the best means of monetizing this remain a larger unproven endeavor.  I wrote a while back on the struggle to monetize Web 2.0 where I explored in detail the strategic and tactical methods for making next generation Web sites financially viable, even successful.

If you refer to my original article on monetizing Web 2.0, I identified three tactical means for generating revenue (advertising, subscriptions, and commissions) and a series of strategies that can support them.  While it's usually fairly clear how the direct revenue models work, it's usually less clear to people how the indirect strategies can directly influence the opportunities.

Strategies for Making the Most from Web 2.0 

• There are direct (the 3 items above) and numerous indirect ways to monetize Web 2.0 that often go unappreciated
• Some of the indirect ways which lead to revenue growth, user growth, and increased resistance to competition -- which in turn lead to increased subscriptions, advertising, and commission revenue -- are:
• Strategic Acquisition: Identifying and acquiring Web 2.0 companies on the exponential growth curve before the rest of the market realizes what it's worth (early exploitation of someone else's network effects.)
• Maintaining control of hard to recreate data sources.  This is basically turning walled gardens into fenced gardens:  Let users access everything, but not let them keep it, such as Google providing access to their search index only over the Web.
• Building Attention Trust - By being patently fair with customer data and leveraging user's loyalty, you can get them to share more information about themselves that in turns leads to much better products and services tailored to them.
• Turning Applications into Platforms: One single use of an application is simply a waste of software.  Turn applications into platforms and get 5, 50, or 5,000 additional uses (Amazon has over 50,000 users of its line of business APIs) for example.  Online platforms are actually very easy to monetize but having compelling content or services first is a prerequisite.
  
• Fully Automated Online Customer Self-Service: Let users get what they want, when they want it, without help.  Seems easy but almost all companies have people in the loop to manage the edge-cases.  Unfortunately, edge cases represent the The Long Tail of customer service.  This is hard but in the end provides goods and services with much tighter feedback loops.  And it's also a mandatory prerequisite for cost effectively serving mass micromarkets.  In other words, you can't directly monetize The Long Tail without this.

Lying directly in the primary tenets of Web 2.0 however, are a series of two-edged issues from a revenue perspective.  Though the concepts and ideas are powerful when applied appropriately, they can also pose significant short-term and long-term challenges.  Below are the basic principles of Web 2.0 along with the positive and negative revenue implications for most companies on the Web today, even ones that aren't fully embracing it yet.

 Revenue Implications for Web 2.0 Principles (not meant to be exhaustive)

• Principle 1: Web as Platform
• Upside:  Revenue scalability (1 billion users on the Web), rapid growth potential and reach through exploitation of network effects
• Downside: Competition is only a URL away, often requiring significant investment in differentiation
  
• Principle 2: Software Above a Single Device
• Upside: More opportunities to deliver products and services to users in more situations
  
• Downside: Upfront costs, more infrastructure, more development/testing/support (costs) to deliver products across multiple devices
  
• Principle 3: Data is the Next "Intel Inside"
• Upside: Customer loyalty and even lock-in
• Downside:  Lack of competitive pressure leading to complacency, long-term potential antitrust issues
  
• Principle 4: Lightweight Programming & Business Models
• Upside: Dramatically reduced production costs, easier integration with partners, suppliers, faster product cycles or more features
• Downside: Unexpected scalability demands, unpredictable capacity, potential security issues, lack of control
  
• Principle 5: Rich User Experiences
• Upside: More productive and satisfied users, competitive advantage
  
• Downside: Higher cost of development, potentially lower new user discoverability and adoption
  
• Principle 6: Harnessing Collective Intelligence
• Upside: Much lower costs of production, higher rate of innovation, dramatically larger overall content output
• Downside: Lower level of direct control, governance issues (increased dependence on user base), content management issues, and legal exposure over IP
  
• Principle 7: Leveraging The Long Tail
• Upside: Cost-effectively reach thousands of small, previously unprofitable market segments resulting in overall customer growth
  
• Downside: Upfront investment costs can be very significant, managing costs of customer service long-term

While a great many startups are not generating revenue in huge quantities yet, the companies that have been diligently exploiting open APIs such as Amazon and Salesforce are in fact generating significant revenue and second order effects from opening up their platforms and being careful not to lose control.  This is actually a large discussion, and as large Web 2.0 sites continue to emerge, we'll continue to keep track of what the successful patterns and practices are.

What other implications are there by putting users in control of content generation and opening everything up? 

I was traveling most of last week and so was unable to weigh in on the Web 3.0 mini-tempest that occurred when John Markoff published his exploratory piece in the NY Times last Sunday.  The premise of the article is that we are finding new ways to mine human intelligence which can be exploited by building a new layer of "meaning" on top of the accumulating mass of global collective intelligence that is growing by leaps and bounds every day on the Internet.  Collective intelligence of course is one key aspects of Web 2.0, namely an Internet that is continually improved by constant and sustained contact with hundreds of millions of users contributing content.  These users can either contribute explicitly via a conscious act or implicitly by their very interaction with the Web which then leaves behind useful behavioral "tracks" that can be fed back into the system.  In this ways, hundreds of millions of people are adding to what we know every day, even if individuals contributions are often minor.

Markoff's description of Web 3.0 was ostensibly prompted by something I'm seeing as well, well beyond pure play Web mashups we're beginning to witness a number of companies building end-user solutions that can automatically navigate the Internet, weave together tapestries of online information to generate new, useful results. They can even take it a step beyond: dynamically generated situational Web applications that fully interact with the Web ecosystem.  Such applications -- self-assembled by these tools -- can perform useful tasks such as planning vacations, managing personal schedules, or even orchestrating complex, collaborative business processes for example including entire real-world projects.  The vision is stunning and futuristic yet and the rich fabric of the Web today, with hundreds of open APIs and even vaster reservoirs of content and raw data, now opens the door to the possibility.

Background Reading: Take a look at eight end-user mashup platforms available today 

I've written a lot recently about the trend of user generated software, applications developed by end-users that use the openness of the Web 2.0 era to interact with high value Web services.  But already we're beginning to see the emergence of the next step beyond that: applications developed and tasks completed intelligently by software itself.  Tim-Berners Lee himself envisioned this as the coming Semantic Web which he brilliantly espoused in Scientific American a few years back and has been the goal of great many companies ever since, but which has been relatively unsuccessful on a large scale even up until now.  The reasons for this are complex but seem to lie in what we learned from Web 1.0; a priori solutions often aren't the right ones, emergent ones are .

So while many might say that the 1,200+ mashups currently listed in the trend graphs on Programmable Web are mostly NOT user generated, one only has to look at the widespread use of badges and widgets on MySpace and other major social networking sites to see that everyday people are getting more and more comfortable with "turfing" their blogs and spaces with content, code, and feeds from elsewhere on the Web.  So while much of the end-user mashup activity we see today is probably shallow and don't represent sophisticated functionality, the new tools we're seeing every day are getting better and better and allowing users to take it deeper, creating a true mashup ecosystem.

The shortage of developers and application backlogs: Not finding the app you need

Here's an significant fact, if you look at the number of professional software developers out there today, they are dwarfed by the number of end-users with the time and motivation to describe the solutions that they need.  And interestingly, the same population is dwarfed by the potential output of computer systems that can be directed to create the applications or carry out the tasks we need, with minimal continuous attention on our part.

If you only look at the enterprise IT space you will see that users usually have a long list of things for which they'd like software solutions, but can't get satisfied by the traditional purchase or build processes in place in most organizations.  Every CIO out there is painfully aware of this application backlog but hasn't had the tools to address it.  And out on the Web, there's a different problem: Lots of Web sites, but little software that will do the specific things that a users needs to get accomplished.  As Steve Borch says , "sit back, relax, and let your customers create your products."

Like IBM is realizing with their exploration of end-user driven development products like QEDWiki, most of us today are already conducting much, if not most, of our software integration manually, by re-entering or cutting and pasting data endlessly between our applications.  This implies that 1) there's demand but not enough access to software that does exactly what people want and 2) there is a very low level of integration between the dozens of pieces of software that we currently use on a daily basis.

And in fact, there really is at least two ways for Semantic Web technologies (and its myriad offshoots, many of them proprietary) to improve the way that we use the Internet.  The first is in fact to provide that "layer" of meaning; making the underlying intent services and content to be made clear to programs and not just developers.  And the second is to actively exploit that layer; building software or carrying out processes intelligently on the behalf of users. 

Traditional software isn't adaptable enough: Mashups and Semantic Web Apps are a better way to do things on the fly  

Need a piece of software to manage the process of planning a wedding and its long list of attendees, suppliers, and dependencies?  How about something to coordinate the delivery of construction materials to a job site for the least total cost including materials and shipping, just in time and in the correct order as the items on the construction schedule are completed?  The possibilities in the consumer and business worlds both are truly endless and reflect that such software can at long lat perhaps fill The Long Tail of IT software demand , which could never cost effectively serve the thousands of mass customized applications that would potentially make using software a dream instead of the chore that it often becomes due to the fact that processes and not just data is what needs to be managed.

And while this -- and by "this" I mean recombinant, self-assembling software that exploits collective intelligence -- is certainly the cutting edge of software development, many companies are beginning to map out this terrain closely and I encourage you to begin tracking them along with me.  Startups and initiatives such as JackBe, Teqlo, OpenKapow, Itensil and a great many others are either wholly or partially enabling the automation of software creation and process management. Interesting, they are usually not via true Semantic Web technology, but by virtue of open, simple, easy-to-describe-and-consume services of the Web 2.0 generation .

This brings us to my last point.  In a panel earlier this year with Adam Bosworth and other notably Web lumuniaries, I responded to an audience question about the difference between Web 2.0 and the Semantic Web by saying "Web 2.0 is what happened while we were waiting for the Semantic Web." And that highlights an interesting point, that this latest generation of tools appears to be built on simple yet proprietary approaches and not on the open but formal Semantic Web technologies.  Whether this points to underlying issue with the usability of Semantic Web 1.0 is hard to say but RSS 1.0 ran into the same issue.  Thus I call this next generation of approaches the "Pragmatic Semantic Web." But I am a bit concerned about the lack of standards and this will be something to watch as we see if this next generation of online software is truly ready to sprout wings and fly. 

What other Web 3.0/Pragmatic Semantic Web companies or projects do you know about?

It's the final day of the three day long Web 2.0 Summit , the leading confab for the Web 2.0 era.  It's been a bustling and busy three days in San Francisco with sessions and discussions on a wide variety of Web 2.0 topics, from Advertising 2.0 and Net Neutrality, to the World of Warcraft and Enterprise 2.0.  Given that the Web 2.0 Summit is an executive level conference, the discussion of business models and company strategies around Web 2.0 has dominated the conversation and not the specific techniques and approaches for actually designing and implementing Web 2.0 services and products.  Those subjects have been moved to the upcoming Web 2.0 Expo next April, which will be a much larger event expo-style conference at Moscone Center.

The leadup to the conference was John Musser's great 100-page update of the famous five page Web 2.0 description from Tim O'Reilly (John's comments on the new report here) and the conference also had an exciting Launchpad event to unveil a series of interesting new Web 2.0 sites.  Richard MacManus has the details with links to the sites here on ZDNet.

The two topics that seemed to come up the most often these last three days was 1) how existing major players on the Web can continue on in their leadership roles without significant changes in their business strategies and 2) the need for Web sites and platforms to be as open as possible in order to draw the broadast range of audience and adoption.  In a profiled afternoon conversation on day two, AOL's Jonathan Miller seemed to clearly understand these issues -- which are actively facing his company today -- as it heads into the world of user generated contact and social networks, two forces that are growing large new Web startups, and hence competition, very rapidly.  These new fast growth site models , such as the ones used with YouTube and MySpace, are not however providing clear paths for way for public company to please their investors (net revenue.) Miller also observed that many large companies are not in a position to acquire hot properties like Google did with YouTube.

In another public conversation in the main ballroom right after Jonathan Miller was Microsoft's Ray Ozzie delved into the issues that Microsoft is facing, that started out by focusing on the challenge of how to adapt Microsoft's flagship operating system product, Vista -- as well as their most profitable product, Office -- more effectively to the Web.  The Internet, particularly with Web 2.0 sites, has become the pre-eminent new "superplatform" and it's a significant challenge to Microsoft to stay relevant in a world where the browser is increasingly the center of attention for the software experience.  Ray seemed sanguine about the opportunities however and Vista certainly has many features, such as pervasive built-in syndication, that will certainly pull Vista closer to the Web.

But it was openness that was clearly the most prevalent topic, with discussions on how companies should free their content and services to be used a wider range of situations, particularly from 3rd party entities, even forming the foundation of other products and services offered by entirely different companies.  Openness can also take many forms, from syndicating content to providing well-defined and monetized Web service APIs, and if you don't provide a technical and legal basis for doing so, challenges will only increase as the limited numbers of ways that content and services will reduce the number of overall business opportunities available.  And it puts companies that don't do this at a competitive disadvantage to companies that do open up.  Finally, openness creates the potential for unintended uses, particular as small, more focused content is opened up (smaller chunks are more reusable and general purpose).  It was clear in many discussions, such as with Jonathan Miller, that it's well understood that walled gardens just aren't a viable online business model any longer.

As culled from Web 2.0 Summit discussions and other known best practices...

1. Liberate content and services via a public, open API.  Content will continue to be separated from the experiences that mediate access to it, this makes adaptable experiences possible. Example: RSS readers let users consume content in the ways they choose and have control over.  Doing this turns your Web application into a platform and is one of the most important habits of highly effective Web sites .
2. Syndicate as well as use Web services to open up data. Each method has clear strengths such as discoverability, ease of consumption, or on-demand control.  Example: This means RSS or Atom as well as REST or SOAP.
3. Make it legal to reuse content.  Don't charge if you can help it, consider monetizing it via advertising, transaction fees, or subscriptions.  Don't cripple unintended uses, such as Yahoo!'s limits on their APIs, vs. Amazon's profitable emphasis on unlimited use.
4. Diligently build trust and credibility.  No one will use your open data or services unless there is trust and credibility in the site.  This is very hard to establish and is easily lost.  This is one of the hardest intangibles of openness to manage.
5. Expect the unexpected.  Opening up a site means that others will dream of ways of using your data and services in ways you couldn't imagine.  Often this means they'll use it as a free resource to achieve something that wasn't possible before in terms of scale or volume.  Be prepared for extreme situations and be sure to monitor your feeds and open services and be prepared to throttle them for mailicous or inadvertant waste.

There were plenty of other good sessions at the show and I attended one of the best ones late on the morning of the first day, a great talk from IBM about enterprise mashups, situational software , SOA, and Web 2.0, which are all colliding and combining to make it easier for companies to clear out their application backlog.

Also see great coverage by Stowe Boyd, Richard MacManus , and I hate to say it but ValleyWag, who has relentlessly live blogged the conference.

Announcing Web 2.0 University 

Finally, and pardon the shameless self-promotion, we did have our own big news at the Web 2.0 Summit, namely that O'Reilly Media -- the company that coined the term "Web 2.0" and described the trend to the world -- and my firm, Hinchcliffe & Company, jointly announced on Wednesday that we've formed strategic partnership to join forces on a series of premier services around Web 2.0.  You can read a complete overview of our first major new service, which is already available, called Web 2.0 University. We believe this full series of education and consulting solutions around Web 2.0 will bring intensive, hands-on services around the specific design patterns and business models of Web 2.0.  The premise is that companies are increasingly becoming aware that they need to apply Web 2.0 models to the core of their existing products and services and these services will help them get there quickly and with a minimum of disruption.  So far, early interest has been intriguingly high.

The Web 2.0 Expo will be upon us before we know it.  The deadline for proposals is tomorrow, November 10th, so get them in if you're interested in presenting. See you there!

The next Web 2.0 Conference will be upon us in early November and things are busier than ever in the Web 2.0 world.  Along the way, I've managed to miss the one year anniversary of this blog, which I began back in late September of last year.  There have been over 2.5 million direct hits on this site since inception, a large percentage of it due to my Web 2.0 lists such as last year's Best Web 2.0 Software List , but I also get e-mail frequently from die-hard readers as well.  Most importantly however, from all my conversations with people all over the world, it's clear that Web 2.0 remains more than ever a topic of major popular interest and industry fascination.

While the general understanding of Web 2.0 is improving all the time, we have a ways to go before we have a concise, generally accepted definition.  My favorite is still networked applications that explicitly leverage network effects. But while most of what we ascribe to the Web 2.0 name falls out of these definition, it's fairly hard for most of us to extrapolate meaningful ramifications from this.

People that read this blog know that I'm in the camp of folks that try to look beyond Ajax and the visual site design aspect of Web 2.0, and try to capture the deeper design patterns and business models that seem to be powering the most successful Web sites and online companies today.  Though concepts such as harnessing collective intelligence and Data as the Next Intel Inside, as described by Tim O'Reilly , most directly capture the spirit of the Web 2.0 era, it does seem to me that there are a few other elements that we haven't nailed down yet.

At the AjaxWorld Conference and Expo earlier this month, I gave my usual talk about how to formally leverage Web 2.0, with plenty of examples coming from things happening out on the Web.  If you accept that it's the power and size of the Web today , particularly the number of highly interactive network nodes (who are mostly people), give them extremely low-barrier tools, and we should be able to find plenty examples of emergent behavior; significant events happening suddenly and unexpectedly.  Tipping points are getting easier and easier to reach as site designers learn how to create better network effect triggers, draw large audiences suddenly, and as those same audiences increasingly self-organize spontaneously, such as in the KatrinaList project (suddenly) or Wikipedia (slower but bigger).

And it's the arrival of Web 2.0 "supersites" like YouTube , which appear suddenly, often riding the coattails of other major Web 2.0 site's ecosystems, and apply aggressive, viral network effects that show us the true, full scale of the possibilities.  Building a Web site worth over one billion dollars in 18 months is a very impressive result, but it's really only a single axis upon which Web 2.0 can be applied successfully.  Another axis upon which to apply Web 2.0 focuses less on pulling in every single user possible with a horizontal network effect, but on building a difficult to reproduce but highly valuable data source, such as the Navteq mapping database, or Zillow's real estate database.  One might argue that these are still very horizontal but these are merely just well known examples.

The variety and depth of the Web is such that not every Web 2.0 site will have tens of millions of users, nor should it.  An effective Web 2.0 site is largely powered by its users, whose feedback and contributions, direct and indirect, make the site a living ecosystem that evolves from day to day, a mosaic as rich and varied as a sites users would like it to be.  In other words, creating a high quality architectures of participation is becoming a strategic competitive advantage in many areas.

I'm often asked, particularly after one of my presentations on Web 2.0, to articulate the most important and effective actions a site designer can take to realize the benefits of Web 2.0.  As a result, I've created the list below in a attempt to catpure a good, general purpose overview of what these steps are.  My plan in the near future, is to dive into each one of these as much as time permits and explain how they make highly effective Web 2.0 sites not only effective, but often possible at all.  In the meantime, please take them for what they're worth, I believe however that they are instrumental in making a Web site or application the most successful possible.

The Essentials of Leveraging Web 2.0 

• Ease of Use is the most important feature of any Web site, Web application, or program.
  
• Open up your data as much possible. There is no future in hoarding data, only controlling it.
• Aggressively add feedback loops to everything.  Pull out the loops that don’t seem to matter and emphasize the ones that give results.
• Continuous release cycles.  The bigger the release, the more unwieldy it becomes (more dependencies, more planning, more disruption.)  Organic growth is the most powerful, adaptive, and resilient.
• Make your users part of your software.  They are your most valuable source of content, feedback, and passion.  Start understanding social architecture.  Give up non-essential control.  Or your users will likely go elsewhere.
• Turn your applications into platforms. An application usually has a single predetermined use while a platform is designed to be the foundation of something much bigger.  Instead of getting a single type of use from your software and data, you might get hundreds or even thousands of additional uses.
• Don’t create social communities just to have them. They aren’t a checklist item.  But do empower inspired users to create them.
  

Of course, there a lot of work in the details and these are just some of the important, general essentials.  Unfortunately, a lot of careful thinking, planning, and engineering goes into any effective Web 2.0 site and it's having these ideas at the core of it, which can help you get the best results.

Final Note:  I'll be on the road the next two weeks and will be at the Web 2.0 Conference in San Francisco from Nov. 7th-9th.  I'll be there writing coverage for the Web 2.0 Journal and here as much as possible.  If you're going to be there, please drop me a line if you'd like to meet. 

It's been approximately 18 months since Jesse James Garrett fatefully coined the term that would go on to nearly reinvent the face of Web development.  A lot has happened in the last year and a half, including the Web 2.0 phenomenon getting into high gear, the creation or resurrection of many a company building or using rich Internet technologies, and the proliferation of really great dynamic, online software.  It's clear that Ajax as a name, a concept, and a popular browser development technique is here to stay, and our Web applications will never be the same again.

While most of us know that the Ajax approach was fairly well known before the term ever came about, the timing was apparently just right for the idea of Ajax to capture our imagination and apply such a pithy name to an important new development trend.  And just as powerful browsers, high-speed connections, online software trends, and development tools were reaching the sweet spot that needed to form for Ajax to be popular, so also came the embrace of a world extremely interested in turning their boring, static Web pages into full-blown, sophisticated applications.  Since then, I've heard of or seen literally hundreds of Ajax products, tools, utilities, debated the disruptive potential of Ajax, speculated about how Ajax will be the face of our SOAs , and even watched as RIA technologies in general have risen up that truly complement the few things that Ajax does not do well, such as multimedia.

Along the way, the Web development community has learned a lot about Ajax including its strengths and weaknesses, appropriate uses, and its inevitable foibles.  So to inaugurate the first print edition of a dedicated Ajax print periodical (see below for details), I thought I'd share my perspective on what I think we've learned in our 18 month journey to remake the face of the Web and the browser.  Ajax has indeed helped give us the next major new platform for software, almost certainly forever surpassing our desktop operating systems as place we develop and use most of our software applications, consumer and business both.  As always, this merely represents my opinion...

What Every Software Project Needs to Know About Ajax

1.  The Browser Was Never Meant For Ajax.  About a week into your first Ajax serious application you'll discover that Ajax pushes the browser nearly beyond its limits and there are definite lower engineering tolerances to get used to. The fact is, without powerful 3rd party development tools, designing clean Javascript software of any size requires some genuine discipline and effort.  So too does Ajax debugging applications in multiple browsers (a real headache), and doing any serious background processing or threading can require heroic measures, particularly if you're mixing in other components that use the rather limited number of simultaneous timers available.  The good news: Simple Ajax -- sprinkling in a little DHTML -- is much less daunting than Ajax In The Large.  But be warned and be prepared to scale up your level of development and testing effort significantly with each doubling or trebling of your application size.
2. You Won't Need As Many Web Services As You Think.  I used to think that going the Ajax route required the development of a bunch of new Web services in order to feed the application data and provide a backing store.  In reality, I'm finding a great many projects are quite happy to scrape HTML and/or use plain old HTTP POSTs to existing service endpoints that have no formal Web service structure.  This is further turning the tide towards Ajax by making it very, very easy to "dip your toe" into Ajax development and reuse almost any preexisting HTTP service on the back end instead of SOAP or REST/WOA.  While this can encourage poor architectural choices, it does make very incremental conversion to Ajax almost effortless and turns out to be a natural thing to do, though it can certainly lead to headaches later.
3. Ajax Is More Involved Than Traditional Web Design and Development.  The loss of HTML user interface conventions, the almost limitless potential for hidden or latent functionality, the programmatic creation of page elements instead of declarative, and other intrinsic aspects of the Ajax approach throw out much of what we know about Web design and development.  Web designers must much more deeply understand the capabilities of the DOM, Javascript, CSS, and how the browser renders graphics, layouts, and elements.  Developers find testing both difficult and tedious.  Though tooling is continuing to improve across the board, it will take years for the industry to develop best practices, lore, patterns, and shared knowledge to make Web application development straightforward.  Huge kudos to folks like Yahoo!'s Bill Scott for trying to fix many of these problems -- particularly the loss of GUI standards -- by actually moving the state of the art considerably forward with things like the Yahoo! UI Design Patterns library.  The bottom line: Ajax development, at least for now, usually takes quite a bit longer than traditional Web development and requires a higher level of skill.
4. Ajax Tooling and Components Are Still Emerging and There Is No Clear Leader Today.  Though Dojo is getting one heck of a running start, the race is very far from over.  For instance, the Dojo framework itself is still just at version 0.3.  And close at its heels are an amazing range of tools, frameworks, and component libraries.  Though OpenAjax will make this mosaic of products play nicer, most developers will get deep experience with two or three of them and stick with them. For now, I would say deeply committing to a particular product is usually not the best idea.  Innovation, competition, and market leadership is likely going to bounce around for a while.  In the meantime, be sure to check out script.aculo.us, Prototype, Google Web Toolkit, Yahoo! UI Library, JackBe, Zapatec, Bindows, Nexaweb, General Interface, Backbase, ActiveWidgets, and last but not least Microsoft Atlas. There are many others and I encourage you to look at Max Kiesler's roundup of 50 Ajax frameworks, with many others in the comments (and growing).  Finally, Microsoft's Harry Pierson has diligently taken me to task for my Ajax spectrum comments, noting that Microsoft actually has more serious experience fostering an interoperable component community than just about anyone else.
5. Good Ajax Programmers are Hard to Find.  Zimbra's Scott Dietzen has lamented recently about the real difficulty in finding good Ajax talent.  See point #3, but building sophisticated Ajax applications requires more computer science skills much more than it does Web design skills.  And I find that experienced programmers tend not to enjoy Javascript programming and debugging. This too shall pass, but not for a few years, and not for a good while in the Bay Area. :-)
6. One Must Actively Address Ajax's Constraints of the Browser Model.  Though the final result can be very rewarding, Ajax is not a perfect Web development approach and it has a few genuine weaknesses.  One is that it tends to break the model of the Web including preventing users from bookmarking content, breaking the use of the Back button, and more.  Fortunately, smart folks like Brad Neuberg have addressed much of this, as long as you're willing to put out the effort and understand why it's important to recover this functionality.  Ajax also lacks much of what still makes desktop software a strong contender; the ability to run disconnected from the network and access to local disk storage, though Flash local storage and the upcoming Apollo platform can help address this.
7. Ajax Is Only One Element of a Successful RIA Strategy. As I've written before, the addition of RIA platforms such as Flex, OpenLaszlo, and WPF/E to a RIA strategy is virtually required to properly exploit the range of capabilities you'll want robust online applications to have.  This is particularly true around rich media support such as audio and video -- which Ajax is virtually incapable of -- but even such mundane things as good printing support.  These are all things that the more sophisticated Flash-based RIA platforms really shine at and are easier to program in to boot.  Ajax will increasingly get a serious run for its money from these platforms, particularly as they provide back-end server support for things like server-side push, formal Web services, enterprise environments, and more.

There are certainly other things software projects should know about Ajax but this is plenty of crucial food for thought.  Looking ahead, we see the growing trend of in-browser mashups which is making the habit of combining pulling together -- entirely on the fly -- sets of Ajax components, Javascript snippets, and Flash widgets from all over the Web into a new set of often user-generated ad hoc software .  Backed by the growing Global SOA , online Ajax components such as Google Maps, that can be referenced over the Web by a line of Javascript, and you have a recipe for an increasingly emphasis on assembly and glue instead of "green field" development of RIAs.  This is an important use of the Web that I've called the "mashosphere " for the lack of a better term, which ushers in a whole new era of dependency and configuration management problems.  The rich palette of software components and high value services on the Web will be a irresistable siren call for developers and expect more and more Ajax applications to be mashups in one form or another.

But all of this talk of the evolution of Ajax does bring up some exciting new industry events... 

Announcing The Premier Issue of AjaxWorld Magazine - Print Edition 

Please do pardon the shameless self-promotion here at the end of this piece, but this is also important Ajax community news.  I've been the editor-in-chief of SYS-CON's AjaxWorld Magazine for a while now and to herald the rise of Ajax, we've just expanded it to a full blown print magazine with the premier issue coming out at the all-star AjaxWorld Conference and Expo and Ajax Bootcamp next week in Santa Clara, California.

For the cover story of the premier print issue, I worked with the OpenAjax Alliance -- a big thanks to IBM's Jon Ferraiolo and Joseph Becker -- to get a premium article series on both the strategic and technical direction of this significant and important new development in the Ajax world.  OpenAjax holds the promise of true Ajax component interoperability, consistent tool support, and much more.  I've urged Microsoft to consider joining -- they're one of the major holdouts -- and they've promised to seriously consider it after they get Atlas shipped, so hopefully we'll see nearly 100% industry support soon.  Thus, the story of OpenAjax has been one of the bigger Ajax stories of the year as the number of vendors on board continues to grow in leaps and bounds, never mind the relatively light hand and welcome avoidance of a heavyweight standards approach to Ajax interoperability.

I'll be blogging more about Ajax and less about Web 2.0 here in the next week or so as coverage of AjaxWorld and the many exciting announcements and information begins to flow forth.

Happy Ajaxing and hope to see you next week in California!

It's been a whirlwind week of Web 2.0 events and I've had a chance to meet a lot of new folks in the last few days and discuss Web 2.0 and how it applies to life and business.  The biggest and most significant event was undoubtedly Wednesday's The New New Internet: Web 2.0 for Business which I helped organize and run here in the DC/Virginia metro area.  Held at the Tysons Corner Ritz-Carlton, the event showed clearly the sustained interest there is in Web 2.0 strategies for business and related topics , and I'm pleased say we had a terrific bunch of speakers and a great audience both.

Though Web 2.0 as applied to business definitely interests a smaller group of people than a general Web 2.0 discussion, it was felt that the topic is still just in the beginning stages of growth, has been underrepresented due to the general consumer hype Web 2.0 gets, and was especially suitable given the conference's proximity to the nation's capital, particularly given the number of large companies and government agencies locally which have expressed interest in how Web 2.0 might apply to them.

It's worth giving a run-down of the particulars of the conference and some of the terrific blog and news coverage of the event for those that weren't able to make it.  Also, I'd like to thank leading-edge Web 2.0 technology firms JackBe and Nexaweb for their terrific support of the conference as well as numerous other sponsors which helped make it possible.

A Fast-Paced and In-Depth Day of Web 2.0 and Business

The day opened up with a rousing speech by Secretary of Technology for the State of Virginia, Aneesh Chopra, who gave quite a talk about technology, the state government, and how they are Web 2.0 capabilities on the state's Web sites for things like online support communities for Iraq veterans and improving job prospects for the less affluent in the south of the state.   Aneesh introduced the day's first keynote speaker, Jobster CEO, Jason Goldberg.

Jason provided a pretty compelling video introduction of the television coverage Web 2.0 has been getting on major media including CNN and then launched into an exploration of Web 2.0 and what it means to him.  His layperson explanation of Web 2.0 (Web 1.0 = Get it online and Web 2.0 = Make it work) came in at a good level for an audience that seem squarely mixed between business folks and technical people.  See below for more details about Jason's talk from Web 2.0 local Ken Yarmosh.

Jason was followed up by Rajen Sheth of Google Enterprise who gave quite a lucid and far ranging overview of the way Google looks at Web 2.0 as well as where the online software market is going including open platform-based application hosting, something Google has been getting increasingly involved with as they release more and more business software online that range from Writely to Google Spreadsheets. Best Concept Discussed: Search has become the most effective navigation paradigm.

After this and right before lunch I hosted a panel to discuss where Web 2.0 is taking business.  We had a great cast of Web 2.0 CTOs and CEOs including Chris Heidelberger of Nexaweb , John Crupi of JackBe , David Temkin of Laszlo Systems , and Jeff Crigler of Voxant .  Our hour-long discussion ranged far and wide and included how rich Internet applications are fairly resistent to indexing and search, to the trend of businesses using software as a service and outsourcing over the Web to solve business problems instead of asking IT deparments for help.  We also touched on whether adding social features to business applications is considered too at risk for distracting employees and much more.

To keep the energy level up, the conference didn't break for lunch and had it served while Web 2.0 ubberblogger and up-and-coming "media 2.0 baron" Michael Arrington of TechCrunch spoke to the audience on "What's Next".  Mike gave one of the more humorous presentations of the day that was also packed with informative analysis on the good, the bad, and ugly of Web 2.0 software today (Good: Digg , Bad: Jigsaw ).  He also presented some terrific analysis of how to be successful in the Web 2.0 era (unfortunately, being lucky topped the list).  Best line (paraphrased): "Be warned that I tend to make blanket incendiary statements when someone asks me a question about something I said in my presentation."  See links below for more coverage.

After lunch, Hart Rossman of SAIC gave an in-depth presentation on Enterprise Web 2.0 and its convergence with service-oriented architecture (SOA), a topic that I find important and write about frequently.  Ross especially dived into the details of how mashups and bringing outside Web services into the enterprise are going to have unique and difficult security challenges and he tackled a tough subject head-on and introduced the new SafeSOA initiative to the audience, which aims to solve, among other things, the issues around using Web 2.0 APIs and platforms within the enterprise.

Microsoft's Michael Platt , from their architecture strategy group, came up next and gave an hour-long tour that ranged far across the landscape of the Internet, the history of computing and how things like Web 2.0 (like the PC and minicomputer revolutions) are recurring phenomona.  Mike talks about user generated content, mobile Web 2.0, Microsoft's Live intiative, the Red vs. Blue videos that young people are generated by recording XBox Live games, and much more including the Chevy Apprentice campaign, one of my favorite social computing stories.  Microsoft has the responsibility of successfuly guiding hundreds of millions of consumers and millions of businesses through the increasingly social jungle that is the Web's next generation and Mike gave a great overview.  And given the time, Mike wasn't even able to discuss major MS-related Web 2.0 news like Soapbox , Microsoft's new YouTube play.

After that, Jeremy Geelan hosted a highly-interactive panel on Web 2.0 and monetization that featured TJ Kang, founder & CEO of ThinkFree , Ben Elowitz, CEO of Wetpaint , Robin ‘Roblimo’ Miller, Editor-in-Chief of OSTG , and Sean Frazier of KnowNow.  The panel provided a "front row seat" to leaders in the Web 2.0 space and Robin Miller, inventor of the Slashdot new format, was clearly in fine form on the panel.

The last speaker was Harvard Business School's Andrew McAfee , whom I'd invited as one of the leading thinkers of how to apply Web 2.0 to the enterprise , a vision he currently calls Enterprise 2.0.  Andrew gave a riveting talk on Enterprise 2.0 and freeform, emergent tools like blogs and wikis to truly enable collaboration and capture discoverable enterprise knowledge, and more.  Citing the success of Wikipedia and its early experiments that encouraged them to make it as easy as possible to contribute and edit entries, McAfee went on to discuss the challenges of getting users to adopt technologies more difficult that e-mail, currently the easiest way for most people to collaborate, and unfortuantely also one of the most private and non-scalable collaboration models.  See blog coverage below for more details. (MP3 Recording)

News and Blog Coverage of The New New Internet

Blog and news coverage of the event was very good, here is some of the best write-ups I've been able to find so far including some good video.  Be sure to add others in the comments below.

• Ed Yourdon writes up his impressions of the conference with plenty of details.
• Dennis McDonald did a bunch of coverage, here's one of the best.
• Ken Yarmosh did a series of great on-the-spot blogs from the event including coverage of Jason Goldberg, video of Rajen Sheth, an exclusive video interview with Michael Arrington, and great coverage of Mike's talk.
• Andrew McAfee did a terrific roll-up of the conference on his HBS blog.
• CNET did a round-up of speakers and panels in Web 2.0 entering corporate world slowly.
• Michael Wade has graciously posted recordings of most of the speakers here.

Turnout was great and interest was high so it looks like we'll do this again next year as well.

Are you looking to apply Web 2.0 to your business? Why or why not? 

Blip.tv is a video sharing site. has already made headlines by winning a deal to provide video services to support media companies including CNN. As Marshall Kirkpatrick pointed out in TechCrunch's August 15th Blog EyeSpot has partnered with Blip.tv. The two startups offer services that are highly complimentary.

Strength: Sharing

Veoh is a video sharing site. While it is possible to view video online the real power of Veoh comes from the downloaded client for PCs and Macs. The client creates a peer-to-peer network for sharing video and allows subscribers to view high quality, full screen video. The benefit of this approach is the distribution of video becomes highly scalable. Veoh is definitely attracting interest and investment. They recently raised $12.5 million in a B venture round from investors including former Disney CEO Michael Eisner and Time Warner.

VideoJug appears to be a video sharing site with a mission. To explain life on film. VideoJug is seeding their site with educational videos but is encouraging the Internet community to contribute instructional video. It will be interesting to see if they can differentiate themselves from the likes of YouTube and build up a critical mass of educational content.

Flurl is a video sharing and search site. It allows media to be uploaded anonymously in a variety of formats. There is a 20MB cap on uploads that limits the size of video content. The service is advertising supported. The terms of service prevent linking to media files directly. Links are made to links that serve the content. This enables Flurl to serve advertising to viewers.

YouTube is now one of the top ten destination sites on the Internet serving over 100 million videos each day. It has achieved this position by making it exceedingly easy to upload and share video. YouTube hosts videos converting them from multiple sources to flash movies. Video is limited to 10 minutes in length or 100MB in size.

The site is worth a visit. Register (its a quick process) and enter a couple of your favorite movies. MovieTally demonstrates many of the traits of a Web 2.0 application where network effects make the site more valuable as more people use it. I have reservations that people will really take the time to enter in Movie information, such as actor, director and plot summaries when Amazon is just a click away - but at the outset very few people expected Wikipedia to grow in to the extensive resource that it has become. I think this issue can be addressed by pulling information from Amazon using their web services toolkit.

The developer, Hayden Metsky, has certainly paid attention to building a participative architecture with cross linkages embedded throughout the site. I intend to keep an eye on the evolution of this site in the coming months. There is an opportunity for MovieTally because people are looking for a movie recommendations and ratings site that they can trust - one that reflects their perspectives and those of their friends.