Pan-Enterprise Search

Gary Szukalski of Autonomy presented on "Pan-Enterprise Search Platforms: Moving Beyond Single-Application Search" at the Gartner Portal and Collaboration Conference.

Context is critical in adding value to search. 

Search is a critical platform in any information-dependent enterprise. E-Discovery and Legal Hold capabilities are increasingly coming in to play. Search is converging with these legally driven applications.

People are a valid and valuable data type in an organization.  Sentiment and reputation analysis can also be captured through search. Visualization is one of the next developments in search. Deducing trends from visual association of results data.

The demands for search capabilities are including:

• Real-time content, including IM and phone/VOIP content
• Content analytics especially from customer service organizations
• Retrieval for business intelligence including connections that live within email correspondence. Search adds complex filtering capabilities to business intelligence.

Enterprise Search is not passive but must be active with updates such as RSS feeds being created when relevant data changes.

What customers are not asking for:

• Complex classification
• Normalization pipelines at ingestion time
• Complex query languages being exposed to users
• Pagination - users don't wait anymore. They will just retry a search with different parameters. 

The Power of Collective Decision Making

I nearly entitled this "the Powr of Collective Decision Making" because it is what we call Web 2.0 that has  empowered collective activity. 

Steven Johnson outlined 3 models:

1. The Web 2.0 Group
2. The Solo Group
3. The Geo Group

Quality filters - Signal v Noise. We started with Yahoo and ended up with Google. Structured organization - the directory - gave way to the self organizing structure of the automated search index. This is a big lesson to learn in any large enterprise. Enterprise search is a critical capability. We don't need to create rigidity if we can discover what we need. The trick is to use the results of search to optimize and organize. This reinforces emergent structure.

Examples of Web 2.0 collaborative models: Digg, Meetup. Wikipedia, Technorati,Outside.in

Yes, the Long tail got a mention. Wikipedia was cited as an example of how groups self organize to produce something and reach agreement in the process.

Where do we hit the limits of Groupthink?

Design is one area.  Apple is a prime example where Steve Jobs and design Guru Johnny Ives control the development of just about every Apple product and every aspect of those products. Design by Committee hasn't worked. This is true of many great Internet sites. At their outset they were typically created by a small team with a clear vision. The secret at Apple is to avoid feature bloat and this is the biggest risk with design by committee.

The Solo Group

Steven Johnson pointed to DEVONThink, a Mac application that builds a web of connections from information that you capture. It has been described as a second brain that makes connections for you. 

"Exploring DEVONThink is a collaboration between, authors, the search algorithm and my 

memory"

The Geo Group

Google and Yahoo's APIs have triggered a renaissance in cartography as  an army of amateurs have enriched mapping data with very focused data. Crowdsourcing is another application where celebrity appearances can be tracked.

 

"The Internet has become the filter and interface for the real world"

"The web is not organized geographically."

What the Internet brings is that we have a standardized Virtual Location - the URL and blog software has provided a standardized time stamp. What we still can't do easily is map Internet information geographically.

This is an "ahah!" moment. This is why Google is interested in the Mobile world because GPS on your phone gives geographic relevance to information that we currently don't have. Is Google going to geo stamp our search requests when they come from our mobile device? The next big step on the Internet is to standardize a machine-readable geo locational stamping.

Pulling it all together the example is dial311. A two-way service for non-emergency information. Every call to 311 gets mapped to a location and to a category of request or complaint. The fundamental issue is that 311 leverages the wisdom and attention of the community. The wisdom of the crowd enables collective issues to be identified and addressed.

Microsoft - backed in to a corner?

This morning Microsoft announced their bid to buy Yahoo for $44.6B. Yahoo has appeared rudderless in the past year, lacking in a clear strategy, profits deteriorating and the share price sagging. On the flip side, Microsoft has been making deals to promote its advertising network and the myriad Live branded sites are still fighting for real traction. This is really two disparate cultures coming together to battle against a bigger rival, Google. If Microsoft did not buy Yahoo there is a possibility that Yahoo might have eventually done a deal with Google to leverage Google's ad network. That would have been the worst possible news for Microsoft. In that light it seems that this deal is essential in order to keep Microsoft in the Advertising game. Expect to see Yahoo and Microsoft's Ad networks merge. This has been rumored for so long it had to finally happen. Fred Wilson makes some interesting observations on his VC blog. Will someone like News Corp come in with a strategic bid? With Yahoo shares trading so cheaply this is Microsoft's only real opportunity to boost their position in search by leveraging the strength of Yahoo. Search is critical because that drives ad revenue. As we are seeing Social Networks are not taking off as sources of Ad revenue, at least from Google's perspective. So that leaves Search as the best monetization bet in town. The deal also helps Microsoft in boosting Vista and Windows Server. Think about - expect to see Yahoo sites running on Windows platforms. That will help Microsoft in the ongoing battle to eat in to Apache's market share. All in all Yahoo's weakness has effectively backed Microsoft in to a corner where they have no choice but to buy Yahoo before some other player comes in and snaps them up or Yahoo starts making strategic deals that would be detrimental to Microsoft. Update: I just read Stowe Boyd's assessment and have to agree. Two tired swimmers colliding. It about sums the situation up.

Reflections on Enterprise 2.0

Unfortunately I only had opportunity to attend the first day of the Enterprise 2.0 conference . Despite this it was a valuable trip. Reflecting on the conference these things stuck out for me:

• One thing the organizers got right was plenty of power outlets in the meeting rooms.
• The wireless worked better than at the Web 2.0 Expo, but then there were far fewer people. Probably 1,000 as against 10,000 at the Expo.
• Microsoft doesn't get it, or is biased by its revenue model. How c an you talk about unified communications across multiple devices and then talk seriously about sharepoint and integrated Office Business Applications that use the Office system - that ties me to my desktop
• Institutional IT doesn't get Web 2.0 yet. This was evidenced by the discussion on Mobile platforms and the desire of IT organizations to lock down mobile devices. Developing revenue growth powerin g mobile applications will spread beyond an enterprise boundary. IT will not be able to lock down, or even prescribe all the devices that will connect to data feeds, applications and services. Developing to open standards and interfaces is essential.
• Enterprise Mashups. I was disappointed to miss this session , which is being held on Wednesday June 21st. My thoughts on this area are that we must first get the basics right. Enterprise search (that really works!) is a critical first step. Add syndication capabilities via RSS and you have a solid foundation on which the organization can build emergent processes with minimal IT investment. When you combine Enterprise search with RSS and a Wiki publishing capability you have the essential components to create new knowledge and to be able to springboard new insights from that knowledge.