Why Is Enterprise Search So Difficult?
“I just want search to work like it does on Amazon and Google.” I can’t tell you how many times I’ve heard that lament from friends, clients and other search folks. Frustration and dissatisfaction are common emotions when it comes to enterprise search — that is, search within the firewall.
Google on the web makes search look easy: you type in a word or two, and you get a list of dozens, if not hundreds of relevant pages. We’d all like search like that for our web and internal repositories too.
But remember that at one point, Google offered an enterprise solution in a box: the Google Search Appliance (GSA). It was a large yellow Google-branded Dell server that would crawl and index internal content, respect security and deliver pretty good results quickly. And the Google logo was available on every page to remind users they were using Google search.
The GSA was marketed to partners and corporations from 2004 through early 2019, when it was removed from the market. The GSA showed decent results, but they never lived up user expectations. What went wrong?
Several IT managers have told me users had anticipated the quality of results to be “just like Google” — but the GSA just didn’t live up to their expectations. One search manager told me that simply adding the GSA logo to their existing non-Google search platform reduced user complaints by 40%.
I’m not proposing that you find a ‘Powered by Google’ graphic and simply add it to your search form. First, that’s misleading; and probably a violation of Google’s intellectual property. And secondly, your users will react to the quality of the results, not the search page logo.
One school of thought was that Google simply decided to focus on their primary business, delivering high quality on the web. In fact, the GSA just didn’t have access to the magic that makes its web search so good: Metadata.
It turns out that internal enterprise search is hard.
Upgrade Your User Search Experience
Partly because of its size and popularity, Google on the web takes advantage of the context available to it. That means the results you see may include queries used and pages that you have viewed in the past. But what really adds value is that Google will also include post-query behavior of other Google users who performed the same query.
The good news is you can likely improve your internal search results by implementing the same approach Google uses on the public web.
Your internal content brings some challenges of its own. On the web, there are sometimes thousands of pages that are nearly identical: if Google web shows you any one of those near duplicates, you’ll probably be satisfied. But behind the firewall, people are typically looking for a single page; and if search can’t find it, users complain.
Internal search comes with its own challenges; but it also has metadata that can be used to improve results.
Almost all of the internal content we’ve seen with clients is secure. While parts of some repositories — think HR — are available across the organization, HR does have secure content such as payroll data, employee reviews, etc. that must not be available to all.
The Solution: Use the Context!
One of the differences between internet and intranet content is security. And repositories fall into one of two general areas: users and content. Security should come into play for both types of content.
User Level Security
In a lot of enterprise environments, many, if not most, repositories apply user or content level security. And typically there are a number of elements here. The fields can be used to add useful metadata. Fields that are available and make sense to be included as user-level metadata may include the following:
Location: Office, Department, Time Zone
Office location, department and time zone
Direct phone & email
List of active clientsUser Level Security
Location, role, title, office, department
Role & title
Manager name & contact info
Key accountsContent Level Security
Content including queries, viewed results pages, and saved and/or rejected-ignored
Actually, this is really a starting data point; examine, experiment, and dive in!
About the Author
Miles Kehoe is founder of New Idea Engineering, a vendor neutral consultancy focused on enterprise search, analytics and big data. In addition to New Idea Engineering, he has also worked at search vendors Verity, Fulcrum Technologies, and most recently at Lucidworks.