Feature
Every enterprise search professional dreads the day when they'll hear, "why can't it be more like Google?" The complaint comes from people who don't understand why the search they experience in their personal lives doesn't carry over to their workplace search.
But as Google search itself undergoes dramatic transformation following generative AI's introduction, enterprise search too is being reimagined for the GenAI era.
This isn't the first time AI has promised to solve some of enterprise search's gnarliest problems. The question is if this time, it can deliver.
What Ails Enterprise Search
Search's challenges in the workplace are a bit more complex than public search. Workers' searches across the enterprise are looking for answers in documents and other formats that may, or may not, have been tagged correctly or saved in the right location. And that’s only the start.
According to Statista, the total volume of enterprise data globally was expected to go from approximately one petabyte to 2.02 petabytes between 2020 to 2022. This represents a 42.2% average annual growth over those two years. While it is impossible to estimate the amount of data that exists in any given enterprise, it's reasonable to surmise there's been a corresponding rise in the amount of content they have in their siloes too.
Getting enterprise search right isn't a nice to have. In a competitive environment that increasingly depends on effectiveness and speed to decision making, could generative AI be the solution to enterprise search problems?
GenAI Can Help, But Isn't a Cure-All
Generative AI will likely help, but not solve everything, OpenSource Connections managing partner Charlie Hull told Reworked. He noted as an example the problem of poor data quality that plagues many enterprises — old documents, bad metadata (titles like "Contract with AnotherCo version 2.1 final edited by Dave committed signed up by Tom copy A"), duplicates and other document-related problems. Needless to say with the growing importance of enterprise video, the problem is getting worse.
"Just piling this into a LLM or RAG [Retrieval-Augmented Generation] system isn't going to help, and sometimes the methods used for generating vector embeddings like chunking actually make things worse," he said. He does note as an aside that vector search is a better way to use semantics, but a lot of vector databases have a primitive approach to processing content — traditional search engines usually do this a lot better.
The bottom line is content quality is still important, Hull said. However, he does see generative AI helping with a number of issues, namely:
- Classifying queries: This goes along the lines of, “This looks like a search for a contract, let's add some metadata to filter by contract.”
- Auto-judging results: It's hard to scale human judgments, but this kind of data is essential for measuring and improving search quality.
- Suggesting metadata: Akin to the classifying query above, this takes the shape of, "this looks like a contract, let's add a field saying it is one as the user hasn't specified this, or put it in the wrong folder."
- Expanding queries: Autogeneration of synonyms, helping you match the language of a search query to the language in your enterprise content.
But the hallucination problem is huge, he cautioned. If a document simply doesn't exist, or the answer to a question can't be created with any confidence, GenAI should not be allowed make it up. Retrieval augmented generation is one potential solution here, but may be superseded by improvements to LLMs in the future.
A language model could be trained specifically on an enterprise's content, however the time and expense of doing so should not be ignored, Hull said.
One final consideration Hull raises is measurement. Without a way to measure how good search results are (be it a search result list or a single GenAI-derived answer to a question), you can't evaluate if you're making enterprise search better or worse.
"We've been saying this for a long time about search," he said. "Gather some test queries based on real behavior, measure the quality of results using human- or click-derived judgement data, experiment with new search configurations, validate against your judgement data, rinse and repeat."
For GenAI you can do the same (the R in RAG is just a search) with the added wrinkle you need to measure the generated answer, the G. There are ways to evaluate this, often using GenAI itself.
“Overall, GenAI is no magic bullet, but applied in the right places it has potential to improve enterprise search,” Hull concluded.
Related Article: When Personalized Enterprise Search Results Are Hidden in a Black Box
GenAI's Potential to Increase Relevancy
Generative AI can also help enterprise search by understanding document content rather than relying on exact terms or metadata, Nucleus senior analyst Charles Brennan told Reworked.
Natural language processing can infer meaning and context, even in cases of missing metadata or inconsistent naming conventions. It can also analyze documents for signs of outdated information and compare versions to prioritize the most current content.
More to the point, by recognizing user intent and learning from past behavior, GenAI delivers more relevant results, Brennan said. However, its success depends on quality data and proper integration — without these, it might struggle to provide optimal results.
A final consideration worth noting, he added, is that while AI-powered enterprise search can be a significant improvement for enterprise search, its success depends on proper implementation. This includes integrating it with existing systems, continuously training it with organization-specific data, and ensuring strong governance to maintain accuracy, relevance and security.
“As the technology evolves, its ability to further enhance search capabilities will grow, offering even more refined solutions to complex search challenges,” Brennan said.
Related Article: Has Microsoft 365 Been Clinically Tested?
Conference
Sep
16
HR Tech Conference Las Vegas 2025
Register
Conference
Oct
27
Gartner HR Symposium/Xpo Orlando 2025
Register
When GenAI Meets Access Controls
Chamomile AI founder and CEO Tirath Ramdas said his company currently works with organizations that are primarily focussed on RAG applications, which in many cases are used to address enterprise search challenges.
LLMs have proven to be surprisingly effective at applying structure on top of unstructured, messy data, he said. Ironically, challenges tend to arise in cases where there is some structure associated with the data, with one particular pain point has been access controls.
However, generative AI-driven search is not ready yet, Ramdas said, as bolting on DLP-like access control mechanisms onto user interfaces has proven problematic.
Research presented at this year’s Black Hat conference demonstrated how effective even simple prompt engineering is at subverting access controls implemented within leading enterprise copilot solutions.
“Problems like this are an absolute blocker for enterprise environments, so it would appear at this stage that genAI-based enterprise search is not quite ready for enterprise-wide deployment,” Ramdas concluded.