Nicki Usiondek on Why KM and AI Are Better Together

Table of Contents

• Modernizing a SharePoint System for Knowledge Management Needs
• Steps to Ensure Documents Were Up to Date
• AI and KM, Like Peanut Butter and Jelly
• New Job, Same Knowledge Management Requirements
• Organizing Information With AI and Humans in Mind
• Exploring Automating Metadata Updates 
• Career Advice for Knowledge Managers and Would Be Knowledge Managers
• Parting Thoughts

Siobhan Fagan: Hi, everybody, and welcome to Three Dots. Today, I am here talking to Nicki Usiondek. She is the Associate Director of Legal Knowledge and Data Management at Eli Lilly and Company. I'm really happy she's here to talk to me about knowledge management and how that feeds into AI. So Nikki, please, welcome to the show.

Nicki Usiondek: Thank you. I'm excited to be here.

Siobhan: I first found out about the work that you're doing through you winning one of the Step Two design awards a few years ago. Can you talk a little bit about what you won the award for?

Nicki: Yes, it was the gold medal award for Microsoft 365 adoption and we did this primarily through the use of SharePoint. I used to always make a joke that I don't watch award ceremonies. And now that I have an award, maybe I should start watching those awards ceremonies?

Siobhan: Oscars are coming up!

Nicki: Yes! So it was really exciting to have not just internal recognition for the work that myself and my team did, but also external because the digital world changes so quickly so to be able to do something and get recognition for it was really validating for me.

Modernizing a SharePoint System for Knowledge Management Needs

Siobhan: Part of what you did was specifically to Microsoft 365. You had a bootstrap team, am I right? And you taught yourself how to work with these tools to help the work that you were doing then, which was with the legal immigration group, am I right?

Nicki: Yes, Fragomen only do immigration law. They practice in over 126 different countries. What that means is they have to capture 126 different countries regulations related to business immigration. And immigration law changes almost daily, especially in today's rapidly changing world. Being able to capture all of the legal requirements for those countries and have it up to date so when they were advising clients in a timely manner, it was very important. This was what drove the project.

I was really fortunate because when I started at Fragmen, there was already a system in place. So this was strictly about modernizing it. And it was bootstraps because they have a really fantastic and robust IT team. However, because of how rapidly immigration law changes, we needed to make sure that if something was down in our system that we could react immediately and fix it.

Obviously we all live in the world where there will be an AWS outage or something like that — those are out of our control. So we looked at ways we could do this so that IT would still be there for the heavy lifts, but for the light touches and emergencies, my team would be able to react.

Siobhan: You gave a sense of the scale of the work that you're doing and the number of countries that your organization was operating in. When you think about all of the regulations and all of the jurisdictions and all of the case types, how do you go about designing an information architecture that's going to accommodate that level of complexity without overwhelming your end users?

Nicki: I think this is where the strength and beauty of SharePoint comes to play. SharePoint is oddly very controversial. People either love it or hate it. I've never really met someone who's indifferent. I've only met people who don't use it either because they never had the exposure or they hate it.

So SharePoint has the beauty of having the front end, or what the end user is using and experiencing, which can look and feel very different from the backend where the SharePoint document libraries are, the Microsoft lists set. Having a way to display data so that it's not overwhelming to end users and isn't like — no offense to Microsoft lists — but they can be ugly and overwhelming when you see the number of columns and roles in it. So being able to pull out information and display it in a way that's meaningful to the end users is a really good strength of SharePoint.

Siobhan: I love that you're using a term that you so rarely hear about SharePoint, which is beauty. I've got a little SharePoint fan club going on here.

Steps to Ensure Documents Were Up to Date

Siobhan: One thing we've heard about SharePoint is the confusion around knowing you have the most current document, which in legal services is critical. What did you have to put in place to ensure that happened and older regulations got pushed down?

Nicki: This is a three-part answer.

The first was the knowledge management team at Fragomen have really strict and really good governance around the information they capture. So they have a 48 hour response turnaround from when there's a legal change to where it's updated in our system. It rarely takes 48 hours. It's usually updated within an eight-hour period only because if it happens overnight.

But also they're global. We have regional managers around the globe to support the different regions as well. So being able to action things quickly, review things quickly, update them quickly. A lot of the rules were captured in the Microsoft lists. And so with those, the information would come in from either an attorney or an employee of Fragomen in one of the regions, or sometimes we had partnerships with third parties. They would let us know about the change.

Having those relationships with the different countries, they don't just say Surprise! They notify people, Hey, this change is coming. So you get everything ready, you make sure it's good to go when the changes happen. So that's one part of it.

The second part of it is there were also products that were created based off of that intelligence that was gathered. So what would happen there is we would update the Microsoft list or the subject matter experts would. And then that would prompt the documents to be updated. Over the course of years we started using Microsoft Power Automate. So when this cell is changed in this list, it will auto populate over here into the documents. And then we would have people that would go make sure the formatting was good, correct, no mistakes, and then PDF it. That would replace what was in the SharePoint document library and then it's available.

The third part, which I think is so important especially with AI, is we had a content curation team, or we started calling it knowledge curation to really differentiate it from content curation in the marketing aspect. That was the team that I led. We would have regular reviews. Is this still up to date? Is this still active? It it's not active anymore, let's move it over to the archive. It was constant checks to make sure that we have the most up to date, have ways to automate when possible and then just doing the final curation.

AI and KM, Like Peanut Butter and Jelly

Siobhan: You had an unexpected realization after you finished this project around AI and how the work that you were doing with this project was better setting up your properties for the use of AI. Can you talk about how you came to that realization and what the results of that have been?

Nicki: We used PnP modern search for the end user experience. And this was where, they have the web parts that sit on a SharePoint page, you can create really targeted queries. For these queries to function, you have to have managed property columns and then associate it with a refinable string in the SharePoint admin center. That's what makes it queryable and findable.

In the course of creating these managed property columns, you're basically creating really strong taxonomy/ontology because now these have relationships to each other and we have a map to it.

I cannot take credit for the information architecture, how it was set up. It was there in place when I joined Fragomen, we were just simply modernizing it. It had always been built in what was at the time called SharePoint lists. I think they're calling them Microsoft lists, unless it's changed recently and I didn't catch that. It was also how it was displayed on the pages.

We had previous consultants that had created the code. And then as they would leave or we would get new consultants, it became what we called spaghetti code. So something would break and then it would impact five different things. We would have to scramble because, you pull one thing, it's not a straight line.

We quickly realized we need to refresh this and retire things that are no longer important to Fragomen or that have just changed, and then refresh it in a way. It was originally one list, but we were nearing 30,000 items. SharePoint has a 5,000 list item threshold, and then it starts to — I'm just going to use the very technical term — get wonky.

We decided to start gathering more data points. As we take on more countries, as new regulations take place, we need to have a little bit more flexibility. So we split that into four lists by region. That way the list sizes would get smaller and SharePoint wouldn't be so angry. That's the beauty of PnP modern search is you're able to target it: look at this list or look at this URL for information, and here's the specific, managed properties.

You would technically point it at the refinable strings to create a full picture. So as we were building this, we finish and it's amazing. It's been well received.

AI starts to become really important and Fragomen starts looking at ways to utilize AI and they started asking, Hey, can you describe these lists to our IT team? Can you describe these lists to us? Can you help us understand what is what and what does this mean? We were able to say, We actually have mapped this out, using it for queries and search queries, which is really fantastic for training AI on what things mean.

And so we were able to hand this off and then IT was able to run with it. And that's when it really hit me that AI and KM, they are basically like peanut butter and jelly, they go together.

New Job, Same Knowledge Management Requirements

Siobhan: So you have since moved on to Eli Lilly. Have you noticed a difference in the knowledge management needs from Fragomen, which was a legal immigration firm, to now pharmaceutical? I imagine they're comparable in that they're both highly regulated, they both have a high demand for current, accurate, information. Can you talk a little bit about that?

Nicki: They are very similar in regards to the amount of regulations that surround both of them and the importance of adherence. I think the commonality is really the importance of how people find information. When people can't find information, it becomes a bottleneck or slows them to their end goal. If I have to spend 30 minutes looking for something and still can't find it, or I find something, but it's from five years ago, I'm frustrated. Now I'm trying to find the person who created it five years ago. Is it still up to date?

It just becomes a really frustrating experience. So that's the commonality. Having a strong knowledge management program, having really good information architecture, having a really good knowledge curation team alleviates that frustration and stress.

Now when people are looking for something, they can find that information, they can make the decision that they need to or complete whatever they were working on and they don't have to get frustrated. They're able to move on to the next task. I think we've all had that job where everything is difficult. So it helps alleviate some of that. Obviously, knowledge management aren't miracle workers, but I think most knowledge managers do care about helping people find what they're looking for.

Siobhan: The frustrations of not being able to find anything and having to turn to this person and that person — if you even know who the person is who might know where that information is — is common across most organizations, just the stakes maybe are a bit higher in the case of a pharmaceutical or legal industry. So it's interesting hearing you talk about that.

Organizing Information With AI and Humans in Mind

Siobhan: Has your knowledge management practice changed at all with what you learned with your award-winning project? When you're designing the knowledge management work at Eli Lilly, are you thinking in terms of AI readability as well as people?

Nicki: I think that's so important. I really feel it just was a logical transition, it made sense after that. I don't know if it is because I have a library science background, so having things organized is something I enjoy. Sometimes it can be a little bit overwhelming when you have so much information. Before it was always, How will people find this? How will people use it? What will the user experience be?

It shifts a little bit and now becomes, Will AI understand this, because as we know with large language models, it's predictive. These aren't geniuses with years and years of education and experience. It's predictive. They're predicting the text. So you want to set it up in a way to where AI is able to understand it. And that's obviously not the right word, but can make the connection.

And then you always have to have the human in the loop. The end user gets the information and uses human experience and human understanding to interpret it. I don't want to understress the importance of humans and the use of AI. It's incredibly important.

Siobhan: When you think of your audience, are you thinking of AI first? And I'm not disregarding the human, but are you thinking of AI first because AI is what's serving the human or are you thinking human first?

Nicki: I'm actually thinking them equal. It's two audiences because no one's fully adopted to AI yet. It's just not there and we're also learning how to leverage it, use it and fine tune it. So I like to approach it from two directions. So I'm the end user. What do I want to see as the end user?

And then also from the AI perspective. AI isn't just like creating text. It can and that helps, but it's also helping you with strategy. It's helping you find patterns. It's helping you be able to sort through large volumes of information quicker. So what will help the AI do that? So it's coming at it from two directions.

Exploring Automating Metadata Updates 

Siobhan: Are you using AI to inform your practice? And if so, how?

Nicki: Right now I'm using it a lot for strategy and bouncing ideas. I tend to have a million thoughts going on in my head at once. Sometimes I have a hard time getting it organized or finding a way to structure it. So I will brain dump a lot of things into, use Claude or whatever, and then see what it spits back. Then I ask Is that what I really am trying to do? No, now I see the spot. Sometimes it's not even about, AI gave me the answer. It's AI showed me where my gap was because my answer isn't in what I'm seeing.

I do use it for a lot for strategy. Some of my goals for this year is to learn how to start fine-tuning more with the large language models in relationship to knowledge management. Also, I want to start finding way so with Claude skills now, this brings the opportunity to automating things that were a little bit more tedious. So I'm looking at ways to broaden my skillset.

Siobhan: When you think of things that are potentially tedious in knowledge management work, what areas are you thinking might be ripe for AI?

Nicki: Updating metadata or creating it. So sometimes, especially in SharePoint, you update a document. Now we have the new updated document, but you have to go through and make sure all of the columns have the correct information. There's obviously the date modified, but that might just mean that someone removed a comma and used a semicolon — it's not true change.

Usually we'll have a column that says last reviewed date. So it's making sure those things are getting updated. So finding ways to make the connection where, and SharePoint is doing this with the knowledge agents and things like that, where it's able to scan the document and know, when was the document last updated and pulling that and populating it. It's little things like that, which are tedious. They can be time-consuming, but a human doesn't really need to do that. You still have to review it, but the actual work could be done with automation and AI.

Siobhan: Do you have any concerns around people getting complacent with AI and not checking the work as closely because you're trusting it and some things float through? What sort of stopgaps will you put in place for that?

Nicki: The bigger issue we're having right now is getting people comfortable trying it or using it. I think a lot of people start using AI for things like, Where should I go to dinner type of thing. That's helpful, but it's not using AI to its full potential. First of all, it's not always going to give you the most information. I laugh because I've also used AI as Google and found things that I wouldn't have found in my Google searches, because SEO changes everything.

I know a lot of people will say AI is not going to take your job, people who know how to use AI will take your job. So the complacency thing, the people who are not going to review it, who are going to be complacent, are the people who are already complacent with things and not necessarily double-checking their work. What you're going to find is the people who tend to be a little bit more in-depth and reviewing their team member's work to double check to make sure everything aligns with what we had discussed prior and then pass it on. Those people I think are going to have no problem.

The people who just did their part only and then passed it along, they're going to be the ones that run into issues. I don't know what those issues will be per se, but I think it'll just be areas of opportunity for them.

Career Advice for Knowledge Managers and Would Be Knowledge Managers

Siobhan: You said earlier that knowledge management and AI go together like peanut butter and jelly. There is in almost all fields a certain amount of panic about the future of work, future of jobs. What would you say to knowledge managers today who might feel threatened by AI?

Nicki: Times change. Technology changes. So it's AI the last few years. Who knows what it's going to be in 10 years. There's always change.  

If you are complacent, you're going to get left behind, whether it's AI or something else. Continue to learn. Always learn. I'm a huge fan of design thinking and iteration. Don't be afraid to fail. And don't be afraid to play around and don't be afraid to be like, Here's my first shot and get feedback and then continue to make changes. Curiosity, to be curious, to be willing to make mistakes and keep learning. Follow people on LinkedIn, watch YouTubes or read blogs. Tinker, try a bunch of different LLMs. They all have different strengths and weaknesses. I discovered last week that Copilot's actually gotten really good at making images, whereas Claude just couldn't do it for me. It's interesting to see the strengths of the different AI and LLMs that are out there. So find different tools, get comfortable with different tools and then, you know, play.

Siobhan: One final question before I turn it over to you: you have a master's in library and information science, you're in a knowledge management, clearly knowledge management with its interest in organizing and structuring information aligns well with AI. So if anyone is graduating from an MLIS program and they're interested in moving into knowledge management, what would you say to them?

Nicki: I love knowledge management. I love my career. And it's weird because I decided to get my master's in library and information science specifically because I was working for a company that was going through the digitization process. They were trying to eliminate their paper files. They had this big push.  

It was a legal department at a corporation where everyone was going to spend a couple hours every day scanning documents. We would take these huge patent files and scan them as one giant PDF — no OCR, no bookmarks. And then the attorneys would go to access that file — and patent files are like 500 plus pages and are organized in very specific ways to make it easy to flip through the file and find what you're looking for. Immediately they were like, Hey guys, you need to rescan all of those and chunk it out by documents.

Scanners at this time didn't have where you could name them at the machine. You would scan 10 things, go to your computer, rename them, upload them. As I spent my summer doing that, I kept thinking there has to be a better way to do this. And that's what led me to library science. Because I realized librarians already do this, but in the non-digital way. And as I started researching that more, realized that libraries had already anticipated the digital change. They had already started making adjustments to their program, which is why I did the masters of library and information science.

The programs prepare you for things like metadata, taxonomy and the importance of digital information architecture. My suggestion would be not to get set on, I am going to be an academic librarian because that doesn't happen right away unless you're really lucky. It does happen, but it's not a normal experience. So get creative with where you want your career to go. A lot of sites out there that will show you, know, non-traditional library fields. And especially now as we become more more digital and AI.

The skill sets that you learn in library school are very transferable. So get creative. For a long time, I would work at public libraries. I would call that my fun job, which is not fair. I love knowledge management. But it's part of being in the community and it's what a lot of people think of when they hear library science. It also helps me understand how people look for things and the importance of the reference question, which is super important in KM.

So you have all the skillsets, you just have to know how to translate it and be willing to try it. If you did the traditional masters of library, try to play more with the digital tools. But I think a lot of library schools understand that non-traditional library positions are more common than a traditional library role.

I always think that it's also like a mini-MBA because you don't just learn how to do cataloging. You also learn about budgeting. You learn about managing people, staff, public relations. It is the complete picture. And you also walk away with really strong digital skills.

Parting Thoughts

Siobhan: Nikki, I really enjoyed this conversation. Is there anything that we didn't touch on that you wanted to raise? Any final words of advice or anything that I didn't ask about that you think would be helpful?

Nicki: The big thing that we've all seen since like 2023 is how everyone in the KM world knows that AI is now part of their day to day. I think that's really interesting. I spent a lot of time going to different conferences last year and some people still fear AI for taking their jobs. Other people hate AI and have this real distrust towards it. I think we need to start looking at AI as not our enemy, not our savior, but as a new colleague who will have strengths and weaknesses and we need to learn how to work with it.

Siobhan: That's a great place to end. So thank you so much, Nikki, and congratulations again on that win, I know it's late, but I enjoyed the story about your Step Two Award.

Nicki: Thank you.