Ben Zweig on a New Approach to Job Architecture

Table of Contents

• Providing a Foundation for Labor Allocation
• A Taxonomy to Divide Human Activities From Automation 
• Who and How Do You Create a Job Taxonomy People Agree On?
• What's the Taxonomy of a Good Manager?
• The Limitations of Using Online Data Only

Siobhan Fagan: Hi everybody, and welcome to this first episode of season three of Three Dots. Today I am happy to have as my guest Ben Zweig. Ben Zweig is the founder and CEO of Revelio Labs. He is an adjunct professor at NYU Stern on the Future of Work.

He's an economist by training and a data scientist by profession. But most importantly, he’s joining us today because he is the author of “Job Architecture: Building a Language for Workforce Intelligence,” which just came out yesterday. So congratulations, Ben.

Ben Zweig: Thank you. Thanks for having me.

Siobhan: So you have a really ambitious plan in this book to create a universal standard taxonomy of jobs. And I just want to know if you can share with the audience a little bit about what prompted this work. I mean, most people don't sit there at night and think, Hey, I really want everyone to get into the taxonomy game.

Ben: It's funny. I think that people who go through analytics projects ultimately come around to the idea that this really lives and dies by the strength of the taxonomy that is being used in the analysis. So I didn’t really think of myself as a taxonomy person, but necessity is the mother of invention. So there you go.

Providing a Foundation for Labor Allocation

Siobhan: You compare the current model of labor allocation — the stuff that we're using in our businesses every day for hiring or moving people around in the workplace — to a house of cards. Why is this?

Ben: Let’s say we think about the economy as being made up of two factors of production. In economics 101, this is one of the first things you learn: there are two factors of production to produce all the goods and services we have in the world. There's labor and there's capital. And capital markets are fairly scientific. There is a whole financial sector which exists for the purpose of providing a science and a rigorous foundation for allocating capital.

There is no such thing when we are allocating labor. So already there's a lot of sloppiness that happens in labor markets, and really from both sides of those markets. You have job seekers who don't have a lot of visibility into what the occupational landscape looks like and how they would fare in certain jobs: what they lead to, what the dimensions of work look like, whether it has good work-life balance or pay or whatever. There's not a lot of visibility there. And firms don't have a lot of visibility. They're mostly hiring people reactively and don't have a good sense of even what people do who work at their company.

And the reason why that's more precarious now is because work is transforming very rapidly. There's new technologies that may automate some components of jobs. We are seeing it all the time. And the nature of labor markets is evolving very rapidly. So I think there is some urgency to kind of get organized and get standardized.

It's also worth noting that the method for measurement in general in companies is very centered around capital, because standards for measurement were solidified in the age of railroads, when the big determinants of company value were capital expenditures. Now the most successful companies are successful because of their people and their code base, much more intangible things. So I think we are getting further and further away from having anything measurable to manage.

A Taxonomy to Divide Human Activities From Automation 

Siobhan: I want to touch on the urgency that you just raised. I was planning on bringing it up later, but here we are — AI. In the book you outline benefits very specifically for employers. You also outline benefits of this approach for investors specifically. You touch on some of the benefits for people, especially when they're looking for job growth and career transformation.

But when we look at this in terms of how our workplaces are changing and how AI is requiring us to fundamentally rethink the jobs that people do, do you see this being a very practical application of this taxonomy — should it come true — where people can actually better divide the labor between the human and the AI?

Ben: That's exactly right. So one way to think about what jobs are, fundamentally jobs are bundles of various work activities. When you get hired into a job, you're hired to do a set of responsibilities, not just one thing.

Technology — really any technology, and AI is included — does not usually automate jobs wholesale. That's very rare. There are some cases, like switchboard operators, that did get automated wholesale. But mostly, what technology does is automate components of jobs. Certain work activities become automated, and then there is a reconfiguration of what people do as a result.

So I think the right framework is not just to have a taxonomy for occupations, but a taxonomy of work activities. What are the things that people do? Because then it becomes possible to determine which activities are most suitable for automation or augmentation and think about how jobs, the collection of work activities, can transform and reconfigure.

Who and How Do You Create a Job Taxonomy People Agree On?

Siobhan: I want to dig in a little bit to the nuts and bolts of how you see this working, because you talk about how the actual activity of doing this is now possible because large language models can do the parsing. I should warn anybody in the audience before they read the book: eat first, because there's a lot of pizza talk. I was very happy that I ate beforehand (and I don’t like pizza with olives).

Typically, it would fall on the HR function to decide what the jobs are and what the job descriptions are. So who in a company do you see owning the creation of these taxonomies?

Ben: Right now, HR typically own the categories of people. That's not to say that they develop them. They rely on consulting firms to survey people and manually develop taxonomies.

I think that's unnecessary in today's day and age. And I think it's a mistake. Because large language models can do that in a way that doesn't have to be repeated every couple of years and can reconfigure very flexibly. So it can be developed in-house with data scientists or data analysts or data engineering. It is about data. It's about categorizing data. Whether they're working on behalf of HR, I think is probably still the right move.

It doesn't have to be done by in-house. It could be done by a vendor or maybe some third party APIs — there are ways to still outsource this. That's less of a concern in my mind about who actually does the implementation. But what I'd say is it is on its face easier than it's ever been and more useful than it's ever been, whether that's done by person A or person B, it's whatever is most suitable for the organization.

Siobhan: I have a library science background and one thing that struck me — and I was laughing because you brought this up — is the example of a product manager and how that same role can change from one company to another, and how frustrating that is for employees and employers alike.

I’m wondering how you get people to agree: If we're going to have a universal standard taxonomy of jobs, how will people agree on the terms? Even inside a single organization, agreeing on a common taxonomy is like pulling teeth.

Is the machine going to add a certain amount of authority so people will just go with it? Or is the urgency going to override those previous problems?

Ben: Ultimately a good taxonomy is a useful taxonomy. And it has to be well-suited for the end users, which could be hiring managers, people doing compensation benchmarking, job seekers, etc. It's in everyone's interest to have alignment between those parts of the organization, those parts of the market.

So having a common language is important in its own right. Now, how do we decide what the right label is is a different story. So the way that I start thinking about this is, you brought up the example of project manager, that is an occupation. And an occupation is a cluster of jobs that have similar sets of work activities. So we really need to define jobs as collections of work activities and be able to analyze what people do in their jobs. Now, there is some ground truth data on what people do. And that really comes from two sources. One is within job listings. Companies post job ads, and they outline what are people expected to do? What are their responsibilities?

In every job posting, and there's billions and billions, there is a section on responsibilities. That is a huge corpus of text that we can train large language models on. There's another big corpus of text which actually comes from the individuals themselves. When someone has a resume or an online profile, they write down what they did. The bullet points on someone's resume, those are an outline of what they had done in their job. So think we have a lot of good text.

What that gets us is the ability to understand what jobs are the same. So if we say project manager, in some organizations, maybe that means an engineering lead. In other organizations, maybe that means someone who organizes events or something. It could mean something totally different if they have different work activities associated with them.

Using a label does not necessarily mean that that's the right label. But once we are able to develop a cluster of jobs that look the same and that have the same set of work activities associated with them, then we need to label that cluster. We need to find out, all these people are doing the same thing. What should we call them?

Here's the easiest way to do it, which is almost good enough, is just pick the most common title. If some people say lawyer, another person says attorney, another person says counsel, just pick lawyer. Just pick something that is the most common, because that's going to have the most salience to users. The difficulty there is when you start introducing hierarchical flexibility.

Every good taxonomy has a really granular version and broader versions. Let's say you have a set of occupations that are, let's say, physical therapists, occupational therapists, speech therapists, and you want to combine those into one broader cluster: you can't really pick one of the most common title because you don't want to just call them a speech therapist. That would be a mislabeling of the broader occupation.

The way that we do that is we use generative LLMs to pick something that does a good enough job. And then it does something called therapeutic occupations or whatever it's called that balances the need to kind of cover the full scope of who's included and be specific enough so that it's differentiable between some neighboring occupation. So you want that to be distinct from the social work cluster that's right nearby or something.

I think these LLMs do a pretty good job. Sometimes when we introduce humans in the loop, sometimes they have useful feedback.

But most of the time, that actually makes things a little worse.

Siobhan: Yeah, I heard the ellipsis there — sometimes they have useful feedback.

Ben: There’s this joke in AI that I love: you need three things to fly a plane — a computer, a human, and a dog. The computer flies the plane, the human feeds the dog, and the dog bites the human if it tries to take over from the computer.

Siobhan: I’ve never heard that before, but I love the introduction of jokes on the show.

Ben: Yeah, sorry for butchering it. But the point stands: sometimes intervention helps, but often it adds obfuscation.

What's the Taxonomy of a Good Manager?

Siobhan: I want to talk about its application in workplaces. Everyone’s had the experience of being hired for a job that looked one way on paper and turned into something completely different. How would this approach help in that scenario?

Ben: It happens all the time. Someone goes into a job and they find themselves doing something different. Even within that example there's a distinction that is worth making about whether it's different on day one. You know, maybe the job was just misrepresented. That is probably a function of not having a good understanding of what you're actually hiring for.

Maybe you use the wrong title. Maybe you outline the responsibilities in a confusing way. Maybe there's an issue with interviewing. It results from a lack of clarity, so it's an unforced error.

On the other hand, maybe it is aligned on day one, but the job transforms over the course of a year or over the course of five years.

That is probably desirable and needs to be that needs to be tracked more rigorously. What we're really talking about is the collection of activities that people do in their job.

A job is a bundle of work activities — I don't know how many times I'll say that, but it's relevant. And that set of activities can evolve from a few different things to a macro shock to the system: a new technology that comes out, it's COVID, it's something that happens in the world that changes the way work gets done. So if a new technology automates some components of a job, that opens up the possibility for that job to be reconfigured and rethought. That's one type of reconfiguration.

But there's another, that is more common than we give it credit for, and that is just the natural day-to-day reconfiguration. In my own firm, we're 70 people, so not a big bureaucratic behemoth. Changes happen all the time. Client requests come in, someone quits, we decide something is strategic. There's new things we want to take on and we figure out who's got the bandwidth, who can take this on. Maybe someone is doing a certain type of work that they're actually not that good at or not that interested in.

It's the job of managers to be the connection point, to be the liaison between the needs of the business and the capabilities of the people that they're managing. And that's how jobs get configured. They are adapting to a changing set of requirements and changing set of circumstances. That's a very fluid process that happens every day and usually has nothing to do with technology.

Siobhan: I’m glad you brought up managers, because people management is always treated as just one activity. Nobody really knows what makes a good manager. How can this approach improve that?

Ben: I’ll go out on a limb and say management is essentially job reconfiguration. Management has to contend with a set of needs. A set of people who have skills and attributes and interests and ambitions — and the work needs to get done in a way that delivers on the business needs.

That requires a constant rethinking of who's doing what, how they're doing it, how those work activities are combined into jobs that exist in one person, where coordination needs to happen or not happen, and when it's appropriate to introduce technology or a vendor or a freelancer. Managers must think about the whole collection of inputs to complete work activities.

You're right that management as an occupation and as a discipline has taken a backseat in recent decades. It used to be, if you went to business school in the '60s, your next step would be to run a division. Now, if you go to business school, you're going to be a level-two analyst and executing things. Management education and formalizing the idea of management has not happened so much recently. But it deserves a front-row seat in how we think about how organizations adapt to technological change. I think they are at the front and center of that.

The Limitations of Using Online Data Only

Siobhan: To your point, in just the last few years we've seen this downgrading of management and of its importance in keeping a business running. At the same time we have all of the studies around the impact of a good manager on general productivity, and also specifically in AI adoption and how management creates the circumstances in which employees are more likely to adopt.

When you think about where you're pulling this information from, and you discuss getting it from online sources, from LinkedIn, from resumes and similar. All information is not online, which we now see with OpenAI. Do you see any way of supplementing that and overcoming the limitations of online data and the unreliability of some online data?

Ben: Supplementation is one approach. Even within a biased sample there are adjustments that can be made that address certain biases — not all, but some of the most important ones. When I say bias, I'm really talking about statistical bias.

In the example of public online profiles, that is not a random sample of the population or of the labor market or even of the formal labor market. It is a biased sample. That's a pretty straightforward thing to solve from a statistical sense. You need sampling weights and to treat it like a stratified sample. You need to basically estimate the likelihood of every observation existing in that sample and give it a weight that's inversely proportional to that likelihood.

That's one approach. Another source of bias are the lags when people report transitions. So if you want to see higher rates of attrition rates or something like that, you can't just look at the most recent set of start dates and end dates. You have to estimate how much time it typically takes someone to update a transition and then model that whole distribution and try to produce some sort of prediction of what you think might have happened. Those are things that have technical solutions.

It's important to note that LLMs don't do that. They take the universe of online profile and say, Here it is. They're starting to downweight or exclude certain types of information. But as we get into very niche subdomains, like this example of employment, we have to take these biases really seriously and do the proper statistical adjustment to address those.

The idea of supplementation is interesting. If there's data out there that exists from different sources — maybe there's private data, maybe companies themselves have HR records, maybe there's ways to get data from someone's footprint within internal company systems. These are all interesting and exciting approaches.

Clearly there's a lot of things to think through. Sometimes when you get data from internal systems, that limits your ability to report on that data. So that's maybe a trade off that needs to be weighed. But as a data guy, I'm so hungry for data. Wherever it is, we want as much of it as possible and then we'll adjust for problems later.

Siobhan: Yes, there's a whole rabbit hole of data localization we can go down, but let's not go down there. I want to wrap up soon, but am curious if you applied this approach inside your own company as you’ve grown?

Ben: When you're first starting, you really can't. In a startup, everyone does everything, everyone's job transforms all the time. It's so fluid. Of course we have structures. We know how many people we have doing sales, doing data engineering, how many economists we have, how many people we have doing front-end development. These are groups and teams and categories of people.

These are the metrics that we think about when we are thinking about who should we hire next. How do we report to the board? How much are we spending on sales and marketing versus R&D? These are things that every company, even small companies, do have to manage. And it's OK when you're a small company to do that manually.

It's like, I know this person. Are they closer to this kind of work or that kind of work? And you just make a call. That's fine when you're really small.

But every company beyond a certain size has to have a more structured way of doing that. The person who is doing finance or reporting or compensation benchmarking, they don't know every person personally. They have to figure out some way to categorize these people with some rigor.

Maybe they have a team that does it manually, maybe they hire a consulting firm to do it also manually. But I think all of those are worse than what can be done using modern technology..

Siobhan: Is there anything we didn’t cover that you want the audience to know?

Ben: Nothing specific comes to mind. There’s the book and plenty of resources.

Siobhan: You’re very active on LinkedIn, which I recommend, and your newsletter has great case studies. Ben, thank you again. Congratulations on the book, and best of luck.

Ben: Awesome. Thank you.