Agile has Not Been Agile

Matt Perez (00:03):

Hi, this is Matt Perez. We're this is rHatchery.live show, and today we have Cliff Berg with us. And Cliff has been doing the actual thing for a long, long time. He mentioned the thing about apple Tool, and that's that Dave, him, and us. And he's got a book that he is, he's got up there. And so we're going to be talking to him about what he sees as a problem, what he, what they're doing today, and where we're seeing where we think we're heading. And with that cliff, do you want to briefly introduce yourself and then get started?

Cliff Berg (00:50):

Sure. Yeah. Well, thank you. Thank you, Matt. And, and, Jose you know, I have been in a career of one type or another for a long time. And, you know, as, as Matt said, you know, my agile journey started when the Agile movement started. I was 20 years into my career at that time, I was CEO of a company that I had co-founded, which had grown to 200 people, and we decided to adopt extreme programming. And that was in 2000. That was in 2000.

Cliff Berg (01:23):

And, it went well. And I, I realize now that it really went well, mostly because the people who led those teams were really good <laugh>. It wouldn't have mattered what method they chose, the matter what they had done, it would've gone well. But you know, my agile journey was in the context, began in the context of a company that built some really difficult mission-critical systems for McKesson Pharmaceuticals bank of America, federal Express Capital One Bank, and United Overseas Bank. And you know, in a context where, you know, lives aren't at stake, but money is at stake and company reputations at stake. And these were non-stop systems that had to work handling customer-facing, very often b2b data and real-time transactions. And the challenge was you know, we had our own performance testing system that we built on our own dollar.

Cliff Berg (02:27):

We did everything fixed price, by the way, and things worked. And my concern was how to move fast, but make sure that stuff really worked. And I ended up writing a book where I, I collected what, you know, kind of what I thought and what I had learned about that. And I actually collaborated with Peter Noman on that who has pretty good credentials and, and system reliability in, in, systems. But I realized that in order to move fast, you really have to be intentional about how things work. And you also have to let that intention evolve because you're constantly learning. And that, that really was the core of agility for, for billing high-reliability systems to, to be intentional, to have a working design, to have the assurance argument, so to speak, which is, you know, the concept for how it's going to, why you think it will be reliable, why you think it will really work and really will scale.

Matt Perez (03:37):

So, so what has happened? What has happened since then? That, has that changed? Or, you know, does the need change? Does the methodology change? Is agile, agile you know, all those things?

Cliff Berg (03:52):

Well yes and no. You know, agile is different things, different people. And I have a lot of debates. I'm pretty active on LinkedIn. You know, I try to be very accessible and I am very happy to engage with people in discussions. I try to always be polite. I, I try to, I, you know, sometimes I slip up, but I try to do my part to make social media a pleasant place and not an antagonistic place. So I try to always stick to the issue. But I find that, in discussions about some of these things there, most people tend to see things in a very fluid way. And, and, and, and think generally that to be agile, you have to be contextual. But there are some people who think that being agile is kind of this very precisely defined thing, and that there's the best way, and I strongly reject that.

Cliff Berg (04:45):

But politely, you know, I, you know, with respect, I reject that. You know, I think it's completely contextual. I don't think you can define best practices for these things. I, I think, I think you can have patterns for people to consider, but then they need to adjust those patterns. They need to take the ideas of those patterns and then come up with their own approach, not copy the pattern. They need to take the idea and then figure out if that idea is useful. And then figure out how they might use that idea to create their own process or their own template or their own whatever. Because there are lots of good ideas out there, but you have to adjust them for you because people are different. People are neurodiverse, you know, some people are very effective doing test-driven development, for example or pairing. And some people are not, and never will be, no matter how long they try to do it <laugh> because they're neurologically different. And that is not something to be ashamed of or being labeled as. You're not agile or something. It's, it's giving people agency about how they work.

Jose Leal (05:56):

Cliff, I, I want to jump in because as I'm listening, I'm thinking of, of myself and, and Matt and going, yep, that all makes sense to us. And then I'm wondering, well, is it making sense to the audience? Because maybe not everybody knows what Agile's about. And so could you give us, just a thumbnail of what was before Agile? What was agile and what's agile too?

Cliff Berg (06:22):

Well, it's followed kind of an interesting arc. You know, as I said, in 2000, 2001, I was 20 years into my career, and I saw a lot of change before that. During the eighties, things were very freewheeling and some of the most agile teams I had ever been on were during the 1980s. During the 1980s, I, I was on a series of teams at two companies where I worked inter metrics and CAD language systems where we collaborated constantly. Even though we had our own offices, we collaborated constantly. We ran fully automated integration tests every night, fully automated integration tests every night. We worked against the feature backlog. We had team leads who were project managers, quote-unquote, but they were not what you think of today when you think of project managers. They were people who were really immersed in the work.

Cliff Berg (07:18):

And, and in those cases were very inquisitive and empowering. And, when they came into the room, you were happy. You weren't afraid. You were thinking, oh, Carl's here, we're going to have a really deep discussion about stuff. Carl, we're working on this. What do you think? You know, and he would never boss you around and tell you what to do. You know, when he left, you felt like someone really helpful had just been there. That was the project manager. You know, so you know, I, I hesitate to call him a servant leader because he was more than that. Servant leadership is a really kind of insufficient model for the kinds of leadership that are needed. It's an important mode, but it's just a mode of leadership. You need more than that. But what happened you know, after the eighties you saw the rise of P M I and a lot of organizations, probably most large organizations decided that they needed to have software development methodologies, and most of them based it on some kind of waterfall approach. And that all arose during the nineties. Before that, it wasn't like that, and it got worse and worse and worse. And when the Agile Manifesto came out, it was a rejection of all that. It, it basically was, you know, this, this phase-based stuff doesn't work. You know, we need to go back to these things. It was about going back to it wasn't new. It was about, it was returned under

Jose Leal (08:54):

A new, under a new name. Under a new philosophy.

Cliff Berg (08:57):

Yeah, under a new, which happens very often in a culture where old ideas get aggregated, and then there's a new label. You know, I if you, I I like to point out that, you know, the pyramids, you know, before the Great Pyramid at Giza was built, they had built a little pyramid and then a slightly larger one, and then a larger one, and then a larger, and then the <laugh>, you know, so when they were building the pyramids, they started small, and then they tried and big and bring them <laugh>.

Jose Leal (09:24):

I wonder if they had a different label on the first one,

Cliff Berg (09:27):

I'm sure. <Laugh> pyramid

Matt Perez (09:31):

One, pyramid, two

Cliff Berg (09:32):

Pyramid. Yeah. So most of this stuff is not new, it's just relabeled or kind of rethought and, you know but then, you know, the, when the Agile manifest, well, well, first of all, extreme programming really launched the agile movement in the U.S. in Europe, there were some other things, but in the US it really was extreme programming. And it was all about xp. And you heard people talking about xp, xp. Have we tried, have you tried xp? You know, we should try xp? Well, my company, we did. But then the Agile Manifesto was created you know, a couple, of two years later after the XP book came out. And then it became a bit, and not xp it became about Agile. But what's interesting is that XP and Agile are actually kind of opposites, because XP is extreme by definition.

Cliff Berg (10:25):

It even says it's extreme, extreme programming. The Agile manifesto is the opposite of the extreme. We value all these things, but this is more than that, so it's all about judgment and context, not about extremes. So the Agile manifesto was the opposite of extreme programming. A lot of people missed that. But then what happened is you know, after that you had, you had kind of XP and Scrum, which existed prior jumping on the Agile bandwagon to kind of benefit from that, claiming retroactively that they're agile, and maybe they are in some way, but, but they're not the same as the Agile manifesto or something else. And then the agile movement took its own arc, and it, it, a lot of really good ideas developed and a lot of really bad ideas developed too. And you saw this mixture of great ideas and toxic ideas within the same thing.

Cliff Berg (11:27):

And a lot of the toxic ideas really took root. And, you know, some of them came sort of from the manifesto, from some, you know, I think maybe misinterpretation of the intent. And, you know, my favorite is the one about self-organizing teams. And, you know, it literally says the correct thing. It's, it, it says that the best thing comes to self-organizing teams. But what it doesn't say is that that's a rare case. You know, in order for teams, if you have, a large organization, an ecosystem to expect that teams will self-organize and perform well, then there are pre-conditions for that. The organization really needs to have a very healthy culture. You know what, what some models call a constructive culture or generative culture.  And you have to have teams where the individual on the team has relatively high EQ. And, you know, if you have those conditions, you will tend to see self-organizing teams because, because people in leadership roles who constitute teams will take people and put them together and have confidence.

Cliff Berg (12:34):

Yeah, they'll, they'll be able to do that. I don't need to mess with them. They'll do that and they will. But in most organizations, it won't work that way. You know, because most organizations do not have a very healthy culture, and the people do not have, by and large, a high eq. And so it won't work. And, you know, in the fairly recent book, accelerate by Nicole Fortran and Judge Hum and Jean Kim Forrest Grand documents, her research, and which was very extensive, and she found that the teams that had the highest performance were the teams that had transformational leadership. You know, not self organizing. So, so there were a lot of things, a lot of narratives that emerged within the Agile community that were extreme or that weren't quite right and kind of, kind of made it very dysfunctional. And the, the goal of Agile two was, was to kind of reset and, and think, you know, what, you know, what, what is real agility?

Cliff Berg (13:37):

Forget agile, what's real agility for organizations? And what does it come from? And you know, kind of starting with ale clean slate and being more scientific about it, looking at what we know in behavioral psychology and leadership theory and cognitive science and operations research. And because a lot has been done in those fields to, to identify like what happens when groups of people come together, you know, what actually happens? What's the behavior you tend to see? And how do you, how do you get groups of people to be effective and, and how do people communicate? Well, it's not just face-to-face. Mm-Hmm. <affirmative>. That's, that's a very simplified view that you just have people talking real collaboration about complex issues. You know, we like to say it's reading, writing, talking, listening, and thinking. You need all those things in different balance at different times, depending on the life cycle of the issue, the discussion, and how complex it is and so on.

Cliff Berg (14:38):

And so, you know, it's, it's not so simple. And, you know, so, and we wanted to, you know, and, and you know, there's a lot we know from cognitive science about how people think and create and, you know, do they need, do they need to collaborate a lot? Do they also need to isolate, you know, the, the, the, the, the I was just listening today to a a podcast or interview. And the topic came up of the incredible importance of, of solitude for creativity. Mm-Hmm. <affirmative>. So, you know, collaboration is absolutely essential. So is solitude <laugh> and different people need them, a different measure, you know, so none of this is simple. You can't take something like an Agile manifesto, which is basically a bunch of bumper stickers, you know, would, and that's not meant in a critical way because bumper stickers are good.

Cliff Berg (15:36):

They have their place, they remind us, but they're not guidance. You, in order to, to understand something like the Manifesto, you already have to understand all of those things. You have to have thought about it for decades, read books about it, discussed it, experienced and lived it, then the manifesto makes sense. And then you'll realize what it's missing. You know, you, it doesn't mention anything about leadership. Mm-Hmm. <Affirmative>, which is probably the most important thing of all. And it says nothing about it. And, but it's okay. It's okay. It doesn't, if we expect it to be this perfect thing where people are parsing the words like a religious document and that, you know, the answer must be there, you know, we just have to think about it the right way. No, the answer might not be there, you know, and some of it might be wrong, and that's okay. It's a great set of reminders. It's not perfect. It's not, it's not sufficient. Yeah.

Matt Perez (16:39):

So, so what are, what are you doing about it? You wrote a book, but what other things are you doing about it? And what are clients doing about it? Were you pushing your clients to do about it? And more from the perspective of where we get to a point where people recognize that they have to have a high IQ, they have to be good at listening, and they have to be good at expressing themselves. It's one thing, to tell somebody, oh, you gotta do this, as opposed to, Hey, we need to do this. How do we do it? Yeah. You know, they have 'em take ownership of that. So I get all that. But what, what's happening? What are you doing? What, are they doing?

Cliff Berg (17:23):

Well, you know, after the Agile two team finished its work you know, and we, we very quickly wrote a book which there's a 400-page book, and we wrote it and published it through a major publisher, Wiley, in three months. And that included all of the rounds of editing that happened. And people said it couldn't be done, but <laugh>, wow. I was determined that we were going to do it <laugh>. But if after, there was a reason why, there was a conference that I wanted the book to be out for. But anyway we then thought about, you know, how can we help people to use these ideas? And, you know, in the Agile community, immediately people assume that, well, you're going to create, you've created a framework and now you're going to have certification or something like that. We didn't do that.

Cliff Berg (18:14):

You know, there's no framework. Agile two is not a framework. Think of it as the Agile manifesto, but much more, much broader, more complete, and with an explanatory narrative. There, there's no how-to in it. There's intentional, there's no how-to, there's no methodology, there's no framework, there are no practices in Agile two, you know, and it was published effectively open source because it's, it's creative comments pub published. So anyone could write a book and repeat the entire Agile two principles, without violating anything. And that was intentional. So someone could create training courses if they wanted to. So I put a, group together too, and the intention of that group, agile two Academy, is to help people to use these ideas, to help them use the ideas. Because people do need help. And, you know, there's, there's a lot of value.

Cliff Berg (19:16):

I think that that's very authentic. You know, we're not selling some kind of process to always run or something, you know, when you help people and you know, we did create a course, but it's not a certification and it doesn't regurgitate agile two, you don't learn Agile two in the course. What you learn are the foundational things that help you to implement the ideas of Agile two. So, the course starts out with four modules about leadership theory, what we know from research path-goal theory, transformational leadership, and leader-member exchange theory, and all these different models and how they work, and why they're useful for thinking about situations. It doesn't say do this and do that. It gives you models for how to think about things. And then,  the course goes into some cognitive science and, and behavioral science behavioral psychology the Transtheoretical model, for example, is a foundational model, in behavioral therapy that explains how people change over time.

Cliff Berg (20:30):

Mm-Hmm. <Affirmative>. And, there are other models that, and, and what we know from how people think in System one and system two, thinking according to Daniel Kahneman's model mm-hmm. <Affirmative> and, then some things from operations research, which happens to be one. I, I have a degree in that. And, and so I, I was able to, to pull some things for that because lean and flow are kind of lenses of, of, of using operations research or, or vice versa. There's a lot of lean and, and flow that it's, it's kind of interesting from an operations research perspective. So the Agile two foundations course, we call it foundations because it doesn't teach you agile two, it teaches you the things you need to, that will be helpful to actually use the idea as an agile two. And it's not a certification because I don't think it should be. We don't, in college, you don't take one course and then get a degree <laugh>, you know, you take

Matt Perez (21:28):

A whole, but somebody, but somebody could, could take your book and, and stuff like that and create a certification program, right?

Cliff Berg (21:36):

They could, they could. And I, you know that would be okay, you know, because, you know you know, I, I've been told by a number of people whose names you'd recognize by the way, that you have to create certification because that's what drives acceptance. But we've just resisted doing that. You know, I, I think, you know, certification is something that's, there should be a very high bar. You know my nephew is a certified financial planner and financial analyst. He's got two certifications and the test for that, for one, each of those took a whole weekend, 16 hours, and he studied for a year. So, you know, to me, certification should be something that is a really significant bar. Not one course should be a whole curriculum of courses. You know, so if someone wants to create, an Agile tube certification, fine, we're not going to.

Cliff Berg (22:40):

But we do have a course that we think has a lot of value. And it also has been really useful for our clients that where we're helping to use these ideas because it gives people, especially people in leadership roles, foundational knowledge, that now we have a vocabulary for how to talk about things. You know, , we run workshops with people too, to help them to think through and generate their ideas, but we also do some instruction. We pull things from the course depending on what issues we're talking about, to give them a little bit of instruction so that now have a model and a vocabulary, and then we let them loose again and ha have them come up with ideas. So we alternate diagnostic instruction generation, you know, dialogic activity, you know, and we go, we alternate between those modes and ending up with a completely generative mode where people come up with their own solutions and you know, that's been working really well. And then people end up having a very strong sense of ownership. They're not implementing any framework. They came up with it so they understand what they came up with, and they're the ones who are in a position to be accountable and to improve it and make it get better and better and better and better.

Jose Leal (24:08):

<Laugh>. Cliff, , you just mentioned something there that speaks to something that we're very strong on, which is the question of ultimately, at the end of the day, can teams be independent enough when there is an op ownership structure and a management structure that is the traditional that we know the mainstream systems? What are your thoughts about that? Do you think that the system as it stands can allow for that to happen?

Cliff Berg (24:42):

Absolutely. Because the structure is not the problem.

Jose Leal (24:47):

Okay.

Cliff Berg (24:47):

The structure's not the problem. You know, very often large organizations need structure in order to decompose, you know, how they fund things or some, you know, how they, you know, where people sit, you know, it has to be, it has to be composable. So a group has to compose them into another group, you know? But how people behave is a whole different thing. And the old model, the old hierarchical company makes identical the structure and the behavior. You know, you, you stay, you collaborate within your group. If you want to go to another group, you talk to your boss, your boss then goes talk to that boss or gives you permission. It's always up and down the hierarchy, right? That doesn't work today. It's too slow. It's actually very toxic. It creates all kinds of dysfunction. It creates territories, you know, what highly agile organizations need, and the highly agile companies that we looked at you see a completely fluid interaction across organizational lines, completely fluid.

Cliff Berg (25:56):

And in fact, you see people in leadership roles making that happen dynamically. Not in the kind of planning way, but dynamically as issues come up, you know, getting people to reach out, go talk to that other group, you know, you and you and you know, solve that problem. Oh, there's a problem there. Okay. You know, who else do you need to talk to? Go talk to them, solve that problem, and it doesn't matter where they set. And the fact that there is structure, that structure is being used only for allocating funds or, or, or, you know, where people sit or something. It's not being used to organize how people actually work. You

Jose Leal (26:47):

Know? And you think that like you just said about allocating funds, so I'm a team, we're very innovative. We want to do something different, but somebody else is allocating funds, and the structure is still mandating what this team is capable of doing. Mm-Hmm. <Affirmative>, be it from a management perspective or a budgeting perspective, I understand what you're saying about I, I totally agree with you. There are environments where managers say, screw the system, we're just going to work together and make this work. But that's rare in today's world. That's very rare. As I'm sure you've noted in your book, the question then is how do you see those limitations becoming overcome? Because there's still constraining teams from being as creative and innovative as they are, they want to be in mis in many cases.

Cliff Berg (27:50):

Yeah. It's <crosstalk>

Jose Leal (27:51):

Two minutes.

Cliff Berg (27:53):

<Laugh>. We have two minutes. Yeah. Oh, oh, okay. Well, very quickly, you know, it's the behavior of the managers, you know, and that starts the top you know, people sit in places in an organization, and then basically that's how they get their paycheck. But that doesn't have to be how they spend their day. Just because they sit here doesn't mean they can't go work over there for two weeks or two months or work. It doesn't matter where they sit. They need to solve a problem with these other people. Why does it matter where they're getting their paycheck from?

Jose Leal (28:27):

Because they have to report.

Cliff Berg (28:29):

That's the problem. <Laugh>. You know, it goes to the managers who are expecting that people will work on their stuff, you know, instead of the manager, you know, owning a problem, I want to solve this problem and to solve that problem, I need some people from another group. Let me go talk to that leader over there. Okay, we're all trying to solve these problems, and they just forget about who's getting paid by what they're all being paid by the organization. You know, and so the incentives at the highest levels have to be aligned for that to work. You know, there has,

Jose Leal (29:07):

Yeah. And what you're describing though, is I think, a critical aspect of what we are focused on, which is it, you have to have a system, a structure that supports that type of agility and

Cliff Berg (29:23):

Incentive.

Jose Leal (29:23):

So when you, and it's in a structure, and when you don't, you don't that's really, really hard. And so that's, that speaks to the question that, that we are really strongly paying attention to. Yes. Which is, can you really have a self-managed team within an organization that does not have that kind of mindset?

Cliff Berg (29:48):

No, no. The organizational culture and the, the incentives originate, you know, from the culture and vice versa, you know, which came first, it doesn't matter. They influence each other. You have to fix the incentives. You have to fix the culture, which is basically the behaviors and expectations, of the people who have the most influence, which is the people with the most positional authority. If you don't fix that, then you're, you're hobbled constantly.

Jose Leal (30:23):

Awesome. Well, Matt, do you want to wrap up or,

Matt Perez (30:28):

Yeah, we're out of time. Unfortunately, this was really interesting. I could have kept listening and talking to you for a while. And then, let me see who's next. So thank you very much Cliff, for your time. And obviously, you thought about this a lot and come up with, you know, a very good approach too, to not spread dogma, which is the problem with the manifesto and things like that. And in any case, the next interview next week I think is Iamena Crolla, founder of CUMBIA a consulting firm, and change maker. And we're going to see what she's done and, and what she thinks the problem is and the usual stuff that we do here. And we expect you guys to listen to that one as well. And with that, I think we're done.

Jose Leal (31:31):

Thanks again, Cliff.

Cliff Berg (31:32):

Thank you. It was a pleasure. Thanks.