Metrics of Success in Development - Part 3

Today we’ll finish the list of ten questions that can give you a quick measure of your development group or department. The purpose is two-fold: to let you see how you measure up compared to other similar departments, and to suggest ways in which you can think about the stresses in your department.

Let’s launch into the final four questions, then we can total them up.

7. Viewed from other departments (outside of Development), how would managers rate your development managers and engineers in each of the following areas?
(a) Cooperativeness (with outside people)
Extremely cooperative - add 3 points
Very cooperative - add 2 points
Cooperative - add 1 point
Uncooperative or unavailable - add 0 points
(b) Flexibility (willing to work with and compromise with others outside the department)
Extremely flexible - add 3 points
Very flexible - add 2 points
Flexible - add 1 point
Inflexible - add 0 points
(c) Team-orientation (beyond the Development department teams)
Extremely team-oriented - add 3 points
Very team-oriented - add 2 points
Team-oriented - add 1 point
Not team-oriented - add 0 points

8. What percentage of the company’s gross revenues in the most recent year were allocated to Development (or R&D)? (If your company has no revenues, or if revenues are less than the Development budget, answer “over 10%”)
(a) over 8% - add 3 points
(b) 4 to 8% - add 2 points
(c) 2 to 4% - add 1 point
(d) less than 2% - add 0 points

9. Comparing this year’s Development budget to last year’s, how did it change?
(a) Increased by 20% or more - add 3 points
(b) Increased by 5 - 20% - add 2 points
(c) Stayed the same or changed by less than 5% - add 1 point
(d) Decreased by more than 5% - add 0 points

10. The number of concurrent development projects now in my department is
(a project is defined as an activity with a timeline and a goal which needs at least 1 full-time person to make progress towards the goal; if you have many small projects, you can count the number of project leaders instead)
(a) over 25 - add 0 points
(b) 15 to 25 - add 1 point
(c) 5 to 15 - add 2 points
(d) 1 to 4 - add 1 point
(e) none - add 0 points


If your total points add up to 52, you have a perfectly-performing development organization and have no need for improvement. For the rest of us, the points are probably in the following ranges:
Excellent: 37 to 52 points
Good: 22 to 36 points
Fair: 13 to 21 points
Poor: 12 points or fewer

How did you do? What does this mean? If you think about the stresses on your department, you can see that the point score is not as significant as the individual issues you’re facing. Are you having a lot of turnover? Slipped schedules? Complaints from other departments about your people? These can all be aggravated by declining budgets which are outside of your control.

In the next few weeks we’ll examine some of these issues and how you can find ways to work around them. In the mean time, click on the Comment button and let us know how you scored.

Metrics of success in development - Part 2

Last week we listed the first three questions of a self-assessment questionnaire for Development managers. Those first three related to project completion, staff turnover, and how well the initial functional or feature list was met. If you are having problems delivering products, most likely you will experience problems in one or more of these initial three areas.

There is a less tangible measure that relates to suitability of the product to the customer. But I'm still trying to figure out how to ask about that in a way that leads to a consistent and useful measure. If you have any suggestions, please make a comment on this blog.

Here are the next three questions:

4. How many hours per week are put in by your project or program staff? This is to be answered in two parts: first for the average over the life of the project, and then for the peak of the project or program. Answer this for each of the most recent 3 projects or programs.

4a. Average over the project or program:
Over 60 hours/week (add 0 points per project)
50 to 60 hours / week (add 1 point per project)
40 to 50 hours / week (add 2 points per project)
less than 40 hours / week (add 0 points per project)

4b. Peak week during the project or program:
Over 80 hours / week (add 0 points per project)
70 to 80 hours / week (add 1 point per project)
60 to 70 hours / week (add 2 points per project)
40 to 60 hours / week (add 3 points per project)

5. How many direct reports do you have? (Direct reports include staff assistants, administrators, secretaries, project leaders, managers and interns)
Over 12 (add 0 points)
10 to 12 (add 1 point)
8 to 9 (add 2 points)
3 to 7 (add 3 points)
0 to 2 (add 0 points)

6. Of your direct reports, how many do you regards as "problem" employees (people you will replace when there is an opportunity)?
0 (add 3 points)
1 (add 1 points)
2 (add 0 points)


Questions 4 and 5 are searching for sustainability in your project loading. It's OK to have peak weeks in which people are working 10 to 20 more hours than is typical during the project. But if you are driving everyone to work more than 60 hours per week on a long-term basis, you are not getting output that is sustainable. It may be time for you to institute metrics that measure results rather than inputs. Then you can experiment with different work schedules to see which ones result in the greatest output.

Sustainability applies to you and your management time, too. If you have more than 9 direct reports, the odds are that you are (a) not managing all of them as well as they could be managed, or (b) overloading yourself with management tasks that give you little time for planning and strategic analysis.

Next week we'll complete the question list and tally the points for a complete score.

Metrics of success in development - Part 1

How do you find out if your development organization is functioning well? Naturally, if you are getting products out on time, consistently, and the world around you is happy with the results, you have nothing to worry about.

But what if there ARE complaints? Can you determine whether you're hearing gripes that have little to do with you? Or whether there really is room for improvement?

I'm working on a self-assessment tool, probably containing about 10 questions, that will help you evaluate whether you are on track. The way to use the tool is to answer the questions and then count the points at the end.

Here are the first three questions in my current working draft:

1. Counting only the past 3 projects and products under your management, how many have been completed on time? For each project/product not completed on time, how much later were they completed?

[4 points] for each project completed on or before the originally scheduled completion date
[3 points] for each project completed within 120% of the originally scheduled duration
[2 points] for each project completed within 150% of the originally scheduled duration
[1 point ] for each project completed after more than 150% of the originally scheduled duration
[0 points] for each project which is never completed

2. During the time that those 3 projects or products were being developed, how much turnover did you experience among your technical staff, including first-level supervisors and managers? Turnover means departed or transferred out from project teams or management without being invited to do so by you or your managers.

[3 points] less than 5%
[2 points] between 5% and 15%
[1 points] between 15% and 30%
[0 points] over 30%

3. In those 3 projects or products, how much of the planned functionality was delivered (at the completion date you used for question 1)?

[add 3 points] for each project in which you delivered all of the planned functionality, plus additional functionality defined after the start of the project.
[add 2 points] for each project in which you delivered all of the planned functionality
[add 1 point] for each project in which you delivered at least half of the planned functionality
[add 0 points] for each project in which you delivered less than half of the planned functionality

If your total points add up to 24, you have a perfectly-performing development organization and have no need for improvement. For the rest of us, the points are probably in the following ranges:
Excellent: 18 to 24 points
Good: 12 to 17 points
Fair: 6 to 11 points
Poor: 5 points or fewer

Shipping products on time is the key result that most of your stakeholders want. Shipping the product with the features and functions that they asked for -- or expect -- is next in line as a measure of your success. But if you are burning out your development crew -- and causing turnover as a result -- you won't be able to sustain the results. Therefore, these are the first few measures that tell you whether the organization is successful and sustainable.

Next time we'll begin looking at other measures that can tell you whether your management practices are helping you build a development organization that is consistent and successful for the long run.

Comments? Click on the comment button.

Constant Reinvention = Survival

Nothing lasts forever. Even the best-conceived business strategies eventually become constraints on growth.

Consider Dell. "Dell succumbed to complacency in the belief that its business model would always keep it far ahead of the pack." But the competitors got better while Dell failed "to invest in new business lines, talent, or innovation that could provide another competitive edge." * [see Business Week citation below]

As a leader in technology or product development, you may think that your entire job is to execute well on the development plans laid out by Marketing or a strategic planning group. But you can do more. You can help the executive staff recognize that the business has other opportunities.

Consider what the Business Week authors went on to say: "Long-term success demands constant reinvention." This means that while you're turning out products that meet the current set of goals for functions, price and quality, the viability of the company may depend on your pointing out where innovative products or services could come from, using the brains you already have on your staff. Reinvention means re-thinking the orthodoxies everyone has accepted as the characteristics of the company. Do you have some independent thinkers on your staff who keep coming up with off-the-wall ideas? Maybe some of those ideas are actually your ticket to survival.

"Nurture the next growth platform long before it's needed." This means you have to carve out the budget to support the radical ideas from the operating budget you're supposed to use for mainstream development. If you can't convince your executive staff to fund a skunkworks operation, then you should look at having some of your key contributors doing some off-the-record investigations. Is this risky in your company? Then maybe the company needs some shaking up.

"Most [companies] don't [nurture the next ideas]. Distracted by the demands of their current success, they re lulled into a false sense of security." It's easy to focus only on the tasks that will satisfy the demands of current customers and current ways of doing business. And while it's not easy to perform on those tasks at extraordinary levels, you can get lost in gunning for the immediate satisfactions of meeting this quarter's goals. Can you be a VP or Director of Engineering and still make time for thinking about next year's products and the businesses that you haven't entered yet? Consider this: who else is better positioned to view what's possible, who is out there needing better functions or services, and what can be combined to make a new business?

I suggest taking on an advocate's role as part of your commitment to the long term success of your company. You don't have to be a marketer or business analyst to know what's exciting and feasible in the next generation of products and services. Carve out a niche as a visionary and a keeper of the wild ideas that can open up new busineses for your company. Do it regularly, and you'll be twice as valuable as the person who only meets the usual goals of Development. Besides, it may help your company survive.

----------
* "Where Dell Went Wrong" by Nanette Byrnes and Peter Burrows in Business Week, February 19, 2007, pages 62-63.

http://www.businessweek.com/magazine/content/07_08/b4022074.htm?chan=search

Technologist or Manager?

Who is helping you to become a better manager?

If you're lucky enough to be employed at a larger company with a long-term view, like IBM and HP, you may actually be getting training and coaching that helps you grow in your job as a manager. But most of the rest of us -- particularly technical people who have come up through the ranks -- are learning on the job, mostly by making mistakes.

In startups and smaller companies, there are no resources for helping managers get better at managing. The executives are satisfied if you simply get the job done, such as getting the product out the door. If the consequences are burnout -- by you or by your team -- that's just collateral damage.

Besides, managing is just good project management, isn't it? If you can fill in the data in Microsoft Project, print the charts every week, and make forward progress on your timeline, everyone will think you're in control and doing what's needed. Won't they?

I don't buy it. Managing is an art. Just because you came into the job with great technical skills and a lot of experience as a project leader, you don't necessarily find yourself as successful as you imagine you could be. There could be better ways to keep the teams aligned on the projects. There could be better ways to coach the managers under you on dealing with "people issues." And you could be inspiring your management team with more innovative ways to develop products.

If you're struggling with being a manager or an executive responsible for high-tech development, you may be part of the collateral damage. It could be time to look for ways to learn about the art of management. First you have to believe that managing is a profession, and that you can be good at it; that it's worthwhile becoming a professional at managing; and that the manager's role goes far beyond being a technology leader.

One of my talks last year was titled, "What does management have to do with innovation." You can get a copy of the outline at http://johnlevyconsulting.com/pdf/Levy - Mgt & innovation.pdf

You have to put yourself out there to become a good manager, to keep your teams from burning out, and to enjoy the role. Make the effort, it's worth it.

Loss Leader

My colleague Joel Harrison is good at encapsulating learnings from his experience. A few weeks ago, while I was visiting him at his startup company, Abrevity, he said, "You can't justify a new product based on a cost analysis of the first-generation product. You have to have a vision."

Joel and I had experienced the frustration of trying to create new products at a company that was in a high-volume, low-margin business -- hard disk drives. On the one hand, Joel had prototyped a product that could have been the first available Ethernet-interfaced free-standing disk storage unit. I had been involved with defining a disk drive that stores and plays back video streams without a computer attached. While our company had funded the early prototyping of these products, it did not make the investment needed to launch them as consumer or end-user products.

Joel's explanation, as I understand it, is that the company did an analysis of the cost of the first products in each case and concluded that the product cost too much to be priced reasonably in the marketplace. Now here's where "vision" comes in. When you're introducing a radical new product, you have to price it not based on the initial product's cost, but based on a combination of the needs of the market and the expected cost curve as volume increases.

Companies selling services, such as cell phone service, do this all the time. To make the service workable, they have to invest a large amount of capital in infrastructure, such as cell phone towers, switching equipment, and so on. But pricing of the phone service must chosen both to make it attractive to the consumer and, when the number of subscribers reaches a reasonable target, to make a reasonable return on the investment.

The same thing is true with new products. The barrier to radical innovation and new product introduction in companies that have been operating in a low-margin high-volume environment for years is primarily a failure of imagination. They need the vision to see that (a) there is a market to be created or captured, (b) the product they have conceived is viable, and (c) initial pricing will lead to losses during the early stages of market development. Venture capital is based on selecting and funding this sort of innovation. But old companies have trouble thinking outside the low-margin, pay-for-itself-or-die product box.

That's what Joel was telling me. If we could have planned the new businesses beyond the first product, and had got a commitment to fund the initial losses, we could have made history in disk drive marketing.

Shifting the focus to longer term

Startup organizations are typically unsustainable and barely stable, because:

1. The pressures to develop and market a first product require taking some expedient shortcuts, such as hiring the most capable, but not necessarily the most team-oriented individuals; placing all priority on getting a workable product out the door, rather than building the product for maintainability and growth; putting in the most features rather than the best-tested features.

2. The top management habitually focuses on the race between funds running out and product delivery, rather than on internal communications, employee satisfaction (except with the potential value of their options), and leadership style. The command-and-control management style is workable for the first few years, but typically fails to inspire the organization to build itself into a self-renewing structure.

3. Having a focus on delivering a product using already-developed technology, the company does not need to invest in longer-term development of underlying technologies, or in the people who will bring in a steady stream of new technology.

The short-term focus of a startup must change soon after the deliver of the first few products. Companies that fail to incorporate longer-term thinking around their third year find themselves living from crisis to crisis. This makes the company unattractive to good managers and good technologists who don't necessarily get their jolllies from living in a startup environment, where "startup" means short-term thinking.

What sort of changes does your organization need, now that the product has been delivered? A new CEO who actually allows the organization to function as if there were competence at all levels? A seasoned technology executive who knows what to do to make the organization attractive to innovative people? A shift in emphasis to listening to customer feedback and involving existing customers in product decisions? Addition of a Quality department that actually has the teeth to delay a product introduction?

Whatever the changes needed, don't be surprised by the shift. Two reactions to the shift are typical:
(1) "What happened to my adrenaline rush?" -- the people who need crises to keep up their energy should pursue another startup.
(2) "I didn't know that a company could actually plan and execute with the future (beyond 1 month) in mind!" -- the people who are stressed by the company's failure to plan and execute for the long term grow into steady, reliable contributors.