Metrics

[Anonymous]

I am reading and implementing Sally McGhee's book Taking Back Your Life (using outlook w/ GTD). There is a suggestion to create "Metrics" around projects, a way to measure your objectives/timelines/mini projects. I understand what they want you to do, but do not know how to do it or implement it. Does anyone have any good examples or an excell document that a Project could be outlined in?

 

[mcogilvie]

Re: Metrics

There is a suggestion to create "Metrics" around projects, a way to measure your objectives/timelines/mini projects.

Metrics are a difficult issue, fraught with danger. I believe more projects have failed because of a preoccupation with metrics than have been saved by them. A classic example of the difficulties of metrics is assessing programmer productivity. Naive measures such as lines of code written per day are known to fail. More robust metrics, such as the time requiired by two different programmers to complete the same project, indicate that programmers may vary in their productivity by a factor of more than 100! If you aren't sure of what you are doing, my advice is to stay away from metrics, except at cocktail parties, where it can be a nice buzzwork to drop into a conversation.

Mike

 

[kewms]

Re: Metrics

Metrics are a difficult issue, fraught with danger. I believe more projects have failed because of a preoccupation with metrics than have been saved by them. A classic example of the difficulties of metrics is assessing programmer productivity. Naive measures such as lines of code written per day are known to fail. More robust metrics, such as the time requiired by two different programmers to complete the same project, indicate that programmers may vary in their productivity by a factor of more than 100! If you aren't sure of what you are doing, my advice is to stay away from metrics, except at cocktail parties, where it can be a nice buzzwork to drop into a conversation.

On the other hand, it's very helpful to have some idea when you might finish a project, when you need to hire more staff, etc. That's difficult to do without some sort of productivity metric.

Katherine

 

[TesTeq]

I think that the problem with metrics is that all data used as a basis for estimation is statistical while it is applied to the real unique project in its real unique environment (team, deadlines, information availability, user cooperation etc.)
It is similar to rolling the dice. You know that statistically every sixth throw five spots will be on top but in real life you can predict nothing.
So metrics can be useful for feasibility study and preliminary estimation but you must be ready to monitor progress and update your plan continuously.
TesTeq

 

[mcogilvie]

Re: Metrics

On the other hand, it's very helpful to have some idea when you might finish a project, when you need to hire more staff, etc. That's difficult to do without some sort of productivity metric.
Katherine

You have to be very careful about personal vs. small group vs. large group. The CEO of a company may know that staff size needs to grow linearly with sales, based on metrics and models. A front-line manager is likely to construct a "metric" to justify staff hires, but the need is usually perceived before "data" is even gathered. Furthermore, there is a disinction between estimate and metric, perhaps subtle. An estimated completion time is an estimate. At the personal level, I think most people need to think more about advancing their personal goals, relying on experience and simple estimates for scheduling and prioritizing. Well-constructed metrics can be useful for large systems where statistical analysis is valid, but a preoccupation with metrics is likely to be a distraction for most people in their day-to-day lives.

 

[etcoutinho]

Re: Metrics

Well-constructed metrics can be useful for large systems where statistical analysis is valid, but a preoccupation with metrics is likely to be a distraction for most people in their day-to-day lives.

I respectfully disagree with you and so does Michael Gerber... Measuring progress is essential in evaluating your performance, and fundamentally there should be no difference between large or small organizations.

For example, tweaking of one's GTD system is a recurring theme in this forum, mainly because it is so difficult to evaluate the impact of a change. I have a routine of always recording the number of less-than-2-minutes actions concluded (Daynotez) and analyze it during my weekly review. Any changes in my system (e.g. new softwares) should increase this number, otherwise it is not worth it. It works for me, and you would be amazed how perceptions can be deceiving...

Regards,
Eduardo

 

[jpm]

Re: Metrics

Well-constructed metrics can be useful for large systems where statistical analysis is valid, but a preoccupation with metrics is likely to be a distraction for most people in their day-to-day lives.

I respectfully disagree with you and so does Michael Gerber... Measuring progress is essential in evaluating your performance, and fundamentally there should be no difference between large or small organizations.

Except of course for statistically significant things like sample size....

For example, tweaking of one's GTD system is a recurring theme in this forum, mainly because it is so difficult to evaluate the impact of a change. I have a routine of always recording the number of less-than-2-minutes actions concluded (Daynotez) and analyze it during my weekly review. Any changes in my system (e.g. new softwares) should increase this number, otherwise it is not worth it. It works for me, and you would be amazed how perceptions can be deceiving...

So can analyzing metrics. Did the number of less-than-2-minute actions drop on average over a number of days? If so how do you know this was do to a change in your GTD use system, or not some other factor (like being on the road vs. home; being in a good mood vs. a bad one, being tired vs. having lots of energy... there are a lot of potential variables and getting at cause and effect requires rigorous statistical practice....

When picking metrics here are some rules of thumb:

1. Pick easy to measure lagging metrics that are directly related to your goals: e.g. Cash in Bank Acount, %lean body mass/waistline, etc.

2. Pick easy to measure leading indicators %under budget, hours worked at gym (maybe intensity level on a scale of 1-10, gluttonous meals eaten, etc.)

3. Track leading indicators more frequently, but look at a rolling average. Track lagging indicators less frequently (no more than monthly).

While not necessarily statistically valid, this method is easier, and will make it less likely that you will make decisions based on anecdotal evidence...

 

[mcogilvie]

Re: Metrics

I respectfully disagree with you and so does Michael Gerber... Measuring progress is essential in evaluating your performance, and fundamentally there should be no difference between large or small organizations.

You are entitiled to your opinion, of course, but there is a difference between large and small organizations in sample size. Statistical fluctuations are larger for individuals and small organizations, and there is a human tendency to correlate those fluctuations with actions taken or not taken. Other, more general problems with basing actions on formal metrics can include:
* bias or subjectivity in the metric
* using an inappropriate metric
* equating correlation with causation

and so on.

By the way, I assume your Michael Gerber is the "e-myth" guy I found on the web. I have never heard of him before. That's one of the problems with "appeals to authority" as an argument: it doesn't work if the other guy has never heard of your authority!

For example, tweaking of one's GTD system is a recurring theme in this forum, mainly because it is so difficult to evaluate the impact of a change.

Here I respectfully disagree. Based on my own experience and the postings of others, I have come to believe that tweaking is essentially a subconscious response to failure in implementing the core habits of GTD. For example, I fail to do a weekly review, so I blame my tools.

I have a routine of always recording the number of less-than-2-minutes actions concluded (Daynotez) and analyze it during my weekly review. Any changes in my system (e.g. new softwares) should increase this number, otherwise it is not worth it. It works for me, and you would be amazed how perceptions can be deceiving...

People differ widely. That kind of dedication to routine is frankly not in me nor does the nature of my work lend itself to it. I went through a period where I exported my completed todo's to DayNotez. I never looked at the exported tasks, and I stopped.

Best,
Mike

 

[etcoutinho]

Except of course for statistically significant things like sample size....

I don't know what you mean by that... Sample size is an important factor (among others) in determining statistical significance. But the whole point in statistics is to make inference: something in a small scale should be valid in a large scale.

Anyway, I have never said that you should use statistics to analyze everything in your life! My point is very simple and direct: do not trust only in your intuition to track your projects. Instead, define something measurable (even if not perfect) and consider it in your reviews.

So can analyzing metrics. Did the number of less-than-2-minute actions drop on average over a number of days? If so how do you know this was do to a change in your GTD use system, or not some other factor (like being on the road vs. home; being in a good mood vs. a bad one, being tired vs. having lots of energy... there are a lot of potential variables and getting at cause and effect requires rigorous statistical practice....

Again, you are over-complicating things... This is not a scientific project, you cannot control everything in your life and all these situations may as well occur in your control group (baseline)! I am just suggesting to use your intuition as usual, but counting also on an objective measurement of success. By the way, most scientists in the last years have abandoned the "p-value dictatorship"!

I generally agree with your rules of thumb. I would just keep away from any kind of subjective measurements (like your 1-10 scores; intuition takes care of that) and try to keep it as simple as possible. The fact that it is not a perfect statistical design is irrelevant.

You are entitiled to your opinion, of course, but there is a difference between large and small organizations in sample size. Statistical fluctuations are larger for individuals and small organizations, and there is a human tendency to correlate those fluctuations with actions taken or not taken. Other, more general problems with basing actions on formal metrics can include:
* bias or subjectivity in the metric
* using an inappropriate metric
* equating correlation with causation
and so on.

Once again, I have never praised the need for complex statistical analysis of your projects or said that one should base actions on formal metrics. Just add an objective measurement of success to your current intuition-based analysis.

The common misconception in all these comments is that if you cannot perfectly use metrics, you shouldn't use it at all. As most things in life, there is no black and white here: come and enjoy the gray zone!

By the way, I assume your Michael Gerber is the "e-myth" guy I found on the web. I have never heard of him before. That's one of the problems with "appeals to authority" as an argument: it doesn't work if the other guy has never heard of your authority!

This is another misconception. Citing someone doesn't mean I am insulting your knowledge or that he is an expert in anything! It just means that these are not my ideas, but someone's else, and I am kindly giving you the reference so that you can consult the originals.

By the way, Michael Gerber has been cited on these forums lots of time and is a very nice read (despite some spiritual fluff). He defends that small businesses should operate since their start as big ones, using dynamic (innovation) standardized systems (orchestration). Metrics connects innovation to orchestration.

Here I respectfully disagree. Based on my own experience and the postings of others, I have come to believe that tweaking is essentially a subconscious response to failure in implementing the core habits of GTD. For example, I fail to do a weekly review, so I blame my tools.

Well, I respectfully dis-disagree. Everyone tweaks their systems because life changes and there is always place for improvements. David Allen wants to develop new collecting methods, make better use of mind maps or implement web-based solutions; Eric Mack is always improving his Lotus template, now using ActiveWords; Marc Orchant is amazingly discovering new ways of doing GTD, collecting web-based information (Onfolio), etc.

These people are not losers, failing to implement GTD! Quite the opposite, they are on the edge, defining what better works for you and me...

People differ widely. That kind of dedication to routine is frankly not in me nor does the nature of my work lend itself to it. I went through a period where I exported my completed todo's to DayNotez. I never looked at the exported tasks, and I stopped.

I agree with you here: people are really different and all my opinions should be taken with a grain of salt. I also had this problem with DayNotez, when I finally understood that it was part of a much bigger one: transforming information into knowledge.

Our old to-do's are generally just information, and if we don't convert them to knowledge, tracking is useless. Recently I came to believe that information is to knowledge what stuff is to actions, and it may overload us as well. To make things worse, Copernik, X1 and Lookout are equally useless, just as searching an engineering library doesn't make an engineer out of you. But this another subject for discussions...

Peace,
Eduardo

 

[severance1970]

Metrics are best used for iterative, industrial processes. Most personal projects don't scale in quite the same linear fashion. It's harder to extrapolate how long it will take to complete a project like buying a house based on the first week of activity than it is to extrapolate how many widgets a factory can deliver based on the first week of activity. Human factors are primary in buying a house: aesthetics, price negotiations, the time is takes for so-and-so to call back, escrow, etc., are all hard to gauge in advance. There are exceptions -- a novelist might know from experience that she can turn out 1000 words a day. Athletic progress is very amenable to metrics.

But either way, a posteri analysis of projects is generally supercilious. If I look at my list of phone calls and see one that I could've made but haven't, it's instantly obvious why: laziness, trepidation, failing to look at the list when I'm at a phone. No emprical data is going to tell me that. My weekly reviews are for identifying my projects and the next actions that will carry them forward. They're not for patting myself on the back for what I've done or flogging myself on the back for what I haven't. I don't have time for rituals.

Creating metrics, like Powerpoint presentations, usually degenerates into a busy-looking activity that could be categorized as "trout production": you get to feel secure staying in place while maintaining the illusion of forward motion.

 

[etcoutinho]

Re: Metrics

I am completely against metrics for posteriori analysis of projects. It simply doesn't work on a personal level.

I use metrics as a tracking tool of success because my perceptions can be deceiving sometimes. Replying to the first post, metrics is/was more important than successful outcome for me in many times. I suggest George Leonard's Mastery book for more on that.

The successful outcome of a project is essential to plan your actions and make sure you are on track. But in the real field, it is often difficult to refer to some higher level definition of success, and that is when metrics become useful. This is so powerful that it has been used in very critical situations, such as addiction recovery. While the higher level outcome is clear (getting rid of the addiction and staying health), one generally needs a day-to-day metrics (just for today, I will have zero of this in my body).

So I have incorporated this in my successful outcome definitions and always look for something measurable to help me on the field. Here goes some examples:
- Outcome: buy a nice house / Metrics: inspect at least 2 houses per week;
- Outcome: be healthy / Metrics: run for 1/2 hour at least 3x/week;
- Outcome: write a book / Metrics: get at least 1 hour of quality writing per day.

The common idea here is simple: although a lot can happen during your project, there are always some obvious measurable things that if not done will put your outcome at risk. But don't try to make it perfect and don't be too harsh on yourself on your weekly review (which is a kind of cleansing ritual, isn't it?).

Regards,
Eduardo

 

[severance1970]

Part of my reservation about metrics and benchmarks has to do with Parkinson's "Law": that work expands to fill the space allocated to it. Someone given an 8-hour day to do a 5-hour project will take 8 hours. That's where I philosophically disagree with McGhee's approach. That's why I found Allen's approach, in contrast to orthodox time management, so intuitively compelling. The stealth corollary to staggering actions on a project according to a contrived schedule is that you actually hold yourself back from completing actions that you could have completed, despite having a populated calendar that makes you look very disciplined. The idea is to complete the errand the next time you're out for errands, not "by the 12th." Next actions are meant to be done as soon possible, i.e. as soon as you're in the context to do them, negotiating which action takes priority within the given context.

Something is either a next action, or it's not. There's a difference between something that can be done now, though you choose to do it later (next actions on your action list), and sometime that depends on a situation which hasn't occured yet (hard landscape items on your calendar). Even if your deliverable is fairly contrived ("Inspect two houses this week"), ultimately it's going to boil down to some next action that's not ("Call Lance to schedule walkthrough of Elm St. house").

 

[kewms]

I think you have to be very careful in selecting the appropriate metric for a particular goal. "Write 1000 words per day" is a useful metric if your goal is to establish a habit of writing regularly. It is not necessarily a useful metric if your goal is to complete and publish a novel: novels also require revision, letters to potential publishers, etc.

I think metrics can be useful for goals that can be met by persistence: losing weight, learning a language or a martial art, conquering an addiction. In all of these cases, taking a few small steps every week will eventually get you there. Metrics can help remind you to take the small steps.

Though persistence can accomplish a lot, many goals are far too fuzzy for reasonable metrics. Metrics can help you finish a novel, but that doesn't mean it's any good.

Katherine

 

[etcoutinho]

I think metrics can be useful for goals that can be met by persistence: losing weight, learning a language or a martial art, conquering an addiction. In all of these cases, taking a few small steps every week will eventually get you there. Metrics can help remind you to take the small steps.

Though persistence can accomplish a lot, many goals are far too fuzzy for reasonable metrics. Metrics can help you finish a novel, but that doesn't mean it's any good.
Katherine

I agree with you almost 100%, metrics have really helped me in persistence-related projects. Nevertheless, it is on the fuzzy real world where they really shine.

I even make more use of metrics in fuzzy-projects, as a baseline, no BS, easy-to-assess tracking method of the critical success factors (as Brian Tracy's definition). In your write a book project, it is of course impossible to control or measure quality; using a 1000-word per day criteria is also useless (are they any good?). But I assure you that if you don't sit down to write, at least for a couple of hours per week, you will NOT finish your book, no matter its quality. So I'd use this as baseline metrics.

Choosing the right metrics is also essential. I generally look for only one, the most critical success factor. For example, in getting rid of an addiction one could try to track a lot of factors (e.g. hours per week of counseling, number of new "clean"relationships, etc). But the real critical factor is "I will have zero of this in my body today". And if this has worked for thousands or millions of people in such a critical, difficult-to-accomplish project, why shouldn't we give it a try?

Eduardo

 

[kewms]

[quote="etcoutinho]I even make more use of metrics in fuzzy-projects, as a baseline, no BS, easy-to-assess tracking method of the critical success factors (as Brian Tracy's definition). In your write a book project, it is of course impossible to control or measure quality; using a 1000-word per day criteria is also useless (are they any good?). But I assure you that if you don't sit down to write, at least for a couple of hours per week, you will NOT finish your book, no matter its quality. So I'd use this as baseline metrics.
[/quote]

I think we're quibbling about semantics but mostly agree.

Some things are amenable to metrics, some are not. I certainly agree that you can't finish a book without consistent effort over an extended period of time. That's a persistence-related task and very susceptible to metrics. However, quality and publishability are almost completely subjective. No suitable metrics exist.

Put another way, "complete a book" is a quantifiable task. "Complete a book that will win the Pulitzer" is not.

Katherine

 

[spring]

Project planning

This project planning technique appeals to me greatly. I'm using it right now to plan a novel that I'm writing. Despite the title of the article, this is good for planning more than just software: Painless Software Schedules

 

[etcoutinho]

Put another way, "complete a book" is a quantifiable task. "Complete a book that will win the Pulitzer" is not.

I am sorry to disagree with you and your example. It seemed to make sense at first, but then I realized that the real difficulty was not in quantifying how to win the Pullitzer, but in defining the next action. Generally, when you can define a next action, you can define something measurable, but this can be tricky when the outcome is out of your direct control.

In these cases I often use one of Brian Tracy's techniques. Basically, forget about winning the Pullitzer, and focus on becoming a person capable of winning it. Define this person and plan your transformation. What kind of education, habbits or relationships would you have? Then you would have quantifyable next actions to work with (e.g. hours of study in a writing course, participation in social groups for authors, etc).

The bottomline is that metrics is much more than just quantifying a task or project for me. It is something that lets me focus more on the journey than in the goal itself. I know it may sound like an heresy, but after doing GTD for some time I realize that life is more than just a succession of goals to be concluded. But don't get me wrong here, clear goals are essential to getting things done. It is just that I prefer to see them during my reviews, to focus my mind and define suitable next actions. When I am in the field, I generally use more some kind of metrics. For example, playing guitar is one of my areas of responsibility (hobby), and I always have an ongoing project for playing some difficult song or solo. Metrics (practice hours) lets me enjoy the journey, enjoy myself playing related scales, arpeggios and exercises and not get obsessed with some silly block. As I suggested before, George Leonard's book on mastery is a nice read if one is interested in the subject.

Finally, I would put metrics in the same category as mind maps. It is not an essential part of GTD, but may help a lot in completing some of your projects. Just give it a try when you are stuck with something.

Eduardo

 

[etcoutinho]

Re: Project planning

This project planning technique appeals to me greatly.

This is gold, Onfolio thanks you for the link... :wink:

I have never thought about it, but metrics can help one defines what is feasible and can be commited to, and what should be somedayed. I would just suggest ResultsManager instead of Excell for a GTDer.

Eduardo

 

[jhegener]

I think you have to be very careful in selecting the appropriate metric for a particular goal. "Write 1000 words per day" is a useful metric if your goal is to establish a habit of writing regularly. It is not necessarily a useful metric if your goal is to complete and publish a novel: novels also require revision, letters to potential publishers, etc.

The single most useful piece of advice that (I believe) applies to metrics is this: Use action quotas, not result quotas.

In other words, "I will make 50 cold calls today." Not, "I will close 5 sales today." We can control our actions, we can't necessarily control how things turn out, though.