Project Reviews, Assurance and Governance
A useful book. Found it a little repetitive and feel that it could have been a lot more compact. Would recommend ‘Rescue The Problem Project’ over this one although I will probably be referring to both in the future.
To me, with my original training in geophysics, this pattern looks a lot like an earthquake
A certain amount of failure is probably inevitable. After all, lack of failure is a sign of lack of ambition. The problem lies in the scale of the failure.
Andrew Doran
I once knew a business stakeholder who considered a ‘green’ project one that wasn’t stretched enough.
It’s easy to avoid project failures. Just don’t do any projects.
The team thinks they’re estimating, but the executive is conducting a negotiation.
A culture that says ‘come to me with solutions, not problems’ may mean that unsolved problems don’t get talked about.
Andrew Doran
Interesting given that I’ve always heard that phrase as positive advice.
When reviewers make it clear that they are interested primarily in clarity of information, and not in underlying agendas, their role as trusted facilitators can be extremely valuable.
APM (2006) recognizes five types of review:
It’s not possible (within bounded cost) to review every aspect of a project at the same time. A single review typically focuses on a small number of attributes
TYPES OF REVIEW
Andrew Doran
This section would make a fantastic mind map. Covers lots of different parameters.
One final point: sponsors frequently ask for a ‘general review of the whole project’. How do I do this without becoming unfocused? I generally find that such reviews can be reframed to focus on objectives and risks. They ask the questions: ‘What are the objectives? Does everyone understand them? What could prevent this project achieving them? What actions are people taking to manage these risks?’ As the review identifies specific risks, it may then drill into other attributes.
distinction between summative and formative evaluations.
Andrew Doran
For the primary benefit of the organisation or the primary benefit of the project. External or internal, respectively.
I’ve since sat in an embarrassing number of reviews like that. Sometimes one of the random suggestions hits the mark and the project team takes away an idea that will actually help their project. Sometimes we all felt better after having a grumble about our project sponsors. That boost to morale must be worth something? Mostly, however, these reviews were a waste of time.
Project managers often had strong opinions as to why elements of the organization’s standard processes didn’t apply to their project. Technologists were prepared to argue endlessly about what constitutes good design. On some reviews, we spent more time debating abstract notions of what constitutes ‘best practice’ than we spent looking at what the project was actually doing.
each review is just a small project.
These questions are mostly about clarifying the outputs from the review: what they are and why they’re needed. For any substantial review, I’d want to agree written terms of reference with the sponsor before I commence.
There are several dimensions to consider when planning interview coverage:
Andrew Doran
Useful checklist.
If significant new issues are coming to our attention after a dozen or more interviews, then that itself says something about the coherence of the project team and effectiveness of its internal communications.
We undertake reviews in order to achieve some outcome, so it is useful to start with outputs – the information that the review is intended to produce.
As we set up reviews, therefore, it pays to identify which stakeholders represent our primary audience and what information they need to inform their decisions and actions.
Standards are subdivided into baseline (that which is agreed for this project prior to the review) and reference models (which are agreed more widely, for example by organizational policy).
The best way I know to maintain focus during the review is to agree up front which reference models apply for this project, and how and why they have been varied. This is done either when setting the initial terms of reference with the review sponsor, or when having the initial meeting with the project leadership.
The best way I know to maintain focus during the review is to agree up front which reference models apply for this project, and how and why they have been varied.
Trying to assess a project against every applicable standard, for example, would probably take longer than the project.
My experience is that reviews add most value when they focus on no more than two or three criteria.
Where does this leave us when someone asks us to ‘just do a general review of the project’? As I noted earlier, this can generally be reframed to focus on two criteria: are the objectives well defined and understood, and are the primary risks to those objectives being managed appropriately?
Andrew Doran
This is exactly where I am right now.
Is this a one-off review or health check, or is it part of an ongoing programme of reviews or assurance? In the latter case, outputs from earlier reviews may influence our investigations, for example, to check that issues identified during those reviews have been addressed.
My rule of thumb is that every hour of interviewing generates about two hours of analysis and other activities.
Plan the event: Kerth (2001) and Derby and Larsen (2006) describe a range of exercises that can be used to kick off a retrospective, gather data about what happened during the project, and hence analyse and structure this data.
Andrew Doran
Useful reference.
it is worthwhile building a way to update and maintain reference models into our review process. Retrospectives are the ideal way to do this.
There are a number of places to begin when creating our initial checklists. Several organizations have published questions they use when conducting reviews. (OGC, 2004b, is a good example). Tables 5.1 and 5.2 (pages 97 and 100 may also be useful). Organizational policies and standards are another potential starting point.
There are a number of places to begin when creating our initial checklists. Several organizations have published questions they use when conducting reviews. (OGC, 2004b, is a good example). Tables 5.1 and 5.2 (pages 97 and 100 may also be useful). Organizational policies and standards are another potential starting point.
Tables 5.1 and 5.2 contain checklists I use when reviewing projects. These may provide useful starting points for your own reviews.
Organizations benefit from this learning in two ways. First, the general level of skills within the organization is raised. Second, knowledge of the organization’s projects and techniques is spread more widely. This provides added resilience in the face of staff turnover and increased capacity to divert resources between projects when necessary.
Many organizations frame reviews as a control mechanism, a way to ensure that their projects are under control. This chapter suggests an alternative and potentially more powerful framing: reviews as a way to enhance and accelerate organizational learning.
It’s hard to kill a project.
Many organizations have an asymmetric need for evidence. They may initiate projects on the basis of partial information and gut feel, but they will only reshape or kill projects when the evidence is very clear.
Most reviews probably start with documents. If we can use them to gain some understanding of the project’s context and history before we meet with the project team, we need waste less of their time as we get up to speed with the project. Documents can also help us identify which areas of the project we want to focus on, who we need to talk to in each area, and what questions we want to ask them.
Table 6.3 describes eight stages for planning and conducting an interview.
Andrew Doran
Useful.
While taking notes, it’s worth separating facts from interpretations. A common way to do this is to separate our notes into two columns.
Table 7.2 Agenda for project leadership briefing
Andrew Doran
Highlighted for reference.
sometimes the interviewee wants to rephrase things once they’ve seen them written down. These rephrasings may point at sensitive areas on the project.
When confirming notes, we’re checking the facts and not our interpretations. We may also choose to share our interpretations with the interviewee, but we should be clear that it’s the facts we want them to check: we have to own our interpretations and recommendations.
This in turn means that scheduling more than a dozen interviews over three days is unlikely to be effective.
Andrew Doran
This book repeats itself too much. I just read the same points only a few pages back.
As we frame our findings, we need to understand the type of actions our stakeholders can take. This means understanding what I call the ‘sphere of influence’ of our review sponsor and the other stakeholders we’re reporting to. We can then ensure that our review addresses the issues which are within their remit.
Governance isn’t management – it defines who manages what, then lets those people get on with the job. It’s also about a lot more than compliance. Compliance is the backward-looking part of governance that helps demonstrate that decisions were taken in accordance with regulations, policies and objectives. The forward-looking part of governance creates structures to help people make good decisions. Those good decisions are what add real value to the organization.
Each of the nine cells in this matrix captures the units and activities involved in making decisions of a given type and strategic scope. Thus for a given project, it helps me think about who is accountable for decisions
Andrew Doran
Useful model.
A review and assurance programme can deliver benefits in three broad areas: avoiding issues within individual projects; better decision making across the project portfolio; and better dissemination of skills and good practices across the organization.
One final point about benefits: some of the benefits may well come from a type of placebo effect. When projects are subject to regular independent reviews, people may set up their projects with a little more care in the first place.
Running a review programme without sufficient resources to do an adequate job risks creating a false sense of security: people may think their projects are being reviewed when they aren’t.
Reviewers need to be clear that they are a complement to other project management practices, not a replacement for them.
I’ve seen reviewers who started to doubt their own sanity: they could see clear signs of major problems on a project, yet the project manager, sponsor and everyone else they spoke to dismissed their views. They started to wonder if they were seeing things. Without appropriate support, this stress can derail a review team.
Reviewers need to put their own preferences on hold and focus on whether the project team, with its mix of skills and resources, can make the chosen approach work.
We managed the plan rather than the reality.
Great analogy.