Anything you can do I can do better . . . performance management and monitoring

Society – in the UK at least – seems obsessed these days with performance. And I’m not referring to the exploits of our athletes in Team GB during London 2012.

No, what I’m concerned about is the focus we seem to have on performance targets and monitoring. Now don’t get me wrong. I do agree that there has to be accountability for many of the things in which society invests. But there are some areas where performance monitoring and appraisal have been taken a little too far in my opinion. I have to be a little circumspect, I guess, since my elder daughter Hannah is a PhD psychologist specializing in industrial and organizational psychology and particularly in issues relating to performance.

In the public eye
Before the Olympics Games finally got under way, there was great concern that the company that had been contracted to provide security cover had not been able to recruit the necessary number of staff or get those that had been trained in time. Thankfully, all went quiet and appears to have gone smoothly as the Games commenced.

But the company in question, G4S, had contracted to provide all these people, yet only came clean a couple of weeks before the Games were due to open that they could not fulfil the terms of the contract. Now it’s not my role here to criticize how G4S managed its contract. What I do find hard to understand is how its failure to deliver was not picked up until so close to the Games. I would have expected the contract to have some very carefully defined targets and a set of milestones that had to be achieved by certain dates. These would have permitted adequate monitoring of contract progress. Maybe these were in place, but no-one bothered to monitor what was going on.

Now we have the furore about GCSE English exam results, and the moving of grade boundaries. Not only will this change in marking practice affect individual students, lower grades affect the targets which schools are expected to achieve in terms of passes at certain grades among students taking exams if the funding from the government is not to be adversely affected. Again, targets, targets, targets.

My own experiences with performance setting and monitoring have been concerned with two aspects. First, as with most employees these days I guess, I’ve had to undergo an annual performance appraisal. And second, as Director for Program Planning and Communications at IRRI between 2001 and 2010 I had responsibility for developing the institute’s Medium Term Plan and helping colleagues to define/refine their annual research targets, as well as respond to the increasingly idiotic and meaningless questions raised by the small-minded accountant hired by the CGIAR Secretariat in Washington, DC who hadn’t the first clue about scientific research (either basic, applied or for development) and was, in reality, the proverbial ‘bean counter’.

Performance appraisal
It was the late 1980s, and I was working at the University of Birmingham, as lecturer in plant biology. The Thatcher government had made any salary increase for academic staff contingent on the introduction of a performance appraisal system – something that was very new in academic circles. We had training courses – both for those supervising staff and for those being appraised. I have to admit I was dead set against this new fangled approach. It seemed to me that if you were found not to be performing as expected there were measures in place to help you do better. If, on the other hand, everything was going well, you might get a pat on the back, and that seemed about it.

I think I surprised myself when, after the first round of performance appraisal, I became a convert. I had found the whole exercise worthwhile and, following a complete reorganization in the School of Biological Sciences into four research groups (I was in the Plant Genetics group), had a better understanding of my niche in the School – and that it was appreciated by my head of group. It was also very useful to be able to have a frank and unconstrained discussion, one-on-one, with my head of group, Dr (later Professor) Mike Kearsey. From this experience, I became convinced that performance appraisal should be more about personal development rather than a means primarily to set remuneration policy and merit increases, although it surely plays a part.

So I was rather shocked when I moved to IRRI in 1991 to find a system of forced ranking, where local staff expected to be rated ‘excellent’ just for doing their job, simply because salary increases were tied to the outcome of the appraisal cycle. In fact during the 19 years I was at IRRI, I think I must have been through more than half a dozen different appraisal systems – and to my mind, none of them was particularly satisfactory. I was able to have some aspects of the development criteria I’d experienced at Birmingham brought into the IRRI system, however, and I think they were appreciated by staff at all levels.

But getting staff performance appraisal just right is a tricky issue, and I do not count myself an expert by any stretch of the imagination. But I think I can recognise a system that is just not delivering – either for the individual staff members or the organization.

Performance targets and monitoring
The days when a researcher could follow his or her scientific curiosity are long gone. Just ask anyone who has had to write a research proposal – for basic research, applied research, or research for development, and the problem of crystal ball gazing emerges once again. Scientists are often asked, as one of the criteria for evaluation, what the impact of their research is likely to be, 10, 20 or 50 years down the road. This is an impossible question for many.

But in the fields of research that I have been associated with for several decades, the success of any grant submission is the ability to clearly demonstrate what the outcomes and impact are expected to be, and to plot a pathway (through milestones) to achieving those. I don’t have much of a problem with that; after all this type of research is not done for its own sake, but has the ultimate aim of improving people’s livelihoods. But while a targets and monitoring scheme can be a framework to assess the benefit-cost of research investment, it had, in my opinion, become a millstone around the collective necks of the international agricultural research community, imposed from above by a group of donors whose staff (well some of them at least who were calling the shots) had little understanding of the nature, complexities and constraints of carrying out research for development, and often in rather challenging conditions.

Among the beefs I had with that accountant in DC were first, the ambiguity of the monitoring metrics – which allowed interpretation and therefore gaming of the system among research centers in the system (after all the ranking that performance monitoring brought about had a direct impact on the next year’s funding), and second, the complete lack of understanding that even though a research project had not met its targets to the letter,  there could have been nevertheless significant impact on the ground. It was the numbers that mattered. And I’m afraid I did, on more than one occasion, let my frustration with system get the better of me, and interact with the Secretariat folks in less than my usual courteous way.

I worry that research for development is increasingly being devised and carried out to a formula, and the performance targets and monitoring are only exacerbating the problem. As I said from the outset, I have no issues with performance assessment per se. But when these exercises take away significant valuable time from active researchers in order to feed into a bureaucratic system (for certain months of the year I was spending over 50% of my time responding to external performance monitoring and auditing requests and having to ask researchers to take time away from their work to meet the deadlines which were imposed on us) then the balance is wrong.

Since retiring I’ve fortunately not had to deal with these issues any more – and it was the increased bureaucracy of international agricultural research that finally decided me to retire. I’m sure this won’t be the end of it. The CGIAR has gone through a major reform and reorganization program, and I’m sure it will have to devise new (better, probably not less complicated?) performance monitoring schemes in order to justify the shape, feel, direction, and expense of spending several years navel gazing to move its agricultural research agenda forward.

I’ve never thought of myself as a cynic. Unfortunately in the last 18 months before I retired I felt myself developing a cynical outlook, and I didn’t like what I saw. Time to get out. I’m happier now.

Postscript (20 March 2014)
Just after the beginning of 2014 I received an email from an old friend, Sirkka Immonen, who works for the CGIAR Independent Evaluation Arrangement, based in Rome. Sirkka and a colleague had made an analysis of the CGIAR’s performance management system, which they had published in the journal Evaluation [1]. One of their compelling conclusions is ‘ . . . that the CGIAR’s PM [Performance Measurement] experiment failed against all the intended purposes. There were inherent difficulties in developing a set of annual indicators with high validity in reflecting the kind of performance that research institutions are expected to demonstrate, on outputs, outcomes and impacts. The system therefore was dominated by simpler observations related to quantitative records and institutional issues with unclear connections to performance of research organizations.’ The whole article is certainly worth a read. And after I had read it myself, I did feel somewhat vindicated for the stance that I had taken and the many concerns that I had raised while trying to implement what I then considered a flawed system in the research context.

[1] Immonen, S and LL Cooksy (2014). Using performance measurement to assess research: Lessons learned from the international agricultural research centres. Evaluation 20 (1), 96-114.
DOI: 10.1 1177/1356389013517444

TV habits . . .

I guess I watch a little too much TV, perhaps. Of one thing I am sure, however. I’m a news junkie, and the availability of a 24 hour news channel on the BBC is good news as far as I’m concerned. So if I miss the scheduled main bulletins at 1 pm and 6 pm, I can always catch up at any time in between.

Funnily enough, I quite like tuning into the BBC Parliament channel to see what our representatives are up to – or not, as the case may be. It’s incredible how empty the Chamber is sometimes. Must be soulless being a Member of Parliament on some days when you decide to make speech on a topic close to your heart and only a handful of colleagues (from all sides of the House) turn up to listen. And there are 650 MPs elected to parliament.

I do like adaptations of the classics – such as Jane Austen and Dickens, and there have been some wonderful series over the years. While we lived abroad we were able to catch up through DVD purchases. So we look forward each year to the autumn schedules and wonder what new adaptations will be presented for our delectation.

And I particularly enjoy history programs very much. We’ve just watched an excellent three-part series on The Churchills by David Starkey (a rather controversial historian), in which he ‘showed’ how Winston Churchill was destined to become a great wartime Prime Minister after having researched and published during the 1930s a magnum opus biography of his ancestor, John Churchill, Duke of Marlborough, one of the greatest generals in the first decade of the 18th century, under Queen Anne.

Other historians that have presented interesting series recently are Amanda Vickery on the Georgians, Cambridge classics scholar Mary Beard on the Romans, and Bettany Hughes, most recently presenting a 3-part series Divine Women and co-presenting Britain’s Secret Treasures (with, I have to say, increasing focus on her ample Rubensesque, Nigella Lawson-like bosom, all plunging necklines and profile shots); Lucy Worsley (Chief Curator at Historic Royal Palaces) rarely fails to please in the various series she has presented. Michael Wood has also made some good series, particularly that about Trojan War (as long ago as 1985 – how time flies).

But there are three BBC programs, all of which have enjoyed multiple series, that I’ve become somewhat ‘addicted’ to, and I’m not quite sure why.

Bargain Hunt
Presented by Chartered Auctioneer and Chartered Surveyor Tim Wonnacott, the program format is surprisingly simple.

Two teams, the Reds and the Blues (of two contestants each) are given £300 each (it was £600 in the early series) – and the support of an expert – to find three bargains at an antiques fair or similar within one hour, which will then be sold at auction (filmed at a later date at an auction house somewhere in the UK). The balance of what they pay for their items is given to the expert (often auctioneers themselves) to spend on a bonus buy. The winners are the team that makes the most profit at auction – whether or not they choose to use the bonus buy – or make the least loss. Any profit is kept by the contestants. A team that makes a profit on all three of their items gets the award of the Golden Gavel – actually a pin button with the BH logo.

What I really like is the last 20 minutes or so of each program when the various purchases are assessed by the auctioneer who is going to sell them. Did they really buy a bargain or a load of junk? It’s impressive the knowledge these auctioneers and the other experts have – but they don’t always come up trumps. It just depends on the ‘buzz’ at the auction on a particular day and, it seems, that some items do better in one part of the country than another (the program is filmed in different auction rooms all over the country). And it’s really interesting to watch the skill of the different auctioneers, and how they move items that don’t look to have any chance of making a profit whatsoever.

Dragons’ Den
This is presented in the UK version by broadcaster Evan Davies (one of the regular anchors of the BBC Radio 4 Today program each morning). Apparently Dragons’ Den began life in Japan, but now there are versions in a number of countries. I’ve seen the Irish and Canadian versions on the TV over here.

Budding entrepreneurs seeking investment in their company or an idea make a pitch to five venture capitalists who have made it, and who are (apparently) prepared to invest their own money in return for an equity stake. The pitch can only last about three minutes, during which time they have to convince the investors about the potential of their idea/product/invention.

And while a few are successful, it never ceases to amaze me how many budding entrepreneurs arrive in the Den ill-prepared. Obviously they are nervous, and some just blow it, and go to pieces. Quite a number do not have the necessary financial details and projections at their fingertips, nor a viable business plan. But there are two ‘mistakes’ that crop up time and again.

The first relates to intellectual property on inventions. The investors are unlikely to invest (maybe up to £200,000) in an invention that has not been protected. The award of a valid patent is sure to attract their attention. And the other mistake is to value their companies or product too high, by asking for an investment and yet unwilling to offer a sufficiently high equity stake. So asking for £100,000 and offering only 10% equity (thus valuing the company or product at £1 million) is sure to end up in some hardball negotiation, and the Dragons usually ask for a much higher stake, even as high as 49%.

But there have been some impressive investments. I have to say however that watching some entrepreneurs squirm under the intense (and sometimes quite hostile) grilling from the Dragons does make for compulsive TV.

QI
Now this is a different kettle of fish – much more light-hearted.

With resident quizmaster, polymath Stephen Fry and resident panelist, comedian and actor Alan Davies (who is joined each program by three other panelists) the show aims to throw some light on things that just might be Quite Interesting – thus QI. It was created by John Lloyd who wrote/produced/created a whole load of other shows on British TV, including Spitting Image and Blackadder, among others.

The panelists are most often other comedians (although Professor Brian Cox was a guest on a recent episode), who are asked about different topics; each show seems to have a particular theme. They are lulled into traps to provide an ‘obvious’ but quite often incorrect answer. Which then allows for much witty banter to-and-fro among the panelists.

The scoring system is a mystery, with Alan Davies most often coming last, with multiple minus points. In recent series the ‘nobody knows’ option has been introduced. In each program there is one question for which nobody knows the answer. Extra points are awarded for correctly identifying this question.

But it’s not about the points – it’s all about the free exchange of wit among Fry and his guests. Forty-five minutes or so of jovial entertainment.

Indiana Me . . . temples in the jungle

Over my career, I was very fortunate to be able to combine business trips with short visits to some of the world’s iconic heritage sites, or take time out for a quick vacation in the region without having to fly half way round the world.

When we lived in Peru, I visited Machu Picchu a couple of times; almost anywhere you travel in Peru you are immersed in archaeology. In Central America we had the opportunity to visit the pyramids of Tikal in Guatemala (and I hope to post photos from here once I have digitized the slides), and also those at Teotihuacan, just north of Mexico City. But one of the most impressive sites must surely be the huge temple complex at Angkor Wat in Cambodia. And we had the chance to visit there in December 2000.

On flights from Bangkok to Manila I have often overflown Angkor Wat, and even from 30,000 feet its extent looks truly impressive (even if there is also evidence over the whole countryside of the intense bombing that Cambodia suffered over several decades of war).

Angkor Wat is located in northwest Cambodia, near the town of Siem Reap, and near the Tonlé Sap, a huge seasonally flooded lake that acts as an overflow for the Mekong River during its flooding.

While we refer to Angkor Wat as a ‘site’, there are in fact many temples and other complexes covering a large area, apparently about 200 square kilometers. The beauty of the stone carvings, the iconic stone faces pointing in four directions, and the wonder of the forest reclaiming the various temples all add to the mystery of Angkor.

I’m not going to attempt to describe in detail what Angkor Wat has to offer, but a visit there has to last more than just one day. We stayed there for three nights, and although we were able to many of the sites and temples, there are plenty more mysteries to uncover, hidden by the jungle that has reclaimed its dominance over the area.

Some of the temple complexes, like the Angkor Wat site itself and Bayon are large with many beautiful buildings to explore, others are much smaller, comprising just a couple of buildings or so. Just click on these photos to open web albums (scanned images rather than original digital photos).

When we visited, it was possible to move freely around all the sites, look inside the temples, climb the towers – and really explore. While it was quite busy in some sites, we did manage to get away from the bulk of the tourists. But the increasing number of visitors to Angkor Wat is now giving rise to concerns, as this recent story on the BBC website discusses.

Settlements at Angkor Wat stretch back thousands of years, but much of what we see today was constructed from about the 11-12th centuries onwards, reaching its peak a couple of centuries later. I’ve read estimates of more than 1 million people were involved in building the temples. And for an ex-rice scientist like myself, that begs the question about the extent and productivity of rice agriculture that was required to keep this huge population fed.

In addition to the Angkor Wat and Bayon sites, these are the other sites you can ‘visit’:

Let me finish with a quote from the Introduction in Dawn Rooney’s guidebook to Angkor Wat [1]: The temples startle with their splendour and perfection, but beyond the emotions they evoke lie complex microcosms of a universe steeped in cosmology. While a thorough understanding may be out of reach for many, the monuments’ profound beauty touches everyone . . . 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[1] Rooney, D (1997). Angkor – an Introduction to the Temples. Passport Books, Lincolnwood (Chicago), Illinois 60646-1975.
ISBN: 0-8442-4766-9