Introduction

Assessments
Elements of an Overview Assessment
Monitoring
Evaluation
Research
Conclusions

References
Annex I: List of Basic Statistics for an Assessment
Annex II: A Proposed Instrument for Assessing Judicial Operations

Introduction:
Judicial reform is a new discipline, but not so new as to escape questions about its intellectual and practical value. In the past few years, external observers and long-term participants have begun a serious reexamination of its accomplishments on both dimensions. This process is not likely to end soon, but an early conclusion is that poor knowledge management constitutes a serious impediment both to disciplinary development and to its application in the resolution of real problems. Here critics have pointed to a series of issues, ranging from a tendency to design, and execute projects in a near informational vacuum, through an absence of systematic efforts to build and test theories and strategies, to a simple failure to draw on past experience or to consolidate and disseminate it adequately. The problems may be inevitable given the field’s dominance by practitioners and advocates more inclined to action than to contemplation. However, they are sufficiently serious to threaten its immediate utility and longer-term survival. For those convinced that there is something worth saving, improved knowledge management has thus become a priority.

The current study looks at a fundamental part of this undertaking, the means by which reform participants can improve the information base for their individual activities and convert it into knowledge about the reform process. Four key elements in that effort are assessments, monitoring, evaluation, and research. While discussed separately, they are clearly interrelated, both in the case of individual projects and in the overall effort to produce a collective knowledge base for the discipline as a whole. The emphasis on field interventions might seem illogical given practitioners’ bias toward action as opposed to reflection. However, this is where most knowledge continues to be generated and where its inadequate utilization poses the greatest costs. In the absence of a strong academic constituency with a vested interest in the discipline’s survival, the practitioners, for better or worse, still have the major responsibility.

This is in the nature of a lessons learned analysis. Many of those lessons are negative, but I don’t propose to spend much time on another issue, why things are as they are. Most of those points are obvious. Those interested in pursuing them are referred to essays, by the present and other authors, who have focused on perverse incentive systems, disciplinary biases, and a host of other contributing factors. At the risk of placing excessive faith in the powers of rationality, my assumption here is that an analysis of what we know about how we generate, collect, and use information might be sufficient to encourage improvements even in the face of organizational and political incentives for doing otherwise.

This incorporates a corollary assumption – the belief that knowledge-based action produces better results than stabs in the dark or uninformed good intentions. That still begs the question of how much you should know before you act and how much time and other resources should be invested in sheer knowledge building as opposed to working actual improvements. An emphasis on immediate action is embedded in most incentive systems, frequently voiced as the argument that we have enough studies and that what we have has not be useful (or used). While believing that more resources should be devoted to our knowledge base, I also would agree that we have made poor use of what we have. More with more would be better, but for the meantime more with the existing investment is a desirable and feasible goal.

The further development of this paper is very simple, looking at how the four activities have been conducted, identifying the strengths and weaknesses of the usual approaches, and deriving some lessons as to how they could be improved. Not discussed, but of equal importance are a series of subsequent steps involving the dissemination of the improved knowledge base, and the means by which the entire reform community can be encouraged to use it. It can be hoped, of course, that the availability of an improved product might provide additional incentives, and that those involved in its creation might also promote its use. However, more direct action will also be required, and more thought directed to the form it should take.