Sample surveys and their analysis
This note will be concerned with the analysis of chronic poverty based on existing large-scale (often nationwide) household survey data, which will generally have been conducted by Statistical Offices and other agencies with the expertise and competence to conduct such surveys effectively. Thus this note will not discuss the collection of survey data, which is a major issue in its own right. See Resources.
Traditionally many such surveys took the form of household budget surveys, which focused predominantly on collecting information on households’ incomes and expenditures. However, over the last fifteen years or so, large numbers of countries have collected multi-purpose household surveys that collect information on many different dimensions of the living conditions and characteristics of households and their members. These have developed as a result of various initiatives including the UN Household Survey Capability Project, the World Bank’s Living Standard Measurement Study (LSMS) project, and its Social Dimensions of Adjustment (SDA) initiative in sub-Saharan Africa. The recent World Bank volume edited by Grosh and Glewwe (2000) provides a detailed discussion of recent developments and issues in designing household surveys.
Here the focus is on the analysis of such data, for which two important issues need to be considered.
- It is important to take account of sample design in using results based on the survey to draw inferences for the population – often some groups of households are oversampled (have a higher probability than average of inclusion in the sample), while others are undersampled. This is achieved using sample weights.
- Analysis based on such data must always be preceded – and accompanied – by an assessment of the quality, strengths and weaknesses of the data. Though there are few scientific procedures for assessing the quality of quantitative data, most commonly used methods take the forms of either:
- Assessing the quality of the survey operation (covering aspects such as questionnaire design, supervision, training of enumerators, sampling etc.)
- Assessing the consistency of the data collected with other credible sources (e.g. age structure of household members compared with that in a recent population census)
- Judging the internal consistency of the data set, to the extent that the available information allows this to be judged (e.g. are levels of household incomes estimated broadly compatible with the estimated levels of household expenditures, or is there evidence of overestimation of one or underestimation of the other).
- The conclusions of such an analysis obviously have implications for the relative reliability of different types of data collected.
Introduction - Quantitative analysis of poverty
The quantitative analysis of poverty is particularly developed in terms of the monetary dimensions of poverty; poverty in this dimension is viewed as a situation of inadequate income or insufficient consumption relative to an appropriate norm or threshold (a poverty line). A number of standard points can be made in this context, among others:
- The standard of living (income or consumption) is most naturally and easily measured at the household level – meaning that it will not take account of intra-household variations.
- The income or consumption standard of living measure needs to take account of differences between households in their size and composition, and in the prices they face.
- The absolute approach to defining a poverty line views this as representing a situation of minimum subsistence needs.
- Various summary poverty indices can summarise its main characteristics (incidence, depth etc.); the P? class of poverty indices proposed by Foster, Greer and Thorbecke (1984) are very widely used.
Absolute poverty lines are widely used in low income developing countries, but it should be recognised that establishing a poverty line is not a precise scientific exercise; poverty lines are usually guided by considerations of nutrient (often calorie) requirements. Poverty is related to the concept of inequality, though distinct from it in that it focuses only on those with inadequate living standards; poverty and inequality are more closely related though if poverty is defined in relative terms (i.e. with reference to the population as a whole).
Quantitative analysis of poverty and deprivation though is not limited to monetary dimensions of living standards (e.g. the Human Poverty Index at the national level). Household surveys typically collect information on other important dimensions of living standards including education, health and nutrition, and this is amenable to quantitative analysis. Such analysis is probably most developed in terms of the use of child anthropometric data in studying malnutrition, a key dimension of deprivation. An important advantage of many of these measures of non-monetary dimensions of living standards is that they are frequently measured at the individual level, overcoming one limitation of commonly used monetary measures. For some dimensions though it may be difficult to establish a concept of deprivation analogous to a poverty line.
Other dimensions of poverty are less amenable to quantitative analysis, in particular using the types of data sets discussed here. The World Development Report 2000/01 stresses the importance of powerlessness and vulnerability as key dimensions of poverty. Standardised surveys are unlikely to be the best tools to collect information on individual and household experiences of powerlessness. And though data from such have sometimes been used in analysing vulnerability, the dimensions of vulnerability they typically capture (often fluctuations in consumption levels) only represent one aspect of a much more complex issue (see the discussion in the World Development Report 2000/01).
Poverty as a dynamic phenomenon
Poverty is fundamentally a dynamic phenomenon, and how it evolves over time for countries, groups within countries and individual households are key dimensions. Such dynamics are fundamental in considering chronic poverty, generally based on a notion of persistence. To consider the changing nature of poverty will require either:
- observations of living conditions and poverty at several points in time; or
- measures of living conditions that capture dynamic aspects even if they are only measured at one point in time – e.g. retrospective questions.
Collecting information based on one-off visits will not generally achieve the former (e.g. measuring income); but often will collect the latter (e.g. finding out about past school attendance or attainments). Repeating the exercise at a later date can achieve the former if the data collected (e.g. to measure income or consumption) is consistent.
Collecting retrospective data is a complex exercise (though see toolbox note on life histories), and it is especially difficult to collect quantitative data covering a long period of recall in this manner [ add reference ]. Most quantitative analysis of poverty dynamics has been based on repeated household surveys. These can take two forms:
- Panel data, where some or all of those surveyed in the first round are surveyed again in subsequent rounds;
- Repeated cross sections, where a new sample is selected each time.
Approach (i) enables the different experience of each individual or household included in the panel to be considered, which can provide information on the factors behind the different experiences of different individuals or groups. Approach (ii) enables an examination of the differing experiences of different groups within the country, where these groups can be defined in different ways (e.g. by geographic residence, by main economic activity). It enables a comparison between the different groups identified, focusing necessarily on the average experience of each group. But the fact that groups can be defined in many different ways enables a detailed picture to be drawn of different experiences of different groups.
Each approach has its advantages and disadvantages. The great strength of panel data is that they enable household specific factors to be identified and taken account of, factors that are lost in the averaging required in analysing repeated cross sections. The household level focus means that panel data does enable distinctions to be drawn between the chronic and transient poor. However, put precisely because it does not involve averaging across different households, household level comparisons within a panel will be more affected by random measurement error than comparisons between group-level averages from repeated cross sections. Measurement error issues are likely to be particularly acute when the analysis is based on already difficult to measure data such as income or consumption; it is difficult to measure the level of such variables accurately but still more difficult to measure accurately their changes between two points in time. Similar points though apply to other data, such as anthropometrics.
An important point about either approach is that the time periods over which dynamics are considered are usually of necessity quite short; this means for example that most panel data is not suitable for addressing life cycle aspects of poverty dynamics at the individual household level. Information on life cycle experiences of individual households will generally be more effectively captured using retrospective techniques with a qualitative focus – see the toolbox note on life histories.
Finally note that some important dynamic dimensions of poverty, notably vulnerability, may be different to capture adequately using a quantitative approach; qualitative approaches may provide richer information on many important dimensions of vulnerability.
Measuring and studying chronic poverty based on panel data
Chronic poverty can be thought of as persistent deprivation of capabilities. There are many dimensions to this, but much of the analysis to date has focused on monetary dimensions of poverty, using panel data sets. In this line of work, two distinct approaches are generally adopted in defining chronic poverty (Yaqub, 2000):
- identifying the chronically poor based on the number or length of spells of poverty they experience (e.g. Baulch and McCulloch, 1998) – so that all poor households are classified chronically poor or not (the latter generally referred to as the transient poor); or
- defining for each household transient and chronic components to poverty (e.g. Jalan and Ravallion, 1998). The transient component of poverty is that due to variability in consumption levels, while the chronic component summarises what the poverty level would be if consumption did not vary about its mean value.
The first approach appears more intuitive, though suffers from the disadvantage that many different criteria might be applied: for example, how many spells does a household have to be poor to be identified as in chronic poverty? In any case the answer will depend on the number of time periods for which data are available. More generally the range of different approaches makes it difficult to compare estimates of chronic poverty across studies that have adopted different approaches – in the way that for example dollar per day poverty levels are compared across countries.
Having distinguished chronic and transient poverty, two key issues of prospective policy relevance that can be addressed using survey data are:
- Examining the characteristics of chronic poverty (e.g. Jalan and Ravallion, 1999, 2000; Gaiha and Deolalikar, 1993)
- Examining the factors associated with poverty transitions – movements into and out of poverty, or those that fail to escape poverty (e.g. Bane and Ellwood, 1986; Stevens, 1994; Baulch and McCulloch, 1999). In the UK CMPO at the University of Bristol are currently analysing issues of poverty dynamics based on the British Household Panel Survey. (Also make reference to IFS).
Most quantitative analysis of chronic poverty has been conducted based on consumption or income based standard of living measures, however similar concepts can also be applied to studying malnutrition based on child anthropometric data – for which a clear concept of deprivation can be defined.
Analysing dynamic poverty issues based on repeated cross-sectional data?
Analysis of a time series of cross sections is based on the principle of following groups of people from one survey to another, where the focus is on average or aggregate characteristics of these groups. Deaton (1997) discusses the types of issues that can be addressed using a time series of cross sections.
See Report on CPRC Workshop on Panel Surveys and Life History Methods (eds. Baulch and Scott, 2006) (pdf file, opens in new window).