Popular Posts

Caveat Emptor

The opinions expressed on this page are mine alone. Any similarities to the views of my employer are completely coincidental.

Thursday, 11 April 2013

The Great British Class Fiasco

It's been a bad week for my blood pressure. I'm trying manfully to ignore the hooplah arising from MT shuffling off her mortal coil, but it's more difficult to pretend that the BBC's Great British Class Survey  (GBCS) didn't take place. In fact I should be feeling very guilty. Back in 2011 I promised that I would write something about it and didn't. Well now I will, but before I get going I'll  reveal to you  how I fared when I took the test.  In fact, in the interest of science, I've taken it twice (test-retest reliability is a good thing to establish) first in 2011 and then just a couple of days ago.  I can confirm that the test (at least in my case) has high reliability (my students will - if they've been listening to my lectures - wish to point out that reliability does not imply validity). On both occasions I scored in the high 90s  (out of 100) on economic capital, and the low 20s on both social and cultural capital. I'll come back to this later, but  on grounds of general ontological security I feel relieved to know that I'm roughly the same man I was 2 years ago. I also took a punt on the BBC's  class calculator and I can proudly announce that it is official: I am a member of the Elite (despite appearing to have the social capital of a Trappist monk and the cultural capital of a Neanderthal). Well, I hope you are listening while I address you de haut en bas. I expect the knighthood is in the post.
Now to more serious matters. I want to ignore for the most part all of the media circus surrounding the story and do a deeply unfashionable and earnest thing: concentrate on the science. Behind all the fun and games is an article in the journal Sociology  with the  title: A New Model of Social Class: Findings from the BBC's Great British Class Survey Experiment (the title is actually slightly misleading because there are in fact no experimental findings  - all the data is purely "observational"). Normally it would be behind a pay-wall but, for reasons that one can only speculate about, it is, at the time of writing,  costless to view. This article, which has almost as many authors as the cast of a Cecil B. DeMille movie, is  the source of the media frenzy. To keep things simple I'll refer to it as Savage et al. with the understanding that the unnamed others are also entitled to a fair share of the praise and the brickbats. We have to assume that it is a serious article, it is after all published in a peer reviewed official journal of the British Sociological Association. I'm going to do it the courtesy of taking it seriously  in the belief that the authors would not wish for anything less.
Let's start with the praise. That can be disposed of pretty quickly. I think Savage et al. deserve a pat on the back for having the bright idea of getting the BBC to help them collect data. It was worth a try even if the results in terms of quality were utterly predictable. Of course the idea of using a mass media organization to collect data for scientific work is far from new. As far back as 1953/4 the BBC broadcast a programme called Science in the Making in which viewers were invited to participate in  live TV experiments and fill in  short questionnaires  printed in the Radio Times. The data from at least one of  these programmes, on right and left handedness, has survived and been used in academic papers.
The GBCS is, in effect, nothing more than a self-completion questionnaire administered over the internet. By their very nature self-completion questionnaires have to be pretty blunt in what they ask (at least if they are  cheap). Respondents are left to their own devices to make what they can of the questions. Their great virtue is that you can collect large amounts of data, but this comes at the cost of losing control over who is actually providing the answers. All a respondent needs is a valid email address. Most people I know have more than one. As I let on above, I've contributed two cases. Somehow I doubt that most of the participants will have been enthusiastic about participating more than once, but strictly speaking there is no way to know that. 
Let's stir in another ingredient that Savage et al. (to their credit) don't try to hide: there is massive self-selection bias in the GBCS. Surprisingly they demonstrate this with reference to the old, bad, inadequate NS-SEC categories that they want to distance themselves from. Managers and professionals are vastly over-represented whilst what one might speak of as the working-class are vastly under-represented. There is only a fifth as many routine  and technical craft workers in the GBCS sample as there should be and only a third as many routine workers. With this level of sampling bias self-selection on unobservables becomes a serious problem. In short even after applying post-stratification weights the working class respondents, for instance, are likely to be atypical members of the working class (after all they have spontaneous chosen to fill in a BBC survey!) and can't give us unbiased information about the missing members of the target population. I want to be scrupulously fair: Savage et al. "fess up" and then pull off their master stroke. Despite having over 160,000 responses to their web survey almost all of the information from which they derive their empirical claims comes from a "nationally representative" quota sample of about a thousand respondents. Let me say that again, just in case you missed it: despite all the hullabaloo about the massive web based survey, almost everything the authors purport to find in their article is actually based on the flimsy basis of  a quota sample n=1026. Without this their web based data is essentially useless. Given that the quota sample  is the bedrock of the whole enterprise we might reasonably expect to be told something substantial about it. What quota controls were applied? Were they interlocking? What was the target sample? Which weights were applied and where did they come from? What was the mode of administration? Look carefully at the article: you won't find answers to any of these questions. Instead all we get is some vague hand waving (pp 6) to the effect that both GfK and the authors are satisfied that "the demographics are nationally representative". I'm not going to beat them up about the quota sample too much, except to point out one fairly obvious thing. With a probability sample one has a coherent set of procedures for making sample to population inferences including, most importantly, inferences about uncertainty: with a quota sample one has none, unless one can make a convincing argument that one has specified the "correct" data generating model. I doubt that the authors will wish to claim that they do the latter - it would, I think, be inconsistent with  their general philosophy of data analysis which seems, to my eye, to favour more "exploratory" approaches. So Savage et al. have a problem. They have a mountain of  highly self-selected data sitting on top of a mole hill of (slightly) better quality data. Ultimately they want to use the latter to make some sensible calibration of the former, but they have to do this without any sense of the uncertainty with which the calibration  reference parameters have been estimated.
To make things concrete assume a simple random sample  with n= 1000 and a characteristic that is found to be distributed 50/50 in the sample. Using the normal approximation to the binomial distribution gives us a 95% confidence interval for the population proportion of roughly 0.47-0.53. Now assume a characteristic that is distributed 6/94 in the sample ie roughly the estimated size of the "elite" class identified by Savage et al. The equivalent calculation gives a confidence interval of roughly 0.05-0.08. In practice these numbers are highly optimistic (they take no account of any source of uncertainty other than a highly specific type of sampling error ie measurement error, model uncertainty and non-sampling sources of uncertainty are ignored) and are but a rough and ready lower bound. Realistic uncertainty estimates will widen the intervals in some cases to a degree that is likely to be  substantively important.
All of this sounds pretty daunting, but Savage et al. are in an even worse place. They don't have any coherent basis for making  uncertainty estimates at all and, in fact, don't make any. Ignorance is truly bliss for some, but it doesn't make good science. The only way out would be to undertake a what if? experiment. In other words pretend that their quota sample was  in fact generated by some known probabilistic mechanism and examine the implications of a range of plausible uncertainty estimates for key parameters on a hypothetical basis. This is really the only sensible way to even half convincingly calibrate the web survey data. I'd settle for that as making the best of a bad job.
That's enough, for the moment,  about the data. I want now to turn to a more fundamental issue: how to think about concepts and typologies. Let's go back to Max Weber not because he is the fount of all truth and wisdom but because he had some rather specific and useful things to say about the matter. Concepts - like for instance social class - are abstract entities in the mind. We use them to selectively organize experience (on this point I could go back to Kant). We are free to organize experience on any basis we like and with any conceptual scheme we like, but there are consequences. Some conceptual schemes are of more practical use than others in the sense that they, as Linnaeus put it, "cleave nature at its joints". Typologies or categorization schemes are closely related to concepts, they are in a sense a step towards making the abstract idea more concrete. There can be as many typologies - and this is Weber's point - as there are points of view. This implies that coherent typologies are made for a purpose. Not everyone need have the same purpose and therefore emphasize the same things, but a purpose they must have. Normally that purpose is given by the part that a concept (and its associated typology) have in a theory that is meant to explain something and naturally this entails spelling out clearly what it is you want your concept to play a part in explaining. A typology is a practical tool that is to be used for a purpose  and is not something that has any intrinsic value in and of itself. Inventing a typology for its own sake is a pretty pointless exercise.
So what is Savage et al.'s purpose? What do they expect their class typology to explain? Read the article as carefully as you like but you won't find an answer. There is a clue however in a pronouncement made by the lead author in a Guardian article:
"Let me be blunt. The concept of class matters, because we need a way of connecting accentuating economic inequalities to social and cultural differences which permeate our society. Rather than seeing our lifestyles and social networks as somehow separate from economic inequalities, there are overlaps that can work together to produce social advantage and disadvantage".
Blunt is definitely the right word here. Lifestyles, social networks, cultural differences and even economic inequalities are exactly what the Savage et al. typology cannot have any role in explaining, for the simple reason that all of these things are built into the definition of the typology. Whether these things "overlap" or not is completely besides the point and does not constitute a good reason for chucking them into the same food blender and then calling the mush that comes out of the other end "social class". If I want to examine the variation in the strength and depth of a population's social network ties - a perfectly reasonable thing to want to do - a tool that already is partly defined in terms of  the explanandum - will not help me to discover any new knowledge. Quare Opium facit dormire: ... Quia est in eo Virtus dormitiva.
 Nobody, it is true, has exclusive rights to the use of a word (or two). If Savage et al. wish to appropriate "social class" and fill it with their own content, so be it. But we are perfectly entitled to ask what precisely has been achieved by such an appropriation? Let's take a few steps back and consider one of the alternative usages.
Goldthorpe's most recent usage refers to aspects of the employment relationship - output/input monitoring difficulty and human asset specificity. I've played a small part in empirically examining the correspondence between these things and the occupational categories that form part of the NS-SEC. In Market, Class and Employment we go so far as to argue that in principle there is no reason why indicators of these things should not be measured at the level of the individual job and that if they were, then, in a sense, the categories of the NS-SEC would be redundant. In practice, of course, this is highly unrealistic. No general purpose survey is going to collect the detailed information required for accurate measurement of the real variables of interest and thus it makes perfect sense to chunk the world of employment up into occupations and then aggregate these into "classes" as long as we have evidence that these classes serve as adequate proxies for the underlying variables that practicalities dictate we can't measure directly. This is all really a matter of pragmatics disciplined by evidence and it would, of course, be absurd to attempt to "explain" say monitoring difficulty in terms of NS-SEC categories.
Savage et al. could say in reply that they were doing something very similar. Goldthorpe defines the content of class in terms of ideas drawn from Williamson's institutional economics, whilst they take their inspiration  from across La Manche. For monitoring difficulty and asset specificity substitute economic capital, cultural capital and social capital and who is to say that this is not acceptable? Fair enough, we can take inspiration from wherever we like, but we then have to live with the consequences. The first thing we might think about is what is to be gained by throwing indicators of the three "capitals" into the blender. After all given the conventional definitions of these things we pretty much know what they refer to. Why muddy the waters by then throwing away information about variability along these three dimensions? -  the inevitable consequence of any process  that combines them to form a typology with a limited number of categories. As I often have occasion to say to my students: no, class is not income or education, we already have a good understanding of what the latter words mean, what in the world do we gain by calling them something else?
Goldthorpe can logically justify the "classification" of the underlying dimensions by saying, I'm creating a tool which can be used in situations and with data where there are no direct measures of the things I'm interested in. By well validated groupings of occupations I can capture a substantial part of what I mean by class - ie the employment relations relevant aspects of jobs. This move though is not open to Savage et al. The logic of their approach impels them to reject the pragmatics of  occupational proxies and so makes it incumbent on them to provide a clear answer to the question: what exactly is gained from a data reduction exercise that reduces variation along three conceptual dimensions to a small set of categories?
Let's return to the theme of having a reason for creating categories. Goldthorpe had a clear reason for creating his class categories. He wanted to describe patterns of mass social class mobility in Britain and therefore he made distinctions that were sensible given that purpose. When that purpose changed - for example to describing cross-national patterns of mass social class mobility - further distinctions became necessary (farmers are numerically not that important in Britain but they are and were very important in France and Poland). Hence the evolution from the original class schema to the EGP schema and its variants (which Savage et al. completely confuse pp 3). Savage et al. perpetuate some highly fanciful ideas about the rationale behind the scheme. For example they tell us (pp 4) that:
"..it is increasingly apparent that a major appeal of the schema lay as a pragmatic means of placing individuals into social classes using standard nationally representative surveys with a moderate sample size. This is especially important since preferred forms of categorical data analysis such as log-linear modelling required reasonable cell sizes. It is for these pragmatic reasons, for instance, that an ‘elite’ was not distinguished (see Penn, 1981) within the Goldthorpe class schema."
This may be an accurate representation of what Penn believed but  the attribution of a kind of technological determinism - log-linear modelling can't cope with small categories - as the primary reason for not distinguishing an elite is completely implausible. Recall, Goldthorpe was interested in mass mobility. The elite are irrelevant to his purposes and in any case in a sample of 10,000 men - the 1972 Oxford Mobility Survey - there would be so few of them sampled that any means of drawing highly accurate inferences about them - log-linear or otherwise - would be doomed to failure. There is a lesson to be learned here, if Savage et al. would care to learn it. It really helps to clarify your thinking when you have a point: what, ultimately, do you want to use your categories to explain?
It's time to move down the ladder of abstraction away from the big conceptual strategic issues and focus on issues of detail. Let's start with a quotation. In the course of discussing Goldthorpe's class schema  Savage et al. say (pp 4):
"...the schema has been shown to be of less use in explicating the wider cultural and social activities and identities (see generally Devine, 1998; Savage,2000), which do not appear to be closely linked to people’s class position, as defined by the Goldthorpe class schema..."
This is an interesting claim, but I wonder how it sits with the evidence they  set before our eyes. To make my point I have to reproduce Savage et al.'s Figures 1 and 2 (pp 9-10) and appeal to the principle of "fair usage". Figure 1 is a bi-plot derived from a multiple correspondence analysis of a whole bunch of categorical measures of the extent to which respondents in the small quota sample take part in various, broadly defined, cultural activities.

For those that care, a multiple correspondence analysis is just a simple correspondence analysis of a so called indicator matrix  with respondents in the rows and dummy variable indicators of the responses chosen in the columns. In essence it is a data reduction technique which throws away vast amounts of information about higher order interactions between the variables.
Personally I find this a horrible graph created by people that obviously have little sympathy for the idea that the purposes of a graph is to convey a message to the reader as simply and clearly as possible - though to be fair, the pukka followers of Bourdieu  do not set great store by clarity. Be that as it may, what do we learn? My first thought after puzzling over this for several minutes was that slightly greater clarity could have been achieved by a 45 degree rotation of all of the points in a clockwise direction, but nevertheless we are told that if we follow a line from NW to SE we can discern a dimension that puts high-brow cultural taste at the NW pole (people visit stately homes, go to the theatre, museums and galleries, dislike fast food etc) and the Royle family in the SE corner. Strange thinks I. If I look at Figure 2 which superimposes so called "supplementary points" onto Figure 1 what do I see?

What I see is that the categories of the old, bad, inadequate NS-SECs which Savage et al. believe  "are not closely linked" to "cultural and social activities and identities" seem to fall from top to bottom roughly in a NW to SE fashion. What this means is that they are indeed predictive of the first cultural dimension of Figure 1. Personally I don't find this particularly surprising, but it beggars belief that Savage et al. didn't notice it. Did they really not see this or do they just not give a damn that their own evidence directly contradicts what they write? But what of the second dimension in Figure 1. This is apparently a great discovery and given the impressive label "emerging cultural capital" though in what sense it is emerging and what it is emerging from is never vouchsafed. This dimension runs from SW to NE and once you grasp the labels it is obvious that it distinguishes the sorts of things that young people like doing from the sorts of things that older people dislike or don't do.
This interpretation is confirmed by inspection of Figure 2. Fine. I don't doubt that younger people enjoy things that older people don't. My 6 year old has a great passion for moshi monsters that I don't share and you won't be able to get my mother-in-law down to the local Indian for a biryani,  but what has that got to do with social class? Think about it further.  Do people's tastes and activities change as they get older? Yes, they do. I haven't been down the gym or played five-aside football for a very long time nor have I recently been to see Coventry City  play (all things I did when I was younger and more carefree). Do people's tastes depend on when they were born? Yes, I believe that is true too. My parents were well into their twenties and thirties when the sixties happened. They never really developed much of a taste for pop-music - Jim Reeves and Frank Ifield were more to my mum's taste. So what dimension 2 actually represents is a confounding of life-cycle and cohort effects. If we  are to take it seriously as a measure of class we would be forced to believe that class can change simply as a function of getting older and being too stiff to get your football boots on  or remains fixed because of the birth cohort you happen to be a member of. Both of these conclusions would be bizarre. Do Savage et al. really hold to them? I'm inclined to believe that they would deny they did, but they would seem to be implied by the logic of what they do. I'd be interested to know how they square this particular circle.

On the back of the multiple correspondence analysis Savage et al. construct two summated ratings scales. The first purports to measure the presence or absence of highbrow cultural capital the second emerging cultural capital. One wonders why they went through all the palaver. They could have just done a simple principal components analysis on all their ratings scales and saved themselves a lot of trouble. It might even have helped them get a clearer picture of their data for they could have given the response categories plausible integer scores rather than having to work with sparse data and categorical measures. Of course they would probably have been summarily drummed out of the Bourdieu club for such a heresy and they would have  denied themselves the chance of using the elegant obfuscatory term "modalities" which in plain Anglo-Saxon means "the levels of a categorical variable". It also allows them to avoid any embarrassing questions about reliability, alpha coefficients and so forth. It really would have been nice to know how much variance was shared among the indicators allocated to each of the dimensions. In the rarefied world of French influenced sociology such trivial matters are clearly for the under-labourers rather than the Olympians that boldly tread where none have gone before.
This seems like an appropriate time to go back to the profile of scores I received on the three capital dimensions, for it raises some basic questions about the validity of the measurement instruments themselves. If you recall, I revealed that I was high on economic capital, but  shamefully low on cultural and social capital. I must say that I was puzzled by this. OK, I exaggerate; most middle aged academics that live in a dual earning household in the South-East of England and work in a Russell group university are probably doing pretty well in income terms. I know enough of the facts about the earnings and income distribution to have a realistic idea of where my household sits in the pecking order. This is why I cringe when I occasionally hear colleagues moaning about how badly they are paid. I also have a house in the South-East with just a small mortgage on it. That makes my housing wealth look pretty healthy compared to the national average, though note gentle reader, the GBCS does not ask respondents to subtract mortgage debt from the estimated value of their property. 
What really threw me was when I tried to answer the question about my savings. Though my liquid assets are meager the GBCS instructed me to include the value of my pension. This is a tricky calculation, and I doubt that most people bothered to attempt it. To give even a moderately sensible answer you need to know what the cash value of your pension will be at your retirement age, estimate how many years you are going to live after retirement and deflate the sum by some appropriate amount to get the present value. If you do all this and are optimistic about your life chances this should augment the current net worth of a middle aged person with a nice final salary occupational pension by a considerable amount. Let’s just hope I live long enough to see any of it. 
It seems to me that there is plenty of scope for people to give silly answers to the savings questions. What’s more it occurs to me  when I look at Savage et al.’s Tables 6 and 7 (pp 12-13) that one of the things that may produce the apparent distinction they discover between the “elite” and the “established middle class” is that the former are the people who bothered to make a sensible calculation about their pension. Take a look at the evidence in Table 6. Whereas the “elite” have salaries that are on average less than twice as high as those of the “established middle class” their savings are more than five times as large. They are also more than 10 years older (Table 7) and some of them may well actually be retired (the average age is 57) and thus in a position to know what their pension is actually worth. Another thing to notice is the average value of a member of the elite’s house. From Table 6 we learn this is £325,000. Immediately I understand why in one of the beautiful grey-scale maps that adorn the closing pages of the article a large share of the “elite” live in London and the South-East. Living in a 3 bedroom terraced house in a region with high housing  prices (as I do)  would propel you quite a long way towards “elite” status even if you  were up to your neck in mortgage debt.
What then of my cultural capital? In my vainer moments I think I’m actually quite a cultured bloke. I read a lot, in various genres, classic novels, contemporary novels, history, philosophy, popular science, current affairs even occasionally a bit of sociology. I watch a fair number of art house films on DVD. I enjoy listening to Radio 4, jazz, classical, opera, rock, folk, world and even country music. I live in a bilingual household and I’ve spent extended periods of time living outside of the UK. By my reckoning that is a lot of cultural capital and we haven’t even got on to my educational qualifications yet. But according to the GBCS I need to broaden my horizons a bit and it isn’t just a matter of listening to a bit of hip-hop, techno or heavy metal. Actually what I need to do is get out more. I don’t go to the gym or watch live sports events at all, and for the past 6 years I’ve rarely gone to a theatre, opera, art gallery, stately-home, museum or restaurant. When I do go to the latter it tends to be either MacDonalds or Pizza Express. Can you guess why? I’m sure you can. When you have young children your life changes. If you eat out at all you eat where they want to eat and you avoid going to places that they find boring. The average 6 year old doesn’t like art-galleries, operas, stately-homes or museums. If you are a reasonable parent, in your spare time you do stuff that your kids like. Before I became a father I did all of those high-brow things and now, for the moment, I don’t. Did I just lose all of my cultural capital (aka cultural resources) overnight? It would be a funny kind of capital if that would be the case. I still, I think, have the capacity to enjoy high-brow activities and it’s an odd operationalization that makes the measurement dependent on life-cycle stage. What I conclude from this is that with respect to cultural capital the GBCS has a bit of a validity issue.
Moving right on to social capital, this is supposed to be about the amount of contact that you have with other people and the social range of those contacts. It’s a perfectly respectable idea, but I don’t see that it has any straightforward connection to the idea of social class. If you are a disadvantaged person I can see that having a broader rather than a narrower social network may bring you some advantages. After all if you are a domestic-cleaner  and become unemployed and you know a merchant banker they might employ you to clean their house or look after their kids. It’s not obvious that if you are a merchant banker the crude economic advantage of having a socially wider rather than a socially narrower set of acquaintances is going to be that great. After all if you don’t know someone who would love to look after your kids  or clean your house you can always find somebody from an agency or ask your next-door neighbour who they use.
In practice Savage et al., while paying lip service to the notion of network diversity, just ignore it and actually only use a measure of the average occupational status score of the respondent’s social contacts and a count of what they call the “mean number of social contacts reported”. The latter is actually inaccurate. It is in fact just a count of the affirmative answers to a sequence of questions that ask: do you know socially anyone who is a X? where X is any one of 34 different occupations. So someone who knows a large number of stockbrokers and nobody else would get a score of 1, as would somebody who has but a single friend in the world. This is not a very sensible way to estimate the size of an ego-centric network. 
There is actually a very simple way to get an idea of the relative size of the ego-centric networks in a population. You give people a list of names taken from the page of a telephone directory or suchlike and you ask them how many people they know who are called Smith, Jones, Johnson etc. Bizarre, but probably inconsequential, is what Savage et al. have to say about the measure of social status they use. This is the so called CAMSIS scale. Savage et al. claim that it is a “widely validated” measure. This is an odd claim because I know of no published or unpublished attempts to demonstrate the criterion-related validity of this instrument. In fact I can report from personal experience having a rather heated exchange with one of the original constructors of the scale during which he both denied that it was either possible or desirable to validate it in this sense just as it was impossible to say what it was that the scale was actually a measure of! I was also present when another of the creators of the scale waxed lyrical for an hour about his special “theory of measurement” but alas proved unable to tell anyone what exactly it was or how it differed from any other theory of measurement. But I digress. Clearly my low social capital score owes something to being an anti-social git, but it owes even more to the fact that those people I do know come from a relatively narrow range of occupations. I know and socialize with academics and professionals of various sorts. But that counts for little in the GBCS. To improve my score I’ll need to get to know a much wider range of people. Like a butterfly collector I should really go for one of each.
It’s time to move on to the piece-de-resistance – the Savage et al. social classes themselves.  These fall out of the application of a particular statistical model – a so called latent profile model – to the 1000 odd responses in the quota sample plus 1 case consisting of the 161,400 responses to the GBCS each  given a weight of 1/161400. This means that the model estimates are derived almost entirely from the small quota sample and the GBCS is basically discarded. Actually it is quite a neat trick to rescue a slightly desperate situation. After all it would have been embarrassing to have collected all that data and not used it. It means that the GBCS data can be allocated to class categories even on the basis of a model that is based almost entirely on the information contained in the smaller sample.
It would be nice to know more about the latent profile model, but we are given few details  and this makes  it difficult to say anything sensible. I’m guessing that it makes some quite strong assumptions – conditional independence of the observed variables given the latent classes as well as marginal normality. The latter, if it is assumed, seems quite implausible to me. I wonder how the referees of the paper managed to make sense of the procedure, or were they just baffled by the apparent science? One thing that is clear is the consequences of the model selection procedure. We are told rather blandly that BIC is minimized when seven latent classes are assumed. We are not told how much worse things are when 6 or 8 are assumed. In fact we are given no sense whatsoever of model uncertainty or indeed why the gospel according to BIC is to be preferred to any other model selection criterion. In fact there is a dark secret hidden away here. The selection of seven classes, much trumpeted in the media as the central finding of the study, is an artefact of the sample size and without any well articulated theoretical grounds for distinguishing the classes it is difficult to see how it could be otherwise. It’s just as well that as things turned out, Savage et al. had, to all intents and purposes, to discard the GBCS data. Imagine they had estimated their latent profile model with 160,000 cases. The logic of their modus operandi would have forced them to conclude that rather than there being seven classes there were more like 77! The moral of the story is this: if you have no purpose in mind behind the application of a data reduction technique or no well articulated expectations, then the data will lead you by the nose. But "the data" or at least the amount of data is completely arbitrary and so will be your results.
The game is given away when Savage et al. attempt to interpret the seven social classes they "discover" (p 12 Tables 5 & 6). Normally one would do that in terms of the profiles of the observed variables associated with each of the latent classes. But looking, especially at Table 6, it becomes quickly apparent that this is rather difficult. Sure there are differences between the classes but which differences are important and which are trivial? There is no information presented here that would allow us to make a reasoned judgment using only the information that went into the latent profile analysis. In fact it is probable that the interpretation and almost certainly the labels applied to the classes actually comes from information quite extraneous to the latent profile analysis itself (pp 13 Table 7).  Bizarrely one of the key ingredients is - wait for it - the percentage in or stemming from one of the old, bad inadequate, occupationally based NS-SEC categories! But Table 7 is  quite revealing - actually of things that if Savage et al. were more Machiavellian they probably wouldn't have wanted to give away. Take for instance their "traditional working class". The average age of this group is 66! They have modest incomes, are low on "emerging cultural capital" yet 30% of them have, or had, jobs that would be classified as professional or managerial (don't you find that strange?). If the average age is 66 then a large proportion of this group are actually pensioners and may have nothing to do with the "working-class" traditional or otherwise. Pensioners are an important interest group, but what do we gain by calling them a social class? Life-cycle also I think plays a role in distinguishing what Savage et al. term the "elite" and the  "established middle class". In terms of the variables tossed into the food blender the latter  differ  from the former principally  because they  have smaller incomes, less savings and live in cheaper houses. In terms of social and cultural capital they don't differ at all. When we look at Table 7 what we find is that the "established middle class" are on average 11 years younger than the elite and other differences are quite marginal. We can't tell how marginal because Table 7 gives us no way of calculating confidence intervals but a substantial part of the difference between these two classes could quite plausibly attributable to the life-cycle (plus not happening to live in the South-East).
It's getting time to wind up, but before I do, I want to bring up a point about the way Savage et al. set up their argument. On pages 4 and 5 of their article they motivate their intellectual project by listing five objections to the Goldthorpe class model.  Fine, but what they say is either misleading, irrelevant or just plain false. Claim 1 is that the Goldthorpe scheme doesn't do well at predicting cultural consumption. Two points. Firstly  it probably also doesn't predict well the colour of peoples' eyes, but who said a class scheme had to predict everything? Secondly, Savage et al.'s own data show that the NS-SECs do in fact predict cultural consumption! Claim 2 about the reasons for the "neglect" of the elite (are 6% really an elite?) I've dealt with above. Claim 3 is that economists have criticized the sociological way of doing social mobility using occupational categories and that income inequality within classes is growing. As far as I can tell these are false. It would be more accurate to say that economists have simply ignored the sociological way of doing social mobility and paid no attention whatsoever to any evidence not produced by an echt economist. So be it, I hold no brief for the way economists do their business. What is definitely not true however is the claim that in the UK within occupational earnings inequality has grown larger than between occupation inequality. If anything the latter has dominated - as has been carefully demonstrated by my ex doctoral student Mark Williams. Claim 4 is that "... a focus on occupations as the sole measure of class occludes the more complex ways that class operates symbolically and culturally, through forms of stigmatisation and marking of personhood and value." Maybe it does, but I can't tell because I don't really understand what that sentence actually means. What is clear though is that it functions in Savage et al.'s discourse in about the same way as saying bugaboo to a child does. Claim 5, at least in the way Savage et al. present it is simply absurd: "Daniel Oesch (2006) questions several of the assumptions in the EGP scheme for lacking the ability to take highly important horizontal cleavages into account, for giving a too homogenous description of the salaried middle class and for overdoing the manual/non-manual divide when separating between male production workers and routine sales and service occupations." 
If Daniel Oesch  does think this (and I've only got Savage et al.'s say so to go on) then he clearly hasn't understood the EGP class schema. What does he think the distinction between employees on the one hand and the self-employed,  farmers and farm labourers on the other is if it is not a "horizontal cleavage"? And hasn't he noticed that in many applications Class IIIb is combined with Class VIIa? Just because a distinction exists it doesn't mean you have to use it. Back to my central theme of having a point.

And finally we hit paydirt, if you've stayed with me so far you have probably guessed that I don't find A New Model of Social Class: Findings from the BBC's Great British Class Survey Experiment a very impressive piece of work. Its high profile in the media is, I think, not a positive thing for British sociology. In some domains people believe that any publicity is good publicity. In science, some sorts of publicity  only serve to make the subject as well as  its practitioners look ridiculous. That creates a negative externality for those of us who try, to the best of our abilities, to do decent science. That's why I will continue to post on conspicuous examples of bad sociology.


druedin.com said...

Thanks for taking the trouble to write up such a detailed account! This post will serve as a useful point of reference (for all colleagues tempted by the GBCS). When I first read the results were out, I wondered how you/Oxford/Nuffield would react; after doing the "what class are you?" calculator and reading the abstract I wondered whether anyone would react at all (is it worth it to react?). Being an optimist, I point to the potential to interest students in class issues, and there's always educational value...

Primula Monkey said...

"what exactly is gained from a data reduction exercise that reduces variation along three conceptual dimensions to a small set of categories?"

Because its a very high profile part of a bigger, ongoing project to revive and/or maintain interest in class analysis?

i.e. it may well be methodologically pants, but there's a political backdrop here (which explains rather than justifies)?

S'funny the essentially arbitrary typology they come up with though given Professor Savage's (and his mate - whose good at counting - Andrew Miles) earlier wee gem of a book on the re-making of the working class and all that entails in terms of class consciousness i.e. how many people actually consider themselves/give a monkey's about "emergent service workers"? (something that could also be said about Goldthorpe's stuff, but then he was presumably asking a different question/has different interests)

Anonymous said...

Thank you for writing such an insightful piece. Your scientific analysis is very impressive. It is never easy to criticise a piece with huge publicity and approved publication in one of the top sociological journals. As you mentioned, the method of data collection and typology do not make any sense. It is more like a piece of work trying to shift the responsibility of social inequality from the government to individuals by addressing the importance of individual choice and taste in determining their social class rather than the unequal distribution of economic capital. I totally agree with your analysis and argument. Please keep writing on it.