It is possible that I’m completely wrong, but I am sort of surprised about the comments thus far, because I thought I immediately understood what people meant by “I prefer fixed effects over random effects because I care about identification”, but no one has attempted a translation yet. As Andrew has pointed out in a paper, “fixed effects” is used in various meanings, but I suppose that what’s meant is the standard usage – fixed effects as unit dummy variables (and perhaps, additionally, time dummies) that control for unmeasured variance between units (points in time) to the extent that it is stable over time (units). Which presupposes you have longitudinal data, although some authors have presented models that use, for example, family fixed effects (dummies). In econometrics, “the identification problem” refers to the problem of being able to make causal claims on the basis of observational data; fixed effects are thought to help with that because they control for an extra portion of the variance (see Stuart Buck’s example above). For a programmatic view, see Halaby, Charles N., 2004: “Panel Models in Sociological Research“, Annual Review of Sociology 30: 507-44; for the technique’s limitations, see Bjerk, David, 2009: “How Much Can We Trust Causal Interpretations of Fixed-Effects Estimators in the Context of Criminality?” Journal of Quantitative Criminology 25: 391-417.

dk says:
October 23, 2010 at 8:57 am
consider using z-score for y-axis. Often the likert measures don't have much intrinsic meaning — worse, they *appear* to have more meaning intrinsically than they do (it's a mistake to assume "slightly agree" vs. "slightly disagree," e.g., is some critically important division for opinion in the world; maybe it is, but probably it isn't). Usually what the likert measure is good for is extracting meaningful effects (of a treatment, say, or of some individual characteristic) by capturing observable degrees of variance in some attitude or latent disposition that you have independent reason to think matters in the world (debates over optimal size of likert are focused on this–the tradeoff between effect size & noise as number of points on scale increases). Using the z-score transformation of the measure for tye y-axis focuses attention on *that* because readers can see how large the variance was between conditions or between different subjects relative to variance across the sample, & aren't tempted to attribute meaning to arbitrary units in the raw likert measure. Also, using z-score as y-axis avoids potentially incorrect interpretations based on where scores fall on the mean or how many likert units differences between conditions or groups span. If, e.g., the sample mean on an 11-pt measure is 6, & you have two conditions that have means of 5 & 7 & SEMs of 0.05, people will think, "gee, everybody is pretty avarge & there's really not much difference between subjects in the two conditions." Sigh. Preempting this inference inference is what motivates people to truncate their y-axis– leading others to say, "hey, don't do that! That's creating a misleading view of your effect size!" Well, *not* doing it can be misleading too if people are unable get a good sense of variance & effect sizes from looking at bars plotted over the whole scale. So use a z-score, usually with -1 & +1 as upper & lower bounds — nothing misleading about that — & plot your data (in form of bars w/ CIs or whatever) within that.
Carl says:
October 25, 2010 at 7:36 am
Thanks dk – that was a really good suggestion and I'm trying it for my survey data results.

Thom says:
November 11, 2010 at 4:55 am
I'd have to disagree with dk's suggestion. If a scales "don't have much intrinsic meaning" then taking z scores don't add in meaning. They merely express the scores in terms of the sample SD. This can differ for all sorts of reason that have nothing to do with what you are measuring (e.g., if the ratings are high or low the z scores will be bigger because of ceiling or floor effects flattening the SD).

Furthermore if the scales are of a disagree/agree type, the most psychologically important information on the scale is probably whether they disagree or agree and this can be obscured by the z scoring.

I do agree that the variability is important and that plotting with CIs is sensible (and z scores with CIs is probably better than raw scores without CIs). However, confounding size of effect with its variability in the sample is problematic for interpretation and decreases transparency.