Part 6: Systematic Measurement Error on Bounded Scales

library(lavaan); library(semTools); library(MASS); library(ggplot2)
library(dplyr); library(tidyr); library(knitr)

---

Connecting Back to Classical Test Theory

In Part 1 of this module we introduced the Classical Test Theory (CTT) decomposition that underlies all measurement in the behavioural sciences:

\[X_{\text{obs}} = T + \varepsilon\]

where \(X_{\text{obs}}\) is the observed score, \(T\) is the respondent’s true latent value on the construct of interest, and \(\varepsilon\) is measurement error. The entire edifice of psychometric measurement — reliability, validity, structural equation modelling — rests on one fundamental assumption about \(\varepsilon\): it is symmetric, centred at zero, with equal probability of positive and negative deviations. This symmetry is what allows the observed mean to function as an unbiased estimator of the true latent mean: the errors cancel in expectation.

This assumption is testable. In a well-specified measurement model, the estimated residuals \(\hat{\varepsilon}_i = X_{\text{obs},i} - \hat{T}_i\) should be symmetrically distributed — mean zero, skewness near zero. A simple Q-Q plot of residuals, D’Agostino’s skewness test, or the Jarque–Bera test can detect systematic asymmetry. If the residuals are skewed, the symmetry assumption is violated and the observed mean is no longer an unbiased proxy for the true latent mean — it will be pulled toward the center of the scale (ceiling effects bias means downward; floor effects bias them upward).

WarningThe symmetry assumption for \(\varepsilon\) rarely holds in practice — bounded scales are a common culprit

The symmetry assumption is more often assumed than checked. One important way it fails is when participants are constrained in the range of responses they can express — most commonly because the scale itself runs out of room. The mechanism described below assumes a truncated latent-score model: the construct has a continuous latent distribution, and the observed score is that latent value clipped to the scale’s endpoints. This is the right model for many bounded continuous measures (sliders, 0–100 ratings, VAS scales), though different scale types — ordinal Likert items, count outcomes, or corner-solution variables with genuine mass at a bound — follow a different data-generating process and may require different diagnostics.

Consider a respondent whose true score is 90 on a 0–100 scale, with genuine response variability of ±15 points. Downward deviations are unconstrained: they can score 75, 80, 85. But upward deviations hit the ceiling: a genuine tendency to score 105 is recorded as 100. Under this truncated model, the error distribution is right-truncated. The respondent’s \(\varepsilon\) is no longer symmetric — it is negatively skewed, and every deviation toward the ceiling is suppressed.

This is not a psychometric failure or a respondent quirk. Under a truncated latent-score DGP, it is a structural consequence of bounded measurement that grows more severe the closer the true score sits to the ceiling or floor. The aggregate result — biased observed means, compressed variances, and skewed residuals — is what this part of the module explores. The takeaway: the symmetry of \(\varepsilon\) should be checked, not assumed, and any researcher using bounded continuous scales should inspect the residual distribution before treating observed means as unbiased.

The Problem in Plain Language

Almost every scale used in behavioral research has bounds — rating scales run from 1 to 7, percentages from 0 to 100, frequency measures from “never” to “always.” These bounds are usually treated as a minor formatting detail. In practice, they introduce a systematic bias that most researchers never think about.

NoteWhere this fits in the broader workshop arc

Bounded scales are ubiquitous across the social, health, economic, and statistical sciences — and so is the bias they create. But the problem appears under different names depending on the discipline: ceiling and floor effects in psychometrics, corner solutions and censored data in econometrics, bounded outcomes in health statistics, item difficulty effects in item response theory. Despite the different terminology, the underlying mathematics is identical: a hard limit on what a scale can record truncates the true distribution, and the truncated mean departs systematically from the truth.

This measurement-layer problem cascades upward through the rest of the inferential chain. Biased observed means corrupt hypothesis tests — because what is being tested is not the construct being theorized about. Those corrupted tests in turn compromise causal inference — because treatment-effect estimates compare biased observations rather than true latent values. A researcher who finds a null result comparing a control group (true mean 50) to a treated group (true mean 75, both on a 0–100 scale with SD 20) may be observing a genuine effect masked by differential truncation bias: the treated group’s distribution is clipped more severely because it sits closer to the ceiling. The measurement layer is not preliminary housekeeping — it shapes every conclusion drawn above it.

Here is the mechanism under a truncated latent-score model. Suppose a participant’s true level of some construct is 85 on a 0–100 scale, and they have a genuine tendency to vary ±15 points around that true value due to measurement noise (mood, attention, question framing). On the low side, they can express that variation freely — they might score 70, 75, 80. On the high side, they hit a ceiling. Scores of 95, 100, 105 are impossible — those attempts to go higher just pile up at 100. The result: the observed distribution of scores is not centered at 85. It is pulled toward the center of the scale. The ceiling cuts off the upper tail, making the observed mean lower than the true mean.

The same thing happens at the lower bound: if the true value is close to the floor, the floor cuts off the lower tail and pulls the observed mean upward. Both effects push observations toward the middle of the scale. (Note: for ordinal Likert items, the response function is categorical rather than continuous-truncation, so the magnitude and direction of bias can differ.)

Three factors make this worse:

1. Proximity to a bound — the closer the true value to a ceiling or floor, the more severe the truncation and the greater the bias.
2. More variability — the more spread there is in true scores (or measurement noise), the more the tails get cut off and the stronger the pull toward the center.
3. Tighter scale — a 7-point scale truncates more than a 100-point scale for the same true variability.

NoteOne-sided vs. two-sided bounds (under a truncated latent-score model)

Under a truncated latent-score DGP, scales bounded on both ends (0–100 slider, bounded continuous rating) produce symmetric pull toward the center: ceiling effects at the top and floor effects at the bottom, each biasing estimates toward the midpoint. Ordinal Likert scales share this topology but follow a different response function — the bias pattern is qualitatively similar but the magnitude depends on item thresholds, not just the raw bounds.

Scales bounded on one end only — like response time (lower bound at 0, no upper bound) or non-negative count data — produce asymmetric pull toward the bounded end only. The unbounded tail is free; the bounded tail is clipped. For corner-solution outcomes (variables with genuine probability mass at zero, such as dollars spent or number of purchases), the econometric treatment differs: a Tobit or hurdle model is more appropriate than a truncation correction.

For right-skewed continuous distributions common in bounded-at-zero measures (reaction time, income, sales), the floor effect is usually the active constraint, and under truncation the bias pushes mean estimates upward.

Interactive Simulator: See the Bias in Real Time

The simulator below shows the observed (truncated) distribution — what researchers actually record after the scale bounds clip the tails of the true distribution. Vertical reference lines mark the true mean (dashed black), the observed mean (solid blue), and the observed median (dashed red). The statistics panel shows the true and observed mean, SD, and skewness — each with its bias relative to the true value. As you move the true mean toward a bound or increase the SD, watch how the observed distribution narrows (SD compression), shifts (mean bias), and tilts (skewness) — and how the mean–median gap signals the direction of the skew. This is the CTT symmetry assumption failing in real time.

viewof bound_mu  = Inputs.range([1, 99], {step: 1, value: 50, label: "True mean (population)"})
viewof bound_sd  = Inputs.range([2, 40], {step: 1, value: 15, label: "True SD (variability / noise)"})

loB = 0
hiB = 100

// Normal PDF and CDF utilities
function phi_fn(x) {
  return Math.exp(-0.5 * x * x) / Math.sqrt(2 * Math.PI);
}
function erf_fn(x) {
  const s = x >= 0 ? 1 : -1, ax = Math.abs(x);
  const t = 1 / (1 + 0.3275911 * ax);
  const y = 1 - (((((1.061405429*t - 1.453152027)*t + 1.421413741)*t - 0.284496736)*t + 0.254829592)*t * Math.exp(-ax*ax));
  return s * y;
}
function Phi_fn(x) { return 0.5 * (1 + erf_fn(x / Math.sqrt(2))); }

// Inverse normal CDF (Acklam approximation)
function qnorm(p) {
  if (p <= 0) return -Infinity; if (p >= 1) return Infinity;
  const a = [-3.969683028665376e1, 2.209460984245205e2, -2.759285104469687e2, 1.383577518672690e2, -3.066479806614716e1, 2.506628277459239];
  const b = [-5.447609879822406e1, 1.615858368580409e2, -1.556989798598866e2, 6.680131188771972e1, -1.328068155288572e1];
  const c = [-7.784894002430293e-3, -3.223964580411365e-1, -2.400758277161838, -2.549732539343734, 4.374664141464968, 2.938163982698783];
  const d = [7.784695709041462e-3, 3.224671290700398e-1, 2.445134137142996, 3.754408661907416];
  const pL = 0.02425;
  let q;
  if (p < pL) {
    q = Math.sqrt(-2*Math.log(p));
    return (((((c[0]*q+c[1])*q+c[2])*q+c[3])*q+c[4])*q+c[5]) / ((((d[0]*q+d[1])*q+d[2])*q+d[3])*q+1);
  } else if (p <= 1-pL) {
    q = p - 0.5; const r = q*q;
    return (((((a[0]*r+a[1])*r+a[2])*r+a[3])*r+a[4])*r+a[5])*q / (((((b[0]*r+b[1])*r+b[2])*r+b[3])*r+b[4])*r+1);
  } else {
    q = Math.sqrt(-2*Math.log(1-p));
    return -(((((c[0]*q+c[1])*q+c[2])*q+c[3])*q+c[4])*q+c[5]) / ((((d[0]*q+d[1])*q+d[2])*q+d[3])*q+1);
  }
}

// Truncated normal moments (analytical)
trunc_moments = {
  const mu = bound_mu, sigma = bound_sd;
  const lo = loB, hi = hiB;
  const alpha = (lo - mu) / sigma;
  const beta  = (hi - mu) / sigma;
  const pA = Phi_fn(alpha);
  const pB = Phi_fn(beta);
  const Z  = pB - pA;
  const fA = phi_fn(alpha);
  const fB = phi_fn(beta);
  const obs_mean = mu + sigma * (fA - fB) / Z;
  const obs_med  = mu + sigma * qnorm((pA + pB) / 2);
  // Note: alpha*fA and beta*fB must be computed explicitly to avoid Infinity*0 = NaN
  const alphaFa  = isFinite(alpha) ? alpha * fA : 0;
  const betaFb   = isFinite(beta)  ? beta  * fB : 0;
  const obs_var  = sigma*sigma * (1 + (alphaFa - betaFb)/Z - ((fA-fB)/Z)**2);
  const obs_sd   = isFinite(obs_var) && obs_var >= 0 ? Math.sqrt(obs_var) : 0;
  return {obs_mean, obs_med, obs_sd, Z};
}

// Truncated (observed) density
density_data = {
  const mu = bound_mu, sigma = bound_sd;
  const lo = loB, hi = hiB;
  const pA = Phi_fn((lo - mu)/sigma);
  const pB = Phi_fn((hi - mu)/sigma);
  const Z  = pB - pA;
  const n  = 300;
  return Array.from({length: n}, (_, i) => {
    const x = lo + (i / (n-1)) * (hi - lo);
    const pdf = phi_fn((x-mu)/sigma) / (sigma * Z);
    return {x, pdf};
  });
}

bias_mean  = trunc_moments.obs_mean - bound_mu
bias_med   = trunc_moments.obs_med  - bound_mu
bias_sd    = trunc_moments.obs_sd   - bound_sd
bias_dir   = Math.abs(bias_mean) < 0.5 ? "negligible" : bias_mean > 0 ? "upward" : "downward"

obs_skewness = {
  // Numerical integration of the 3rd standardised central moment of the truncated distribution
  const mu = bound_mu, sigma = bound_sd;
  const lo = loB, hi = hiB;
  const pA = Phi_fn((lo - mu) / sigma);
  const pB = Phi_fn((hi - mu) / sigma);
  const Z  = pB - pA;
  const mu_obs = trunc_moments.obs_mean;
  const sd_obs = trunc_moments.obs_sd;
  if (sd_obs < 1e-10) return 0;
  const n  = 800;
  const dx = (hi - lo) / n;
  let s3 = 0;
  for (let i = 0; i < n; i++) {
    const x = lo + (i + 0.5) * dx;
    s3 += Math.pow((x - mu_obs) / sd_obs, 3) * phi_fn((x - mu) / sigma) / (sigma * Z) * dx;
  }
  return s3;
}

skew_label = Math.abs(obs_skewness) < 0.05
  ? "≈ 0 (symmetric)"
  : obs_skewness > 0
    ? `+${obs_skewness.toFixed(3)} (right-skewed — floor clips left tail)`
    : `${obs_skewness.toFixed(3)} (left-skewed — ceiling clips right tail)`
skew_color = Math.abs(obs_skewness) < 0.1 ? "#888" : "#e63946"

med_gap       = trunc_moments.obs_mean - trunc_moments.obs_med
med_gap_label = Math.abs(med_gap) < 0.05
  ? "mean ≈ median (symmetric ε)"
  : med_gap > 0
    ? `mean > median by ${med_gap.toFixed(2)} → right-skewed ε`
    : `mean < median by ${Math.abs(med_gap).toFixed(2)} → left-skewed ε`

html`<div style="display:flex;gap:10px;flex-wrap:wrap;margin:14px 0;font-size:.86em;line-height:1.6;">

  <div style="flex:1;min-width:190px;border-left:4px solid #444;padding:8px 14px;background:#f5f5f5;border-radius:4px;">
    <div style="font-weight:700;margin-bottom:4px;color:#444;">True distribution</div>
    <div><strong>Mean:</strong> ${bound_mu.toFixed(1)}</div>
    <div><strong>SD:</strong> ${bound_sd.toFixed(1)}</div>
    <div style="color:#888;font-size:.92em;"><strong>Skewness:</strong> 0 — ε is symmetric</div>
  </div>

  <div style="flex:1;min-width:190px;border-left:4px solid #457b9d;padding:8px 14px;background:#f0f4f8;border-radius:4px;">
    <div style="font-weight:700;margin-bottom:4px;color:#457b9d;">Observed distribution</div>
    <div><strong>Mean:</strong> ${trunc_moments.obs_mean.toFixed(2)}
      <span style="color:${Math.abs(bias_mean)>1?'#e63946':'#888'};">
        (Δ = ${bias_mean > 0 ? '+' : ''}${bias_mean.toFixed(2)})</span></div>
    <div><strong>SD:</strong> ${trunc_moments.obs_sd.toFixed(2)}
      <span style="color:${Math.abs(bias_sd)>0.5?'#e63946':'#888'};">
        (Δ = ${bias_sd > 0 ? '+' : ''}${bias_sd.toFixed(2)})</span></div>
    <div><strong>Skewness:</strong>
      <span style="color:${skew_color};">${skew_label}</span></div>
  </div>

  <div style="flex:1;min-width:190px;border-left:4px solid #e63946;padding:8px 14px;background:#fdf0f0;border-radius:4px;">
    <div style="font-weight:700;margin-bottom:4px;color:#e63946;">CTT symmetry check</div>
    <div><strong>Median:</strong> ${trunc_moments.obs_med.toFixed(2)}
      <span style="color:${Math.abs(bias_med)>1?'#e63946':'#888'};">
        (Δ = ${bias_med > 0 ? '+' : ''}${bias_med.toFixed(2)})</span></div>
    <div style="margin-top:4px;color:${Math.abs(med_gap)>0.5?'#e63946':'#666'};">
      ${med_gap_label}</div>
    <div style="color:#888;font-size:.91em;margin-top:2px;">
      Mean–median gap reveals<br>asymmetry in ε</div>
  </div>

</div>`

Plot.plot({
  width: 650, height: 300,
  marginLeft: 55, marginBottom: 45, marginTop: 20, marginRight: 20,
  x: {label: "Scale value →", domain: [-2, 102]},
  y: {label: "Density", zero: true},
  marks: [
    // Observed (truncated) distribution — what researchers actually record
    Plot.areaY(density_data, {x: d => d.x, y: d => d.pdf, fill: "#457b9d", fillOpacity: 0.45}),
    Plot.line(density_data,  {x: d => d.x, y: d => d.pdf, stroke: "#457b9d", strokeWidth: 2.2}),
    // Reference lines
    Plot.ruleX([bound_mu],               {stroke: "#222",    strokeDasharray: "6,3", strokeWidth: 2.2}),
    Plot.ruleX([trunc_moments.obs_mean], {stroke: "#457b9d",                         strokeWidth: 2.5}),
    Plot.ruleX([trunc_moments.obs_med],  {stroke: "#e63946", strokeDasharray: "4,3", strokeWidth: 2.5}),
    Plot.ruleX([loB], {stroke: "#aaa", strokeWidth: 1.5}),
    Plot.ruleX([hiB], {stroke: "#aaa", strokeWidth: 1.5})
  ]
})

Figure 8.1: Blue filled area = observed (truncated) distribution — what researchers actually record after the scale bounds clip the tails. Dashed black vertical line = true mean; solid blue = observed mean; dashed red = observed median. The gap between the true mean (black) and the observed mean (blue) is the truncation bias. The gap between the observed mean and median indicates skewness in the measurement error.

WarningWhat this means for your research

Means are biased toward the scale midpoint. When comparing groups or conditions, if one group has a true mean closer to a bound than the other, the truncation bias differs between groups — the apparent treatment effect is distorted by the scale itself, not just the construct. This is particularly dangerous in pre–post designs, where an intervention may move participants toward a ceiling, making the post-treatment distribution more severely clipped and therefore making the estimated gain look smaller than it truly is.

The problem gets worse as variability increases. High within-person variability (which researchers often interpret as low reliability) amplifies truncation bias. Ironically, a noisy measure produces more bias, not just more noise. This also means that between-group comparisons of variance — common in psychometric analyses — are confounded whenever the groups sit at different distances from a bound.

Transformations can help, but not fully eliminate the problem. Logit-transforming proportions (e.g., log(p/(1-p))) or arcsine-transforming bounded scores can partially stabilize the bias. Tobit regression (Wooldridge, 2010) models the truncated distribution explicitly and estimates the latent mean directly — it was developed in econometrics for exactly this structure, where household expenditure is zero for non-buyers and positive (and therefore bounded below) for buyers.

The problem has different names across disciplines — but the same solution. In econometrics it is censored data; in health research it is ceiling and floor effects in patient-reported outcome (PRO) instruments; in psychometrics it appears as item difficulty in item response theory; in statistics it is truncated distribution estimation. Each field has converged on similar remedies: model the latent distribution explicitly (Tobit / truncated regression), use response scales whose range substantially exceeds the likely spread of true scores, and — as a minimum — report the proportion of respondents at or near each bound before interpreting means and treatment comparisons.

References

Pieters, R., Srivastava, J., & Bagchi, R. (2025). Improving the discriminant validation of multi-item scales. Journal of Marketing Research. https://doi.org/10.1177/00222437251322089

Henseler, J., Ringle, C. M., & Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the Academy of Marketing Science, 43(1), 115–135.

Millsap, R. E., & Kwok, O.-M. (2004). Evaluating the impact of partial factor loading and intercept invariance on selection in two populations. Psychological Methods, 9(1), 93–115.

Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71–90.

Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36.

Scrucca, L., Fop, M., Murphy, T. B., & Raftery, A. E. (2016). mclust 5: Clustering, classification and density estimation using Gaussian finite mixture models. The R Journal, 8(1), 289–317.

Breunig, M. M., Kriegel, H.-P., Ng, R. T., & Sander, J. (2000). LOF: Identifying density-based local outliers. ACM SIGMOD Record, 29(2), 93–104.

Ester, M., Kriegel, H.-P., Sander, J., & Xu, X. (1996). A density-based algorithm for discovering clusters in large spatial databases with noise. Proceedings of the 2nd International Conference on Knowledge Discovery and Data Mining (KDD-96), 226–231.

Venables, W. N., & Ripley, B. D. (2002). Modern applied statistics with S (4th ed.). Springer.

Wooldridge, J. M. (2010). Econometric analysis of cross section and panel data (2nd ed.). MIT Press.

---

Researcher Checklist: Bounded Scales

NoteKey questions when using Likert or other bounded measures

• Are any of your groups near the ceiling or floor of the scale? If so, the observed difference between groups is mechanically compressed — the scale ran out of room, not because the treatment was weaker.
• Is the variance lower in the group closer to the bound? Compressed variance distorts standardized effect sizes (Cohen’s d) and signals truncation bias.
• Does your treatment effect hold among participants who were not at ceiling before treatment? If it disappears in this subsample, truncation is driving the result.
• Is a failed manipulation check actually a ceiling effect? Check the distribution of the manipulation check before concluding the manipulation was weak.

---