Introduction to Robust Estimation and Hypothesis Testing

Introduction to Robust Estimation and Hypothesis Testing......Page 2
Copyright......Page 3
Preface......Page 4
1.1 Problems with Assuming Normality......Page 7
1.2 Transformations......Page 11
1.3 The Influence Curve......Page 13
1.4 The Central Limit Theorem......Page 14
1.5 Is the ANOVA F Robust?......Page 15
1.6 Regression......Page 16
1.8 Using the Computer: R......Page 17
1.9 Some Data Management Issues......Page 19
1.9.1 Eliminating Missing Values......Page 28
2.1 Basic Tools for Judging Robustness......Page 29
2.1.1 Qualitative Robustness......Page 30
2.1.2 Infinitesimal Robustness......Page 33
2.1.3 Quantitative Robustness......Page 34
2.2.1 Quantiles......Page 35
2.2.2 The Winsorized Mean......Page 36
2.2.4 M-Measures of Location......Page 38
2.2.5 R-Measures of Location......Page 41
2.3 Measures of Scale......Page 42
2.4 Scale Equivariant M-Measures of Location......Page 44
2.5 Winsorized Expected Values......Page 45
3.1 A Bootstrap Estimate of a Standard Error......Page 49
3.1.1 R Function bootse......Page 51
3.2.1 Normal Kernel......Page 52
3.2.3 The Expected Frequency Curve......Page 53
3.2.4 An Adaptive Kernel Estimator......Page 54
3.2.5 R Functions skerd, kerden, kdplot, rdplot, akerd, and splot......Page 55
3.3 The Sample Trimmed Mean......Page 60
3.3.2 Estimating the Standard Error of the Trimmed Mean......Page 63
3.3.5 Estimating the Standard Error of the Sample Median, M......Page 68
3.4 The Finite Sample Breakdown Point......Page 69
3.5 Estimating Quantiles......Page 70
3.5.1 Estimating the Standard Error of the Sample Quantile......Page 71
3.5.2 R Function qse......Page 72
3.5.3 The Maritz–Jarrett Estimate of the Standard Error of x?q......Page 73
3.5.5 The Harrell–Davis Estimator......Page 74
3.5.7 A Bootstrap Estimate of the Standard Error of θ?q......Page 75
3.6 An M-Estimator of Location......Page 76
3.6.2 Computing an M-estimator of Location......Page 81
3.6.3 R Functions mest......Page 83
3.6.4 Estimating the Standard Error of the M-estimator......Page 84
3.6.6 A Bootstrap Estimate of the Standard Error of μ?m......Page 86
3.6.7 R Function mestseb......Page 87
3.7 One-Step M-estimator......Page 88
3.8 W-estimators......Page 89
3.8.1 Tau Measure of Location......Page 90
3.10 Skipped Estimators......Page 91
3.11 Some Comparisons of the Location Estimators......Page 92
3.12 More Measures of Scale......Page 95
3.12.1 The Biweight Midvariance......Page 96
3.12.3 The Percentage Bend Midvariance and tau Measure of Variation......Page 98
3.12.4 R Functions pbvar, tauvar......Page 100
3.12.5 The Interquartile Range......Page 101
3.13.2 A Method Based on the Interquartile Range......Page 102
3.13.4 A MAD-Median Rule......Page 103
3.13.5 R Functions outbox, out, and boxplot......Page 104
3.13.6 Skewness and the Boxplot Rule......Page 105
3.14 Exercises......Page 106
4.1 Problems when Working with Means......Page 108
4.2 The g-and-h Distribution......Page 112
4.2.1 R Functions ghdist and rmul......Page 115
4.3 Inferences About the Trimmed and Winsorized Means......Page 116
4.3.1 R Functions trimci and winci......Page 119
4.4.1 The Percentile Bootstrap Method......Page 120
4.4.2 R Function onesampb......Page 121
4.4.3 Bootstrap-t Method......Page 122
4.4.4 Bootstrap Methods when Using a Trimmed Mean......Page 123
4.4.6 R Functions trimpb and trimcibt......Page 128
4.5 Inferences About M-Estimators......Page 129
4.6 Confidence Intervals for Quantiles......Page 131
4.6.1 Beware of Tied Values when Using the Median......Page 134
4.6.2 Alternative Method for the Median......Page 135
4.6.3 R Functions qmjci, hdci, sint, sintv2, qci, and qint......Page 136
4.7 Empirical Likelihood......Page 137
4.7.1 Bartlett Corrected Empirical Likelihood......Page 138
4.9 Exercises......Page 140
5 Comparing Two Groups......Page 142
5.1 The Shift Function......Page 143
5.1.1 The Kolmogorov–Smirnov Test......Page 146
5.1.2 R Functions ks, kssig, kswsig, and kstiesig......Page 149
5.1.3 The S Band and W Band for the Shift Function......Page 151
5.1.4 R Functions sband and wband......Page 152
5.1.5 Confidence Band for the Deciles Only......Page 155
5.1.6 R Function shifthd......Page 156
5.2 Student\'s t-test......Page 158
5.3 Comparing Medians and Other Trimmed Means......Page 162
5.3.1 R Function yuen......Page 165
5.3.2 A Bootstrap-t Method for Comparing Trimmed Means......Page 166
5.3.3 R Functions yuenbt and yhbt......Page 168
5.3.4 Measuring Effect Size: Robust Analogs of Cohen\'s d......Page 171
5.3.5 R Functions akp.effect, yuenv2, and ees.ci......Page 174
5.4 Inferences Based on a Percentile Bootstrap Method......Page 175
5.4.1 Comparing M-Estimators......Page 176
5.4.2 Comparing Trimmed Means and Medians......Page 177
5.4.3 R Functions trimpb2, pb2gen, m2ci, and medpb2......Page 178
5.5.1 Comparing Variances......Page 179
5.5.2 R Function comvar2......Page 180
5.6 Permutation Tests......Page 181
5.7 Inferences About a Probabilistic Measure of Effect Size......Page 182
5.7.1 R Function mee......Page 184
5.7.2 The Cliff and Bruner–Munzel Methods: Handling Tied Values......Page 185
5.7.3 R Functions cid, cidv2, bmp, and wmwloc......Page 189
5.8 Comparing Two Independent Binomials......Page 191
5.8.1 Storer–Kim Method......Page 192
5.8.2 Beal\'s Method......Page 193
5.8.4 R Functions twobinom, twobici, bi2KMS, bi2KMSv2, and bi2CR......Page 194
5.9.1 A Shift Function for Dependent Groups......Page 195
5.9.3 Comparing Deciles......Page 197
5.9.4 R Function shiftdhd......Page 198
5.9.5 Comparing Trimmed Means......Page 200
5.9.6 R Functions yuend and yuendv2......Page 202
5.9.8 R Function ydbt......Page 203
5.9.9 Inferences about the Distribution of Difference Scores......Page 204
5.9.10 R Functions loc2dif and l2drmci......Page 205
5.9.11 Percentile Bootstrap: Comparing Medians, M-Estimators and Other Measures of Location and Scale......Page 206
5.9.12 R Function bootdpci......Page 207
5.9.13 Handling Missing Values......Page 208
Method M1......Page 209
Method M3......Page 211
5.9.14 R Functions rm2miss and rmmismcp......Page 212
5.9.16 The Sign Test and Inferences about the Binomial Distribution......Page 213
5.9.17 R Functions binomci and acbinomci......Page 216
5.10 Exercises......Page 217
6.1 Generalized Variance......Page 219
6.2.2 Halfspace Depth......Page 220
6.2.3 Computing Halfspace Depth......Page 222
6.2.4 R Functions depth2, depth, fdepth, fdepthv2, and unidepth......Page 225
6.2.5 Projection Depth......Page 226
6.2.7 Other Measures of Depth......Page 227
6.3 Some Affine Equivariant Estimators......Page 228
6.3.1 Minimum Volume Ellipsoid Estimator......Page 229
6.3.2 The Minimum Covariance Determinant Estimator......Page 230
6.3.3 S-Estimators and Constrained M-Estimators......Page 231
6.3.5 Donoho–Gasko Generalization of a Trimmed Mean......Page 232
6.3.6 R Functions dmean and dcov......Page 233
6.3.7 The Stahel–Donoho W-Estimator......Page 234
6.3.9 Median Ball Algorithm......Page 235
6.3.11 OGK Estimator......Page 236
6.3.12 R Function ogk......Page 237
6.3.14 R Function MARest......Page 238
6.4 Multivariate Outlier Detection Methods......Page 239
6.4.1 A Relplot......Page 240
6.4.2 R Function relplot......Page 242
6.4.5 R Functions covmve and covmcd......Page 243
6.4.6 R function out......Page 244
6.4.7 The MGV Method......Page 245
6.4.8 R Function outmgv......Page 247
6.4.9 A Projection Method......Page 248
6.4.10 R functions outpro and out3d......Page 250
6.4.12 R Function outproad and outmgvad......Page 251
6.4.14 Comments on Choosing a Method......Page 252
6.5 A Skipped Estimator of Location and Scatter......Page 254
6.5.1 R Functions smean, wmcd, wmve, mgvmean, L1medcen, spat,mgvcov, skip, skipcov, and dcov......Page 256
6.6 Robust Generalized Variance......Page 258
6.7.1 Inferences Based on the OP Measure of Location......Page 259
6.7.2 Extension of Hotelling\'s T2 to Trimmed Means......Page 260
6.7.3 R Functions smeancrv2 and hotel1.tr......Page 261
6.8 Two-Sample Case......Page 263
Data Management......Page 264
6.9 Multivariate Density Estimators......Page 266
6.10 A Two-Sample, Projection-Type Extension of the Wilcoxon–Mann–Whitney Test......Page 267
6.10.1 R functions mulwmw and mulwmwv2......Page 269
6.11 A Relative Depth Analog of the Wilcoxon–Mann–Whitney Test......Page 271
6.11.1 R function mwmw......Page 272
6.12 Comparisons Based on Depth......Page 273
6.12.1 R Functions lsqs3 and depthg2......Page 276
6.13 Comparing Dependent Groups Based on All Pairwise Differences......Page 279
6.14 Robust Principal Components Analysis......Page 281
6.14.2 Maronna\'s Method......Page 283
6.14.4 Method HRVB......Page 284
6.14.6 Method PPCA......Page 285
6.14.7 R Functions outpca, robpca, robpcaS, SPCA, Ppca, and Ppca.summary......Page 286
6.14.8 Comments on Choosing the Number of Components......Page 287
6.15 Cluster Analysis......Page 291
6.16 Exercises......Page 292
7 One-Way and Higher Designs for Independent Groups......Page 294
7.1 Trimmed Means and a One-Way Design......Page 295
7.1.1 A Welch-Type Procedure and a Robust Measure of Effect Size......Page 296
A Robust, Heteroscedastic Measure of Effect Size......Page 297
7.1.2 R Functions t1way, t1wayv2, esmcp, fac2list, and t1wayF......Page 298
Data Management......Page 300
7.1.3 A Generalization of Box\'s Method......Page 301
7.1.4 R Function box1way......Page 302
7.1.5 Comparing Medians......Page 303
7.1.7 A Bootstrap-t method......Page 304
7.1.8 R Functions t1waybt and btrim......Page 305
7.2 Two-Way Designs and Trimmed Means......Page 307
7.2.1 R Functions t2way......Page 311
7.2.2 Comparing Medians......Page 313
7.3 Three-Way Designs and Trimmed Means......Page 314
7.3.1 R Functions t3way and fac2list......Page 316
7.4 Multiple Comparisons Based on Medians and Other Trimmed Means......Page 319
7.4.1 An Extension of Yuen\'s Method to Trimmed Means......Page 320
7.4.2 R Function lincon......Page 322
7.4.3 Multiple Comparisons for Two-way and Three-Way Designs......Page 325
7.4.4 R Functions mcp2atm, mcp2med, mcp3atm, mcp3med, con2way, and con3way......Page 326
7.4.5 A Bootstrap-t Procedure......Page 328
7.4.6 R Functions linconb, bbtrim, and bbbtrim......Page 330
Rom’s Method......Page 332
Benjamini–Hochberg Method......Page 333
7.4.8 R Functions tmcppb, bbmcppb, bbbmcppb, medpb, med2mcp,med3mcp, and mcppb20......Page 334
7.4.9 Judging Sample Sizes......Page 336
7.4.10 R Function hochberg......Page 337
7.4.12 R Functions ESmainMCP and esImcp......Page 338
7.5 A Random Effects Model for Trimmed Means......Page 339
7.5.1 A Winsorized Intraclass Correlation......Page 341
7.5.2 R Function rananova......Page 342
7.6 Global Tests Based on M-Measures of Location......Page 343
7.6.1 R Functions b1way and pbadepth......Page 346
7.6.2 M-estimators and Multiple Comparisons......Page 347
Variation of a Bootstrap-t Method......Page 348
A Percentile Bootstrap Method: Method SR......Page 349
7.6.3 R Functions linconm and pbmcp......Page 350
7.7 M-Measures of Location and a Two-Way Design......Page 351
7.8 Ranked-Based Methods for a One-Way Design......Page 354
7.8.1 The Rust–Fligner Method......Page 355
7.8.2 R Function rfanova......Page 356
7.8.4 R Function bdm......Page 357
7.8.5 Inferences about a Probabilistic Measure of Effect Size......Page 359
7.8.6 R Functions cidmulv2, wmwaov and cidM......Page 361
7.9 A Rank-Based Method for a Two-Way Design......Page 362
7.9.2 The Patel–Hoel Approach to Interactions......Page 364
7.9.3 R Function rimul......Page 365
7.10 MANOVA Based on Trimmed Means......Page 366
7.10.1 R Functions MULtr.anova, MULAOVp, bw2list, and YYmanova......Page 368
7.10.2 Linear Contrasts......Page 370
7.10.3 R Functions linconMpb, linconSpb, YYmcp, fac2Mlist, and fac2BBMlist......Page 372
Data Management......Page 373
7.11 Nested Designs......Page 374
7.12 Exercises......Page 377
8.1 Comparing Trimmed Means......Page 381
8.1.2 R Function rmanova......Page 382
8.1.3 Pairwise Comparisons and Linear Contrasts Based on Trimmed Means......Page 383
8.1.4 Linear Contrasts Based on the Marginal Random Variables......Page 386
8.1.5 R Function rmmcp and rmmismcp......Page 387
8.1.6 Judging the Sample Size......Page 388
8.2.1 Comparing Trimmed Means......Page 389
8.2.3 Multiple Comparisons Based on Trimmed Means......Page 390
8.2.4 R Functions pairdepb and bptd......Page 392
8.2.5 Percentile Bootstrap Methods......Page 394
8.2.6 R Functions bd1way and ddep......Page 396
8.2.7 Multiple Comparisons Using M-estimators or Skipped Estimators......Page 397
8.2.8 R Functions lindm and mcpOV......Page 399
8.3 Bootstrap Methods Based on Difference Scores......Page 400
8.3.2 Multiple Comparisons......Page 402
8.3.3 R Functions rmmcppb, wmcppb, dmedpb, and lindepbt......Page 404
8.4 Comments on which Method to Use......Page 406
8.5 Some Rank-Based Methods......Page 408
8.6.1 Analyzing a Between-by-Within Design Based on Trimmed Means......Page 410
8.6.2 R Functions bwtrim and tsplit......Page 412
8.6.3 Data Management: R Function bw2list......Page 414
8.6.4 Bootstrap-t Method for a Between-by-Within Design......Page 415
8.6.6 Percentile Bootstrap Methods for a Between-by-Within Design......Page 416
8.6.7 R Functions sppba, sppbb, and sppbi......Page 419
8.6.8 Multiple Comparisons......Page 420
8.6.9 R Functions bwmcp, bwamcp, bwbmcp, bwimcp, spmcpa, spmcpb, and spmcpi......Page 423
8.6.10 Within-by-Within Designs......Page 424
8.6.12 A Rank-Based Approach......Page 425
8.6.13 R Function bwrank......Page 429
8.6.16 Multiple Comparisons when Using a Patel–Hoel Approach to Interactions......Page 431
8.7.1 The Munzel–Brunner Method......Page 433
8.7.2 R Function mulrank......Page 435
8.7.3 The Choi–Marden Multivariate Rank Test......Page 436
8.7.4 R Function cmanova......Page 437
8.8.1 Global Tests Based on Trimmed Means......Page 438
8.8.3 Data Management: R Functions bw2list and bbw2list......Page 439
8.8.4 Multiple Comparisons......Page 440
8.8.6 R Functions bbwmcp, bwwmcp, bbwmcppb, bwwmcppb, and wwwmcppb......Page 441
8.9 Exercises......Page 442
9.1 Problems with the Product Moment Correlation......Page 443
9.1.1 Features of Data that Affect r and T......Page 446
9.1.2 Heteroscedasticity and the Classic Test that ρ=0......Page 447
9.3.1 The Percentage Bend Correlation......Page 448
9.3.2 A Test of Independence Based on ρpb......Page 449
9.3.4 A Test of Zero Correlation among p Random Variables......Page 451
9.3.5 R Function pball......Page 453
9.3.6 The Winsorized Correlation......Page 454
9.3.7 R Functions wincor and winall......Page 455
9.3.8 The Biweight Midcovariance......Page 456
9.3.9 R Functions bicov and bicovm......Page 457
9.3.10 Kendall\'s tau......Page 458
9.3.12 R Functions tau, spear, cor, and taureg......Page 459
9.3.13 Heteroscedastic Tests of Zero Correlation......Page 460
9.3.14 R Functions corb, pcorb, and pcorhc4......Page 461
9.4.2 Skipped Measures of Correlation......Page 462
9.4.4 Inferences Based on Multiple Skipped Correlations......Page 463
9.4.5 R Functions scor and mscor......Page 465
9.5 A Test of Independence Sensitive to Curvature......Page 466
9.5.1 R Functions indt, indtall, and medind......Page 468
9.6.1 Comparing Pearson Correlations......Page 469
9.7 Exercises......Page 470
10 Robust Regression......Page 472
10.1 Problems with Ordinary Least Squares......Page 473
10.1.1 Computing Confidence Intervals under Heteroscedasticity......Page 476
10.1.2 An Omnibus Test......Page 480
10.1.3 R Functions lsfitNci, lsfitci, olshc4, hc4test, and hc4wtest......Page 481
10.1.5 Comments on Trying to Salvage the Homoscedasticity Assumption......Page 484
10.2 Theil–Sen Estimator......Page 485
10.2.1 R Functions tsreg, correg, and regplot......Page 487
10.3.1 R Function lmsreg......Page 488
10.5 Least Trimmed Absolute Value Estimator......Page 489
10.6 M-Estimators......Page 490
10.7 The Hat Matrix......Page 491
10.8 Generalized M-Estimators......Page 494
10.8.1 R Function bmreg......Page 498
10.9 The Coakley–Hettmansperger and Yohai Estimators......Page 499
10.9.1 MM-Estimator......Page 500
10.10 Skipped Estimators......Page 501
10.10.1 R Functions mgvreg and opreg......Page 502
10.11.1 R Function mdepreg......Page 503
10.13.1 S-Estimators and τ-Estimators......Page 504
10.13.2 R Functions snmreg and stsreg......Page 505
10.13.3 E-Type Skipped Estimators......Page 506
10.13.4 R Functions mbmreg, tstsreg, and gyreg......Page 507
10.13.5 Methods Based on Robust Covariances......Page 508
10.13.6 R Functions bireg, winreg, and COVreg......Page 510
10.13.8 L1 and Quantile Regression......Page 511
10.13.10 Methods Based on Estimates of the Optimal Weights......Page 512
10.13.11 Projection Estimators......Page 513
10.13.12 Methods Based on Ranks......Page 514
10.14 Comments About Various Estimators......Page 515
10.14.1 Contamination Bias......Page 516
10.15.1 Detecting Regression Outliers......Page 521
10.15.2 R Function reglev......Page 522
10.16 Logistic Regression and the General Linear Model......Page 523
10.16.1 R Functions glm, logreg, wlogreg, and logreg.plot......Page 524
10.16.2 The General Linear Model......Page 525
10.17 Multivariate Regression......Page 526
10.17.1 The RADA Estimator......Page 527
10.17.2 The Least Distance Estimator......Page 528
10.17.3 R Functions mlrreg and Mreglde......Page 529
10.17.4 Multivariate Least Trimmed Squares Estimator......Page 530
10.18 Exercises......Page 531
11.1 Inferences About Robust Regression Parameters......Page 534
11.1.1 Omnibus Tests for Regression Parameters......Page 535
11.1.2 R Function regtest......Page 539
11.1.3 Inferences About Individual Parameters......Page 540
11.1.4 R Functions regci and wlogregci......Page 542
11.1.5 Methods Based on the Quantile Regression Estimator......Page 544
11.1.6 R Functions rqtest, qregci, and qrchk......Page 545
11.1.7 Inferences Based on the OP-Estimator......Page 546
11.1.8 R Functions opregpb and opregpbMC......Page 547
11.1.9 Hypothesis Testing when Using the Multivariate Regression Estimator RADA......Page 548
11.1.10 R Function mlrGtest......Page 549
11.2 Comparing the Parameters of Two Independent Groups......Page 550
11.2.1 R Function reg2ci......Page 552
11.3.1 A Quantile Regression Approach......Page 554
11.3.2 Koenker\'s Method......Page 555
11.4 Curvature and Half-Slope Ratios......Page 556
11.4.1 R Function hratio......Page 557
11.5.1 Smoothers......Page 559
11.5.2 Kernel Estimators and Cleveland\'s LOWESS......Page 560
11.5.4 The Running Interval Smoother......Page 562
11.5.5 R Functions runmean, rungen, runmbo, and runhat......Page 567
11.5.7 Smoothers for Estimating Quantiles via Splines......Page 570
11.5.9 Special Methods for Binary Outcomes......Page 571
11.5.10 R Functions logrsm bkreg, logSM, and rplot.bin......Page 573
11.5.11 Smoothing with More than One Predictor......Page 574
11.5.12 R Functions runm3d, run3hat, rung3d, run3bo, rung3hat, rplot, rplotsm, and runpd......Page 575
11.5.13 LOESS......Page 579
11.5.14 Other Approaches......Page 582
11.5.15 R Function adrun, adrunl, gamplot, and gamplotINT......Page 584
11.6 Checking the Specification of a Regression Model......Page 585
11.6.1 Testing the Hypothesis of a Linear Association......Page 586
11.6.3 Testing the Hypothesis of a Generalized Additive Model......Page 587
11.6.4 R Function adtest......Page 588
11.6.6 R Function adcom......Page 589
11.7 Regression Interactions and Moderator Analysis......Page 590
11.7.1 R Functions kercon, riplot, runsm2g, ols.plot.inter, and reg.plot.inter......Page 592
11.7.2 Mediation Analysis......Page 595
11.8 Comparing Parametric, Additive, and Nonparametric Fits......Page 597
11.8.1 R Functions adpchk and pmodchk......Page 598
11.9 Measuring the Strength of an Association Given a Fit to the Data......Page 599
11.9.2 Comparing Two Independent Groups via Explanatory Power......Page 601
11.10  Comparing Predictors......Page 602
11.10.1 Comparing Pearson Correlations......Page 603
11.10.2 Methods Based on Estimating Prediction Error......Page 604
11.10.3 R Functions TWOpov, regpre, and regpreCV......Page 606
11.10.5 Comparing Predictors via Explanatory Power and a Robust Fit......Page 608
11.10.6 R Functions ts2str and sm2strv7......Page 609
11.11 ANCOVA......Page 610
11.11.1 Methods Based on Specific Design Points......Page 611
11.11.2 R Functions ancova, ancpb, runmean2g, lplot2g, ancboot,ancbbpb, and cobs2g......Page 614
11.11.3 Multiple Covariates......Page 619
11.11.4 R Functions ancdes, ancovamp, ancmppb, and ancmg......Page 620
11.11.5 Some Global Tests......Page 621
11.12 Marginal Longitudinal Data Analysis: Comments on Comparing Groups......Page 625
11.12.1 R Functions long2g, longreg, longreg.plot, and xyplot......Page 627
11.13 Exercises......Page 628
References......Page 631
I......Page 686
R......Page 687
W......Page 689