Influence measures in nonparametric regression model with symmetric random errors

Abstract

In this paper we present several diagnostic measures for the class of nonparametric regression models with symmetric random errors, which includes all continuous and symmetric distributions. In particular, we derive some diagnostic measures of global influence such as residuals, leverage values, Cook’s distance and the influence measure proposed by Peña (Technometrics 47(1):1–12, 2005) to measure the influence of an observation when it is influenced by the rest of the observations. A simulation study to evaluate the effectiveness of the diagnostic measures is presented. In addition, we develop the local influence measure to assess the sensitivity of the maximum penalized likelihood estimator of smooth function. Finally, an example with real data is given for illustration.

Appendix

1.1 Hessian matrix

Let ${\mathbf {L}}_{\mathrm {p}}\left( p^{*} \times p^{*}\right)$ be the Hessian matrix with $\left( j^{*}, \ell ^{*}\right)$ -element given by $\partial ^{2} L_{\mathrm {p}}(\varvec{\theta }, \varvec{\alpha }) / \partial \theta _{j^{*}} \theta _{\ell ^{*}}$ for $j^{*}, \ell ^{*}=1, \ldots , p^{*} .$ After some algebraic manipulations we find

$$\begin{aligned} \begin{array}{l}{ \frac{\partial ^{2} L_{\mathrm {p}}(\varvec{\theta }, \varvec{\alpha })}{\partial {\mathbf {f}} \partial {\mathbf {f}}^{T}}=-\frac{1}{\phi } {\mathbf {N}}^{T} {\mathbf {D}}_{a} {\mathbf {N}}-\alpha {\mathbf {K}}} , \\ {\frac{\partial ^{2} L_{\mathrm {p}}(\varvec{\theta }, \varvec{\alpha })}{\partial \phi ^{2}}=\frac{1}{\phi ^{2}}\left\{ \frac{n}{2}+\varvec{\delta }^{T} {\mathbf {D}}_{\zeta ^{\prime }} \varvec{\delta }-\frac{1}{\phi } \varvec{\epsilon }^{T} {\mathbf {D}}_{v} \varvec{\epsilon }\right\} } {\mathrm {and}}\\ {\frac{\partial ^{2} L_{\mathrm {p}}(\varvec{\theta }, \varvec{\alpha })}{\partial {\mathbf {f}} \partial \phi }=\frac{2}{\phi ^{2}} {\mathbf {N}}^{T} {\mathbf {b}}}, \end{array} \end{aligned}$$

where ${\mathbf {D}}_{a}={\mathrm {diag}}_{1 \le i \le n}\left( a_{i}\right) , {\mathbf {D}}_{\zeta ^{\prime }}={\mathrm {diag}}_{1 \le i \le n}\left( \zeta _{i}^{\prime }\right) , {\mathbf {b}}=\left( b_{1}, \ldots , b_{n}\right) ^{T}, \varvec{\delta }=\left( \delta _{1}, \ldots , \delta _{n}\right) ^{T}$, $\varvec{\epsilon } = \left( \epsilon _{1}, \ldots , \epsilon _{n}\right) ^{T}, a_{i}=-2\left( \zeta _{i}+2 \zeta _{i}^{\prime } \delta _{i}\right) , b_{i}=\left( \zeta _{i}+\zeta _{i}^{\prime } \delta _{i}\right) \epsilon _{i}, \epsilon _{i} = \textit{y}_{i} - \mu _{i}, \zeta _{i}^{\prime }=\frac{d \zeta _{i}}{d \delta _{i}}$ and $\delta _{i}=\phi ^{-1} \epsilon _{i}^{2}.$

1.1.1 Cases-weight perturbation

Let us consider cases-weight perturbation for the observations in the penalized log-likelihood function as

$$\begin{aligned} L_{\mathrm {p}}(\varvec{\theta }, \alpha | \varvec{\omega })=\sum _{i=1}^{n} \omega _{i} L_{i}(\varvec{\theta })-\frac{\alpha }{2} {\mathbf {f}}^{T} {\mathbf {K f}}, \end{aligned}$$

where $\varvec{\omega }=\left( \omega _{1}, \ldots , \omega _{n}\right) ^{T}$ is the vector of weights, with $0 \le \omega _{i} \le 1,$ for $i=1, \ldots , n$. In this case, the vector of no perturbation is given by $\varvec{\omega }_{0}=(1, \ldots , 1)^{T}$. Differentiating $L_{\mathrm {p}}(\varvec{\theta }, \alpha | \varvec{\omega })$ with respect to the elements of $\varvec{\theta }$ and $\omega _{i},$ we obtain after some algebraic manipulation

$$\begin{aligned} \left. \frac{\partial ^{2} L_{\mathrm {p}_{i}}(\varvec{\theta }, \alpha | \varvec{\omega })}{\partial {\mathbf {f}} \partial \omega _{i}}\right| _{\theta =\hat{\theta }, \omega =\omega _{0}}= & {} -\frac{2}{\widehat{\phi }} \widehat{\zeta }_{i} \widehat{\epsilon }_{i} {\mathbf {n}}_{i} \quad \text {and}\\ \left. \frac{\partial ^{2} L_{\mathrm {p}_{i}}(\varvec{\theta }, \alpha | \varvec{\omega })}{\partial \phi \partial \omega _{i}}\right| _{\theta =\hat{\theta }, \omega =\omega _{0}}= & {} -\frac{1}{2 \widehat{\phi }}-\frac{1}{2 \widehat{\phi }} \widehat{\zeta }_{i} \widehat{\delta }_{i} , \end{aligned}$$

where $\widehat{\epsilon }_{i} = y_i - \widehat{\mu {_i}}$, for $i=1, \ldots , n$.

1.1.2 Scale perturbation

Under scale parameter perturbation scheme it is assumed that ${\mathrm {y}}_{i} \sim S(\mu _{i}, \omega _{i}^{-1} \phi , g ),$ where $\omega =\left( \omega _{1}, \ldots , \omega _{n}\right) ^{T}$ is the vector of perturbations, with $\omega _{i}>0,$ for $i=1, \ldots , n .$ In this case, the vector of no perturbation is given by $\omega _{0}=(1, \ldots , 1)^{T}$ such that $L_{\mathrm {p}}(\varvec{\theta }, \alpha | \varvec{\omega =\omega _{0}})=L_{\mathrm {p}}(\varvec{\theta }, \alpha ).$ Taking differentials of $L_{\mathrm {P}}(\varvec{\theta }, \alpha | \varvec{\omega })$ with respect to the elements of $\varvec{\theta }$ and $\omega _{i},$ we obtain after some algebraic manipulation

$$\begin{aligned} \frac{ \partial ^{2} L_{\mathrm {p}_{i}} (\varvec{\theta }, \alpha | \varvec{\omega })}{\partial {\mathbf {f}} \partial \omega _{i}}|_{\theta =\hat{\theta }, \omega =\omega _{0}}= & {} -\frac{2}{\widehat{\phi }}\left\{ \widehat{\zeta }_{i}^{\prime } \hat{\delta }_{i}+\widehat{\zeta }_{i}\right\} \widehat{\epsilon }_{i} {\mathbf {n}}_{i}^{T}\\ \frac{\partial ^{2} L_{\mathrm {p}_{i}}(\varvec{\theta }, \alpha | \varvec{\omega })}{\partial \phi \partial \omega _{i}}|_{\theta =\hat{\theta }, \omega =\omega _{0}}= & {} -\frac{1}{2\widehat{\phi }}-\frac{1}{\hat{\phi }}\left\{ \widehat{\zeta }_{i}^{\prime } \hat{\delta }_{i}+\widehat{\zeta }_{i}\right\} \hat{\delta }_{i} , \end{aligned}$$

where $\widehat{\epsilon }_{i} = y_i - \widehat{\mu {_i}}$, for $i=1, \ldots , n$.

1.1.3 Response variable perturbation

To perturb the response variable values we consider $y_{i \omega }=y_{i}+\omega _{i},$ for $i=1, \ldots , n$, where $\omega =\left( \omega _{1}, \ldots , \omega _{n}\right) ^{T}$ is the vector of perturbations. Here, the vector of no perturbation is given by $\varvec{\omega }_{0}=(0, \ldots , 0)^{T}$ and the perturbed penalized log-likelihood function is constructed from (5) with ${\mathrm {y}}_{i}$ replaced by ${\mathrm {y}}_{i \omega },$ that is,

$$\begin{aligned} L_{\mathrm {p}}(\varvec{\theta }, \alpha | \varvec{\omega })=L(\varvec{\theta } | \varvec{\omega })-\frac{\alpha }{2} {\mathbf {f}}^{T} {\mathbf {K}} {\mathbf {f}}, \end{aligned}$$

where $L(\cdot )$ is given by (4) with $\delta _{i \omega }=\phi ^{-1}\left( {\mathrm {y}}_{i \omega }-\mu _{i}\right) ^{2}$ in the place of $\delta _{i} .$ Differentiating $L_{\mathrm {p}}(\varvec{\theta }, \alpha | \varvec{\omega })$ with respect to the elements of $\varvec{\theta }$ and $\omega _{i},$ we obtain, after some algebraic manipulation, that

$$\begin{aligned} \frac{\partial ^{2} L_{\mathrm {p}_{i}}({\varvec{\theta }}, \alpha | {\varvec{\omega }})}{\partial {\mathbf {f}} \partial \omega _{i}}|_{\theta =\hat{\theta }, \omega =\omega _{0}}= & {} -\frac{1}{\widehat{\phi }}\left\{ 2 \widehat{\zeta _{i}}^{\prime } \widehat{\delta }_{i}+2 \widehat{\zeta }_{i} \right\} \quad \text {and} \\ \frac{\partial ^{2} L_{\mathrm {P}_{i}}({\varvec{\theta }}, \alpha | {\varvec{\omega }})}{\partial \phi \partial \omega _{i}}|_{\theta =\hat{\theta }, \omega =\omega _{0}}= & {} -\frac{\widehat{\epsilon }_{i}}{\widehat{\phi }^{2}}\left\{ 2 \widehat{\zeta }_{i}^{\prime } \widehat{\delta }_{i}+2 \widehat{\zeta }_{i}\right\} , \end{aligned}$$

where $\widehat{\epsilon }_{i} = y_i - \widehat{\mu {_i}}$, for $i=1, \ldots , n$.