1) How to calculate 100(1-$\alpha$)$\%$ confidence regions of $\theta$?
If we consider the information matrix
\[ I(\theta) = - \frac{\partial^2 Q}{\partial \theta^2 } |_{\theta = \hat{\theta} } \]
where $Q(\theta | \theta^{(t)}) = E_{Z_i | Y_i, \theta^{(t)}} [ L_0(\theta | X) ]$, then it was previously shown a $100(1-\alpha)\%$ confidence interval for $\theta$
\[ [\hat{\theta} - Z_{\alpha/2} (\frac{1}{\sqrt{I(\theta)}}) , \hat{\theta} + Z_{\alpha/2} (\frac{1}{\sqrt{I(\theta)}}) ] \]
If we consider a p-dimensional $\theta$, a 100(1-\alpha)$\%$ likelihood-ratio confidence region would be given by
\[ \{ \theta: 2(l (\hat{\theta}) - l(\theta)) \leq \chi^2_{p, \alpha} \} \]
On the other than a 100(1-\alpha)$\%$ Wald confidence region would given by
\[ \{ \theta: (\theta - \hat{\theta})^{T} V^{-1} (\theta - \hat{\theta}) \leq \chi^2_{p, \alpha} \} \]
where $V^{-1}$ is given by $- Q^{''}(\hat{\theta})$.
2) How to calculate 100(1-$\alpha$)$\%$ confidence regions of $h(\theta)$ where $\frac{\partial h(\theta)}{\partial \theta}$ exists and is non-zero?
For this question, we are interested in confidence regions of $h(\theta)$ which is a q-dimensional where $q \leq p$ Using the Delta Method and asymptotic efficiency of MLEs [see Casella and Berger (2002) Second Edition, pg 473], the variance of $h(\hat{\theta})$ is approximated by
\[ \text{Var}[h(\hat{\theta})] \approx [ \frac{\partial h(\theta)}{\partial \theta} |_{\theta = \hat{\theta} } ]^T [I(\theta) |_{\theta = \hat{\theta} } ]^{-1} [ \frac{\partial h(\theta)}{\partial \theta} |_{\theta = \hat{\theta} } ] \]
Therefore, a 100(1-\alpha)$\%$ Wald confidence region for $h(\theta)$ would given by
\[ \{ \theta: (h(\theta) - h(\hat{\theta}))^{T} V^{-1} (h(\theta) - h(\hat{\theta})) \leq \chi^2_{p, \alpha} \} \]
where
\[ V^{-1} = [\text{Var}(h(\hat{\theta}))]^{-1} \]
For p or q = 2, these confidence regions can easily be plotted using the contour2d() function in R.
