1) How to calculate 100(1-\alpha)\% confidence regions of \theta?
If we consider the information matrix
I(\theta) = - \frac{\partial^2 Q}{\partial \theta^2 } |_{\theta = \hat{\theta} }
where Q(\theta | \theta^{(t)}) = E_{Z_i | Y_i, \theta^{(t)}} [ L_0(\theta | X) ], then it was previously shown a 100(1-\alpha)\% confidence interval for \theta
[\hat{\theta} - Z_{\alpha/2} (\frac{1}{\sqrt{I(\theta)}}) , \hat{\theta} + Z_{\alpha/2} (\frac{1}{\sqrt{I(\theta)}}) ]
If we consider a p-dimensional \theta, a 100(1-\alpha)\% likelihood-ratio confidence region would be given by
\{ \theta: 2(l (\hat{\theta}) - l(\theta)) \leq \chi^2_{p, \alpha} \}
On the other than a 100(1-\alpha)\% Wald confidence region would given by
\{ \theta: (\theta - \hat{\theta})^{T} V^{-1} (\theta - \hat{\theta}) \leq \chi^2_{p, \alpha} \}
where V^{-1} is given by - Q^{''}(\hat{\theta}).
2) How to calculate 100(1-\alpha)\% confidence regions of h(\theta) where \frac{\partial h(\theta)}{\partial \theta} exists and is non-zero?
For this question, we are interested in confidence regions of h(\theta) which is a q-dimensional where q \leq p Using the Delta Method and asymptotic efficiency of MLEs [see Casella and Berger (2002) Second Edition, pg 473], the variance of h(\hat{\theta}) is approximated by
\text{Var}[h(\hat{\theta})] \approx [ \frac{\partial h(\theta)}{\partial \theta} |_{\theta = \hat{\theta} } ]^T [I(\theta) |_{\theta = \hat{\theta} } ]^{-1} [ \frac{\partial h(\theta)}{\partial \theta} |_{\theta = \hat{\theta} } ]
Therefore, a 100(1-\alpha)\% Wald confidence region for h(\theta) would given by
\{ \theta: (h(\theta) - h(\hat{\theta}))^{T} V^{-1} (h(\theta) - h(\hat{\theta})) \leq \chi^2_{p, \alpha} \}
where
V^{-1} = [\text{Var}(h(\hat{\theta}))]^{-1}
For p or q = 2, these confidence regions can easily be plotted using the contour2d() function in R.
