CI method name for lmer models?

[burdonmark]

Hi,
I can't find in the documentation what the name is of the default type of CI or PI that are produced for lmer models.
For prediction intervals the documentation options are: "normal" (default), "satterthwaite", and
"kenward".
Other packages talk about "Wald" and "profile" CIs.
Just wondering how to describe it in a research paper.

[bwiernik]

The best approach to use for your models is ci_method = "profile" and to refer to them in your paper as "profile-likelihood confidence intervals".

These intervals are generated using the same method used to fit the model initially--by picking a value for the parameter and computing the likelihood. For the intervals, the parameter is moved away from the maximum likelihood estimate until the model fit worsens to a significant degree. Those values above and below are the confidence interval.

The default method uses normal distribution Wald intervals if the model has a z statistic (GLMMs) and t distribution Wald intervals with residual degrees of freedom if the model has a t statistic (LMMs). These are an approximation (similar to what glmmTMB reports by default). Depending on the structure of your model, they may tend to give overly narrow/optimistic intervals. There are several other methods for approximately degrees of freedom for mixed models that might apply depending on the nature of your model, such as Satterthwaite, Kenward-Roger, between-within, and m-l-1. Each of these makes important strong assumptions about the nature of your model. Personally, I don't care for any of these df approximations and suggest strongly just using profile likelihood to compute CIs.

For additional discussion, see https://bbolker.github.io/mixedmodels-misc/glmmFAQ.html#what-are-the-p-values-listed-by-summaryglmerfit-etc.-are-they-reliable

For random effects SDs and correlations, intervals will be reported if you use profile or boot methods. For the default Wald method, intervals will be reported if you have the {merDeriv} package installed and set ci_random = TRUE. Those intervals are Wald intervals (t or normal following what I say above) computed using log or Fisher z transformation and back-transforming.

[strengejacke]

I think @burdonmark refers to the results from get_predicted() here. And the documentation is indeed a bit vague here. Currently, method "normal" is a Wald t-distribution, using the residual df (i.e. the calculation of degrees of freedoms relies on the default of get_df()), see:

insight/R/get_predicted_ci.R

Lines 231 to 250 in 94b0502

	# data is required for satterthwaite
	if (isTRUE(ci_method %in% c("satterthwaite", "kenward", "kenward-roger"))) {
	dof <- get_df(x, type = ci_method, data = data)
	} else {
	dof <- get_df(x)
	}
	# Return NA
	if (is.null(se)) {
	se <- ci_low <- ci_high <- rep(NA, length(predictions))
	# Get CI
	# TODO: Does this cover all the model families?
	} else {
	# use `all()` because `dof` can be a vector with satterthwaite
	if (is.null(dof) || all(is.infinite(dof)) || find_statistic(x) == "z-statistic") {
	crit_val <- stats::qnorm(p = (1 + ci) / 2)
	} else {
	crit_val <- stats::qt(p = (1 + ci) / 2, df = dof)
	}

Your comment here was the reason for me to open this PR: #648

The goal should be to be more precise. @bwiernik has implemented and documented this for model parameters, see https://easystats.github.io/parameters/reference/model_parameters.html#confidence-intervals-and-approximation-of-degrees-of-freedom

There will be less options for predictions, but once the PR is finished, we can update the docs and add some more options for predictions as well (maybe having options residual, wald, normal for classical freq. regression, and additonally satterthwaite and kenward-roger for linear mixed models).

[strengejacke]

Some of the docs (and the underlying functions) are now more comprehensive and consistent, see:

https://easystats.github.io/insight/reference/get_predicted.html
https://easystats.github.io/insight/reference/get_df.html

And, as already referred to before, the comprehensive documentation here:
https://easystats.github.io/parameters/reference/model_parameters.html#confidence-intervals-and-approximation-of-degrees-of-freedom

I hope that this both clarifies what kind of df/ci-types you get from the easystats-functions, and how to report this in a paper.