References to variables which are not set are left expanded

[JonathonReinhart]

Summary

If .env references a variable (e.g. ${FOO}) which is not already set (either via the external environment, or in the .env file, it is left unexpanded.

This is in contrast to the following other languages / implementations which will expand the variable to an empty string:

• Bash (source)
• Python: python-dotenv

Demo

Consider this .env file:

# A simple, constant env var
CONST='A constant env var'

# Derived from another variable
DERIV="I came from ${CONST} with some more"

# An environment variable we expect to be set on the outside
EXT_SET=${ENV_EXT_SET}

# An environment variable we expect *not* to be set on the outside
EXT_UNSET=${ENV_EXT_UNSET}

When loaded by bash, EXT_UNSET will get the value "" (that is, the empty string).

When loaded by phpdotenv v5.2.0, EXT_UNSET will get the value "${ENV_EXT_UNSET}" (literally).

See this gist for a full reproducible example:

$ pip3 install scuba
$ ./prepare.sh
$ ./run_test.sh
------------------------------------------------------
Bash:
CONST=A constant env var
DERIV=I came from A constant env var with some more
EXT_SET=set externally
EXT_UNSET=

------------------------------------------------------
phpdotenv:
$_ENV = Array
(
    [CONST] => A constant env var
    [DERIV] => I came from A constant env var with some more
    [EXT_SET] => set externally
    [EXT_UNSET] => ${ENV_EXT_UNSET}
)

------------------------------------------------------
python-dotenv:
CONST=A constant env var
DERIV=I came from A constant env var with some more
EXT_SET=set externally
EXT_UNSET=

References

• Upstream issue: docker/docker.env variables not expanded to empty strings snipe/snipe-it#6578

[GrahamCampbell]

This seems to be the intended behaviour, and we even have tests for it.

        $this->assertSame('', $_ENV['NVAR9']);  // nested variable is empty string
        $this->assertSame('${NVAR888}', $_ENV['NVAR10']);  // nested variable is not set

I even back-ported these test to the 2.3 branch, locally, to see if the passed there too, and they do, so it seems this has always been the behaviour.

I think if we want to change this behaviour, we need to consider it to be breaking a change, and not a bug fix, and thus be making the change for the next major release.

[GrahamCampbell]

NB Neither we, nor the Python dotenv package claim to have parity with the bash implementation. We both are just "similar".

[JonathonReinhart]

Thanks for the quick response, @GrahamCampbell. I was initially looking for confirmation that this was either a bug or the intended behavior.

I can't really think of any use case where the current behavior would be desired or useful, but I definitely understand the desire for backwards compatibility and avoiding breaking changes.

Another route would be an optional argument that controls the expansion behavior ("POSIX mode" one might call it). There are other constructs that the shell language supports, too.

I'm just a user of an open source project facing this limitation due to that project's use of Laravel and it's implementation of Docker image. (See referenced issue.) I'm attempting to work around this using envsubst in ths Docker startup script, but obviously that is not ideal:

(set -a; source .env; envsubst < .env > .envnew && mv .envnew .env)

It took a fair amount of effort to track this down (through Snipe-it, through Laravel, finally to phpdotenv) and realize it was the intended behavior. Perhaps the simplest solution here would be just a README change clarifying the behavior when expanding variables?

[GrahamCampbell]

I am not averse to changing the behaviour in the next major version, but just need to be sure there won't be any other unintended side effects. One thing that immediately comes to mind is the nested processing of interpolation.

[GrahamCampbell]

I think, possibly the original reason for this behaviour was because there was no way to write a literal dollar, say, as part of a password, without it being processed as a variable, however, we now have single quote syntax for dealing with this.

[bhushan]

Note: Typing from mobile

Ok lets imagine we agreed on empty string approach.

now

how we handle such things ?

APP_ENV=‘${SOME_VARIABLE}-app’

APP_ENV=“SOME_VARIABLE-app”

how to identify and replace it with empty string ?

[judgej]

I'll just throw this here, in case it raises any interest.

bash allows a default value to be used if the referenced variable is not set or null, like this:

${VARIABLE:-default}

A default empty string would then be ${VARIABLE:-}. That could be implemented as a new feature without it being a breaking change. There may be no interest in this, but I'm only here because I was searching for a solution.