Versioning supporting library code

[jmgilman]

Hi all,

We've been using tanka/jsonnet for several months now. One issue we've been running into is how to safely change common library code shared between environments without inadvertently breaking other environments. For example, if I have a lib/common.libsonnet file that is imported into all of the environments, and I want to test a single (long-term) change without breaking prod, the only workaround I've found is by making a temporary branch which greatly disturbs our gitops setup.

We use Terraform/TerraGrunt for configuring our infrastructure. One of the nice things about how this is set up is that we can store our Terraform modules in a separate repository. This way, I can change the modules repository, bump the version, and then back in our gitops repo, I can bump the version on a single environment at a time.

I'm wondering if there's a way to emulate this behavior with tanka? I want to be able to version our supporting code and be able to choose which version a specific environment is using selectively. This would give us the same benefit as Terragrunt.

Thoughts?

[Duologic]

You can create a temporary jsonnet mixin that overrides bits from lib/common.libsonnet. It's hard to give advice as I don't know your set up but here goes:

Let's say you have locations in common:

// lib/common.libsonnet
{
  locations: [
    'us-central1'
  ],
}

And your application in dev and prod use this:

// environment/{dev,prod}/main.jsonnet
local common = import 'common.libsonnet';

{
  app: {
    locations: common.locations,
  }
}

Now you want to add another location in in dev, what we can do is add a new
common.libsonnet in the dev environment:

// environments/dev/overrides.libsonnet
local common = import 'common.libsonnet';

common {
  locations+: [
    'us-east1'
  ],
}

And update dev accordingly:

// environment/dev/main.jsonnet
local common = import './overrides.libsonnet';

{
  app: {
    locations: common.locations,
  }
}

Now there is a structure in place to set overrides for dev, once your happy with it, you
can promote the changes to lib/common.libsonnet for all environments.

This can grow quite complex, at Grafana Labs we work with "waves", with 'overries' for
dev/canary/prod. Temporary config changes (short term) are often directly configured on
the environment itself.

[jmgilman]

Interesting. This approach works for overriding parameters, but for my approach I was more thinking about being able to swap out entire sets of functions. For example:

local common = import 'common.libsonnet';
{
  deployment = common.getDeploy(param1, param2, param3),
}

Where common might bring in several additional files:

• /lib/common.libsonnet
• /lib/kube/deploy.jsonnet
• /lib/kube/secrets.jsonnet

During a refactor, all of these files may need to be touched. Presently, this will impact all environments that depend on them. In the world of Terragrunt/Terraform, we could call this collection a module and place it into a separate repository. Then, we could version it at v1.0.0 and version the refactor at v1.1.0, giving us the ability to have prod use one and other environments use the other. Promoting prod is as simple as changing the version it's pointing to.

Does this make sense?

[Duologic]

It's a matter of structuring your code to allow for gradual roll out, with feature flags and optional components this can be accomplished in many ways.