Organize GitHub Parameters for Infrastructure as Code

Here are more details on how I organize parameters and variables as a follow-up to my previous post on Infrastructure as Code Patterns.

Remember:

• Domains help me decide where to define variables and when to pass parameters.
• Tools are the processes that use variables and parameters.
• Variables are values that can change between deployments.
• Parameters are variables used to pass information between domains.

Domains have additional areas to consider:

• Framework
  • Command Plane: Variables available to the framework and commands. For example:
    • GitHub Actions: github.run_id, runner.os
    • bicep: managementGroup()
  • Runner Plane: Variables available to the host machine or environment executing the commands. For example:
    • GitHub Actions. GITHUB_RUN_ID available as an OS environment variables.
• Reusabilty
  • Orchestration: Uses resuable modules.
    • GitHub Action caller workflow
    • Azure ARM Parent template
    • Bicep Parent file.
  • Modules: Generic resuable components.
    • GitHub Action reusable workflow
    • Azure ARM linked template
    • bicep module

Domain Paramter Sequence

1. HCI/Trigger - The user or automation triggers the automation orchestration which calls the first child modules which calls the deployment tool.
2. Deployment - The orchestration calls two child modules which call the resource api to create the target resources.
3. Deployment - The automation orchestration calls the second module which calls a deployment with only one child module.

GitHub Variables and Parameters

In the GitHub world there are several places to set variables and parameters.

Mapping GitHub Variables and Parameters to Domains

Other Stuff

| Type | Capability | Domain | Usage | Example | | —– | ——— | —— | —– | ——- | | Environments | Group Custom Variables and Secrets | HCI/Trigger | Configure environments | dev, test, prod, stable, canary, release | | | Input variables | HCI/Trigger | Parameters passed to an action, resuable workflow, or manually triggered workflow. Parameters that change for each deployment. | env, skip_coverage, release_type, profile | | Org - Custom Variables | Global variable | Automation Plane | Org wide variable used between repos and workflows. | DOCKERHUB_REPO, APPLE_DEVELOPER_ID, DEFAULT_REGION, ORG_TENANT_ID | | Org - Secrets | Global Secure variable | Automation Plane | Org wide secure variable used between repos and workflows. | CODE_SCANNING_TOKEN, CERTIFICATE_BASE64 | | Repo - Custom Variables | Global variable | Automation Plane | Repo wide variable used between workflows. | VERSION, CHANGELOG_BRANCH, BASE_FOLDER_PATH, IAC_PATH | | Repo - Secrets | Global Secure variable | Automation Plane | Repo wide secure variable used between workflows. | CODECOV_TOKEN, REGISTRY_PASSWORD, DOCKERHUB_TOKEN, REPO_DISPATCH_TOKEN | | Workflow - Custom Variables | Global variable | Automation Plane | Variable used between workflow jobs. | PYTHON_VERSION, CMAKE_OPTIONS, CUDA_PATH | | Workflow - Secrets | Global secure variable | Automation Plane | Secure variable used between workflow jobs. | | | Job - Custom Variables | Global variable | Automation Plane | Variable used between job steps. | PYTHON_VERSION, CMAKE_OPTIONS, CUDA_PATH | | Job - Secrets | Global secure variable | Automation Plane | Secure variable used between job steps. | | | Step - Custom Variables | Variable | Automation Plane | Variable used for a single step. | CLIENT_SECRET, PYPI_API_TOKEN, AWS_ACCESS_KEY_ID, API_TOKEN | | Step - Secrets | Secure variable | Automation Plane | Secure variable used for a single step. | GITHUB_TOKEN, OAUTH_TOKEN, OPENAI_API_KEY, CLIENT_ID |

IaC Parameter Types

ARM Templates

Bicept

General Best Practices

• Always use parameters for user names, passwords, secrets, and other sensitive information.
• Don’t store secrets in the code or parameter files. Use a secret store.
• Use variables for values that you need to use more than once in a workflow or IaC template.
• Use variables for values that must be unique
• Use variables for complex expressions.
• Remove unused parameters and variables.
• Set variable values in the right domain.
• Use parameters to pass information between domains.
• Avoid outputing sensitive data.

Example Configuration

This example uses the following parameters.

ENV_CONTEXT_VAR
ORGANIZATION_VAR
REPOSITORY_VAR 
JOB_NAME
USE_VARIABLES
RUNNER
ENVIRONMENT_STAGE
OVERRIDE_VAR
HELLO_WORLD_STEP
HELLO_WORLD_ENABLED
GREET_NAME

on:
  workflow_dispatch:
env:
  # Setting an environment variable with the value of a configuration variable
  env_var: $

jobs:
  display-variables:
    name: $
    # You can use configuration variables with the `vars` context for dynamic jobs
    if: $
    runs-on: $
    environment: $
    steps:
    - name: Use variables
      run: |
        echo "repository variable : $REPOSITORY_VAR"
        echo "organization variable : $ORGANIZATION_VAR"
        echo "overridden variable : $OVERRIDE_VAR"
        echo "variable from shell environment : $env_var"
      env:
        REPOSITORY_VAR: $
        ORGANIZATION_VAR: $
        OVERRIDE_VAR: $
        
    - name: $
      if: $
      uses: actions/hello-world-javascript-action@main
      with:
        who-to-greet: $

GitHub Environments are a way to group parameters. This is useful for defining parameters for different environments. For example, you might have a dev, test, and prod environment. Each environment would have different parameters.

References

• Example Workflows https://github.com/supertokens/next.js/tree/canary/.github/workflows
• Example CI Workflow https://github.com/ITJamie/salt/blob/master/.github/workflows/ci.yml
• Example choose environment based on eventy type, branch name https://github.com/Fuzzkatt/pytorch/blob/ec18ef62f44e68e154b939ed4f860a2bda69716d/.github/workflows/_binary-upload.yml#L77
• https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment