Manage dev and prod using terraform and local yaml files
Manage Dev and Prod Using Terraform and Local yaml FilesMake your terraform spin in multiple projects using yamlIn a corporate production setting, typically cloud projects are split into development (dev) and production (prod) to segregate the two environments. As an engineer, it is important to be able to do the necessary testing and development in (dev) and move them into (prod) once approved, in a quick and efficient manner.Photo by Alvaro Reyes on UnsplashAs expected, there are several solutions to do this, terragrunt is an immediate one that comes to mind. However, not everyone is familiar with terraform, much less terragrunt. How can I simplify this template to enable others in a team simply using terraform and yaml files? Here is my take on a simplified implementation.1. Repository StructureAs a reference for this article, here was how i structured my repository..├── environment # Folder to contain both dev and prod yaml│ ├── dev-config.yaml│ └── prod-config.yaml├── modules # Folder to contain terraform code by services│ ├── module1│ │ └──main.tf│ └── module2│ └── main.tf├── backend.tf # backend file to configure terraform backend├── locals.tf # locals file to point to environment yaml└── main.tf # main file to point to modules2. Create your terraform locals.tfThe locals.tf file is the default way terraform interacts with locally stored files. Your locals.tf should point to the terraform workspace that you would like to work in, this in turn will call the correct config file.# Select the workspaceterraform workspace select dev# This should run terraform plan against 'environment/dev-config.yaml' fileterraform planUsing a local files that can be committed via git is much preferred over maintaining several .tfvars individually across several developers. Please take care not to commit sensitive information in the yaml file.Also terraform does allow for other types of config files other than yaml, including json and text [see link].# filename: locals.tflocals { config = yamldecode(file("environment/${terraform.workspace}-config.yaml"))}2. Create configuration files for both dev and prodIn the environment directory, create your config files. Be sure in your naming that the name is also the same one as your terraform workspace as we will be using the variable `${terraform.workspace}` to differentiate between the two files.# filename: dev-config.yamlproject_id: "cloud-project-dev" terraform_state_bucket_name: "terraform_state_bucket-name"some_dataset_map: dataset_1: description: "dataset_1" dataset_2: description: "dataset_2"# filename: prod-config.yamlproject_id: "cloud-project-prod" terraform_state_bucket_name: "terraform_state_bucket-name"3. ModulesI use module blocks to separate resources used. There is no strict rule as to what goes in a module, but I typically group resources that are spun up for the same service together (e.g. An airflow job that writes to a SQL table, or a VM and its associated load balancers).The main thing here is that your modules point to a local.config in this case the yaml file that we created. The locals.tf should then take care of the rest.# filename: main.tfmodule "module1" { source = "./modules/module1" terraform_state_bucket_name = local.config.terraform_state_bucket_name location = local.config.location}module "module2" { source = "./modules/module2" project_id = local.config.project_id}SourcesTerraform workspaces: linkTerraform locals: linkManage dev and prod using terraform and local yaml files was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.
Welcome to Billionaire Club Co LLC, your gateway to a brand-new social media experience! Sign up today and dive into over 10,000 fresh daily articles and videos curated just for your enjoyment. Enjoy the ad free experience, unlimited content interactions, and get that coveted blue check verification—all for just $1 a month!
Account Frozen
Your account is frozen. You can still view content but cannot interact with it.
Please go to your settings to update your account status.
Open Profile Settings