Code development platform for open source projects from the European Union institutions

Skip to content
Snippets Groups Projects
Select Git revision
  • main default protected
  • AITED-226_add-rds-instance
  • AITED_fix_glue_crawler_policies
  • AITED-132_dashboard-dns
4 results
Compare
Name Last commit Last update
modules
.gitignore
.gitlab-ci.yml
README.md
dev.tfvars
main.tf
variables.tf
README.md

TEDAI IAC

This project hosts terraform modules to deploy TED AI environment in AWS.

The Gitlab runner in AWS is responsible to deploy the environment updates after a successful Gitlab CI pipeline on main branch.

Requirements

  • Terraform 1.3.9+
  • AWS credentials with administration rights to create the bootstrap (S3 bucket and DynamoDB) to manages Terraform state

Project initialization

This must be done for a new environment only.

The infrastructure needs to be deployed in several phases.

First phase: variables updates

  • Replace the values in dev.tfvars to match your environment:
    • project account id (required)
    • vpc id & vpc cidr (required)
    • all subnets id (required)
    • all the other values related to TED AI (optional)

Second phase: bootstrap the environment for Terraform state

Before being able to use Gitlab CI pipeline to deploy Terraform updates, it is needed to create the configuration to hold the state. This is done with an S3 bucket and a DynamoDB table to handle concurrent execution.

This process is described in the following blogpost.

  • comment the storage and other modules in main.tf
  • comment the backend s3 block in the terraform provider
  • plan and apply using your AWS profile from your local machine, this will create the S3 bucket and DynamoDB tables for the state

Now, it is possible to update the terraform and plan/apply via a Gitlab CI pipeline. The state will be stored in S3 and used by the CI pipeline.

Third phase: create the storage (S3, DynamoDB, Elastic Container Repository, ...)

Once the bootstrap part is successfully done. It is needed to first create the storage that will hold assets for the full infrastructure.

  • uncomment the storage module in main.tf
  • plan and apply using your AWS profile from your local machine, this will create the S3 buckets, ECR repositories, ...

Fourth phase: push satellite projects in ECR, ...

The terraform script of this repository references docker images for the classifiers, the dashboard, and the ingestion/lawfulness application. These images must be pushed to the ECR repositories created in the third phase. The tags need then to be updated in dev.tfvars file.

The following projects need to be deployed:

Once this is done, it is possible to uncomment all modules from main.tf and then deploy. This can either be done via a branch and the CI or directly from a local machine.

Terraform update process

  1. Create a branch from origin/main in Gitlab
  2. Update the modules and commit your changes on the branch. The Gitlab CI will validate the changes and create a plan for you to review
  3. Once validated, create a merge request in Gitlab, review it and merge it
  4. In the CI pipeline of the main branch, a new manual step is available: apply
  5. Check successful completion in apply step in Gitlab CI pipeline

TODO

  • have a fully functional dev/test/prod environment setup, for the moment, only dev is supported.

For further information about the platform and support, please see https://code.europa.eu/info/about