tl;dr

Applying Ansible playbooks to Google Cloud Platform Compute Engine instances via the google.cloud plugin requires a YAML inventory file, named <hostname>.gcp.yml. If you want a single-host, single-inventory layout, maintaining these by hand is a pain, so I wrote a small python script to generate them from one list of instances. As a bonus it also writes host_vars, so playbooks get per-host metadata without anyone copying it into three places.

intro

Ansible is a great tool for managing infrastructure configuration. It lets you apply playbooks (a list of idempotent tasks that apply common configuration) to hosts, which can be physical machines, virtual machines, or cloud instances.

Using the GCP compute plugin requires a YAML inventory file that tells Ansible how to find hosts, things like:

• projects
• filters
• auth_kind
• hostnames

Yes, typically your inventory in Ansible declares multiple hosts, but for various reasons you may not want to run playbooks against all of them. In idiomatic Ansible you would use the hosts key at runtime to declare the hosts you want, but that gets cumbersome with a large number of hosts, or when you want an extra defensive guardrail: knowing that the inventory you pointed at only declares one host.

Maintaining those files manually is tedious and error prone. Copy one, forget to change the filter, and you are now targeting the wrong VM with a lot of confidence. The script reads a YAML list and generates one inventory file per instance. It can also extract useful metadata into a host_vars file, which playbooks and templates can use later.

inventory file format

So what are we aiming for? The plugin will take this as a minimum viable inventory file:

plugin: google.cloud.gcp_compute
projects:
  - example-project-dev
filters:
  - name = web-server-app-dev
  - status = RUNNING
auth_kind: application
scopes:
  - https://www.googleapis.com/auth/compute.readonly
hostnames:
  - name
keyed_groups:
  - key: zone
    prefix: zone
compose:
  ansible_host: name
  ansible_gcloud_zone: zone
  ansible_gcloud_project: project

It would also be really nice to define host_vars not only for a single host, but for a group of hosts or environments, so we put the following in the inventory too:

groups:
  env_dev: true
  service_app: true

That makes any host in the env_dev group inherit variables from:

• inventory/group_vars/env_dev/*.yml
• inventory/group_vars/service_app/*.yml

which is a great way to keep the inventory DRY and avoid duplication. Environment-wide bits (log levels, backup windows, which apt pocket to use) live on the env group. Service-wide bits (which role, which ports) live on the service group. The per-host file only has to say things that are actually unique to that VM.

host_vars file format

The generated inventory gets the host into the right groups. The generated host_vars file is the other half: values you will template into configs, unit files, nginx vhosts, whatever.

For the same instance, that looks like:

---
project_id: example-project-dev
service_slug: app
service: example-app
env_slug: dev
env: development
site_domain: dev.example.com

This lands at inventory/host_vars/<instance_name>/metadata.yml. Ansible loads it automatically for that host, so a role can just use {{ site_domain }} without copying it into the playbook. Extra per-host keys from the source list get copied through the same way; keep anything environment-wide in group_vars instead.

the source of truth

The only file you should be editing by hand is scripts/inventory/instances.yml. Everything else is generated. It is just a YAML list:

---
# scripts/inventory/instances.yml
# Declarative list of instances for which inventories + host_vars should be generated.

- instance_name: web-server-app-dev
  project_id: example-project-dev
  service_slug: app
  env_slug: dev
  service: example-app
  env: development
  site_domain: dev.example.com

- instance_name: web-server-app-prod
  project_id: example-project-prod
  service_slug: app
  env_slug: prod
  service: example-app
  env: production
  site_domain: example.com

Onboarding a host is: add a list item, run the script, commit the generated files. If a field is the same for every host in an environment, it does not belong here, it belongs in group_vars.

the templates

Two Jinja templates sit next to the script. One becomes inventory/<instance_name>.gcp.yml, the other becomes inventory/host_vars/<instance_name>/metadata.yml.

Inventory template:

plugin: google.cloud.gcp_compute

projects:
  - {{ project_id }}

filters:
  - name = {{ instance_name }}
  - status = RUNNING

auth_kind: application

scopes:
  - https://www.googleapis.com/auth/compute.readonly

hostnames:
  - name

groups:
  env_{{ env_slug }}: true
  service_{{ service_slug }}: true

keyed_groups:
  - key: zone
    prefix: zone

compose:
  ansible_host: name
  ansible_gcloud_zone: zone
  ansible_gcloud_project: project

filters is the whole point of the single-host layout. The plugin can see every instance in the project; these two lines make sure this inventory file can only return this one, and only while it is RUNNING. auth_kind: application means Application Default Credentials, so the same file works on a laptop with gcloud auth application-default login and in CI with a workload identity.

compose copies GCP attributes onto the host as Ansible facts. ansible_host: name is a choice: I target by instance name (and resolve via IAP / SSH config) rather than by ephemeral public IP. If you want the plugin to use the external IP, you would compose ansible_host from networkInterfaces[0].accessConfigs[0].natIP instead.

Host vars template:

---
project_id: {{ project_id }}
service_slug: {{ service_slug }}
service: {{ service }}
env_slug: {{ env_slug }}
env: {{ env }}
site_domain: {{ site_domain }}

the script

Nothing clever. Load the list, skip incomplete rows, render both templates, write the files. It lives at scripts/inventory/engine.py and assumes the repo layout:

scripts/inventory/
  engine.py
  instances.yml
  instance.gcp.yml.j2
  instance_host_vars.yml.j2
inventory/
  <instance_name>.gcp.yml          # generated
  host_vars/<instance_name>/
    metadata.yml                   # generated

#!/usr/bin/env python3
import sys
from pathlib import Path

try:
    import yaml
    from jinja2 import Environment, FileSystemLoader
except ImportError as exc:
    sys.exit(f"Error: {exc.name} is not installed.\nInstall with: pip install pyyaml jinja2")

scripts = Path(__file__).resolve().parent
inventory = scripts.parents[1] / "inventory"
config = scripts / "instances.yml"

if not config.exists():
    sys.exit(f"Error: inventory config not found at {config}")

instances = yaml.safe_load(config.read_text(encoding="utf-8")) or []
if not isinstance(instances, list):
    sys.exit("Error: scripts/inventory/instances.yml must be a YAML list.")

jinja = Environment(loader=FileSystemLoader(scripts), keep_trailing_newline=True)

def render(template, dest, **ctx):
    dest.parent.mkdir(parents=True, exist_ok=True)
    dest.write_text(jinja.get_template(template).render(**ctx), encoding="utf-8")

for item in instances:
    name = str(item.get("instance_name", "")).strip()
    project = str(item.get("project_id", "")).strip()
    if not name or not project:
        print(f"Skipping entry without instance_name/project_id: {item!r}", file=sys.stderr)
        continue

    print(f"Generating files for {name} ({project})")
    render("instance_host_vars.yml.j2", inventory / "host_vars" / name / "metadata.yml", **item)
    render("instance.gcp.yml.j2", inventory / f"{name}.gcp.yml", **item)

print("Generation complete.")

I used Jinja over yaml.dump() so the output stays diffable. A round-trip through PyYAML would reshuffle keys every run, which is a miserable time in code review.

usage

Needs pyyaml and jinja2. Then from the repo root:

pip install pyyaml jinja2
python scripts/inventory/engine.py

You should see one line per instance, then Generation complete. Point Ansible at a single generated file when you want the guardrail:

ansible-playbook -i inventory/web-server-app-dev.gcp.yml playbooks/site.yml

or at the whole inventory/ directory when you actually do want every host. I commit the generated files. CI and anyone else running playbooks should not need Python and Jinja just to know which VM is which. The script is for changing the set of hosts, not for applying configuration.

If you add a field to instances.yml, add it to the host_vars template in the same change. Otherwise you will get a generated file that looks complete and a playbook that has no idea the field exists.

conclusion

This is not a replacement for Ansible's inventory plugins, grouping, or host_vars. It is a thin generator in front of them, so the single-host layout stays cheap. The plugin still does the GCP lookup. Groups still do the sharing. The YAML list is just the bit humans edit.

The failure mode I was trying to kill is "inventory file drifted from reality". One list, two templates, a script that is allowed to be boring.

resources

• google.cloud.gcp_compute inventory plugin
• Ansible inventory, host_vars, and group_vars
• Jinja2 template designer documentation
• Application Default Credentials