Skip to content

Commit 9c77c6c

Browse files
IIBenIIericyz
IIBenII
and
ericyz
authored
feat!: Promote managed_prometheus to GA (#1505)
Co-authored-by: Eric Zhao <zhaoe@google.com>
1 parent f6128b7 commit 9c77c6c

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

44 files changed

+254
-127
lines changed

Diff for: ‎README.md

+4
Original file line numberDiff line numberDiff line change
@@ -148,6 +148,7 @@ Then perform the following commands on the root folder:
148148
| dns\_cache | The status of the NodeLocal DNSCache addon. | `bool` | `false` | no |
149149
| enable\_binary\_authorization | Enable BinAuthZ Admission controller | `bool` | `false` | no |
150150
| enable\_cost\_allocation | Enables Cost Allocation Feature and the cluster name and namespace of your GKE workloads appear in the labels field of the billing export to BigQuery | `bool` | `false` | no |
151+
| enable\_kubernetes\_alpha | Whether to enable Kubernetes Alpha features for this cluster. Note that when this option is enabled, the cluster cannot be upgraded and will be automatically deleted after 30 days. | `bool` | `false` | no |
151152
| enable\_network\_egress\_export | Whether to enable network egress metering for this cluster. If enabled, a daemonset will be created in the cluster to meter network egress traffic. | `bool` | `false` | no |
152153
| enable\_resource\_consumption\_export | Whether to enable resource consumption metering on this cluster. When enabled, a table will be created in the resource export BigQuery dataset to store resource consumption data. The resulting table can be joined with the resource usage table or with BigQuery billing export. | `bool` | `true` | no |
153154
| enable\_shielded\_nodes | Enable Shielded Nodes features on all nodes in this cluster | `bool` | `true` | no |
@@ -168,12 +169,15 @@ Then perform the following commands on the root folder:
168169
| ip\_range\_services | The _name_ of the secondary subnet range to use for services | `string` | n/a | yes |
169170
| issue\_client\_certificate | Issues a client certificate to authenticate to the cluster endpoint. To maximize the security of your cluster, leave this option disabled. Client certificates don't automatically rotate and aren't easily revocable. WARNING: changing this after cluster creation is destructive! | `bool` | `false` | no |
170171
| kubernetes\_version | The Kubernetes version of the masters. If set to 'latest' it will pull latest available version in the selected region. | `string` | `"latest"` | no |
172+
| logging\_enabled\_components | List of services to monitor: SYSTEM\_COMPONENTS, WORKLOADS. Empty list is default GKE configuration. | `list(string)` | `[]` | no |
171173
| logging\_service | The logging service that the cluster should write logs to. Available options include logging.googleapis.com, logging.googleapis.com/kubernetes (beta), and none | `string` | `"logging.googleapis.com/kubernetes"` | no |
172174
| maintenance\_end\_time | Time window specified for recurring maintenance operations in RFC3339 format | `string` | `""` | no |
173175
| maintenance\_exclusions | List of maintenance exclusions. A cluster can have up to three | `list(object({ name = string, start_time = string, end_time = string, exclusion_scope = string }))` | `[]` | no |
174176
| maintenance\_recurrence | Frequency of the recurring maintenance window in RFC5545 format. | `string` | `""` | no |
175177
| maintenance\_start\_time | Time window specified for daily or recurring maintenance operations in RFC3339 format | `string` | `"05:00"` | no |
176178
| master\_authorized\_networks | List of master authorized networks. If none are provided, disallow external access (except the cluster node IPs, which GKE automatically whitelists). | `list(object({ cidr_block = string, display_name = string }))` | `[]` | no |
179+
| monitoring\_enable\_managed\_prometheus | Configuration for Managed Service for Prometheus. Whether or not the managed collection is enabled. | `bool` | `false` | no |
180+
| monitoring\_enabled\_components | List of services to monitor: SYSTEM\_COMPONENTS, WORKLOADS (provider version >= 3.89.0). Empty list is default GKE configuration. | `list(string)` | `[]` | no |
177181
| monitoring\_service | The monitoring service that the cluster should write metrics to. Automatically send metrics from pods in the cluster to the Google Cloud Monitoring API. VM metrics will be collected by Google Compute Engine regardless of this setting Available options include monitoring.googleapis.com, monitoring.googleapis.com/kubernetes (beta) and none | `string` | `"monitoring.googleapis.com/kubernetes"` | no |
178182
| name | The name of the cluster (required) | `string` | n/a | yes |
179183
| network | The VPC network to host the cluster in (required) | `string` | n/a | yes |

Diff for: ‎autogen/main/cluster.tf.tmpl

+15-20
Original file line numberDiff line numberDiff line change
@@ -92,21 +92,31 @@ resource "google_container_cluster" "primary" {
9292
type = var.cluster_telemetry_type
9393
}
9494
}
95+
{% endif %}
96+
{% if autopilot_cluster != true %}
9597
# only one of logging/monitoring_service or logging/monitoring_config can be specified
96-
logging_service = local.cluster_telemetry_type_is_set || local.logmon_config_is_set ? null : var.logging_service
98+
{% if beta_cluster %}
99+
logging_service = local.cluster_telemetry_type_is_set || local.logmon_config_is_set ? null : var.logging_service
100+
{% else %}
101+
logging_service = local.logmon_config_is_set ? null : var.logging_service
102+
{% endif %}
97103
dynamic "logging_config" {
98104
for_each = length(var.logging_enabled_components) > 0 ? [1] : []
99105

100106
content {
101107
enable_components = var.logging_enabled_components
102108
}
103109
}
110+
{% if beta_cluster %}
104111
monitoring_service = local.cluster_telemetry_type_is_set || local.logmon_config_is_set ? null : var.monitoring_service
112+
{% else %}
113+
monitoring_service = local.logmon_config_is_set ? null : var.monitoring_service
114+
{% endif %}
105115
dynamic "monitoring_config" {
106116
for_each = length(var.monitoring_enabled_components) > 0 || var.monitoring_enable_managed_prometheus ? [1] : []
107117

108118
content {
109-
enable_components = length(var.monitoring_enabled_components) > 0 ? var.monitoring_enabled_components : null
119+
enable_components = length(var.monitoring_enabled_components) > 0 ? var.monitoring_enabled_components : []
110120

111121
dynamic "managed_prometheus" {
112122
for_each = var.monitoring_enable_managed_prometheus ? [1] : []
@@ -117,22 +127,6 @@ resource "google_container_cluster" "primary" {
117127
}
118128
}
119129
}
120-
{% else %}
121-
logging_service = var.logging_service
122-
monitoring_service = var.monitoring_service
123-
{% if beta_cluster %}
124-
dynamic "monitoring_config" {
125-
for_each = var.monitoring_enable_managed_prometheus ? [1] : []
126-
127-
content {
128-
managed_prometheus {
129-
enabled = var.monitoring_enable_managed_prometheus
130-
}
131-
}
132-
}
133-
{% endif %}
134-
{% endif %}
135-
{% if autopilot_cluster != true %}
136130
cluster_autoscaling {
137131
enabled = var.cluster_autoscaling.enabled
138132
dynamic "auto_provisioning_defaults" {
@@ -158,7 +152,7 @@ resource "google_container_cluster" "primary" {
158152
}
159153
}
160154
}
161-
{% endif %}
155+
{% endif %}
162156
{% if autopilot_cluster == true %}
163157
cluster_autoscaling {
164158
dynamic "auto_provisioning_defaults" {
@@ -184,9 +178,10 @@ resource "google_container_cluster" "primary" {
184178
}
185179
}
186180

181+
enable_kubernetes_alpha = var.enable_kubernetes_alpha
182+
187183
{% if beta_cluster %}
188184
enable_intranode_visibility = var.enable_intranode_visibility
189-
enable_kubernetes_alpha = var.enable_kubernetes_alpha
190185
enable_tpu = var.enable_tpu
191186

192187
dynamic "pod_security_policy_config" {

Diff for: ‎autogen/main/main.tf.tmpl

+1-1
Original file line numberDiff line numberDiff line change
@@ -102,6 +102,7 @@ locals {
102102
provider = null
103103
}]
104104
cluster_gce_pd_csi_config = var.gce_pd_csi_driver ? [{ enabled = true }] : [{ enabled = false }]
105+
logmon_config_is_set = length(var.logging_enabled_components) > 0 || length(var.monitoring_enabled_components) > 0 || var.monitoring_enable_managed_prometheus
105106
{% endif %}
106107
{% if beta_cluster and autopilot_cluster != true %}
107108
cluster_cloudrun_config_load_balancer_config = (var.cloudrun && var.cloudrun_load_balancer_type != "") ? {
@@ -117,7 +118,6 @@ locals {
117118
] : []
118119
cluster_cloudrun_enabled = var.cloudrun
119120
gke_backup_agent_config = var.gke_backup_agent_config ? [{ enabled = true }] : [{ enabled = false }]
120-
logmon_config_is_set = length(var.logging_enabled_components) > 0 || length(var.monitoring_enabled_components) > 0 || var.monitoring_enable_managed_prometheus
121121
{% endif %}
122122

123123
cluster_authenticator_security_group = var.authenticator_security_group == null ? [] : [{

Diff for: ‎autogen/main/variables.tf.tmpl

+11-11
Original file line numberDiff line numberDiff line change
@@ -651,33 +651,33 @@ variable "timeouts" {
651651
error_message = "Only create, update, delete timeouts can be specified."
652652
}
653653
}
654-
{% if beta_cluster %}
655-
{% if autopilot_cluster != true %}
656654

657-
variable "enable_kubernetes_alpha" {
655+
{% if autopilot_cluster != true %}
656+
variable "monitoring_enable_managed_prometheus" {
658657
type = bool
659-
description = "Whether to enable Kubernetes Alpha features for this cluster. Note that when this option is enabled, the cluster cannot be upgraded and will be automatically deleted after 30 days."
658+
description = "Configuration for Managed Service for Prometheus. Whether or not the managed collection is enabled."
660659
default = false
661660
}
662661

663-
variable "logging_enabled_components" {
662+
variable "monitoring_enabled_components" {
664663
type = list(string)
665-
description = "List of services to monitor: SYSTEM_COMPONENTS, WORKLOADS. Empty list is default GKE configuration."
664+
description = "List of services to monitor: SYSTEM_COMPONENTS, WORKLOADS (provider version >= 3.89.0). Empty list is default GKE configuration."
666665
default = []
667666
}
668667

669-
variable "monitoring_enabled_components" {
668+
variable "logging_enabled_components" {
670669
type = list(string)
671-
description = "List of services to monitor: SYSTEM_COMPONENTS, WORKLOADS (provider version >= 3.89.0). Empty list is default GKE configuration."
670+
description = "List of services to monitor: SYSTEM_COMPONENTS, WORKLOADS. Empty list is default GKE configuration."
672671
default = []
673672
}
674-
{% endif %}
675673

676-
variable "monitoring_enable_managed_prometheus" {
674+
variable "enable_kubernetes_alpha" {
677675
type = bool
678-
description = "(Beta) Configuration for Managed Service for Prometheus. Whether or not the managed collection is enabled."
676+
description = "Whether to enable Kubernetes Alpha features for this cluster. Note that when this option is enabled, the cluster cannot be upgraded and will be automatically deleted after 30 days."
679677
default = false
680678
}
679+
{% endif %}
680+
{% if beta_cluster %}
681681
{% if autopilot_cluster != true %}
682682

683683
variable "istio" {

Diff for: ‎autogen/main/versions.tf.tmpl

+1-1
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ terraform {
2424
required_providers {
2525
google-beta = {
2626
source = "hashicorp/google-beta"
27-
version = ">= 4.45.0, < 5.0"
27+
version = ">= 4.46.0, < 5.0"
2828
}
2929
kubernetes = {
3030
source = "hashicorp/kubernetes"

Diff for: ‎cluster.tf

+27-2
Original file line numberDiff line numberDiff line change
@@ -71,8 +71,31 @@ resource "google_container_cluster" "primary" {
7171

7272
min_master_version = var.release_channel == null || var.release_channel == "UNSPECIFIED" ? local.master_version : null
7373

74-
logging_service = var.logging_service
75-
monitoring_service = var.monitoring_service
74+
# only one of logging/monitoring_service or logging/monitoring_config can be specified
75+
logging_service = local.logmon_config_is_set ? null : var.logging_service
76+
dynamic "logging_config" {
77+
for_each = length(var.logging_enabled_components) > 0 ? [1] : []
78+
79+
content {
80+
enable_components = var.logging_enabled_components
81+
}
82+
}
83+
monitoring_service = local.logmon_config_is_set ? null : var.monitoring_service
84+
dynamic "monitoring_config" {
85+
for_each = length(var.monitoring_enabled_components) > 0 || var.monitoring_enable_managed_prometheus ? [1] : []
86+
87+
content {
88+
enable_components = length(var.monitoring_enabled_components) > 0 ? var.monitoring_enabled_components : []
89+
90+
dynamic "managed_prometheus" {
91+
for_each = var.monitoring_enable_managed_prometheus ? [1] : []
92+
93+
content {
94+
enabled = var.monitoring_enable_managed_prometheus
95+
}
96+
}
97+
}
98+
}
7699
cluster_autoscaling {
77100
enabled = var.cluster_autoscaling.enabled
78101
dynamic "auto_provisioning_defaults" {
@@ -105,6 +128,8 @@ resource "google_container_cluster" "primary" {
105128
}
106129
}
107130

131+
enable_kubernetes_alpha = var.enable_kubernetes_alpha
132+
108133
dynamic "master_authorized_networks_config" {
109134
for_each = local.master_authorized_networks_config
110135
content {

Diff for: ‎main.tf

+1
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,7 @@ locals {
8484
provider = null
8585
}]
8686
cluster_gce_pd_csi_config = var.gce_pd_csi_driver ? [{ enabled = true }] : [{ enabled = false }]
87+
logmon_config_is_set = length(var.logging_enabled_components) > 0 || length(var.monitoring_enabled_components) > 0 || var.monitoring_enable_managed_prometheus
8788

8889
cluster_authenticator_security_group = var.authenticator_security_group == null ? [] : [{
8990
security_group = var.authenticator_security_group

Diff for: ‎modules/beta-autopilot-private-cluster/README.md

-1
Original file line numberDiff line numberDiff line change
@@ -111,7 +111,6 @@ Then perform the following commands on the root folder:
111111
| master\_authorized\_networks | List of master authorized networks. If none are provided, disallow external access (except the cluster node IPs, which GKE automatically whitelists). | `list(object({ cidr_block = string, display_name = string }))` | `[]` | no |
112112
| master\_global\_access\_enabled | Whether the cluster master is accessible globally (from any region) or only within the same region as the private endpoint. | `bool` | `true` | no |
113113
| master\_ipv4\_cidr\_block | (Beta) The IP range in CIDR notation to use for the hosted master network | `string` | `"10.0.0.0/28"` | no |
114-
| monitoring\_enable\_managed\_prometheus | (Beta) Configuration for Managed Service for Prometheus. Whether or not the managed collection is enabled. | `bool` | `false` | no |
115114
| monitoring\_service | The monitoring service that the cluster should write metrics to. Automatically send metrics from pods in the cluster to the Google Cloud Monitoring API. VM metrics will be collected by Google Compute Engine regardless of this setting Available options include monitoring.googleapis.com, monitoring.googleapis.com/kubernetes (beta) and none | `string` | `"monitoring.googleapis.com/kubernetes"` | no |
116115
| name | The name of the cluster (required) | `string` | n/a | yes |
117116
| network | The VPC network to host the cluster in (required) | `string` | n/a | yes |

Diff for: ‎modules/beta-autopilot-private-cluster/cluster.tf

-11
Original file line numberDiff line numberDiff line change
@@ -69,17 +69,6 @@ resource "google_container_cluster" "primary" {
6969

7070
min_master_version = var.release_channel == null || var.release_channel == "UNSPECIFIED" ? local.master_version : null
7171

72-
logging_service = var.logging_service
73-
monitoring_service = var.monitoring_service
74-
dynamic "monitoring_config" {
75-
for_each = var.monitoring_enable_managed_prometheus ? [1] : []
76-
77-
content {
78-
managed_prometheus {
79-
enabled = var.monitoring_enable_managed_prometheus
80-
}
81-
}
82-
}
8372
cluster_autoscaling {
8473
dynamic "auto_provisioning_defaults" {
8574
for_each = var.create_service_account ? [1] : []

Diff for: ‎modules/beta-autopilot-private-cluster/variables.tf

-5
Original file line numberDiff line numberDiff line change
@@ -423,8 +423,3 @@ variable "timeouts" {
423423
}
424424
}
425425

426-
variable "monitoring_enable_managed_prometheus" {
427-
type = bool
428-
description = "(Beta) Configuration for Managed Service for Prometheus. Whether or not the managed collection is enabled."
429-
default = false
430-
}

Diff for: ‎modules/beta-autopilot-private-cluster/versions.tf

+1-1
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ terraform {
2121
required_providers {
2222
google-beta = {
2323
source = "hashicorp/google-beta"
24-
version = ">= 4.45.0, < 5.0"
24+
version = ">= 4.46.0, < 5.0"
2525
}
2626
kubernetes = {
2727
source = "hashicorp/kubernetes"

Diff for: ‎modules/beta-autopilot-public-cluster/README.md

-1
Original file line numberDiff line numberDiff line change
@@ -100,7 +100,6 @@ Then perform the following commands on the root folder:
100100
| maintenance\_recurrence | Frequency of the recurring maintenance window in RFC5545 format. | `string` | `""` | no |
101101
| maintenance\_start\_time | Time window specified for daily or recurring maintenance operations in RFC3339 format | `string` | `"05:00"` | no |
102102
| master\_authorized\_networks | List of master authorized networks. If none are provided, disallow external access (except the cluster node IPs, which GKE automatically whitelists). | `list(object({ cidr_block = string, display_name = string }))` | `[]` | no |
103-
| monitoring\_enable\_managed\_prometheus | (Beta) Configuration for Managed Service for Prometheus. Whether or not the managed collection is enabled. | `bool` | `false` | no |
104103
| monitoring\_service | The monitoring service that the cluster should write metrics to. Automatically send metrics from pods in the cluster to the Google Cloud Monitoring API. VM metrics will be collected by Google Compute Engine regardless of this setting Available options include monitoring.googleapis.com, monitoring.googleapis.com/kubernetes (beta) and none | `string` | `"monitoring.googleapis.com/kubernetes"` | no |
105104
| name | The name of the cluster (required) | `string` | n/a | yes |
106105
| network | The VPC network to host the cluster in (required) | `string` | n/a | yes |

Diff for: ‎modules/beta-autopilot-public-cluster/cluster.tf

-11
Original file line numberDiff line numberDiff line change
@@ -69,17 +69,6 @@ resource "google_container_cluster" "primary" {
6969

7070
min_master_version = var.release_channel == null || var.release_channel == "UNSPECIFIED" ? local.master_version : null
7171

72-
logging_service = var.logging_service
73-
monitoring_service = var.monitoring_service
74-
dynamic "monitoring_config" {
75-
for_each = var.monitoring_enable_managed_prometheus ? [1] : []
76-
77-
content {
78-
managed_prometheus {
79-
enabled = var.monitoring_enable_managed_prometheus
80-
}
81-
}
82-
}
8372
cluster_autoscaling {
8473
dynamic "auto_provisioning_defaults" {
8574
for_each = var.create_service_account ? [1] : []

Diff for: ‎modules/beta-autopilot-public-cluster/variables.tf

-5
Original file line numberDiff line numberDiff line change
@@ -393,8 +393,3 @@ variable "timeouts" {
393393
}
394394
}
395395

396-
variable "monitoring_enable_managed_prometheus" {
397-
type = bool
398-
description = "(Beta) Configuration for Managed Service for Prometheus. Whether or not the managed collection is enabled."
399-
default = false
400-
}

Diff for: ‎modules/beta-autopilot-public-cluster/versions.tf

+1-1
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ terraform {
2121
required_providers {
2222
google-beta = {
2323
source = "hashicorp/google-beta"
24-
version = ">= 4.45.0, < 5.0"
24+
version = ">= 4.46.0, < 5.0"
2525
}
2626
kubernetes = {
2727
source = "hashicorp/kubernetes"

Diff for: ‎modules/beta-private-cluster-update-variant/README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -227,7 +227,7 @@ Then perform the following commands on the root folder:
227227
| master\_authorized\_networks | List of master authorized networks. If none are provided, disallow external access (except the cluster node IPs, which GKE automatically whitelists). | `list(object({ cidr_block = string, display_name = string }))` | `[]` | no |
228228
| master\_global\_access\_enabled | Whether the cluster master is accessible globally (from any region) or only within the same region as the private endpoint. | `bool` | `true` | no |
229229
| master\_ipv4\_cidr\_block | (Beta) The IP range in CIDR notation to use for the hosted master network | `string` | `"10.0.0.0/28"` | no |
230-
| monitoring\_enable\_managed\_prometheus | (Beta) Configuration for Managed Service for Prometheus. Whether or not the managed collection is enabled. | `bool` | `false` | no |
230+
| monitoring\_enable\_managed\_prometheus | Configuration for Managed Service for Prometheus. Whether or not the managed collection is enabled. | `bool` | `false` | no |
231231
| monitoring\_enabled\_components | List of services to monitor: SYSTEM\_COMPONENTS, WORKLOADS (provider version >= 3.89.0). Empty list is default GKE configuration. | `list(string)` | `[]` | no |
232232
| monitoring\_service | The monitoring service that the cluster should write metrics to. Automatically send metrics from pods in the cluster to the Google Cloud Monitoring API. VM metrics will be collected by Google Compute Engine regardless of this setting Available options include monitoring.googleapis.com, monitoring.googleapis.com/kubernetes (beta) and none | `string` | `"monitoring.googleapis.com/kubernetes"` | no |
233233
| name | The name of the cluster (required) | `string` | n/a | yes |

Diff for: ‎modules/beta-private-cluster-update-variant/cluster.tf

+3-2
Original file line numberDiff line numberDiff line change
@@ -97,7 +97,7 @@ resource "google_container_cluster" "primary" {
9797
for_each = length(var.monitoring_enabled_components) > 0 || var.monitoring_enable_managed_prometheus ? [1] : []
9898

9999
content {
100-
enable_components = length(var.monitoring_enabled_components) > 0 ? var.monitoring_enabled_components : null
100+
enable_components = length(var.monitoring_enabled_components) > 0 ? var.monitoring_enabled_components : []
101101

102102
dynamic "managed_prometheus" {
103103
for_each = var.monitoring_enable_managed_prometheus ? [1] : []
@@ -142,8 +142,9 @@ resource "google_container_cluster" "primary" {
142142
}
143143
}
144144

145+
enable_kubernetes_alpha = var.enable_kubernetes_alpha
146+
145147
enable_intranode_visibility = var.enable_intranode_visibility
146-
enable_kubernetes_alpha = var.enable_kubernetes_alpha
147148
enable_tpu = var.enable_tpu
148149

149150
dynamic "pod_security_policy_config" {

0 commit comments

Comments
 (0)