Unable to connect to Google Cloud SQL PostgreSQL instance (private IP) using Go SQL Connector: iam.serviceAccounts.getAccessToke

admin管理员组

文章数量:1122832

I am trying to connect to a PostgreSQL database instance in Google Cloud (with a private IP connection) using the SQL Connector library in Go. However, I am encountering the following error:

ERROR Failed to dial instance {"error": "failed to get instance: Refresh error: failed to get instance metadata (connection name = \"<connection-string>\"): Get \"/\/instances/postgres/connectSettings?alt=json&prettyPrint=false\": compute: Received 403 `Unable to generate access token; IAM returned 403 Forbidden: Permission 'iam.serviceAccounts.getAccessToken' denied on resource (or it may not exist).
This error could be caused by a missing IAM policy binding on the target IAM service account.
For more information, refer to the Workload Identity documentation:

Environment Details:

Google Cloud: PostgreSQL instance with private IP connection enabled

Service Account Permissions:

• Cloud SQL Admin
• Cloud SQL Client
• Compute Network Admin
• Project IAM Admin
• Service Account Admin
• Service Account Token Creator
• Storage Admin
• Workload Identity User

Go Environment: Running the code in a VM instance that is part of the GKE cluster in the same VPC.

Here’s the relevant portion of my Go code:

func connectDB() error {
    user := "postgres"
    password := "password"
    connectionName := "connectionName"
    sslmode := "require"
    dbName := "databaseName"

    config, err := pgx.ParseConfig(connStr)
    if err != nil {
        return err
    }

    opts := append([]cloudsqlconn.Option{},
        cloudsqlconn.WithDefaultDialOptions(cloudsqlconn.WithPrivateIP()),
    )

    d, err := cloudsqlconn.NewDialer(context.Background(), opts...)
    if err != nil {
        return err
    }

    config.DialFunc = func(ctx context.Context, network, instance string) (net.Conn, error) {
        log.FromContext(ctx).Info(fmt.Sprintf("Dialing instance: %s", instance))
        conn, err := d.Dial(ctx, connectionName)
        if err != nil {
            log.FromContext(ctx).Error(err, "Failed to dial instance")
        }
        return conn, err
    }

    dbURI := stdlib.RegisterConnConfig(config)
    dbPool, err := sql.Open("pgx", dbURI)
    if err != nil {
        return err
    }   

    err = dbPool.Ping()
    if err != nil {
        return err
    }   
    return nil
}

Troubleshooting Attempts:

Verified that the database instance has a private IP and is reachable within the same VPC. Checked and ensured that the service account has the following permissions: Cloud SQL Admin Cloud SQL Client Service Account Token Creator Workload Identity User, among others. Ensured that the Go application is running inside a VM that has access to the private VPC.

I am trying to connect to a PostgreSQL database instance in Google Cloud (with a private IP connection) using the SQL Connector library in Go. However, I am encountering the following error:

ERROR Failed to dial instance {"error": "failed to get instance: Refresh error: failed to get instance metadata (connection name = \"<connection-string>\"): Get \"https://sqladmin.googleapis.com/sql/v1beta4/projects/\/instances/postgres/connectSettings?alt=json&prettyPrint=false\": compute: Received 403 `Unable to generate access token; IAM returned 403 Forbidden: Permission 'iam.serviceAccounts.getAccessToken' denied on resource (or it may not exist).
This error could be caused by a missing IAM policy binding on the target IAM service account.
For more information, refer to the Workload Identity documentation:

Environment Details:

Google Cloud: PostgreSQL instance with private IP connection enabled

Service Account Permissions:

• Cloud SQL Admin
• Cloud SQL Client
• Compute Network Admin
• Project IAM Admin
• Service Account Admin
• Service Account Token Creator
• Storage Admin
• Workload Identity User

Go Environment: Running the code in a VM instance that is part of the GKE cluster in the same VPC.

Here’s the relevant portion of my Go code:

func connectDB() error {
    user := "postgres"
    password := "password"
    connectionName := "connectionName"
    sslmode := "require"
    dbName := "databaseName"

    config, err := pgx.ParseConfig(connStr)
    if err != nil {
        return err
    }

    opts := append([]cloudsqlconn.Option{},
        cloudsqlconn.WithDefaultDialOptions(cloudsqlconn.WithPrivateIP()),
    )

    d, err := cloudsqlconn.NewDialer(context.Background(), opts...)
    if err != nil {
        return err
    }

    config.DialFunc = func(ctx context.Context, network, instance string) (net.Conn, error) {
        log.FromContext(ctx).Info(fmt.Sprintf("Dialing instance: %s", instance))
        conn, err := d.Dial(ctx, connectionName)
        if err != nil {
            log.FromContext(ctx).Error(err, "Failed to dial instance")
        }
        return conn, err
    }

    dbURI := stdlib.RegisterConnConfig(config)
    dbPool, err := sql.Open("pgx", dbURI)
    if err != nil {
        return err
    }   

    err = dbPool.Ping()
    if err != nil {
        return err
    }   
    return nil
}

Troubleshooting Attempts:

Verified that the database instance has a private IP and is reachable within the same VPC. Checked and ensured that the service account has the following permissions: Cloud SQL Admin Cloud SQL Client Service Account Token Creator Workload Identity User, among others. Ensured that the Go application is running inside a VM that has access to the private VPC.

• postgresql
• go
• google-cloud-platform
• google-cloud-sql

Share Improve this question Follow asked Nov 22, 2024 at 12:39 piyushpiyush 5311 silver badge33 bronze badges 5

• Are you using workload identity? Recommended way is workload identity cloud.google.com/kubernetes-engine/docs/how-to/… – dany L Commented Nov 22, 2024 at 14:39
• Also are you following this documentation? cloud.google.com/sql/docs/postgres/… – dany L Commented Nov 22, 2024 at 14:48
• @danyL yes I am using the workload identity and using the documentation only. – piyush Commented Nov 23, 2024 at 8:18
• I wonder if you need to gran the VM service account the Service Account Token Creator role instead of the Workload Identity service account. – Jack Wotherspoon Commented Nov 28, 2024 at 15:41
• This may also be useful: googlecloudcommunity.com/gc/Developer-Tools/… – Jack Wotherspoon Commented Nov 28, 2024 at 15:42

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 0

Notice this part of the error you linked: "Permission 'iam.serviceAccounts.getAccessToken' denied on resource (or it may not exist)". The only two roles that you mentioned that have that particular permissions are

• Service Account Token Creator
• Workload Identity User

In the documentation, only workload identity user is mentioned, so I would to try to troubleshoot where that role is mentioned:

1. Follow this best practice if you did not
2. Make sure you complete all the steps as per the doc. Tips: make sure you complete all the required steps, 1 to 6 ( from namespace creation to policy to impersonation to annotation). For step 5, do you get

The member name must include the namespace and Kubernetes ServiceAccount name. For example, serviceAccount:example-project.svc.id.goog[example-namespace/example-serviceaccount].

Other troubleshooting steps you can take for Workload Identity Federation for GKE

Check step 5 (Check that the IAM service account is configured correctly)

gcloud iam service-accounts get-iam-policy \
GSA_NAME@GSA_PROJECT.iam.gserviceaccount.com

Expected:

  - members:
  - serviceAccount:PROJECT_ID.svc.id.goog[NAMESPACE/KSA_NAME]
  role: roles/iam.workloadIdentityUser

本文标签：