Google Cloud Platform (GCP) continues to expand its offerings and remains a top-tier cloud infrastructure service provider. It offers a wide range of services, including Compute Engine (VMs), Google Kubernetes Engine (GKE), BigQuery, Cloud Storage, and Cloud AI Platform. However, in recent years, several vulnerabilities have been reported in some of GCP's services, such as privilege escalation in GKE and CloudSQL.

As the adoption of GCP grows and more organizations choose it as their cloud infrastructure, it is essential to understand how to conduct forensic investigations and analyze logs in GCP. A key aspect of any investigation is to determine the principals and identities involved in the incident.

In this blog post, we'll explain how to identify principles and understand the identities present in GCP logs, including impersonation and third-party identities.

The first part of this two-part blog series will cover the logging system in Google Cloud Platform (GCP) and its fields. We will also provide a comprehensive overview of the various identity types in GCP, along with a detailed explanation of service accounts and impersonations within the platform.

  • Overview of GCP Logs
  • Identity Types in GCP: the Different Identities Available
  • authenticationInfo in GCP Logs: Understanding Who, What, and Where
  • Service Accounts: An Introduction
  • How to Use Service Accounts: Authentication and Impersonation

Overview of GCP Logs

GCP has several logging categories, such as Cloud Audit Logs, Service Logs, Agent Logs and Network Logs.

Different logs in GCP

Cloud Audit Logs:

Audit logs answer the question many various cloud audit logs answer and that is “who did what, where, and when?”

  • Admin Activity: contain log entries for API calls of configuration changes or changes to resource metadata.
  • System Event: contain log entries of actions altering resource configurations, specifically non-human administrative actions, generated by Google systems.
  • Data Access: contain log entries of API calls that access resource configuration or metadata, along with user-initiated calls that create, modify, or read user-provided resource data.
  • Access Transparency: contain log entries of actions taken by Google personnel when accessing your content.

Service Logs:

Service logs are logs that developers can create for deploying code to Google Cloud. For example, when deploying a web application to Google App Engine, logging statements in the application code are captured and sent to Cloud Logging.

Agent Logs: 

Agent logs in Google Cloud Platform (GCP) are logs generated by agents running on virtual machines, like the Ops Agent or legacy Cloud Logging agent (fluentd), which collect and send data from known file locations or logging services to Cloud Logging for monitoring and management.

Network Logs: 

Network logs provide detailed telemetry for network and security operations and can be listed in these logs:

  • VPC Flow: contain entries of VPC network flows and can be used for network monitoring, security analysis and forensic investigations related to attacks such as Denial of Service (DoS), Port Scanning, Reconnaissance, Data Exfiltration, and other security incidents.
  • NAT Gateway: contain information on NAT network connections and errors.
  • Firewall Rules: contain logs that track and analyze the effects of firewall rule on network traffic.

With a better understanding of the available logs in GCP, this blog post will focus on the Audit Logs, specifically the Admin Activity and Data Access logs.

Identity Types in GCP: the Different Identities Available

Before discussing logging that assists forensic investigators in identifying the principals behind an operation, it's important to understand and map out the entities existing in GCP.

Different entities in GCP

In GCP, four main entities operate. The following sections will discuss each of these identities individually. Here's a brief overview:

  • User: A user account that interacts with GCP resources(human users). Typically authenticated using Google accounts or other identity providers integrated with Google Cloud Identity and Access Management (IAM).
  • Service Accounts: Accounts intended for non-human entities, like applications and virtual machines to access Google Cloud services programmatically. Authenticate with a private/public key pair.
  • Service Agents: - Google-managed service accounts used by some Google Cloud services to access user resources. These service agents are created by Google when necessary and are visible in a project's allow policy and audit logs, but not in the service account section.
  • Workload Identities: - Used in Google Kubernetes Engine (GKE) for workloads to authenticate and authorize with other Google services without manual management of service account keys. They can also be used with other Identity Providers (IDPs) and cloud providers like AWS (Amazon Web Services) for secure cross-platform communication.

AuthenticationInfo in GCP Logs: Understanding Who, What, and Where

Audit logs, which include Admin Activity and Data Access logs, record the identity of the entity performing operations on a Google Cloud resource. The caller's identity is stored in the authenticationInfo field of the AuditLog objects.

The AuthenticationInfo field contains details such as the authentication method used and the email address or service account key associated with the caller, providing details about the authentication context for the logged operation.

To understand the importance of the fields of the authenticationInfo field, it would be beneficial to understand its structure and fields. Google’s documentation provides detailed information on this.

An example of a full possible occurrence of an authenticationInfo:

AuthenticationInfo structure

A description of the fields (from Google’s documentation):

Field Meaning and type
principalEmail
string

The email address of the authenticated user (or service account on behalf of third-party principal) making the request. For third party identity callers, the principalSubject field is populated instead of this field. For privacy reasons, the principal email address is sometimes redacted. For more information, see Caller identities in audit logs.
authoritySelector
string

The authority selector specified by the requestor, if any. It is not guaranteed that the principal was allowed to use this authority.
thirdPartyPrincipal
object (Struct format)

The third-party identification (if any) of the authenticated user making the request. When the JSON object represented here has a proto equivalent, the proto name will be indicated in the @type property.
serviceAccountKeyName
string

The name of the service account key used to create or exchange credentials for authenticating the service account making the request. This is a scheme-less URI full resource name.
For example: "//iam.googleapis.com/projects/{PROJECT_ID}/serviceAccounts/{ACCOUNT}/keys/{key}"
serviceAccountDelegationInfo[]
object (ServiceAccountDelegationInfo)

Identity delegation history of an authenticated service account that makes the request. It contains information on the real authorities that try to access GCP resources by delegating on a service account. When multiple authorities present, they are guaranteed to be sorted based on the original ordering of the identity delegation events.
principalSubject
string

String representation of identity of requesting party. Populated for both first- and third-party identities.

It’s apparent that there are two additional structs in the authenticationInfo field: serviceAccountDelegationInfo and thirdPartyPrincipal.

The serviceAccountDelegationInfo is related to service account delegation which will be covered later on in this blog post.

thirdPartyPrincipal is used to track authentication from entities outside of Google's services, such as external identity providers (if applicable).

An example of a serviceAccountDelegationInfo:

ServiceAccountDelegationInfo structure

A description of the fields (from Google’s documentation):

Field Meaning and type
principalSubject
string

A string representing the principalSubject associated with the identity. For most identities, the format will be principal://iam.googleapis.com/{identity pool name}/subject/{subject) except for some GKE identities (GKE_WORKLOAD, FREEFORM, GKE_HUB_WORKLOAD) that are still in the legacy format serviceAccount:{identity pool name}[{subject}]
Union field Authority. Entity that creates credentials for service account and assumes its identity for authentication. Authority can be only one of the following:
firstPartyPrincipal
object (FirstPartyPrincipal)

First party (Google) identity as the real authority.
thirdPartyPrincipal
object (ThirdPartyPrincipal)

Third party identity as the real authority.

A deeper look into the firstPartyPrincipal and thirdPartyPrincipal:

FirstPartyPrincipal
Field Meaning and type
principalEmail
string

The email address of a Google account.
serviceMetadata
object (Struct format)

Metadata about the service that uses the service account.
ThirdPartyPrincipal
Field Meaning and type
thirdPartyClaims
object (Struct format)

Metadata about third party identity.

After examining the authenticationInfo field in detail, the next step is to review the implementation of Service Accounts.

Service Accounts: An Introduction

As mentioned, one of the entities in GCP is Service Accounts.

Service Accounts provides an identity for carrying out service to service interactions in a project without supplying credentials.

An example usage of a service account can be writing an application running on a Compute Engine VM instance, and said application needs to read a file from Cloud Storage. A service account with the correct role (set of permissions, given the write scope) can be assigned to the instance. Then the application running on the instance would be able to use the service account without embedding any secret keys or credentials in the instance or application code.

Service accounts can be identified with a Google-managed email with the prefix of gserviceaccount.com.

There are three types of service accounts:

  • User created (custom)
  • Built-in service accounts
  • Google managed

User created (custom):

Users can create a service account to meet their specific needs. When managing a service account, users must provide user-managed keypairs for authentication, with Google storing only the public portion.

Up to 10 user-managed service account keys per service can be createdand managed using the IAM API, console, or gcloud.

These service accounts are considered as user managed service accounts.

Built-in service accounts:

Built-in service accounts which are a subset of user managed service accounts, are created automatically when enabling certain Google Cloud services. These service accounts have key-pairs managed by Google for both the private and public keys. The public key can be used for signing for up to two weeks, while the private key is securely held in escrow and is not directly accessible.

The Default Compute Engine service account is an example of a Built-in service account. It is automatically created for each project with an auto-generated name and email address, such as 3xxxxxxx5-compute@developer.gserviceaccount.com. This service account is automatically assigned the Editor role for the project and is enabled by default on all instances created using gcloud or the console. When starting a new instance with gcloud, the default service account is automatically enabled. Changing this default setting is possible by specifying a different service account or disabling service accounts for the instance.

Google managed:

Google-managed service accounts, like the Google APIs Service Agent (identified as <project_number>@cloudservices.gserviceaccount.com),are created and managed by Google for service management purposes. They are commonly used to support service functionality (and additionally run operations on behalf of services or users). These accounts, also known as service agents, are visible in IAM policies and audit logs but are not listed in the service account section. We will explore these accounts further in the next part of the blog.

How to Use Service Accounts: Authentication, Impersonation and How to Find Them in the Logs

Using a service account can be achieved by either activating the service account or impersonating it.

Service Account Activation:

When activating a service account with gcloud, you authorize gcloud and other tools in the Google Cloud CLI to use the service account's credentials for making requests to Google Cloud services. This activation process imports the service account's credentials from a file containing a private authorization key (JSON or p12) and activates them for use with gcloud. 

The activation imports the service accounts credentials from a file containing a private authorization key (JSON or p12) and activates it for gcloud in this case.

Activating a service account is like logging in with a user account (using gcloud auth login) but using a service account instead.

An example of an activation in gcloud:

gcloud auth activate-service-account <SERVICE_ACCOUNT>@<DOMAIN>.COM --key-file=<path_to_key_file> --project=<PROJECT_ID>

Service Account Impersonation:

Impersonation for service accounts allows users to delegate permissions for a service account to act on behalf of another user or service account. This enables the service account to access resources and perform actions requiring specific permissions usually associated with the user or service account being impersonated. It enables secure usage of applications or users, ensuring they have the necessary permissions without revealing sensitive information or requiring significant code modifications.

Some of Google’s documentation on impersonation:

“When an authenticated principal, such as a user or another service account, authenticates as a service account to gain the service account's permissions, it's called impersonating the service account. Impersonating a service account lets an authenticated principal access whatever the service account can access. Only authenticated principals with the appropriate permissions can impersonate service accounts.
Impersonation is useful when you want to change a user's permissions without changing your Identity and Access Management (IAM) policies. For example, you can use impersonation to temporarily grant a user elevated access, or to test whether a specific set of permissions is sufficient for a task. You can also use impersonation to locally develop applications that can only run as a service account, or to authenticate applications that run outside of Google Cloud.”

Why impersonate?

Legitimate service account impersonations in GCP can have use cases such as:

  • Delegated access: When a service account needs to access resources that it wouldn't normally have access to, such as when a service account needs to access resources in a different project.
  • Automation: Service accounts are used to automate tasks, and impersonation allows one service account to perform actions on behalf of another, simplifying access management.
  • Access to external services: A service account might need to access external services that require a specific identity for authentication purposes.

On the other hand, malicious actors might use impersonation for unauthorized access to resources or to bypass security controls. Some examples of malicious use cases include:

  • Data exfiltration: A malicious actor might impersonate a service account to access and exfiltrate sensitive data.
  • Privilege escalation: Impersonation can be used to escalate privileges and gain access to resources that are restricted to higher privileged accounts.
  • Covering tracks: Impersonation can be used to perform malicious activities while masking the identity of the attacker, making it harder to trace the actions back to the original source.

Impersonation can take two forms:

  • Impersonation through service activation
  • Impersonation on a specific invocation

Impersonation through service account activation

This involves logging in with a service account (and activating it) using credentials from a file as mentioned earlier. It allows gcloud to use the service account credentials for requests, effectively setting the environment to that of the service account we logged in as.

An example of such an impersonation:

gcloud auth activate-service-account --key-file= <path_to_key_file> --impersonate-service-account=<SERVICE_ACCOUNT_EMAIL>

Impersonation through service account activation: How to identify it in the logs?

Under this specific impersonation, the logs will display the impersonated service account as the principalEmail in the authenticationInfo field. Additionally, the serviceAccountKeyName in the logs indicates authentication was done using a key for activation.

Impersonation with SA activation

The credentials loaded can be revoked using gcloud auth revoke.

Impersonation on a specific invocation

Specific invocation impersonation enables impersonation for a specific action, granting temporary access to resources as the impersonated service account without impacting the broader environment. This approach allows for impersonation without the need to create or activate a key. Additionally, it supports creating a chain of service accounts for delegation chain impersonation.

To perform this kind of impersonation, a user or service account must have the necessary permissions. These permissions are listed in a role named Service Account Token Creator, which includes the following permissions:

iam.serviceAccounts.get
iam.serviceAccounts.getAccessToken
iam.serviceAccounts.getOpenIdToken
iam.serviceAccounts.implicitDelegation
iam.serviceAccounts.list
iam.serviceAccounts.signBlob
iam.serviceAccounts.signJwt
resourcemanager.projects.get
resourcemanager.projects.list

These permissions are necessary for impersonation because the account initiating the impersonation uses the GenerateAccessToken method to obtain a temporary token for the gcloud operation they wish to perform.

Specific invocation impersonation using GetAccessToken

The user who generated the access token is specified in authenticationInfo.PrincipalEmail, along with details of the service account for which the token was created, found in request.name and resource.labels.email_id.

A little about impersonation with delegation chains

Some information provided from Google’s documentation:

“For this gcloud invocation, all API requests will be made as the given service account or target service account in an impersonation delegation chain instead of the currently selected account. You can specify either a single service account as the impersonator, or a comma-separated list of service accounts to create an impersonation delegation chain. The impersonation is done without needing to create, download, and activate a key for the service account or accounts.
To make API requests as a service account, your currently selected account must have an IAM role that includes the iam.serviceAccounts.getAccessToken permission for the service account or accounts.
The roles/iam.serviceAccountTokenCreator role has the iam.serviceAccounts.getAccessToken permission. You can also create a custom role.
You can specify a list of service accounts, separated with commas. This creates an impersonation delegation chain in which each service account delegates its permissions to the next service account in the chain. Each service account in the list must have the roles/iam.serviceAccountTokenCreator role on the next service account in the list. For example, when --impersonate-service-account= SERVICE_ACCOUNT_1,SERVICE_ACCOUNT_2, the active account must have the roles/iam.serviceAccountTokenCreator role on SERVICE_ACCOUNT_1, which must have the roles/iam.serviceAccountTokenCreator role on SERVICE_ACCOUNT_2. SERVICE_ACCOUNT_1 is the impersonated service account and SERVICE_ACCOUNT_2 is the delegate.”

An example command using an impersonation with a delegation chain:

gcloud storage buckets list --impersonate-service-account=[SA_email](or comma separated emails -> SA1,SA2,SA3)

In delegation chains with multiple service accounts, each entity in the chain generates an access token using GenerateAccessToken. The chain is shown in metadata.identityDelegationChain as an array, with each entity creating a token resulting in a shorter chain.

GenerateAccessToken with the delegation chain

When executing the gcloud operation within the impersonation chain, the operation appears as if it was performed by the last service account in the chain. However, the entire delegation chain can be viewed in authenticationInfo.serviceAccountDelegationInfo.[<item location>].firstPartyPrincipal.principalEmail, with the final object indicating the original user who initiated the impersonation chain (essentially, the chain in reverse).

Delegation chain impersonation – last SA in the chain is shown as actor

Authorization levels when impersonating

When impersonating a service account using GenerateAccessToken, two authorization levels can be observed based on the delegation chain. For single impersonations, the permission scope is iam.serviceAccounts.getAccessToken:

GenerateAccessToken with a single impersonation in the chain (SA1 impersonates SA2)

When impersonating with a longer delegation chain, the permission scope is iam.serviceAccounts.implicitDelegation:

GenerateAccessToken with multiple delegations in the chain

Default Compute Engine service account delegation

As previously mentioned, Google Compute Engine instances use the Default Compute Engine service account by default. However, it's possible to attach a different service account to a VM. When a specific service account is attached to a VM, operations from that VM will be seen as delegated operations originating from the Default Compute Engine service account. The service account attached to the VM can be identified in authenticationInfo.principalEmail, while the delegation from the Default Compute Engine service account can be found in authenticationInfo.serviceAccountDelegationInfo.firstPartyPrincipal.principalEmail.

Operations form the SA attached to the VM delegated by the default Compute Engine SA

Summary

In this first part of our two-part blog series, we've covered the basics of GCP logs for forensic investigations, the different identities in GCP, and a detailed exploration of how service accounts are used, impersonated, and how to analyze log components.

The second part will explore the remaining identities and instances where GCP does not log identities.

LAST UPDATED:

April 20, 2024

Don't miss these stories:

5 Common Threat Actor Tactics Used in Cloud, Identity, and SaaS Attacks

Explore five common tactics used in cloud attacks and recommendations on how to defend against them.

Tactical Guide to Threat Hunting in Snowflake Environments

It was brought to our attention that a threat actor has been observed using stolen customer credentials to target organizations utilizing Snowflake databases.

Unlocking Cloud Security with Managed Detection and Response

See how Mitiga’s Cloud Managed Detection and Response tackles complex cyber threats with proactive threat management and advanced automation at scale.

Rethinking Crown Jewels Analysis: Mitigating Cybersecurity Bias

Uncover the risks of bias in Crown Jewels Analysis (CJA) and learn strategies to protect your organization's most valuable assets with a comprehensive approach.

Microsoft Breach by Midnight Blizzard (APT29): What Happened?

Understand the Midnight Blizzard Microsoft breach by APT29, what happened, and key steps organizations should take to strengthen their defenses.

Understanding Lateral Movement Attacks in Hybrid Environments

Learn how lateral movement attacks pose serious risks in on-prem, cloud, or hybrid environments, and discover effective strategies to mitigate these threats.