Kafka Over the Public Internet in Google Cloud: Security, Architecture, and the Real Cost of Access

Public access to Google Cloud Managed Service for Apache Kafka simplifies connectivity for external clients, but it adds CIDR management, DNS, identities, ACLs, observability, and internet traffic to the cost model. This article explains when that path is justified—and when private connectivity through PSC is the better option.

Connecting an external client to a Kafka cluster is usually more involved than providing a URL and credentials. The client first contacts a bootstrap address, retrieves metadata, and then establishes connections to specific brokers. Network access must therefore work not for a single entry point, but for a set of addresses that changes with the cluster.

Public access in Google Cloud Managed Service for Apache Kafka removes the need to route an external client through a VPC, VPN, or proxy. But Kafka does not become an internet API as a result. The team gains another network path that must be secured, observed, and included in the cost model.

What Google made public

The feature appeared in the release notes on September 10, 2026, and Google Cloud announced its general availability on September 17.

When public access is enabled, the service allocates external IPv4 addresses for the bootstrap endpoint and brokers. Existing DNS names become resolvable through public DNS.

Private networking remains in place:

• the cluster must still have at least one connected subnet;

• within a VPC with a connected subnet, the same DNS names resolve to private addresses;

• outside that VPC, they resolve to public addresses;

• an external client can connect without routing traffic through a connected subnet.

Public access is therefore not a replacement for the existing network architecture, but an additional access plane. Internal services can continue to use Private Service Connect (PSC), while selected external clients connect over the internet. The full topology is described in the networking configuration documentation.

This distinction matters: enabling public access does not require moving all client traffic onto the public path.

Why a public bootstrap address alone is not enough

A Kafka client does not maintain a permanent connection only to its bootstrap address. It uses that address to retrieve initial metadata, then connects directly to brokers.

The external network must therefore permit the entire client path:

1. DNS resolution of the bootstrap address.

2. A connection to the bootstrap endpoint.

3. Retrieval of broker addresses from Kafka metadata.

4. DNS resolution and connections to every required broker.

5. Reconnection when cluster membership or addresses change.

Google warns that a cluster's public IP addresses must not be treated as static. The address set can change or expand when the cluster scales, and addresses change if public access is disabled and then enabled again.

For outbound firewall configuration, Google publishes dedicated discovery DNS A records. Each record contains no more than 30 IP addresses; larger clusters have additional records. Their purpose is to help firewalls that support FQDN-based rules discover current addresses.

Kafka clients, however, must use the bootstrap address—not the discovery records. Mixing these purposes creates a fragile configuration: the firewall may know the broker addresses, but the client should not bypass Kafka's standard discovery mechanism.

For external networks, Google recommends allowing:

• TCP/9092 for SASL;

• TCP/9192 for mTLS.

The practical implication is clear: if a partner's firewall accepts only a static IP list, connecting requires a process to update that list. If it supports FQDN-based rules, discovery records can automate the process.

Public access, PSC, or a self-managed Kafka cluster

Start by choosing the network path based not on whether Kafka can be exposed to the internet, but on where clients reside and where trust boundaries lie.

Internal services: PSC remains the default option

For applications in GKE, Compute Engine, or Cloud Run, private connectivity through PSC preserves reachability over private IP addresses. For each connected VPC, the service creates an endpoint for the bootstrap address and separate endpoints for brokers. DNS names remain the same, but resolve to addresses local to the relevant client network.

This is especially useful when clients already run in Google Cloud. In that case, the public route rarely removes enough complexity to justify the additional external attack surface and internet traffic.

PSC should not, however, be treated as entirely free or design-free. Cross-zone traffic between a client and broker incurs billable data processing, while cross-project scenarios require network and IAM coordination.

External clients: the public path may be simpler

If a partner operates in another cloud or in its own infrastructure, internet connectivity may be more practical than a VPN, bastion host, or complex VPC connectivity.

The most manageable scenario looks like this:

• the partner has a fixed egress NAT CIDR;

• that range is added to the cluster allowlist;

• the partner's firewall tracks broker addresses through DNS discovery records;

• authentication uses short-lived tokens or mTLS;

• Kafka ACLs restrict access to specific topics and consumer groups.

With dynamic egress IPs, that convenience disappears quickly. An address change becomes an allowlist change request, while temporarily adding a broad range weakens the security model.

Self-hosting: control brings operational responsibilities back

Google automatically replaces failed brokers, patches critical vulnerabilities, and exports logs and metrics. With self-managed Kafka, all of that returns to the engineering team.

On the other hand, the managed service does not remove responsibility for:

• client applications;

• topic and ACL design;

• identity management;

• capacity planning;

• disaster recovery;

• observability across the full client path.

The service also does not protect against a regional failure or simultaneous loss of two zones. That level of resilience requires two regional clusters and a synchronization mechanism such as MirrorMaker 2.0. These limitations are listed in the Managed Service for Apache Kafka overview.

CIDR is only the first barrier

A public cluster accepts connections only from allowed source ranges. The following restrictions apply to the allowlist:

• only publicly routable IPv4 ranges are supported;

• prefixes must be between /16 and /32;

• no more than 500 ranges are allowed;

• ranges must not overlap;

• RFC 1918 ranges and IPv6 are not supported.

Filtering is implemented through Cloud Next Generation Firewall. Google explicitly recommends against adding untrusted ranges.

This makes public access unsuitable for clients with mobile addresses, IPv6-only networks, or unpredictable carrier NAT. The theoretical ability to connect IoT devices or remote sites does not mean every device should connect directly to Kafka.

For edge scenarios, a more realistic design uses a controlled gateway:

Devices → local or regional gateway → fixed NAT IPv4 → Kafka

The gateway aggregates connections, provides a predictable source address, and can handle buffering. The documentation does not establish that direct access is suitable for arbitrary constrained devices, mobile addresses, or IPv6-only fleets.

A public address does not mean unencrypted or anonymous access

All client connections to Google Managed Kafka are protected with TLS. Anonymous access is not permitted. A public IP therefore does not mean an open plaintext listener.

But TLS and IP validation do not answer what an authenticated client is authorized to do. That requires identity and Kafka ACLs.

The preferred approach: WIF and SASL/OAUTHBEARER

For clients in AWS, Azure, or on-premises infrastructure, Google recommends Workload Identity Federation (WIF) with SASL/OAUTHBEARER. The external identity obtains a short-lived Google access token and acts as a service account.

This approach reduces reliance on long-lived secrets. SASL/PLAIN with a service-account key is technically supported, but Google does not recommend it for production because it relies on static, long-lived credentials.

mTLS for organizations with their own PKI

mTLS is not tied to a specific cloud provider and suits organizations with an established PKI. Client identity is determined from the certificate Subject Name.

This model has an important limitation: mTLS client authorization is performed through Kafka ACLs, not Google Cloud IAM roles. Certificate issuance, revocation, and renewal processes must therefore align with Kafka authorization rules.

ACLs cannot be deferred

If Kafka ACLs are not configured, the documented default behavior grants every authenticated principal full access.

This is a dangerous trap: a team may carefully restrict CIDRs and configure WIF, yet a partner can still access every topic and consumer group after authenticating successfully.

At a minimum, the model should restrict:

• allowed topics;

• read and write operations;

• consumer groups;

• administrative operations;

• separate principals for different applications and environments.

The network range answers, “Where did the connection come from?” Identity answers, “Who connected?” ACLs answer, “What are they allowed to do?” None of these layers replaces the others.

Access revocation and re-authentication

Removing a CIDR from the allowlist applies only to new connections. An already established Kafka session is not necessarily terminated immediately.

An incident-response procedure should therefore not consist of a single step: “remove the partner's IP.” In the event of compromise, use multiple layers:

1. Remove or narrow the allowed CIDR range to stop new connections.

2. Disable or revoke the compromised identity and certificate; for already issued tokens, verify the available invalidation mechanism and account for their lifetime.

3. Restrict or remove the relevant Kafka ACLs.

4. Review activity for affected topics and consumer groups.

5. Assess which data the client may have read or written.

6. Verify the actual time to access termination with a test connection.

Public documentation does not describe a supported operator mechanism that guarantees closure of all already open public sessions. Neither an allowlist change nor any other individual control should therefore be assumed in advance to provide immediate isolation; validate the behavior in drills.

There is another operational consideration: Managed Kafka requires SASL re-authentication every 30 minutes. Clients without re-authentication support may lose connections regularly; Google identifies Kafka client 2.2.0 as the minimum version supporting this capability. Validate it through a long-running test rather than a brief test that sends only a few messages. Details are available in the troubleshooting guide.

Observability must cover the client path

The service has no JMX API for broker metrics. Cloud Monitoring metrics and broker logs in Cloud Logging are available, but they show only the server side.

For example, topic_error_count does not cover all failure classes. An authorization error can block writes without appearing in that metric, while an unreachable broker may never respond at all.

Google therefore recommends supplementing server-side data with client metrics. A practical baseline should include:

• authentication and authorization failures;

• DNS and TLS errors;

• request latency;

• timeouts and retries;

• reconnections;

• producer local queue size;

• consumer lag;

• errors when contacting individual brokers.

For the public path, also monitor:

• changes to allowed CIDR ranges;

• changes to broker addresses;

• DNS resolution success from the external network;

• TCP/9092 or TCP/9192 availability;

• the state of the client's egress NAT;

• internet traffic volume;

• disconnects that coincide with 30-minute SASL re-authentication.

Cloud Audit Logs help investigate changes to clusters, topics, ACLs, and identities. But they do not record produce and consume operations. Moreover, Data Access logs for metadata operations are disabled by default until the team explicitly enables them. The purpose and limitations of auditing are described in the Cloud Audit Logs documentation for Managed Kafka.

In other words, control-plane logs show who changed an ACL, but they do not replace client telemetry when investigating message reads or writes.

TCO: account for traffic and operational work

The cost of public access cannot be assessed by whether there is a separate endpoint fee. It is more useful to break total cost of ownership into several components:

TCO = cluster resources
    + storage
    + cross-zone replication
    + client network-path cost
    + disaster recovery
    + engineering and operational work

Costs shared by public and private access

The default replication factor of 3 creates billable cross-zone traffic regardless of the client path. An example on the pricing page shows that writing 10 GiB in one zone generates another 20 GiB of transfer for two additional replicas. At $0.01/GiB, that is $0.20 in cross-zone traffic.

Moving from PSC to a public endpoint does not eliminate this component.

The public-path variable

Traffic from the cluster to an external consumer is charged at standard internet egress rates. The documentation provides an indicative range of $0.08 to $0.023 per GiB, depending on destination and volume.

As a result, public consumers with high read volumes can change the economics of the solution even when there are only a few clients. Calculate the exact amount for the relevant geography and validate it against the actual billing export.

The private-path variable

For PSC, data processing is charged when traffic flows between a client and a broker in different zones—approximately $0.004 to $0.01 per GiB depending on topology. The hourly endpoint fee for Managed Kafka is listed as waived. Current rates should be checked on the Managed Kafka pricing and VPC pricing pages.

These ranges do not establish in advance that PSC is always cheaper than public access. The result depends on client zones, traffic direction, geography, and volume. The specific workload must be measured.

Organizational costs

The public path adds work that is not visible on the Kafka bill:

• tracking and updating partner CIDRs;

• DNS and firewall automation;

• issuing and revoking identities or certificates;

• ACL design;

• supporting client libraries and re-authentication;

• diagnosing the external network path;

• access-revocation drills;

• monitoring internet traffic and cost anomalies.

The private path, in turn, requires subnet, zone, IAM, and DNS planning. A self-managed Kafka cluster adds upgrades, broker replacement, operational tooling, and on-call responsibilities. Comparing compute resource prices alone is incomplete.

Kafka Connect also deserves separate consideration. Its workers run in a dedicated subnet and require Cloud NAT for internet access. A public endpoint on another Kafka cluster does not remove that requirement. For example, MirrorMaker connecting to an internet-accessible Kafka cluster still needs an outbound route through Cloud NAT, as noted in the documentation for creating a Connect cluster.

Practical scenarios

Local development

A public cluster lets a developer connect without a bastion host or proxy. It genuinely shortens time to the first message.

But the workstation must use a stable public IPv4 address, and the client must support TLS, the selected authentication method, and—when using SASL—re-authentication.

For a production cluster, do not automatically allow home or broad corporate ranges. It is safer to use a separate development cluster, a separate identity, and a limited set of topics.

A SaaS partner in another cloud

This is the strongest use case for public access. It works when both parties can establish a network and identity contract:

• the partner provides a stable NAT CIDR;

• CIDR changes follow an agreed process;

• the outbound firewall uses DNS discovery records;

• WIF/OAUTHBEARER or mTLS establishes identity;

• ACLs restrict topics and groups;

• both parties collect client telemetry.

If the partner does not control its egress addresses, a public endpoint may be more complicated than a VPN or intermediary gateway.

IoT, retail, and telecom sites

Google lists these as use cases, but their suitability depends on network topology. The device fleet needs a controlled aggregating gateway or NAT with a predictable IPv4 range.

Direct connections from individual devices with changing mobile addresses, IPv6-only connectivity, unstable links, or difficult certificate management should not be considered an automatically supported architecture.

How to run a like-for-like public/private experiment

Without measurement, you cannot reliably compare the latency, throughput, or cost of the two paths. The experiment should use the same workload and differ only in networking.

1. Define the threat model

Before testing, identify:

• the data and topics being protected;

• external participants and operators;

• boundaries between the client, the internet, Google Cloud, and Kafka;

• identity theft vectors;

• the risk of an overly broad CIDR;

• the consequences of incorrect ACLs;

• potential reading, writing, or deletion of data;

• available ways to limit damage.

2. Prepare two equivalent clients

Place one client in a connected VPC and route it through PSC. Place the other in an external environment with controlled NAT and public access.

They should use:

• the same Kafka client version;

• identical producer and consumer settings;

• identical topics and message sizes;

• the same workload intensity;

• comparable identities and ACLs.

3. Measure more than average latency

Collect:

• p50, p95, and p99 latency;

• throughput;

• retry and error rates;

• timeouts;

• consumer lag;

• reconnection counts;

• DNS, TLS, and authentication errors;

• internet traffic or PSC data processing in billing;

• the number of manual actions needed to operate each path.

4. Test configuration changes

Normal operation is only half the validation. Test separately:

• removing and adding CIDRs;

• changing the client's egress NAT address;

• refreshing a token or certificate;

• 30-minute re-authentication;

• changing broker public IPs;

• updating FQDN-based firewall rules;

• an incorrect ACL;

• loss of one broker.

5. Define decision criteria in advance

Public access is a rational choice only when all of the following conditions are met:

• the client is physically outside a connectable VPC;

• it has a stable, controlled egress IPv4 address;

• WIF or mTLS can be integrated into the identity lifecycle;

• ACLs restrict access to the minimum required;

• the firewall can accommodate changes to broker addresses;

• the team observes the client path;

• internet traffic costs are acceptable;

• the access-revocation procedure has been tested in practice.

If the client already runs within Google Cloud, private connectivity through PSC remains the stronger default architecture. Public access is best treated as a constrained, segmented path for clients that genuinely cannot be connected to a VPC or for which doing so is not practical—not as a new way to connect every application to Kafka.