Connect and Manage On-Prem Environments From Multi-Cloud Environments

For a long time I was thinking about hybrid cloud deploymentand, and how it becomes a very common option either azure to azure or On-premises, AWS, Google cloud, so I decided to share with you all the way to connect between with pro and Con of each one.

Azure to On-premises

VPN connection
A VPN gateway is a type of virtual network gateway that sends encrypted traffic between an Azure virtual network and an on-premises location. The encrypted traffic goes over the public Internet.
This architecture is suitable for hybrid applications where the traffic between on-premises hardware and the cloud is likely to be light, or you are willing to trade slightly extended latency for the flexibility and processing power of the cloud.

Benefits

• Simple to configure.
  
• Much higher bandwidth available; up to 10 Gbps depending on the VPN Gateway SKU.

Challenges

• Requires an on-premises VPN device.
• Although Microsoft guarantees 99.9% availability for each VPN Gateway
  

Site-to-Site
Azure S2S VPN connections provide secure, cross-premises connectivity between customer premises and Azure. This tutorial walks through IPsec S2S VPN connection life cycles such as creating and managing a S2S VPN connection.

Azure Supported Services is Cloud Services and Virtual Machines
 
Typical Bandwidths is Typically < 1 Gbps aggregate
Protocols Supported work as IPSEC
Routing We support Policy Based (static routing) and Route Based (dynamic routing VPN)
Connection resiliency active-passive or active-active
Typical use case Dev/test/lab scenarios and small-scale production workloads for cloud services and virtual machines
 
Point-to-Site VPN connection
you can also use P2S instead of a Site-to-Site VPN when you have only a few clients that need to connect to a VNet. Point-to-Site connections do not require a VPN device or a public-facing IP address. P2S creates the VPN connection over either SSTP (Secure Socket Tunneling Protocol), or IKEv2. For more information about Point-to-Site VPN
Azure Supported Services: Cloud Services and Virtual Machines
Typical Bandwidths is Based on the gateway SKU
Protocols Supported      Secure Sockets Tunneling Protocol (SSTP), OpenVPN and IPsec
Routing : Route Based (dynamic)
Connection resiliency: active passive
Typical use case Prototyping, dev / test / lab scenarios for cloud services and virtual machines

Azure ExpressRoute connection
ExpressRoute connections use a private, dedicated connection through a third-party connectivity provider. The private connection extends your on-premises network into Azure.
This architecture is suitable for hybrid applications running large-scale, mission-critical workloads that require a high degree of scalability.
Benefits

• Much higher bandwidth available; up to 10 Gbps depending on the connectivity provider.
• Supports dynamic scaling of bandwidth to help reduce costs during periods of lower demand. However, not all connectivity providers have this option.
• May allow your organization direct access to national clouds, depending on the connectivity provider.
• 99.9% availability SLA across the entire connection.

Challenges

• Can be complex to set up. Creating an ExpressRoute connection requires working with a third-party connectivity provider. The provider is responsible for provisioning the network connection.
• Requires high-bandwidth routers on-premises.

Architecture
The architecture consists of the following components.

• On-premises corporate network. A private local-area network running within an organization.
• ExpressRoute circuit. A layer 2 or layer 3 circuit supplied by the connectivity provider that joins the on-premises network with Azure through the edge routers. The circuit uses the hardware infrastructure managed by the connectivity provider.
• Local edge routers. Routers that connect the on-premises network to the circuit managed by the provider. Depending on how your connection is provisioned, you may need to provide the public IP addresses used by the routers.
• Microsoft edge routers. Two routers in an active-active highly available configuration. These routers enable a connectivity provider to connect their circuits directly to their datacenter. Depending on how your connection is provisioned, you may need to provide the public IP addresses used by the routers.
• Azure virtual networks (VNets). Each VNet resides in a single Azure region, and can host multiple application tiers. Application tiers can be segmented using subnets in each VNet.
• Azure public services. Azure services that can be used within a hybrid application. These services are also available over the Internet, but accessing them using an ExpressRoute circuit provides low latency and more predictable performance, because traffic does not go through the Internet.
• Microsoft 365 services. The publicly available Microsoft 365 applications and services provided by Microsoft. Connections are performed using Microsoft peering, with addresses that are either owned by your organization or supplied by your connectivity provider. You can also connect directly to Microsoft CRM Online through Microsoft peering.
• Connectivity providers (not shown). Companies that provide a connection either using layer 2 or layer 3 connectivity between your datacenter and an Azure datacenter.

ExpressRoute with VPN failover
This options combines the previous two, using ExpressRoute in normal conditions, but failing over to a VPN connection if there is a loss of connectivity in the ExpressRoute circuit.
This architecture is suitable for hybrid applications that need the higher bandwidth of ExpressRoute, and also require highly available network connectivity.
Benefits

• High availability if the ExpressRoute circuit fails, although the fallback connection is on a lower bandwidth network.

Challenges

• Complex to configure. You need to set up both a VPN connection and an ExpressRoute circuit.
• Requires redundant hardware (VPN appliances), and a redundant Azure VPN Gateway connection for which you pay charges.

Azure App Service Hybrid Connections
Hybrid Connections is both a service in Azure and a feature in Azure App Service. As a service, it has uses and capabilities beyond those that are used in App Service. To learn more about Hybrid Connections and their usage outside App Service,
Within App Service, Hybrid Connections can be used to access application resources in any network that can make outbound calls to Azure over port 443. Hybrid Connections provides access from your app to a TCP endpoint and does not enable a new way to access your app. As used in App Service, each Hybrid Connection correlates to a single TCP host and port combination. This enables your apps to access resources on any OS, provided it is a TCP endpoint. The Hybrid Connections feature does not know or care what the application protocol is, or what you are accessing. It simply provides network access.

How it works
Hybrid Connections requires a relay agent to be deployed where it can reach both the desired endpoint as well as to Azure. The relay agent, Hybrid Connection Manager (HCM), calls out to Azure Relay over port 443. From the web app site, the App Service infrastructure also connects to Azure Relay on your application's behalf. Through the joined connections, your app is able to access the desired endpoint. The connection uses TLS 1.2 for security and shared access signature (SAS) keys for authentication and authorization

When your app makes a DNS request that matches a configured Hybrid Connection endpoint, the outbound TCP traffic will be redirected through the Hybrid Connection.
 Note: This means that you should try to always use a DNS name for your Hybrid Connection. Some client software does not do a DNS lookup if the endpoint uses an IP address instead.
App Service Hybrid Connection benefits
There are several benefits to the Hybrid Connections capability, including:

• Apps can access on-premises systems and services securely.
• The feature does not require an internet-accessible endpoint.
• It is quick and easy to set up. No gateways required
• Each Hybrid Connection matches to a single host:port combination, helpful for security.
• It normally does not require firewall holes. The connections are all outbound over standard web ports.
• Because the feature is network level, it is agnostic to the language used by your app and the technology used by the endpoint.
• It can be used to provide access in multiple networks from a single app.
• It is supported in GA for Windows native apps and is in preview for Linux apps. It is not supported for Windows container apps.

Things you cannot do with Hybrid Connections
Things you cannot do with Hybrid Connections include:

• Mount a drive.
• Use UDP.
• Access TCP-based services that use dynamic ports, such as FTP Passive Mode or Extended Passive Mode.
• Support LDAP, because it can require UDP.
• Support Active Directory, because you cannot domain join an App Service worker.
  

AWS to On-premise

• AWS Site-to-Site VPN
• AWS Client VPN

AWS Site-to-Site VPNBy default, instances that you launch into an Amazon VPC can't communicate with your own (remote) network. You can enable access to your remote network from your VPC by creating an AWS Site-to-Site VPN (Site-to-Site VPN) connection, and configuring routing to pass traffic through the connection.
Although the term VPN connection is a general term, in this documentation, a VPN connection refers to the connection between your VPC and your own on-premises network. Site-to-Site VPN supports Internet Protocol security (IPsec) VPN connections.
Your Site-to-Site VPN connection is either an AWS Classic VPN or an AWS VPN. For more information
You can create, access, and manage your Site-to-Site VPN resources using any of the following interfaces:

• AWS Management Console— Provides a web interface that you can use to access your Site-to-Site VPN resources.
• AWS Command Line Interface (AWS CLI) — Provides commands for a broad set of AWS services, including Amazon VPC, and is supported on Windows, macOS, and Linux.
• AWS SDKs — Provide language-specific APIs and takes care of many of the connection details, such as calculating signatures, handling request retries, and error handling.
• Query API— Provides low-level API actions that you call using HTTPS requests. Using the Query API is the most direct way to access Amazon VPC, but it requires that your application handle low-level details such as generating the hash to sign the request, and error handling

Single Site-to-Site VPN connection
The VPC has an attached virtual private gateway, and your on-premises (remote) network includes a customer gateway device, which you must configure to enable the Site-to-Site VPN connection. You set up the routing so that any traffic from the VPC bound for your network is routed to the virtual private gateway

Single Site-to-Site VPN connection with a transit gateway
The VPC has an attached transit gateway, and your on-premises (remote) network includes a customer gateway device, which you must configure to enable the Site-to-Site VPN connection. You set up the routing so that any traffic from the VPC bound for your network is routed to the transit gateway.

Multiple Site-to-Site VPN connections
The VPC has an attached virtual private gateway, and you have multiple Site-to-Site VPN connections to multiple on-premises locations. You set up the routing so that any traffic from the VPC bound for your networks is routed to the virtual private gateway.
You can also use this scenario to create Site-to-Site VPN connections to multiple geographic locations and provide secure communication between sites.

When you create multiple Site-to-Site VPN connections to a single VPC, you can configure a second customer gateway to create a redundant connection to the same external location.Multiple Site-to-Site VPN connections with a transit gateway
The VPC has an attached transit gateway, and you have multiple Site-to-Site VPN connections to multiple on-premises locations. You set up the routing so that any traffic from the VPC bound for your networks is routed to the transit gateway.
You can also use this scenario to create Site-to-Site VPN connections to multiple geographic locations and provide secure communication between sites.

When you create multiple Site-to-Site VPN connections to a single transit gateway, you can configure a second customer gateway to create a redundant connection to the same external location.Client VPNAWS Client VPN is a managed client-based VPN service that enables you to securely access your AWS resources and resources in your on-premises network. With Client VPN, you can access your resources from any location using an OpenVPN-based VPN client.
Client VPN offers the following features and functionality:

• Secure connections — It provides a secure TLS connection from any location using the OpenVPN client.
• Managed service — It is an AWS managed service, so it removes the operational burden of deploying and managing a third-party remote access VPN solution.
• High availability and elasticity — It automatically scales to the number of users connecting to your AWS resources and on-premises resources.
• Authentication — It supports client authentication using Active Directory, federated authentication, and certificate-based authentication.
• Granular control — It enables you to implement custom security controls by defining network-based access rules. These rules can be configured at the granularity of Active Directory groups. You can also implement access control using security groups.
• Ease of use — It enables you to access your AWS resources and on-premises resources using a single VPN tunnel.
• Manageability — It enables you to view connection logs, which provide details on client connection attempts. You can also manage active client connections, with the ability to terminate active client connections.
• Deep integration — It integrates with existing AWS services, including AWS Directory Service and Amazon VPC.

 
Access to a VPC
The configuration for this scenario includes a single target VPC. We recommend this configuration if you need to give clients access to the resources inside a single VPC only

Before you begin, do the following:

• Create or identify a VPC with at least one subnet. Identify the subnet in the VPC that you want to associate with the Client VPN endpoint and note its IPv4 CIDR ranges.
• Identify a suitable CIDR range for the client IP addresses that does not overlap with the VPC CIDR.
• Review the rules and limitations for Client VPN endpoints in Limitations and rules of Client VPN.

To implement this configuration

1. Create a Client VPN endpoint in the same Region as the VPC. To do this, perform the steps described in Create a Client VPN endpoint.
2. Associate the subnet with the Client VPN endpoint. To do this, perform the steps described in Associate a target network with a Client VPN endpoint and select the subnet and the VPC you identified earlier.
3. Add an authorization rule to give clients access to the VPC. To do this, perform the steps described in Add an authorization rule to a Client VPN endpoint, and for Destination network, enter the IPv4 CIDR range of the VPC.
4. Add a rule to your resources' security groups to allow traffic from the security group that was applied to the subnet association in step 2.

Access to a peered VPC
The configuration for this scenario includes a target VPC (VPC A) that is peered with an additional VPC (VPC B). We recommend this configuration if you need to give clients access to the resources inside a target VPC and other VPCs that are peered with it (such as VPC B).

Before you begin, do the following:

• Create or identify a VPC with at least one subnet. Identify the subnet in the VPC that you want to associate with the Client VPN endpoint and note its IPv4 CIDR ranges.
• Identify a suitable CIDR range for the client IP addresses that does not overlap with the VPC CIDR.

To implement this configuration

1. Establish the VPC peering connection between the VPCs. Follow the steps at Creating and accepting a VPC peering connection in the Amazon VPC Peering Guide.
2. Test the VPC peering connection. Confirm that instances in either VPC can communicate with each other as if they are within the same network. If the peering connection works as expected, continue to the next step.
3. Create a Client VPN endpoint in the same Region as the target VPC. In the preceding example, this is VPC A. Perform the steps described in Create a Client VPN endpoint.
4. Associate the subnet you identified earlier with the Client VPN endpoint that you created. To do this, perform the steps described in Associate a target network with a Client VPN endpoint and select the subnet and the VPC.
5. Add an authorization rule to give clients access to the target VPC. To do this, perform the steps described in Add an authorization rule to a Client VPN endpoint, and for Destination network to enable, enter the IPv4 CIDR range of the VPC.
6. Add a route to direct traffic to the peered VPC. In the preceding example, this is VPC B. To do this, perform the steps described in Create an endpoint route; for Route destination, enter IPv4 CIDR range of the peered VPC, and for Target VPC Subnet ID, select the subnet you associated with the Client VPN endpoint.
7. Add an authorization rule to give clients access to peered VPC. To do this, perform the steps described in Add an authorization rule to a Client VPN endpoint; for Destination network, enter IPv4 CIDR range of the peered VPC.
8. Add a rule to your resources' security groups in VPC A and VPC B to allow traffic from the security group that was applied to the subnet association in step 2.
   

 
Access to an on-premises network
The configuration for this scenario includes access to an on-premises network only. We recommend this configuration if you need to give clients access to the resources inside an on-premises network only.

Before you begin, do the following:

• Create or identify a VPC with at least one subnet. Identify the subnet in the VPC that you want to associate with the Client VPN endpoint and note its IPv4 CIDR ranges
• Identify a suitable CIDR range for the client IP addresses that does not overlap with the VPC CIDR.

To implement this configuration

1. Enable communication between the VPC and your own on-premises network over an AWS Site-to-Site VPN connection.

Note: Alternatively, you can implement this scenario by using an AWS Direct Connect connection between your VPC and your on-premises network.

1. Test the AWS Site-to-Site VPN connection you created in the previous step. To do this, perform the steps described in Testing the Site-to-Site VPN connection in the AWS Site-to-Site VPN User Guide. If the VPN connection is functioning as expected, continue to the next step.
2. Create a Client VPN endpoint in the same Region as the VPC. To do this, perform the steps described in Create a Client VPN endpoint.
3. Associate the subnet that you identified earlier with the Client VPN endpoint. To do this, perform the steps described in Associate a target network with a Client VPN endpoint and select the VPC and the subnet.
4. Add a route that allows access to the AWS Site-to-Site VPN connection. To do this, perform the steps described in Create an endpoint route; for Route destination, enter the IPv4 CIDR range of the AWS Site-to-Site VPN connection, and for Target VPC Subnet ID, select the subnet you associated with the Client VPN endpoint.
5. Add an authorization rule to give clients access to the AWS Site-to-Site VPN connection. To do this, perform the steps described in Add an authorization rule to a Client VPN endpoint; for Destination network, enter the AWS Site-to-Site VPN connection IPv4 CIDR range.

Access to the internet
The configuration for this scenario includes a single target VPC and access to the internet. We recommend this configuration if you need to give clients access to the resources inside a single target VPC and allow access to the internet.

Before you begin, do the following:

• Create or identify a VPC with at least one subnet. Identify the subnet in the VPC that you want to associate with the Client VPN endpoint and note its IPv4 CIDR ranges.
• Identify a suitable CIDR range for the client IP addresses that does not overlap with the VPC CIDR.

To implement this configuration

1. Ensure that the security group that you'll use for the Client VPN endpoint allows inbound and outbound traffic to and from the internet. To do this, add inbound and outbound rules that allow traffic to and from 0.0.0.0/0 for HTTP and HTTPS traffic.
2. Create an internet gateway and attach it to your VPC.
3. Make your subnet public by adding a route to the internet gateway to its route table. In the VPC console, choose Subnets, select the subnet you intend to associate with the Client VPN endpoint, choose Route Table, and then choose the route table ID. Choose Actions, choose Edit routes, and choose Add route. For Destination, enter 0.0.0.0/0, and for Target, choose the internet gateway from the previous step.
4. Create a Client VPN endpoint in the same Region as the VPC. To do this, perform the steps described in Create a Client VPN endpoint.
5. Associate the subnet that you identified earlier with the Client VPN endpoint. To do this, perform the steps described in Associate a target network with a Client VPN endpoint and select the VPC and the subnet.
6. Add an authorization rule to give clients access to the VPC. To do this, perform the steps described in Add an authorization rule to a Client VPN endpoint; and for Destination network to enable , enter the IPv4 CIDR range of the VPC.
7. Add a route that enables traffic to the internet. To do this, perform the steps described in Create an endpoint route; for Route destination, enter 0.0.0.0/0, and for Target VPC Subnet ID, select the subnet you associated with the Client VPN endpoint.
8. Add an authorization rule to give clients access to the internet. To do this, perform the steps described in Add an authorization rule to a Client VPN endpoint; for Destination network, enter 0.0.0.0/0.
9. Ensure that the security group for the subnet association in step 5 has an outbound rule that allows internet access (the destination is 0.0.0.0/0).

Client-to-client access
The configuration for this scenario enables clients to access a single VPC, and enables clients to route traffic to each other. We recommend this configuration if the clients that connect to the same Client VPN endpoint also need to communicate with each other. Clients can communicate with each other using the unique IP address that's assigned to them from the client CIDR range when they connect to the Client VPN endpoint.

Before you begin, do the following:

• Create or identify a VPC with at least one subnet. Identify the subnet in the VPC that you want to associate with the Client VPN endpoint and note its IPv4 CIDR ranges
• Identify a suitable CIDR range for the client IP addresses that does not overlap with the VPC CIDR.

To implement this configuration

1. Create a Client VPN endpoint in the same Region as the VPC. To do this, perform the steps described in Create a Client VPN endpoint.
2. Associate the subnet that you identified earlier with the Client VPN endpoint. To do this, perform the steps described in Associate a target network with a Client VPN endpoint and select the VPC and the subnet.
3. Add a route to the local network in the route table. To do this, perform the steps described in Create an endpoint route. For Route destination, enter the client CIDR range, and for Target VPC Subnet ID, specify local.
4. Add an authorization rule to give clients access to the VPC. To do this, perform the steps described in Add an authorization rule to a Client VPN endpoint. For Destination network to enable , enter the IPv4 CIDR range of the VPC.
5. Add an authorization rule to give clients access to the client CIDR range. To do this, perform the steps described in Add an authorization rule to a Client VPN endpoint. For Destination network to enable, enter the client CIDR range.

Restrict access to your network
You can configure your Client VPN endpoint to restrict access to specific resources in your VPC. For user-based authentication, you can also restrict access to parts of your network, based on the user group that accesses the Client VPN endpoint.
Restrict access using security groups
You can grant or deny access to specific resources in your VPC by adding or removing security group rules that reference the security group that was applied to the target network association (the Client VPN security group). This configuration expands on the scenario described in Access to a VPC. This configuration is applied in addition to the authorization rule configured in that scenario.
To grant access to a specific resource, identify the security group that's associated with the instance on which your resource is running. Then, create a rule that allows traffic from the Client VPN security group.
In the following example, sg-xyz is the Client VPN security group, security group sg-aaa is associated with instance A, and security group sg-bbb is associated with instance B. You add a rule to sg-aaa that allows access from sg-xyz, therefore, clients can access your resources in instance A. Security group sg-bbb does not have a rule that allows access from sg-xyz or the Client VPN network interface. Clients cannot access the resources in instance B.

Before you begin, check if the Client VPN security group is associated with other resources in your VPC. If you add or remove rules that reference the Client VPN security group, you might grant or deny access for the other associated resources too. To prevent this, use a security group that is specifically created for use with your Client VPN endpoint.
To create a security group rule

1. Open the Amazon VPC console at https://console.aws.amazon.com/vpc/.
2. In the navigation pane, choose Security Groups.
3. Choose the security group that's associated with the instance on which your resource is running.
4. Choose Actions, Edit inbound rules.
5. Choose Add rule, and then do the following:
   • For Type, choose All traffic, or a specific type of traffic that you want to allow.
   • For Source, choose Custom, and then enter or choose the ID of the Client VPN security group.
6. Choose Save rules

To remove access to a specific resource, check the security group that's associated with the instance on which your resource is running. If there is a rule that allows traffic from the Client VPN security group, delete it.
To check your security group rules

1. Open the Amazon VPC console at https://console.aws.amazon.com/vpc/.
2. In the navigation pane, choose Security Groups.
3. Choose Inbound Rules.
4. Review the list of rules. If there is a rule where Source is the Client VPN security group, choose Edit Rules, and choose Delete (the x icon) for the rule. Choose Save rules.

Restrict access based on user groups
If your Client VPN endpoint is configured for user-based authentication, you can grant specific groups of users access to specific parts of your network. To do this, complete the following steps:

1. Configure users and groups in AWS Directory Service or your IdP. For more information, see the following topics:
   • Active Directory authentication
   • Requirements and considerations for SAML-based federated authentication
2. Create an authorization rule for your Client VPN endpoint that allows a specified group access to all or part of your network.

If your Client VPN endpoint is configured for mutual authentication, you cannot configure user groups. When you create an authorization rule, you must grant access to all users. To enable specific groups of users access to specific parts of your network, you can create multiple Client VPN endpoints. For example, for each group of users that accesses your network, do the following:

1. Create a set of server and client certificates and keys for that group of users.
2. Create a Client VPN endpoint.

Create an authorization rule that grants access to all or part of your network. For example, for a Client VPN endpoint that is used by administrators, you might create an authorization rule that grants access to the entire network

Google Cloud to On-premises
Cloud VPN overviewCloud VPN securely connects your peer network to your Virtual Private Cloud (VPC) network through an IPsec VPN connection. Traffic traveling between the two networks is encrypted by one VPN gateway, and then decrypted by the other VPN gateway. This protects your data as it travels over the internet. You can also connect two instances of Cloud VPN to each other.
Types of Cloud VPN
Google Cloud offers two types of Cloud VPN gateways, HA VPN and Classic VPN.
HA VPNHA VPN is a high availability (HA) Cloud VPN solution that lets you securely connect your on-premises network to your Virtual Private Cloud (VPC) network through an IPsec VPN connection in a single region. HA VPN provides an SLA of 99.99% service availability.
When you create an HA VPN gateway, Google Cloud automatically chooses two external IP addresses, one for each of its fixed number of two interfaces. Each IP address is automatically chosen from a unique address pool to support high availability. Each of the HA VPN gateway interfaces supports multiple tunnels. You can also create multiple HA VPN gateways. When you delete the HA VPN gateway, Google Cloud releases the IP addresses for reuse. You can configure an HA VPN gateway with only one active interface and one public IP address; however, this configuration does not provide a 99.99% service availability SLA.
In the API documentation and in gcloud commands, HA VPN gateways are referred to as VPN gateways rather than target VPN gateways. You don't need to create any forwarding rules for HA VPN gateways.
HA VPN uses an external VPN gateway resource in Google Cloud to provide information to Google Cloud about your peer VPN gateway or gateways.
HA VPN requirementsYour Cloud VPN configuration must meet the following requirements to achieve a service-level availability of 99.99% for HA VPN:
When you connect an HA VPN gateway to your peer gateway, 99.99% availability is guaranteed only on the Google Cloud side of the connection. End-to-end availability is subject to proper configuration of the peer VPN gateway.
If both sides are Google Cloud gateways and are properly configured, end-to-end 99.99% availability is guaranteed.
To achieve high availability when both VPN gateways are located in VPC networks, you must use two HA VPN gateways, and both of them must be located in the same region.
Even though both gateways must be located in the same region, the routes to their subnets that they share with each other can be located in any region if your VPC network uses global dynamic routing mode. If your VPC network uses regional dynamic routing mode, only routes to subnets in the same region are shared with the peer network, and learned routes are applied only to subnets in the same region as the VPN tunnel.

• HA VPN rejects Google Cloud IP addresses when they are configured in an external VPN gateway resource. An example of this is using the external IP address of a VM instance as the external IP address for the external VPN gateway resource. The only supported HA VPN Google Cloud-to-Google Cloud topology is where HA VPN is used on both sides

You must configure two VPN tunnels from the perspective of the Cloud VPN gateway:
If you have two peer VPN gateway devices, each of the tunnels from each interface on the Cloud VPN gateway must be connected to its own peer gateway.
If you have a single peer VPN gateway device with two interfaces, each of the tunnels from each interface on the Cloud VPN gateway must be connected to its own interface on the peer gateway.
If you have a single peer VPN gateway device with a single interface, both of the tunnels from each interface on the Cloud VPN gateway must be connected to the same interface on the peer gateway.
A peer VPN device must be configured with adequate redundancy. The details of an adequately redundant configuration are specified by the device vendor, and may or may not include multiple hardware instances. For details, see the vendor documentation for the peer VPN device.
If two peer devices are required, each peer device must be connected to a different HA VPN gateway interface. If the peer side is another cloud provider like AWS, VPN connections must be configured with adequate redundancy on the AWS side as well.
Your peer VPN gateway device must support dynamic (BGP) routing.
The following diagram shows the HA VPN concept, showing a topology that includes the two interfaces of a HA VPN gateway connected to two peer VPN gateways

Classic VPN
WARNING: Classic VPN is deprecating certain functionality on October 31, 2021.
In contrast, Classic VPN gateways have a single interface, a single external IP address, and support tunnels using dynamic (BGP) or static routing (route based or policy based). They provide an SLA of 99.9% service availability.
Classic VPNs are referred to as target VPN gateways in the API documentation and in cloud commands

Using a second peer VPN gateway
For Classic VPN, if your on-premises side is hardware based, having a second peer VPN gateway provides redundancy and failover on that side of the connection. A second physical gateway allows you to take one of the gateways offline for software upgrades or other scheduled maintenance. It also protects you in case of an outright failure in one of the devices.
To configure a tunnel from your Cloud VPN gateway to a second on-premises-side VPN gateway, do the following:
Configure a second on-premises VPN gateway and a tunnel.
Set up a second tunnel on your Cloud VPN gateway pointing to the second on-premises gateway
Forward the same routes for the second tunnel as you did for the first. If you want both tunnels to balance traffic, set their route priorities to be the same. If you want one tunnel to be primary, set a lower priority on the second tunnel.
If either VPN tunnel fails due to network issues along the path, or a problem with an on-premises gateway, the Cloud VPN gateway will continue sending traffic over the healthy tunnel and will automatically resume using both tunnels once the failed tunnel recovers.

Increased throughput and load balancing options
There are three options for scaling a Cloud VPN configuration:

• Option 1: Scale the on-premises VPN gateway.

Set up a second on-premises VPN gateway device with a different external IP address. Create a second tunnel on your existing Cloud VPN gateway that forwards the same IP range, but pointing at the second on-premises gateway IP. Your Cloud VPN gateway automatically load balances between the configured tunnels. You can set up the VPN gateways to have multiple tunnels load balanced this way to increase the aggregate VPN connectivity throughput.

• Option 2: Scale the Cloud VPN gateway. If you’re on-premises VPN gateway's throughput capabilities are higher, and you want to scale higher throughput from the Cloud VPN gateway, you can set up a second Cloud VPN gateway.
  

• Option 3: Scale both the on-premises VPN gateway and the Cloud VPN gateway.