Networking

The Oracle Backend for Microservices and AI has specific networking requirements to ensure resource communication while providing security through isolation and networking rules.

The standard installation will provision a new Virtual Cloud Network (VCN) with the required subnets and all networking rules to get you started using the Oracle Backend for Microservices and AI quickly. To use an existing VCN, please follow the Bring Your Own VCN instructions.

For custom installations, including On-Premises, it is the responsibility of the customer to ensure network access controls to provide both operational access and security. The Oracle Cloud Infrastructure (OCI) standard installation setup can be used as a general template.

All infrastructure resources are split between two subnets: one public and one private subnet with access restricted by Network Security Groups (Firewalls). All access is blocked by default with specific IP and port opened, as documented below, for proper operation.

Note minimum CIDRs are based on an Oracle Cloud Infrastructure (OCI) deployments. Other cloud vendors reserve usable IPs within a CIDR reducing the usable IPs that can be used by the infrastructure.

The public subnet allows external access to the Oracle Backend for Microservices and AI resources. Ingress/Egress to the public subnet and from the Internet is provided by an Internet Gateway.

At a minimum, the public subnet requires two usable IP Addresses. In OCI this equates to a 30-bit network identifier (/30).

ICMP traffic is allowed between the public and private Subnets for resource discovery.

NOTE: Resources within the public subnet are not inherently accessible from the internet. The public subnet allows the resources in it to obtain a public IP address, however assigning a public IP is not required. When a public IP is not assigned to the resource, it is only accessible from within the subnet itself. All access to resources in the public subnet are restricted by configurable Network Security Groups (Firewalls).

An external Load Balancer is used in conjunction with Ingress resources to expose Services to the internet. Ingress to the Load Balancer should be restricted by CIDR and ports (default: 0.0.0.0/0; 80,443). All TLS communications terminate at the LoadBalancer. For more information on the certificate for TLS, please see OCI Load Balancer TLS Certificate Setup.

The Kubernetes API Endpoint, within the Control Plane, is used to manage the Kubernetes Cluster as well as providing access to Kubernetes services that are not exposed to the Load Balancer via Port Forwarding. Ingress to the Endpoint is restricted to port 6443 and should be further restricted by CIDR (default: 0.0.0.0/0).

In Oracle Cloud Infrastructure (OCI), the Oracle Kubernetes Engine (OKE) Control Plane is allowed egress to OCI services via port 443. Access to these services are required for the OKE Managed Control Plane.

The private subnet isolates its resources from direct external access by prohibiting the allocation of public IP address to those resources. Ingress to resources in the private subnet is restricted to the Load Balancer and Kubernetes API Endpoint via specific Network Security Group (Firewall) rules. Optionally, a Bastion Service, within the Private subnet, can provide additional non-restricted access to the resources in the private subnet.

Egress to the Internet is provided by a NAT Gateway, while Egress to other Cloud Services (such as a Container Repository) is provided by a Service Gateway.

The CIDR of the Private Subnet is dependent on the number of Kubernetes Work Nodes and other resources, such as databases. It is recommended to specify a CIDR providing at least six usable IP Addresses. In OCI, this equates to a 29-bit network identifier (/29).

ICMP traffic is allowed between the public and private subnets for resource discovery.

The worker nodes are allowed to communicate with other worker nodes on all ports.

The Worker Nodes and Load Balancer are allowed to communicate on port 10256 for Health Check/Availability purposes and ports 30000-32767 for forwarding external internet access to exposed services.

Worker Nodes are allowed to communicate with the API Endpoint on port 6443.

The Kubernetes Control Plane is allowed to communicate to Worker Nodes on all ports. The Worker Nodes are allowed to communicate with the Control Plane on port 12250.

It is recommended to place the Oracle Databases inside the Private Subnet. Worker Nodes and the Oracle Database will be allowed to communicate freely within the Private Subnet.

When using the Oracle Autonomous Database (ADB), it can either be placed in the Private Subnet with a Private Endpoint, or outside the Virtual Cloud Network with Secured Access. When the ADB is configured for Secured Access, access is allowed from the Virtual Cloud Network and additional, configurable IPs.

Using the Standard Edition you can use a pre-created VCN for the Oracle Backend for Microservices and AI resources.

The following are minimum requirements for a BYO VCN:

• Public Subnet
  • Minimum CIDR: /30
  • Route Table:
    • Internet Gateway to 0.0.0.0/0
• Private Subnet
  • Minimum CIDR: /29
  • Route Table:
    • Service Gateway to “All <Region> Services in the Oracle Services Network”
    • NAT Gateway to 0.0.0.0/0

During the configuration of the Oracle Backend for Microservices and AI, ensure that the Edition is set to Standard:

Tick the “Bring Your Own Virtual Network” checkbox and fill in the appropriate values for the VCN Compartment/Name, Public Subnet Compartment/Name, and Private Subnet Compartment/Name:

• Virtual Cloud Network Compartment : The compartment of the existing VCN.
• Virtual Cloud Network (VCN) : The VCN name.
• Private Subnet Compartment : The compartment of the existing Private Subnet in the VCN.
• Private Subnet : The Private Subnet name (this will automatically translate the name to an OCID).
• Public Subnet Compartment : The compartment of the existing Public Subnet in the VCN.
• Public Subnet : The Public Subnet name (this will automatically translate the name to an OCID).

In OCI, example self-signed certificates are used for TLS communication to the Load Balancer. The self-signed certificates should be replaced with certificates signed by a publicly trusted certificate authority (CA).

1. Ensure your Domain Name System (DNS) entry points to the public IP address associated with the service/ingress-nginx-controller in the ingress-nginx namespace:

   

2. Obtain a new TLS certificate. In a production environment, the most common scenario is to use a public certificate that has been signed by a certificate authority.

3. Create a new Kubernetes secret in the ingress-nginx namespace. For example:

   kubectl -n ingress-nginx create secret tls my-tls-cert --key new-tls.key --cert new-tls.crt
   

4. Modify the service definition to reference the new Kubernetes secret by changing the service.beta.kubernetes.io/oci-load-balancer-tls-secret annotation in the service configuration. For example:

   kubectl patch service ingress-nginx-controller -n ingress-nginx \
       -p '{"metadata":{"annotations":{"service.beta.kubernetes.io/oci-load-balancer-tls-secret":"my-tls-cert"}}}' \
       --type=merge