Domain and Cluster resources

This document describes how to create your own Domain and Cluster resources.

Contents

Overview

Domain resources reference WebLogic domain configuration, a WebLogic install, images, and anything else necessary to run the domain. Beginning with operator 4.0, WebLogic clusters that are within a WebLogic domain configuration may optionally be associated with a Cluster resource in addition to a Domain resource. This Cluster resource makes it simpler to scale the number of running member servers using kubectl scale, the Kubernetes built-in Horizontal Pod Autoscaling, or similar tools. These Cluster resources are only active if referenced by a Domain resource. NOTE: This Cluster resource is a new custom resource different from the Domain resource. In earlier operator versions, you specified the life cycle attributes of WebLogic clusters in the Domain resource cluster section. Beginning with operator 4.0, the Domain resource cluster section references only the name of the Cluster resources; all the life cycle attributes are specified in the new Cluster resource objects.

Many of the samples accompanying the operator project include scripts to generate initial Domain and Cluster resources from a set of simplified inputs; however, these resources are the actual source of truth for how the operator will manage each WebLogic Server domain. You are encouraged to either start with the YAML files generated by the various samples, or create Domain and Cluster resources manually or by using other tools, based on the schema referenced here or in this documentation.

Prerequisites

The following prerequisites must be fulfilled before proceeding with the creation of domain and cluster resources:

  • Create a Kubernetes Namespace unless your intention is to use the default namespace.
  • Make sure the WebLogic Kubernetes Operator is running and is configured to monitor the namespace. Depending on the configuration, you may need to update the operator by running helm upgrade to be able to monitor a new namespace. For more information, see Namespace management.
  • Choose a domain home source type.
  • Make sure any resources that will be referenced are deployed to the same namespace. For example, all domain resources have a spec.webLogicCredentialsSecret field that references a Kubernetes Secret containing the username and password of the WebLogic Server administrative account.
  • Make sure any domain and cluster resources and the corresponding WebLogic configuration meet Kubernetes resource name restrictions.

Deploying domain and cluster resource YAML files

Domains and clusters are defined using YAML files. For each WebLogic Server domain you want to run, you should create one YAML file defining a Domain resource and, optionally, one Cluster resource for each WebLogic cluster, and apply it. In the following referenced example, the sample scripts generate a YAML file that you can use as a basis. Copy the file and override the default settings so that it matches all the WebLogic Server domain parameters that define your domain.

See the WebLogic Server samples, Domain home on a PV, Domain home in Image, and Model in Image.

After you have written your YAML files, you use them to create your domain artifacts using the kubectl apply -f command.

$ kubectl apply -f domain-resource.yaml
$ kubectl apply -f cluster-resource.yaml

To confirm that the Domain and any Clusters were created, use this command:

$ kubectl get weblogic -n <namespace>

To view all of the attributes of a running domain, including the domain’s status, use this command:

$ kubectl describe domain <domain-resource-name> -n <namespace>

Or this command:

$ kubectl get domain <domain-resource-name> -n <namespace> -o yaml

To view all of the attributes of a cluster, use this command:

$ kubectl describe cluster <cluster-resource-name> -n <namespace>

Domain and cluster resource attribute references

The domain resource metadata section names the Domain and its namespace. The name of the Domain is the default value for the domainUID which is used by the operator to distinguish domains running in the Kubernetes cluster that may have the same domain name. The Domain name must be unique in the namespace and the domainUID should be unique across the cluster. The domainUID, Domain resource name, and domain name (from the WebLogic domain configuration) may all be different. Similarly, the cluster resource metadata section names the Cluster and its namespace. The metadata.name of a cluster resource must be referenced from the spec.clusters field of the domain resource. This name does not need to be the same as the name of the WebLogic cluster. The spec.clusterName field of the cluster resource must name the WebLogic cluster from the domain configuration.

The domain resource spec section describes the intended running state of the domain, including intended runtime state of WebLogic Server instances, and details about Kubernetes Pod or Service generation, such as resource constraints, scheduling requirements, or volume mounts. The cluster resource spec section describes the intended number of member servers of that cluster to run and can set or override the other settings inherited from the domain resource.

The operator automatically updates the status section of a deployed domain or cluster resource to describe the actual running state, including WebLogic Server instance runtime states and current health.

Here are some references you can use for the fields in these sections:

Using kubectl explain

You can access the description of any field using kubectl explain. For instance, the following command displays the description of the domainUID field:

$ kubectl explain domain.spec.domainUID
KIND:     Domain
VERSION:  weblogic.oracle/v9

FIELD:    domainUID <string>

DESCRIPTION:
     Domain unique identifier. It is recommended that this value be unique to
     assist in future work to identify related domains in active-passive
     scenarios across data centers; however, it is only required that this value
     be unique within the namespace, similarly to the names of Kubernetes
     resources. This value is distinct and need not match the domain name from
     the WebLogic domain configuration. Defaults to the value of
     `metadata.name`.

NOTE: The VERSION field’s value may be different, depending on the operator version you are using.

Domain and cluster spec elements

The Domain spec section contains elements for configuring the domain operation and sub-sections specific to the Administration Server, specific clusters, or specific Managed Servers. The Cluster spec section contains elements for configuring the lifecycle options for all of the Managed Server members of a WebLogic cluster, including Java options, environment variables, additional Pod content, and the ability to explicitly start, stop, or restart cluster members.

Domain spec elements

Elements related to domain identification, container image, and domain home:

  • domainUID: Domain unique identifier. This identifier is required to be no more than 45 characters (for more details, see Meet Kubernetes resource name restrictions. It is recommended that this value be unique to assist in future work to identify related domains in active-passive scenarios across data centers; however, it is only required that this value be unique within the namespace, similarly to the names of Kubernetes resources. This value is distinct and need not match the domain name from the WebLogic domain configuration. Defaults to the value of metadata.name.
  • image: The WebLogic container image; required when domainHomeSourceType is Image or FromModel (when not specifying an auxiliary image); otherwise, defaults to container-registry.oracle.com/middleware/weblogic:12.2.1.4. WARNING: The default image is unpatched and therefore, should always be specified to use an image with the latest patches installed (for example, container-registry.oracle.com/middleware/weblogic_cpu:12.2.1.4-slim-jdk8-ol8). For more information, see Understand Oracle Container Registry images.
  • imagePullPolicy: The image pull policy for the WebLogic container image. Legal values are Always, Never, and IfNotPresent. Defaults to Always if the image ends in :latest; IfNotPresent, otherwise. To ensure that all nodes have the same image, any images that are updated without changing the tag should be set to Always.
  • imagePullSecrets: A list of image pull Secrets for the WebLogic container image.
  • domainHome: The directory containing the WebLogic domain configuration inside the container. Defaults to /shared/domains/domains/ if domainHomeSourceType is PersistentVolume. Defaults to /u01/oracle/user_projects/domains/ if domainHomeSourceType is Image. Defaults to /u01/domains/ if domainHomeSourceType is FromModel.
  • domainHomeSourceType: Domain home file system source type: Legal values: Image, PersistentVolume, FromModel. Image indicates that the domain home file system is present in the container image specified by the image field. PersistentVolume indicates that the domain home file system is located on a persistent volume. FromModel indicates that the domain home file system will be created and managed by the operator based on a WDT domain model.
  • dataHome: An optional directory in a server’s container for data storage of default and custom file stores. If dataHome is not specified or its value is either not set or empty, then the data storage directories are determined from the WebLogic domain configuration.

Elements related to logging:

  • includeServerOutInPodLog: Specifies whether the server .out file will be included in the Pod’s log. Defaults to true.
  • logHome: The directory in a server’s container in which to store the domain, Node Manager, server logs, server *.out, introspector .out, and optionally HTTP access log files if httpAccessLogInLogHome is true. Ignored if logHomeEnabled is false. See also logHomeLayout.
  • logHomeEnabled: Specifies whether the log home folder is enabled. Defaults to true if domainHomeSourceType is PersistentVolume; false, otherwise.
  • logHomeLayout: Specifies how log files are organized under the logHome directory when logHomeEnabled is set to true. Defaults to ByServers, where domain logs and introspector.out are placed at root level and all other log files are organized under logHome/servers/<server-name>/logs. If Flat is specified, then all files are placed at root level.
  • httpAccessLogInLogHome: Specifies whether the server HTTP access log files will be written to the same directory specified in logHome. Otherwise, server HTTP access log files will be written to the directory configured in the WebLogic domain configuration. Defaults to true.
  • fluentdSpecification: Automatic fluentd sidecar injection. If specified, the operator will deploy a sidecar container alongside each WebLogic Server instance that runs the fluentd,

Elements related to failure retry:

  • failureRetryIntervalSeconds: The wait time in seconds before the start of the next retry after a Severe failure. Defaults to 120.
  • failureRetryLimitMinutes: The time in minutes before the operator will stop retrying Severe failures. Defaults to 1440.

Elements related to security:

  • webLogicCredentialsSecret: Reference to a Kubernetes Secret that contains the user name and password needed to boot a WebLogic Server under the username and password fields.
  • See also the following elements under configuration.

Elements related to domain startup and shutdown:

  • serverStartPolicy: The strategy for deciding whether to start a WebLogic Server instance. Legal values are AdminOnly, Never, or IfNeeded. Defaults to IfNeeded.
  • restartVersion: Changes to this field cause the operator to restart WebLogic Server instances.
  • replicas: The default number of cluster member Managed Server instances to start for each WebLogic cluster in the domain configuration, unless replicas is specified for that cluster in its Cluster resource.
    • For each cluster, first the operator will sort cluster member Managed Server names from the WebLogic domain configuration by normalizing any numbers in the Managed Server name and then sorting alphabetically. This is done so that server names such as “managed-server10” come after “managed-server9”.
    • Then, the operator will start Managed Servers from the sorted list, up to the replicas count, unless specific Managed Servers are specified as starting in their entry under the managedServers field. In that case, the specified Managed Servers will be started and then additional cluster members will be started, up to the replicas count, by finding further cluster members in the sorted list that are not already started. Note that if cluster members are started because of their entries under managedServers, then a cluster may have more cluster members running than its replicas count. Defaults to 1.
  • maxClusterConcurrentStartup: The maximum number of cluster member Managed Server instances that the operator will start in parallel for a given cluster, if maxConcurrentStartup is not specified for a specific cluster under the clusters field. A value of 0 means there is no configured limit. Defaults to 0.
  • maxClusterConcurrentShutdown: The default maximum number of WebLogic Server instances that a cluster will shut down in parallel when it is being partially shut down by lowering its replica count.
  • introspectVersion: Changes to this field cause the operator to repeat its introspection of the WebLogic domain configuration (see Initiating introspection). Repeating introspection is required for the operator to recognize changes to the domain configuration, such as adding a new WebLogic cluster or Managed Server instance, to regenerate configuration overrides, or to regenerate the WebLogic domain home when the domainHomeSourceType is FromModel. Introspection occurs automatically, without requiring change to this field, when servers are first started or restarted after a full domain shut down. For the FromModel domainHomeSourceType, introspection also occurs when a running server must be restarted because of changes to any of the fields listed here. See also overrideDistributionStrategy.

Elements related to specifying and overriding WebLogic domain configuration:

  • These elements are under configuration.

    • overridesConfigMap: The name of the ConfigMap for WebLogic configuration overrides.
    • overrideDistributionStrategy: Determines how updated configuration overrides are distributed to already running WebLogic Server instances following introspection when the domainHomeSourceType is PersistentVolume or Image. Configuration overrides are generated during introspection from Secrets, the overridesConfigMap field, and WebLogic domain topology. Legal values are Dynamic, which means that the operator will distribute updated configuration overrides dynamically to running servers, and OnRestart, which means that servers will use updated configuration overrides only after the server’s next restart. The selection of OnRestart will not cause servers to restart when there are updated configuration overrides available. See also introspectVersion. Defaults to Dynamic.
    • secrets: A list of names of the Secrets for WebLogic configuration overrides or model.
    • introspectorJobActiveDeadlineSeconds: The introspector job timeout value in seconds. If this field is specified, then the operator’s ConfigMap data.introspectorJobActiveDeadlineSeconds value is ignored. Defaults to 120 seconds.
  • These elements are under configuration.model, only apply if the domainHomeSourceType is FromModel, and are discussed in Model in Image.

    • configMap: (Optional) Name of a ConfigMap containing WebLogic Deploy Tooling model YAML files or .properties files.
    • domainType: WebLogic Deploy Tooling domain type. Legal values: WLS, RestrictedJRF, JRF. Defaults to WLS.
    • runtimeEncryptionSecret: The name of the Secret containing the runtime encryption password, which must be in a field named password. Required when domainHomeSourceType is set to FromModel.
    • modelHome: Location of the WebLogic Deploy Tooling model home directory, which can include model YAML files, .properties variable files, and application .zip archives. Defaults to /u01/wdt/models.
    • onlineUpdate.*: Settings related to the online update option for Model In Image dynamic updates.
      • onlineUpdate.enabled: Enable online update for model changes to a running domain. Default is false.
      • onlineUpdate.onNonDynamicChanges: Controls behavior when non-dynamic WebLogic configuration changes are detected during an online update. Non-dynamic changes are changes that require a domain restart to take effect. Valid values are CommitUpdateOnly (default) and CommitUpdateAndRoll. For more information, see Online update handling of non-dynamic WebLogic configuration changes in the Runtime Updates section of the Model in Image user guide.
      • onlineUpdate.wdtTimeouts.*: Rarely needed timeout settings for online update calls to the WebLogic domain from WebLogic Deploy Tooling within the introspector job. All timeouts are specified in milliseconds and default to two or three minutes. For a full list of timeouts, call kubectl explain domain.spec.configuration.model.onlineUpdate.wdtTimeouts.
  • These elements are under configuration.opss, and only apply if the domainHomeSourceType is FromModel and the domainType is JRF.

    • walletPasswordSecret: Name of a Secret containing the OPSS key passphrase, which must be in a field named walletPassword. Used to encrypt and decrypt the wallet that is used for accessing the domain’s entries in its RCU database.
    • walletFileSecret: Name of a Secret containing the OPSS key wallet file, which must be in a field named walletFile. Use this to allow a JRF domain to reuse its entries in the RCU database. This allows you to specify a wallet file that was obtained from the domain home after the domain was booted for the first time.

Elements related to Kubernetes Pod and Service generation:

  • serverPod: Customization affecting the generation of Pods for WebLogic Server instances.
  • serverService: Customization affecting the generation of ClusterIP Services for WebLogic Server instances.

Sub-sections related to the Administration Server, specific clusters, or specific Managed Servers:

  • adminServer: Lifecycle options for the Administration Server, including Java options, environment variables, additional Pod content, and which channels or network access points should be exposed using a NodePort Service.
  • clusters: Beginning with operator 4.0 and the weblogic.oracle/v9 API version of the domain resource, this field lists the referenced Cluster resources. The Cluster resource now has lifecycle options for all the Managed Server members of a WebLogic cluster, including Java options, environment variables, additional Pod content, and the ability to explicitly start, stop, or restart cluster members. The spec.clusterName field of each Cluster resource must match a cluster that already exists in the WebLogic domain configuration.
  • managedServers: Lifecycle options for individual Managed Servers, including Java options, environment variables, additional Pod content, and the ability to explicitly start, stop, or restart a named server instance. The serverName field of each entry must match a Managed Server that already exists in the WebLogic domain configuration or that matches a dynamic cluster member based on the server template.

The elements serverStartPolicy, serverPod and serverService are repeated under adminServer and managedServers and as part of each Cluster resource. The values directly under spec, set the defaults for the entire domain. The values that are part of a Cluster resource set the defaults for cluster members of that cluster. The values under adminServer or an entry under managedServers, set the values for that specific server. Values from the domain scope and values from the cluster (for cluster members) are merged with or overridden by the setting for the specific server depending on the element. See Startup and shutdown for details about serverStartPolicy combinations.

Elements related to the customization of liveness and readiness probes:

Cluster spec elements

For a complete list of the Cluster spec elements, see Cluster Spec.

Additional information for the clusterService.sessionAffinity element:

  • sessionAffinity: This is an advanced setting that is applicable only when the kube-proxy is running in non-default proxy modes, such as User space (legacy) proxy mode and IPVS proxy mode. It is used to enable session affinity based on the client’s IP addresses. For more information, see the Virtual IPs and service proxies. Must be ClientIP or None. Defaults to None.

    NOTE: This setting is not applicable when the kube-proxy is running in the default iptables proxy mode.

For additional domain and cluster resource attribute reference material, see Domain and cluster resource attribute references.

JVM memory and Java option environment variables

You can use the following environment variables to specify JVM memory and JVM option arguments to WebLogic Server Managed Server and Node Manager instances:

  • JAVA_OPTIONS: Java options for starting WebLogic Server.
  • USER_MEM_ARGS: JVM memory arguments for starting WebLogic Server.
  • NODEMGR_JAVA_OPTIONS: Java options for starting a Node Manager instance.
  • NODEMGR_MEM_ARGS: JVM memory arguments for starting a Node Manager instance; this will take precedence over JAVA_OPTIONS and USER_MEM_ARGS.
  • WLST_PROPERTIES: System properties for WLST commands in introspector jobs or WebLogic Server instance containers.
  • WLST_EXTRA_PROPERTIES: System properties appended to WLST_PROPERTIES for WLST commands in introspector jobs or WebLogic Server instance containers.
  • WLSDEPLOY_PROPERTIES: System properties for WebLogic Deploy Tooling commands during Model in Image introspector jobs or WebLogic Server instance containers.
  • PRE_CLASSPATH: Path(s) that are prepended to the WebLogic Server system classpath; delimit multiple paths with a colon :.
  • CLASSPATH: Path(s) that are appended to the WebLogic Server system classpath; delimit multiple paths with a colon :.

NOTES:

  • The following behavior occurs depending on whether or not NODEMGR_JAVA_OPTIONS and NODEMGR_MEM_ARGS are defined:
    • If NODEMGR_JAVA_OPTIONS is not defined and JAVA_OPTIONS is defined, then the JAVA_OPTIONS value will be applied to the Node Manager instance.
    • If NODEMGR_MEM_ARGS is not defined and USER_MEM_ARGS is defined, then the USER_MEM_ARGS value will be applied to the Node Manager instance.
    • If nothing else is specified, then the default Java property values (-Xms64m -Xmx100m -Djava.security.egd=file:/dev/./urandom) will be applied to the Node Manager instance.
  • The USER_MEM_ARGS and WLST_EXTRA_PROPERTIES environment variables both default to -Djava.security.egd=file:/dev/./urandom in all WebLogic Server pods and the WebLogic introspection job. They can be explicitly set to another value in your Domain or Cluster YAML file using the env attribute under the serverPod configuration.
  • Notice that the NODEMGR_MEM_ARGS, USER_MEM_ARGS, and WLST_EXTRA_PROPERTIES environment variables all include -Djava.security.egd=file:/dev/./urandom by default. This helps to speed up the Node Manager and WebLogic Server startup on systems with low entropy, plus similarly helps to speed up introspection job usage of the WLST encrypt command.
  • For a detailed description of Java and pod memory tuning see the Pod memory and CPU resources FAQ.
  • You can use JAVA_OPTIONS and WLSDEPLOY_PROPERTIES to disable Fast Application Notifications (FAN); see the Disable Fast Application Notifications FAQ for details.

This example snippet illustrates how to add some of the previously mentioned environment variables using the env attribute under the serverPod configuration in your Domain or Cluster YAML file.

# Copyright (c) 2017, 2021, Oracle and/or its affiliates.
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl.
#
apiVersion: "weblogic.oracle/v9"
kind: Domain
metadata:
  name: domain1
  namespace: domains23
  labels:
    weblogic.domainUID: domain1
spec:
  serverPod:
    # an (optional) list of environment variable to be set on the servers
    env:
    - name: JAVA_OPTIONS
      value: "-Dweblogic.StdoutDebugEnabled=false "
    - name: USER_MEM_ARGS
      value: "-Djava.security.egd=file:/dev/./urandom "
    - name: NODEMGR_JAVA_OPTIONS
      value: "-Dweblogic.StdoutDebugEnabled=false "
    - name: NODEMGR_MEM_ARGS
      value: "-Xms64m -Xmx100m -Djava.security.egd=file:/dev/./urandom "
    - name: PRE_CLASSPATH
      value: "/sample/path/prepended/to/weblogic/system/classpath:/another/prepended/path"
    - name: CLASSPATH
      value: "/sample/path/appended/to/weblogic/system/classpath:/another/appended/path"

Pod generation

The operator creates a Pod for each running WebLogic Server instance. This Pod will have a container, named weblogic-server, based on the container image specified by the image field. Additional Pod or container content can be specified using the elements under serverPod. This includes Kubernetes sidecar and init containers, labels, annotations, volumes, volume mounts, scheduling constraints, including anti-affinity, resource requirements, or security context.

Customer provided labels and annotations may not begin with “weblogic” and the operator will generate the following labels:

  • weblogic.createdByOperator: "true"
  • weblogic.domainName: <domain-name>, where <domain-name> is the name of the WebLogic domain
  • weblogic.domainUID: <uid>, where <uid> is the domain UID from the Domain resource
  • weblogic.serverName: <server-name>, where <server-name> is the name of the WebLogic Server instance
  • weblogic.clusterName: <cluster-name>, where <cluster-name> is the name of the cluster of which this instance is a member, if any
  • weblogic.domainRestartVersion: <restart-version>, matches domain.spec.restartVersion after the pod is up to date with this version
  • weblogic.introspectVersion: <introspect-version>, matches domain.spec.introspectVersion after the pod is up to date with this version

Prior to creating a Pod, the operator replaces variable references allowing the Pod content to be templates. The format of these variable references is $(VARIABLE_NAME) where VARIABLE_NAME is one of the variable names available in the container for the WebLogic Server instance. The default set of environment variables includes:

  • DOMAIN_NAME: The WebLogic Server domain name.
  • DOMAIN_UID: The domain unique identifier.
  • DOMAIN_HOME: The domain home location as a file system path within the container.
  • SERVER_NAME: The WebLogic Server instance name.
  • CLUSTER_NAME: The WebLogic cluster name, if this is a cluster member.
  • LOG_HOME: If the `domain.spec.logHomeEnabled’ attribute is set to true, then this contains the WebLogic log location as a file system path within the container

This example Domain and Cluster YAML file specifies that Pods for WebLogic Server instances in the cluster-1 cluster will have a per-Managed Server volume and volume mount (similar to a Kubernetes StatefulSet), an init container to initialize some files in that volume, and anti-affinity scheduling so that the server instances are scheduled, as much as possible, on different Nodes:

# Copyright (c) 2019, 2021, Oracle and/or its affiliates.
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl.
#
apiVersion: "weblogic.oracle/v9"
kind: Domain
metadata:
  name: domain1
  namespace: domains23
  labels:
    weblogic.domainUID: domain1
spec:
  domainHome: /u01/oracle/user_projects/domains/domain1
  domainHomeSourceType: Image
  image: "phx.ocir.io/weblogick8s/my-domain-home-in-image:12.2.1.4"
  imagePullPolicy: "IfNotPresent"
  imagePullSecrets:
  - name: ocirsecret
  webLogicCredentialsSecret:
    name: domain1-weblogic-credentials
  includeServerOutInPodLog: true
  serverStartPolicy: IfNeeded
  serverPod:
    env:
    - name: JAVA_OPTIONS
      value: "-Dweblogic.StdoutDebugEnabled=false"
    - name: USER_MEM_ARGS
      value: "-Djava.security.egd=file:/dev/./urandom "

  clusters:
  - domain1-cluster-1
---
apiVersion: "weblogic.oracle/v1"
kind: Cluster
metadata:
  name: domain1-cluster-1
  namespace: domains23
  labels:
    weblogic.domainUID: domain1
spec:
- clusterName: cluster-1
  serverPod:
    volumes:
    - name: $(SERVER_NAME)-volume
      emptyDir: {}
    volumeMounts:
    - mountPath: /server-volume
      name: $(SERVER_NAME)-volume
    initContainers:
    - name: volumeinit
      image: "oraclelinux:7-slim"
      imagePullPolicy: IfNotPresent
      command: ["/usr/bin/sh"]
      args: ["echo", "Replace with command to initialize files in /init-volume"]
      volumeMounts:
      - mountPath: /init-volume
        name: $(SERVER_NAME)-volume
  replicas: 2

The operator uses an “introspection” job to discover details about the WebLogic domain configuration, such as the list of clusters and network access points. The Job Pod for the introspector is generated using the serverPod entries for the Administration Server. Because the Administration Server name is not known until the introspection step is complete, the value of the $(SERVER_NAME) variable for the introspection job will be “introspector”.