Status conditions

Contents

Overview

The WebLogic Kubernetes Operator populates status conditions on Domain and Cluster resources to provide high-level status reporting. Status conditions are a Kubernetes standard mechanism and the conditions generated by the operator are similar to those that Kubernetes provides for Pod and Deployment resources.

Checking domain or cluster conditions

Conditions can be found under the status.conditions field in a Domain resource or a Cluster resource.

You can check the conditions in a Domain resource by using: kubectl -n MY_NAMESPACE describe domain MY_DOMAIN_RESOURCE_NAME. Similarly, you can check the conditions in a Cluster resource by using: kubectl -n MY_NAMESPACE describe cluster MY_CLUSTER_NAME.

Click here for an example of a cluster status showing its conditions. 
Status:
  Cluster Name:  cluster-1
  Conditions:
    Last Transition Time:  2022-10-25T16:31:22.682605Z
    Status:                True
    Type:                  Available
    Last Transition Time:  2022-10-25T16:31:22.683156Z
    Status:                True
    Type:                  Completed

The conditions for Cluster resources referenced by a domain are also listed in the domain status under domain.status.clusters.

Click here for an example of a domain status showing both domain and cluster conditions. 
Status:  
  Clusters:
    Cluster Name:  cluster-1
    Conditions:
      Last Transition Time:  2022-10-25T16:31:22.682605Z
      Status:                True
      Type:                  Available
      Last Transition Time:  2022-10-25T16:31:22.682605Z
      Status:                True
      Type:                  Completed
    Label Selector:          weblogic.domainUID=sample-domain1,weblogic.clusterName=cluster-1
    Maximum Replicas:        5
    Minimum Replicas:        0
    Observed Generation:     1
    Ready Replicas:          1
    Replicas:                1
    Replicas Goal:           1
  Conditions:
    Last Transition Time:  2022-10-25T16:44:27.104854Z
    Status:                True
    Type:                  Available
    Last Transition Time:  2022-10-25T16:44:27.104766Z
    Status:                True
    Type:                  Completed

Attributes in a condition

The following attributes can be found in a condition:

  • type - The type of the condition, such as Failed or Available. See Types of domain status conditions .
  • status - The status of the condition, such as True or False.
  • message - An optional, human-readable message providing more details about the condition.
  • reason - The reason for the Failed condition. Not applicable to other types of condition.
  • severity - The severity for the Failed condition. Not applicable to other types of condition.
  • lastTransitionTime - A timestamp of when the condition was created or the last time time the condition transitioned from one status to another.

Types of domain conditions

The following is a list of condition types for a Domain resource.

Failed

  • The desired state of the Domain resource cannot be achieved due to failures encountered in processing the Domain resource. For information about how to diagnose failures, see debugging .
  • The status attribute is always True for a Failed condition. The Failed condition is removed from the domain status when the underlying failure is resolved.
  • The message attribute contains an error message with details of the failure.
  • The reason attribute is set to one of the reasons listed in domain failure reasons .
  • The severity attribute is set to one of the severity levels listed in domain failure severities .
    Click here for an example of a domain status with a Failed condition. 
    Status:
  ...
  Conditions:
    Last Transition Time:  2022-10-24T23:54:49.486543Z
    Message:               One or more server pods that are supposed to be available did not start within the period of time defined in 'serverPod.maxPendingWaitTimeSeconds' under "domain.spec', 'domain.adminServer', 'managedServer', or 'domain.cluster'. Check the server status in the domain status, the server pod status and logs, and WebLogic Server logs for possible reasons. One common cause of this issue is a problem pulling the WebLogicServer image. Adjust the value of 'serverPod.maxPendingWaitTimeSeconds' setting if needed."
    Reason:                ServerPod
    Severity:              Severe
    Status:                True
    Type:                  Failed
    Last Transition Time:  2022-10-24T23:49:35.974905Z
    Message:               No application servers are ready.
    Status:                False
    Type:                  Available
    Last Transition Time:  2022-10-24T23:49:35.974897Z
    Status:                False
    Type:                  Completed
    Last Transition Time:  2022-10-24T23:49:36.173977Z
    Status:                True
    Type:                  Rolling

Completed

  • The status attribute of a Completed condition indicates whether the desired state of the Domain resource has been fully achieved.
  • The status attribute is set to True when all of the following are true:
    • There are no Failed conditions, for example, no failures are detected.
    • One of the following conditions are met:
      • All WebLogic Server pods that are expected to be running are ready at their target image or images, restartVersion, and introspectVersion.
      • No WebLogic Server pods are running and this is the expected state.
      • The domain is configured to have a spec.serverStartPolicy value of AdminOnly and the Administration Server pod is running and ready.
    • There are no pending server shutdown requests.

Available

  • The status attribute is set to True when a sufficient number of pods are ready such that all of the following are true:
    • At least one WebLogic Server pod is ready.
    • Every non-clustered server with a serverStartPolicy value of IfNeeded or Always is ready.
    • Every Cluster resource referenced by the domain has True in its Available condition. Clusters that are configured with a replicas value of 0 or a serverStartPolicy value of Never are ignored.
  • NOTE: The Available status can be True even when the status for the Completed condition is False, a Failed condition is reported, or a cluster has up to cluster.spec.maxUnavailable pods that are not ready.

ConfigChangesPendingRestart

  • This condition tracks the progress of runtime updates to the WebLogic Deploy Tooling model of a Model in Image domain home source type.
  • The status attribute is True if all of the following are true:
    • The Domain resource attribute domain.spec.configuration.model.onlineUpdate.onNonDynamicChanges is CommitUpdateOnly.
    • The Domain resource attribute domain.spec.configuration.model.onlineUpdate.enabled is True.
    • There were model changes and these changes modify non-dynamic WebLogic configuration.
    • Processing successfully completed, including the introspector job.
    • The administrator has not subsequently rolled or restarted each WebLogic Server pod (to propagate the pending non-dynamic changes).
  • The ConfigChangesPendingRestart condition is removed from the domain status when pending non-dynamic runtime updates complete their processing due to a roll or restart.
  • For how to see which pods are awaiting restart using WebLogic pod labels, see Online update status and labels .
    Click here for an example of a domain status with a ConfigChangesPendingRestart condition. 
    Status:
  ...
  Conditions:
    Last Transition Time:  2021-01-20T15:09:15.209Z
    Message:               Online update completed successfully, but the changes require restart and the Domain resource specified 'spec.configuration.model.onlineUpdate.onNonDynamicChanges=CommitUpdateOnly' or not set. The changes are committed but the domain require manually restart to  make the changes effective. The changes are: Server re-start is REQUIRED for the set of changes in progress.

    The following non-dynamic attribute(s) have been changed on MBeans
    that require server re-start:
    MBean Changed : com.bea:Name=oracle.jdbc.fanEnabled,Type=weblogic.j2ee.descriptor.wl.JDBCPropertyBean,Parent=[sample-domain1]/JDBCSystemResources[Bubba-DS],Path=JDBCResource[Bubba-DS]/JDBCDriverParams/Properties/Properties[oracle.jdbc.fanEnabled]
    Attributes changed : Value
    Reason:                      Online update applied, introspectVersion updated to 82
    Status:                      True
    Type:                        ConfigChangesPendingRestart

Rolling

  • This condition indicates that the operator is rolling the server pods in a domain, such as after it has detected an update to the Domain resource or Model in Image model that requires it to perform a rolling restart of the domain.
  • The status attribute is always True for a Rolling condition.
  • The Rolling condition is removed from the domain status when the rolling is completed.

Types of cluster conditions

The following is a list of condition types for a Cluster resource.

Completed

  • The status attribute of a Completed condition indicates whether the desired state of the Cluster resource has been fully achieved.
  • The status attribute is set to True when all of the following are true:
    • There are no Failed domain.status.conditions.
    • No WebLogic Server pods are expected to be running, or all server pods in the cluster that are expected to be running are ready at their target image or images, restartVersion, and introspectVersion
    • There are no pending server shutdown requests.

Available

  • The status attribute is set to True when a sufficient number of WebLogic Server pods are ready in the cluster. Both of the following must be true:
    • At least one pod in the cluster is expected to run and is ready.
    • The number of not ready server pods, which are expected to run, is less than or equal to the value of cluster.spec.maxUnavailable, which defaults to 1.
  • Examples:
    • If a cluster has a serverStartPolicy of Never, or replicas is 0, or a cluster is in a domain with serverStartPolicy of AdminOnly or Never, then the cluster will have an Available condition that is False.
    • If a cluster and domain both have a serverStartPolicy of IfNeeded, and cluster.spec.replicas is 1, then the cluster will have an Available condition that is True only when its single pod is ready.
    • If a cluster and domain both have a serverStartPolicy of IfNeeded, cluster.spec.replicas is 4, and cluster.spec.maxUnavailable is 1 (the default), then the cluster will have an Available condition that is True only when three or four of its pods are ready.
  • NOTE: The Available condition’s status can be True even when the status for the Completed condition is False, a Failed condition is reported on the Domain resource, or the cluster has up to cluster.spec.maxUnavailable pods that are not ready.

Conditions life cycle

After the operator detects a Cluster or Domain resource, it will ensure that each resource always has exactly one occurrence of an Available and of a Completed condition. The operator will not subsequently remove these conditions but instead, will change their status to True or False as appropriate. You can use this fact as a way to determine when an operator has detected a Domain or Cluster resource.

The Failure, Rolling, and ConfigChangesPendingRestart conditions are ephemeral; they are only added when they apply, and are removed when they no longer apply.

The Failure condition is the only condition that can have multiple occurrences. This occurs when there are multiple concurrent failures of different types.

Conditions and generations

Use metadata.generation and status.observedGeneration in Domain and Cluster resources to detect when their conditions and other status are up-to-date. Specifically, conditions may not be up-to-date when domain.status.observedGeneration generation does not equal domain.metadata.generation, or when cluster.status.observedGeneration does not equal cluster.metadata.generation in any of the domain’s Cluster resources. This may indicate either the operator is in the process of updating the status or the operator is not running.

Conditions and events

A corresponding event is generated when an Available, Completed, Failure, or Rolling condition has changed. For details, see operator-generated event types .