Troubleshooting

This document describes common issues that may occur during the deployment of Oracle SOA Suite on Kubernetes and the steps to troubleshoot them. Also refer to the FAQs page for frequent issues and steps to resolve them.

WebLogic Kubernetes Operator installation failure

If the WebLogic Kubernetes Operator installation failed with timing out:

  • Check the status of the operator Helm release using the command helm ls -n <operator-namespace>.
  • Check if the operator pod is successfully created in the operator namespace.
  • Describe the operator pod using kubectl describe pod <operator-pod-name> -n <operator-namespace> to identify any obvious errors.

RCU schema creation failure

When creating the RCU schema using create-rcu-schema.sh, the possible causes for RCU schema creation failure are:

  • Database is not up and running
  • Incorrect database connection URL used
  • Invalid database credentials used
  • Schema prefix already exists

Make sure that all the above causes are reviewed and corrected as needed.
Also drop the existing schema with the same prefix before rerunning the create-rcu-schema.sh with correct values.

Domain creation failure

If the Oracle SOA Suite domain creation fails when running create-domain.sh, perform the following steps to diagnose the issue:

  1. Run the following command to diagnose the create domain job:

    $ kubectl logs jobs/<domain_job> -n <domain_namespace>

    For example:

    $ kubectl logs jobs/soainfra-create-soa-infra-domain-job -n soans

    Also run:

    $ kubectl describe pod <domain_job> -n <domain_namespace>

    For example:

    $ kubectl describe pod soainfra-create-soa-infra-domain-job-mcc6v -n soans

    Use the output to diagnose the problem and resolve the issue.

  2. Clean up the failed domain creation:

    1. Delete the failed domain creation job in the domain namespace using the command kubectl delete job <domain-creation-job-name> -n <domain-namespace>.
    2. Delete the contents of the domain home directory
    3. Drop the existing RCU schema

  3. Recreate the domain:

    1. Recreate the RCU schema
    2. Make sure the Persistent Volume and Persistent Volume Claim used for the domain are created with correct permissions and bound together.
    3. Rerun the create domain script

Common domain creation issues

A common domain creation issue is error Failed to build JDBC Connection object in the create domain job logs.

Click here to see the error stack trace:

This error is reported when there is an issue with database schema access during domain creation. The possible causes are:

  • Incorrect schema name specified in create-domain-inputs.yaml.
  • RCU schema credentials specified in the secret soainfra-rcu-credentials are different from the credentials specified while creating the RCU schema using create-rcu-schema.sh.

To resolve these possible causes, check that the schema name and credentials used during the domain creation are the same as when the RCU schema was created.

Server pods not started after applying domain configuration file

This issue usually happens when the WebLogic Kubernetes Operator is not configured to manage the domain namespace. You can verify the configuration by running the command helm get values <operator-release> -n <operator-namespace> and checking the values under the domainNamespaces section.

For example:

$ helm get values  weblogic-kubernetes-operator -n opns
USER-SUPPLIED VALUES:
domainNamespaces:
- soans
image: ghcr.io/oracle/weblogic-kubernetes-operator:3.4.4
javaLoggingLevel: FINE
serviceAccount: op-sa
$

If you don’t see the domain namespace value under the domainNamespaces section, run the helm upgrade command in the operator namespace with appropriate values to configure the operator to manage the domain namespace.

$ helm upgrade --reuse-values --namespace opns --set "domainNamespaces={soans}" --wait weblogic-kubernetes-operator charts/weblogic-operator

Ingress controller not serving the domain URLs

To diagnose this issue:

  1. Verify that the Ingress controller is installed successfully.
    For example, to verify the Traefik Ingress controller status, run the following command: 
    $ helm list -n traefik
NAME                    NAMESPACE       REVISION        UPDATED                                 STATUS          CHART               APP VERSION
traefik                  traefik         2               2022-11-30 11:31:18.599876918 +0000 UTC deployed        traefik-20.5.3       v2.9.5
$
  2. Verify that the Ingress controller is setup to monitor the domain namespace.
    For example, to verify the Traefik Ingress controller manages the soans domain namespace, run the following command and check the values under namespaces section. 
    $ helm get values traefik-operator -n traefik
USER-SUPPLIED VALUES:
kubernetes:
   namespaces:
   - traefik
   - soans
$
  3. Verify that the Ingress chart is installed correctly in domain namespace. For example, run the following command: 
    $ helm list -n soans
NAME                    NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                       APP VERSION
soainfra-traefik        soans           1               2021-10-27 11:24:31.7572791 +0000 UTC   deployed        ingress-per-domain-0.1.0    1.0
$
  4. Verify that the Ingress URL paths and hostnames are configured correctly by running the following commands:
    Click here to see the sample commands and output

Security warnings reported in WebLogic Administration console

With the July 2021 PSU applied, WebLogic Server regularly validates your domain configuration settings against a set of security configuration guidelines to determine whether the domain meets key security guidelines recommended by Oracle. If your domain does not meet a recommendation for a security configuration setting, a warning is logged in the Security Warnings Report in the WebLogic Administration Console.

See Review Potential Security Issues in Securing a Production Environment for Oracle WebLogic Server for more information.

Disable Remote Anonymous RMI T3 and IIOP Requests

If you see security warning message that Remote Anonymous RMI T3 or IIOP requests are enabled, resolve this warning by setting RemoteAnonymousRMIT3Enabled and RemoteAnonymousRMIIIOPEnabled attributes to false in domain.yaml with JAVA_OPTIONS before starting the domain as shown below:

serverPod:
    # an (optional) list of environment variable to be set on the servers
    env:
    - name: JAVA_OPTIONS
      value: "-Dweblogic.StdoutDebugEnabled=false -Dweblogic.ssl.Enabled=true -Dweblogic.security.SSL.ignoreHostnameVerification=true -Dweblogic.security.remoteAnonymousRMIT3Enabled=false -Dweblogic.security.remoteAnonymousRMIIIOPEnabled=false"

See link for more details.