Monitor a domain and publish logs

After the Oracle SOA Suite domain is set up, you can:

Monitor the Oracle SOA Suite instance using Prometheus and Grafana

Using the WebLogic Monitoring Exporter you can scrape runtime information from a running Oracle SOA Suite instance and monitor them using Prometheus and Grafana.

Set up monitoring

Follow these steps to set up monitoring for an Oracle SOA Suite instance. For more details on WebLogic Monitoring Exporter, see here.

Publish WebLogic Server logs into Elasticsearch

WebLogic Server logs can be published to Elasticsearch using Fluentd. See Fluentd configuration steps.

Publish SOA server diagnostics logs into Elasticsearch

This section shows you how to publish diagnostics logs to Elasticsearch and view them in Kibana. For publishing operator logs, see this sample.

Prerequisites

If you have not already set up Elasticsearch and Kibana for logs collection, refer to this document and complete the setup.

Publish to Elasticsearch

The diagnostics or other logs can be pushed to Elasticsearch server using logstash pod. The logstash pod should have access to the shared domain home or the log location. In case of the Oracle SOA Suite domain, the persistent volume of the domain home can be used in the logstash pod. To create the logstash pod, follow these steps:

  1. Get the domain home persistence volume claim details of the domain home of the Oracle SOA Suite domain. The following command lists the persistent volume claim details in the namespace - soans. In the example below, the persistent volume claim is soainfra-domain-pvc:

    $ kubectl get pvc -n soans

    Sample output:

    NAME                  STATUS   VOLUME               CAPACITY   ACCESS MODES   STORAGECLASS                    AGE
soainfra-domain-pvc   Bound    soainfra-domain-pv   10Gi       RWX            soainfra-domain-storage-class   xxd

  2. Create the logstash configuration file (logstash.conf). Below is a sample logstash configuration to push diagnostic logs of all servers available at DOMAIN_HOME/servers/<server_name>/logs/-diagnostic.log:

    input {                                                                                                                
  file {                                                                                                               
    path => "/u01/oracle/user_projects/domains/soainfra/servers/**/logs/*-diagnostic.log"                                          
    start_position => beginning                                                                                        
  }                                                                                                                    
}                                                                                                                         
filter {                                                                                                               
  grok {                                                                                                               
    match => [ "message", "<%{DATA:log_timestamp}> <%{WORD:log_level}> <%{WORD:thread}> <%{HOSTNAME:hostname}> <%{HOSTNAME:servername}> <%{DATA:timer}> <<%{DATA:kernel}>> <> <%{DATA:uuid}> <%{NUMBER:timestamp}> <%{DATA:misc}> <%{DATA:log_number}> <%{DATA:log_message}>" ]                                                                                        
  }                                                                                                                    
}                                                                                                                         
output {                                                                                                               
  elasticsearch {                                                                                                      
    hosts => ["elasticsearch.default.svc.cluster.local:9200"]                                                          
  }                                                                                                                    
}

  3. Copy the logstash.conf into /u01/oracle/user_projects/domains so that it can be used for logstash deployment, using the Administration Server pod (for example soainfra-adminserver pod in namespace soans):

    $ kubectl cp logstash.conf  soans/soainfra-adminserver:/u01/oracle/user_projects/domains --namespace soans

  4. Create a deployment YAML (logstash.yaml) for the logstash pod using the domain home persistence volume claim. Make sure to point the logstash configuration file to the correct location (for example, copy logstash.conf to /u01/oracle/user_projects/domains/logstash.conf) and also the correct domain home persistence volume claim. Below is a sample logstash deployment YAML:

    apiVersion: apps/v1
kind: Deployment
metadata:
  name: logstash-soa
  namespace: soans
spec:
  selector:
    matchLabels:
      app: logstash-soa
  template: # create pods using pod definition in this template
    metadata:
      labels:
        app: logstash-soa
    spec:
      volumes:
      - name: soainfra-domain-storage-volume
        persistentVolumeClaim:
          claimName: soainfra-domain-pvc
      - name: shared-logs
        emptyDir: {}
      containers:
      - name: logstash
        image: logstash:6.6.0
        command: ["/bin/sh"]
        args: ["/usr/share/logstash/bin/logstash", "-f", "/u01/oracle/user_projects/domains/logstash.conf"]
        imagePullPolicy: IfNotPresent
        volumeMounts:
        - mountPath: /u01/oracle/user_projects
          name: soainfra-domain-storage-volume
        - name: shared-logs
          mountPath: /shared-logs
        ports:
        - containerPort: 5044
          name: logstash

  5. Deploy logstash to start publish logs to Elasticsearch:

    $ kubectl create -f  logstash.yaml

  6. Now, you can view the diagnostics logs using Kibana with index pattern “logstash-*”.