Websphere Application server failed to start after Applying 8.5.5.18 patch . From the logs i see below error .
[11/21/20 20:32:17:992 GMT+08:00] 00000001 SSLComponentI A CWPKI0007I: SSL service failed to start successfully
[11/21/20 20:32:17:994 GMT+08:00] 00000001 WsServerImpl E WSVR0100W: An error occurred initializing, server1 [class com.ibm.ws.runtime.component.ServerImpl]
com.ibm.ws.exception.ConfigurationError: com.ibm.websphere.ssl.SSLException: java.lang.NullPointerException
at com.ibm.ws.ssl.core.SSLComponentImpl.initialize(SSLComponentImpl.java:208)
...
Caused by: java.lang.NullPointerException
at com.ibm.ws.ssl.config.WSKeyStore$1.run(WSKeyStore.java:745)
at com.ibm.ws.security.util.AccessController.doPrivileged(AccessController.java:118)
at com.ibm.ws.ssl.config.WSKeyStore.do_getKeyStore(WSKeyStore.java:645)
at com.ibm.ws.ssl.config.WSKeyStore.getKeyStore(WSKeyStore.java:911)
at com.ibm.ws.ssl.config.WSKeyStore.initializeKeyStore(WSKeyStore.java:1098)
at com.ibm.ws.ssl.config.SSLConfigManager.parseSecureSocketLayer1(SSLConfigManager.java:1191)
at com.ibm.ws.ssl.config.SSLConfigManager.parseSSLConfig(SSLConfigManager.java:751)
at com.ibm.ws.ssl.config.SSLConfigManager.initializeServerSSL(SSLConfigManager.java:302)
FFDC:
[11/21/20 20:32:17:937 GMT+08:00] FFDC Exception:java.lang.NullPointerException SourceId:com.ibm.ws.ssl.core.SSLConfigManager.initializeServerSSL ProbeId:312 Reporter:com.ibm.ws.ssl.config.SSLConfigManager@6e37fe8a
java.lang.NullPointerException
at com.ibm.ws.ssl.config.WSKeyStore$1.run(WSKeyStore.java:745)
at com.ibm.ws.security.util.AccessController.doPrivileged(AccessController.java:118)
at com.ibm.ws.ssl.config.WSKeyStore.do_getKeyStore(WSKeyStore.java:645)
at com.ibm.ws.ssl.config.WSKeyStore.getKeyStore(WSKeyStore.java:911)
at com.ibm.ws.ssl.config.WSKeyStore.initializeKeyStore(WSKeyStore.java:1098)
at com.ibm.ws.ssl.config.SSLConfigManager.parseSecureSocketLayer1(SSLConfigManager.java:1191)
at com.ibm.ws.ssl.config.SSLConfigManager.parseSSLConfig(SSLConfigManager.java:751)
at com.ibm.ws.ssl.config.SSLConfigManager.initializeServerSSL(SSLConfigManager.java:302)
at com.ibm.ws.ssl.core.SSLComponentImpl.initialize(SSLComponentImpl.java:145)
.
So something related to your security settings..
This error occurs on the environment where custom password jar is used . and there is are testfixes available .Just like add JVM arguements
The client channel definition table (CCDT) determines the channel definitions and authentication information used by client applications to connect to the queue manager.
Ex:It has channel name ,ConnName ,QMGRName,cipher … etc
The purpose of the client channel definition table (CCDT) is to determine the channel definitions used by client applications to connect to the queue manager. The channel definition also specifies the authentication information that applies to the connections. The CCDT is a binary file. It is generated by a queue manager. The queue manager does not read the CCDT file.
There are a number of ways for a client application to use a CCDT. The CCDT can be copied to the client computer. You can copy the CCDT to a location shared by more than one client. You can make the CCDT accessible to the client as a shared file, while it remains located on the server.
If you use FTP to copy the file, use the bin option to set binary mode; do not use the default ASCII mode. Whichever method you choose to make the CCDT available, the location must be secure to prevent unauthorized changes to the channels.
The CCDT is created when the queue manager is created. Each entry of a CCDT represents a client connection to a specific queue manager. A new entry is added when you define a client-connection channel using the DEFINE CHANNEL command, and the entry is updated when you alter the client-connection channels by using the ALTER CHANNEL command.
The path to the CCDT can be changed by setting MQCHLLIB. If you do set MQCHLLIB, be aware, if you have multiple queue managers on the same server, they share the same CCDT location.
On Linux/AIX
/var/mqm/qmgrs/QUEUEMANAGERNAME/@ipcc
On Windows
MQ_INSTALLATION_PATH\data\qmgrs\QUEUEMANAGERNAME\@ipcc
MQCHLTAB specifies the name of the file containing the client channel definition table (ccdt). The default file name is AMQCLCHL.TAB.
On Windows:
SET MQCHLTAB=filenameCopy code
On UNIX and Linux® systems:
export MQCHLTAB=filename
Let us create a chaneel tab file for the below connection from QMGR
[mqm@ip-172-31-14-154 ~]$ runmqsc APPQM1
5724-H72 (C) Copyright IBM Corp. 1994, 2020.
Starting MQSC for queue manager APPQM1.
DEFINE CHANNEL(CL.APPQM1) CHLTYPE(CLNTCONN) TRPTYPE(TCP) CONNAME('172.31.14.154(1417)') QMNAME(APPQM1)
1 : DEFINE CHANNEL(CL.APPQM1) CHLTYPE(CLNTCONN) TRPTYPE(TCP) CONNAME('172.31.14.154(1417)') QMNAME(APPQM1)
AMQ8014I: IBM MQ channel created.
DEFINE CHANNEL(CL.APPQM1) CHLTYPE(SVRCONN) TRPTYPE(TCP)
2 : DEFINE CHANNEL(CL.APPQM1) CHLTYPE(SVRCONN) TRPTYPE(TCP)
AMQ8014I: IBM MQ channel created.
end
3 : end
2 MQSC commands read.
No commands have a syntax error.
All valid MQSC commands were processed.
[mqm@ip-172-31-14-154 ~]$
AMCLCHL.TAB file is a binary file . If we view we can see connection information but cannot read complete file .
Now if i want to create another Channel tab from the same QMGR with different channel then it is difficult because this file will be modified or updated with all the channel information from the QMGR .
From the Latest version onwards we have an easy way to generate Channel Tab using runmsc -n command . By default runmqsc -n Look for the channle tab in /var/mqm . If we run this command then it will read the file from that location if exist otherwise it will create new so that we can update the definitons of channels that we intended to create for Application connectivity .
Note: We only need SVRCONN on QMGR Side . CLNTCONN is only need for Application that used CHLTAB file . so we will define only CLNTCONN on channel tab file .
[mqm@ip-172-31-14-154 @ipcc]$ cd /var/mqm
[mqm@ip-172-31-14-154 ~]$ ls -lrt
total 24
drwxrwsr-x. 3 mqm mqm 19 Dec 29 15:56 conv
drwxrwsr-x. 2 mqm mqm 6 Dec 29 15:56 config
drwxrwsr-x. 3 mqm mqm 27 Dec 29 15:56 web
drwxrwsrwx. 2 mqm mqm 6 Dec 29 15:56 trace
-rw-rw-r--. 1 mqm mqm 1941 Dec 29 15:56 service.env
drwxrwsr-x. 5 mqm mqm 53 Dec 29 15:56 mqft
drwxrwsr-x. 4 mqm mqm 32 Dec 29 15:56 shared
-rw-rw-r--. 1 mqm mqm 661 Dec 29 15:56 mqclient.ini
drwxrwsr-x. 3 mqm mqm 27 Dec 29 15:56 exits
drwxrwsr-x. 3 mqm mqm 27 Dec 29 15:56 exits64
drwxrwsr-x. 15 mqm mqm 177 Jan 4 11:44 sockets
drwxrwsr-x. 15 mqm mqm 177 Jan 4 11:44 qmgrs
-rw-rw-r--. 1 mqm mqm 3674 Jan 4 11:44 mqs.ini
drwxrwsr-x. 14 mqm mqm 162 Jan 4 11:44 log
drwxrwsrwx. 2 mqm mqm 8192 Jan 9 07:06 errors
[mqm@ip-172-31-14-154 ~]$
[mqm@ip-172-31-14-154 ~]$ runmqsc -n
5724-H72 (C) Copyright IBM Corp. 1994, 2020.
Starting local MQSC for 'AMQCLCHL.TAB'.
dis chl(*)
1 : dis chl(*)
AMQ9518E: File '/var/mqm/AMQCLCHL.TAB' not found.
end
2 : end
No commands have a syntax error.
[mqm@ip-172-31-14-154 ~]$
This is when try to read a channel tab if the file diesnot exist . Let's create one .refer below
[mqm@ip-172-31-14-154 ~]$ ls -lrt /var/mqm/AMQCLCHL.TAB
-rw-rw-r--. 1 mqm mqm 2106 Jan 10 05:53 /var/mqm/AMQCLCHL.TAB
[mqm@ip-172-31-14-154 ~]$
If you want to create in different location then we have to set MQCHLLIB.
For Windows:
SET MQCHLLIB=pathname
For UNIX and Linux systems:
export MQCHLLIB=pathname
Migration and client channel definition tables (CCDT) In general, the internal format of the client channel definition table might change from one release level of IBM WebSphere MQ to the next. As a result, an IBM WebSphere MQ MQI client can use a client channel definition table only when it has been prepared by a server queue manager that is at the same release level as the client, or at an earlier release level.
If we generate a CHLTAB and try to use in Older version of client it may not work . Take the working TAB file and edit using runmqsc -n then modify then it should work fine .
Every one have requirement to monitor the Weblogic performance in terms for Heap, GC, class loading ..etc during project development phase for performance stats /Load test results . As a part of this we will be seeing how we can enable JMX port in Weblogic and monitor each JVM for the details statics .
Please note there is some level of Monitoring available in Weblogic but that is instant value at that current point of time . We can not see the historical data for ex past 1 hour data . So enable JMX port on Weblogic and connect to Weblogic using tools like Jconsole,JVisualVM ,Java Machine control (JMC) using it core technology protocol . by having this we can do highly efficient performance tuning debugging tasks like flight recordings etc
Steps for enabling JMX is sam in Weblogic 10 ,11g ,12c, and 14c .
setDomainEnv.sh (or) setDomainEnv.cmd file can be found in the weblogic DOMAIN_HOME/bin directory to configure JVM arguments.
set JAVA_OPTIONS=%JAVA_OPTIONS% -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=11111 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false
Add the above in the respective files and restart Weblogic .
setDomainEnv file is global and the parameter . so the port which is configure can be usable by only one JVM which ever starts first and other will get port conflicts . So We need to add different port for each JVM for this . Port can be anything available on the server.
As per my Experience there will be custom script defined for each JVM so better Add the above given JVM arguments individually for each JVM so that which ever JVM you want to connect can easily connect .
In other way you can add in setDomanEnv file with an if condition for each of you JVM like below . In your case JVM name's may be different .
if [ "$SERVER_NAME" == "AdminSer" ] then JAVA_OPTIONS="${JAVA_OPTIONS} -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=11111 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false export JAVA_OPTIONS
elif [ "$SERVER_NAME" == "AppSrv01" ] then JAVA_OPTIONS="${JAVA_OPTIONS} -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=11112 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false export JAVA_OPTIONS fi
use jconsole hostname:port to connect from jconsoe
or double click jvisualVM and connect using IP and PORT without credentials .
Above is an insecure way of connecting . If you want to try in Highr environments then better use below
-Dcom.sun.management.jmxremote.password.file=jmxremote.password -Dcom.sun.management.jmxremote.authenticate=true Here we have used the access configuration files named jmxremote.access and jmxremote.password
jmxremote.access file
# The "monitor" role has readonly access.
# The "admin" role has readwrite access.
monitor readonly
admin readwrite
jmxremote.password file
# specify actual password instead of the text password
monitor password
admin password
A queue manager on which the interface queues are not located, but on which they are defined as cluster queues. Such a queue manager is called a gateway queue manager.
Here FR1 and FR2 actually hold the interface queue .Ex QL1 Application connecting to GRQM doenst hold the actual Queue but puts a message into an interface queue because the queue is not managed by GWQM .GWQM checks its cluster queue defninitions.It recognizes that QL1 is managed by FR1,Fr2 so GWQM puts the message to QL1 on one of the QMGR's (FR1,FR2). subsequent messages will be load balenced to FR1 and FR2
define ql(QL1) cluster(Test) defbind(notfixed) Defined QL1 on Both FR1 and FR2
Sending 10 messages to QL1 using GWQM
bash-4.4$ cat put.sh
#!/bin/bash
for i in `seq 10` ; do
echo "Sending $i message"
echo "Test MSG $i" |/opt/mqm/samp/bin/amqsput QL1 GWQM >/dev/null 2>&1
done
bash-4.4$
Messages can also be routed to a gateway queue manager from queue managers that are external to the cluster. Refer below Diagram .
Application have its own QMGR (non-cluster) and it connects to GWQM . If the Application side has another MQ cluster then GWQM will be used to OverLap two cluster's
