A shell script is a file containing series of commands .The shell reads this file and run each command as if like it entered on the command line.On a Linux Operating System there are multiple Interpreters which will execute the commands that we pass to them . Default is Bash and this is widely available on various operating systems . Other 's are KSH,SH . Some commands may different on there interpreters but majority will support by all.
In short and simple a shell script is a file containing series of commands .The shell reads this file and run each command as if like it entered on the command line.
The benefit of this is to simplify the day to day mundane activities . Shell is a scripting language interpreter.
Scripts unlock the power of our Linux machine. imagine you have 2 files on with 100 words and another with 100000 words and you need to check first file words on second file and print the word existence and no of times present .
This is where the power of scripting comes .Once you write a simple script then start using it and enhance further to fit into day to day activities .
#!/bin/bash
echo "Hello World"
First line #!/bin/bash is the interpreter which will execute the commands . Next to that are the commands . echo will print the values in the screen .
2.Variables
Variable is a keyword to store some values . We have some System predefined variable and user can also define variables in the scripts to store the results of commands .
Example :
PATH is the OS define variable .run echo $PATH to print the values
VALUE=`date` && echo $VALUE --> here VALUE is user defined variable and it exists till the script finishes .Once finished VALUE variable no more exist.
3.File Manipulations
Below table gives the basic file manipulation operation help full while handling the files.
File Commands
Explaination
> file
Create/Overwrite file
>> file
Append to file
>file 2>&1
redirect both output and error to file
< file
read from file
file1 | file1
pipe output of file1 as input to file2
4.Common Iterations/Loops
Below given blocks are very important because in the scrips for each command that we use we need to check something before executing and store the results some where and iterate through some repetitive steps .
read text file line by line . in many scenarios we need to read file line by line and apply the logic .In the below we are reading and printing with echo.
while read line
do
echo "Line is $line
done < file
Find matching lines
grep foo file --> print the lines that has foo matching word
egrep 'foo|bar' file -->find multiple word and print the matching lines
grep -i FOO file --> same as above but i for ignore case
grep -v foo file --> -v used to print not matching lines
Get the output of command to a variable
FILELIST=`ls`
COUNTOFFILES=`ls -lrt |wc -l`
Case is a good way to Iterate avoiding multiple if/elif blocks
case.sh
#!/bin/bash
# case example
$1 means first argument in the script
# Usage ./case.sh argument
case $1 in
start)
echo starting
;;
stop)
echo stoping
;;
restart)
echo restarting
;;
*)
echo don\'t know
;;
esac
Function declaration and calling ...
multiply() {
expr $1 \*2
}
multiply 3
For loop iterates the in the list of give values and run the logic
for i in 1 2 3 4 5
do
echo "In $i loop"
echo "Do some logic here "
done
5.Useful System Variables
$? --> what the shell returned or previous command return status 0 means success other value means failure This $? verification is very useful to check the status of previous command to take next decession $* --> all arguements ./abc.sh 1 2 --> in this $* is 3 $# --> No of the variable values . where $0 is name of script , $1 is 1st arguement ,$2 is 2nd arguement
6.Test Operators
# Compare 2 variable and do something
if [ "$x" -lt "$y" ] ; then
# Do something
fi
7.Numeric Tests
Numeric Test operations
Explaination
lt
less than
gt
greate than
eq
equal
ne
not equal
ge
greate or equal
le
less than or equal
8.String tests
File Test operations
Explaination
nt
newer than
d
is a directory
f
is a file
r
readable
w
writable
x
executable
9.Logical & String Tests
Logical & String Tests
Explaination
=
equal to
z
zero lenth
n
not zero length
&&
Logical AND
||
logical OR
!
logical Not
10.Argument Variables
Argument Variables
Explaination
$0
Program name or script name
$1
1st argument
$2
2nd arguments
…
…
$9
9th argument
$*
all aguements
$#
No of Arguments
11.Some Advance commands ,utilities ,network and file handling commands
below are some of the commands that we daily use for the day-to-day activities.
Linux Commands
Usage
command1 ||Command2
run command1; if it fails, run Command2
command1 && Command2
run command1; if it works, run Command2
command1 ; Command2
keeping 2 commands on the same line
ls -lSt
list files biggest last
ls -lrt
list files newest last
ls -al
show all fils including hidden
sort -n
sort numarically
wget URL
download url
read x
read some value from user /keyboard
touch file
create empty file
cmd |tee file.txt
command output to stdout also to file.txt
ifconfig -a
list all network interfaces .can see ip here
netstat -r
show routers
netstat -tnpl |grep -I listen
shows all listening ports on the server
ssh u@host
login to host as user u
scp file.txt u@host:\tmp
copy file.txt to host /tmp/ wit u user
alias l='ls -lrt'
creating alias to a command
df -h
show dis mount points
find . -type f -name a.txt
find a.txt in current directory
find . -type f -name *.txt -print
find all the txt files
find /foo -type d -ls
list all directories under foo
awk -F":" '{ print $1 " " $NF}'
print file value in each line of the file delimted with : and NF for last value
tar -cvf abc.tar a.txt b.txt
create acrchive file
tar -tvf abc.tar
check the list of file in abc.tar
tar -rvf abc.tar c.txt
add c.txt to existing abc.tar
tar -xvf abc.tar
extract abc.tar file
tar -zcvf my_archive.tar.gz *
create zip and then tar
tar -zxvf my_archive.tar.gz *
extract zip tar
zcat
view the file without decompressing it like cat
ps -ef |grep keyword
grep some process running
kill -9 pid
kill process with pid
pwd
present working dir
cd
Go to user home
cd ..
Go to previous directory
hosname
hostname of the server
Hope the blog gives some useful information for starting writing scripts ...
There is a requirement to find and replace a string from file's in the in the directory name across all the files and directories in the given directory.
We already know some basic commands like mv,sed/perl & find commands . We will use these basic commands to write a script and get the expected result .
basic commands
mv
mv ABC DEF -> to rename a directory
find
find /dir -type f -exec grep -l "ABC" {} \; | xargs perl -pi -e 's/ABC/DEF/g'
Above command is used find and replace ABC in all the files with DEF on /dir
sed
sed 's/ABC/DEF/g' -->To replace a string in a word
perl
perl -pi -e 's#ABC#DEF#g' file --> this searches and replaces ABC to DEF .
We combine this in to 2 scenarison .
To find and replace the string in given file .
To find and replace the string in the directy recursivley along with matched sub directory string .
On the below we have ABC directory with abc.txt and ACBC directory . Test results for both scenarios 1 and 2 given below .
Scenario 1 :
Scenario2 :
1.sh
#!/bin/bash
echo "Enter file Name"
read fileName
echo "Enter findstring "
read findstring
echo "Enter replacestring "
read replacestring
if [ -f $fileName ] ; then
perl -pi -e 's#'$findstring'#'$replacestring'#g' $fileName
echo "$findstring replace with $replacestring "
else
echo "Given file is not available "
fi
2.sh
#!/bin/bash
echo "Enter Dir Name"
read dir
echo "Enter findstring "
read findstring
echo "Enter replacestring "
read replacestring
if [ -d $dir ] ; then
find $dir -type f -exec grep -l "$findstring" {} \; | xargs perl -pi -e 's/'$findstring'/'$replacestring'/g'
echo " Word $findstring replaced with $replacestring successfully ..."
for directory in `find $dir -type d -name "$findstring"`
do
newDirName=`echo $directory |sed 's/'$findstring'/'$replacestring'/g'`
mv $directory $newDirName
echo " $directory name changed to $newDirName "
done
else
echo "Directory not found : $dir "
fi
set -x and set +x are for debugging . Hope this article helps in detail for find and replace . ..
/host=master/subsystem=elytron/key-store=httpsKS:add(path="${jboss.home.dir}/ssl/jboss.jks", credential-reference={clear-text=jboss@123}, type=JKS) 2.Connect JBOSS Cli mode to configure keystore, key-manager and ssl-context in Elytron
It is better to always remove temp and wstemp when there is any update with application or re-deploy the application . But safer to take backup of the complete profile before making any deletion .
profile_root//temp and profile_root//wstemp these are the location of the temporary files in Websphere Application server .
WebSphere Application Server uses multiple temporary locations for many reasons. This post explains the most commonly used temporary files, why they are used and when they can be removed. This blog will also explain the files and directories that can be removed under the profile direction with caution.
Important:
Be careful in deleting any temporary, cache and log files in WebSphere Application Server!
Before making any changes to the environment , take a backup of the profile. It can be a tape backup, using the backupconfig tool, or using the manageprofile -backupProfile option.
Let's describe the different files and their locations:
profile_root/wstemp Usage: wstemp is a workspace temporary directory. Any changes that you make to the configuration are stored in the wstemp directory temporarily. For example, if you are changing the heap size for an application server, the change is stored in the wstemp location until you save the changes. The concept is same for any administrative client, such as the Integrated solution console, wsadmin or JMX, that you use to make the changes.
Caution: The WebSphere Application Server administrative console stores a preferences.xml file in install_root/wstemp/<workspace_id>. This file contains user preferences on administrative console layout and actions. It is created when you log onto the administrative console. If you remove this file, you lose the user preferences; however, the preferences can be created again the next time you log onto the administrative console.
Do not delete the wstemp files when the server is running (especially deployment manager or node agent servers). This approach can cause unexpected results. Also, do not delete the files when you are unsure about the changes that you made to the configuration. Save any pending changes, stop the deployment manager or node agent, which depends on whether you are removing the dmgr wstemp or node wstemp, and then delete the wstemp files.
Why remove these files?: Files in the profile_root/wstemp directory can be removed. Restart the server process after removal. Because the directory is used by multiple clients, some times you might see multiple files and subdirectories left behind in this directory. For example, when you use the ConfigService MBean to make changes to configuration and you do not discard the session in the code, this directory will never get deleted. Another reason is corruption in the workspace. Corruption can happen when multiple users make changes to the same configuration at the same time.
profile_root/temp Usage: The temp directory is used by multiple WebSphere components. Two good examples are compiled Java ServerPages (JSP) files and web service cache files. Compiled JSP class files (servlets) are stored in this location. The directory might get regenerated when you invoke the JSP again. However, you might experience a performance issue when you invoke the JSP for the very first time after the JSP compiled files have been removed.
Caution: Be cautious if you have a web services application deployed on the node. The wscache.xml is generated during the deployment process and stored under the temp directory. You have to redeploy the web service application to generate the wscache.xml again. You may experience some performance issue with large and complicate webservices application
Why remove these files?: Corrupted JSP files or any non-root permission issues might cause the server start up issue.
Never delete any other files or directories for WebSphere Application Server unless otherwise directed by the IBM Support team.
It is better to always remove tmp and cache when there is any update with application or re-deploy the application . But safer to take backup of the complete directory before making any deletion .
Recently when working with new application update I found that just bouncing/ restarting server is not enough, we will also need to clean up the cache so that new changes take effect. We will see how to clean temporary directories in Weblogic .
Find out the Domain directory under which our Managed instance folder's exist . Some customised Weblogic domain existing on a different location .If you dont know the location then on the Weblogic installation location find domain_registry.xml file . In this file you can see the list of Domain's created and it's paths .
Each managed server have its own tmp and cache directory .
--> Shut down Server.
--> Delete the contents of the folder ORACLE_HOME/user_projects/domains/your_domain/servers/your_server/tmp…
You can also delete ORACLE_HOME/user_projects/domains/your_domain/servers/your_server/cache
-->Restart Server.
You can also do something like this . In your startup script you can add delete statements so that every time the server restarted new tmp and cache will be re-created .
Note: Do not delete any other directories under the server . If you deleted then restoration is difficult and lead to startup issues .
- SSL 3.0 and TLS 1.0 are susceptible to known attacks on the protocol; they are disabled entirely.
- Disabling TLS 1.1 is (as of August 2016) mostly optional; TLS 1.2 provides stronger encryption options, but 1.1 is not yet known to be broken. Disabling 1.1 may mitigate attacks against some broken TLS implementations.
In addition, you can use SSL server which accepts strong encryption only by adding following directives too
~~~~~~~
SSLHonorCipherOrder on
SSLCompression off
SSLSessionTickets off
~~~~~~~
- Enabling SSLHonorCipherOrder ensures that the server's cipher preferences are followed instead of the client's.
There will a situation that the admin account is locked or no back up id to unlock or reset the admin id or completely forget the password then we may need to reset Weblogic Admin security.
Windows
Linux/AIX/Solaris
Assuming Domain path : cd D:\ABC\ABCDomain\
Assuming Domain path : /u01/ABC/ABCDomain/bin
cd D:\ABC\ABCDomain\bin and run below command
cd /u01/ABC/ABCDomain/bin and run below command
setDomainEnv.cmd
. ./setDomainEnv.cmd [ dot then space ]
cd D:\ABC\ABCDomain\security\
cd /u01/ABC/ABCDomain/security
move DefaultAuthenticatorInit.ldift [ Rename the file ]
As a part of OS user authentication there is a possibility to exchange the keys between ID's on same or different OS so that they can communicate without password . it is call password less authentication . Using this one can login to the server with our any "Entering password" / do file transfer from the script where we don't require to Enter password .
In this Post we will use same Server with 2 ID's . We will see how to connect one to another with password then will see how we can configure password less authentication .
In the organisation we may have server to server authentication mostly . some time with in the same server between multiple id's it may need to exchange for ease of day to day operations .
You want to use Linux and OpenSSH to automate your tasks. Therefore you need an automatic login from host A / user source to Host B / user destination. You don't want to enter any passwords, because you want to call ssh from a within a shell script.
How to Create a New User in Linux
To create a new user account, invoke the useradd command followed by the name of the user.
For example to create a new user named username you would run:
Set some password for each of the source and destination id's using passwd userid command
root@ip-172-31-14-154 destination]# passwd source
Changing password for user source.
New password:
BAD PASSWORD: The password fails the dictionary check - it is based on a dictionary word
Retype new password:
passwd: all authentication tokens updated successfully.
[root@ip-172-31-14-154 destination]#
[root@ip-172-31-14-154 destination]#
[root@ip-172-31-14-154 destination]# passwd destination
Changing password for user destination.
New password:
BAD PASSWORD: The password fails the dictionary check - it is based on a dictionary word
Retype new password:
passwd: all authentication tokens updated successfully.
[root@ip-172-31-14-154 destination]#
password of source is zaq12wsx and password of destination is mko09ijn
Let us see how we can do ssh from source to destination . Some OS will have tectia where sshg3 and scpg3 are available instead of ssh and scp
[source@ip-172-31-14-154 ~]$ ssh destination@ip-172-31-14-154
The authenticity of host 'ip-172-31-14-154 (172.31.14.154)' can't be established.
ECDSA key fingerprint is SHA256:bASX/U9HJi3iu0CUsUY+VcYlZR4mE8/b1tJQcl69RpM.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'ip-172-31-14-154,172.31.14.154' (ECDSA) to the list of known hosts.
destination@ip-172-31-14-154: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
[source@ip-172-31-14-154 ~]$
Update /etc/ssh/sshd_config with PasswordAuthentication yes if it was no and then restart sshd service to get rid of the above error
[root@ip-172-31-14-154 destination]# grep -i PasswordAuthentication /etc/ssh/sshd_config
#PasswordAuthentication yes
PasswordAuthentication yes
# PasswordAuthentication. Depending on your PAM configuration,
# PAM authentication, then enable this but set PasswordAuthentication
[root@ip-172-31-14-154 destination]# systemctl restart sshd
[root@ip-172-31-14-154 destination]#
SSH from Source to Destination . Now it will ask to enter destination id password
With Password we are able to authenticate from source to destination id successfully . Now we will see how we can make this password less using the public and private keys of the id's .
Let's do the password authentication by generating a pair of public and private keys of id's and exchange for authentication using below command
ssh-keygen -t rsa -b 4096
bit size can be 2048,1024,3072 or 4098 or any other bit size that supports
Run the command for source id
source@ip-172-31-14-154 ~]$ ssh-keygen -t rsa -b 4096
Generating public/private rsa key pair.
Enter file in which to save the key (/home/source/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/source/.ssh/id_rsa.
Your public key has been saved in /home/source/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:jtcQnDlr00yb91sOVgNn4dH4Gj0cqu79GJ2pZ+Kv3mA source@ip-172-31-14-154.us-east-2.compute.internal
The key's randomart image is:
+---[RSA 4096]----+
| +.|
| . o o.+|
| * . .o*.|
| B o .=oo|
| S = o +o|
| + + o .o.+|
| . o o E++.|
| . .oo*B |
| ..oBO+.|
+----[SHA256]-----+
[source@ip-172-31-14-154 ~]$ cd .ssh/
[source@ip-172-31-14-154 .ssh]$ ls -lrt
total 12
-rw-r--r--. 1 source source 192 May 2 04:49 known_hosts
-rw-r--r--. 1 source source 776 May 2 05:04 id_rsa.pub
-rw-------. 1 source source 3422 May 2 05:04 id_rsa
[source@ip-172-31-14-154 .ssh]$
Run the same command for destination id
[destination@ip-172-31-14-154 ~]$ ssh-keygen -t rsa -b 4096
Generating public/private rsa key pair.
Enter file in which to save the key (/home/destination/.ssh/id_rsa):
Created directory '/home/destination/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/destination/.ssh/id_rsa.
Your public key has been saved in /home/destination/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:V8THYedkP6waQjWcxK+4uFITKsk6ul7gXbqW6m+KDO8 destination@ip-172-31-14-154.us-east-2.compute.internal
The key's randomart image is:
+---[RSA 4096]----+
| +=o.o.+|
| .=o.+=.|
| . o. oo|
| .. . .. .|
| .. ... S.o... |
|. o+o. o o..o |
|...+o . o .. |
|++.+.. . . |
|BOEo ... |
+----[SHA256]-----+
[destination@ip-172-31-14-154 ~]$
ssh-keygen command will create 2 files one id_rsa [ Private key ] and id_rsa.pub [ public key ]
Note : For Source id to connect to Destination then source public key need to upload to destination
When i try to connect still asking password . so trying to un on debug mode vith -vvv
[source@ip-172-31-14-154 .ssh]$ ssh destination@ip-172-31-14-154 -vvv
OpenSSH_8.0p1, OpenSSL 1.1.1g FIPS 21 Apr 2020
debug1: Reading configuration data /etc/ssh/ssh_config
debug3: /etc/ssh/ssh_config line 52: Including file /etc/ssh/ssh_config.d/05-redhat.conf depth 0
debug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf
debug2: checking match for 'final all' host ip-172-31-14-154 originally ip-172-31-14-154
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 3: not matched 'final'
debug2: match not found
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-,gss-gex-sha1-,gss-group14-sha1-]
debug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1]
debug1: configuration requests final Match pass
debug1: re-parsing configuration
debug1: Reading configuration data /etc/ssh/ssh_config
debug3: /etc/ssh/ssh_config line 52: Including file /etc/ssh/ssh_config.d/05-redhat.conf depth 0
debug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf
debug2: checking match for 'final all' host ip-172-31-14-154 originally ip-172-31-14-154
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 3: matched 'final'
debug2: match found
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-,gss-gex-sha1-,gss-group14-sha1-]
debug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1]
debug2: resolving "ip-172-31-14-154" port 22
debug2: ssh_connect_direct
debug1: Connecting to ip-172-31-14-154 [172.31.14.154] port 22.
debug1: Connection established.
debug1: identity file /home/source/.ssh/id_rsa type 0
debug1: identity file /home/source/.ssh/id_rsa-cert type -1
debug1: identity file /home/source/.ssh/id_dsa type -1
debug1: identity file /home/source/.ssh/id_dsa-cert type -1
debug1: identity file /home/source/.ssh/id_ecdsa type -1
debug1: identity file /home/source/.ssh/id_ecdsa-cert type -1
debug1: identity file /home/source/.ssh/id_ed25519 type -1
debug1: identity file /home/source/.ssh/id_ed25519-cert type -1
debug1: identity file /home/source/.ssh/id_xmss type -1
debug1: identity file /home/source/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.0
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.0
debug1: match: OpenSSH_8.0 pat OpenSSH* compat 0x04000000
debug2: fd 4 setting O_NONBLOCK
debug1: Authenticating to ip-172-31-14-154:22 as 'destination'
debug3: hostkeys_foreach: reading file "/home/source/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /home/source/.ssh/known_hosts:1
debug3: load_hostkeys: loaded 1 keys from ip-172-31-14-154
debug3: order_hostkeyalgs: prefer hostkeyalgs: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,ext-info-c
debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes256-cbc,aes128-gcm@openssh.com,aes128-ctr,aes128-cbc
debug2: ciphers stoc: aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes256-cbc,aes128-gcm@openssh.com,aes128-ctr,aes128-cbc
debug2: MACs ctos: hmac-sha2-256-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha1,umac-128@openssh.com,hmac-sha2-512
debug2: MACs stoc: hmac-sha2-256-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha1,umac-128@openssh.com,hmac-sha2-512
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1
debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes256-cbc,aes128-gcm@openssh.com,aes128-ctr,aes128-cbc
debug2: ciphers stoc: aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes256-cbc,aes128-gcm@openssh.com,aes128-ctr,aes128-cbc
debug2: MACs ctos: hmac-sha2-256-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha1,umac-128@openssh.com,hmac-sha2-512
debug2: MACs stoc: hmac-sha2-256-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha1,umac-128@openssh.com,hmac-sha2-512
debug2: compression ctos: none,zlib@openssh.com
debug2: compression stoc: none,zlib@openssh.com
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: aes256-gcm@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: aes256-gcm@openssh.com MAC: <implicit> compression: none
debug1: kex: curve25519-sha256 need=32 dh_need=32
debug1: kex: curve25519-sha256 need=32 dh_need=32
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:bASX/U9HJi3iu0CUsUY+VcYlZR4mE8/b1tJQcl69RpM
debug3: hostkeys_foreach: reading file "/home/source/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /home/source/.ssh/known_hosts:1
debug3: load_hostkeys: loaded 1 keys from ip-172-31-14-154
debug3: hostkeys_foreach: reading file "/home/source/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /home/source/.ssh/known_hosts:1
debug3: load_hostkeys: loaded 1 keys from 172.31.14.154
debug1: Host 'ip-172-31-14-154' is known and matches the ECDSA host key.
debug1: Found key in /home/source/.ssh/known_hosts:1
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey out after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey in after 4294967296 blocks
debug1: Will attempt key: /home/source/.ssh/id_rsa RSA SHA256:jtcQnDlr00yb91sOVgNn4dH4Gj0cqu79GJ2pZ+Kv3mA
debug1: Will attempt key: /home/source/.ssh/id_dsa
debug1: Will attempt key: /home/source/.ssh/id_ecdsa
debug1: Will attempt key: /home/source/.ssh/id_ed25519
debug1: Will attempt key: /home/source/.ssh/id_xmss
debug2: pubkey_prepare: done
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic,password
debug3: preferred gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_lookup gssapi-with-mic
debug3: remaining preferred: publickey,keyboard-interactive,password
debug3: authmethod_is_enabled gssapi-with-mic
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure. Minor code may provide more information
No Kerberos credentials available (default cache: KCM:)
debug1: Unspecified GSS failure. Minor code may provide more information
No Kerberos credentials available (default cache: KCM:)
debug2: we did not send a packet, disable method
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering public key: /home/source/.ssh/id_rsa RSA SHA256:jtcQnDlr00yb91sOVgNn4dH4Gj0cqu79GJ2pZ+Kv3mA
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Trying private key: /home/source/.ssh/id_dsa
debug3: no such identity: /home/source/.ssh/id_dsa: No such file or directory
debug1: Trying private key: /home/source/.ssh/id_ecdsa
debug3: no such identity: /home/source/.ssh/id_ecdsa: No such file or directory
debug1: Trying private key: /home/source/.ssh/id_ed25519
debug3: no such identity: /home/source/.ssh/id_ed25519: No such file or directory
debug1: Trying private key: /home/source/.ssh/id_xmss
debug3: no such identity: /home/source/.ssh/id_xmss: No such file or directory
debug2: we did not send a packet, disable method
debug3: authmethod_lookup password
debug3: remaining preferred: ,password
debug3: authmethod_is_enabled password
debug1: Next authentication method: password
destination@ip-172-31-14-154's password:
chmod 0600 /home/your_home/.ssh/authorized_keys
After that go to /etc/ssh/sshd_config
PubkeyAuthentication yes
systemctl restart sshd
Source Public key is updated on the destination authorized keys . Now try
destination@ip-172-31-14-154 .ssh]$ cat authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqLtvCpcoVJi5NmVNEwgf1+RyNvpvVF5iIKGRAZYZZJv18qeOx5Rm+cIhw3xPmqrEDQrxOJmSwC5Z+CRVO0BHNf8uvln1c6ES+4QevfmKgmTFMy5oIpfyr+00AUHEBAvxlb/y0slvg0LvdP/f5S2UW97AkjioHcb0YUUHGchUuCgePKt/HbVlomoTfQdlf7gm2QejynWIADD6KgOHCqxP4rJuYe9sp3nxC/ViwIqSOsGKHi8bCKf1g8OU9M57A4vlwaORnUKZ8v2mJeXUQkkGJI4cfVfv2tS/Y9jETyBjDR4/m46Kb0que08Swe0pMouOJFK4pXMjNLXOLAiCpIwmtzkiq6Q9G5Qw5FDCo6AiFRXv1IQNCDqXcXRDfhMh6C81u969Xpk+uWi6tYOXIV4cib+aIEU295GXOAEcSbJKujhmXuF6FHUUsHeK1+0etaB29XPjbkYvOHdsbnBwspS0tKp+Siba1+HutKn3kuZGn9HJWdVn83zBh8FIjMIKtb69S9P2zgprM1Y/1M+Bo0yRMxZwTY/FPHx0BXrGIGCzR2qhWeNk/w/N1qkG3nUkDFfX4Lh1IvrCQjHuZwH88JBVsSC7MqFoMlfd7D+1f6q+oQ73ARtdCMwEo2irmUbv5rpnFNFn84MdIhk6UlDs8xhjYAOmhfDKTEVyyATjGIN5r3Q== source@ip-172-31-14-154.us-east-2.compute.internal
[destination@ip-172-31-14-154 .ssh]$ ls -lrt
total 12
-rw-r--r--. 1 destination destination 781 May 2 05:05 id_rsa.pub
-rw-------. 1 destination destination 3434 May 2 05:05 id_rsa
-rw-------. 1 destination destination 776 May 2 05:45 authorized_keys
[destination@ip-172-31-14-154 .ssh]$
[source@ip-172-31-14-154 ~]$ ssh destination@ip-172-31-14-154
Last login: Sun May 2 05:45:50 2021 from 172.31.14.154
[destination@ip-172-31-14-154 ~]$
YESSSS Successful after a couple of issues . Now source is able to connect to destination without password . scp also now works without password authentication
Now we can do vice versa . Means upload destination public key [.pub file ] to source and configure in authorized_keys then destination will be able to connect to source without password .Lets do it quickly .
[destination@ip-172-31-14-154 ~]$ ssh source@ip-172-31-14-154
The authenticity of host 'ip-172-31-14-154 (172.31.14.154)' can't be established.
ECDSA key fingerprint is SHA256:bASX/U9HJi3iu0CUsUY+VcYlZR4mE8/b1tJQcl69RpM.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'ip-172-31-14-154,172.31.14.154' (ECDSA) to the list of known hosts.
source@ip-172-31-14-154's password:
Last login: Sun May 2 05:44:10 2021
[source@ip-172-31-14-154 ~]$
[source@ip-172-31-14-154 ~]$
[source@ip-172-31-14-154 ~]$ ls -lrt
total 4
-rw-r--r--. 1 source source 781 May 2 05:54 id_rsa.pub
[source@ip-172-31-14-154 ~]$
[source@ip-172-31-14-154 .ssh]$ ls -lrt
total 20
-rw-r--r--. 1 source source 776 May 2 05:04 id_rsa.pub
-rw-------. 1 source source 3422 May 2 05:04 id_rsa
-rw-r--r--. 1 source source 776 May 2 05:14 destination@localhost
-rw-r--r--. 1 source source 363 May 2 05:15 known_hosts
-rw-rw-r--. 1 source source 781 May 2 05:55 authorized_keys
Finally we are able to connect to both id's vice versa without password . SCP also now can without password authentication .
[source@ip-172-31-14-154 .ssh]$ ssh destination@ip-172-31-14-154
Last login: Sun May 2 05:56:42 2021 from 172.31.14.154
[destination@ip-172-31-14-154 ~]$ ssh source@ip-172-31-14-154
Last login: Sun May 2 05:57:09 2021 from 172.31.14.154
[source@ip-172-31-14-154 ~]$
If SSH Tectia installed then sshg3 instead of ssh and scpg3 instead of scp tools available . rest of the steps remain same .
awk works on programs that contain rules comprised of patterns and actions. The action is executed on the text that matches the pattern. Patterns are enclosed in curly braces ({}). Together, a pattern and an action form a rule. The entire awk program is enclosed in single quotes (').
Here is the output of -h
Perhaps i don’t need all the output I want only mounted file systems to print and it size let say.
In the above we are ignoring first description line using grep –v. Then printing required values .Infact we can print as many values as we want . NF holds last values .
$0 represents entire line
$1 represents the first field
$2 represent second field
$7 represent 7th field
$NF represents the last record.
In the previous command if we don’t specify delimiter then space is the default . so df –h delimited with space then we are printing the values.Also we used “ “ space when printing .If we use , then space used while printing.
Another example just want to print the date in Mon DD YYYY format
date | awk '{print $2,$3,$6}'
OFS --> Output field Separator
date | awk 'OFS="-" {print$2,$3,$6}'
The BEGIN and END Rules
A BEGIN rule is executed once before any text processing starts. In fact, it’s executed before awk even reads any text. An END rule is executed after all processing has completed. You can have multiple BEGIN and END rules, and they’ll execute in order.
We can also add patterns before printing with AWK . In the below example we will check if the 3rd value greater or equal 1000 the print those rows or fields on those rows .
Combining both BEGIN and patterns on awk
In case if we want to develop our own logic using awk follow below
The first line of the script tells the shell which executable to use
#!/usr/bin/awk -f
BEGIN {
# set the input and output field separators
FS=":"
OFS=":"
# zero the accounts counter
accounts=0
}
{
# set field 2 to nothing
$2=""
# print the entire line
print $0
# count another account
accounts++
}
END {
# print the results
print accounts " accounts.\n"
}
Hope this article help to use awk for data processing with advance utility ...
Follow below steps to get SSL certificate for an JAVA based Application server .In the below steps we will use keytool which is availale in JAVA_HOME/bin/
Create a new keystore keystore.jks for managing your public/private key pairs and certificates.
Note : -v option is for detailed output
Keytool help for the commands
[wlsuser@localhost tmp]$ keytool
Key and Certificate Management Tool
Commands:
-certreq Generates a certificate request
-changealias Changes an entry's alias
-delete Deletes an entry
-exportcert Exports certificate
-genkeypair Generates a key pair
-genseckey Generates a secret key
-gencert Generates certificate from a certificate request
-importcert Imports a certificate or a certificate chain
-importpass Imports a password
-importkeystore Imports one or all entries from another keystore
-keypasswd Changes the key password of an entry
-list Lists entries in a keystore
-printcert Prints the content of a certificate
-printcertreq Prints the content of a certificate request
-printcrl Prints the content of a CRL file
-storepasswd Changes the store password of a keystore
Use "keytool -command_name -help" for usage of command_name
[wlsuser@localhost tmp]$
Generate a CSR in the file carequest.csr for submission to a CA. The CA signs and returns a certificate or a certificate chain that authenticates your public key.
CSR
keytool -certreq -v -alias mycert -file carequest.csr -keystore keystore.jks -storepass storepassword
Send or upload the csr file to the third pary site like and get it signed . Then download root ,intermediate certificates and carequest.cer file .
Print the contents of a certificate file in a human-readable form.
keytool -printcert -v -file carequest.cer
Delete the certificate with the alias aliasname from the keystore keystore.jks.
keytool -delete -alias aliasname -keystore keystore.jks -storepass storepassword
print the certificate
keytool -printcert -f mycert.cer
Now the keystore is ready. we can use this keystore to configure in Application server's like Jboss,Weblogic,Websphere,tomcat ..etc
In a certificate if both Owner and Issuer are same then it is Self signed certificate . We can use self signed cert for the server but it is not secured and not recommended. Every root certificate of the Third party provider are self signed it self and used to sign other certificates .
If we have any SSL handshake issue first we need to understand the details steps that take place during SSL handshake then only we will be able to address it easily . Find the below some of the details .
There are multiple reasons for SSL handshake . When we get this Error on the logs . First of all we need to verify server side whether the SSL configured is 2 Ways SSL or 1 Way SSL .
SSL Handshake Issue troubleshooting :
Make sure the Public keys [ Trusted certificates (root & Intermediate )] are imported in the client truststore .
For 2 Ways SSL both side Signer certificate should be imported .
If Server side enforce the certificate then server certificate also need to import at the client trust store .
Both side should have at-least a common Allowed Protocol . Ex TLS1.2 and a cipher
If something is not satisfied in the above then make the changes accordingly .Like importing the signers in to truststore
keytool -import -file /tmp/root.cer -alias root -keysoore /pathToSSL/***.jks then enter and provide the password .
In some case we can not figure out what is the issue .Then enable SSL debug using below parameter in startup script or in the server JVM arguement .then restart the JVM
-Djavax.net.debug=ssl:handshake:verbose
once restarted test the connectivity . In the logs we can see full debug statements . and also we can see what trusted certs are loaded .There are certain steps in SSL handshake all are printed . Refer below table for SSL Handshake to get better idea .
In the above error handshake failed because Server side rejectClientNegotiation is set to true . and client tried to negotiate for multiple call's renegotiation_info that is where it failed .
Now you can match every step in the SSL handshake debug log with the attached screen to see at what step the failure happened .
