Category: Linux

Create a new C file.

#include <stdio.h>

int main()
{
    puts((void *)((unsigned long long)-1));
    return 0;

}

Compile and run.

[root@slave log]# gcc -o seg seg.c
[root@slave log]# ./seg
Segmentation fault
[root@slave log]# tail /var/log/messages
Aug 26 23:39:18 slave kernel: seg[7788]: segfault at fffffffffffffff0 ip 00007fee2413c231 
sp 00007ffdb268c488 error 4 in libc-2.12.so[7fee240bb000+18a000]
[root@slave log]#

It is very useful if you are working on custom monitoring to track segfault daemons/services/applications on your server.

Sometimes you will see this error when you want to extend filesystem / logical volume.

Error Message:  0516-787 extendlv: Maximum allocation for logical volume <LV_Name> is 512.
Maximum number of LPs for the logical volume has been exceeded – must increase the allocation

It indicates that your LP allocation is insufficient, hence it needs to be increased. Maximum LP value can be increased on-the-fly without unmounting the LV first.

Case study.

I’m trying to increase additional 300GB to my SAP backup directory but failed with 0516-787 error.

sappr08(DMP)# chfs -a size=+300G /sapdb/DMP/backup
0516-787 extendlv: Maximum allocation for logical volume lvDMPbackup
is 5120.

Use “lslv lvDMPbackup” to verify max LPs value for your logical volume.

sappr08(DMP)# lslv lvDMPbackup
LOGICAL VOLUME:     lvDMPbackup            VOLUME GROUP:   sdbdmpvg
LV IDENTIFIER:      00f60f5200004c000000012abb2ed482.9 PERMISSION:     read/write
VG STATE:           active/complete        LV STATE:       opened/syncd
TYPE:               jfs2                   WRITE VERIFY:   off
MAX LPs:            5120                   PP SIZE:        256 megabyte(s)
COPIES:             1                      SCHED POLICY:   parallel
LPs:                4594                   PPs:            4594
STALE PPs:          0                      BB POLICY:      relocatable
INTER-POLICY:       minimum                RELOCATABLE:    yes
INTRA-POLICY:       middle                 UPPER BOUND:    1024
MOUNT POINT:        /sapdb/DMP/backup      LABEL:          /sapdb/DMP/backup
MIRROR WRITE CONSISTENCY: on/ACTIVE
EACH LP COPY ON A SEPARATE PV ?: no
Serialize IO ?:     NO

Now what we need is to increase LP allocation for your logical volume. You can calculate how many LP needed before we can expand the filesystem successfully.

Formula: LV Size in MB / LP size in MB

How to calculate LP value.

1. Get your LV size in MB. Use df -m <filesystem>

sappr08(DMP)# df -m /sapdb/DMP/backup
Filesystem    MB blocks      Free %Used    Iused %Iused Mounted on
/dev/lvDMPbackup 1483264.00 511005.14   66%     4690     1% /sapdb/DMP/backup
sappr08(DMP)#

So, LV size = 1483264 MB

2. Get  PP size from your LV

sappr08(DMP)# lslv lvDMPbackup
LOGICAL VOLUME:     lvDMPbackup            VOLUME GROUP:   sdbdmpvg
LV IDENTIFIER:      00f60f5200004c000000012abb2ed482.9 PERMISSION:     read/write
VG STATE:           active/complete        LV STATE:       opened/syncd
TYPE:               jfs2                   WRITE VERIFY:   off
MAX LPs:            6000                   PP SIZE:        256 megabyte(s)
COPIES:             1                      SCHED POLICY:   parallel
LPs:                4594                   PPs:            4594
STALE PPs:          0                      BB POLICY:      relocatable
INTER-POLICY:       minimum                RELOCATABLE:    yes
INTRA-POLICY:       middle                 UPPER BOUND:    1024
MOUNT POINT:        /sapdb/DMP/backup      LABEL:          /sapdb/DMP/backup

PP size: 256

Now, pop out your trusty calculator and find new LP value.

sappr08(DMP)# echo "1483264/256" | bc
5794
sappr08(DMP)#

So you need to increase at least 5794 PP to make 300GB filesystem extension successful. In this case, I’ll just raise my max PP to 6000.

sappr08(DMP)# chlv -x 6000 lvDMPbackup

Verify new PP value again with “lslv” command.

sappr08(DMP)# lslv lvDMPbackup
LOGICAL VOLUME:     lvDMPbackup            VOLUME GROUP:   sdbdmpvg
LV IDENTIFIER:      00f60f5200004c000000012abb2ed482.9 PERMISSION:     read/write
VG STATE:           active/complete        LV STATE:       opened/syncd
TYPE:               jfs2                   WRITE VERIFY:   off
MAX LPs:            6000                   PP SIZE:        256 megabyte(s)
COPIES:             1                      SCHED POLICY:   parallel
LPs:                4594                   PPs:            4594
STALE PPs:          0                      BB POLICY:      relocatable
INTER-POLICY:       minimum                RELOCATABLE:    yes
INTRA-POLICY:       middle                 UPPER BOUND:    1024
MOUNT POINT:        /sapdb/DMP/backup      LABEL:          /sapdb/DMP/backup
MIRROR WRITE CONSISTENCY: on/ACTIVE
EACH LP COPY ON A SEPARATE PV ?: no
Serialize IO ?:     NO
INFINITE RETRY:     no
DEVICESUBTYPE:      DS_LVZ
COPY 1 MIRROR POOL: None
COPY 2 MIRROR POOL: None
COPY 3 MIRROR POOL: None

Now you may proceed with filesystem extension.

sappr08(DMP)# chfs -a size=+300G /sapdb/DMP/backup
Filesystem size changed to 3037724672
sappr08(DMP)# df -g /sapdb/DMP/backup
Filesystem    GB blocks      Free %Used    Iused %Iused Mounted on
/dev/lvDMPbackup   1448.50    499.03   66%     4690     1% /sapdb/DMP/backup

This post solved your problem? I appreciate if you can treat me a cup of coffee 😉

Symptom

User is not able to “su” to another account (be it local, or LDAP based)

[admazwan_ngali@oracle~]$ su - servicesoasit
Password:
Password:
su: incorrect password

/var/log/secure will display this error log.

Jul 21 23:53:37 oracle su[22863]: pam_vas: Authentication <succeeded> for <Active Directory> 
user: <servicesoasit> account: <ServiceSOASIT@AMER.DELL.COM> service: <su-l> reason: <N/A> Access Control Identifier(NT Name):<AMERICAS\ServiceSOASIT>
Jul 21 23:53:38 oracle su[22863]: pam_unix(su-l:auth): authentication failure; 
logname=admazwan_ngali uid=2184379 euid=2184379 tty=pts/6 ruser= rhost=  user=servicesoasit

As you can see pam_vas already verified authentication is a success, but “su” still refused you to switch user.

Troubleshooting

Ensure setuid is set on /bin/su file. Not sure why it’s changed, probably it happened during OEL upgrade recently.

[root@oracle pam.d]# ll /bin/su
-rwxr-xr-x 1 root root 28336 Oct 16  2012 /bin/su

Try to “su” to another account again. Issue should be fixed.

[root@oracle pam.d]# chmod +s /bin/su
[root@oracle pam.d]# ll /bin/su
-rwsr-xr-x 1 root root 28336 Oct 16  2012 /bin/su

[admazwan_ngali@oracle ~]$ su - servicesoasit
Password:
[servicesoasit@oracle~]$

Working on some performance related issue today and user requested average CPU/Memory utilization history on previous days, so I came out with quick script to pull the data.

Tested on Oracle Enterprise Linux. Should be working fine on any RHEL based distribution. Sysstat package is required to enable sar report on your server.

#!/bin/bash

# Get Average CPU/Memory Utilization History from sysstat file in /var/log/sa/*
# Author: azwan.ngali[AT]gmail.com


for file in $(ls -la /var/log/sa/* | grep sa[0-9] | awk '{print $9}')
do
        sar -f $file | head -n 1
        printf "\n"

        # Get CPU idle average, it's pretty straight forward.

        printf "CPU average: "
        sar -u -f $file | grep Average: | awk -F " " '{sum = (100 - $8) } END { print sum "%" }'

        # Get Average Memory utilization

        # Information being displayed in sar -r command is somewhat misleading.
        # As it is merely calculated by the formula kbmemused/(kbmemused+kbmemfree) * 100
        # But actually that was not the case, in order to get memory calculation, 
        # here's the revised formula to include memory cache/buffer information into account.
        # 
        # Formula:
        # (kbmemused-kbbuffers-kbcached) / (kbmemfree + kbmemused) * 100
        # The reason behind this is Linux treats unused memory as a wasted resource and so uses as 
        # much RAM as it can to cache process/kernel information
		
        printf "Memory Average: "
        sar -r -f $file | grep Average | awk -F " " '{ sum = ($3-$5-$6)/($2+$3) * 100   } END { print sum "%" }'

        printf "\n"
done

Upon execution, it will search all sa* file in /var/log/sa and perform basic calculation to display CPU/memory average. It may be handy if you’re lazy like me.

[root@ausuovmfmtap3 sa]# ./averagesar.sh
Linux 2.6.18-274.el5 (ausuovmfmtap3.xx)        07/01/2014
CPU average: 4.76%
Memory Average: 15.6925%
 
Linux 2.6.18-274.el5 (ausuovmfmtap3.xx)        07/02/2014
CPU average: 3.4%
Memory Average: 14.3805%
 
Linux 2.6.18-274.el5 (ausuovmfmtap3.xx)        07/03/2014
CPU average: 3.35%
Memory Average: 14.576%
 
Linux 2.6.18-274.el5 (ausuovmfmtap3.xx)        07/04/2014
CPU average: 3.97%
Memory Average: 17.8241%
 
Linux 2.6.18-274.el5 (ausuovmfmtap3.xx)        07/05/2014
CPU average: 4.44%
Memory Average: 20.4096%
 
Linux 2.6.18-274.el5 (ausuovmfmtap3.xx)        07/06/2014
CPU average: 4.58%
Memory Average: 20.6211%
 
Linux 2.6.18-274.el5 (ausuovmfmtap3.xx)        07/07/2014
CPU average: 4.77%
Memory Average: 18.3188%
 
Linux 2.6.18-274.el5 (ausuovmfmtap3.xx)        07/08/2014
CPU average: 3.34%
Memory Average: 14.8783%
 
Linux 2.6.18-274.el5 (ausuovmfmtap3.xx)        07/09/2014
CPU average: 3.44%
Memory Average: 15.1599%
 
[root@ausuovmfmtap3 sa]#

Toodles.

Last week one of our business partner had issue mapping their Samba path to their workstation with access denied error.

C:\Users\azwan_ngali>net use * \\sambaserver\sys2\dellsftw\barcodes "password" /
user:domain\serviceaccount

System error 5 has occured.

Access is denied.

Despite user’s service account is a part of valid users group in /etc/samba/smb.conf, he’s still not able to map it.

Found out that the samba path is actually a symlink pointing to another directory which the service account has read / write access. Service account is a part of ap2_dev_cpdev_bar member.

lrwxrwxrwx 1 root root 51 Mar 17 2011 barcodes -> /stornext/snfs1/data/common/sys2/dellsftw/barcodes/

drwxrwxr-x 25510 root ap2_dev_cpdev_bar 960033 Jun 16 17:08 barcodes

To fix this problem you need to modify your SAMBA configuration.

Add / modify these three lines to enable SAMBA mapping to symlink paths.

follow symlinks = yes
wide links = yes
unix extensions = yes

Save the file and restart Samba service.

service smb restart

Try remapping the Samba path again. Access denied error will be gone.