Friday, May 3, 2013

Permissions broken on /bin/chmod

Imagine you have a Linux system and some administrator ran the following command:

# chmod a-x /bin/chmod

If you try reversing that after it's been done you'll get an error.

# chmod a+x /bin/chmod

-bash: /bin/chmod: Permission denied

I was asked this exact scenario in a Job Interview recently. The interesting thing to me is that one of the interviewers was leading me to believe that if you touched a new file (say /bin/changemethen the default permissions for that new file would be rwxr-xr-x. This is incorrect. The default permissions for a new file on Linux are 666. After you apply the default root umask of 022, the permissions on that new file would be 644, or rw-r--r--

So how do you go about fixing this problem? The real resolution is to copy a file that already exists in /bin. For example the file /bin/touch. You can then use dd or cat to get the contents of the /bin/chmod command into the copy file. You will then be able to use the copy file to "chmod" the /bin/chmod file to the correct permissions. See the example below for the actual commands to do this.

# cp /bin/touch /bin/changeme
# cat /bin/chmod > /bin/changeme
# dd if=/bin/chmod of=/bin/changeme
# /bin/changeme a+x /bin/chmod

Your /bin/chmod command will now function as normal.

Monday, February 7, 2011

Storix and SAN storage detection

As noted in the comments, this issue no longer exists since version I want to also make it clear that I have been very happy both with the Storix software and with their support.

I ran into an issue in using Storix backup software. When beginning a backup, Storix does a query to make sure that / is not on an exclude list. It will query LVM volume groups to get this information about /. I discovered (with the help of Storix support) that SBAdmin by default will only query /dev/sd[a-z] for the root filesystem.
If you have the LVM volume group that the root filesystem is on, on SAN storage, the Storix backup will give an error similar to the following:

The / filesystem exists on the system but is not included on the
backup because it is in a logical volume you did not select to
include. You must include volume group rootvg on a system backup.

In looking further you may notice this warning in your Storix logs:

Volume Group rootvg will be ignored because it contains physical
volume sdba, which is not a supported device type.

This gives you the physical volume name that is important in the next step.

To fix the problem, on the client that is being backed up edit the file:
There will be a section that is called "Hard Disk". Inside that section look for the line that contains the disk that your root volume group is on. In our example above, it is physical volume sdba. So we look for the line that is:

#disk:sdb:a-z::1-15:SCSI disk

Uncomment this line so it reads as follows.

disk:sdb:a-z::1-15:SCSI disk

If our physical volume had been sdey. then we would have been looking for the line that looked like:

#disk:sde:a-z::1-15:SCSI disk

We would have uncommented that line so that it read:

disk:sde:a-z::1-15:SCSI disk

When you have made this change, save the changes and retry the backup. At this point it should succeed.

Friday, July 16, 2010

Timekeeping on Linux VMware Guests

Timekeeping on Linux VMware guests can be a cumbersome and frustrating experience. I'm creating this post to be able to remind myself of the various settings required to make things work. The bulk of this information is taken from VMware KB article #1006427 located at:

VMware recommends using NTP and turning off VMware Tools periodic time synchronization in the guest. To turn this off, there are three options:

  1. Set tools.syncTime = "False" in the configuration file (.vmx file) of the virtual machine.

  2. OR

  3. Deselect Time synchronization between the virtual machine and the host operating system in the VMware Tools toolbox GUI of the guest operating system.

  4. OR

  5. Run the vmware-guestd --cmd "vmx.set_option synctime 1 0" command in the guest operating system.
To work properly in a VM, some vendor kernels require certain kernel parameters. In our environment, we only run SLES and RHEL so I'm only concerned about those two. If you run something different, check the VMware KB article linked to above.


Linux Version

Kernel Parameters

64bitRHEL 5.4 and later
No additional kernel parameters.
32bitRHEL 5.4 and later
No additional kernel parameters.
64bitRHEL 5.3notsc divider=10
32bitRHEL 5.3divider=10 clocksource=acpi_pm
64bitSLES 10.2 and later
No additional kernel parameters.
32bitSLES 10.2 and later
Use a VMI enabled kernel.

You now want to make sure that NTP is configured correctly for your environment.

The following is a sample /etc/ntp.conf :

restrict default nomodify nopeer
driftfile /var/lib/ntp/drift

Wednesday, January 6, 2010

Remove LUNs from Linux

I have a previous post here on how to add SAN Presentations (LUNs) to Linux. This post uses the same QLogic Dynamic Target and LUN Discovery utilities as the previous post. The steps below apply to SLES 10 and RHEL 5 servers using QLogic HBAs and PowerPath. That is what we have at my workplace and so that is what I'm most familiar with. This post also assumes that you have put the qlogic utilities into /tmp/ql_utils as described in the other post.

To remove a LUN and all associated PowerPath and Linux devices from the host environment follow these steps. These steps must be run with root privileges. (Note that it is critical to follow the procedures in exact order because deviating from the procedures can cause the host to panic.) These steps were put together using steps from QLogic KB articles.

1. Stop any I/O to the device being removed. [Unmount the filesystem or make sure that the disk has been completely removed from within Oracle ASM control.] It is critical to stop all I/O on the device that is being removed.

2. Run the following command to determine which native SCSI devices are mapped to the pseudo device:

powermt display dev=all

Find the name of the LUN to be removed and match it up with the emcpower device name. This will need to be done on each server individually.

3. Run the command:

powermt remove dev=emcpower

Where corresponds to the LUN to be removed.

4. Run the command:

powermt release

Failing to run this command results in the pseudo device still being visible in /dev and /sys/block and may lead to some complications when the new devices are dynamically added.

5. In the /tmp/ql_utils directory, there should be a script to rescan the qlogic hbas. (If this script doesn't exist, see the previous post for the steps to get the script.) The script is called Run the script:


6. You will now remove the device from the storage array using your array admin utilities. On the Linux server, run powermt display to verify that the device has been removed.

7. Now remove the LUN from the Storage Group (CLARiiON) or unmap it from the FA ports (DMX).

Display WWN in SLES 9

I have another post regarding how to do this on SLES 10, so I decided to create a post on how to do this in SLES 9. In SLES 9 you can easily find the WWN for QLogic HBA's by running:

# cat /proc/scsi/qla2xxx/* | grep node

Monday, May 11, 2009

Setup SAN presentation on Linux

The steps below are to scan for new LUNs from a SAN after the LUNs have been presented from the storage side when using QLogic HBA's . This post uses the QLogic Dynamic Target and LUN Discovery utilities which can be downloaded from:

The steps below will work on SUSE Linux Enterprise Server (SLES) 10 and Red Hat Enterprise Linux (RHEL) 5.
You can discover what version of Linux you are on by running:

cat /etc/*ease

Create a directory to hold the utilites. In this examle we will use the /tmp/ql_utils directory:

mkdir /tmp/ql_utils

Change to the /tmp/ql_utils directory:

cd /tmp/ql_utils

Retrieve the utilities from the download above and extract them from the zip file into the /tmp/ql_utils directory.
This will put five files into /tmp/ql_utils:

The file that we are going to be using is the file. We need to ensure that it is set to executable:

chmod a+x /tmp/ql_utils/

The script has several options available. You can see what these options are by running:

/tmp/ql_utils/ -h

The commands to scan for new LUNs are listed below. After the commands we'll describe what each command does. These commands should be run with root priveledges while inside the /tmp/ql_utils directory.

powermt display dev=all | egrep "Pseudo|Logical" > before
Please make sure there is no active I/O before running this script
Do you want to continue: (yes/no)? yes
powermt config
powermt display dev=all | egrep "Pseudo|Logical" > after
diff after before
powermt save

The first line outputs the current list of LUNs to the file named before. This is optional, but, makes it easy to see what new LUNs have been discovered later on.
The second line actually does the scan for new LUNs and prompts you to make sure that it's ok to run the script. Answer yes to the prompt.
The fifth line (powermt config) creates the emcpower devices in /dev.
The sixth line outputs the new list of LUNs to the file named after.
The seventh line runs a command to compare the list of new LUNs to the list of old LUNs. The differences will be displayed on screen. Make sure that names of the new LUNs show up in the output.
The last line saves the configuration.

After this is done, the LUNs can be setup with LVM. (see LVM post)

Wednesday, April 8, 2009

Slides from SLLUG PAM training

I taught a training session at the daytime Salt Lake Linux Users Group meeting today. I wanted to make the slides available to those that attended. They probably won't make a whole lot of sense if you didn't attend the meeting.
Slides from Pam Training presentation