Storix may fail with SAN storage

I ran into an issue in using Storix backup software. When beginning a backup, Storix does a query to make sure that / is not on an exclude list. It will query LVM volume groups to get this information about /. I discovered (with the help of Storix support) that SBAdmin by default will only query /dev/sd[a-z] for the root filesystem.
If you have the LVM volume group that the root filesystem is on, on SAN storage, the Storix backup will give an error similar to the following:

The / filesystem exists on the system but is not included on the
backup because it is in a logical volume you did not select to
include. You must include volume group rootvg on a system backup.

In looking further you may notice this warning in your Storix logs:

Volume Group rootvg will be ignored because it contains physical
volume sdba, which is not a supported device type.

This gives you the physical volume name that is important in the next step.

To fix the problem, on the client that is being backed up edit the file:
/opt/storix/config/device_info
There will be a section that is called "Hard Disk". Inside that section look for the line that contains the disk that your root volume group is on. In our example above, it is physical volume sdba. So we look for the line that is:

#disk:sdb:a-z::1-15:SCSI disk

Uncomment this line so it reads as follows.

disk:sdb:a-z::1-15:SCSI disk

If our physical volume had been sdey. then we would have been looking for the line that looked like:

#disk:sde:a-z::1-15:SCSI disk

We would have uncommented that line so that it read:

disk:sde:a-z::1-15:SCSI disk

When you have made this change, save the changes and retry the backup. At this point it should succeed.

Timekeeping on Linux VMware Guests

Timekeeping on Linux VMware guests can be a cumbersome and frustrating experience. I'm creating this post to be able to remind myself of the various settings required to make things work. The bulk of this information is taken from VMware KB article #1006427 located at:
http://kb.vmware.com/kb/1006427.

VMware recommends using NTP and turning off VMware Tools periodic time synchronization in the guest. To turn this off, there are three options:

1. Set tools.syncTime = "False" in the configuration file (.vmx file) of the virtual machine.

OR


2. Deselect Time synchronization between the virtual machine and the host operating system in the VMware Tools toolbox GUI of the guest operating system.

OR


3. Run the vmware-guestd --cmd "vmx.set_option synctime 1 0" command in the guest operating system.

To work properly in a VM, some vendor kernels require certain kernel parameters. In our environment, we only run SLES and RHEL so I'm only concerned about those two. If you run something different, check the VMware KB article linked to above.

Architecture	Linux Version	Kernel Parameters	Notes
64bit	RHEL 5.4 and later		No additional kernel parameters.
32bit	RHEL 5.4 and later		No additional kernel parameters.
64bit	RHEL 5.3	notsc divider=10
32bit	RHEL 5.3	divider=10 clocksource=acpi_pm
64bit	SLES 10.2 and later		No additional kernel parameters.
32bit	SLES 10.2 and later		Use a VMI enabled kernel.

You now want to make sure that NTP is configured correctly for your environment.

The following is a sample /etc/ntp.conf :

server ntp1.somedomain.org

server ntp2.somedomain.org

restrict 127.0.0.1

restrict default nomodify nopeer

driftfile /var/lib/ntp/drift

Remove LUNs from Linux

I have a previous post here on how to add SAN Presentations (LUNs) to Linux. This post uses the same QLogic Dynamic Target and LUN Discovery utilities as the previous post. The steps below apply to SLES 10 and RHEL 5 servers using QLogic HBAs and PowerPath. That is what we have at my workplace and so that is what I'm most familiar with. This post also assumes that you have put the qlogic utilities into /tmp/ql_utils as described in the other post.

To remove a LUN and all associated PowerPath and Linux devices from the host environment follow these steps. These steps must be run with root privileges. (Note that it is critical to follow the procedures in exact order because deviating from the procedures can cause the host to panic.) These steps were put together using steps from QLogic KB articles.

1. Stop any I/O to the device being removed. [Unmount the filesystem or make sure that the disk has been completely removed from within Oracle ASM control.] It is critical to stop all I/O on the device that is being removed.

2. Run the following command to determine which native SCSI devices are mapped to the pseudo device:

powermt display dev=all

Find the name of the LUN to be removed and match it up with the emcpower device name. This will need to be done on each server individually.

3. Run the command:

powermt remove dev=emcpower

Where corresponds to the LUN to be removed.

4. Run the command:

powermt release

Failing to run this command results in the pseudo device still being visible in /dev and /sys/block and may lead to some complications when the new devices are dynamically added.

5. In the /tmp/ql_utils directory, there should be a script to rescan the qlogic hbas. (If this script doesn't exist, see the previous post for the steps to get the script.) The script is called ql-dynamic-tgt-lun-disc.sh. Run the script:

/tmp/ql_utils/ql-dynamic-tgt-lun-disc.sh

6. You will now remove the device from the storage array using your array admin utilities. On the Linux server, run powermt display to verify that the device has been removed.

7. Now remove the LUN from the Storage Group (CLARiiON) or unmap it from the FA ports (DMX).

Display WWN in SLES 9

I have another post regarding how to do this on SLES 10, so I decided to create a post on how to do this in SLES 9. In SLES 9 you can easily find the WWN for QLogic HBA's by running:

# cat /proc/scsi/qla2xxx/* | grep node

Setup SAN presentation on Linux

The steps below are to scan for new LUNs from a SAN after the LUNs have been presented from the storage side when using QLogic HBA's . This post uses the QLogic Dynamic Target and LUN Discovery utilities which can be downloaded from:

http://driverdownloads.qlogic.com/QLogicDriverDownloads_UI/SearchByProduct.aspx?ProductCategory=39&Product=935&Os=126

The steps below will work on SUSE Linux Enterprise Server (SLES) 10 and Red Hat Enterprise Linux (RHEL) 5.

You can discover what version of Linux you are on by running:

cat /etc/*ease

Create a directory to hold the utilites. In this examle we will use the /tmp/ql_utils directory:

mkdir /tmp/ql_utils

Change to the /tmp/ql_utils directory:

cd /tmp/ql_utils

Retrieve the utilities from the download above and extract them from the zip file into the /tmp/ql_utils directory.

This will put five files into /tmp/ql_utils:

ql-dynamic-tgt-lun-disc.sh

README.ql-dynamic-tgt-lun-disc.txt

Copying

revision.qldynamic.txt

sg3_utils-1.23.tgz

The file that we are going to be using is the ql-dynamic-tgt-lun-disc.sh file. We need to ensure that it is set to executable:

chmod a+x /tmp/ql_utils/ql-dynamic-tgt-lun-disc.sh

The ql-dynamic-tgt-lun-disc.sh script has several options available. You can see what these options are by running:

/tmp/ql_utils/ql-dynamic-tgt-lun-disc.sh -h

The commands to scan for new LUNs are listed below. After the commands we'll describe what each command does. These commands should be run with root priveledges while inside the /tmp/ql_utils directory.

powermt display dev=all | egrep "Pseudo|Logical" > before

./ql-dynamic-tgt-lun-disc.sh

Please make sure there is no active I/O before running this script
Do you want to continue: (yes/no)? yes

powermt config

powermt display dev=all | egrep "Pseudo|Logical" > after

diff after before

powermt save

The first line outputs the current list of LUNs to the file named before. This is optional, but, makes it easy to see what new LUNs have been discovered later on.

The second line actually does the scan for new LUNs and prompts you to make sure that it's ok to run the script. Answer yes to the prompt.

The fifth line (powermt config) creates the emcpower devices in /dev.

The sixth line outputs the new list of LUNs to the file named after.

The seventh line runs a command to compare the list of new LUNs to the list of old LUNs. The differences will be displayed on screen. Make sure that names of the new LUNs show up in the output.

The last line saves the configuration.

After this is done, the LUNs can be setup with LVM. (see LVM post)

Slides from SLLUG PAM training

I taught a training session at the daytime Salt Lake Linux Users Group meeting today. I wanted to make the slides available to those that attended. They probably won't make a whole lot of sense if you didn't attend the meeting.
Slides from Pam Training presentation

Publish at Scribd or explore others:

Clear Swap Space in Linux

There have been times where it has been necessary for me to clear out the swap space on a Linux system. In order to do this, you must first make sure that you have enough free memory to hold what is being used by swap.
I usually do this in a few steps. You do need to be root to do this. First I want to see what is currently being used.

free

Then I run the actual commands that empty the swap:

swapoff -a && swapon -a

Then I check what is being used after doing this.

free

I have these three steps in a script and then I just run the script, but, the commands could easily be run from the command line.