Thursday, November 17, 2011

Oracle Solaris 11 Summit Day at USENIX LISA 2011

LISA '11 is just around the corner and once again includes an Oracle Solaris Summit the day before the main conference.  Please come to the summit and to as my esteemed colleagues and I introduce many of the great improvements found in Solaris 11.  I'll be giving a talk on Zones.

Even with a full day to talk about Solaris 11, we'll certainly be unable to get into the depth in the areas that concern you the most.  To get some face time with Oracle engineers, stop by the Oracle demo booth - I'll be there Wednesday from 2:00 - 4:00.

If I have any updates, I'll be posting them to The Zones Zone.

Wednesday, November 09, 2011

The Zones Zone

New posts related to Solaris can be found at The Zones Zone

Monday, January 03, 2011

dtrace PID provider vs. libraries loaded by python

As I was doing some work on beadm, I really wanted to know who was mucking with properties on datasets. It looked as though I could do something like:

# dtrace -qn 'pid$target:libzfs.so.1:zfs_prop_set:entry / copyinstr(arg0 + 8) == $1 /
{
    printf("%s(%s,%s,%s)\n", probefunc, copyinstr(arg0 + 8), copyinstr(arg1), copyinstr(arg2));
    ustack();
}' -c 'beadm create test' $dataset
But that doesn't work:
... probe description pid12942:libzfs.so.1:zfs_prop_set:entry does not match any probes
My guess is because that this is because the python executable is not linked against libzfs - a very good thing. To get around this, we need to preload libzfs. For example:
# LD_PRELOAD_32=libzfs.so.1 dtrace -qn 'pid$target:libzfs.so.1:zfs_prop_set:entry / copyinstr(arg0 + 8) == $1 /
{
    printf("%s(%s,%s,%s)\n", probefunc, copyinstr(arg0 + 8), copyinstr(arg1), copyinstr(arg2));
    ustack();
}' -c 'beadm create test' rpool/zones/z1/rpool/ROOT/zbe-1
zfs_prop_set(rpool/zones/z1/rpool/ROOT/zbe-1,mountpoint,legacy)

              libzfs.so.1`zfs_prop_set
              libbe.so.1`be_unmount_zone_root+0xa9
              libbe.so.1`be_update_zone_vfstab+0x106
              libbe.so.1`be_copy_zones+0x9d9
              libbe.so.1`be_copy+0x8f2
              libbe_py.so`beCopy+0x245
...
The only change here was the addition of LD_PRELOAD_32=libzfs.so.1 at the beginning of the command. LD_PRELOAD would have worked as well, but it wouldn't work if a different library that does not have a 64-bit variant was being traced. That is because dtrace would fail to start because the 64-bit library was not able to be found. I found this when I needed to poke around at calls within libbe.so.1, which is 32-bit only.

Sunday, January 02, 2011

ksh93 backtraces

It is often times handy to get a backtrace to help you understand how a program got to an error condition. Unfortunately, shell scripting languages tend to not provide an easy mechanism to get a backtrace. The following examines how it can be accomplished with ksh93.

#! /bin/ksh

function backtrace {
        typeset -a stack 
        # Use "set -u" and an undefined variable access in a subshell
        # to figure out how we got here.  Each token of the result is
        # stored as an element in an indexed array named "stack".
        set -A stack $(exec 2>&1; set -u; unset __unset__; echo $__unset__)

        # Trim the last entries in stack array until we find the one that
        # matches the name of this function.
        typeset i=0
        for (( i = ${#stack[@]} - 1; i >= 0; i-- )); do
                [[ "${stack[i]}" == "${.sh.fun}:" ]] && break
        done

        # Print the name of the function that called this one, stripping off
        # the [lineno] and appending any arguments provided to this function.
        print -u2 "${stack[i-1]/\[[0-9]*\]} $*"
        # Print the backtrace.
        for (( i--; i >= 0; i-- )); do
                print -u2 "\t${stack[i]%:}"
        done
}

# A couple functions to illustrate the output
function a {
        b "$@"
}

function b {
        c "$@"
}

function c {
        # Trigger a backtrace and exit if no arguments were passed
        (( $# == 0 )) && backtrace "missing arg" && exit 1
        print -- "$@"
}

a "$@"
A couple example runs:
$ ./backtrace.ksh hello world!
hello world!
$ ./backtrace.ksh
c: missing arg
        c[37]
        b[32]
        a[28]
        ./backtrace.ksh[41]

Saturday, April 03, 2010

Connecting to twist server

Once your development environment is set up, it is useful to know what is required to connect to the twist server.

#! /opt/opsware/smopython2/python2

import sys
sys.path.append("/opt/opsware/smopylibs2")
from pytwist import *

# Establish an unauthenticated connection to twist server.  Note that the
# hostname "twist" must resolve either through /etc/hosts or DNS.
ts = twistserver.TwistServer()

In the event the hostname twist does not resolve or points to the wrong twist server, you can do...

ts = twistserver.TwistServer("twist.mycompany.com")

See the TwistServer Method Syntax section on page 75 of the Developer's Guide for other options that may be important to you.

It is also quite likely that you will need to do an authenticated session. This is covered in the Error Handling section on page 76 of the Developer's Guide. In order to prompt for a username and password, I have found something like the following works well...

def authenticate(server):
        for tries in range(1,3):
                try:
                        sys.stdout.write("HPSA login: ")
                        user = sys.stdin.readline().strip()
                        pw = getpass.getpass("HPSA password: ")
                        server.authenticate(user, pw)
                        return True
                except:
                        sys.stderr.write("Authentication failed.\n")
        return False

...
ts = twistserver.TwistServer()
authenticate(ts)

Opsware/HPSA API: Getting Started

The HP Server Automation Platform Developer's Guide (for HPSA 7.50, September, 2008) and any other resources I've been able to find are far from complete and sometimes misleading with respect to using the HPSA API. As I've tried to clear up my confusion with the API, google has helped me exactly 0% of the time. Maybe my rambling on the subject will lead to someone leading me to a better approach...

In this first post on the subject, let's start out with enough to get a working Python environment. Chapter 3 of the developer's guide indicates that Pytwist relies on Python 1.5.2. Ugh. However, pytwist can be used with Python 2.4. Rather than following the advice to install the /Opsware/Tools/Opsware API Access software policy, instead install the following policies:

  • /Opsware/Tools/Python 2/Python 2 for Server Modules
  • /Opsware/Tools/Python 2 Opsware API Access for Server Modules/Python 2 Opsware API Access for Server Modules

The permissions are likely such that you need to be root to execute. Fix this with:

# chmod -R a+r /opt/opsware/smopython2
# exit
$ /opt/opsware/smopython2/python -V
Python 2.4.4

Ahhh, much better.

Notice the path to python above. It is actually a shell script that sets PYTHON_HOME then tries to execute python. Unfortunately, Solaris doesn't like a shebang line in one script referring to another script. The resulting error looks like:

$ ./twister1 
import: Unable to connect to X server ().
./twister1: line 4: syntax error near unexpected token `"/opt/opsware/smopylibs2"'
./twister1: line 4: `sys.path.append("/opt/opsware/smopylibs2")'

In other words, the shebang (#!) line python scripts will be useless as the software policy installs it.

Workaround 1:

$ export PYTHONHOME=/opt/opsware/smopython2/34c.0.0.5.31-1

Then use a shebang line like:

#! /opt/opsware/smopython2/34c.0.0.5.31-1/bin/python2

Workaround 2: Compile this program and put it at /opt/opsware/smoptyhon2/python2

#include <stdlib.h>
#include <unistd.h>

#define PYTHONHOME "/opt/opsware/smopython2/34c.0.0.5.31-1"

int main(int argc, char **argv) {
        char *python = PYTHONHOME "/bin/python2";

        argv[0] = python;
        setenv("PYTHONHOME", PYTHONHOME, 1);

        return execv(python, argv);
}

Then use a shebang line like:

#! /opt/opsware/smopython2/python2

In any case (even if you used Python 1.5 like the developer's guide suggests) the path to python does not match reality. Adjust your scripts accordingly.

Friday, June 19, 2009

Shell Programming and PATH

As most readers of this blog will already know, the PATH environment variable is used to locate commands that are executed. Key things to remember as you read this post are:

  • Environment variables (including PATH) are inherited by child processes
  • Child processes are unaffected by the parent process subsequently changing PATH to something else

So what's the big deal? Suppose you have a shell script that calls ps -fe. It works great for you because you have /usr/bin first in your PATH. However, the guy down the hall that cut his teeth on a BSD system has /usr/ucb first. If your shell script does not set PATH=/usr/bin:... prior to calling ps, your shell script will work for you but give strange errors for the guy down the hall. Of course, your shell script could just specify /usr/bin/ps -fe...

This brings up four different styles that are seen...

Style 1: Just hope for the best

#! /usr/bin/ksh

count=$(ps -fe | wc -l)
echo "There are $count processes running"

Style 2: Specify full path whenever calling a program

#! /usr/bin/ksh

count=$(/usr/bin/ps -fe | wc -l)
/usr/bin/echo "There are $count processes running"

Style 3: Create variables to store full path to all programs

#! /usr/bin/ksh

PS=/usr/bin/ps
WC=/usr/bin/wc
ECHO=/usr/bin/echo

count=$($PS -fe | $WC -l)
$ECHO "There are $count processes running"

Style 4: Set PATH to use the commands you want to use

#! /usr/bin/ksh

export PATH=/usr/bin
count=$(ps -fe | wc -l)
echo "There are $count processes running"

With Style 1, the script is only reliable for the subset of users that have the right version of ps first in their PATH.

A workaround for this is shown in Style 2. However, this example has an intentional problem that is somewhat common when this approach is used. Notice that wc is not specified by its full path. This will work fine until someone with a really messed up (or unset) PATH tries to execute the script.

Style 3 fixes the ps and wc problems, but introduces another small problem: it forces a fork() and exec*() to run something that could be more efficiently done via a built-in. I'll talk more about this in a future post.

Style 4 keeps the simplicity of Style 1, but ensures that each user will get the same version of the commands. The author of the script can tailor PATH to contain the minimum set to find the required commands and test the script to gain a high degree of confidence that the script will work for others.

I have a strong preference for Style 4. Performing shell programming retains the feel of using a shell interactively, keeps the code understandable, and performs reliably. But this doesn't mean that it is always the right thing to do. Consider the batch command. It doesn't set PATH and it is very correct in not doing so. That is, if the following:

exec /usr/bin/at -qb $*
were replaced with
PATH=/usr/bin; export PATH
...
at -qb $*
This would change the environment that at(1) attaches to the job - potentially breaking it.