I Developed this commandline tool to make it easier to maintain stuff on a groing amount of servers. nexec is a tool that can help you gather ad-hoc info when it’s needed quickly. nexec is written in python, it depends on codespeak’s pylib, and does all it’s communication over SSH. The only thing you need on ‘the other side’ is a python interpreter. (that’s right, no further libs or modules). Oh, and you need your SSH key pairs in place, otherwise it’s ‘password galore’.

Of course, the bash-ers out there would say that the same functionality can be achieved with a for loop. True! But that’s exactly what nexec does for you so you can concentrate on what you’re doing remote. And ofcourse you can have nightly cron-ed scripts in place that do this every day. But nexec aids in all those things that are not ‘in place’.

Examples

Before getting into any details, first some tricks of the ‘nexec’ trade to get you warmed up …

Are my server systems running the proper time:

$ ./nexec.py -n 'server1 server2  server3' -c 'date'
server2: Fri May 23 16:21:18 CEST 2008
server1: Fri May 23 16:21:17 CEST 2008
server3: Fri May 23 16:21:19 CEST 2008

See if all your servers are running ssh protocol 2 only:

$ ./nexec.py -n 'server1 server2  server3' -c 'cat /etc/ssh/sshd_config | grep -v ^# | grep Protocol' | sort
server1: Protocol 2
server2: Protocol 2
server3: Protocol 2

(You could also do a “-n ALL”, then nexec reads your server list from a config file. More on that in a minute)

Get a list with all different kernel versions that are running on your systems:

$ ./nexec.py -n ALL -q -c 'uname -r' | sort -u
2.6.18-5-686
2.6.18.8-xen

(The “-q” parameter suppresses the ‘per line’ hostname prefix).

See if ‘john’ is a member of the sudo group on servers 1 and 2:

./nexec.py -v -q -n 'server1 server2' -c 'cat /etc/group | grep sudo | grep -v john'
Host:  server1.example.com
sudo:x:27:jane,jerry
Host:  server2.example.com
sudo:x:27:jane

(The”-v” adds the “Host: ….” header, per server output. Note that “-v” and “-q” are interchangeable)

As you can see, by combining local and remote greps and using the -v and/or -q parameters there are some interesting ways to obtain live data from your server.

Lets look at the configuration.

Configuration

Run without parms, nexec shows help output. Furthermore the only two files nexec depends on (and not even necessarily) are a “~/.nexec.conf” file and a possible “~/.ssh/config”.

The “~/.nexec.conf” does not contain much, just “key: value” pairs under a hosts section. The actual ‘value’ is only obtained when using the “-v” option to display the header above each server output “Host: server1.example.com“.

$ cat .nexec.conf
[hosts]
server1:   server1.example.com
server2:   server2.example.com
#server3:   server3.example.com

Other then that, nexec simply expands “-c ALL” to all available keys under the “[hosts]” section, skipping the usual hashsign ‘#’ being a comment. In this case the “ALL” parameter given to “-c” would be expanded to “server1 server2″ in the previous example. After that the full name is taken from the “~.ssh/config” file.

$ cat .ssh/config
# server1
Host server1
Hostname server1.example.com
User john
Port 4321
# server2
Host server2
Hostname server2.example.com
User john
Port 4321

I deliberately left all that in there so other usernames and port numbers can be specified. Besides, that’s all access related and shouldn’t be anywhere else anyway.

Then one more note on the “-q” and “-v” option. What they basically do is mark (or suppress mark’s) on the output that get’s returned. So you can mark output per server adding the “-v”, and output get’s marked per line (default) if not suppressed with “-q”.

Download

The script can be downloaded here: nexec-10.tgz

Set the ‘execute’ bit with chmod and put it in your $PATH somewhere. And for the “ALL” parameter, set up the config file as the example shows in the Configuration section.

Gotcha’s

Escape your backticks with with a backslash if you want remote expansion.

This runs the uname part locally and then sends the command to the other side.

$ nexec.py -n ALL -c "date | mailx -s `uname -n` root"

This also executes the uname command on the other side.

$ nexec.py -n ALL -c "date | mailx -s \`uname -n\` root"

Todo

There’s two things still on my list.

• Catch a non-existing hostname when taken from the “.ssh/config” file. The script breaks if a host is not resolvable. (Being a sysadmin you should know what your doing ..
• Make the “Host: server1.example.com” header not show when no output is returned from a host. That is when using the “-c ALL”.

New tricks might be added in the future, so if there’s stuff that you think should be added drop me a reply.

That’s all folks …