Redcay Lab

A couple of the machines have cd roms that do not even eject...

anyways, I installed it a couple of days ago. Then the screen was messed up. When I checked today though the x system looked ok, so I'll just go with it.

I used the following to sync it with the lab machines
dpkg --get-selections "*" > /tmp/sels
on one of the hawkins machines

then on the redcay prototype
dpkg --set-selections < /tmp/sels
apt-get -u dselect-upgrade

it seems to be working, although there is a huge amount of stuff to download and install.

cont

machine 124 is still hosed, Kevin replaced the hard drive, so I guess it must be the motherboard

now I need to setup the kerberos/AD authentication
the files I need to change are
krb5.conf
ldap/ldap.conf
libnss-ldap.conf
pam_ldap.conf
pam.d/common-auth
pam.d/common-session

I set the skelton files up in /local/admin/support/skel so they would not need to be changed on each machine.

The packages that I need are
krb5-user
krb5-config
libpam-krb5
libkrb53
ntp

so in my script I will try to install those first, then copy over the corresponding files

differentiating after imaging

here are a list of files that need to be fixed
hostname
hosts
network/interfaces

are the files to be changed
script seems to work ok, but the machines are being slow, with very long latencies at time
I hope it is because nfs was confused by so many of the same ips

kerberos changeover

Worked on kerberos for server and unix.

It took forever to backup the home directories, the problem was that it was large and I started by doing a copy over nfs. That ended up being too slow, so eventually I switched to netcat and bunzip2 (although, it was a bad idea to use bunzip2 for netcat for the robot lab image because the rescue system only has gunzip available). Use netcat was about 3-4 times faster. It still took all night.

This morning I pruned the old directories of users who can not log in and I reset the UIDs to correspond to the new ones given by kerberos. That went ok. Then I put kerberos on unix, that was a pain in the but. The main problem was there was a clock drift, and I could not figure out why it was giving me problems for the life of me. It is better now, but it was a real pain to figure that out.

fall 07 checklist

still to do:
new machines/old machines in redcay lab
first day of class/lab
prepare my classes
finish imaging bad machines in hawkins lab - done 8/25
finish configuring imaged machines in hawkins and robot labs - hawkins done 8/25
find a way to image two remaining machines in robot lab
ntp all machines - done 8/25
setup kerberos on all machines - done 8/25
make sure skel files are in place - done 8/25
include robot lab links in skel
reconfigure www to use kerberos
java/emacs/etc reconfiguration

wordpress

upgraded to 2.2, went very smooth, just untarred the new files and then ran the upgrade script, nice.

netcat

I forgot to log netcat last year ...

here is what I've actually ended up using
nc -l -p 3333 | gunzip | dd of=/dev/sda bs=4k
cat img | nc -q 1 addr 3333

anyways, it is fairly simple just a

netcat -p 2222 -l |bzip2 -d | dd of=/dev/sdb
on the receiving side, and on the sending side

bzip2 -c /dev/sda | netcat hostA 2222

(alternative syntax
nc -l 2222 | bzip2 -d > /dev/sdb
bzip2 -c /dev/sda | nc 192.168.1.1 2222
netcat -p 2222 -l |bzip2 -d | dd of=/dev/sdb bs=16M


dd if=/dev/hda bs=16065b | netcat targethost-IP 1234
netcat -l -p 1234 | dd of=/dev/hdc bs=16065b

)

of course I usually create the image file and save it
what I will probably do this time is bring csc127 down to single user mode
use it as a template, and then send its image to server to save it.
Then use server to distribute the image to the machines that need to be reimaged.
again, I was fairly disappointed that I seem to need to reimage so many, but it seems
to be necessary.

I have some simple scripts to log the packages installed to a database and track differences.
It surprises me too, that there is such a large discrepancy on the machines since they all
started identical. It is unclear to me what the source of the nondeterminism is....


waxing delays

I've been stymied the past couple of days by the waxers in preventing physical access to the 053b lab, hopefully I'll be able to get in soon to finish up

It also looks like I'm going to take over the robot lab, it should go ok

mail problems

well, the whole vlan was stuck without mail access in or out, apparently the culprit was some excessive messages by cslab-104, over a long period of time.

annoying. Called Symen to get it cleared up.

I discovered what the problem was,
it was tripwire was trying to send a mail report, but the address had been mangled, and then when I removed the package, it never took the cron job out, so it kept complaining every day.

lab upgrades

just spent several hours baby sitting the upgrades of the lab,
it didn't go as smoothly as I had hoped/expected. nscd gave me problems in terms of it
not configuring correctly and confusing dpkg, then there are several machines that are just out of it, and it looks like I will have to either reimage or reinstall, they are 105, 109, 113, 118, 119, and 121

blech, that is in addition to 106 and 124, which are currently out of commission.

active directory

Active directory/kerberos set up seems to be going fairly well
Plus I'm setting it up to auto create home directories, so hopefully the new classes will go
smoothly. I need to set up the skel directory for the new directories.

skel is set up in /local/admin/support/skel, there will be an extra .svn subdirectory, but I can filter that out later.

Now I need to make a script to sync the uid/gid with the new ones in active directory

I also need one to move people to an archived folder, I think I will do that one first.

I think the scripts are in place, contacted media services about including a couple of instructors in the kerberos group. After they are put in, then I will run the scripts.

Also, I spent about a half hour trying to figure out why something was not printing out any more. It turns out I had it as debugging info, and I was not passing the debugging flag in.

Upgrades, semester prep

Upgrading to postgresql 8.2
- did a pg_dumpall to save the old data
- it did not create a new 8.2 etc config, so I copied the 8.1
- now it is complaining about not having a locale to start under
- created a new database with initdb
- it still does not run from /etc/init.d
- it seems to run from /usr/lib/postgresql/8.2/bin/postgres -D /var/lib/postgresql/8.2
- strange
- data seems moved over, (pg_restore), but unclear if it will boot on its own now

Apache
- php is giving me fits
- it seems it was not enabled
- removed a couple of AllowOverride None, since it was complaining that they
were useless, I thought they were nice for documentation though

Subversion
- with the move to the new authentication, need to update this
- some progress, but I am having trouble with the authz parts of things

I spent hours on this, finally it seems that I was missing a line
AuthBasicProvider ldap
in my config. blech, it had been working before, and after the upgrade it stopped.
those are the sorts of things that are really frustrating

Active Directory/LDAP

Here are some useful notes,
to get ldapsearch to work use

ldapsearch -v -w password -x -D username

in the ldap.conf file the nss_passwd field needed to be changed

Ubuntu Feisty upgrade

I've been working on upgrading the machines to feisty, with mixed results.
unix - the easiest of the bunch, no problems
www - was running fedora, and then forgot how to talk to anyone outside the subnet,
I suspect that somehow the networking config got messed up, which is weird
because I haven't touched it at all. I wanted to reinstall anyways, so it is
better now
server - not so good. I was going to save this one for last, but it started crashing
with kernel panics. I'm hoping there isn't a hardware problem. Anyways
I got half-way through the upgrade, and it started bogging down and having
lots of problems with responsiveness. I had to reboot, and it did not come
back up. After struggling with a rescue disc and trying to get it back up
I eventually succeeded. I'm not sure if it was part of the problem, but
mdadm and evms might have been part of the problem since I could not
boot to the kernels that they had messed with. (That bugs me, they should
not be messing with the kernel images without first asking).
hawkins 053b - somewhat messy. I've had to log into them individually to work on
the update, and they have had some problems. My next step is to
write some software to log their current status to a central db and
then try to do something to get them all synchronized.
I'm also working on getting them authenticated against the kerberos
system. I met with Symen and Shawn last week, and now need to get
the query part working.
office - went ok, nothing too exciting