Works in Progress

It's been a few months since this thing has been updated, but over the course of those few months focuses have somewhat changed. Probably because of the crazy amount of schoolwork and other projects (like a job) that have cropped up, I've been unable to work a bit on some of the WebDev/MySQL replication things. I still plan on doing this, however, but it doesn't seem like a very high priority for now.

DeCal Submissions Site

A friend of mine wanted to learn the PHP scripting language, so I originally came up with this project as a way to get a head-start into web development. A lot of the specifications require database interaction, so essentially this project would serve as a good introduction to PHP and SQL. It ended up getting put off for a bit, and since I was busy working on lecture notes for the System Administration DeCal, I wasn't able to get a working prototype or even start work on it until just recently.

I've done something similar with PHP before and pieces are being put together slowly. This project probably would have been a good candidate for learning something new, especially RoR, but I'd like to get this mostly functional and in a package-like format before the semester starts. So far, the registration and authentication has been finished; now I need to get to the real bulk of the site: the actual homework submissions.

Things have changed a bit since working with PHP4 a few years back; PHP5 is out with object-oriented programming support. I can't say I've switched over completely, but it's something I'll give a whirl once I'm satisfied with what I have. Speaking of development tools, I tried a beta release of Eclipse PDT in hopes of having the DreamWeaver-like functionality of being able to edit in an IDE and synchronize with the server instantaneously to view the changes. The SSH2/SFTP protocol support is only available with a plug-in, and it doesn't have the instantaneous synchronization that I like with DreamWeaver. Eclipse is a nifty IDE (I've never actually used it before), but for now I'm sticking with vim. All the commands have been ingrained in muscle-memory that it's a bit difficult to not use them in other editors.

System Administration Booklet

I came up with an idea for doing this for the DeCal classes that are taught (usually) every semester. Rather than having a makeshift amount of lecture notes for each of the beginning, intermediate, and advanced sections, I thought it would be nifty if the bulk of the material was consolidated in a booklet. Of course, advanced and in-depth material wouldn't be assigned for students in the beginning section, but those who wanted to explore more detail in a particular topic could do so if he or she wished.

As I expected, work on this project is on a much larger scale. Though I'm re-using many of the lecture notes written up in last semester's class, a lot of it needs to be re-written and updated. I've enlisted the help of previous DeCal facilitators in hopes of getting progress done, but I don't see a rough copy being available anytime soon.

ipv6

One of the topics for the System Administration for Beginners course that I'm facilitating is the history of the Internet and some details about it. I discussed a bit about IP addresses and the two main versions, one that we're currently using and another one that we're moving to. I found an interesting article on Ars Technica that explains a bit more about IPv6. I encourage anyone interested to find out more to take a look.

solaris: helpful things

Here are some of things that I've found while making a transition from GNU/Linux to Solaris. There are different configurations and you can change the way things are done, so this isn't necessarily the only way to do things.

• /var/adm/messages - Unless you change your syslogd.conf, logs will generally be stored here rather than /var/log/


• If you've installed apache2 on Debian, the configuration files are stored under /etc/apache2/sites-available, etc. If you have compiled apache2, look for the configuration file in httpd.conf.


• Also note that /etc/init.d/apache2ctl does not exist. You will probably want to use $APACHE/bin/httpd -k {start|stop|restart|graceful}, where $APACHE is where you have installed apache2


• It's helpful to import the pgp key for pkg-get. Try these commands:

  # wget --output-document=pgp.key http://blastwave.org/mirrors.html
  # gpg --import pgp.key

• catman -w - Missing man pages? This command will build the indexes for the man pages.

webdev: apache install

We have reached the stage where our server box is ultimately stable. Except forthe fact that we haven't installed anything on it and the load is the ultimate 0.00. It is now time to install Apache, while hoping that doing so will not break stuff.

It is rather easy to dismiss an install to something along the lines of ./configure, make, make install. Since we are also attempting to learn about the way systems are configured, we are trying to make sure we mimic the settings that the OCF uses as best we can.

For our development server, we will be using the latest available stable version, Apache 2.2.4. There are a few files to grab using wget, since this is a fresh install of the machine. It is also important to note that many of the programs that you may be used to are not in their usual paths. For example:

$ echo $PATH
/bin

So this probably means you'll want to change the PATH environment variable to have at least /bin:/opt/sfw/bin:/usr/bin.

Here are the files we want:

httpd-2.2.4.tar.gz
httpd-2.2.4.tar.gz.asc
httpd-2.2.4.tar.gz.md5
KEYS

The files are, respectively, the gzipped tarball of the daemon we want to install, a signature file, an md5 hash file, and a list of PGP/GPG keys. The last three files are for verifying the integrity of the gzipped tarball that we downloaded, essentially to make sure that what we are downloading is what the developers wanted to release. Since their files are hosted on mirrors, it is possible that a malicious host would want to modify the code a bit to create security holes. The details in how verifying is done will not be covered here.

Next, we can check the integrity of files using gpg and md5sum:

$ gpg --import KEYS
$ gpg --verify httpd-2.2.4.tar.gz.asc httpd-2.2.4.tar.gz
$ md5sum -v -c httpd-2.2.4.tar.gz.md5

Checking both the md5 hashes and verifying the gpg signature is a bit redundant, but both options are there. Keep in mind that both the hash and signature are kept on Apache's servers.

Now we want to start the process of installing. Since there are various sorts of customizations that are needed to make in order to work with the OCF's current setup, we have streamlined this a bit using a nifty sh script that will handle all the options for us. For future reference, the build script is named BUILD_INFO and should be edited manually by hand before actually running it (in order to customize it for each package being installed). It is probably a good idea to check with a root staffer to make sure that everything will work fine with the script; specifically the one listed under Packager.

What needs to be edited is the server name you are installing this package to, and the correct path for cc. We can then test the make process and install it:

# ./BUILD_INFO make
# ./BUILD_INFO

All that is left is to use some working configuration files (or write them from scratch) to make it usable.

rpcbind relic and random rhetoric

Update (05 Mar): Someone referred me to this Google search, making the terms a bit more specific. Sun's documentation on rpcbind can be found there.

A more experienced staffer recommended that I take a look at rpcbind. This thing, apparently, will cause future headaches and problems and suggested that I understand what exactly it does before stuff breaks. A quick Google on the subject didn't reveal much; not even on Sun documentation! Apparently it seems to be an almost 12 year old relic of the past. GNU/Linux's implementation of rpcbind is portwrapper.

So what does it do? rpcbind will run on a standard port 111, waiting for requests. When another machine, say cyclone wants to connect to a service running on the current machine, famine, then a query will be sent to rpcbind in order to find out the port number. cyclone will know the name of the service it wants to use, but not the port number. Apparently, it seems like certain services (especially ones like NFS, NIS and NIS+) will use random ports hovering around the > 1024 range.

What happens next? rpcbind will respond with the port number to use and then our machine cyclone will happily connect to it. Hopefully. Since I do not think the WebDev server will make use of NIS/NIS+ services, I'm not sure if it is necessary to implement or configure rpcbind until needed.

So how can we tell if it's running on a machine? Let's take a look using rpcinfo.

$ rpcinfo -p death
program vers proto port
100000 4 tcp 111 portmapper
100000 3 tcp 111 portmapper
100000 2 tcp 111 portmapper
100000 4 udp 111 portmapper
100000 3 udp 111 portmapper
100000 2 udp 111 portmapper
...

As we can see, the portmapper is listening to requests on port 111 on both tcp and udp. Note that its listings under program always seems to be 100000. To support our diversity of systems and versions, there are multiple versions that portmapper is listening for.

Something that we may want to do if this server goes into production is to secure rpcbind properly. Either through some iptables-like program or tcpwrapper, we will want to restrict access to specific IP addresses. I am still really sure as to how rpcbind works in its entirety, so probably looking through RFC 1833 [1] would help me out. Maybe when I'm bored out of my mind (or when it breaks).

Server Additions

Next up is adding some utilities and failsafe things when we break our server. Staffers have suggested either hooking up a serial console, doing something with inetd (I somehow didn't catch what was said), or installing another sshd on a different port when I happen to break ssh and lose connection.

Compiling Apache2/MySQL 5.x

To mimic the current setup of the OCF, we have several BUILD_INFOs strewn about. We will be compiling these packages by hand with some specific settings we'd like to have. At the moment, we're not entirely sure what we're doing, so it is likely that we'll change some of the current BUILD_INFOs around to support our setup and pass it around for review before actually doing it (or run it through continuous testing).

With MySQL replication, we will be replicating the staff databases to test the setup. Another experienced (and old) staffer noted that MySQL uses query-based replication, so it shouldn't be too bad in terms of setup. Master-slave replication will most likely be done, with some load-balancing fiddled around with. If this goes into production, it will serve as a failsafe and perhaps used for load balancing (maybe 80/20 or 70/30).

Apache2 replication does not seem very likely at the moment. It would actually be wonderful to do load-balancing for our webserver and being able to move some of the traffic hitting popular sites that we host for faster response times and speeds in general. As for now, this is of the lowest priority and something I will try to work out in the near future.

Too bad it's not as simple as apt-get install apache2. But then it wouldn't be Solaris, right?

[1] http://www.ietf.org/rfc/rfc1833.txt

webdev: solaris installed

Solaris 10 has been successfully installed on one of AMD 2U rackmounts. Sporting 2x AMD Athlon MP 1800+ with 1 GB of RAM, the server we were working on caused us some trouble in the beginning trying to get the installation process to work. A CMOS reset and a PS/2 keyboard allowed us to finally get started.

We didn't realize that the Solaris installer used GRUB, and since there was a previous Solaris install, we continuously rebooted the machine in order to try and get the server to boot off the install CDs. After that, it was pretty much (relatively) smooth sailing. We didn't come across any hardware compatibility issues, which I was somewhat expecting.

Other than getting used to some of the Solaris nuances, we seem to have secured the box by locking down most of the unneeded services and ports. For example, we didn't need an X server running. I'm used to working with Debian GNU/Linux, so this would only take a few simple updates to rc.d. In Solaris:

# svcs -a
# svcadm disable cde-login

This allowed us to get a list of some of the services running and disable the graphical login manager, since we will be administrating this remotely. To get a look at some of the CPU information, I did a quick Google search and found that the kstat command works well (as opposed to cat /proc/cpuinfo).

# kstat cpu_info:::brand cpu_info:::implementation

Next up is poking around at what rpcbind does before installing some of the software required to get MySQL replication up and running.

while i'm still inspired

Similar to the OCF blog, this page will show a hazy glimpse into the inner workings of a staff member at the OCF. Hopefully this will also help serve as some sort of reminder to get stuff done.

At the moment I have two major projects I'm working on: a DeCal class that Jameson and I are teaching, and a jack-of-all-trades WebDev experimental server that Abhi and I are spearheading in order to get some of the latest web technology to the users (and abusers) of the OCF.

Web Development Testing

As some users have requested, we are currently testing the possibilities of being able to have the latest web technologies available to our users. This includes one of the more popular frameworks, Ruby on Rails. This testbed will also serve as a training tool for some staff members to set up a server with Solaris 10 and as a playground for Ruby on Rails, PHP5, and so on. In the future, as more and more web technologies/frameworks are requested, it will undergo an evaluation through this environment.

MySQL Replication

For those that remember some of the instabilities that occurred last semester, we had some of our users' MySQL tables corrupted and lost. MySQL replication will not only allow increased robustness in case the main server goes down, but provide load-balancing and easier creation and maintenance of backups.

System Administration DeCal

This DeCal hopes to teach others about UNIX System Administration by allowing for a hands-on approach to managing UNIX-based servers. Students will learn basic concepts and commands of UNIX-like operating systems (specifically Solaris and Debian GNU/Linux) and learn how to properly maintain their own server. By the end of the course, students will know how to set up a web/database server and administer a multi-user environment.

More details on the DeCal here.