Monday, December 11, 2006

Current Project Status

A few people asked for information about where we are currently with the computer lab, network etc. Basically the technical aspects of the project. Here you are:

When we first got our shipments from the port, we went over the inventory, and were quite pleased. Many Dell Optiplex GX110s, some newer lower quality off brand Pentium III systems, one Pentium 4 tower, and some fairly useless Pentium Pro/Pentium towers.

When we got to Kumasi, we found that the Dell systems had been imaged by the organization that donated them to have Windows 98, OpenOffice, and some other software installed, which was nice since they just worked out of the box. Others had no hard drive, wiped hard drives, or old personal installs (complete with personal information).

At first we copied the Win98 image from machine to machine, and I was trying to install Edubuntu on others. This was just to get the machines working so class could resume. The sad truth about Edubuntu (a desktop Linux-based OS, focused on schools) is that it is basically unusable on machines with less than 512 mb of ram. This is very disappointing.

Sorry if this upsets people, but a School-focused linux distribution should not require top of the line equipment. Boston Public Schools are barely better off with regards to processing power, so I find the Edubuntu desktop, basically, a failure for developing nations.

Harsh words, but frankly Gnome 2 is slow, there are no decent network/user management tools unless you are deploying the LTSP (which is awesome, however), and yes I am going to say it... it should LOOK like windows. Sorry, I don't care for idealistic GUI politics, I run Mac OS X, OpenBSD, and Windows2k/XP as desktops (when required), and don't care if it is windows or not (give my a CLI, a text editor, and firefox), but PARENTS/EDUCATORS DO. Basically, unless you are deploying terminals, or using very fast hardware, Edubuntu should be avoided. Though I do like Xubuntu, a lightweight non-education focused distro by the same people.

I've come to the realization that I will have to heavily customize an LTSP-based linux distro (or make my own) to run on this hardware and our network infrastructure. Which is not good when we are running decent hardware, probably some of the best in Ghana, and a fast network. I still plan on doing it, though. And I will actually spend a LOT of time building it, but It just isn't going to happen for another 6 months.

Due to virus/network/security related (read: student) issues, we had to jump away from windows 98 as soon as possible. I found the nLiteOS project again and started working (read: chucking about a dozen burned CDs on the floor) on a new custom windows XP build. I got one working after about 2 days of testing (and numerous power outages) cd images and installs.

The first build was designed to work on our Ubuntu/Samba-based Primary Domain Controller server. I quickly took the server down after the near-constant power outages were making me worry about losing our 300GB hard drive. The second build included SP 2, and some further network/workgroup customizations.

The outages have been every few days, but the days they happen, it is either on/off every 20 minutes, or off for several hours. This coupled with Africa Online's (AOL) service, has made my job exceedingly difficult.

Due to network/equipment/power issues, I decided to hold off on deploying parts of the network that will make it easy to manage and started my focuses on routers/dhcp server/dns/etc issues. I really want to roll out the Domain-based/group policy/terminals/RandomAdvancedNetworkIdea setup seamlessly and quietly as possible. I will probably deploy the enterprise-level aspects of windows domain networking within the next 6 months.

There were however certain more advanced aspects of the network that deploying immediately have already shown improvements in the network. Though I still think I am a month behind schedule due to power/internet issues (sadly I did not plan for outages from both so frequently).

When I first got here, replacing the old router with my airport extreme base station made significant improvements in network performance, latency, lookup times, etc. Switching to OpenBSD as a router/DHCP Server/Firewall/Simple Web&FTP server had added some minor speed gains under heavier load (on a 900mhz p3 with 256mb of ram). I hope to separate these servers for security/bandwidth shaping reasons using Xen.

It was a little tricky to initially configure, the syntax changed on a few things from the older versions (some FAQs/Guides are almost worthless now), also I had to locally mirror most of the packages before I could really attempt configuring it fully. But after I got it working as a simple NAT/Firewall/DHCP Server life became easier for everything. Also this let me bring my Wireless Base Station into my room, which is always nice.

OpenBSD 4.0 is what most people actually want when they look at Linux distros for network server setups. They have an amazing security record, modifying many opensource packages to be more secure (chrooted environments, etc) and really make package management a breeze. you set the pkg_path (location of applications/packages, ftp, local, http, network-based) and then use pkg_add packagename or pkg_delete. Very elegant, especially compared to the dependency hell that is Linux-bsed distributions.

Anyway, one cool thing we did with the OpenBSD server was that we added a DNS caching server. What a DNS server does is when your computer requests http://www.google.com it asks your dns server (in our case an OpenBSD 4.0 box) to do a "lookup" in global DNS servers for the ip address, and then connects you to the ip address through your gateway (home routers/cable modems/wifi base stations, etc).

What a caching DNS server does is it installs a BIND server (DNS server), and when BIND looks up a domain name (www.domain.tld) it saves the ip address to the server, so it doesn't have to do a lookup again (which on our connection takes a second or three). This saves a significant amount of time on networks with high latency about 2 seconds per new domain, which can be x.domain.tld, xxxxxxxx.tld, or xxx.x.xxx.x.tld. It is much akin to using a phone book, as it looks up the name, and returns the phone number.

This is not something most people with low latency/fast connections would need, but with our 1-3 second each way latency (worst during the day), it saves a lot of time. And in the future with Satellite this will get worse. We will actually have speed of light delays.

My plans for December/Early January:

Setup a transparent Squid Proxy Cache. What this will do is it will locally cache websites we go to on a server, and only download new information if the website has changed. Which on slow/high latency connections can be a tremendous time saver.

Also, I want to setup ntop to gather network utilization statistics to help with fundraising/grant writing.

Then I will take a look at the WPKG and Unattended projects for creating a simple to maintain/administrate windows network.

Start working on designing new install builds in a virtual environment alla VMWare, and doing significant testing/configuration before installing onto a "production machine." Oh no, there it was, my first use of a very specific network deployment term.

And finally I wish to start working with Xen to consolidate low cpu utilization servers into a virtualized environment that will allow multiple operating systems to run at once on a server.

Right now we are basically testing various ways of doing things. We started with trying to do imaging, which I actually found to be not as good as unattended clean installs. We continued on with trying a domain controller from the start, but changed to a workgroup environment until we have a more stable electrical infrastructure. I tried using Edubuntu as a desktop for some of the machines, but due to the piggishness of gnome2 with regards to memory/cpu utilization, I will be holding off on Linux desktops until I have time to develop a customized (read: fast and responsive) LTSP-based linux distro.

All in all, I am learning a lot about setting up enterprise level networks, figuring out what works and what doesn't, and trying to integrate as much free software was we can.

Apologies...

I have recently taken the time to read the postings and e-mails I have written. I know exactly how horribly written many sentences are or how the paragraphs aren't fluid. I hate it, but that is the cost of writing when either deliriously tired, or being delirious from Malaria. I will spend some time cleaning them up. Though I do feel that some sort of update is more important than a perfectly written update.

-Evan

Sunday, December 10, 2006

The Project

I figured I should outline the general goals of my project as clearly and concisely as possible.

This year is about putting together an advanced, easily maintainable, secure, expandable computer network, begin writing a new Desktop Computing textbook for the school, setting up an internet cafe to subsidize the cost of the internet connection ($600 USD a month) and computer maintenance.

I will be Offering a class in fundamentals of desktop computing the internet, and networking concepts for the faculty after school hours, and also allowing teachers to use the computers to take online classes offered through universities.

And finally laying groundwork for future technology advances including but not restricted to solar power, DC-powered computer lab, amongst other things.

There are many possibilities in this project, and I really have to limit myself to not spending time on ideas that I cannot fully see into fruition. The most important requirement is "Easily Maintainable." I am not going to waste my time here by creating something that will fall apart as soon as Yaw, Toon, and I leave.

Right now we have a manageable build system that isn't too difficult to setup. We use a customized Windows XP install CD I made using tools from the nLiteOS project. It allowed me to create a bootable windows XP CD which includes almost all modern drivers for ethernet, audio, chipset, cpu, and video cards using driverpacks. It also let me disable many unneeded windows services, both useless, or resource intensive (Themes, balloon tips, fancy effects).

One of the best things about using nLite is that it lets me setup networking, user accounts, and workgroup/domain settings before I burn the CD. This lets us do unattended installs, we were able to do full installs on 20 machines in about 4 hours by simply putting one of the 4 CDs we burnt into the computers and selecting the drive to format and install onto. We come back about an hour later (depending on CD-ROM speed) and put the CD in a different computer.

This let us install software off our network shares onto the computers while we waited for other computers to have windows installed. This part actually takes time and requires user input. I am looking into ways to make both of these aspects of deployment easier. The unattended and WPKG projects are free network based "push" and "pull" methods for both windows installs and software deployments.

I plan on setting up a Deployment Server that will let one "Network Boot" (Booting off the network card, as opposed to the CD-ROM or Floppy) their machine on a small, separate computer network. This will let us do unattended windows XP installs without even using a CD, CD-ROM, or Floppy Disk, and is also a little cleaner than "Imaging" (Creating a Windows XP install and copying it to all other computers).

It is cleaner in a sense that I can add service packs, security updates, etc to the network-stored windows install files, and have them automatically put on any new computer systems. Many of you are familiar with the idea of reinstalling windows due to one problem or another. Now imagine not having to download hundreds of megabytes of updates and security fixes after each install.

WPKG is another beast entirely. It simplifies application installation on networks. Allowing an Administrator to select which computer builds receive what applications, and automates the whole process if need be. I can test software updates on one computer and then push the software down to all the computers on the network and have them automatically, silently, and without user input, install.

If I include the "Client" of wpkg on the windows deployment, I can have it automatically grab all the software upon first boot, install it, and reboot. Basically taking 2-3 hours (reduced from what a normal user would have to go through already) of Administrator input and futzing with a computer and having them automate all but about 5 minutes of it.

Friday, December 08, 2006

Quick Update

I've spent the last few days contemplating letters, post cards, skypeouts, and other means of communication with those of you stuck in that bitter cold wasteland that is The States (or those of you pour souls simply trapped on the left coast).

Anti-Malarial drugs are kinda neat, Malaria itself not so bad. Haven't had much of the bad symptoms, and except for neck and joint paint and some mild dizziness I really can't complain. I get dizzy from standing up too quickly, and I immediately sit down, then my neck hurts later on. Not entirely dissimilar from a good roller coaster. It has however made me rest a bit and read, which is never a bad thing.

Cicero's On the Good Life, what Roman and Greek histories I can find, some bits of The Bible, and random biology and computing articles online. I'm happy to be reading again it is not something I have done of my own volition in many years. Though a few writer friends of mine had been pushing me as long as I knew them.

I've finally gotten our internet connection to something manageable, so I have been able to upload pictures. Although I was in a rush just to see if it would work and haven't labeled or organized them yet.

Here you go: http://picasaweb.google.com/evandt/Ghana

There will be more to come, hopefully categorized, captioned, cropped, and rotated. There aren't many for the last few weeks because I haven't found any good batteries I want the ones that cost more than 12 cents for my camera, the others last about 2 minutes in such a device.

You can see my room in Accra when I stayed with John and Nana, the pool, the vendor lined streets, and the state of suburban sprawl in Legon. Then there are a few pictures from Ghana University, Legon with Toon, Yaw, the driver and John, followed by some of Yaw and I in Tema, the port city, getting our equipment. There there is the school and it's cultural event day planned partially for some American Tourists who came to visit, along with Toon and the children in the schoolyard.

Toon is at ICYE (www.icye.org) volunteer. He is about 24 has a teaching degree, and was a European History major from Belgium. Pretty easy for me to get along with based on these few things alone. My only complaints about him are that he is entirely too helpful, nice, polite, well-mannered, and doesn't drink beer.

He came with a group of mostly European volunteers, all of whom, excepting him, are female and under 21. They all take trips on weekends to various towns or each other's different projects. Sometimes I join them, sometimes they come here, sometimes their parents come to visit them. All in all it is nice to have some westerners close to my own age to talk to.

The differences between us are more pronounced than similarities to me. I do not consider myself a "volunteer" like they do. Most volunteers pay about 5000 dollars to come and volunteer for 6 months to a year. They go through an organization and are placed at host sites, usually with host families.

I came because I was asked, I have family nearly as close as my own brother here. I am not much one for being a tourist in this great country. When I see how Ghana runs things I think of how I wish the west were more like that, rather than how "strange" it may seem. I've really experienced very little in Ghana that I haven't experienced in Boston. Plantains, peanut soup, hip hop, bureaucracy, bribes, mild corruption, dangerous driving and traffic. The only real differences I see are a lack of infrastructure, and less cosmopolitan ways of thinking. But no idea I have seen or heard here is at all new to me, and I have never felt more "free" in my life.

Unlike my European compatriots I am staying for over a year, and if all goes well I've committed to returning once a year (or as needed) for the next 5 years. And because of my own political and philosophical beliefs I simply could not come to this country and do this type of work without committing fully to such a project who's main requirement is that we put together a maintainable, easily managed, remotely manageable network and an upgradeable infrastructure.

I need some semblance of a lasting permanence, otherwise I would be very disappointed to have spent a year here for everything to fall apart when I leave. Basically, if the school is willing and we have the ability, I am not leaving until I feel I am truly "done."

Life is good here, though I think I could use a change of scenery, haven't left the school or it's immediate surrounding areas in some time. The dry season hasn't been as harsh as I had been warned, actually nothing has been as bad as I had been warned, but it has been more humid than expected. I really can't complain, I suppose, though I miss the torrential downpours there were truly spectacular and quite refreshing.

I hope to hear more of the goings on in Boston, I would rather get second hand information about local news than actually read that rag The Boston Globe, but should I have to resort to such desperate measures I may. Oh I take back any nice things I said about Deval Patrick as he put a Microsoft Shill on the MA Tech Advisory board, the only person on that board who is not involved in any IT departments for MA-based companies... He works for Microsoft.

Anyway, this update is getting far too technical, and I promised myself to separate technical posts from personal ones. I will continue on the more technical aspects of this project tomorrow and it will not be mass e-mailed out like this one is, but available at the same website as the personal updates: http://evandtaylor.blogspot.com

Happy Holidays to those of you not enjoying the 78 degree weather, and having to slave over cooking family meals rather than being served three fairly good meals a day.

Sincerely,

Evan Taylor

Wednesday, November 08, 2006

Ok, so it takes me a little longer to get anything done...

Hello All,

Sorry for the delay, it is only mildly inexcusable that I have taken two whole months to send some sort of update, especially to those who are probably a bit more interested, if not concerned, or just plain worrying about my wellbeing. I would like to say it was the power outages (nearly daily for less than an hour where I am now, but on a 3 day am-pm and then pm-am rotation), internet outages, mild (read: massive) frustration with various online tools I was trying to originally do this with (googlepages is horrible), or my wanting to spend what precious time I had of all these things working getting the initially useless computer lab back up and running so the PTA here likes me. Finally, however I can send this out, because the lab is completely up and running as of earlier today, and I have had ample time to write as I have finally succumbed to a normal and mild traveler's stomach plight.

Ghana is... Ghana is gorgeous. The weather is humid less than the oppressive Boston summers, and a might cooler. The Sun is only abusive for the first two or three sunburns, but after that it is enjoyable, though many Ghanaians tell me otherwise. The colors are different, though that may simply be the Mefloquine I am taking (causes hallucinations, says so on the packaging). The dirt is reddish-brown, and the sky has more vibrant blues than anything I've seen before, probably due to less pollution, again the Mefloquine, and possibly just the surrounding colors being equally vibrant and bright.

The food is a different color and flavor altogether. Oranges are yellow or green, Yams are white, Pineapples are red, there is this strange Sucre ingredient (apparently made from Canes or something) where my classic ingredient High Fructose Corn Syrup is supposed to be, and the tomotatoes... Well the tomatoes have flavor, flavor unlike anything I ever tasted before, you simple could not describe them as "watery," they are very robust.

Pretty standard fair as far as food goes, peanut soup, various stews (usually tomato based) with okra among other things not commonly found in American 'Cuisine' (Apologies Helen, Tanya, Frank, Harvey etc, I would never lump your cooking in with common American Cuisine), pounded corn made into dough like balls for tearing small bits off and dipping them into your stew, Chinese food, and lots of spaghetti or rice. And of course, plantains, lots and lots of plantains.

I had to spend a month in Accra in a rather boxy cement house, nothing too special for the community it was in, for the first computer shipment to be cleared out of customs.

Often the power was out for 12 hours at a time, alternating daylight and night hours every 3 days. Some of these days I "aerated" the pool (You know, to keep mosquitoes from breeding in it!), others I read (Yes Harvey, Ron, Helen, I am reading again, and quite a bit, and almost some of it isn't trash).


I love Ghana, but I am sorry to say I absolutely hate the port city Tema. After multiple days of 6-14 hour waits dealing with an overzealous Fantis spinster, while nothing happened with Yaw ("Yow" the computer teacher) and briefly Toon ("Tone" the Belgian volunteer from ICYE), we finally got our first of 3 shipments out.

I was happy to find almost all the computers were Pentium IIIs with at least 450mhz on them and a decent amount of ram. I had been warned they may be Pentium Is, possibly worse. We loaded up our van, and took the equipment first to the house in Accra, and then the 4-5 hour drive inland to Kumasi and the school.

On my first arrival there with the van full of computer equipment, it was an experience like no other. The small children swarmed me all wanting to shake my hand, hold it, guide me to the building I was to live in, or wherever I wanted to go, or they wished to take me. It was something I simply cannot put into words.

Shortly after arriving Toon, Yaw and I got to work on sorting out the hardware, good from bad, testing, configuring, planning (all half-assedly as I could muster, in hind sight), etc. Strangely enough I kept losing my blank CDs. At first I thought I misplaced a few, but a few days later my entire spindle (30+) had decided to go for a walk. I had realized the CDs were being stolen, but no one entered the computer room without Toon or Yaw or I in it supervising... There's no way someone could have stolen thirty plus CDs out from under us... I am not nearly the criminal investigative genius I thought I was. One of the students told me one day when I was complaining where my CDs got lost to that children were taking them. He then proceeded to tell me how.

The children brought books into the computer lab and would sneak empty CDs in between the pages while we weren't looking. They would then LEAVE the books in the classroom when they had to go to class next. Later after school or near the end of school they would come back to the computer lab and ask Toon or I if they could get their books... If there is anything in my life more humbling than one student feeling sorry for me enough to tell me how horribly I had been tricked I hope to avoid it. These children were less than 10 years old, and I was thoroughly impressed, amused, and bloody annoyed.

Now that the lab is setup, we almost have our hands on a backup generator, and the internet is working and the systems are nearly secured from user error (kids will be kids...), I have begun work on rewriting the antiquated computer text book (MS-DOS is the primary focus of the book, you remember DOS, right?) for the school (partly to start putting together a curriculum for the class I will be offering the faculty in a few weeks or months), and possibly other schools.

I am truly enjoying the work I do here and the freedom that comes from the project, it's ever-evolving goals, and what I am experiencing of Ghanaian life in general. Each day I wake up feeling that the work I am doing is worthwhile, have delicious meals, find time to read (sometimes forcibly by candlelight), research new ways to solve computer problems, or just hang out with some of the older boarding students whom have befriended me and been an almost endless supply of laughter and hijinx in the evenings or on weekends.

I've made some friends and they make me get away from the computer on weekends and go out and enjoy the 85 cent (delicious) lager pints, or go to various towns nearby to meet other volunteers, see funerals (which are actually somewhat enjoyable), and other sights and sounds of this great, and truly free, country.

It is somewhat late and my ability to write (already hampered by our very great American Education system) is slowing being worn down by this pesky call from my pillow. Goodnight.

Sincerely,

Evan Taylor