Sabayon Stabiliser

The following talks about how you can gradually move Sabayon from the testing tree to the stable tree of portage.

Sabayon. Very cool. However, it is a bit disturbing that it bases everything on the testing tree of portage.

In make.conf, there is a line:

ACCEPT_KEYWORDS=~amd64

That last part may read ~x86 if you are on a 32bit platform. In gentoo speak a package is "masked" in the portage tree (the software repository) until it has completed testing. The ~x86 or ~amd64 keyword identifies a packages as being in testing. Once testing is complete, the keyword changes to "x86". Ie, without the tilda.

The line above is essentially saying that whenever you install a package, it should install the latest testing version.

This can lead to problems, as the packages in testing haven't been tested to see if they are ready for release - that they haven't been QAed to work with everything else you may have installed.

So just remove the line from make.conf right? Wrong. The dependencies that packages have on each other are far too complex for this to work. Another strategy would be to work out groups of packages that work together, and roll them back to stable using /etc/portage/package.keywords and /etc/portage/package.mask

Forget it. Portage is the tangled web we weave.

The following approach works on this principle: You accept that you currently have a bunch of testing packages installed,but everything seems to be working. Then take advantage of the fact that lots of people are volunteering their time to ensure that these packages are suitable for release, or fixing them so that they are. So if you could say "Ok, no package can be upgraded from now on until they are in the release tree".

So look at this:

equery -i list | sed 's/(.*)/\>\=$1 -~amd64 amd64/' > packages

This command will get a list all of your currently installed packages, add a ">" to the beginning of each line, and add " -~amd64 amd64" to the end of each line, and output them all to a file called "packages"

So if the package was this:

media-video/vlc-0.8.6-r1

Now it looks like this:

>media-video/vlc-0.8.6-r1 -~amd64 amd64


If this line were put into /etc/portage/package.keywords, it would be saying "remove the "~amd64" mask from this package, and add the release mask, if the version number is greater than the one shown. In other words, don't install any more test versions for this package.

So if the file you generated is added to the end of /etc/portage/keywords then any updates to these packages will not be installed unless they are in the release tree. So now, all you need to do is wait until enough packages are in release that you feel confident you can remove the ACCEPT_KEYWORDS line from make.conf.

Note that for the above file, the first line of the "packages" file will not be valid. Remove the first line. You can append the file to package.keywords with (as root):

cat packages >> /etc/portage/package.keywords

Note that this is not a fire and forget solution. There are likely to be dependency tweaks you need to make as you go along, and you will have to enter a line in package.keywords for any new packages you install, but this at least will stop "emerge world" from ensuring your system is never stable.

GridStat

Are you looking for the Gridstat grid.org statistics graphics generator?

It is offline at the moment while I resolve hosting for it. If this is something you value, please leave a message here, and I will expedite it.

Windows the Boot

Yeah, Windows sucks. It is truly awful. But a necessary evil for myself and many others.

One of the suckiest things is how it refuses to accept that there can be in existence any other operating systems in the world but itself.

So I have a PC, and had a Windows and Linux install for some time running together, with grub in the MBR handling the booting both Linix and Windows. Or at least passing booting off to ntldr to bring windows up. No problem.

But then I wanted to reinstall Windows in another partition. It is always good to do a reinstall of Windows into a fresh partition so that you can copy over bits and pieces of the old install as you need them (notably stuff in the Application Data folder in your profile). So it was time. My Windows installation had gotten slow and bloated after only six months, as I randomly install software to make the experience more palatable. My new strategy was to run everything from Linux primarily, and do Windows stuff from a windows VM on my server, and just use native Windows for games.

I did try and use VM Server as a way of reinstalling Windows into the partition without having to leave Linux, by creating a partition based disk for the vm. VM Server reports that the partition table is invalid if you do this on my install. There are a few people who have experienced this before, but none seemed quite the same as mine, and I didn't get to the bottom of it.

So it was time for a native install. So a PC fairly useless while Windows gets on with copying itself. Except it didn't work. This was new - most of the problems you get with Windows I have encountered one time or another but this one I hadn't seen.

The CD would boot, and do the "Checking Hardware configuration" message that precedes the blue install screen, but it never got to the blue. It just cleared the screen and hung.

So.. a bit of a googling (sorry Google for using your name as a verb, but seriously, it is one now) and I discover that Windows XP install will not work if there is a boot sector that it doesn't know. Ie, if there is one present, and it isn't a Windows one. Either that or the partition tables aren't aligned the way it expects.

So a bit of monkeying around later - reorganising the partitions in a way that I did not want, and having to boot the disk with no hard drives connected, I finally got Windows installed. Of course my boot sector is now trashed with the Windows one instead of grub.

But clearly putting grub back is only going to give me problems next time I want to wipe windows.

The good news is that you can boot Linux - essentially chain grub - from NTLDR - the windows boot manager. There are a stack of articles about it around, and from my experience GRLDR doesn't work - this is the grub4dos boot manager that should be able to read and process a standard menu.lst file. The documentation is erratic, and says that it must on the boot drive, but cannot be on ntfs. Not sure how you resolve that if your boot drive is ntfs.

So the other way is to get a copy of the grubbed boot sector, make a file out of it, put it in the root of your boot drive, and just add it to boot.ini.

You need to understand which is your boot drive. In my case grub was installed in the MBR of the primary disk /dev/hda. After all this stuffing around, that had been wiped, so I reinstalled it into another drive:


# grub
grub> root (hd0,7)
grub> setup (hd0,7)


This will install grub into /dev/hda8 (the eighth partition of the first ide drive - grub itself numbers partitions from zero). If your first drive is SATA, then this may refer to /dev/sda

Now make a copy of the first 512 sectors:


dd if=/dev/hda8 of=linux.boot bs=512 count=1


Copy this to your primary Windows partition - wherever boot.ini resides (usually c:\). Then edit boot.ini and put in the line:


c:\linux.boot="Linux"


Reboot, and this will appear in the standard boot menu for Windows.

Sabayon

Sabayon linux is an interesting distro. It is based on Gentoo, which is a favorite of many, but seeks to eliminate some of the compilation hassle of Gentoo by providing a live CD and an installer that installs precompiled packages.

A desktop Gentoo install would be up and running in a day or so, whereas you can get Sabayon installed and running as quickly as any other distro - the install time is a function of the number of packages you install.

So how does it work in real life? Well, installation is indeed a breeze - using the mini-CD option you are up and running in no time.

The mini-CD has a subset of the full Sabayon install DVD, which comes in at 3.3 gigs. The Sabayon team say that they have eliminated all the "useless" things to make the mini version. These useless things are presumably included on the DVD.

So once you have installed, you will want to add a few of your own packages. And this is where things get interesting. Firstly Sabayon has a release schedule, and an existing Sabayon installation can be upgraded to the new release, though only those packages that are part of the distro. I would imagine this would often cause non-Sabayon packages to break.

However, if you are a happy to manage this, then install some extras. But this is where things get tangled. For example, it is recommended that an

emerge --sync
layman -S
emerge --newuse --update --deep world

is performed to get everything up to date. This is pretty important as anything you want to install over and above standard Sabayon, is likely have library version dependencies that are not met by the install disk libraries.

So what happens here in effect is that everything in standard install is almost immediately replaced by updated and recompiled versions. And something is bound to want to look at the kernel in /usr/src/linux, so you had better put one there. And build it. And then install it (thankfully Sabayon does provide the .config file for the default kernel in /proc/config.gz). But the whole point of Sabayon is to not have to do this... I had around 6 gigs of downloads once I had updated everything.

But then there is something else Sabayonic that comes into play. Sabayon states that it is cutting edge - and it is - it comes with late release nvidia and ati drivers, and aiglx, xgl and beryl are there out of the box. But in order to be cutting edge, masked packages need to be accepted.

Masked packages are flagged with ~amd64 or ~x86 depending on your architecture. These are packages that have not been fully tested and so it is up to you if you want to risk installing them.

A sabayon default installation has "ACCEPT_KEYWORDS=~amd64" in make.conf, which means that it will emerge anything in testing. This is how it achieves cutting edge-ness. But obviously not everything in testing will work - thats the whole point of testing it. So the emerge will fail on various packages, which will cause you to try and find a path through the dependencies that will allow everything to work - by managing what goes into /etc/portage/package.mask. This is pretty difficult.

And it seems to me that it is impossible to go back to not bring in untested packages. Well, not impossible, but hard. The keyword can be switched off, but then everything needs recompilation. But some of the libraries cannot be downgraded - you are warned that downgrading will break the system - like glibc.

So anything that relies on a library that is currently in testing cannot be downgraded must by definition be an "in-testing" package. So trying to unravel the dependency web is intractable.

Having gentoo on the desktop is pretty cool, and Sabayon does a good job of getting you there - but the nature of gentoo is that you are compiled to your architecture, so having precompiled binaries doesn't give you any of the benefits that gentoo is about - except emerge's excellent dependency management. But that demands that you recompile everything, which then makes Sabayon redundant.

Perhaps the better approach would be to install vanilla gentoo, then use layman to bring in the sabayon overlay, and use the Sabayon part of the portage tree to bring beryl and all the extra goodies that make Sabayon worthwhile.

The Bank Counter-Conspiracy....

Hello. If you work at a bank, please look away now. This information is not for you.

We are all aware, to some degree, that the worlds banks are conspiring against us in order to extract as much money as they can for us.

I can now reveal a conspiracy that you can be part of against the banks. Here it is. Take a look at one of your many credit cards. Now turn it over and observe the signature strip... notice how narrow it is? Try writing on it... notice how it seems to repel most ordinary forms of pen delivered ink?

Now this is probably something you have known in the back of your mind for many years - since you sent off your first credit card application and got your sweaty credit virgin hands on a plastic rectangle that was your ticket to free money. All you had to do was turn it over and sign on the... d'oh! The strip is too small for any reasonable signature and damn it, none of these pens leave a mark. You probably put this down to early inexperience and let it fester into a growing feeling of inadequacy as you grew through credit filled life.

Well, it doesn't have to be that way. There are millions like you. Billions. In fact, here is the kicker: Most people on the planet are aware that credit card signature strips are hopeless for recording signatures. It isn't just you.

The truth is, the only people on the planet that do not know this fact are those working in the bank and credit industries. This is the one morsel of information that we have and they don't. Huzzah! Please don't tell them.

Media PC: Tuners

You are probably thinking of buying or building a media PC. With the advent of Vista, I foresee a rapid increase in the use of media PCs in the living room. That's if anyone buys Vista of course.

If you have digital TV broadcasts in your area, then get a DVB-T tuner. Forget about analog - the price difference make it a no brainer, and analog will be switched off pretty soon. Hopefully. So all that frequency space can be freed up and used for something better.

How many tuners? Well there are 7 free-to-air channels where I am, with content on about five of them that I would watch. Still, for that many channels, I'd say two is the minimum, and three would be good.

Now you may be thinking - well hold on, I managed perfectly well with a single VCR, why would I suddenly need to be recording 3 shows concurrently? How often would that happen?

The thing about switching to digital recording is that your whole approach to TV will change. If you have regular shows that you would watch, you'll find that you will watch almost all of them "time-shifted". The first thing you do when you get your MPC set up is to go through the programme guide and tell it to record all your favorite shows. You are likely to get a overlap here. And you may be thinking, well I can just watch one show live, and record the other...

The thing is that once you get used to having all your shows recorded whenever they are on, you will become more flexible about when and how you watch them. You'll no longer be timetabling your life around what is on (you do this right?) - everything gets recorded, and you watch when you can, or when you want. So the show that you used to watch while the other show was being VCRd on the other channel becomes a hassle, and you'll want both of them to be recorded. Not least of all so you can pause the show and come back with that cup of coffee. Besides, recorded shows means no more adbreaks - that you need to pay attention to - you can break whenever you want, not when the network wants to sell you something.

Overrun is also an important thing. In this country, they take great pride in considering a TV schedule a 'guide' - a kind of "nice to have", but not really that important. So even channels with no live content still do not trouble themselves with shows starting when they are advertised. So you need to plan for overrun. And if you have a show back to back with another on a different channel, you need two tuners to cater for one show overrunning the start time of another.

Dvico do a fusion card that has two DVB-T tuners built in. This is a good starting point. Don't get one. Or you'll want to get another. And when this whole media PC thing works for you, you'll want to put the kit into a nice low profile case. These support microATX size motherboards, which often only have one PCI slot, and one PCI-X. Most tuner cards are still PCI, so a dual tuner card is a good idea.

Final Conflict

Remember Earth: Final Conflict? Maybe not, because I was the only one who watched it.

But they had these communicators with roll out screens... so lookee here:

http://www.polymervision.com/News-Center/Press-Releases/TelecomItaliaandPolymerVisionannouncetheCE.html

Ruby Scraping

You've always got to pick the right tool for the right job, and many would say that Ruby is the right tool for most jobs. It is pretty good, thats for sure.

But with hpricot, it is a no brainer when it comes to web scraping. hpricot is a library for extracting contents from web pages to do with what you will. Chief amongst the features you'll want for such a library are simple and fast ways to parse the tree of the site you are scraping, and hpricot has them in abundance. I haven't found anything simpler.

And then just now I find out about the firebug extension for firefox. One of the tricky things with scraping is manually figuring out the path through the tree you need to traverse to get to the bit of the page you are looking for. This blog shows how much simpler it is with firebug...

Ruby Screen-Scraper in 60 Seconds