Linux, Java & Open Source

Manual Logback Configuration

2012-01-27T00:10:00.000-08:00

Programmatic configuration of Logback is really simple, when you know how. I'm not sure why they don't have more examples like this in the manual, because in some situations, its really important that logs don't get lost even if a configuration file isn't found (particularly when STDOUT goes nowhere). Anyway, here's one for you to get you started.

import ch.qos.logback.classic.Logger;
import ch.qos.logback.classic.LoggerContext;
import ch.qos.logback.classic.encoder.PatternLayoutEncoder;
import ch.qos.logback.classic.spi.ILoggingEvent;
import ch.qos.logback.core.FileAppender;

FileAppender<ILoggingEvent> fa = new FileAppender<ILoggingEvent>();
fa.setName("file");
fa.setFile("mylog.log");
LoggerContext con = (LoggerContext) LoggerFactory.getILoggerFactory();

PatternLayoutEncoder pl = new PatternLayoutEncoder();
pl.setContext(con);
pl.setPattern("%d %5p %t [%c:%L] %m%n)");
pl.start();

fileAppender.setEncoder(pl);
fileAppender.start();

Logger rootLogger = con.getLogger(Logger.ROOT_LOGGER_NAME);
rootLogger.addAppender(fa);

Open source repository manager

2011-11-30T00:05:00.001-08:00

Looking for a open source, web based, code repository manager? Sick of having to ssh into the code server to configure new repositories? Try this out:

http://www.scm-manager.org/

Its easy to setup, supports git, svn, bzr, hg, and it works on both Windows and Linux. No need to edit configuration files; everything can be done via the web interface.

It has a plug-in framework, (and you can install them completely within the interface) which extend its functionality which includes integration with continuous build systems (such as Bamboo and Jenkins), and support for LDAP authentication.

I would love to see this project take off, so get behind it, if you can.

Auto authenticating proxy - Windows

2011-11-24T23:31:00.001-08:00

If your running Windows applications that either don't support Windows (NTLM) proxy authentication, or don't support it well, it can be difficult to connect them to the internet in a corporate environment. Often the effort in making changes to a firewall (if indeed it's possible) outweighs the benefits of using the application.

I found myself in this position, and spent a while looking for a local proxy that would automatically authenticate connections from localhost to the internet. After some hours of searching, I came across this:

http://ntlmaps.sourceforge.net/

Which did the trick! This also has an added benefit of keeping all your authentication details in once place. Unfortunately it stores the password in an un-encrypted configuration file, but it does have an option to leave it blank so you can enter it once when the application starts.

Postgres 9.0 to 9.1 upgrade - Gentoo

2011-10-02T02:46:00.000-07:00

If your running a virtual machine, with an automatic upgrade script you may find you need up upgrade your data to work with 9.1, and that the old version has been removed. To do the upgrade you need both version installed.

First step, install the old version of 9.0, if its not already there. You can check via:

eselect postgresql list

Check /var/lib/postgres, and see if the 9.0 data directory exists. Before doing anything else, make a copy this folder to a safe place, so you can revert if required.

If eselect didn't come up with any 9.0 version, install it using this.

emerge =dev-db/postgresql-server-9.0.5

Try and start 9.05, and make sure there are no errors. I found I had to move some configuration files around, and change some permissions, but the errors were very explicit and easy to solve.

Now, copy 9.0's configuration files.

cp -R /etc/postgresql-9.0/* /etc/postgresql-9.1/

Next step, create a blank 9.1 database in /var/lib/postgresql/9.1/data

su postgres

initdb91 /var/lib/postgresql/9.1/data

Make sure 9.1 can start.

/etc/init.d/postgresql-9.1 start

Stop it again, and your ready to run the conversion.

su postgres

pg_upgrade91 -v --old-datadir=/var/lib/postgresql/9.0/data/ --new-datadir=/var/lib/postgresql/9.1/data --old-bindir=/usr/lib/postgresql-9.0/bin/ --new-bindir=/usr/lib/postgresql-9.1/bin/

If this completes, start 9.1. You can run it using the init.d script, but I found I had an error which was not going to the logs. You can debug it by running pg_ctrl directly, which will print any errors to the screen.

su -l postgres -c 'env PGPORT="5432"  /usr/lib/postgresql-9.1/bin/pg_ctl start -w -t 60 -s -D /var/lib/postgresql/9.1/data -o '\''-D /etc/postgresql-9.1/ --data-directory=/var/lib/postgresql/9.1/data'\'''

If everything is fine, your data migration should be complete, and all thats left to do is tidy up by removing 9.0's binary and the old data. i.e

emerge --unmerge dev-db/postgresql-server-9.0.3

Redmine on Arch Linux

2011-08-28T01:59:00.000-07:00

Installing redmine seems a bit "clunky" to me, but thats probably because I've never used ruby jems before. I've put down steps involved in getting it working, in an NON-PRODUCTION enviroment (I.e not secure, just quick and dirty to get it running fast). This worked with release 1.2.1, and it installs mysql, but redmine works off other databases too, so modify to your needs.

First up, install all the prerequisits.

pacman -S rubygems

gem install -v=0.4.2 i18n

gem install rack -v=1.1.0

pacman -S mysql

/etc/rc.d/mysqld start

/usr/bin/mysqladmin -u root password 'root'

pacman -S make

gem install mysql

Download and unpack the release. All the configuration files are in ./config/, so you need to go there next. We just change a few things in the example configs, so move them into place.

mv database.yml.example database.yml

mv configuration.yml.example configuration.yml

Edit database.yml, and change the top to this.

production:

adapter: mysql

database: redmine

host: localhost

username: root

password: root

encoding: utf8

Next, you have to edit the environment.rb file. Within the Rails::Initializer.run do |config| section,

add this line:

config.action_controller.session = { :key => "_myapp_session", :secret => "$t =>"ee7f29f2bb6f5006ff0674b7838e5dd7" }

Finally, populate the database.

RAILS_ENV=production rake db:migrate

RAILS_ENV=production rake redmine:load_default_data

And now your ready to go. Start the server and browse to http://localhost:30003

The default login is admin/admin.

script/server -e production

Netbeans 7.01 Startup Error

2011-08-03T02:34:00.001-07:00

I've come across a dbus error while starting Netbeans.

arguments to dbus_message_iter_append_basic() were incorrect, assertion "*bool_p == 0 || *bool_p == 1" failed in file dbus-message.c line 2541.

This is normally a bug in some application using the D-Bus library.

D-Bus not built with -rdynamic so unable to print a backtrace

/usr/share/netbeans/bin/../platform/lib/nbexec: line 548: 2593 Aborted

This was running Java 7, and Netbeans 7.01. I found the answer here:

https://bbs.archlinux.org/viewtopic.php?pid=939873

This states that appending this:

-J-Dnetbeans.keyring.no.native=true

to netbeans_default_options in netbeans.conf (in /usr/share/netbeans/etc using Arch Linux), will fix the problem; and it does!

Virtual Audio Cable - Linux and Windows

2011-07-22T12:33:00.000-07:00

If your looking for a virtual audio cable (VAC), thats free, open source and works on Linux, Windows and OSX check out jackaudio. For some reason this neat bit of software doesn't get nearly the attention it deserves and despite what you'll read out on the net, its most definitely does work on Windows (even Windows 7, 64 bit). Its really simple to use, just:

Install it.

Start Jack Control (will come up in your list of programs after the install).

Go into the settings, and select "portaudio" as the driver.

Start Jack Port Audio.

Restart Jack control.

If you right click on the sound control icon (on the far right of your task bar), and select recording devices, you should see a new device called "Rec. Playback". You can record from this device and it will capture the sound from your sound card. (Note you have to set it to the "default" recording device if your just using the standard windows recorder). If you need any fancy routing, you can specify what you want where in the "Connect" screen within Jack Control.

Theres ALOT of other options, none of which I've played with but for just recording your soundcard output to a file this will do the job well.

Mac Address String to byte[]

2011-06-01T00:56:00.000-07:00

The Android SDK returns a String mac address, while the Java SDK returns a byte array. This makes it somewhat difficult to compare the two.

Android:

WifiManager manager = (WifiManager) getSystemService(Context.WIFI_SERVICE);
WifiInfo info = manager.getConnectionInfo();
String address = info.getMacAddress());

Java SDK

InetAddress inet = InetAddress.getLocalHost();
NetworkInterface ni = NetworkInterface.getByInetAddress(inet);
byte[] address = ni.getHardwareAddress();

The code below illustrates how to convert the string returned by Android, into a byte[].

public static byte[] parseMacAddress(String macAddress)
    {
        String[] bytes = macAddress.split(":");
        byte[] parsed = new byte[bytes.length];

        for (int x = 0; x < bytes.length; x++)
        {
            BigInteger temp = new BigInteger(bytes[x], 16);
            byte[] raw = temp.toByteArray();
            parsed[x] = raw[raw.length - 1];
        }
        return parsed;
    }

Django and processing JSON

2011-05-29T11:21:00.000-07:00

Django ships with the JSON processing library simplejson. Although there is talk that it may be removed in the future, I see no disadvantage in using it; if it is removed then just install it manually.

This tutorial will explain how to create a form that allows the user to upload a file, then processes the json contained within it.

First, define your model (models.py)

class Import(models.Model): 
        file = models.FileField(upload_to='tmp') 
 
        def __unicode__(self): 
                return self.file

Then define the form (forms.py)

class ImportForm(forms.Form): 
 
 file =  forms.FileField() 
 class Meta: 
  model = ImportModel

Create your template, making sure you include enctype="multipart/form-data" in your html form declaration.

Now, the actual processing of the file goes into your form's view.

# other standard django imports here.. 
from django.utils import simplejson 
 
def import_form(request): 
    
    
    if request.method == 'POST': 
        form = ImportForm(request.user, request.FILES) 
        
        if form.is_valid(): 
            file = request.FILES['file'] # get reference to the file here 
            my_data = simplejson.loads(file.read()) 
            
            # now you can process  
            items =  my_data['items'] 
 
            for item in items: 
                # do your processing here... 
                print "item is: " + item['attribute'] 
 
            return HttpResponseRedirect('/somewhere') 
        else: 
            print "form invalid!" 
    
    else: 
        form = ImportForm() 
            
    return render_to_response('your_template.html', {'form': form}, context_instance=RequestContext(request))

Arch Linux DELL Studio 1558 Notebook Video card problems

2011-04-06T00:24:00.000-07:00

I was having problems with the fan going really fast all the time (at max speed) with my laptop, after switching from the propriety ATI graphics card driver to the open source one. The only reason I switched was because an update caused the propriety one to die, and I couldn't get it back again.

This problem is especially bad for laptops, when running on battery. The card this laptop has is:

02:00.0 VGA compatible controller: ATI Technologies Inc Manhattan [Mobility Radeon HD 5000 Series]

Anyway, long story short, with the open source driver (at least), you need to enable KMS. Its enabled via a kernel option in your boot loader (radeon.modeset=1), by adding the radeon module in /etc/rc.conf on the MODULES line. A complete explanation is here:

https://wiki.archlinux.org/index.php/ATI#Enabling_KMS

After you have done that, you can then specify the profiles, which is stated here:

http://www.overclock.net/9332962-post1.html

Stress testing Asterisk with Sipp

2011-02-25T02:21:00.001-08:00

Firstly, I had problems with running sipp in client mode from Ubuntu 10.10. The UDP packets were leaving my machine, but never gettting to the remote machine. I have no idea why, but I suggest you build sipp from scrach (its pretty easy), and use that.

There is heaps of documentation on sipp's website, but it assumes you have a decent knowledge of sip, in which case you probably won't be reading this anyway.

For a start, you need to understand that sipp can run in a client and server mode. This means, it can act as caller, or a reciever of calls. Thus, in order for you to be able to record statistics, you need to setup your asterisk box so there is a loop; the client creates calls, which end calling the server. Both the client and server display statistics about the calls being made, and from that you can see how your box is handling it.

To setup asterisk, you need to edit the sip.conf and extensions.conf.

------------ sip.conf --------------

[sippuac]
type=friend
username=sippuac
host=
port=5060
context=default
dtmfmode=rfc2833
insecure=very
canreinvite=no
nat=yes

[sippuas]
type=friend
username=sippuas
host=
port=5061
context=default
dtmfmode=rfc2833
insecure=very
canreinvite=no
nat=yes
------------ sip.conf --------------

There are two extensions setup in this example, one is for client(sippuac), and one is outbound to the server(sippuas). The client will connect on the default sip port 5060 in this example, and the server connection will be made to 5061. I have disabled security in this example just to make things simple, as thats all the default senario can handle. You need to change the host for each to wherever each sipp client and server is running. You cannot set the host to dynamic, as asterisk needs to make the connection (I could be wrong here, but dynamic did not work for me).

Next is extensions.conf.

---------- extensions.conf ---------

[default]
exten=>testing,1,Dial(SIP/sippuas,20)
---------- extensions.conf ---------

In the default context I put an extension called "testing", which then calls the server. Reload asterisks using the cli, so all we have to do is setup sipp to call this extension, and listen using sippuas.

To run sipp in server mode, you need to specify something like this:

sipp -sn uas -p 5061

The -sn flag specifies a seneriao. UAS is a built in scenario, which just receives a call and hangs up. the -p flag is the UDP port that it will listen for incoming calls on (this port must match whats in sippuas).

To run sipp in client mode, you need to specify something like this:

sipp -sn uac 127.0.0.1 -s testing

The scenario uac now specifys a built in client scenario. 127.0.0.1 is the IP address of the asterisk box (Im running it from the asterisk server itself, but you don't need to), and -s testing specifies the extension to call. The default port is 5060 which we are using anyway.

So, hopefully now, you should see stats appear on your screen!

Telia E1750 Mobile Broadband on Linux

2011-02-23T06:54:00.000-08:00

This explains how I got my Telia Mobile Broadband (Telia Mobilt bredband) usb modem working with Arch linux. The theory should be the same though, for any distribution.

What you need:

1. A Huawei E1750 usb modem
2. Windows installation (To reset the pin and test the modem)
3. An existing internet connection (To install packages)
4. Linux with ppp and ppp/usb support built into its kernel (*)

*Any big distribution should have this by default.

Firstly, install and connection your modem in Windows using Telia's software. You need to do this to establish that your modem is working properly, and, more importantly to disable the PIN on it (Its in the options menus). (Note that doing this makes your usb unsecure as anybody can steal it and start using your modem, but I can't seem to get the pin to enter in correctly within Linux).

Once you have connected properly, and disabled your pin, boot into linux. You first need to install usb_modeswitch(http://www.draisberghof.de/usb_modeswitch/), as this will allow your modem to be detected as both a usb drive and as a modem. In arch linux you do it like this:

pacman -S usb_modeswitch
ldconfig

Connect your usb. In dmessg you should see this:

usbcore: registered new interface driver usbserial
USB Serial support registered for generic
usbcore: registered new interface driver usbserial_generic
usbserial: USB Serial Driver core
USB Serial support registered for GSM modem (1-port)
option 2-1.1:1.0: GSM modem (1-port) converter detected
usb 2-1.1: GSM modem (1-port) converter now attached to ttyUSB0
option 2-1.1:1.1: GSM modem (1-port) converter detected
usb 2-1.1: GSM modem (1-port) converter now attached to ttyUSB1
option 2-1.1:1.2: GSM modem (1-port) converter detected
usb 2-1.1: GSM modem (1-port) converter now attached to ttyUSB2
usbcore: registered new interface driver option

And if you do lsusb you should see something like this:

Bus 002 Device 006: ID 12d1:1001 Huawei Technologies Co., Ltd. E620 USB Modem

Now that your USB modem is detected, its time to install the ppp applications used to manage it, and they are ppp and wvdial. Again, with arch

pacman -S ppp
pacman -S wvdial

Setting up wvdial is as simple as editing /etc/wvdial.conf and adding the following:

[Dialer Defaults]
Init1 = ATZ
Init2 = ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
Init3 = AT+CGDCONT=1,"IP","online.telia.se"
Modem Type = Analog Modem
ISDN = 0
Password = dummy
Username = dummy
Phone = *99#
Modem = /dev/ttyUSB0
Baud = 9600
Dial Command = ATDT
Stupid Mode = 1

Type wvdial and hopefully this appears:

--> WvDial: Internet dialer version 1.61
--> Cannot get information for serial port.
--> Initializing modem.
--> Sending: ATZ
ATZ
OK
--> Sending: ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
OK
--> Sending: AT+CGDCONT=1,"IP","online.telia.se"
AT+CGDCONT=1,"IP","online.telia.se"
OK
--> Modem initialized.
--> Sending: ATDT*99#
--> Waiting for carrier.
ATDT*99#
CONNECT
--> Carrier detected.  Starting PPP immediately.
--> Starting pppd at Wed Feb 23 15:35:31 2011
--> Pid of pppd: 4416
--> Using interface ppp0
--> local  IP address 
--> remote IP address 
--> primary   DNS address 
--> secondary DNS address

Your now connected to the internet! For some reason pppd did not set my default gateway the first time, so I needed to add it myself by doing this, although you may not need to.

route add default gw

Gentoo Install (Not live) on a USB stick

2011-02-18T09:36:00.000-08:00

Here is a quick tutorial of how to install Gentoo on a USB stick, using the gentoo minimal cd. We start out installing the live cd onto the USB, then migrate it to a proper installation. You don't need to be running any particular distribution, you don't need a cdrom drive, but what you will need is:

1. unetbootin
2. extlinux
3. A usb stick
4. Something to partition the usb stick with
5. The usb minimal iso

First of all, partition your usb stick (using gparted or similar). As we are going to start installing straight from the usb, you need to set this up now, so make room for a swap partition and your root partition. I created one root partition EXT4, and one swap partition, without bothering for a boot partition.

Use unetbootin to install the live cd onto the usb, and specify the root partition that you created just before. When thats done, mount the usb, go to the mount point and run these commands:

mv isolinux extlinux
mv extlinux/isolinux.cfg extlinux/extlinux.conf
rm extlinux/isolinux.bin
rsync -av extlinux
extlinux -i extlinux

Unmount the USB. You should now be able to boot with it.

The USB will boot to the livecd console. If your running DHCP everything should be working (if not, look at gentoo.org), and you can now begin setting up your operating system.

If you check fdisk, your root partition should be mounted in /mnt/cdrom, as readonly. Remount it as read/write.

mount -n -o remount,rw /mnt/cdrom

Now, chroot there.

chroot /mnt/cdrom
mount -t proc none /proc
mount -o bind dev /dev
mount -t devpts devpts /dev/pts
env-update
source /etc/profile

You will need to put in a nameserver in /etc/resolv.conf, as the live cd doesn't have anything in there. Once your done, you should now be able to continue with section 5 of the gentoo handbook, "Installing the Gentoo Installation files". That is, download the stage 3 file, untar in the root directory, and extract portage into /usr.

When its time to compile your kernel, make sure you have all the drivers required for USB (i.e SATA). In this example I have not created a boot partition, so, you will need grub to mount the boot partition rw, and enable Support for large (2TB) block devices (Under enable block devices) EVEN if your USB is no where near that size.

Also, you will need to add another parameter into grub on your kernel line, otherwise risk the error "cannot find root". This delays the kernel from accessing the root partition, allowing your usb drivers to setup properly. Heres an example grub.conf

title Gentoo Linux 2.6.34
root (hd0, 1)
kernel /boot/kernel-2.6.34-gentoo-r1 root/dev/hda2 rw rootdelay=10

If its still not detected, try making it longer.

All thats left to do is remove all the old live cd install files, such as the ones I've listed below in the root directory, and your ready to go.

extlinux           livecd               ubninit
extlinux.conf           menu.c32             ubnkern
gentoo.efimg            stage3-i686-20110215.tar.bz2  
gentoo.efimg.mountPoint syslinux.cfg         ubnpathl.txt
image.squashfs          ubnfilel.txt

Finally, be careful about the USB devices you have; your usb stick may not be always /dev/hda!

Good Luck!

Simple Java Project with Gradle

2010-12-22T02:29:00.001-08:00

Heres how to setup a very simple project in Gradle.

I don't use any repositories. All the jar files required for compiling the project are contained within a sub-directory of the project. I use TestNG for unit testing.

Ive found Gradle easier to use than Maven, not least because the documentation is clearer with examples all the way through. In addition, setting up Gradle for small projects seems to require a lot less effort.

This is the contents of build.gradle, which is placed in the root directory of the project. ./lib contains all the required jar files for the source, which is in ./src. Likewise, the test code is in ./test/src, and associated libraries (I.e TestNG's libraries) are in ./test/lib.

You don't need to include the "jar" section unless your planning on deploying the project as a jar file.

apply plugin: 'java'

sourceCompatibility = 1.6
version = '1.0'

dependencies {
   compile files(fileTree(dir: 'lib', includes: ['*.jar'])) 
   testCompile files(fileTree(dir: 'test/lib', includes: ['*.jar']))

}

sourceSets {
    main {
        java {
            srcDir 'src'
        }
    }    

    test {
        java {
            srcDir 'test/src'
        }
    } 
}

jar {
    manifest {
        attributes 'Implementation-Title': 'My Project', 
                   'Implementation-Version': version,
                   'Main-Class': 'package.Main'
    }
}

test {
    useTestNG()
}

SmartGWT cross domain JSON Data Source

2010-08-20T13:37:00.000-07:00

I was struggling to find an XJSON data source example (That is, a data source that is capable of loading JSON from a domain other than where the scripts resides), so battled through it myself.
The example below shows how to do it.


// Create your XJSONDataSource object. Its just a standard data source with a few whistles.

DataSource ds = new XJSONDataSource();
ds.setRecordXPath("/item");

// Set it up with whatever fields you want, as per usual

DataSourceTextField name = new DataSourceTextField("name", "Data Name");
name.setValueXPath("name");

DataSourceIntegerField id = new DataSourceIntegerField("id", "Data Id");
id.setValueXPath("id");
id.setPrimaryKey(true);

ds.setFields(id, name);

// DataSource is now ready for data. Give it a url.

ds.setDataURL("http://your_json_url_feed");

Once this loads (and remember to fetch the data through your tree grid or whatever else is viewing it), it will send a request that looks something like this:


http://your_json_url_feed?callback=isc.Comm._scriptIncludeReply_0

The JSON that your server must send must be wrapped around this callback label. For example:


isc.Comm._scriptIncludeReply_0 ({"item": [ {"id": "1","name": "Monkey"}, 
{"id": "2","name": "Tree"}, 
{"id": "3","name": "Banana"} ] })

The callback value can change, so make sure your server code picks it up and puts it in the right place within the response accordingly.

Connecting your android phone to the internet via your PC or Laptop

2010-07-21T05:05:00.000-07:00

First, lets get one thing straight. This explains how to connect your PHONE to the internet VIA your computer, which which already has a internet connection. It does NOT explain how to connect your laptop or computer to the internet VIA your phone (i.e using your phone as a modem).

If you want to do this, there are a multitude of ways, but probably the easiest is by using an application called PDAnet. The process of doing this is called "tethering". What I am going to explain is known as "reverse tethering".

Why would I want this?

Because your laptop has internet access, and your phone doesnt, and you want it on your phone.

Options that didnt work

USB Cable

Unless you have a rooted phone, you cant do this. And if you've already rooted your phone, then I doubt you'll be reading this anyway.

VIA an ADHOC network

You can create an adhoc network in windows quite easily. Most(?) phones cant connect to adhoc networks, and searching for it in your wifi settings comes up with nothing.

Solution

VIA Connectify

You will need, windows 7, your phone and the connectify software.

This method works on the Samsung GT-7500, and probably ANY phone that supports wifi.

Download the connectify application from http://www.connectify.me/

Install the application and create a new wifi hotspot.

Make sure "Share internet" is ticked in your Connectify settings (A box should popup as soon as its started)

Connect to it using your phone.

If you cant get onto the internet, make sure the connection (In Network Connections) that has internet is set to allow other network users to connect through this computers internet connection, and specify the newly created connectify network in the drop box. Press OK.

You should now be able to connect to the internet.

Arch Linux on a DELL Studio 1558 Notebook

2010-05-07T01:38:00.000-07:00

Ive switched to Arch from Ubuntu for the moment. For a couple reasons; I wanted a more configurable system from the start but mostly Im getting a little bored of Ubuntu. No other reason than that. I went with the 64bit version.

Before I get any further, the following things are working:

Wireless
Network Card
ATI Graphics card (Propritary driver)
Web Cam
Sound

Untested

HDMI Port
Card Reader

Whats not working

Media Buttons (*)
CD-ROM Eject Button(**)

* Seems to work but crashes the computer
** You can still right click in gnome and eject the cd. This comes up in dmessg.


dell-wmi: Unknown key 0 pressed

Product Description

This is what is described on the receipt from dell.


- DELL Studio 1558 Notebook-OCQT
- Intel(R)Core(TM)i7-620M(2.66GHz,4 Threads,turbo boost up to 3.33GHz,4M cache)
- 15.6 " Full HD(1080p)WLED (1920 x 1080)Display with True
- ATI Mobility Radeon 5470 1GB
- Dell(TM) Wireless 365 Bluetooth Module
- Dell(TM) Wireless 1520 802.11n Mini-Card
- 640GB 5400RPM SATA Hard Drive
- 4GB Dual-channel 1333MHz DDR3 SDRAM (2 x 2GB)
- Bezel with Integrated 2.0MP Web Cam

lspci


00:00.0 Host bridge: Intel Corporation Core Processor DRAM Controller (rev 12)
00:01.0 PCI bridge: Intel Corporation Core Processor PCI Express x16 Root Port (rev 12)
00:16.0 Communication controller: Intel Corporation 5 Series/3400 Series Chipset HECI Controller (rev 06)
00:1a.0 USB Controller: Intel Corporation 5 Series/3400 Series Chipset USB2 Enhanced Host Controller (rev 06)
00:1b.0 Audio device: Intel Corporation 5 Series/3400 Series Chipset High Definition Audio (rev 06)
00:1c.0 PCI bridge: Intel Corporation 5 Series/3400 Series Chipset PCI Express Root Port 1 (rev 06)
00:1c.1 PCI bridge: Intel Corporation 5 Series/3400 Series Chipset PCI Express Root Port 2 (rev 06)
00:1c.3 PCI bridge: Intel Corporation 5 Series/3400 Series Chipset PCI Express Root Port 4 (rev 06)
00:1c.4 PCI bridge: Intel Corporation 5 Series/3400 Series Chipset PCI Express Root Port 5 (rev 06)
00:1c.5 PCI bridge: Intel Corporation 5 Series/3400 Series Chipset PCI Express Root Port 6 (rev 06)
00:1d.0 USB Controller: Intel Corporation 5 Series/3400 Series Chipset USB2 Enhanced Host Controller (rev 06)
00:1e.0 PCI bridge: Intel Corporation 82801 Mobile PCI Bridge (rev a6)
00:1f.0 ISA bridge: Intel Corporation Mobile 5 Series Chipset LPC Interface Controller (rev 06)
00:1f.2 SATA controller: Intel Corporation 5 Series/3400 Series Chipset 4 port SATA AHCI Controller (rev 06)
00:1f.3 SMBus: Intel Corporation 5 Series/3400 Series Chipset SMBus Controller (rev 06)
00:1f.6 Signal processing controller: Intel Corporation 5 Series/3400 Series Chipset Thermal Subsystem (rev 06)
02:00.0 VGA compatible controller: ATI Technologies Inc Device 68e0
02:00.1 Audio device: ATI Technologies Inc Device aa68
04:00.0 Network controller: Broadcom Corporation Device 4353 (rev 01)
07:00.0 SD Host controller: Ricoh Co Ltd Device e822 (rev 01)
07:00.1 System peripheral: Ricoh Co Ltd Device e230 (rev 01)
07:00.2 System peripheral: Ricoh Co Ltd Device e852 (rev 01)
07:00.3 FireWire (IEEE 1394): Ricoh Co Ltd Device e832 (rev 01)
09:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 03)
ff:00.0 Host bridge: Intel Corporation Core Processor QuickPath Architecture Generic Non-core Registers (rev 02)
ff:00.1 Host bridge: Intel Corporation Core Processor QuickPath Architecture System Address Decoder (rev 02)
ff:02.0 Host bridge: Intel Corporation Core Processor QPI Link 0 (rev 02)
ff:02.1 Host bridge: Intel Corporation Core Processor QPI Physical 0 (rev 02)
ff:02.2 Host bridge: Intel Corporation Core Processor Reserved (rev 02)
ff:02.3 Host bridge: Intel Corporation Core Processor Reserved (rev 02)

lsusb


Bus 001 Device 006: ID 413c:8162 Dell Computer Corp. 
Bus 001 Device 005: ID 413c:8161 Dell Computer Corp. 
Bus 001 Device 004: ID 0c45:6417 Microdia 
Bus 001 Device 003: ID 0a5c:4500 Broadcom Corp. BCM2046B1 USB 2.0 Hub (part of BCM2046 Bluetooth)
Bus 001 Device 002: ID 8087:0020 Intel Corp. Integrated Rate Matching Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 002: ID 8087:0020 Intel Corp. Integrated Rate Matching Hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Installation

I had some issues with partitions. I wanted to keep the inital Windows 7 install and have another partition for Arch (Well two, / and swap). My inital solution was to resize the partitions using gparted with Ubuntu's live cd. This worked; I created some free space, and restarted with the Arch installation cd. When it changed to the disk resizing tool, cfdisk (when using manually set up partitions) I got this error message:


FATAL ERROR: Bad primary partition 3: Partition ends after end-of-disk
Press any key to exit cfdisk

Everywhere on the web seemed to suggest that I had to remove all the partitions and start again. This isn't the case; use fdisk, remove the parititions that were created in Ubuntu, create two new blank partitions, and write the partition table to disk. When you restart the installer, you should be able to format and select your new partitions without error.

Wireless

First thing I fixed was wireless. The wireless card requires the broadcom-wl driver, which is only available from yaourt. Append the yaourt repo to you pacman configuration file.

/etc/pacman.conf


[archlinuxfr]
Server = http://repo.archlinux.fr/x86_64

Install it.


pacman -Sy yaourt
pacman -S base-devel
yaourt -S broadcom-wl
modprobe wl

Now, eth0 or eth1 should be your wireless card.

dmesg:


eth0: Broadcom BCM4353 802.11 Hybrid Wireless Controller 5.60.48.36

I install wicd. Its, in my opinion the easiest way to connect to wireless internet. Sure, there are more lightweight options but this allows you to easily find, connect, and create profiles very easily.

pacman -S wicd

Fix for eth0 and eth1 swapping

Sometimes eth0 and eth1 swap, which is a bit of a pain when using wicd as you have to keep changing the settings. You can fix this by creating/editing the following file, and assigning the appropriate ATTR parameter to your wifi cards mac address.

File: /etc/udev/rules.d/10-network.rules


SUBSYSTEM=="net", ATTR{address}=="ff:ee:dd:cc:bb:aa", NAME="wlan0"

Graphics Card

I tried using the standard radeon driver (radeonhd does not work with this card). It worked ok, but the graphics card was going full speed all the time. This isn't very good when running on battery, but installing the propriety driver (now also in yaourt) fixed this.


# Note, you only have to run the first two commands if you currently have the open source
# driver installed.

# pacman -Rd libgl
# pacman -Rd xf86-video-ati

yaourt catalyst

Now, just follow the instructions as copied directly from yaourt.


You can use the tool 'aticonfig' to generate an xorg.conf file.
Remember to add fglrx to the MODULES list in /etc/rc.conf.
--------------------------------------------------------------
Add nomodeset to your kernel line in /boot/grub/menu.lst , ie.:
kernel /boot/vmlinuz26 root=/dev/disk/by-uuid/1aa731c0-f626-3d01-a551-8eed1c2a34f3 ro nomodeset
--------------------------------------------------------------

Sound

Note, I did not install pulseaudio. All i installed alsautil, and added alsa to the DAEMONS line in rc.conf.


pacman -S alsautils

ls -l /sys/module/snd/holders


lrwxrwxrwx 1 root root 0 May  6 13:08 snd_hda_codec -> ../../snd_hda_codec
lrwxrwxrwx 1 root root 0 May  6 13:08 snd_hda_codec_idt -> ../../snd_hda_codec_idt
lrwxrwxrwx 1 root root 0 May  6 13:08 snd_hda_intel -> ../../snd_hda_intel
lrwxrwxrwx 1 root root 0 May  6 13:08 snd_hwdep -> ../../snd_hwdep
lrwxrwxrwx 1 root root 0 May  6 13:08 snd_mixer_oss -> ../../snd_mixer_oss
lrwxrwxrwx 1 root root 0 May  6 13:08 snd_pcm -> ../../snd_pcm
lrwxrwxrwx 1 root root 0 May  6 13:08 snd_pcm_oss -> ../../snd_pcm_oss
lrwxrwxrwx 1 root root 0 May  6 13:08 snd_seq -> ../../snd_seq
lrwxrwxrwx 1 root root 0 May  6 13:08 snd_seq_device -> ../../snd_seq_device
lrwxrwxrwx 1 root root 0 May  6 13:08 snd_seq_oss -> ../../snd_seq_oss
lrwxrwxrwx 1 root root 0 May  6 13:08 snd_timer -> ../../snd_timer

The module required for this sound card is snd_hda_intel.

As per the instructions on the wiki, I added this line to /etc/modprobe.d/modprobe.conf


options snd_hda_intel index=0

After restarting or modprobing snd_hda_indel, you aplay -l should look something like this:


card 0: Intel [HDA Intel], device 0: STAC92xx Analog [STAC92xx Analog]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 1: Generic [HD-Audio Generic], device 3: ATI HDMI [ATI HDMI]
Subdevices: 1/1
Subdevice #0: subdevice #0

You can test your sound with this:


aplay -D plughw:0,0 /usr/share/sounds/alsa/Front_Center.wav

Add your user to audio group, and restart. Sound should be working.

Webcam

The driver for the webcam is included in the stock kernel. Just modprobe uvcvideo.


modprobe uvcvideo

CPU Frequency Scaling

I set cpu frequency governor to powersave. There are plenty of tutorials on the arch wiki on how to do this. One thing to watch out for though, is that you have to set the governor for EACH core. Setting it for one will not affect the other. I didn't install a daemon, so I have to change between the modules manually (Not a big deal; powersave seems fast enough anyway).
All you need is:


pacman -S cpufrequtils
modprobe acpi-cpufreq

So, add something like the below to a script that gets run on startup, say /etc/rc.local


# set all cores to powersave governer

cpufreq-set -c 0 -g powersave
cpufreq-set -c 1 -g powersave
cpufreq-set -c 2 -g powersave
cpufreq-set -c 3 -g powersave

# add to rc.conf, along with the governors you want to use.


MODULES=(acpi-cpufreq cpufreq_powersave ....)

Upgrading the kernel

After a kernel upgrade its more than likely your wireless and graphics cards will stop working.

When you try and start X11, you'll get:


(EE) unable to load module fglrx

Add this line to your /etc/pacman.conf


[catalyst]
Server = http://catalyst.apocalypsus.net/repo/catalyst/x86_64

Then


pacman -Syu # this should prompt you to upgrade catalyst; if not reinstall it anyway

Your broadcom wireless driver will have to be rebuilt again.


pacman -S base-devel
yaourt -S broadcom-wl

Note, if you get the error


Unable to read PKGBUILD

Try running yaourt as a NON root user (which you should be doing anyway).

Extras

Gnome - "unable to mount DEVICE - not authorized"

This is not an Arch/Dell issue, but something I came across. Making your xinitrc read as below, fixes it.

[~/.xinitrc]


exec ck-launch-session gnome-session

Conclusion

Thats pretty much everything I use my laptop for, so I'm happy with Arch for the moment.

Ubuntu Wireless On Toshbia L510

2010-02-06T05:54:00.001-08:00

This is on the Ubuntu forums, but I wanted to keep a reference here, as so I wouldn't forget.
The current release of Ubuntu (Karmic) does NOT have the latest kernel which includes the required drivers for the wireless card on this laptop. The wifi card this laptop has is as described in by lspci below.


03:00.0 Network controller: Realtek Semiconductor Co., Ltd. Device 8192 (rev 01)

At the time of writing, the development version (Ubuntu lucid) ships with the 2.6.32 kernel which HAS the required module but NOT the required FIRMWARE (I'll get to that later).

So, either install this kernel manually, or update to lucid using synaptic or update-manager -d, or do-release-upgrade -d.

Once you have done that, the network manager will list the wifi card as wlan0, but wifi operations will not be permitted (For example, the Network Manager will display "disconnected" under the wireless networks menu, as the firmware cannot be found, and you will not be able to do anything with it.

To install the firmware, do as follows. (You will need to apt-get git-core if you haven't already done so)


cd /tmp
git clone git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/firmware.git
sudo cp -av firmware/RTL8192E /lib/firmware/

Reboot your system. You should now have wireless!

Colouring a Swing JButton - Java

2009-11-22T20:03:00.000-08:00

This is extremely simple, but not intuitive. Without setting Opaque, and content area filled, the colour will not show; instead it will still display whatever is defined by the default look and feel. The below example will set the background of the button blue.


JButton button = new javax.swing.JButton();
button.setBackground(Color.BLUE);
button.setContentAreaFilled(false);
button.setOpaque(true);

Joomla On Ubuntu

2009-11-18T02:14:00.000-08:00

Below is a simple installation guide for installing Joomla on Ubuntu.

Installing dependences

First, install these packages.



# The mysql database that joomla uses
apt-get install mysql-server

# The Apache 2 web server
apt-get install apache2

# The php module for Apache 2
apt-get install libapache2-mod-php

# Note that if you have tried to access your site before installing the php mod or restarting 
# apache your browser may still prompt you to "download" the php file, rarther than run it.

#  If you see this error:
# *** Unable to connect to the database:The MySQL adapter "mysql" is not available.
# you need to install the folowing package:

apt-get install php5-mysql

Pre Configuation

Extract the joomla archive (you can download it from www.joomla.org into a folder within /var/www/. From now on I am going to refer to that folder as "joomla_home".

For some reason I need to create a blank configuration file in order for the configuration procedure to work.


touch /var/www/joomla_home/configuration.php
chown www-data /var/www/joomla_home/configuration.php
chgrp www-data /var/www/joomla_home/configuration.php
chmod 755 /var/www/joomla_home/configuration.php

That should be all you need for the configuration stage. You need to set up the permissions for other parts of Joomla though;



chown www-data joomla_home/ -R
chmod 755 joomla_home/ -R

#  If you get this error "*** uploading template - * Warning! Failed to move file."
#  when trying to install a template, your web user (www-data) doesnt have write access to 
# joomla_home/tmp

Setting up the database.

First, set password for root. Then login to mysql and create the database, and a user that joomla can use to access this database. In the example below the database name is "joomla_cms", the username is "joomla" and the password is "joomla_pass".


create database joomla_cms;
CREATE USER 'joomla'@'localhost' IDENTIFIED BY 'joomla_pass'
use joomla_cms;
grant all privileges on joomla_cms.* to 'joomla'@'localhost';

Configuration

Now, you should be able to browse to the joomla configuration page.
Once thats completed, remove the configuration folder, and you should be able to log into joomla.

Extras

Some modules require the php-gd package to make thumbnails of bigger pictures.
You can install it using this:


apt-get install php5-gd       # required to to make thumbnails

permissions:

-- installing GD library for event plugin


apt-get install php5-gd # required to make thumbnails

Manually setting up a PPTP connection

2009-11-17T03:39:00.000-08:00

A Ubuntu upgrade rendered network-manager useless. So I switched to wicd, which I personally think is better, except for the fact that its missing PPTP vpn support. Turns out setting one of these via the command line is very easy. The script below demonstrates how to do it.

#!/bin/bash
pptpsetup --create work --server 10.0.0.1 --username yourusername \
--password yourpassword --encrypt --start

# add routes

route add -net 192.168.7.0/24 ppp0
route add -net 192.168.9.0/24 ppp0

10.0.0.1 is the IP address of your VPN server. This will more than likely be a public IP address.
yourusername is the username you use to connect to the pptp vpn with.
yourpassword is the password you use to connect to the pptp vpn with.

Note pptpsetup seems to cache your password along side the created vpn. For example, you
may have run the above command with an invalid command and got this:


MS-CHAP authentication failed: I don't like you.  Go 'way.
CHAP authentication failed

If your sure you now have the correct password, try changing "--create work" to some other name.

After running the pptpsetup command, you'll hopefully get something like this:

Using interface ppp0
Connect: ppp0 <--> /dev/pts/3
CHAP authentication succeeded
MPPE 128-bit stateless compression enabled
local  IP address 192.168.7.29
remote IP address 192.168.7.1

This tells us that the command has created adapter "ppp0" for our vpn connection.
Finally, we must add the required routes into our routing table to route out this adapter. In the example it will route all traffic destined to .7 and .9 out our newly created vpn interface.

Running java with limited memory

2009-11-12T19:11:00.000-08:00

If you've subscribed to a shared VPS service, chances are you have a limited amount of memory to worth with. This can be a problem for java. If java cannot reserve enough memory for the VM, you'll likely get the error below.


Error occurred during initialization of VM
Could not reserve enough space for object heap
Could not create the Java virtual machine.

My first instinct was to create a swap file. You can do this by doing the following.


# create a blank file (542 Megs)
dd if=/dev/zero of=myswapfile bs=1024 count=524288
# make it a swap file
mkswap myswapfile
# Activate swap space:
swapon /myswapfile

This command is restricted though, on most VPS, meaning you'll get this error.


swapon: myswapfile: Operation not permitted

Solution

By running the following command, you can set the amount of heap space that java will use. You will want to adjust the parameters to suit your situation.


export _JAVA_OPTIONS="-Xms20m -Xmx64m -XX:MinHeapFreeRatio=20 -XX:MaxHeapFreeRatio=40 
-XX:NewSize=10m -XX:MaxNewSize=10m -XX:SurvivorRatio=6 -XX:TargetSurvivorRatio=80 
-XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled"

Controlling mouse with keyboard in Ubuntu

2009-11-10T19:56:00.000-08:00

Its only when your stuck on a machine where you don't have a mouse, or when your mouse decides not to work that the short-cut to control your mouse with your keyboard suddenly becomes really handy. The short-cut is:


SHIFT + CTRL + ALT + NUM LOCK

Once you press this, you can then control your mouse using the keypad! Pressing 5, is the same as a left click. I believe this is a short-cut that's controlled by X-org, so should work in most Linux distributions, but I've only tested it with Ubuntu.

Arch Linux 64 bit installation

2009-11-09T19:03:00.000-08:00

If you love linux, Arch Linux is great. The installer is easy to use (although follow the steps in order or you'll run into trouble), and you can create a very lean system, quite quickly. Dont get me wrong; Ubuntu is also great, and will suit alot of people, but for those that dont have the time for the builds of Gentoo (which I highly recommend for anybody wanting to dig a bit deeper) but want a more customized system, Arch Linux is a fantastic choice.
Below, Ive listed a few hurdles I had getting up and running.

Fixing open office fonts

If you dont the correct fonts installed, you'll see squares instead of characters when using open office. The most common ones are in these packages.


artwiz-fonts ttf-ms-fonts

See The open office wiki for details.

Pidgin

to fix the spell check in Pidgin (assuming your using english) install the aspell english language pack:


aspell-en

MP3, DivX, Xvid

If you want to play mp3's, DivX and Xvid in Banshee (or any other media player) you'll need to install these packages:


gstreamer-bad-plugins
gstreamer-ugly

If you dont, you might see errors like this:


85] nereid Client Started
** Message: don't know how to handle audio/mpeg, mpegversion=(int)1, mpegaudioversion=(int)1, layer=(int)3, rate=(int)44100, channels=(int)2, parsed=(boolean)true
** Message: don't know how to handle audio/mpeg, mpegversion=(int)1, mpegaudioversion=(int)1, layer=(int)3, rate=(int)44100, channels=(int)2, parsed=(boolean)true
[Error 14:09:41.784] GStreamer resource error: NotFound

No Gstreamer Plugins Found Error

If you get the error "no gstreamer plugins found" when trying to play mp3/xvid/divx files, you may need to add your user to the audio group, shown below.


sudo chmod -R a+rwx /dev/snd
usermod --append audio your_username

Display Settings

It took me some time to work out how to get the display to work correctly on my 1400x900 monitor working with the correct resolution. It wasnt until I found out that the default driver "vesa" does NOT support this resolution (!!!) that I was able to solve it. My graphics card is:


00:02.0 VGA compatible controller: Intel Corporation 82Q35 Express Integrated Graphics Controller (rev 02)

You can install the driver required for this using:


pacman -S  xf86-video-intel

Then, in your /etx/X11/xorg.conf, make sure you specify the correct driver and resolution.


Section "Device"
        Identifier  "Card0"
        Driver      "intel"
        VendorName  "All"
        BoardName   "All"
EndSection

Section "Screen"
        Identifier "Screen0"
        Device     "Card0"
        Monitor    "Monitor0"
        DefaultColorDepth 24
        SubSection "Display"
                Depth     24
                Modes "1440x900"
        EndSubSection
EndSection

Simple SVN repository Verification Script

2009-11-09T17:03:00.000-08:00

Ive been using rsync to back up an SVN repository, which has been working great. Unfortunately though, if a problem occurs within the repo, those problems will then be replicated to the backup location. Ive written a small script (below) that will detect if there is a problem reported by the svnadmin verify command. This obviously isnt the complete script; the complete one performs a backup if the verify is successful, and sends an email if not, but those tasks are well documented elsewhere on the net, so I'll leave them out for clarities sake.

I would love to see a more elegant way of doing this, if anybody has one.

#!/bin/bash

# replace /path/to/svnroot to your svnroot

i=`svnadmin verify /path/to/svnroot  2>&1 | grep svnadmin:`

if [[ ${i/ /} == "" ]]

then
        echo "SVN repo is verified.";
else
        echo "SVN repo error!";
fi

Ziproxy - Speed up your internet connection

2009-11-07T16:39:00.000-08:00

If you have a slow Internet connection, or have been capped, using ziproxy can help speed up browsing by compressing all the html that goes through it. Its easy to setup, all you need is a server that has a fast connection, that you can connect to.

Installation

From your server, either get it from your repository, or download it (Its easy to compile from source) from http://ziproxy.sourceforge.net/.

In ubuntu you can use:


apt-get install ziproxy

Source Install

If its not in your repository, you can install it by downloading the source, then installing its dependancies.
In fiesty, they are:


apt-get install libgif-dev
apt-get install libungif4-dev
apt-get install libjpeg62-dev
apt-get install libpng12-dev

If your missing these libries, you'll get one/more of these error messages:


configure: error: Both libgif and libungif not found.
configure: error: libjpeg not found.
configure: error: libpng not found.

Once installed, it will look for the config file (unless otherwise specified on the command line) here;


/etc/ziproxy/ziproxy.conf

The easiest way to configure ziproxy, is to create this file. You can now start ziproxy by using:


ziproxy --daemon

When you start it, it will then print out the process id. Please note that if it cant run (for example, it cant use the default port 8080) it will fail silently.

Configuring

There are alot of options you can play with in ziproxy.conf, mostly to do with the compression of various images, but some interesting ones are:


Port=9900
ImageQuality={-15,20,25,0}
UseContentLength=false
ProcessJS=true
ProcessCSS=true
ProcessHTML=true
ProcessHTML_CSS=true
ProcessHTML_JS=true
ProcessHTML_tags=true
ProcessHTML_text=true

Once thats done, change your proxy settings in your browser, and enjoy the speedy surfing!

pl/sql trigger that limits table size

2009-08-20T20:04:00.000-07:00

Description

The trigger below executes a pl/sql function that will delete the oldest rows in a table. If a row is inserted into the database it will check to see if the number of rows is greater than 10. If it is, it will then delete the last two. You will want to customise this
to your needs. This function is particularly useful when dumping log4j records to a postgres database, as it ensures the table size does not get out of control.

"table" should be replaced with the table name that you wish to limit
"table_limit_trigger" is the name of the trigger.
"limit_table_size()" is the name of the function that performs the delete
Obviously you can replace the maximum size of the table (currently 10), and the number of
records it deletes (currently 2) with whatever you require.

Code

DROP TRIGGER table_limit_trigger on <table>;

CREATE OR REPLACE FUNCTION limit_table_size() RETURNS TRIGGER AS $table_limit_trigger$

DECLARE cnt INTEGER;

BEGIN

 -- count the number of rows
 SELECT INTO cnt COUNT(*) FROM <table>;

 IF cnt > 10 THEN
  --- sql that deletes the oldest rows
  DELETE FROM <table> WHERE id IN (SELECT id FROM <table> ORDER BY id ASC LIMIT 2);

 END IF;

RETURN NEW;
END;
$table_limit_trigger$ LANGUAGE plpgsql;

CREATE TRIGGER table_limit_trigger AFTER INSERT ON <table> FOR EACH ROW EXECUTE PROCEDURE limit_table_size();

SVN tools - All Distributions

2009-08-15T05:26:00.000-07:00

Revert

To revert a folder on your working copy back to a revision within SVN, the following syntax can be used:


svn copy -r 2450 svn+ssh://username@192.168.1.3/svnroot/project/trunk/dir@2450 ./dir

This command will revert the specified directory to revision 2450.

Remove svn directories

The command below will remove all svn directories within a directory tree.


find . -name .svn -exec rm -fdr {} \;

SVN update cron

The command below is a crontab line that will automatically update your svn repository every day at 4am. Make sure your ssh key is setup properly, so that svn doesn't prompt for a password.


1 4     * * *   root   svn up --accept postpone /home/user/workspace/svnroot

Adding directories to repository

The following command will add all folders from a directory, including subdirectories (even if that directory is already in svn) to the working copy.


svn add --force ./dir

Trac project updater script

2009-08-05T23:08:00.000-07:00

Updates and upgrades all trac projects in a given directory.

# Updates from trac svn, then upgrades all trac projects in given directory.
# Finally, restarts apache.

#!/bin/bash
# files
TRAC_INSTALL=/root/installs/trac_install/trac
TRAC_PROJECTS=/usr/trac/trac_projects
FILES=`ls -l | grep $1^d | gawk '{ print $8 }'`

APACHE_RESTART_CMD="/etc/init.d/apache2 restart"

echo "updating repository"

svn up $TRAC_INSTALL

# re-install

cd $TRAC_INSTALL
./setup.py install
cd -

echo "update the projects/wiki"

cd $TRAC_PROJECTS

for f in $FILES
do
  echo "Processing project: $f"
  trac-admin $f upgrade
  trac-admin $f wiki upgrade

done

echo "restarting apache"

$APACHE_RESTART

DropBox - Online Storage

2009-07-31T15:48:00.000-07:00

If your looking for some FREE online storage, and for a quick way to share files between computers, look no further than DropBox! The app supplied works flawlessly in BOTH Windows AND Linux (Ubuntu), and is very useful for storing...well pretty much anything! You get 2 gigs to use absolutely FREE, and can pay for more if required. Plus, you can view and manage all your files via the web, AND share folders with friends.

Get DropBox Here!

Flash Sound Not Working On Jaunty 9.04

2009-07-25T18:07:00.000-07:00

If your sound isnt working with flash in Firefox (i.e youtube) on Ubuntu Jaunty (9.04), try installing this package:


sudo apt-get install  flashplugin-nonfree-extrasound

Then restart your browser.

Port Fowarding via SSH

2009-07-14T21:15:00.000-07:00

Port forwarding through SSH is really useful, for many reasons such as wrapping an insecure protocol with ssh, or if you need to bypass a restrictive server and have ssh access to a server within a required network.
I think learning this is best by example, so I'll explain one below.


ssh -L local_port:end_server:end_port user@mid_point

local_port is the port that ssh will listen to on your local machine.
end_server is the server at the end of the tunnel.
end_port is the servers port at the end of the tunnel.
user@mid_point is the ssh server in between you and the server.

For example, if you want to tunnel terminal services, you could use this command


ssh -L 3389:192.168.1.99:3389 myuser@192.168.1.55

The first 3389 is the port on your local machine that ssh will listen on.

192.168.1.99 is the ip address of the remote server that's running terminal services that you wish to connect to.

The second 3389 is the port on the remote machine (in this case, the terminal services port).

myuser@192.168.1.55 is the login details for the machine on the same network as the terminal services server.

Running that command will then prompt for "myuser"'s password. Entering it in will open the port 3389 on your machine. To connect to the remote machine, point your terminal services client (in Ubuntu thats tsclient) to localhost, and put in the username and password that you usually use to log in to terminal service.

Simple Linux and Windows File Encryption

2009-07-12T04:08:00.000-07:00

If your after a quick and easy way to encrypt files with a simple password (and cant be bothered with keys etc) in Linux AND Windows check out gpg. Gpg supports "symmetric", which only requires a pass phrase to encrypt & decrypt. To encrypt in Linux, just use this command:


gpg --symmetric

If your lucky enough to be using Ubuntu (Im sure all the other major distro's will support this too), you can right click on a *.gpg, and it will prompt for the pass-phrase.

To encrypt/decrypt in windows you need to install the GPG package. It can be found at the site below:

http://www.gnupg.org/

Please don't use this for highly sensitive data; the encryption is only as safe as the password you choose, and there are more secure ways of encrypting data. Check out all the features of gpg for details.

Manually Mounting A Partition From An Image

2009-06-29T22:36:00.000-07:00

Using dd, you can create a dump of a harddrive, directly to a disk. This is useful if you want to make a backup of a compact flash. For example


dd in=/dev/sdb0 of=myfile.dmp

Will write the entire contents of the sdb0 device, into a file called myfile.dmp. You can then use dd to write that file back to another flashcard.

Mounting image to file system

If you want to see the contents of an image, you need to use mount with -o (loop back) specified. The tricky part is working out the offset, which is the byte that the partition you wish to load starts at.


Disk myfile.dmp: 0 MB, 0 bytes
255 heads, 63 sectors/track, 0 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x0003ba1c

Device Boot           Start         End      Blocks    Id  System
myfile.dmp1   *           1         182      1461883+  83  Linux
myfile.dmp2             183         249      538177+   82  Linux swap / Solaris

To mount the first partition all you need to do is specify an offset of 512 (bytes) * 63 (sectors) = 32256B, which is where the first partition starts. The full command is:


mount -o loop,ro,offset=32256 myfile.dmp

The formula for calculating partitions after the first one is is (unit - 1) * unit size.

And to mount the second partition it would be 182 * 8225280 = 1497000960 (In this example you cant mount it, as its swap space). To mount it you would use


mount -o loop,ro,offset=1497000960 myfile.dmp

You can also use parted to do the calculations for you. Here is a very useful link:

Mounting Linux Images

KnowledgeTree Manual Install

2009-05-10T02:40:00.000-07:00

This tutorial describes how to install the open source version of knowledge tree (www.knowledgetree.com/ ) manually on Ubuntu (8.10, but any recent version should be similar). This is required when you have an existing install of apache and/or mysql, and you wish to use the same installs for KT as the installer binary cannot handle using existing versions of these services.
Remember, whenever I have quoted a directory its is relative to wherever you have uncompressed the source.

Installation

Firstly download the installation files (http://www.knowledgetree.com/community-download)and, place in /var/www. (Not the .bin installer file; you need the source files; it will have a name like: ktdms-src-oss-3.6.0.tgz)

Chown and chgrp all files to www-data (or whatever user is used to run scripts by apache). For example


chown www-data -R /var/www/kt-dms-oss
chgrp www-data -R /var/www/kt-dms-oss

Update the file with /etc/apache2/sites-enabled/000-sites-enabled, with the site details, as shown below.



        AddType text/html .php .phps
        AddHandler application/x-httpd-php .php
        AddHandler application/x-httpd-php-source .phps
        Options Indexes FollowSymLinks MultiViews
        AllowOverride None
        Order allow,deny
        allow from all

Next, install apaches php5 and php-mysql modules, so that KT can execute its php scripts.


apt-get install libapache2-mod-php5 php5-mysql

Note, if you do not have the php5 module installed, it will attempt to download the php files when you browse to them. After you install the modules you will need to force-reload apache, AND clear the cache of your browser.

If you do not have php-mysql installed, you will get this error:


[db_error: message="DB Error: extension not found" code=-25 mode=return 
level=notice prefix="" info="Array"]

I needed to manually set the document root, otherwise KT would redirect to the wrong location.
The settings for setting document root in /config/config.ini


rootUrl = /kt-dms-oss

Next, you need to update the database details in the same config.ini file. Set it up with a valid msql user.


; Database login details
dbHost           = localhost
dbName           = dms
dbUser           = root
dbPass           =
dbPort           = default

dbAdminUser      = 
dbAdminPass      =

If you do not setup the database correctly you will get this error.


[db_error: message="DB Error: connect failed" code=-24 mode=return level=notice prefix="" 
info=" [nativecode=Access denied for user 'root'@'localhost' (using password: YES)] ** Array"]

The problem is that KT cannot log in with username and password. You need to set these settings in config.ini with a valid user.

Next, create the database

log into mysql, and execute


create database dms;

Logout of mysql.

Next, we need to init the database structure and content. The scripts are located here:

sql/mysql/install

Run these commands, (in sql/mysql/install)


mysql -uroot dms < structure.sql
mysql -uroot dms < data.sql

If you do not have the database installed correctly you will receive this error:


[db_error: message="DB Error: no such database" code=-27 mode=return 
level=notice prefix="" info=" [nativecode=Unknown database 'dms'] ** Array"]

Or,


Fatal error: Cannot use object of type DB_Error as array in /var/www/kt-dms-oss/lib/session/Session.inc on line 253

Now you must upgrade the database.

run setup/upgrade.php. You may get the following error:


-bash: ./upgrade.sh: /bin/sh^M: bad interpreter: No such file or directory

If you do, run:


perl -i -pe's/\r$//;' upgrade.php

If the upgrade has not been performed, you may get the error "Session timed out" when trying to log in.

Now, to check the status of the install run:

setup/precheckup.php

Restart apache.

You should now be able to login to Knowledge via login.php, using username "admin", password "admin".

Memory Issues

I had to increase the amount of memory php was allowed to use. Errors like these would come up:


Fatal error: Allowed memory size of 16777216 bytes exhausted 
(tried to allocate 19456 bytes) in /var/www/kt-dms-oss/plugins/ktstandard/workflow/FolderAssociator.php on line 174

To fix this, increase the memory limit in /etc/php5/apache2/php.ini


memory_limit = 32M      ; Maximum amount of memory a script may consume (16MB)

restart apache.


/etc/init.d/apache2 restart

Bulk Importing

Be careful doing bulk file uploads on the server. Make sure:

1. The folders don't contain any files bigger than 2M (you can increase this limit in php.ini).
2. KT has access to all the folders and files within the directory you choose to upload.
3. You have some spare time; the process can take a while!

The scheduler

The scheduler takes care of managing and running tasks or batch jobs at regular intervals in the background. If its running correctly, you will see useful server stats when logged in as an administrator. To execute php scripts on the server you need to install php5-cli package.


sudo apt-get install php5-cli

Then, add the following line to your /etc/crontab


# execute knowledgetree scheduler to execute at 3am every night
0 3 * * * root (php /var/www/kt-dms-oss/bin/scheduler.php > /dev/null)

Document Indexer

If the indexer isnt setup correctly, you will get this error when searching:


The Document Indexer did not respond correctly. Your search results will not include content results. Please notify the system administrator to investigate why the Document Indexer is not running.

Looking under the administration tab, you will likely find errors like this.


    * Message: Cannot connect to Open Office Server on host '127.0.0.1:8100'.
    * This could be due to OpenOffice.org not having been started, OR not installed
    * For more information, please see: http://wiki.knowledgetree.com/Troubleshooting_OpenOffice.org_Service

To fix this, install open office as explained below

Installing Open Office

In Ubuntu, there's a simple way to solve this.


apt-get install java-6-sun-jre
update-alternatives --config java # Select the appropriate option
apt-get install openoffice.org-headless openoffice.org-java-common
sudo soffice -headless -accept="socket,host=localhost,port=8100;urp;" -nofirststartwizard&

Note, if you get this error if you dont have openoffice.org-java-common installed:


javaldx: Could not find a Java Runtime Environment!

Now, as per the documention, running this command


netstat -an | grep 8100

will show the open office server running on that port.

Installation on Ubuntu Feisty

Exporting to PDF


An error occurred generating the PDF - please contact the system administrator. PyODConverter: Cannot find the pyuno.so library in sys.path and known paths.

To fix it:


sudo apt-get install python-uno # will install all the required libraries (note that openoffice-server doesnt exist in this version)

Now you need to start the server and setup a fake display (if your running this on a headless server anyway), otherwise you'll get this error on the client:


An error occurred generating the PDF - please contact the system administrator. OpenOffice process not found or not listening (Connector : couldn't connect to socket (Success))

And this error when you try start the open office server:


sudo soffice -headless -accept="socket,host=localhost,port=8100;urp;" -nofirststartwizard&

/usr/lib/openoffice/program/soffice.bin X11 error: Can't open display: 
   Set DISPLAY environment variable, use -display option
   or check permissions of your X-Server
   (See "man X" resp. "man xhost" for details)

To setup a fake display, run this:


apt-get install xvfb # install required package

# start display
unset XAUTHORITY
DISPLAY=:1000
export DISPLAY
killall Xvfb
Xvfb $DISPLAY &

Now you should be able to start the server using this command:


sudo soffice -headless -accept="socket,host=localhost,port=8100;urp;" -nofirststartwizard&

The next step is to install the required open office packages that will export the required documents. If you dont, you'll get an error message like this:


An error occurred generating the PDF - please contact the system administrator. Given parameters don't conform to the specification ( URL seems to be an unsupported one.)

Openoffice needs the required packages to process the request. i.e an odt. doc needs openoffice.org-writer installed.
To get all of them quickly, just install the whole openoffice.org package.


apt-get install openoffice.org

you will need to restart the soffice daemon for the changes to take effect.

Indexer

Once the openoffice server is running you need to start KT_LOCATION/bin/luceneserver/ktlucene.jar
Starting this, and the open office daemon is shown below.

OpenOffice and Indexer Startup Script


#!/bin/bash

KT_LOCATION=/var/www/knowledgetree

# start indexer

cd $KT_LOCATION/bin/luceneserver/
java -jar ktlucene.jar&

# start fake screen

set DISPLAY=:1000
export DISPLAY
Xvfb $DISPLAY &

# start open office server

soffice -accept="socket,host=localhost,port=8100;urp" -nofirststartwizard&

cd  -

Be careful when you run this command; the indexer may not be configured correctly. You may get this error:


2    [main] INFO  com.knowledgetree  - Loading properties file: KnowledgeTreeIndexer.properties
423  [main] INFO  com.knowledgetree  - Server in paranoid mode!
443  [main] INFO  com.knowledgetree  -  Accepting connections from: 127.0.0.1
444  [main] INFO  com.knowledgetree  -  Accepting connections from: 192.168.1.1
555  [main] INFO  com.knowledgetree.lucene  - Indexer starting up...
583  [main] INFO  com.knowledgetree.lucene  - Loading properties file: KnowledgeTreeIndexer.properties
583  [main] INFO  com.knowledgetree.lucene  - Using index directory: ../../../var/indexes
Exception in thread "main" java.lang.Exception: Invalid index directory specified: ../../../var/indexes
 at com.knowledgetree.indexer.IndexerManager.(IndexerManager.java:160)
 at com.knowledgetree.indexer.IndexerManager.get(IndexerManager.java:92)
 at com.knowledgetree.lucene.KTLuceneServer.main(KTLuceneServer.java:71)

If you do, edit KnowledgeTreeIndexer.properties (in bin/luceneserver), and change the indexer.directory line to:


indexer.directory=../../var/indexes

Streaming database to remote machine

2009-02-28T03:17:00.000-08:00

If you need to backup a remote database but dont have room on your remote machine to make a database dump file (of which you could use scp to transfer), you can use ssh to stream the dump as its created.

Executing this command on your local machine will run postgres's dump command on the remote machine (in this case 192.168.1.2) and pipe it back through the ssh pipe, into a file on your local machine called database_dump_file.


ssh databaseuser@192.168.1.2 'pg_dump your_database' > database_dump_file

If this is over the internet, you may wish to use ssh's -C flag, to compress the stream.

Glorious Tinc - Ubuntu

2008-10-11T20:50:00.000-07:00

Tinc (http://www.tinc-vpn.org/) is an awesome VPN daemon. Its secure, its fast and its lightweight.

I am going to explain how to setup a simple network where each machine is on the same subnet, and connects to and routes through a central computer, which behaves like a switch. This is useful when you need to create a network where some of the computers behind a firewall. With this configuration, only the central computer on the network needs to allow incoming connections.
For simplicity I will call the computer that must allow incoming connections and acts like a switch "central", and go through configuring it and one "leaf" pc that connects to it. Setting up additional "leaf" pcs, is just the same, except you will need to give them each a unique name and IP address.

You will need to do this on the central computer:

1) install tinc (apt-get install tinc on Ubuntu)
2) install brctl (apt-get install bridge-utils)
3) install tun (this is a kernel module; compiled in by default with Ubuntu)
4) setup tinc's configuration

Leaf configuration

Tinc's configuration is in /etc/tinc

Firstly, you need to create a directory for your vpn configuration. Call it what you like, but in this case Im going to call it "vpn".


mkdir /etc/tinc/vpn

Next, edit nets.boot, and add vpn at the bottom of it, like so:

/etc/tinc/net.boot


## This file contains all names of the networks to be started on system startup.
vpn

From now on, all configuration is done from within the "vpn" folder.
The tinc-up file is executed when the vpn is brought "up". It must be executable by root (as root, chmod u+x tinc-up) See below for the example of a nodes tinc-up file. Note that 192.168.7.22 is the address that our node will be given on the vpn.

/etc/tinc/vpn/tinc-up


#!/bin/sh
modprobe tun
ifconfig vpn 0.0.0.0
ifconfig vpn up

brctl addbr bridge
brctl addif bridge vpn
ifconfig bridge 192.168.7.22 netmask 255.255.255.0 #this IP address is our leafs VPN address
ifconfig bridge up

Next, we need to configure the tinc.conf file for the vpn.
Name is the name of the local node. Connects to is the node that our leaf node connects to (in this case the central node). Switch mode means that any packets received by the bridge will be passed through to our vpn and tinc keeps and ARP cache to determine if any packets we wish to send should be sent out our vpn interface.

/etc/tinc/vpn/tinc.conf


Name = leafPC
ConnectTo = central
Device = /dev/net/tun
Mode = switch
PrivateKeyFile = /etc/tinc/vpn/rsa_key.priv

Dont worry about the private key reference; we'll generate that shortly. Now, we need to setup the hosts we will communicate on the vpn. Seeing as we only connect to the central node, we only need to define this leaf and the central node. Create a directory in vpn called "hosts". Now create a file called "central". This is the details of the central computer we will communicate through. Note that the address field is the ACTUAL IP of the central PC, not its VPN address. Subnet is the subnet that the node is responsible for, in this case its our entire VPN.

# /etc/tinc/vpn/hosts/central


Address = 192.168.1.44   # actual IP address of central
Subnet = 192.168.7.0/24   # vlan subnet
TCPOnly = yes

We also need to put our leafs detail in as well. The address field is the leafs PUBLIC IP address

# /etc/tinc/vpn/hosts/leafPC
Address = 192.168.1.22 # public ip address
TCPOnly = yes

Generating the keys

The next thing to do is to generate a private key for the node. This private key needs to be copied into every node that this one will connect to; in our case, the central. Do this by entering this command:

tincd -n vpn -K

The command should then output something like this:


Please enter a file to save private RSA key to [/etc/tinc/vpn/rsa_key.priv]: 
Please enter a file to save public RSA key to [/etc/tinc/vpn/hosts/leaf]:

Your leaf node configuration is nearly complete. The only thing left to do is to copy the leafs public key to the central computer, and visa versa so that the connection can be authenticated.

Central configuration

Configuring the central is similar to setting up a leaf node.

1) add VPN to nets.boot
2) create directory called vpn
3) create executable tinc-up file in this folder, with the following contents:


#!/bin/sh
modprobe tun
ifconfig $INTERFACE 192.168.7.2 netmask 255.255.255.0

4) create tinc.conf file with the following contents:

Name = central
Device = /dev/net/tun
Mode = switch
PrivateKeyFile = /etc/tinc/vpn/rsa_key.priv

5) create hosts folder

6) create a file called central which should contain this:

Address = 192.168.1.44 # PUBLIC IP address of central
Subnet = 192.168.7.0/24 # Subnet of vpn
TCPOnly = yes

8) rerun tincd -n vpn -K to general your public and private keys.

Setting up public / private keys

The final step is to copy the PUBLIC keys and all of the settings of each node to each other. For example, you must copy the host file ./hosts/central to ./hosts/central on your leaf machine, and ./hosts/leaf on your leaf machine to ./hosts/leaf. Each of theses should both contain the RSA PUBLIC key at the bottom of the file.

You should now be able to start tinc using


/etc/init.d/tinc start

If everything is going to plan, you should now have an interface called bridge with the IP address you specified, and should be able to ping the vlan central ip address!

# troubleshooting

To run tinc in Debug Mode:


tincd -n vpn -d5 -D

common errors

the host entry on the central and leaf need to be EXACTLY the same. If your getting error messages like "bogus" data received or the connection being closed repeatedly, copy and paste the entry from the leaf into the appropriate server file, and restart the tinc on the server.


tincd 1.0.8 (Nov  2 2007 17:40:21) starting, debug level 5
Invalid name for myself!
Terminating

Name cannot contain some characters, such as "-".

Merging SVN repositories - Linux

2008-08-14T20:26:00.000-07:00

Merging SVN repositories

Initially, in my finite wisdom, I thought that putting each project into a separate SVN repository was a good idea. It turns out get a little annoying, mostly because every time we need to create a new project, I have to create a new repository on the server. It will also get more annoying down the track, when we want to setup mirrors of the repository. Plus, there really no point in having multiple repositories, so I decided to merge them.

Firstly, because each of the repositories will have different UUID's (SVN's repository identifier), you will need to make sure you check in ALL changes of the repos to be merged before doing the migration because you cannot switch your existing working copies to commit to the new repo. (See below if you accidentally do this)

Once thats done, create a dump file of your old repo:


svnadmin dump myRepo > dumpfile.dump

Next step is to load it into you new repository. Create a new folder in your new repository (if required; I needed to because I had just put my trunk, branches and tags folder into the root folder). and commit it. Now all you need to do is load the old repo into your new repo.


svnadmin load myNewRepo  --parent-dir projectRoot < dumpfile.dump

Ive been making changes on the wrong working copy; what do I do?

Easy; just use svn diff on your old working copy, then patch your new working copy.


svn diff > oldrep.diff

Then from the same directory in the new working copy:


patch -p0 < oldrepo.diff

If you missed adding any files you can add them using this:


svn status | grep "^\?" | awk '{print $2}' | xargs svn add

Note that this handy snip-it this is not mine; I got it from this web site: http://snipplr.com/view/5745/svn-add-recursively/

PhpPgAdmin - Ubuntu

2008-08-13T23:52:00.000-07:00

Phppgadmin is an great tool for browsing PostgreSQL databases via a web interface. Installation is easy:


apt-get install phppgadmin

In /etc/phpmyadmin/apache.conf copy paste the contents into apaches configuration..i.e


/etc/apache2/sites-enabled# nano 000-default

Now, you'll need to setup a non-root user to login. If you dont, using a root user will result in this error:


Login disallowed for security reasons.

To do so, edit the phppgadmin config file here:


/etc/phppgadmin/config.inc.php

Modify the line, to allow access for your root user ( postgres )


$conf['extra_login_security'] = false;

To create a user, login to phppgadmin using postgres, click on users create a user with desired privileges and logout.

Finally, edit config.inc.php and reset extra_login_security to true. You can now login with your newly created user.

Dar Backup Script

2008-08-10T23:24:00.000-07:00

The below script allows you to backup selective directories using dar. It works by creating a custom dar command, executing the command and burning each dar file to CD / DVD. It allows you to selectively ignore directories that contain a file (by default it looks for a file called .nobackup), which is helpful if you have some large files in selective directories that you dont wish to backup.

#!/usr/bin/python
import os

import io
import string

#######################Python Backup Script#########################################
##
##This script allows you to backup selective directories using dar. 
##It has a couple of handy features:

##
##1) It will ignore any directory containing a file called ".nobackup".(configurable) 
##2) Any directories put into the ignore list will be ignored recursively.
##3) It will write these to DVD / CD, and prompt to enter next disk.
##4) It dar files once burnt to disk.
##
##Source: http://programminglinuxblog.blogspot.com/ 
##License: GPLv3. See http://www.gnu.org/licenses/gpl.html 
####################################################################################

# All directories off ROOT_PATH are by included by default unless they are put in the
# exclude or ignore lists.

# IGNORE_FILENAME if a file with this file name is found dar will not backup anything 
# in the directory its contained in. It is NOT recursive, thus subdirectories will be 
# considered for backup.

IGNORE_FILENAME=".nobackup"

# Saves the executed dar command in this file:

COMMAND_FILE = "command"

# Saves the output of the dar command in this file

COMMAND_OUTPUT = "output";

# BACK_SAVE_LOC is where the dar file will be saved. WARNING: this directory should 
# be an ignored directory or in a sub directory of one of these directories.

BACK_SAVE_LOC="/root/backups/"

# the name of the dar file(s) that will be created

BACKUP_FILENAME="full"

# The root directory to backup from

ROOT_PATH = "/"

# exclude is a list of directories to exclude from backup. 
# WARNING: these directories must be RELATIVE to ROOT_PATH.

exclude = []

# NOTE: all subdirectories of these directories are ignored (i.e a recursive ignore)

# Example Ignore:
# ignore = ["bin", "dev","initrd","lib","media","proc","tmp", "vmlinuz","boot","etc", 
#            "mnt","root","srv","cdrom","lost+found","opt","sbin","sys","var"]

ignore.append(BACK_SAVE_LOC)      # ( WARNING: dont modify )

stack = []   # stack for directories to include   ( WARNING: dont modify )

# recursive method that sets up stack and exclude lists with the appropriate directories.
def get_backup_directories(rootDir):

    for name in os.listdir(rootDir):
        path = os.path.join(rootDir, name)

        if os.path.isdir(path):  # if current file is a dir

            ignoreDir = False

            for ig in ignore:  # check if its in ignore list
                if path.startswith(ROOT_PATH + ig):

                    ignoreDir = True
                    break

            if not ignoreDir:

                stack.append(path)
                get_backup_directories(path)
            else:

               exclude.append(path)
 else:
     shortpath = os.path.basename(path)

  
     if shortpath == IGNORE_FILENAME: # remove directory if ignore file present
                stack.pop()

                
  exclude.append(path.replace(IGNORE_FILENAME,"")) # remove filename 
    

get_backup_directories(ROOT_PATH)

# excplicitly included directories also includes sub-directories; this removes any sub-directories
# of already included directories

print "normalising include stack"

i = len(stack)

while i != 0:
    i = i - 1

    
    x = len(stack)
    while x != 0:

        x = x - 1
        if stack[i].startswith(stack[x]) and not stack[i] == stack[x]:

            del stack[i]
            break

for i in stack: 
    print "normalised: " + i

# now to make all exclude paths realitive

print "relativising exclude paths"

i = len(exclude)

while i != 0:
    i = i - 1

    curr = exclude[i]
    curr = curr.replace(ROOT_PATH,"",1)

    curr = curr.replace(" ","\ ")       # delmit space characters
    exclude[i] = curr

# same for include paths

print "relativising include paths"

i = len(stack)

while i != 0:
    i = i - 1

    curr = stack[i]    
    curr = curr.replace(ROOT_PATH,"",1) 
    curr = curr.replace(" ","\ ")  # delmit space characters

    stack[i] = curr

# NOTE: -v verbose,-c filename, -s size, -z gzip  compression, set root folder, ignore All files
baseCmd = "dar -v -c "+ BACK_SAVE_LOC + BACKUP_FILENAME + " -s 4000M -z -R " + ROOT_PATH +" -P . "

# include directories
for i in stack:
    baseCmd += "-g " + i + " "

# exclude directories
for i in exclude:
   baseCmd += "-P " + i + " " 


print "executing command: " + baseCmd

os.system("echo "  + baseCmd + " > " + COMMAND_FILE) # save to file

output = os.system(baseCmd + " > " + COMMAND_OUTPUT)    # also save to file

# write each file to cd

files = os.listdir(".")

# remove all non-dar files

i = len(files)
while i != 0:

        i = i - 1
        if not files[i].endswith(".dar"):

                del files[i]
 
print "you will need " + str(len(files)) + " disks. Starting burning process..."

for curr in files:
 x = raw_input('Please insert a CD/DVD. Press any key to continue.')

 output = os.system("growisofs -Z /dev/dvd -R -J " + BACK_SAVE_LOC + curr) 
 os.system("eject")

# cleanup

os.system("rm *.dar")

Common Trac issues

2008-08-07T22:04:00.000-07:00

Below I have listed some common trac errors, and how I have solved them.

Error


Warning: Can't synchronize with the repository 
(Unsupported version control system "svn": "No module named svn" ).
Look in the Trac log for more information.

Solution

you need to install python subversion bindings. In Ubuntu, thats the python-subversion package. (you'll need to restart apache after this)

Error


SubversionException: ("Expected format '3' of repository; found format '5'", 165005)

This means that you have different SVN version on your client and server.
You can check it via svn --version. For example:


svn --version

version 1.4.0 (r21228)
   compiled Sep 18 2006, 15:25:13

Solution

Currently, Subversion 1.5 is not supported in any Ubuntu distribution other than Intrepid, which hasn't been released yet. Thus, you must find a backport, or install it manually. There is a backport for Hardy, available here:

https://launchpad.net/~clazzes.org/+archive

If you add this to your sources.list file, it will become available:


deb http://ppa.launchpad.net/clazzes.org/ubuntu hardy main
deb-src http://ppa.launchpad.net/clazzes.org/ubuntu hardy main

Make sure you remove your old subversion, and python-subversion bindings. For some reason, I had to manually remove the old svn binaries (i.e svnadmin etc) out of /usr/local/bin and copy the new ones into there. The new ones were located in /usr/bin

Error


Traceback (most recent call last):
  File "/usr/lib/python2.5/site-packages/Trac-0.12dev_r7454-py2.5.egg/trac/web/api.py", line 344, in send_error
    data, 'text/html')
  File "/usr/lib/python2.5/site-packages/Trac-0.12dev_r7454-py2.5.egg/trac/web/chrome.py", line 697, in render_template
    data = self.populate_data(req, data)
  File "/usr/lib/python2.5/site-packages/Trac-0.12dev_r7454-py2.5.egg/trac/web/chrome.py", line 597, in populate_data
    d['chrome'].update(req.chrome)
  File "/usr/lib/python2.5/site-packages/Trac-0.12dev_r7454-py2.5.egg/trac/web/api.py", line 170, in __getattr__
    value = self.callbacks[name](self)
  File "/usr/lib/python2.5/site-packages/Trac-0.12dev_r7454-py2.5.egg/trac/web/chrome.py", line 466, in prepare_request
    for category, name, text in contributor.get_navigation_items(req):
  File "/usr/lib/python2.5/site-packages/Trac-0.12dev_r7454-py2.5.egg/trac/ticket/web_ui.py", line 151, in get_navigation_items
    if 'TICKET_CREATE' in req.perm:
  File "/usr/lib/python2.5/site-packages/Trac-0.12dev_r7454-py2.5.egg/trac/web/api.py", line 170, in __getattr__
    value = self.callbacks[name](self)
  File "/usr/lib/python2.5/site-packages/Trac-0.12dev_r7454-py2.5.egg/trac/web/main.py", line 267, in _get_perm
    return PermissionCache(self.env, self.authenticate(req))
  File "/usr/lib/python2.5/site-packages/Trac-0.12dev_r7454-py2.5.egg/trac/web/main.py", line 135, in authenticate
    authname = authenticator.authenticate(req)
  File "/usr/lib/python2.5/site-packages/Trac-0.12dev_r7454-py2.5.egg/trac/web/auth.py", line 71, in authenticate
    authname = self._get_name_for_cookie(req, req.incookie['trac_auth'])
  File "/usr/lib/python2.5/site-packages/Trac-0.12dev_r7454-py2.5.egg/trac/web/auth.py", line 187, in _get_name_for_cookie
    (cookie.value, req.remote_addr))
  File "/usr/lib/python2.5/site-packages/Trac-0.12dev_r7454-py2.5.egg/trac/db/util.py", line 50, in execute
    return self.cursor.execute(sql_escape_percent(sql), args)
  File "/usr/lib/python2.5/site-packages/Trac-0.12dev_r7454-py2.5.egg/trac/db/sqlite_backend.py", line 59, in execute
    args or [])
  File "/usr/lib/python2.5/site-packages/Trac-0.12dev_r7454-py2.5.egg/trac/db/sqlite_backend.py", line 51, in _rollback_on_error
    return function(self, *args, **kwargs)

DatabaseError: file is encrypted or is not a database

Solution

You need to update your database. Go into $TRAC_PROJECT_DIR/db/

and type:


mv trac.db trac2.db  && sqlite trac2.db .dump | sqlite3 trac.db

Running Tomcat With Apache - Ubuntu 7.10

2008-06-15T20:49:00.001-07:00

This blog will explain how to run Tomcat behind Apache so both can be used to serve content. This is useful for when you need to run servlet/jsp projects and script based content such as PHP. Its pretty simple; all the configuration is done in apache. I will present two ways; one using mod-jk and one using mod-proxy. Mod-Proxy is slightly more flexible, as it allows you to redirect from any context to any other context, On the other hand, I think mod-proxy would probably be slower as it is forwarding the requests not delegating. That is a guess though; so if you are worried about performance you best do some more research.

Firstly, you need to install apache and tomcat. Not going to explain that, except that you need to use apache2 for this tutorial. Once these are installed, you need to configure apache to run on the port you wish for all the content to be listed on; most likely port 80 (in ubuntu this is default).You can change this in /etc/apache2/ports.conf.

mod-jk

Install mod-jk; the apache 2 connector. This is the package that connects apache with tomcat.


apt-get install libapache2-mod-jk

Once this is installed, you need to configure it. Add these lines to /etc/apache2/apache2.conf


# Worker properties file
JkWorkersFile /etc/apache2/workers.properties

# Logging
JkLogFile /var/log/apache2/mod_jk.log

# Set the jk log level [debug/error/info]
JkLogLevel info

# Log format
JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "

Next, you need to modify workers.properties file, so that it knows where to find your tomcat install. You need to point your worker to your tomcat and java directories, and specify the port (8080 in this case) that it is running on.


workers.tomcat_home=/usr/local/tomcat/tomcat
workers.java_home=/usr/lib/jvm/java-6-sun
ps=/
worker.list=worker1

worker.default.port=8080
worker.default.host=localhost
worker.default.type=ajp13
worker.default.lbfactor=1

Now, go back into /etc/apache2/apache2.conf, and add in the tomcat context's you wish to share. Unfortunately (as far as I know) you cant translate to different contexts (i.e your TestContext in apache must translate to TestContext in tomcat. If you wish to translate it, you'll need to do use apaches (awesome) module mod_rewrite.)



# Send servlet for context / TestContext to worker named worker1

JkMount /TestContext worker1
JkMount /TestContext/* worker1

JkMount /AnotherContext worker1
JkMount /AnotherContext/* worker1

Restart apache (/etc/init.d/apache2 restart). Now, any requests sent to apache (i.e http://myserver/TestContext), will be forwarded through to tomcat.

mod-proxy

You can use mod-proxy to achieve the same goal. The advantages I can see with using mod-proxy, is that you can transparently redirect a URL to any other URL. This is useful if you want to deploy different versions of the same application, but want the root context to redirect to only one, transparently.

First you need to load proxy and http proxy. In /etc/apache2/mods-enabled you need to create a link back to mods-available to each module.


ln -s ../mods-available/proxy.load proxy.load
ln -s ../mods-available/proxy.conf proxy.conf
ln -s ../mods-available/proxy_http.load proxy_http.load

Now, edit /etc/apache2/httpd.conf.


ProxyTimeout 1
ProxyRequests Off
ProxyPreserveHost On
ProxyPass / http://localhost:8080/TestContext/
ProxyPassReverse / http://localhost:8080/TestContext/


        Order allow,deny
        Allow from all

The configuration above will allow any hosts to be redirected from the root context,to MyApp. The proxy element allows you to configure permissions like a normal apache context. Restart apache, and everything should work.

SVN On Commit MSN Notification

2008-05-02T05:35:00.000-07:00

SVN Commit MSN Notification

When working on a project with other developers it can be useful to be notified when the repository is committed to. Although its fairly easy to setup an email to be sent on commit, the update emails often get lost in your inbox, and generally your only really interested in what is happening now anyway(or at least I was), so end up ignoring most of them.

I have created a script based MSN client which, when combined with a SVN post-commit hook, will send an MSN message when the SVN repository is committed to.

Download

I have now a sourceforge project for this application; you can get it here

Setting up the client

To set up the MSN client you need a notifier email account. This is the MSN email address the MSN client will use to send messages to you and your team. Once you have created it, log into it using pidgin or another MSN messenger, and add each of your team's email address into your contact list. Make sure they accept your invitation.

Once this is done, you need to create a settings file. If you run the MSN client without a ini file present a template one will be created for you. It needs to be called "ScriptMSN.ini", be located in the same folder as the jar file, and look something like this:


[broadcastGroup]
contacts=first_email@hotmail.com,second_email@hotmail.com

[login]
username=notifier_account@hotmail.com
state=away
password=notifier_password

The broadcast group should contain every contact that needs to be notified of commits. The login section contains the credentials for the notifier account.

Setting up post-commit hook

Firstly; I'm not sure HOW svn decides where it will run things from; "/" seems to reference your home directory. Below is an example of a simple post-commit hook:

/projects/svn/my_svn_repo/hooks/post-commit

#!/bin/sh
REPOS="$1"
REV="$2"
java -jar `pwd`/ScriptMSN.jar -m revision:$REV repo:$REPOS details:`svnlook info /projects/svn/my_svn_repo ` > output

This is broadcast to each user:


(13:17:50) Notifier: revision:165 repo:/projects/svn/my_svn_repo details:server 2008-05-02 13:20:41 +1000 (Fri, 02 May 2008) 11 IMPL: added a big fancy gold button, just for fun!

Connection Pooling with Java - All

2008-03-31T19:12:00.000-07:00

Establishing a connection from your application to a database can be a time consuming task. In a standalone application, this is probably acceptable; the connection is usually made once when started, maintained throughout the lifetime of the app, which may be a period of hours or longer, then closed once the application is finished.
In web-based applications it is a different story. Due to the way the HTTP protocol works, a connection from a clients browser to our server is not maintained throughout a clients session. As servlet programmers, we have no idea when the next request will be made, if indeed another will be made at all. So, to make sure we dont use up all our allowed database connections we must open and close database at the start and end of each request. This might be fine if your running your app off your local machine and connections can be made and closed quickly, but if your application is hosted elsewhere, it may slow the time it takes to process requests significantly. Another (not so good) option is not to close connections at all, and let them time out. Again, this is workable if the frequency of requests is low or the timeout on the database connections is short, but as more users start making requests open connections will start to build up. We need an option that will scale.

If the number of connections is limited, and you dont close connections, you will start getting messages like these:



Attempted reconnect 3 times. Giving up.

Due to underlying exception: 'com.mysql.jdbc.exceptions.MySQLSyntaxErrorException: User user already has more than 'max_user_connections' active connections'.

** BEGIN NESTED EXCEPTION ** 

com.mysql.jdbc.exceptions.MySQLSyntaxErrorException
MESSAGE: User user already has more than 'max_user_connections' active connections

STACKTRACE:

com.mysql.jdbc.exceptions.MySQLSyntaxErrorException: User user already has more than 'max_user_connections' active connections
 at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:936)
 at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2870)
 at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:812)
 at com.mysql.jdbc.MysqlIO.secureAuth411(MysqlIO.java:3269)
 at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1182)
 at com.mysql.jdbc.Connection.createNewIO(Connection.java:2818)
 at com.mysql.jdbc.Connection.(Connection.java:1531)
 at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:266)
 at java.sql.DriverManager.getConnection(DriverManager.java:582)
 at java.sql.DriverManager.getConnection(DriverManager.java:185)
 at gwtlib.server.db.ConnectionThread.run(ConnectionThread.java:43)
 at java.lang.Thread.run(Thread.java:619)

How can we get around this? One option is to use connection pooling. Connection pooling keeps a set of open connections to the database available in a "pool". When requests are made, a connection can be retrieved from the pool, used, and put back into the pool. The actual connection remains open throughout the entire operation and is not closed until the application is shutdown. This way we only suffer the delay of making a connection to the database when the pool does not already contain a spare open connection.

Before we get started, there are a few things you MUST be careful of, especially if you are not used to using pools, or you are converting an existing application:

Closing resultset / prepared statements

You MUST close all result sets or prepared statements. Because we never actually close a connection ( its just returned to the pool ) all resultsets / prepared statements will be retained in memory. You will only notice this if your running an app thats making a lot ( ~ 1 a second or so ) of requests, but definitely a good idea to write code that wont cause memory leaks the first time round as they can be hard to find.

The JConnection MySSQL API states that closing a prepared statements also closed any associated prepared statements, but as closing a result set resources it might save you some memory closing the result sets as your finished with them, and closing any prepared statements at the end.

Below is a method for closing result sets that will save you catching SQL exceptions every time you what to close a result set

 public static void ( ResultSet rs ) 

{
 if ( rs != null )
 {

  try
  {
   rs.close();
  }
  catch ( SQLException sql )

  {
   // ignored
  }
 }

Dereferencing result set objects

ResultSet rs = sqlConn.execute("SELECT 1");

// rs.close(); 
rs = conn.execute("SELECT * FROM users");
rs.close();

This code will create in a memory leak; the first result set is never closed.
You must explicitly close the first and second result set ( uncomment the rs.close() line ).

Static variables

Using a static variable for a connection in servlets is a bad idea, as they are maintained between requests. This will cause problems if multiple requests come in at the same time, as the first one to finish will close the connection on the second one. If you do this, you will get all sorts of errors, especially Null Pointer Exceptions that occur within resultsets.

Implementation

Apache commons (here) provides a good, standard way to implement connection pooling. It is good because everybody knows it, its tested and it works. There are two options available; direct integration with Tomcat, or Manual Pooling.

Pooling with Tomcat

Pooling with tomcat is fairly easy. It requires three things:

1. A data source to be configured in your META-INF/context.xml
2. Some sort of driver, that tomcat will use to connect to your database.
3. Some code to initialise a data source, from which you can get connections.

Configuring your data source
Below is an example of what your context.xml could look like:

<Resource name="jdbc/myDB" 

   auth="Container"

   type="javax.sql.DataSource" 

   maxActive="20"
   maxIdle="20" 
 
   username="user"
   password="pass"

   
   driverClassName="com.mysql.jdbc.Driver"
   
   removeAbandoned="true"
   
   validationQuery="select 1"
   
   url="jdbc:mysql://192.168.0.21/myDatabase"/>

</Context>

This is straightforward. Your resource is accessable via its name; the rest of the details are concerned with connecting to your database and managing your pool.

Setting up your driver
All you have to do is put your jdbc connection driver in a place tomcat knows about, usually in $TOMCAT_DIR/lib or $TOMCAT_DIR/commons/lib.

Initialising your data source

Most commonly this is done via an application context listener.

Using it
If everything is going to plan, you can now get a reference to your datasource, and call getConnection() to get a new connection for your application to use.

Manual pooling

Manual pooling has an advantage if you run your application as a daemon and as a web app. If you create it manually both your dameon and web sessions can share the pool. Make sure you create the pool only once, either when your context is started, or your daemon is started.

 Class.forName("org.postgresql.Driver").newInstance(); // register your driver as usual (postgres in this example)

        connectionPool = new GenericObjectPool(null);

 // setup your connection factory so it knows where to get connections from

        ConnectionFactory connectionFactory = new DriverManagerConnectionFactory("jdbc:postgresql://192.168.1.1:5432/MyDB", null);

 // hook the pool up to the factory
        PoolableConnectionFactory poolableConnectionFactory = 
                              new PoolableConnectionFactory(connectionFactory,

                              connectionPool, null, null, false, true);

 PoolingDataSource dataSource = new PoolingDataSource(connectionPool);

 // now you can get connections from the pool using this:

 // Connection conn = dataSource.getConnection();

Again, this works the same as any pooling; you MUST close your resultsets and connections so that the memory is freed and your connections return back to the pool to be reused.

Errors

If tomcat cannot find your drivers, you will get an error like this:


2/04/2008 08:17:46 org.apache.catalina.core.StandardContext start
INFO: Container org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/MyContext] has already been started
org.apache.tomcat.dbcp.dbcp.SQLNestedException: Cannot load JDBC driver class 'com.mysql.jdbc.Driver'
        at org.apache.tomcat.dbcp.dbcp.BasicDataSource.createDataSource(BasicDataSource.java:766)
        at org.apache.tomcat.dbcp.dbcp.BasicDataSource.getConnection(BasicDataSource.java:540)
        at server.core.database.BravoDB.setupDatabases(BravoDB.java:83)
        at server.core.database.BravoDB.getAuthDB(BravoDB.java:214)
        at server.core.service.authentication.Authenticate.doPost(Authenticate.java:45)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
        at org.netbeans.modules.web.monitor.server.MonitorFilter.doFilter(MonitorFilter.java:368)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
        at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
        at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
        at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
        at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.Driver
        at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
        at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:169)
        at org.apache.tomcat.dbcp.dbcp.BasicDataSource.createDataSource(BasicDataSource.java:760)

You must place your drivers in $TOMCAT_HOME/commons/lib or $TOMCAT_HOME/lib (depending on your version of tomcat).

Unique URL file sharing - All

2008-03-09T19:52:00.001-07:00

Works on: Ubuntu 7.10 Server Edition
Requires: mod python enabled on your apache install

This script generates unique HTTP URL's for every file in a given directory, meaning you can give a user the link in the format http://server/files/index.py?file=12386123978624914, which will then provide a download link for a given file. Providing the directory doesnt have browse access, the scripts allow you to give anonymous access to a particular file. It is NOT very secure; if a user guesses any file in that directory it will allow access. This is NOT designed as a secure way to distribute sensitive data. To run these files you need mod python enabled on your apache server.

First off, we need add a directory to our apache config, which in ubuntu is located here:

/etc/apache2/sites-enabled/000-default

in the virtual host tag add:


<Directory /var/www/files>

Options -Indexes FollowSymLinks MultiViews
AllowOverride AuthConfig

Order allow,deny
Allow from all

AddHandler mod_python .py
PythonHandler mod_python.publisher
PythonDebug On
</Directory>

This will allow any python script in /var/www/files to be executed by the python handler.

This script is designed to take a file parameter on the URL...i.e if this was in a file index.py, you could call it with index.py?file=1923782936025293856 from your browser, where that long number is the MD hash of the file you want. See the script below this one to get the required URL's for each file.

#!/usr/bin/python

###################### Unique File URL Generator ##################################
## This script generates unique HTTP URL's for every file in a given directory.
## It provides a fairly secure way of providing anonymous file access to third party
## without having to supply a username and password or any other form of 
## authentication.
## The links are in the format: http://server/files/index.py?file=12386123978624914
##
##Source: http://programminglinuxblog.blogspot.com/ 

##License: GPLv3. See http://www.gnu.org/licenses/gpl.html 
####################################################################################

import os
from mod_python import apache

from mod_python import util

import hashlib

ROOT_FILE_DIR = "/var/www/files" # the directory our files are located

def index(req ):
#   req.content_type = "text/plain"
webpage = ""

webpage += "<html><head><title>File Repository</title></head><body>"   # HTML output
form = util.FieldStorage(req,keep_blank_values=1)

if (form.has_key("file")):  # process file parameter
reqHash = form["file"]

# get dictionary of MD hash values for all files in directory

allHashs = getMDDirectoryList()

if ( reqHash in allHashs ):

webpage +="<h2>File Found: " + allHashs.get(reqHash) + "</h2>"

webpage += "<a href='" + allHashs.get(reqHash) + "'>Download</a>"

else:
webpage += "<h2>unknown file requested</h2>" 

else:
webpage += "<h2>No file requested</h2>"

return webpage + "</body></html>"

# returns dictionary of md5 hashs of files in current directory
def getMDDirectoryList():

filelist = os.walk(ROOT_FILE_DIR)
md5list = {}

# recurse through all directories
# calculate root dir name size so only realtive dirs are shown

rootFileNameLen = len(ROOT_FILE_DIR) + 1

for root, dirs, files in filelist:

for name in files:
fullPath = os.path.join(root,name)

md5val = getMDString(fullPath)
md5list[md5val] = fullPath[rootFileNameLen:len(fullPath)]

return md5list


# returns the MD of a particular file
def getMDString( filename ):

fileObj = file(filename)

h = hashlib.new('ripemd160')

h.update(fileObj.read());
return h.hexdigest()

def cgiFieldStorageToDict( fieldStorage ):
"""Get a plain dictionary, rather than the '.value' system used by the cgi module."""
params = {}

for key in fieldStorage.keys():
params[ key ] = fieldStorage[ key ].value

return params

The script below prints out a list of URLs that will point to all the files in this directory. (so filename | http://server?file=12355125245). You can give these URLs to whoever you wish to share the files with. This file should not be available to the general public, and should probably be password protected by Apache (see below script).

#!/usr/bin/python

###################### Unique File URL Generator - list ###########################
## This script generates unique HTTP URL's for every file in a given directory. 
## It provides a fairly secure way of providing anonymous file access to third party
## without having to supply a username and password or any other form of 
## authentication.    
## The links are in the format: http://server/files/index.py?file=12386123978624914 
##
##Source: http://programminglinuxblog.blogspot.com/

##License: GPLv3. See http://www.gnu.org/licenses/gpl.html
####################################################################################

import os
import sys
from mod_python import apache

from mod_python import util

import hashlib

ROOT_FILE_DIR = "/var/www/files" # the directory our files are located

# lists all files in directory and its associated hash value

def index( ):

webpage = ""

webpage += "<html><head><title>File Repository</title></head><body>"   # HTML
webpage += "<h2>File Listing</h2>"

webpage += "<table><tr><td>FileName</td><td>URL</td></tr>"
allHashs = getMDDirectoryList()

for fileName in allHashs.keys():
url = "http://your_server_ip/files/index.py?file=" + fileName

webpage += "<tr><td>" + allHashs.get(fileName) + "</td>" + "<td><a href='" + url +  "'>Download Link</a></

td></tr>"  

webpage +="</table></html>"

return webpage

def getMDDirectoryList():
filelist = os.walk(ROOT_FILE_DIR)

md5list = {}

# recurse through all directories
# calculate root dir name size so only realtive dirs are shown

rootFileNameLen = len(ROOT_FILE_DIR) + 1


for root, dirs, files in filelist:
for name in files:

fullPath = os.path.join(root,name)
md5val = getMDString(fullPath)

md5list[md5val] = fullPath[rootFileNameLen:len(fullPath)]

return md5list

# returns the MD of a particular file
def getMDString(filename):

fileObj = file(filename)

h = hashlib.new('ripemd160')

h.update(fileObj.read());
return h.hexdigest()

index()

To password protect add this into your /etc/apache2/sites-enabled/000-default

in the virtual host tag add this, substituting list_file_hashs.py for the location that you put the script above, and /files/htpasswd with the file that contains the passwords of valid users.

<Location /files/list_file_hashs.py>
AuthType Basic
AuthName "AuthName"

AuthUserFile /files/htpasswd
Require valid-user
</Location>

Automatic SSH Login Script - All

2008-02-19T14:21:00.000-08:00

Allows you to automatically login to your server via ssh.

#!/bin/bash

if [ ! $1 ]; then
echo "usage: pushssh.sh user@remoteserver "
exit
fi

PORT=22

if [ $2 ]; then
PORT=$2
fi

# Uploads your id_rsa.pub to the specified host, wrapped for readability

if [ ! -r ${HOME}/.ssh/id_rsa.pub ]; then
ssh-keygen -b 2048 -t rsa
fi

# Make sure auth file exists and chmod to 600

ssh $1 -p $PORT 0> echo "mkdir ~/.ssh; touch ~/.ssh/authorized_keys; 
chmod u+rw .ssh/authorized_keys"

# Append to the copy on the remote server

cat ~/.ssh/id_rsa.pub | ssh $1 -p $PORT "cat - >> .ssh/authorized_keys"

if [ $? -eq 0 ]; then
echo "Success"
fi

Tomcat 5.5 Socket Exception - Ubuntu

2008-02-13T13:50:00.000-08:00

This error message:


java.net.SocketException
MESSAGE: java.security.AccessControlException: access denied (java.net.SocketPermission 
192.168.1.253:3306 connect,resolve)

STACKTRACE:

java.net.SocketException: java.security.AccessControlException: access denied (java.net.SocketPermission 192.168.1.253:3306 
connect,resolve)
 at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:156)
 at com.mysql.jdbc.MysqlIO.(MysqlIO.java:276)
 at com.mysql.jdbc.Connection.createNewIO(Connection.java:2666)
 at com.mysql.jdbc.Connection.(Connection.java:1531)
 at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:266)
 at java.sql.DriverManager.getConnection(DriverManager.java:525)
 at java.sql.DriverManager.getConnection(DriverManager.java:171)
 at server.core.database.ConnectionThread.run(ConnectionThread.java:54)
 at java.lang.Thread.run(Thread.java:595)

is caused by a restrictive java policy file. Note that tomcat doesnt use the standard java.policy file (i.e. /usr/lib/jvm/java-sun-6/jre/lib/security/java.policy), but instead uses /etc/tomcat5/policy.d/04webapps.policy.

To fix the above example, you need to add something like this inside your grant { block:


grant {

   // grant permissions to sql server
    permission java.net.SocketPermission 
       "192.168.1.253:3306","connect,resolve,listen,accept";

Apt-Cacher - Cache Updates, Source List Script - Ubuntu

2008-02-09T20:00:00.000-08:00

Apt-cacher provides a way of caching your repository updates, and is useful to minimise the amount you have to download from the net if you have multiple Ubuntu machines running on your LAN. There are plenty of tutorials for apt-cacher on the web; so I wont explain how to use it, but just provide an example of how I have it setup, which may save you some time.

modify:
etc/default/apt-cacher


AUTOSTART=1

to setup your path map modify:
/etc/apt-cacher/apt-cacher.conf


path_map = ubuntu-archive au.archive.ubuntu.com/ubuntu ; ubuntu-canonical 
archive.canonical.com/ubuntu ; ubuntu-security security.ubuntu.com/ubuntu ; ubuntu-skype 
download.skype.com/linux/repos/debian ;

Restart apt-cache (/etc/init.d/apt-cacher restart) Now for each client computer (using the cache) you can reference them like this (where 10.0.0.10 is the IP address of your update server)
/etc/apt/sources.list


# deb cdrom:[Ubuntu 8.04 _Hardy Heron_ - Alpha i386 (20080109)]/ hardy main restricted
# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.

deb http://10.0.0.10:3142/ubuntu-archive  hardy main restricted universe
deb-src http://10.0.0.10:3142/ubuntu-archive  hardy main restricted

## Major bug fix updates produced after the final release of the
## distribution.
deb http://10.0.0.10:3142/ubuntu-archive  hardy-updates main restricted universe
# Line commented out by installer because it failed to verify:
# deb-src http://10.0.0.10:3142/ubuntu-archive  hardy-updates main restricted

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team, and may not be under a free licence. Please satisfy yourself as to
## your rights to use the software. Also, please note that software in
## universe WILL NOT receive any review or updates from the Ubuntu security
## team.
# Line commented out by installer because it failed to verify:
# deb http://10.0.0.10:3142/ubuntu-archive  hardy universe
# Line commented out by installer because it failed to verify:
# deb-src http://10.0.0.10:3142/ubuntu-archive  hardy universe
# Line commented out by installer because it failed to verify:
# deb http://10.0.0.10:3142/ubuntu-archive  hardy-updates universe
deb-src http://10.0.0.10:3142/ubuntu-archive  hardy-updates universe

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu 
## team, and may not be under a free licence. Please satisfy yourself as to 
## your rights to use the software. Also, please note that software in 
## multiverse WILL NOT receive any review or updates from the Ubuntu
## security team.
deb http://10.0.0.10:3142/ubuntu-archive  hardy multiverse
deb-src http://10.0.0.10:3142/ubuntu-archive  hardy multiverse
deb http://10.0.0.10:3142/ubuntu-archive  hardy-updates multiverse
deb-src http://10.0.0.10:3142/ubuntu-archive  hardy-updates multiverse

## Uncomment the following two lines to add software from the 'backports'
## repository.
## N.B. software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
deb http://10.0.0.10:3142/ubuntu-archive  hardy-backports main restricted universe multiverse
# deb-src http://10.0.0.10:3142/ubuntu-archive  hardy-backports main restricted universe multiverse

## Uncomment the following two lines to add software from Canonical's
## 'partner' repository. This software is not part of Ubuntu, but is
## offered by Canonical and the respective vendors as a service to Ubuntu
## users.
# deb http://archive.canonical.com/ubuntu hardy partner
# deb-src http://archive.canonical.com/ubuntu hardy partner

deb http://10.0.0.10:3142/ubuntu-security hardy-security main restricted
deb-src http://10.0.0.10:3142/ubuntu-security hardy-security main restricted
deb http://10.0.0.10:3142/ubuntu-security hardy-security universe
deb-src http://10.0.0.10:3142/ubuntu-security hardy-security universe
deb http://10.0.0.10:3142/ubuntu-security hardy-security multiverse
deb http://10.0.0.10:3142/ubuntu-archive  hardy-proposed restricted main multiverse universe
deb-src http://10.0.0.10:3142/ubuntu-security hardy-security multiverse
# deb http://download.skype.com/linux/repos/debian/ stable non-free
deb http://10.0.0.10:3142/ubuntu-skype stable non-free

Source List Script

I have written a Python script to automatically update a sources list file with your apt-cacher details and specified mappings. My Python skills arnt the greatest, so if you have any improvements to offer, let me know.

#!/usr/bin/python
import os

# ------------- Parameters --------------------

# input / output source list files
INPUT_SRC='/etc/apt/sources.list';
OUTPUT_DEST='source.list.new';

# apt cacher details
APT_CACHER_SERVER="192.168.1.250";

APT_CACHER_PORT=3142;
APT_CACHER_TYPE="http";

# archive urls
OLD_ARCHIVE="http://archive.ubuntu.com/ubuntu/";

OLD_CANONICAL="http://au.archive.ubuntu.com/ubuntu/";
OLD_SECURITY="http://security.ubuntu.com/ubuntu/";

# new mappings (configured in apt-cacher)

ARC_MAP="ubuntu-update";

CAN_MAP="ubuntu-canonical";
SEC_MAP="ubuntu-security";

# ---------------------------------------------

print "source updater started";

TMP_DEST='/tmp/source.tmp';

# sed delimited archives
SED_ARCHIVE = str.replace ( OLD_ARCHIVE , "/", "\/" );

SED_CANONICAL = str.replace ( OLD_CANONICAL , "/", "\/" );

SED_SECURITY = str.replace ( OLD_SECURITY , "/", "\/" );

# complete sed delimited apt server url
APT_SERVER=APT_CACHER_TYPE + ":\/\/" + APT_CACHER_SERVER + ":" + str(APT_CACHER_PORT) + "\/";

# print "apt server url: (sed): " + APT_SERVER ;

ARCH_SED_CMD="sed -e '" + "s/" + SED_ARCHIVE + "/" + APT_SERVER + ARC_MAP + "/'";

CAN_SED_CMD="sed -e '" + "s/" + SED_CANONICAL + "/" + APT_SERVER + CAN_MAP + "/'";

SEC_SED_CMD="sed -e '" + "s/" + SED_SECURITY + "/" + APT_SERVER + SEC_MAP + "/'";

# copy file to new tmp location
COPY_CMD = "cp " + INPUT_SRC + " " + TMP_DEST;

os.system(COPY_CMD);

IN_CMD = "cat " + TMP_DEST + " | ";

OUT_CMD = " > " + OUTPUT_DEST;

# cat from tmp to new
COPY_BACK = "cp " + OUTPUT_DEST + " " + TMP_DEST;

# copy from new back to tmp
os.system( IN_CMD  + ARCH_SED_CMD + OUT_CMD );

os.system(COPY_BACK); 
os.system( IN_CMD  + CAN_SED_CMD + OUT_CMD );

os.system(COPY_BACK); 
os.system( IN_CMD  + SEC_SED_CMD + OUT_CMD );

print "source updater finished";

init-d template - All

2008-01-21T18:08:00.000-08:00


#!/bin/bash

case "${1:-''}" in
  'start')
           # start commands here
        ;;
  'stop')
           # stop commands here
        ;;
  'restart')
           # restart commands here
        ;;
  *)      # no parameter specified
        echo "Usage: $SELF start|stop|restart|reload|force-reload|status"
        exit 1
        ;;
esac

MySQL - Java connection Exception - All

2008-01-20T16:14:00.000-08:00

I came across this exception when trying to setup an tomcat app that connected to a mySQL server (on the same server).

Problem


** BEGIN NESTED EXCEPTION **

java.net.SocketException
MESSAGE: java.security.AccessControlException: access denied (java.net.SocketPermission 
127.0.0.1:3306 connect,resolve)

STACKTRACE:

java.net.SocketException: java.security.AccessControlException: access denied 
(java.net.SocketPermission 127.0.0.1:3306 connect,resolve)
        at com.mysql.jdbc.StandardSocketFactory.unwrapExceptionToProperClassAndThrowIt(StandardSocketFactory.
java:407)
        at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:268)

......

** END NESTED EXCEPTION **


Attempted reconnect 3 times. Giving up.
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessor

Solution

This is a Java exception, caused by your java.policy file. Its in your JAVA_HOME/jre/lib/security folder. Heres where it is in Ubuntu (using java 6).

/usr/lib/jvm/java-6-sun/jre/lib/security/java.policy

You need to add the following lines:


grant{
permission java.net.SocketPermission "127.0.0.1:3306", "connect, resolve";
};

Tomcat 5.5 on Ubuntu (Gutsy)

2007-11-27T13:02:00.000-08:00

Setting up Tomcat 5.5 on Ubuntu is not as straight forward as it should be, but is very easy when you know how. Firstly,


apt-get install tomcat5.5 tomcat5.5-admin

This will install and start Tomcat. This is misleading; you MUST restart your machine in order for tomcat to start reading your users.xml file, to allow you to login to the manager page.

Configuration

Security

Tomcats configuration is in /etc/tomcat5.5. (which is linked to /var/lib/tomcat5.5/) Once you have restarted, add a user tomcat-users.xml in this directory making sure they have the role admin/manager. For example,


#user username="admin" password="admin" roles="standard,manager"/#

You must restart Tomcat for this to take effect:


/etc/init.d/tomcat5.5 restart

You can configure tomcat's port for connections in server.xml. By default it runs on 8180.

Deployment

Tomcat is deployed here


/usr/share/tomcat5.5/

Configuration pages


http://localhost:8180/manager/html/


http://localhost:8180/host-manager/html/

Logs

By default, all logs (i.e all output from your servlets) will end up in,


/var/logs/syslog

Note that you may also need to set JAVA_HOME (i.e put export JAVA_HOME=/usr/lib/java-6-sun in ~/.bashrc, and start a fresh session)

Detecting IP address from Mac Address - Linux

2007-11-01T17:22:00.000-07:00

Operating System: Linux

Im sure there's a more elegant way of doing it, but the finding out a computers IP address from MAC address can be done like this:

ip neigh flush all &gt; /dev/null; # Clears out your arp cache

fping -c 1 -g -q 192.168.1.0/24 2&gt; /dev/null;  # pings all of subnet

arp -n | grep "00:1B:4C:22:28:B1"   # Search cache for MAC address

By pinging all computers on the subnet, your arp cache will be refreshed with MAC -> Translations. The arp -n command lists all entries in your arp cache, and by piping to grep you only see what your looking for. NOTE: You can remove the pipes to /dev/null, Ive only done this to stop unnecessary output.

Python Glade MAC Address To IP Address Application

I have written a small (free, under GPL) application to demonstrate the uses of fping or raw sockets. Raw sockets will be used if the fping command isn't avaiable, but require root access and is much slower. The application has good examples of:

A simple Linux GUI (gtk.glade library)

Execute system commands

Using raw sockets

Using threads

It is available here. If you would like to extend the application, send me a message and I'll upload your updates.

Wireless via Linksys WRT56G using WPA-PSK - Ubuntu

2007-10-24T05:12:00.000-07:00

Operating System: Xubuntu
Below I will explain how I got my Linksys WRT56G wireless router going with ubuntu (Xubuntu) linux running WPA-PSK. Replace eth1 with your network adapter.

Finding your network

If you have set your ssid visible on the router, you should be able to view your networking using


iwconfig eth1 scan

Open

Connecting to an open network is easy. Run the following


iwconfig eth1 essid "Your Network" && dhclient eth1

If your using a static IP address, you will need to configure /etc/network/interfaces appropriately.

WPA

I never had much luck with wifi-radar, so found manually configuring wpa_supplicant easier.
Edit /etc/wpa_supplicant/wpa_supplicant.conf and fill in your network details:


ctrl_interface=/var/run/wpa_supplicant
ctrl_interface_group=0
eapol_version=1
ap_scan=2
fast_reauth=1

network={
        ssid="linksys_SES_26841"
        proto=WPA
        key_mgmt=WPA-PSK
        pairwise=TKIP
        group=TKIP
        psk="yourprivatekeyhere"
}

To activate your connection run (remember to replace eth1 with your wireless card):


wpa_supplicant -c /etc/wpa_supplicant/wpa_supplicant.conf -D wext -dd -i eth1 -B && dhclient eth1

Running this should connect to your router using DHCP.

Startup Script

Running this every time you start up is annoying, so copy the wpa_supplicant line into a file in /etc/init.d/ (you will need to be root to do this) and run the command:


update-rc.d defaults wireless

where "wireless" is the name of the file you created in init.d. Now when you reboot, your wireless should connect automatically.

Wireless a netgear WG111v2 and kistmet - Ubuntu

2007-10-02T20:29:00.000-07:00

This blog will cover how to setup a usb WG111v2 adapter to be used with kismet on Ubuntu.

Introduction

The usb netgear WG111v2 adapter has a Realtek rt8180 chipset. The current kernel has support for this adapter via the r8187 module, but does not support monitoring mode (meaning kismet and aircrack will not work).

To get it to work, you need to manually install the rtl-wifi driver.
Note: this is NOT the same as the rtl8180-sa2400 project. See the wiki
for details:

http://rtl-wifi.sourceforge.net/wiki/

Installing

Firstly, to check to see if your adapter has been detected by Ubuntu running lsusb,
should give you a line looking like this:


Bus 005 Device 002: ID 0846:6a00 NetGear, Inc. WG111 WiFi (v2)

Typing lsmod will probably reveal that Ubuntu has loaded the standard
driver, as you should be able to find the module "r8187" somewhere.

Remember

Make sure you reboot your machine WITHOUT your adapter pluged in. If you do not, you may run into this error:


insmod: error inserting ieee80211/iee80211-rtl.ko: -1 Invalid symbol in module
insmod: error inserting rtl18187-newstack/r8187.ko: -1 Unknown symbol in module

Also, make sure you make clean all directories before rebulding. (i.e the stack, in directory ieee80211 and the module rtl8187-newstack)

Finally, make sure you backup your /lib/module/2.6.XX-XX-generic/kernel/net/ieee80211/ directory, so you can revert back to your old settings.

Building

Before following the instructions here to install:

http://rtl-wifi.sourceforge.net/wiki/Installing#Installing_the_rtl-wifi_driver

NOTE that the current version does NOT work the the latest version of the Ubuntu kernel (2.6.22-12-generic); you must checkout revision 62 (use svn -r 62, or if you've already check it out, use svn switch).


svn sw -r 62 https://rtl-wifi.svn.sourceforge.net/svnroot/rtl-wifi rtl-wifi

If it doesnt compile, try checking out a previous version. If it does, set your source in kismet.conf.to


source=rt8180,wlan0,RTL

Remember, make sure you remove your USB wifi adapter and "make clean" in ALL directories of rtl-wifi; otherwise you may not be able to build correctly.

Reverting

To revert your system back, just copy the ieee80211 you backed up into into /lib/module/2.6.XX-XX-generic/kernel/net/ieee80211/. If you didnt back it up, use synaptic to competely remove, then reinstall linux-image, and restart your system.

Linux on iPAQ rx3715

2007-10-01T20:00:00.000-07:00

Introduction

I will explain how to install linux (Familar, running Opie) on an iPAQ rx3715. I would think this would be applicable to any rx3000 series IPAQ, but who knows. Here are some references to sites that will help you. I have subsequently sold my iPAQ, but have published what I know in the hopes it helps somebody else.

There are two options:

1) Format your memory card to have one fat (vfat) partition (as partition 1) and one ext3 partition (as partition 2). The first partition holds the image, the second holds the file system including Opie. Once the image and filesystem are in place, you can run Haret to boot into Linux.

2) Format your memory card to have one partition, all ext3 and place the image onto your Ipaq File Storage. This is probably the easiest way of doing it.

References

http://www.handhelds.org/projects/rx3715.html
http://opie.handhelds.org/

Requirements

Firstly, you need the ability to do these things before you can start:

Format a SD / MMC card with Linux filesystem (ext2 / ext3)

This can be difficult. Alot of card readers are not compatible with linux. In addition to this, SD is a proprietary format, so you will probably need to use an MMC (MultiMediaCard) card.

I managed to use my MP3 player to do it, but its still a little flakey, and wont format properly all of the time. I'd invest in a SD/MMC card reader that's properly supported in Linux. Your other option is to use windows to format it, but using a driver.

linux

to format it in linux, you must be root and execute the following:


fdisk -l


umount /dev/sdb1   # make sure your partitions arent mounted

# this will show you which which device your card is registered as.


fdisk /dev/sdb    # note: do NOT put the parition number in
mkfs.vfat
mkfs.ext3
fsk /dev/sdb1     # can use this to check partition

Transfer information between your Ipaq and a computer

Linux:

Use synce.

Windows:

ActiveSync

Resouces.

--Installing--

The first parition (vfat) is what will hold:

1. Kernel Image (zImage)
2. Haret ( Unloads Windows CE out of memory, and boots the kernel )
3. default.txt ( Tells Haret what image to use and where to mount root )

VirtualBox network bridge script - All

2007-09-27T22:18:00.000-07:00


# requires apt-get install uml-utilities bridge-utils (Ubuntu)
# and to be run as root

# Set your machine in VirtualBox to use  “Host Interface”, and set
# Interface name to "tap1". Run this script every time you require the bridge set up,
# before you start your virtual machine.

#!/bin/sh

USERNAME=user           # login name of HOST system
DHCP=1      # set to 1 to use dynamic ip for bridge
IP_ADDRESS=192.168.1.1    # static ip address of bridge (only used if DHCP set to 0)

tunctl -t tap1 -u $USERNAME
brctl addbr br0
ifconfig eth0 0.0.0.0 promisc
brctl addif br0 eth0

if [ DHCP = 1 ]; then
 dhclient br0
else
 ifconfig br0 $IP_ADDRESS
fi 

brctl addif br0 tap1
ifconfig tap1 up
chmod 0666 /dev/net/tun

Hamachi - Gutsy Gibon

2007-09-27T14:47:00.000-07:00

Hamachi doesn't work correctly with Gutsy Gibon. It will complete with no response. To make it work, you need to install:


upx-ucl-beta

then run (as root)


upx -d /usr/bin/hamachi

It should now work correctly.
See this thread on the ubuntu forms for details.

Automatic FTP Transfer Script - Linux

2007-09-26T17:30:00.000-07:00

Operating System: Linux
Tools Required yafc (Yet Another FTP Client)

To do automated FTP transfers any command line FTP tool can be used. Ive chosen yafc because its commands are similar to standard Linux commands. The code below will connect to FTP server 192.168.1.3, on port 21, using a username of "username" and password of "password". It then lists the directory (ls), removes two files (somefile.txt and anotherfile.txt), then sends all the files in /home/user to the current directory on the FTP server. Finally it closes the connect to the server. Simple.


#!/bin/bash
yafc  <<**
open ftp://username:password@192.168.1.3:21/ 
ls 
rm -f dir/somefile.txt
rm -f dir/anotherfile.txt
put -f -r /home/user/*
close
**

You can do an authenticated GET request also, using wget. This is useful if you require to restart your server after uploading, and you're able to do it from a web interface. Using the spider flag means wget doesn't actually download the page.


wget --spider http://192.168.1.3/page --http-user=youruser --http-password=yourpassword

Java - Writing XML document - All

2007-09-25T04:06:00.000-07:00

Creating an XML document with java is very easy. If you know how to use System.out.println(), then you can create an XML document. This may be enough for trivial programs, but for any application that requires the xml to be well formatted, parsed or understood by others, then a framework of sorts is needed. Below I have explained how to create a simple XML document using org.w3c.dom.


import org.w3c.dom.Document;
import org.w3c.dom.Element;

public void createXML() 
{
        DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
        DocumentBuilder builder;
        Document doc;

        try 
        {
            
            builder = factory.newDocumentBuilder();    // create document
            doc = builder.newDocument();
            
            Element e = doc.createElement("root");      // create root element
            
            doc.appendChild(e);     // append to document

            e.setAttribute("attributeOne","valueOne");       // set a couple of attributes
            e.setAttribute("attributeTwo","valueTwo");
            
            Element e2 = doc.createElement("SubElement");    // add another element to the first
            e.setAttribute("attributeThree","valueThree");
            e.appendChild(e2);

        } 
        catch (ParserConfigurationException ex) 
        {
            ex.printStackTrace();
        }
}

Parsing XML:


public void parseXML(String xml)
{
            DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
            DocumentBuilder builder = factory.newDocumentBuilder();
            InputStream is = new ByteArrayInputStream(xml.getBytes());
            Document doc = builder.parse(is);
            
            NodeList list = doc.getChildNodes();
            Node n = list.item(0);         // root element
            NamedNodeMap attributes  = n.getAttributes();
            
            Node itemA = attributes.getNamedItem("myAttribute");
}

Printing the XML document:


        StringWriter sw = new StringWriter();       // write to a string
        XMLSerializer ser = new XMLSerializer(sw, new OutputFormat(doc));
        ser.serialize(doc.getDocumentElement());
        String XMLStr = sw.toString();
        System.out.println(XMLStr);

Booting from floppy disk - Ubuntu

2007-09-24T20:08:00.000-07:00

If you can not boot from your CD-ROM drive but can read from it, then you need to use the SmartBootManager. To install it under windows look here:

https://help.ubuntu.com/community/SmartBootManagerHowto

to install it under linux, go to the installs directory of the ubuntu install cd, put in a empty floppy, and type (as root):


dd if=/dev/fd0 of=~/sbm.bin

it should print something like this when finished:


2880+0 records in
2880+0 records out
1474560 bytes (1.5 MB) copied, 47.6615 seconds, 30.9 kB/s

Assuming your floppy disk dirve is fd0.

Common Problems - Xbuntu

2007-09-23T15:16:00.001-07:00

Problem: When starting an application it says the display cannot be found, or, for example


cutecom: cannot connect to X server

Solution:


export DISPLAY=:0.0

then as your normal user


xhost local:root

Problem: Adding a startup service:

As root, add a file in /etc/init.d/ with the commands you want executed, then make it executable with the following command.

chmod +x file

Now, add it to a runlevel.

update-rc.d file defaults

This will create links from /etc/rcX.d to your new script.

Problem: Select newly installed java runtime


update-alternatives --config java

And select which option you require.
Now, add the directory into /etc/jvm...i.e


/usr/lib/java-x-xx

Virtual Box - Xubuntu

2007-09-02T20:20:00.001-07:00

Operating System: Xubuntu (Gutsy Gibon)
Host : Ubuntu 6.10

VirtualBox is an awesome VM for Linux, akin to VMWare and Parallels. Awesome because its free, and awesome because it works.

Installation is easy; add in your repo and go. Im not going to explain that; its already explained at www.virtualbox.org. Setting up Guest Additions is a little more difficult, and this is what I will explain below.

Guest Additions is required to share directly between the host and guest systems. On your guest, you need the linux kernel, headers and build utils. You also need to create a kernel config file, and to build it.

Firstly though, start your VM, and select on the Virtual Machine Window menu -> Devices -> Install Guest Additions. This should mount a cd in /media/ on the guest, which you should copy to a write enabled directory. Now, build your kernel.

apt-get install linux-source             # the kernel source code with patches
apt-get install linux-kernel-devel       # You probably only need build-essentials
apt-get install libncurses5-dev          # Only required for  make menuconfig
apt-get install autoconf                 # guest additions needs this

Now, unzip your source, create your linux sym link, and make your kernel


tar xvf linux-source-2.6.xx.tar.bz2
ln -s /usr/src/linux /usr/src/linux-source-2.6.xx
cd /usr/src/linux
make menuconfig   # close, and save changes (creates a .config file)
make && make modules_install

Now you have a compiled copy of the kernel source for guest utils to link against. Go back to the place you copied the guest additions folder to. Run


./VBoxLinuxAddtions.run # You may need to chmod +x this file

If something errors, look in /var/log/vboxadd-install.log.

Restart your host. All going well, you can now add transient or machine folders. Transient shared folders only last for the length of the guest session, where as Machine folders last until removed. To create a transient directory (Virtual Machine Window menu -> Device-> Shared folders). And add a transient folder.
Within your guest environment, you should now be able to mount this directory:

sudo mount -t vboxsf transientFolderName folder

Files copied to this folder should be readable by your host and guest environment.

Machine Folders are added via your machines settings, under the "Shared folder" option. They work in exactly the same way as transient folders.

NOTE: Every time you update on your kernel on your guest operating system, you will need to reinstall guest additions.

Netbeans Errors - All

2007-08-19T15:35:00.000-07:00

SVN (via SSH)

Problem

In a popup you'll get this message:
org.tigris.subversion.svnclientadapter.commandline.CmdLineException: ssh_askpass: exec(/usr/bin/ssh-askpass): No such file or directory

Permission denied, please try again.
ssh_askpass: exec(/usr/bin/ssh-askpass): No such file or directory
Permission denied, please try again.
ssh_askpass: exec(/usr/bin/ssh-askpass): No such file or directory
Permission denied (publickey,password).
svn: Connection closed unexpectedly

up -r HEAD /repo... --non-interactive --config-dir /pathtobeans/.netbeans/5.5.1/config/svn/config

Fix

Install ssh-askpass. In ubuntu its as simpe as:


apt-get install ssh-askpass

Installing Beast 0.7.1 - Fedora 7

2007-08-15T05:11:00.000-07:00

Fedora 7: Installing beast-0.7.1 (http://beast.gtk.org/)

Summary

Building beast requires a few libraires to be installed, and you will need the livna repo enabled. Heres a summarized list of what you will need to install:

yum install gcc-c++ glibc2-devel glibc2-devel intltool perl-XML-Parser
libmad-devel libvorbis-devel libogg-devel (might not be required)
guile-devel  pango-devel gtk2-devel ruby-libart2-devel
libxml2-python libgnomecanvas-devel

Python 2.4 is required. If you dont have it, or have 2.5 you can make a python2.4 link to python2.5 and it seems to work ok. Also, there seems to be some files that you need to get from the SVN repository, so svn will need to be "yummed", and you'll need to check out the Beast repository.

Configuring Beast

Ive listed all the errors i've had below, and how I got around them. To get anything to compile, you need the gcc c++ compiler.


yum install gcc-c++

configure: error: Failed to detect or find glib-2.0 >= 2.6.4 gthread-2.0


yum install glibc2-devel

checking for XML::Parser... configure: error: XML::Parser perl module is
required for intltool

yum install perl-XML-Parser

configure: WARNING: failed to detect Ogg/Vorbis library (>=1.0) or
headers
checking for mad_stream_errorstr in -lmad... no
configure: WARNING: failed to detect libmad (MPEG I-III audio decoder)
or headers checking the required GSL-Loader library set...
configure: error: Ogg/Vorbis is missing, but required

yum install libmad-devel # installs libmad

yum install libvorbis-devel # install vorbis

# NOTE: I also installed libogg-devel, but i dont think this is
requried

configure: error: failed to detect guile headers and library v1.6
(available from http://www.gnu.org/software/guile/guile.html) - detected
library version: none

yum install guile-devel

Installing:
guile-devel

Installing for dependencies:
gmp i386 4.1.4-12.3 fedora
gmp x86_64 4.1.4-12.3 fedora
gmp-devel x86_64 4.1.4-12.3 fedora
guile i386 5:1.8.1-3.fc7 fedora
guile x86_64 5:1.8.1-3.fc7 fedora
libtool-ltdl i386 1.5.22-11.fc7 fedora
libtool-ltdl x86_64 1.5.22-11.fc7 fedora

configure: WARNING: failed to detect proper sound device files
checking for LIBPANGO_CFLAGS...
checking for LIBPANGO_LIBS...
configure: error: Package requirements (pangoft2 >= 1.4.0) were not met.
Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

Alternatively you may set the LIBPANGO_CFLAGS and LIBPANGO_LIBS environment variables
to avoid the need to call pkg-config. See the pkg-config man page for
more details.

yum install pango-devel

configure: error: Package requirements (gtk+-2.0 >= 2.6.4) were not met.
Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

Alternatively you may set the LIBGTK_CFLAGS and LIBGTK_LIBS environment variables
to avoid the need to call pkg-config. See the pkg-config man page for
more details.

yum install gtk2-devel

Installing:
gtk2-devel i386 2.10.13-1.fc7 updates 3.0 M
gtk2-devel x86_64 2.10.13-1.fc7 updates 3.0 M
Installing for dependencies:
atk-devel x86_64 1.18.0-1.fc7 fedora 129 k
libXcursor-devel x86_64 1.1.8-1 fedora 14 k
libXfixes-devel x86_64 4.0.3-1 fedora 9.5 k
libXi-devel x86_64 1.0.4-1 fedora 52 k
libXinerama-devel x86_64 1.0.2-1.fc7 fedora 8.2 k
libXrandr-devel

configure: error: Package requirements (libart-2.0 >= 2.3.8) were not met.
Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

yum install ruby-libart2-devel

configure: error: Package requirements (libgnomecanvas-2.0 >= 2.4.0) were not met.

yum install libgnomecanvas-devel

Making Beast

Compiling with guile-1.8.1

Trying to compile with guile-1.8.1 will give this error:

bsescminterp.c:651: error: ‘scm_catch_body_t’ undeclared (first use in this function)
bsescminterp.c:651: error: (Each undeclared identifier is reported only once
bsescminterp.c:651: error: for each function it appears in.)
bsescminterp.c:651: error: expected ‘)’ before ‘signal_marshal_sproc’

...followed by some warnings, then...

make[2]: *** [bsescminterp.o] Error 1
make[2]: Leaving directory `/home/fedora/Desktop/music/beast-0.7.1/shell'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/fedora/Desktop/music/beast-0.7.1'
make: *** [all] Error 2

It seems it is not possible to get beast-0.7.1 working with guile-1.8.1
without applying this patch.
http://bugzilla.gnome.org/attachment.cgi?id=80947&action=view

http://www.nabble.com/compiling-beast-for-guile-1.8-t3447427.html

shell/bsescminterp.h, shell/bsescm.c, and shell/bsescminterp.c need to be modified.

I manually incorporated these differences. If you require these files let me know.

make[3]: Entering directory
`/home/fedora/Desktop/music/beast-0.7.1/docs'
../doxer/doxer.py doxi2man -d mans/ -D TOP_WEBDIR "." -D BST_VERSION
0.7.1 -I . -D DOXI_INCLUDE_FILE \"docframe.doxi\"
/bin/sh: ../doxer/doxer.py: No such file or directory

You need to copy all of ./doxer and ./docs the directory from the SVN
version, and make sure python2.4 is installed

, ManGenerator, qdoxygen, qxmlparser
File "/home/fedora/Desktop/music/beast-0.7.1/doxer/qxmlparser.py",
line 21, in ?
import os, sys, re, libxml2
ImportError: No module named libxml2

yum install libxml2-python -y (Python bindings for the libxml2 library)

* NOTE: To save installing 2.4, i copied the 2.5 exe to python2.4. i.e:


cp /usr/bin/python2.5 /usr/local/bin/python2.4

make[3]: *** No rule to make target `html/style/beast-style.css', needed
by `all-am'. Stop.
make[3]: Leaving directory `/home/fedora/Desktop/music/beast-0.7.1/docs'

cp ./web/styles from svn

0.7.1/docs/html/architecture.html'
/usr/bin/install: cannot stat `./html/architecture.html': No such file
or directory
/usr/bin/install -c -m 644 './html/beast-index.html'
'/usr/local/share/beast/v0.7.1/docs/html/beast-index.html'
/usr/bin/install: cannot stat `./html/beast-index.html': No such file or
directory

You should now be able to run "make install" even with the above errors, and it should install.

64bit Linux Installs - All

2007-07-30T17:34:00.000-07:00

Adobe Flash

Yes you can install flash on a 64bit distribution. Ofcourse it doesn't run in 64bit mode, but who really cares? Heres how you install it in Fedora (from the Livna repo)


yum install swfdec-mozilla

Java Applets

Again, embedded java applets can be run fairly easily, just install a jre and link your gcj directory to your mozilla plugins directory. The below example is for jre-4.1.2...


mkdir ~/.mozilla/plugins/
ln -s /usr/lib64/gcj-4.1.2/libgcjwebplugin.so .mozilla/plugins

Restart firefox, then load this page to test

WebDav - Apache Tomcat

2007-07-15T21:12:00.000-07:00

An example WebDav deploy comes installed by deafult when you first install Tomcat. The default setup is readonly, and doesn't allow users to login, and write data (for things such as sharing calendar in Sunbird, Mozilla calendar etc). To enable write access you need to have a user within tomcat that has a role thats allowed to access WebDav. Users and roles can be added easily by using the tomcat admin page. Once you have done this, its a simple matter of editing the webapps/webdav/web.xml.

Server Configuration

Firstly uncomment these lines (NOTE: I have removed all the "<" and ">" characters)


   init-param
      param-name
                readonly
    /param-name
      param-value
                    false
     /param-value
   /init-param

You then need to uncomment this:


  login-config
    auth-method
                    BASIC
   /auth-method
    realm-name
                Tomcat Supported Realm
    /realm-name
 /login-config

Now, add in any roles you wish to have the ability to use WebDav. In this example, the role "webdav" will have access.


  security-role
         description
                 An example role defined in "conf/tomcat-users.xml"
        /description
        role-name
                webdav
        /role-name
 /security-role

Now add a role in ./conf/tomcat-users.xml, such as:

to give the user "auser" access to the webdav folder. Redeploy WebDav via the tomcat manager, and WebDav folder should be now be accessible.

Testing

If you wish to check access, you can use davfs2 to mount webdav folders.


sudo apt-get install davfs2  # Using Ubuntu

Put this entry in your /etc/fstab


http://myserver/webdav/  /mnt/mydocs davfs rw,noauto,user 0 0

Then mount it as


mount /mnt/mydocs

You will be prompted for a username and password. If sucessful, you shoul be able to read and write to that directory.

Go here for more info.

Clients

Both sunbird and Mozilla calendar support sharing calendar information via WebDav.

Sunbird

www.mozilla.org/projects/calendar/sunbird

Mozilla calendar

http://www.mozilla.org/projects/calendar/faq.html#remote_server
http://www.twilight-systems.com/flacco/mozcal/mozcal-webdav.html

Locales problem - Ubuntu 6.06

2007-07-03T21:13:00.000-07:00

Operating System Ubuntu 6.06.

Doing an apt-get update && apt-get upgrade gave this error:

/var/cache/apt/archives/locales_2.3.18.3_all.deb (--unpack): trying to overwrite `/usr/share/zoneinfo/Africa/Algiers', which is also in package tzdata
dpkg-deb: subprocess paste killed by signal (Broken pipe)

There may be other ways of fixing this, but the easiest way is to force the package to be installed:

dpkg -i --force-overwrite /var/cache/apt/archives/locales_2.3.18.3_all.deb

Which will then complain about belocs-locates-bin not being installed. (I uninstalled this previously, so you may not get this error)

Preparing to replace locales 2.3.18.3 (using .../locales_2.3.18.3_all.deb) ...
Unpacking replacement locales ...
dpkg: dependency problems prevent configuration of locales:
locales depends on belocs-locales-bin (>= 2.3.5-5ubuntu1); however:
 Package belocs-locales-bin is not configured yet.
dpkg: error processing locales (--install):
dependency problems - leaving unconfigured
Errors were encountered while processing:
locales

So install the dependancy.

    apt-get install belocs-locales-bin

And try forcing the install again.

SVN

When I uninstalled locales, I got this SVN error (on the client):


svn: warning: cannot set LC_CTYPE locale
svn: warning: environment variable LANG is en_AU.UTF-8
svn: warning: please check that your locale name is correct
At revision 32.

To fix, install locales:


apt-get install locales

You may also get this problem, when using puttty in windows to connect to the server:

svn: Can't create tunnel: The system cannot find the file specified.

To fix this edit your svn config file located here:


%USERPROFILE%/Application Data/Subversion/config


Then:


Uncomment [tunnel]
[tunnel]: ssh = C:/Program Files/TortoiseSVN/bin/TortoisePlink.exe

Windows File Sharing (Samba) - Xubuntu

2007-06-23T21:39:00.000-07:00

By default, xubuntu doesn't come with a samba network browser, and
samba sharing isnt enabled. This is easy to fix. To add a share
go into


System-> Shared Folders

and share a folder. This will install samba (smb) for you. You can also do it
via the synaptic package manager.

Clients - pyNeighborhood

As far as clients go, there are a few options, but pyNeighborhood
is small and easy to install.


apt-get install pyNeighborhood

You should be able to browse the network by navigating through the tree
on the left.

By default pyNeighbourhood needs root privilages in order to mount shares
automatically under the default directory (which is /mnt/).
If you try and mount a partition without root privilages you will get
a "failed to mount" error message.

NOTE: If you wish to see exactly why pyNeighbourhood didn't work, run it in debug mode via
the command line:


 pyNeighbourhood --debug

The first thing to do to fix this is to create a directory writable by your user and
set pyNeighbourhood to use that as its root mount point.

Edit -> Preferences - Mount Point

Once this is done, you need to allow your user access to smbmnt and smbumount. If you dont do this
you will get this error (in --debug mode).


 smbmnt must be installed suid root for direct user mounts (1000,1000)
 smbmnt failed: 1
 Failed to mount

So to allow access run this:


 sudo chmod +s /usr/bin/smbmnt
 sudo chmod +s /usr/bin/smbumount

When you share your files, pyNeighbourhood will complain that mc isn't avaiable. This is not compulsary,
but if you have it install you can use it to browse your files.


 apt-get install mc  # File browser

Command Line

    smbtree - lists avaiable samba servers

Mounting a samba share in your filesystem is very easy; Ive provided an example below.
NOTE: UBUNTU is the name of the server, upload is the folder on that server,
and /shares/ubuntu is the directory in which it will be mounted.


 mount -t smbfs //UBUNTU/upload /shares/ubuntu

If you have your shares setup with passwords you will need to append this line with


 -o username=mywindowsusername,password=mywindowspassword

If you need this directory to be permanenty mounted, add it into
your /etc/fstab


 //servername/sharename /mountdirectory smbfs username=windowsuserename,password=windowspassword 0 0

Finally, creating a symbolic link on your desktop to your root share directory can be very handy.


 ln -s /shares/ /home/ubuntu/Desktop/shares

For more complete information on pyNeighbourhood check out this site:

Samba Shares

And for more info about putting samba shares into fstab:

Samba & Fstab

Trac Adminstration - All

2007-06-21T15:05:00.000-07:00

Create Project :
trac-admin initenv /my/new/project

Essential Plugins:

Pygments ( java, C, xml highlighting)

project/conf/trac.ini - Trac Configuration

To setup trac to send email alerts smtp must be enabled, and the from address must be vaild (actually ALL the email fields must be valid, so be careful):


[notification]
smtp_enabled = true
smtp_from = trac@no-reply.com

Attachment Size:


[attachment]
max_size = 262144



Running Standalone (Runs all projects is /usr/trac/trac_projects using password file users.htdigest)

tracd -p 8080 --auth *,/usr/trac/trac_projects/users.htdigest,mydomain.com -e /usr/tr
ac/trac_projects

 

SVN Upgrade script: (This script updates your svn to the latest version, then updates all your projects)

#!/bin/bash

svn update /trac_install/trac
python /trac_install/trac/setup.py install
trac-admin /trac_projects/MyProject upgrade # NOTE: Needs to be run on every project

Fedora 7 Upgrade - Fedora 6

2007-06-21T04:47:00.000-07:00

Upgrading from Fedora Core 6 (64bit) to Fedora 7 (64bit) was fairly straightfoward, apart from removing alot of packages.

Following the advice given here:
Fedora Core 7 Update

Running this command, to switch your repo's:

rpm -Uhv ftp://download.fedora.redhat.com/pub/fedora/linux/releases/7/
Fedora/i386/os/Fedora/fedora-release-*.noarch.rpm

Gave me this error:

fedora-release = 6 is needed by (installed)
yum-fedorafaq-6-2007.02.03.noarch

Which can be resoved by uninstalling yum-fedorafaq (

yum remove yum-fedorafaq

After issuing a:

yum upgrade

I then had various dependency problems. To resolve them I had to remove these packages:

* gaim
* livna-config-display
* kmod-nvidia
* xorg-x11-drv-nvidia
* eclipse-jdt
* mkinitrd

Uninstalling mkinird meant these packages also had to be removed:

* compiz
* fuse
* gnome-session
* gnome-volume-manager
* kernel
* mkbootdisk
* ntfs-3g
* pcmicautils

After removing these packages I was then able to run yum upgrade.

WARNING: Do NOT reboot until you have reinstalled these packages after the upgrade!!

At the end of the upgrade, i was presented with this error:

Transaction Check Error:
file /usr/bin/gcore from install of gdb-6.6-15.fc7 conflicts with file from package gdb-6.5-15.fc6
file /usr/share/info/gdb.info.gz from install of gdb-6.6-15.fc7 conflicts with file from package gdb-6.5-15.fc6
file /usr/share/info/gdbint.info.gz from install of gdb-6.6-15.fc7 conflicts with file from package gdb-6.5-15.fc6
file /usr/share/info/stabs.info.gz from install of gdb-6.6-15.fc7 conflicts with file from package gdb-6.5-15.fc6
file /etc/esd.conf from install of esound-libs-0.2.38-1.fc7 conflicts with file from package esound-0.2.36-3
file /usr/share/man/man1/esd.1.gz from install of esound-0.2.38-1.fc7 conflicts with file from package esound-0.2.36-3

Error Summary
-------------

yum remove gdb esound

# WARNING: This will cause a lot of packages to be removed. Make sure you keep a note of them so you can reinstall them later.

yum upgrade # should complete now.

yum install groupinstall Base # reinstall compulsary components

Nvidia drivers

# update to the correct livna repo:

rpm -ivh http://rpm.livna.org/livna-release-7.rpm

# if your already using the livna repo, updating will probably give you this error:

[root@localhost fedora]# rpm -ivh http://rpm.livna.org/livna-release-7.rpm
Retrieving http://rpm.livna.org/livna-release-7.rpm
Preparing... ########################################### [100%]
file /etc/yum.repos.d/livna.repo from install of livna-release-7-2 conflicts with file from package livna-release-6-1

# In which case you should run this first:

yum remove livna-release

#

yum install kmod-nvidia

=============================================================================
Package Arch Version Repository Size
=============================================================================
Installing:
kmod-nvidia x86_64 100.14.09-1.2.6.21_1.3228.fc7 livna 2.5 M
Installing for dependencies:
livna-config-display noarch 0.0.12-1.lvn7 livna 55 k
xorg-x11-drv-nvidia x86_64 100.14.09-2.lvn7 livna 5.6 M

Transaction Summary
=============================================================================
Install 3 Package(s)
Update 0 Package(s)
Remove 0 Package(s)

livna-config-display

# All going well you should have this somewhere in /etc/X11/xorg.conf:

Section "Device"
Identifier "Videocard0"
Driver "nvidia"

# If you restart x (ctrl-alt-backspace) and it wont load properly, change that driver line to:

Driver "nv"

Linux to Linux Remote X Session - All

2007-06-17T19:06:00.000-07:00

This should work with any two computers running X-Windows.

The easy way

On your client machine, run xhost +server_address. i.e


xhost +192.168.1.34

Within the same shell, connect to the remote pc, via ssh -X user@server_address. i.e


ssh -X admin@192.168.1.34

Now run your application.

The hard way (manually)

xhost +server_address. ie.


xhost +192.168.1.34

ssh -X user@server_address. i.e


ssh -X admin@192.168.1.34

Then, export your display, using export DISPLAY=client_address:0.0
i.e


export DISPLAY=192.168.1.5:0.0

Then run your application.

FreeNX: Ubuntu 6.06

2007-06-17T16:24:00.000-07:00

Description: Setting up FreeNX client on Ubuntu 6.06.

Homepage

This article here lists how to do it:

Forums
Here is a summary...

Add this to sources.list:

deb http://seveas.imbrandon.com dapper-seveas freenx
deb-src http://seveas.imbrandon.com dapper-seveas freenx

Then add your key and update

wget http://seveas.imbrandon.com/1135D466.gpg -O- | sudo apt-key add -
sudo apt-get update

now you should be able to install "freenx" and "nxclient"

Note: The install isnt really complete: theres no symblink in bin or any other folder on path, and no icon is put in gnome.

The client program is here...(dont run as root):



/usr/NX/bin/nxclient

Now you should be able to login. My system is currently comming up with no icons though....

Description: Free NX - 64bit Ubuntu Fiesty

Aparently this cannot be done, and you will get this error:

PHP & PostgreSQL - All

2007-06-15T00:50:00.000-07:00

Heres a quick guide to getting PHP going with PostgreSQL.

Ive listed a few useful functions below.

Database Connection:

 $DB_HOST = "192.168.1.10";
$DB_USER = "database_user";  // login username
$DB_PASS = "donthackme";     // login password

/** Create DB connection and returns connection ID. */
function getDBConnection() {

global $DB_HOST,$DB_NAME, $DB_USER,$DB_PASS;

$id = pg_connect("host=" . $DB_HOST . " dbname=" . $DB_NAME .
   " user=" . $DB_USER . " password=" . $DB_PASS ) or die('Could
   not connect: ' . pg_last_error());

 if (!$id) {
         die("getDBConnection : Could not connect -> "
         . pg_errormessage($id));
         exit;
 }
 return $id;
}

Queries:

/** Executes query on @param conn */
function executeQuery($conn, $query) {

 $resultRS = pg_query($conn, $query) or die('ExecuteQuery :
            Query failed: '. $query . ':' . pg_last_error());

 if (!$resultRS) {
         echo "ExecuteQuery :: An error occured : " . $query . "\n";
         exit;
 }

 return $resultRS;
}

Example Query:

executeQuery($conn, 'INSERT INTO "database"."tableName"
          ("tableID","name") VALUES (' . $id . ',\'a name\');'

Auto Increment:

Auto Increment works differently to MySQL. To set a
column to auto increment, you should
set the columns default value to the value of a sequence. i.e.

nextval('dbname.tablename_tableID_seq'::text) ,where
'dbname.tablename_tableID_seq' is a sequence.

/** Returns the next id number of an auto incremented field.
  $sequenceNumber should be in the format "tableName_tableID_seq"
  (i.e. dont include the database name. */
$DB_SCHEMA = "dbschema";

function getNextID($conn, $sequenceNumber) {

 global $DB_SCHEMA;

 $query = 'SELECT nextval(\'' . $DB_SCHEMA . '.' .
 $sequenceNumber . '\') AS id';
 $rs = executeQuery($conn, $query);
 $id = getFirstResult($rs);
 return $id;
}

/** Returns first result in the first row in
  resultset $rs. Returns -1 on fail. */

function getFirstResult($rs) {
      $ID = -1;

      if ($row = pg_fetch_row($rs)) {

              if ( $row[0] != NULL ) {
                      $ID = $row[0];
              }
      }

      pg_free_result($rs);
      return $ID;
}

Linux From Scratch 6.06 - Live CD

2007-06-07T15:30:00.000-07:00

Installing LFS 6.06

After trying to install LFS using Ubuntu 7.04, and running in to several problems, I decided to use the LFS live CD as my host system. If this is your first LFS install or are a Linux noob, I'd suggest you do the same. I have listed errors that I have had with both, and how I got around them.

Most of the problems I've had have been from following the instructions incorrectly.

There are a few things to keep in mind:

- Do NOT ignore any errors until you are satisfied they will not affect anything.
- Always check config.log for any errors during a build
- Unless otherwise specified make sure you unpack the source, cd to a DIFFERENT directory and run ../package-name/configure from that directory,
then make && make install.
- If you've had errors when trying to build, or are coming back to rebuild a package, make sure you run "make distclean", or "make clean" so that the package is completely rebuilt.

- If you restart your PC make sure you setup your chroot environment correctly again:

export LFS=/lfs/dir

mount -vt devpts devpts $LFS/dev/pts
mount -vt tmpfs shm $LFS/dev/shm
mount -vt proc proc $LFS/proc
mount -vt sysfs sysfs $LFS/sys

chroot "$LFS" /usr/bin/env -i \
HOME=/root TERM="$TERM" PS1='\u:\w\$ ' \
PATH=/bin:/usr/bin:/sbin:/usr/sbin \
/bin/bash --login

- the "which" command will tell you where the command is that your running
- find /boot/lfskernel-2.6.16.27 will tell you which partition your linux image is on

2. Preparing a new partition

This is fairly straight forward if you know how to use fdisk, mount, swap and mk2fs. If your struggling, turn to your friend google.

3. Packages

Ive listed problems and solutions below:

---------- Live CD problems ----------

Package: GCC

Error:

configure: error: Building in the source directory is not supported in this release. See http://gcc.gnu.org/install/configure.html for more details.

Solution:

create a new directory, and run configure and make from there

Package: BASH
Error:

/tools/bin/sh: error while loading shared libraries: libncursesw.so.5: cannot open shared object file: No such file or directory

Solution:

copy the ncurses libraries in manually. I think this may have been caused by installing a later version by accident.

cp /lib/libncursesw.so.5* /tools/lib/

Package: GCC
(http://www.linuxfromscratch.org/lfs/view/stable/chapter06/gcc.html)

Error:

In file included from ../../gcc-4.0.3/gcc/crtstuff.c:64:
../../gcc-4.0.3/gcc/tsystem.h:90:19: error: stdio.h: No such file or directory
../../gcc-4.0.3/gcc/tsystem.h:93:23: error: sys/types.h: No such file or directory
../../gcc-4.0.3/gcc/tsystem.h:96:19: error: errno.h: No such file or directory
../../gcc-4.0.3/gcc/tsystem.h:103:20: error: string.h: No such file or directory
../../gcc-4.0.3/gcc/tsystem.h:104:20: error: stdlib.h: No such file or directory
../../gcc-4.0.3/gcc/tsystem.h:105:20: error: unistd.h: No such file or directory
In file included from /build/gcc-build/build/gcc/include/syslimits.h:7,
from /build/gcc-build/build/gcc/include/limits.h:11,
from ../../gcc-4.0.3/gcc/tsystem.h:108,
from ../../gcc-4.0.3/gcc/crtstuff.c:64:
/build/gcc-build/build/gcc/include/limits.h:122:61: error: limits.h: No such file or directory
In file included from ../../gcc-4.0.3/gcc/crtstuff.c:64:
../../gcc-4.0.3/gcc/tsystem.h:111:18: error: time.h: No such file or directory
make[1]: *** [crtbegin.o] Error 1
make[1]: Leaving directory `/build/gcc-build/build/gcc'
make: *** [all-gcc] Error 2

Solution:
Patch gcc-4.0.3-specs-1.patch was applied to remove /usr/include from GCC's search. Remove patched version of GCC build, replace with original version and rebuild.

Package: GCC
Error: (make)

Make[3]: Leaving directory `/build/gcc-build/build/i686-pc-linux-gnu/libmudflap'
make[2]: Leaving directory `/build/gcc-build/build/i686-pc-linux-gnu/libmudflap'
make[1]: *** [check-recursive] Error 1
make[1]: Target `check' not remade because of errors.
make[1]: Leaving directory `/build/gcc-build/build/i686-pc-linux-gnu/libmudflap'
make: *** [check-target-libmudflap] Error 2
make[1]: Entering directory `/build/gcc-build/build/i686-pc-linux-gnu/libiberty'
make[2]: Entering directory `/build/gcc-build/build/i686-pc-linux-gnu/libiberty/testsuite'
make[2]: Nothing to be done for `check'.
make[2]: Leaving directory `/build/gcc-build/build/i686-pc-linux-gnu/libiberty/testsuite'
make[1]: Leaving directory `/build/gcc-build/build/i686-pc-linux-gnu/libiberty'

Solution:

make distclean and try again

Package: Kernel
Error: (clean / make)

root:/build/linux/linux-2.6.16.27# make mrproper
/bin/sh: sed: command not found
Makefile:524: /build/linux/linux-2.6.16.27/arch//Makefile: No such file or directory
make: *** No rule to make target `/build/linux/linux-2.6.16.27/arch//Makefile'. Stop.
root:/build/linux/linux-2.6.16.27# make menuconfig
/bin/sh: sed: command not found
Makefile:438: /build/linux/linux-2.6.16.27/arch//Makefile: No such file or directory
make: *** No rule to make target `/build/linux/linux-2.6.16.27/arch//Makefile'. Stop.

Solution:

- sed hasnt been installed correctly

reinstall sed.

I did this outside of the chrooted session, using this:
./configure --prefix=/mnt/lfs/usr --bindir=/mnt/lfs/bin &&amp;amp;amp;amp;amp; make && make install
check to see if sed is now in /bin/

Package: Grub
Error: (configuring)

grub> root (hd0,3)
Error 21: Selected disk does not exist

Solution:

/dev/ devices did not exist.

reinstalling lfs_bootscripts (while NOT chrooted) will populate /dev/
and /proc

Tip:

- find /boot/lfskernel-2.6.16.27 will show you if grub can find your linux image.

Notes:

"Filesystem type is ext2fs, partition type 0x83" will be displayed even if your partition is ext3

Package: Grub
Error: (running)

error 15 file not found

Solution:

Make sure you have spelt your linux image version correctly in /boot/grub/menu.lst

-------------------- Ubuntu 6.06 errors --------------------

Note: I didnt complete this install, so what I have suggested is quite probably wrong

Package: BinUtils
Error: (configure)

configure: error: installation or configuration problem: C compiler cannot create executables.

Check config.log in bin-utils directory... Most likely its says this:

configure:1938: gcc -o conftest conftest.c 1>&5
/usr/bin/ld: crt1.o: No such file: No such file or directory
collect2: ld returned 1 exit status

Solution:

apt-get install libc6-dev

Package: GCC
Error: (configure)

configure: error: installation or configuration problem: C compiler cannot create executables.

config.log:

/tools/bin/ld: unrecognized option '--hash-style=both'

Solution:

append CC compile flag to configure

CC="gcc -B/usr/bin" gcc-4.0.3/configure --prefix=/tools ...

Package: GCC
Error: (make)

make[1]: Entering directory `/mnt/lfs/build/gcc-build/gcc-4.0.3/build-i686-pc-linux-gnu/libiberty'
make[1]: *** No rule to make target `../include/ansidecl.h', needed by `regex.o'. Stop.
make[1]: Leaving directory `/mnt/lfs/build/gcc-build/gcc-4.0.3/build-i686-pc-linux-gnu/libiberty'
make: *** [all-build-libiberty] Error 2

Solution:

This is caused by building inside the source directory...just do it outside it and it will work fine

Package: GLIBC
Error: (make)

/build/glibc/csu/version.o.dt -MT /mnt/lfs/build/glibc/csu/version.o
In file included from version.c:33:
/mnt/lfs/build/glibc/csu/version-info.h:2: error: missing terminating " character
/mnt/lfs/build/glibc/csu/version-info.h:3: error: missing terminating " character
version.c:40: error: syntax error before string constant
make[2]: *** [/mnt/lfs/build/glibc/csu/version.o] Error 1
make[2]: Leaving directory `/mnt/lfs/build/glibc/glibc-2.3.6/csu'
make[1]: *** [csu/subdir_lib] Error 2
make[1]: Leaving directory `/mnt/lfs/build/glibc/glibc-2.3.6'
make: *** [all] Error 2

Solution:

modify /mnt/lfs/build/glibc/csu/version-info.h: and add a " character on line 2 & 3

LFS extensions

Package management (epkg):

http://www.encap.org/

My computer cannot seem to access the package repo, making it hard to test the install...will update soon.

Runlevel manager: runit (http://smarden.org/runit/)

By default LFS uses init. If you wish to experiment with something different, especially to see how run levels work (i.e what happens after the kernel is loaded) installing runinit is an excellent way of doing it.

Installation:

Follow the steps outlined here:

http://smarden.org/runit/replaceinit.html#sysv

Setting up the terminal script

Your getty terminal run script is contained here.
/etc/sv/getty-5/*

If you look at the run script within this directory, you'll see that it is set to execute /sbin/getty...this needs to be changed to /sbin/agetty.
It is also set to run on tty5. This means that when it starts you have to switch to virtual terminal 5 (ALT-5) in order to see the prompt.

Migrating the runlevels

LFS puts it inital run level scripts in /etc/rc.d/init.d/*. This needs to be translated into /etc/runit/1 . This is what I have so far:

/etc/rc.d/init.d/mountkernelfs start # start virtual file systems (proc)
/etc/rc.d/init.d/udev start # mount devices
/etc/rc.d/init.d/swap start # activate swap files
/etc/rc.d/init.d/checkfs start # check the disk
/etc/rc.d/init.d/mountfs start # mount the disk
/etc/rc.d/init.d/cleanfs start # clean tmp dirs
/etc/rc.d/init.d/setclock start # setup clock
/etc/rc.d/init.d/rc 3 # general runlevel script
/etc/rc.d/init.d/console start # configure console

to test, modify your grub and set init=/sbin/runit-init.
It should load the kernel as per normal, then kick into runit.
Remember you will need to switch to tty5 (ALT-5) in order to get to the terminal
after runit reaches run level 2.

TODO: Shutdown script (/dev/initctl not found)

Linux Applications - All

2007-06-05T20:28:00.000-07:00

# Applications

# Development Environments

netbeans # General IDE
eclipse # General IDE
ktechlab # PIC micro-controller IDE

# Development tools

trac # Bug tracking and project mangment (compatiable with svn)
trac - pygments # Syntax highlighting
svn # Code repository
svn-stats # SVN statistics generator
meld # diff viewer
vncviewer # vnc client
virtualbox # virtual machine application (like VMware and parallels)
quanta plus # web development tool

jad # Java decompiler (NOTE: get statically linked version for less hassles)
proguard # Java obfuscator and optimizer
firefox # Webbrowser
agave # Colour Scheme designer

# Backup

dar # Differential backup

# General

gaim # Chat client ( mutiprotocol)
amsn # Chat client (webcam)
mpc # music client
mantis # php bug tracking application

# Accounting

compiere

Incremental Backup - dar - All

2007-06-04T18:25:00.000-07:00

--- backup ---
dar

FLAGS:

-c /root/filename # backup filename (NOTE: extention and slice number are added by dar)
-R / # Backup directory. (root in this case)

-s 100M # Slice size is 100Megs
-S 100M # First slice 100 megs
-p -b # Pause flag; -b terminal bell
-z # gzip compression. -z1 bzip2 compression
-X "*~" -X ".*~" # Excluded files (re-flag each option)
-P dev/pts # Excude directory (re-flag each option; NOTE: must ALWAYS be relative to -R dir)
-D # Add excluded directories as empty directories

-A /file/backup # Creates differential backup from base backup specified

EXAMPLE:

dar -c /mnt/zip/linux_full -s 100M -S 95M -p -b -z -R / -X "*~" -X ".*~" -P dev/pts -P proc -P mnt/zip -D

TASKS:

dar -l backupfile # Lists backup file
dar -t backupfile # Check backup file
dar -d /path/backupfile -R /backup/path # Check dar "backupfile" against backup'ed filesystem

--- restoring ---

# Run dar binary file (self extracting), extracts
# to zip/linux_full, then restores to /disk

/disk/dar_static -x /zip/linux_full -R /disk -b

FLAGS:

-w # dont warn
-r # dont overwrite more recent files
-b # beep on prompt

# Additional files are done the same way

/disk/dar_static -x /zip/linux_diff2 -R /disk -b -w

--- Restore single files ---

# Restore fromdar -R / -x /zip/linux_diff29 -v -g home/denis/my_precious_file

# If file doesn't exist in this diff

dar -x /zip/linux_diff29 -R / -r -v -g home/denis/my_precious_file -g etc/fstab
sepcified diff file

Apache Tomcat & SSL - ALL

2007-05-08T18:46:00.000-07:00

To setup SSL in tomcat you have to create a certificate keystore by executing the following command:


%JAVA_HOME%\bin\keytool -genkey -alias tomcat -keyalg RSA

and specify a password value of "changeit".

This will generate a key file in your home directory. WARNING: If you load tomcat as a different user, it will display an error, and you will need to regenerate or move this key file into their home directory.

Uncomment the "SSL HTTP/1.1 Connector" entry in $CATALINA_HOME/conf/server.xml and tweak as necessary.

If your using Java version 1.3 or lower:

* Download the Java Secure Socket Extensions (JSSE) package, version 1.0.2 or later, from http://java.sun.com/products/jsse/

* Install JSSE extension by copying all three JAR files (jcert.jar, jnet.jar, and jsse.jar) into your %JAVA_HOME%/jre/lib/ext directory.

Tomcat - Publicly Listed Directory - All

2007-05-07T18:50:00.000-07:00

# To setup a directory that is browesable, create a new directory in your tomcat webapps folder. In this example I called it "public". You then need to create a directory within this, called WEB-INF. In this folder create a new file called web.xml as listed below.

<?xml version="1.0" encoding="ISO-8859-1"?>

<!DOCTYPE web-app
PUBLIC "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"
"http://java.sun.com/dtd/web-app_2_3.dtd">

<web-app>

<display-name>public</display-name>
<description>File Browsing Application for the Document Share</description>



<servlet>
<servlet-name>DefaultServletOverride</servlet-name>
<servlet-class>org.apache.catalina.servlets.DefaultServlet</servlet-class>
<init-param>
<param-name>listings</param-name>
<param-value>true</param-value>
</init-param>
<load-on-startup>1</load-on-startup>
</servlet>


<servlet-mapping>
<servlet-name>DefaultServletOverride</servlet-name>
<url-pattern>/</url-pattern>
</servlet-mapping>

</web-app>

(NOTE: Heres the link to display XML in a blog http://www.stanford.edu/~bsuter/js/convert.html)

# Restart tomcat. Tomcat Manager should now list a new application called "public".

Cygwin Remote Desktop - Ubuntu

2007-04-26T03:21:00.000-07:00

Server Operating System: Ubuntu 7.04
Client Operating System: Windows XP

#Two ways you can connect to a linux operating system remotely are VNC and using Cywin, which is a Windows ports of many GNU packages. As VNC is fairly easy to configure, I will explain how to get Cygwin to remotely connect to a remote X Windows (Gnome, KDE etc) server.

#Install Cygwin on the Client. It can be downloaded from here

#Install SSH package during the Cygwin install. It can be found in the category "net", and is called "openssh"

# Once installed, you can start a local xsession (in windows) via the command:


startx.

# You should now have a "X" icon on your taskbar. If you hover over the icon, it will tell what your display is running on; most properly "0.0". If this is the case, run this command (and substitue 0.0 with whatever you get):

export DISPLAY=localhost:0.0

# Tell xserver to allow connections from your remote server; substitue "remote_host" with your remote server's IP address.

xhost +remote_host

# Now you are ready to connect to your remote server using ssh. Install the ssh daemon on your remote server. In ubunu this is done via:

apt-get install openssh-server

# It should start automatically; but if it doesn't, run:

/etc/init.d/ssh start

# From your client (the Windows machine), connect to your remote server using SSH. This example assumes "user" is the username on the server, and 10.0.0.6 is the servers IP:

ssh user@10.0.0.6

# When connected, run this command, replacing 10.0.0.9 with the IP address of your client(the windows machine)

export DISPLAY=10.0.0.9:0.0

# Now if you run a xwindows application, it should appear on your clients machine.

Linux to Linux - Connecting to X Windows Remotely:

ssh -X # This is the easiest way

Sendmail - PLAIN / LOGIN authentction - Ubuntu

2007-04-25T16:00:00.000-07:00

Operating System Ubuntu 6.06

apt-get install sendmail

# By default, sendmail only allows access from localhost. Configuration files are in /etc/mail

# In access, you need to allow access to your network. In this case 192.168.X.X

Connect:192.168 RELAY
GreetPause:192.168 0
ClientRate:192.168 0
ClientConn:192.168 0

# In sendmail.mc you need to update these lines, and removed the Addr="" clause, so that it will receive from any addresses:

FEATURE(`no_default_msa')dnl
dnl DAEMON_OPTIONS(`Family=inet6, Name=MTA-v6, Port=smtp')dnl
DAEMON_OPTIONS(`Family=inet, Name=MTA-v4, Port=smtp')dnl
dnl DAEMON_OPTIONS(`Family=inet6, Name=MSP-v6, Port=submission')dnl
DAEMON_OPTIONS(`Family=inet, Name=MSP-v4, Port=submission')dnl

# Now if you run one of these...

# * `sendmailconfig`
# * `make`
# * `m4 /etc/mail/sendmail.mc > /etc/mail/sendmail.cf`

# And restart the service (/etc/init.d/sendmail restart) you should now be able to use your server.

# Settings up Authentication - Add these lines to sendmail.cf, if they dont exist:

define(`confAUTH_OPTIONS', `A')dnl
dnl #
TRUST_AUTH_MECH(`DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
dnl #

# Setting up users:

http://www.elwoods.org/home/2006/12/13/sendmail-ubuntu-yahoo-and-you/

CD burning Script - All

2007-04-12T15:06:00.000-07:00

# This is a simple script that will burn a cd via command line. You need mkisofs and cdrecord.

# The device your cdrom is assigned to can be found in /etc/fstab (hopefully)

# make iso. This makes a iso with label Backup of all files in ./backup_iso/
# Remember you can put symbolic links in this directory.

mkisofs -f -R -r -l -J -V1 -ABackup -o backup.iso ./backup_iso/

# Find out what your device is called, and look for devname: ''

cdrecord -checkdrive

# record cd at 15 speed. My cdrom device is /dev/cdrw

cdrecord -v -pad speed=15 dev=/dev/cdrw backup.iso

# clean up

rm backup.iso

# NOTE: You can also burn an iso via:

growisofs -Z /dev/cdrw=filename.iso

Linux Command Line Apps - All

2007-04-11T23:19:00.000-07:00

Command Line Applications

If your Linux system is low on resources, or if you just feel like you've been away from the command line too long, here are a few command line applications you can use.

Music Player (mp3 etc)

mpd (server) ncmpc (client) http://www.musicpd.org/.
mplayer

Msn Client

tmsn Ncurses based Msn client.

Bittorrent Client

btdownload (or btdownloadcurses). Note: if btdownload runs slowly, it may be because your running in passive mode (i.e you haven't allowed incoming connections. You can do this by opening up incomming port(s) on your router (and firewall), and specfying the --minport flag. btdownload will now test each port going up from --minport. NOTE: Each new instance of btdownload will need its own port.

FTP Client

yafc Easy to use FTP client. Connection is in the format ftp://user:password@myftpserver.com. Other commands are very similar to standard linux commands (i.e ls, cd). Download and upload files via get and put respectively.

MySQL Adding Auto Increment Column to existing table

2007-04-11T18:06:00.000-07:00

This alters table "tableName" , and adds a new column "autoIncrementColumn".


ALTER TABLE tableName ADD autoIncrementColumn MEDIUMINT NOT NULL AUTO_INCREMENT KEY

This modifies an existing column to be auto increment.


ALTER TABLE tableName MODIFY tableNameID MEDIUMINT NOT NULL AUTO_INCREMENT;

This sets the next auto increment value to 10. Useful if you already have data in the table that you are modifying.


ALTER TABLE tableName AUTO_INCREMENT = 10;

This modifies "myColumn" to have a default value of 0.


ALTER TABLE myTable ALTER COLUMN myColumn SET DEFAULT 0;

Enabling Core 2 Duo Support - Ubuntu

2007-04-07T17:48:00.000-07:00

Operating System: Ubuntu 7.04 Beta

# After installing Fiesty on a Core 2 Duo machine, I noticed that Ubuntu's performance was poor. After installing the kernel I see that by default the kernel is compiled for the x68 instruction set (for compatibility reasons I assume), and does not have SMP (i.e. hyper threading) enabled . This isn't good! We have to get our hands dirty for this one; recompiling the kernel is required.

# You need the kernel source. Get it via apt-get linux-source, or via manually at http://packages.ubuntu.com/, Once this has run, you should have a .tar.bz2 file in /usr/src. Extract this here:

bunzip2
tar -xvf

# Create linux symbolic link to the newly created directory (while in the /usr/src directory)

ln -s /usr/linux/ linux

# all the tools required for installing the kernel:

apt-get install build-essential
apt-get install kernel-package
apt-get install gcc-3.4
apt-get install libncurses5

# Configure

make menuconfig

# You need to enable these options in menuconfig:

Processor Type and Features -> SubArchitecture Type -> Support for ScalarMP vSMP
Processor Type and Features -> Intel Core2
Processor Type and Features -> Symmetric multi-processor support
Processor Type and Features -> SMP (Hyperthreading) Scheduler support
Processor Type and Features -> Multi-Core Scheduler support

# Now use ubuntu to create a .deb install. Note smp12 is the revision name; you can call it what you wish, but it must contain a number.

/usr/src/linux# make-kpkg -initrd --revision=smp12 kernel_image kernel_headers modules_image

# When this has finished, install the package that has been created (i.e)

dpgk -i

# After reinstalling any kernel you will need to reinstall any video driver patches (i.e. Nvdia) or X windows will complain.

NTFS Read / Write Support - Fedora

2007-04-07T17:15:00.000-07:00

Operating System: Fedora 6

Linux now supports NTFS read/write support using Third Generation NTFS Driver. To install:

yum install ntfs-3g

# Find your ntfs partition

/sbin/fdisk -l
....
/dev/sda5 7649 30400 182755408+ 7 HPFS/NTFS
....

# Mount partiton to the /mnt/ntfs directory:

ntfs-3g /dev/sda5 /mnt/ntfs

# You should now have read/write support to your NTFS partition.

Enabling divx, mp3 and other restricted codecs - Ubuntu

2007-04-04T16:47:00.000-07:00

Ensure the relevant repositories are enabled. Click System → Administration → Synaptic Package Manager → Settings → Repositories and then click Add. Check the Community maintained (Universe) and Non-free (Multiverse) boxes. When you close the window, click Reload.
Install the packages. While you could install packages individually using Synaptic, here is one case where any Ubuntu user can save a lot of time by using the command line. Quit out of Synaptic, then click Application → Accessories → Terminal and install all these packages:

sudo apt-get install

Edgy:

gstreamer0.10-pitfdll gstreamer0.10-ffmpeg gstreamer0.10-plugins-bad gstreamer0.10-plugins-bad-multiverse gstreamer0.10-plugins-ugly gstreamer0.10-plugins-ugly-multiverse gxine libxine-main1 libxine-extracodecs ogle ogle-gui

Feisty:

apt-get install gstreamer0.10-plugins-ugly gstreamer0.10-ffmpeg gstreamer0.10-plugins-bad

Gusty:

sudo apt-get install gstreamer0.10-ffmpeg gstreamer0.10-plugins-bad gstreamer0.10-plugins-bad-multiverse gstreamer0.10-plugins-ugly-multiverse gxine libxine1-ffmpeg ogle ogle-gui

NOTE: If you still dont hear sound your volume is probabaly not set correctly; use alsamixer
 to set it up correctly (Press "M" to unmute a channel). For some reason sound will not work at all if I have Duplicate Front muted.

Gaim Google Talk Settings - All

2007-04-04T16:14:00.001-07:00

Operating System : All

Protocol: Jabber
Screen Name: Full Gmail username (no @gmail.com)
Server: gmail.com

--Advanced --

“Use TLS if available”
Port: 5223
Connect server: talk.google.com

Command Line Bittorrent - Ubuntu

2007-03-31T18:57:00.000-07:00

Operating System Ubuntu 6.10

# To install Bittornado, run

apt-get bittornado

# All bittornado commands are prefixed with "bt"

# To start downloading a torrent:

btdownloadcurses.bittornado <.torrent url>

Installing Nvidia Drivers - Ubuntu

2007-03-31T17:03:00.000-07:00

Operating System: Ubuntu 6.10 

# Install these modules:

sudo apt-get install nvidia-glx nvidia-kernel-common

# This script automatically updates your X11 (display) configuration.

sudo nvidia-xconfig

Should the above not enable the new driver, you can enable it manually by opening the X config file:

sudo gedit /etc/X11/xorg.conf

and replacing "nv" with "nvidia" in the driver section :

Section "Device"
Identifier "Videocard0"
Driver "nvidia"
Option "AddARGBGLXVisuals" "True"
EndSection

Restart gnome ( Ctrl - Alt - Backspace)

# If Gnome doesn't load correctly, you can check your logs in /var/log/Xorg.*.log (* will be a number) or any errors. Check for any lines with (EE) , as this is an error. (i.e. more /var/log/Xorg.0.log | grep "(EE)"). If you change your driver line back to "nv", gnome will use the default driver, so you can load Gnome again while you fix the problem.

Screen resolution

If you wan to change your screen resolution and it is not listed under Settings -> Display Settings, you can configure your Screen device via


sudo displayconfig-gtk

This will bring up a Screen and Graphics preferences configuration window. It doesn't seem to be listed on any menu. NOTE: you may need to restart X after you've finshed modifying things.

64Bit Linux (FATAL: Error running install command for nvidia) error:

Details

After a kernel update when running 64bit Ubuntu, the kernel wasn't patched properly. When logging into gdm I got the usual nvidia module errors. Doing a modprobe gave me this error:


$ sudo modprobe nvidia
FATAL: Error running install command for nvidia

Fix

Firstly, to find out if the drivers are installed, check to see if nvidia.ko exists:

$ find /lib/modules/`uname -r` -name "nvidia.ko" -exec ls -la {} \;
-rw-r--r-- 1 root root 8644254 2007-06-13 07:49 /lib/modules/2.6.20-16-generic/kernel/drivers/video/nvidia.ko

If the second line comes up, your nvidia driver exists. If not, manually install your drivers, replacing --kerenel-source-path with whatever kernel you wish to use (NOTE: Always try using the package manager first, that way updates will be installed automatically)

sh NVIDIA-Linux-x86_64-1.0-9755-pkg2.run --kernel-source-path=/usr/src/linux-headers-2.6.20-16-lowlatency

By now you should have your nvidia drivers installed; issue the previous find command to verify. Now, try modprobe nvidia again. If this still fails, remove the file below.

rm -f /lib/linux-restricted-modules/.nvidia_new_installed

If this still doesn't work try removing the line referencing nvidia in /etc/modprobe.d/lrm-video

ASUS Z8100 Laptop Wireless - Fedora

2007-03-28T20:07:00.000-07:00

# Operating system: Fedora Core 6
# After battling with ndiswrapper, i found a driver that supports the ASUS USB2.0 wireless adapter (zd1211).

# Development wiki is here:

http://zd1211.ath.cx/wiki/

# The driver is installed as a module using the standard install kernel, so to get it working all thats required is

modprobe z1211 # Load module
ifconfig eth1 up # Bring up adapter

# Once your adapter has come to life, you can see it using
iwconfig eth1

# You can then scan for networks using

iwlist eth1 scan

# From here you can see your access points ESSID. You need to set your adapter's essid to the same as your access points:

iwconfig eth1 ESSID

# Start DHCP

dhclient eth1

# To configure your adapter to use WEP or WPA, use wpa_supplicant