I recently decided that I my half hearted backups to an external drive wasn’t really good enough, so I started to look at alternatives.

Note, this howto is for Mac and *nix system, if you need a windows version please try to google it.

I tried Crash plan, considered a extended dropbox plan but none of them seemed like the correct solution (still I love dropbox, its a great app). I ended up registering an Amazon S3 account.

After a quick google search I found this brilliant little blog post “Using Amazon S3 for personal backups” which describe how to use jetS3t to semi automate the synchronization of the iPhoto library on a Mac OSX.

I am not going to repeat the instructions found on the “Using Amazon S3 for personal backups” blog post, I did however have to do a few minor tweaks to get it to work, I also extended the functionality slightly, to enable a fully automated backup solution.

JetS3t has had a few releases since the “Using Amazon S3 for personal backups” article was written, I downloaded jets3t-0.8.1.zip. Adding the link of the jetS3t directory to the path in .profile didn’t work for me (I’m on Mac osX 10.6), so I added it to my .bash_profile file. You can find .bash_profile in /Users/YOUR-USERNAME/, you’ll have to replace YOUR-USERNAME with your proper username of course. Other than that I followed the examples in “Using Amazon S3 for personal backups”.

Once that was in place and worked properly I started to add some custom scripts.

1. Create a scripts directory in your home directory
2. Create a logs directory in the scripts directory

2. Navigate to your scripts directory and create a amazon-s3-backup.sh file.

3. Use your favourite text editor to add the following code to the script

Replace YOUR-IPHOTO-BUCKET with the amazon S3 bucket you want to use, and replace YOUR-USERNAME with your username on your mac or *nix system. We pipe the output from the sync process to files in the logs directory, one log file for the iphoto and one log file for the documents process.

4. Edit your crontab. The crontab is a file where you can add files that you want to automatically execute on a regular basis.

5. Add the script to your crontab, I’ve chosen to run it once a week at 20:00 every Sunday, which is expressed by (0 20 * * 0).

6. Save your cronjob

There you go, now you’ll backup your iPhoto library and Documents every Sunday at 20:00 as long as your computer is turned on and online. You can of course tweak when and how often you want to sync to Amazon S3, you can read more about scheduling cron jobs at wikipedia

Cassandra is a NoSQL distributed database developed by Facebook, it is built to handle huge amounts of data and to perform CRUD operations quickly. The Cassandra site’s strap line says:

“The Apache Cassandra Project develops a highly scalable second-generation distributed database, bringing together Dynamo’s fully distributed design and Bigtable’s ColumnFamily-based data model.

Thrift is also developed by Facebook and is a software framework for service development and is used as an interface to Cassandra. The Thrift page site says:

“Thrift is a software framework for scalable cross-language services development. It combines a software stack with a code generation engine to build services that work efficiently and seamlessly between C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, Smalltalk, and OCaml.”

Both Cassandra and Thrift are apache incubator projects.

Installing Cassandra

1. Download cassandra

Download from: http://incubator.apache.org/cassandra/.

2. Create and set the correct paths in the storage-conf.xml

You can find the storage-conf.xml file in your cassandra root directory /conf

My configuration file settings in storage-conf.xml:

Notice: You have to create all these directories for cassandra to run properly.

3. Set a log directory in the log4j.properties file

This file is found in the same directory as the storage-conf.xml

4. Check that you are running java 6 as default

If you are running an earlier version of java you will have to change your version. Java 6 should already be installed on your mac if you keep your os in sync with the automatic updates from apple. You can change your java version by using the “Java Settings” application located in your /Application/Utilities directory.

5. Starting Cassandra

You should be ready to go now, navigate to the root directory of your cassandra installation and start cassandra by typing:

If you dont see any error messages cassandra is probably running as it should, so it is time to test it out.

Cassandra comes with a CLI interface which allowes you to do simple queries to the database. Notice that the CLI interface is not not as powerful as the thrift interface. You can for instance not execute get queries in Super Columns, those queries will create a java exception.

To test the CLI interface, run the following command from the cassandra root directory:

Inserting values to the keyspace:

Retrieving data from the keyspace:

Installing Thrift

Update: I found this manual after I had installed thrift: http://wiki.apache.org/thrift/ThriftInstallationMacOSX, using this install guide will probably fix the issues I had with compiling thrift.

1. Download Thrift

Download from http://incubator.apache.org/thrift/download/ and extract it.

2. Check that you have installed the following:

• g++ 3.3.5+
• Runtime libraries for lex and yacc might be needed for the compiler.
• boost 1.33.1+ (1.34.0 for building all tests) http://www.boost.org/.

I had to install boost manually:

Notice: the boost installation might take a while, It took about 5 – 10 minutes on my Macbook PRO (2.53GHz).

You can see the full requirements for thrift at http://wiki.apache.org/thrift/ThriftRequirements.

3. Start the installation

This ended up in an error message for me:

To fix this I had to copy my pkg.m4 file from /opt/local/share/aclocal/pkg.m4 to my thrift/aclocal directory.

Navigate to your thrift root directory:

Thanks to http://aaronspotlatch.appspot.com/archive/Jul-2008 and
http://qslack.com/post/thrift-macosx-104 for pointing me in the right direction.

You should now be ready to run make

and

You should now be able to run thrift on your mac.

You should now be ready to build your amazing application with Cassandra if both your installation of Cassandra and Thrift were successful.

I will try to post another blog post shortly on using Cassandra, Thrift and PHP. Stay tuned.

Resources

• The Cassandra Project
• The Cassandra Wiki
• The Thrift Wiki
• http://jetfar.com/installing-cassandra-and-thrift-on-snow-leopard-a-quick-start-guide/
• Setting up a Cassandra cluster
• http://www.onemanclapping.org/2009/11/building-cassandra-thrift-bindings-on.html

Today I started to work on a little project that required a crawler, and Nutch seemed to do most of what I needed. The nutch team conveniently released Nutch 1.0 late in March 2009, so I had a brand new release to test out. Installing nutch 1.0 on a mac is not as straight forward as I thought, I ran into a lot of unexpected issues and here is my cook book description of how to successfully install nutch 1.0 on your mac.

1. Download the latest source code from the Apache SVN repository http://svn.apache.org/repos/asf/lucene/nutch/. I tried running it from the tarball without success, I also tried to compile the source from the tarball, but a post on the nutch forum clearly states that this will not work.
2. Set your JAVA_HOME and NUTCH_JAVA_HOME variables, again this is not straight forward, they both need to point to your real installation of Java 1.6 (earlier versions of Java will fail). I sat these variables to: /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home, I could not get the /Library/Java/Home symbolic link to work properly.
3. Compile the source code using Ant (I built it in Eclipse).
4. Setup your nutch configuration, by following the tutorial by Peter P. Wang
5. Run your first crawl with: ./bin/nutch crawl urls -dir crawl -depth 3 -topN 50

Most of the issues I encountered was related to the Java version and the fact that using /Application/Utilities/Java/Java preferences application do not really change the JAVA_HOME directory /Library/Java/Home properly. So make sure you have set both JAVA_HOME and NUTCH_JAVA_HOME, and that your OSX does not fool you when it pretend to be symbolically linking to the 1.6 installation.

Good luck.

In a previous post I wrote about my new life running on a MacBook Pro and OSX. It has now been over a month since I switch over to this unix hybrid, and I am quite liking it. It is very stable, I almost never turn off my mac, but I put it to sleep, this is working fine and my last reboot is over two weeks ago. I have also gotten used to some of the new weird keys on the keyboard and the shortcuts, but I am not yet as efficent on a mac as I am/were on ubuntu/windows. During the last few weeks I have discovered that Apple and other in most cases provide me with the applications I need, but not always, and here is the list of applications I really miss:

1. TortoiseSVN
Windows application which integrates itself with Windows Explorer and provide a SVN client. I would say that this is the best graphical SVN client I have ever used. SVNX which I currently use on the mac is not a very good replacement.

2. Kate / Notepad++
Kate is a KDE text editor for unix based systems. Notpad++ is Kates equivalent on Windows. Both editors provide a simple and intuitive user interface, and a lot of syntax highlight files for all the obscure programming languages you can think of.
I know Mac have the TextMate application, but that is third party software and you have to pay €48 or something for a license, and that is probably what I probably will do. The TextMate application is really good and provide most if not all the functionality that Kate and Notepad++ provide.

In my desperation for a good text editor I almost went off and tried to install KDE on Mac, but that was said to be experimental and could break my entire system. So that is a no go for now. The article however was really interesting:

URL: http://www.simplehelp.net/2007/07/22/how-to-install-kde-4-in-os-x/.

I still miss my ubuntu system and will probably go off and install parallels or vmware and ubuntu, just to have it accessible

At Orange Bus we all work on MacBook Pro, and when I started up at Orange Bus it was the very first time I used a Mac and OS X. It is now 12 days since that first experience with an Apple computer and though I am still a bit unfamiliar with certain features I am getting in to it, and I really like it. The stuff that I have most problem with are:

The keyboard
I have some trouble finding the keys and key combinations I am used to from both Ubuntu and WinXP. And it probably does not help that the keyboard on the laptop is English, and my external keyboard at work is Norwegian (I was thinking it would be a good idea to bring a Norwegian keyboard to the UK, yeah right).

Changing windows
Moving between different set of windows and the different set of their instances are still a bit unfamiliar. In particular moving between a set of windows of the same application.

Universal access
Universal access is a piece of software which enhances for instance the readability on the Mac, and the key command to activating the speach on the mac must be some command I am used to from Windows or Ubuntu. I am always turning it on, very annoying.

That was the “negative” part about Mac, but that is just things I need to learn. Now let us have a look at the positive stuff in OS X.

Simplicty
I have heard it a million times by mac users, it is so easy to use a mac but I have never really believed them. But they are correct. Using a mac is extremely simple, Apple must have some of the best UI designers in the world. Everything is so amazingly simple. And another thing you just have to love in OS X is how you install software. You open the install file and drag it into the Application folder, and woosh, the application is installed. I would go as far to say that if you are a novice computer user, Mac would probably be the best choice. Lets compare it to moving around. Using a mac is not much harder than walking, in comparison to Microsoft Windows which would be something like controlling a space shuttle.

Flexibility
Well I have been praising Mac for their simplicity right now, but even though it is very easy to use you still have the possibility to do advanced stuff. For instance, since Mac is based on Unix you have a terminal easily accessible and you can use almost all your regular commands

Mail
Mail is a Mac application for managing….. yes you are right, mail. Well Mail is a okay application it works fine, but it is not as good as Microsoft Outlook, but it is yet another example of how simple and intuitive an Apple product can be.

Quicksilver
Quicksilver is a third party software for easy launch of applications. Type “ctrl” + “space” and then start typing the name of the application you want to launch, it automatically find the application and launches it when you hit enter. Brilliant!

At this point I can not see one good reason for going back to MS Windows. I am still very fond of Linux distros like Ubuntu, but Mac and OS X have really impressed me. So I suppose I have to line in with all the other Mac fanatics now (no I am not a mac fanatic, at least not yet).