Kristian Lunde

I recently decided that I my half hearted backups to an external drive wasn’t really good enough, so I started to look at alternatives.

Note, this howto is for Mac and *nix system, if you need a windows version please try to google it.

I tried Crash plan, considered a extended dropbox plan but none of them seemed like the correct solution (still I love dropbox, its a great app). I ended up registering an Amazon S3 account.

After a quick google search I found this brilliant little blog post “Using Amazon S3 for personal backups” which describe how to use jetS3t to semi automate the synchronization of the iPhoto library on a Mac OSX.

I am not going to repeat the instructions found on the “Using Amazon S3 for personal backups” blog post, I did however have to do a few minor tweaks to get it to work, I also extended the functionality slightly, to enable a fully automated backup solution.

JetS3t has had a few releases since the “Using Amazon S3 for personal backups” article was written, I downloaded jets3t-0.8.1.zip. Adding the link of the jetS3t directory to the path in .profile didn’t work for me (I’m on Mac osX 10.6), so I added it to my .bash_profile file. You can find .bash_profile in /Users/YOUR-USERNAME/, you’ll have to replace YOUR-USERNAME with your proper username of course. Other than that I followed the examples in “Using Amazon S3 for personal backups”.

Once that was in place and worked properly I started to add some custom scripts.

1. Create a scripts directory in your home directory
2. Create a logs directory in the scripts directory

2. Navigate to your scripts directory and create a amazon-s3-backup.sh file.

3. Use your favourite text editor to add the following code to the script

Replace YOUR-IPHOTO-BUCKET with the amazon S3 bucket you want to use, and replace YOUR-USERNAME with your username on your mac or *nix system. We pipe the output from the sync process to files in the logs directory, one log file for the iphoto and one log file for the documents process.

4. Edit your crontab. The crontab is a file where you can add files that you want to automatically execute on a regular basis.

5. Add the script to your crontab, I’ve chosen to run it once a week at 20:00 every Sunday, which is expressed by (0 20 * * 0).

6. Save your cronjob

There you go, now you’ll backup your iPhoto library and Documents every Sunday at 20:00 as long as your computer is turned on and online. You can of course tweak when and how often you want to sync to Amazon S3, you can read more about scheduling cron jobs at wikipedia

Packt publishing contacted me couple of months back, asking me if I was interested in reviewing “PHP5 Social Networking” by Michael Peacock. The title intrigued me as I’ve built several social networking applications and I was interested in seeing Michaels approach to this.

The first chapter starts by defining social networking, it looks at the most known social networks out there. It also reviews available social network software out there such as Elgg, drupal and joomla. The approach taken in this book is to roll out an own application from scratch with no base framework. The basics of this framework is described in chapter 2 and 3.

Chapter 4 to 10 deals with the core elements of social networks, such as friends, friend requests, statuses, streams, messaging and events. Each chapter describe how to the functionality should work, a detailed implementation plan is shown and ideas for further development is discussed.

Chapter 11 show the reader how the application easily can support an API, I really like that Michael chose to add this chapter as I think it is a crucial element to any social network that is going to go somewhere.

Maintenance, security and deployment is discussed in chapter 12. I find this chapter a bit odd as it describe very basic steps to get the application up and running on the production server. I would have liked to see more about automated deployment in this chapter.

All social network sites need to get users and hopefully generate some revenue, this is the topic of chapter 13 which describe solutions and approaches to marketing, SEO techniques and monetization.

The last chapter is “Planning for growth” a very important chapter in my mind. It describes approaches for profiling of web applications, server scaling, redundancy and more.

It is a well written book with good examples and it describes the build of the application in detail so the user easily can follow and copy the examples. I would definitely recommend this book to novice developers as it is a good introduction to a lot of good programming practices. It can also be a decent book to read if you are an experienced developer that want to learn more about the core concepts of building a social network.

Packt Publishing has asked me to review Michael Peacocks “PHP 5 Social Networking” something I’ve happily agreed to do. I am expecting to receive the book soon and look forward to read it. Social networks and PHP is something I’ve worked quite a bit with and I am interested to see Michaels take on this topic.

I’ll be posting the review as soon as I’ve finished the book.

Say hi to Lunde Development.

The 3rd of August 2010 was the first official day of my own company, Lunde Development. I had at that point already been working with the company ideas and proposals for a while and was able to hit the ground running with my first project already secured. Since then I’ve had several cool clients such as hoverState a high-end US based digital agency, the Newcastle based software company Band-X and Sharpe Recruitment a recruitment agency in Newcastle. You can read more about these projects on my portfolio page.

The future looks bright for Lunde Development and I enjoy every second of running my own business, it is challenging, I learn so much and most important of all, it is fun.

Please check out http://www.kristianlunde.com.

You know where to go if you need a new web site, a piece of software or consultancy

The Zend_Input_Filter is a very useful tool when you need to validate and filter the input to your application. It allows you to both filter and validate the input without a lot of hassle. One of the cool features it has is that it allows you to add the validators you need and meta commands to each validator. For instance you can set an Alnum validator to allow empty fields, set a default text and so on.

I came over this annoying issue the other day when I tried to setup the Alnum validator to allow white spaces and have a few meta commands attached to the validator chain. The manual says that you can do this:

Which I assumed would also work like this:

This does not work though. You need to remove the validator type string and replace it with an instance of the Zend_Validate_Alnum validator to get it to accept whitespaces and meta commands. This is the right way to do it:

The entire script ends up looking like this:

There is a lot of things going right now. We (@ramonahz, my daughter and myself) have decided to move back to Norway. We have thought about this for a while now and finally made the decision a few weeks ago. The main reason for moving back is that we want to be closer to our family, and not restrict our daughter to only see her grandparents a few times a year.

The move has affected my work slightly, I have resigned from my position at MySapient LTD and are now looking to do some freelance work from Norway. I also have a few other interesting things coming up, but I’ll leave that for a later blog post.

We have already started the process of moving back, and we’ll be leaving Newcastle and the UK on the 26th of July. We have spent almost 2 years in Newcastle, I have met some amazing people during these past two years, made some good friends and worked for two really cool companies. I am really going to miss living here, but hey, I’ll be back to visit soon.

Arrivederci Newcastle.

I encountered a rather strange issue on one of MySapients CentOS 5 servers the other day, the server seemed to run fine, without any issues what so ever, but it seem to run very low on memory. In fact it only had 150MB memory available of a total 4GB.

This obviously caused me a bit of a worry, but after some digging around I found the answer, linux is borrowing unused memory to do disk caching. Memory is automatically reassigned when the server needs to allocate memory to other services, in other words there is nothing to worry about.

There is a good site about this describing this functionality in more detail, you can read it at:

http://www.linuxatemyram.com/

Yesterday I installed the openbook plugin for wordpress on this site. My intentions was to build a “My bookshelf” page similar to what Mats Lindh and Trond Huso have on their sites.

I installed the plugin without any problems but the layout was a bit nasty. I don’t know if that was because of the theme I am running on this site or if the openbook markup was messed up. Anyway I rewrote the html and css for the openbooks template on its configuration page and added some custom css to the site to get it working.

This is my openbook template:

This is the css I added:

At last I added this custom html in the “my bookshelf” page:

You can see the result here: My Bookshelf .

This is a great article describing super columns and the general structure of cassandra. A must read for NoSQL newbies.

http://arin.me/code/wtf-is-a-supercolumn-cassandra-data-model

Cassandra is a NoSQL distributed database developed by Facebook, it is built to handle huge amounts of data and to perform CRUD operations quickly. The Cassandra site’s strap line says:

“The Apache Cassandra Project develops a highly scalable second-generation distributed database, bringing together Dynamo’s fully distributed design and Bigtable’s ColumnFamily-based data model.

Thrift is also developed by Facebook and is a software framework for service development and is used as an interface to Cassandra. The Thrift page site says:

“Thrift is a software framework for scalable cross-language services development. It combines a software stack with a code generation engine to build services that work efficiently and seamlessly between C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, Smalltalk, and OCaml.”

Both Cassandra and Thrift are apache incubator projects.

Installing Cassandra

1. Download cassandra

Download from: http://incubator.apache.org/cassandra/.

2. Create and set the correct paths in the storage-conf.xml

You can find the storage-conf.xml file in your cassandra root directory /conf

My configuration file settings in storage-conf.xml:

Notice: You have to create all these directories for cassandra to run properly.

3. Set a log directory in the log4j.properties file

This file is found in the same directory as the storage-conf.xml

4. Check that you are running java 6 as default

If you are running an earlier version of java you will have to change your version. Java 6 should already be installed on your mac if you keep your os in sync with the automatic updates from apple. You can change your java version by using the “Java Settings” application located in your /Application/Utilities directory.

5. Starting Cassandra

You should be ready to go now, navigate to the root directory of your cassandra installation and start cassandra by typing:

If you dont see any error messages cassandra is probably running as it should, so it is time to test it out.

Cassandra comes with a CLI interface which allowes you to do simple queries to the database. Notice that the CLI interface is not not as powerful as the thrift interface. You can for instance not execute get queries in Super Columns, those queries will create a java exception.

To test the CLI interface, run the following command from the cassandra root directory:

Inserting values to the keyspace:

Retrieving data from the keyspace:

Installing Thrift

Update: I found this manual after I had installed thrift: http://wiki.apache.org/thrift/ThriftInstallationMacOSX, using this install guide will probably fix the issues I had with compiling thrift.

1. Download Thrift

Download from http://incubator.apache.org/thrift/download/ and extract it.

2. Check that you have installed the following:

• g++ 3.3.5+
• Runtime libraries for lex and yacc might be needed for the compiler.
• boost 1.33.1+ (1.34.0 for building all tests) http://www.boost.org/.

I had to install boost manually:

Notice: the boost installation might take a while, It took about 5 – 10 minutes on my Macbook PRO (2.53GHz).

You can see the full requirements for thrift at http://wiki.apache.org/thrift/ThriftRequirements.

3. Start the installation

This ended up in an error message for me:

To fix this I had to copy my pkg.m4 file from /opt/local/share/aclocal/pkg.m4 to my thrift/aclocal directory.

Navigate to your thrift root directory:

Thanks to http://aaronspotlatch.appspot.com/archive/Jul-2008 and
http://qslack.com/post/thrift-macosx-104 for pointing me in the right direction.

You should now be ready to run make

and

You should now be able to run thrift on your mac.

You should now be ready to build your amazing application with Cassandra if both your installation of Cassandra and Thrift were successful.

I will try to post another blog post shortly on using Cassandra, Thrift and PHP. Stay tuned.

Resources

• The Cassandra Project
• The Cassandra Wiki
• The Thrift Wiki
• http://jetfar.com/installing-cassandra-and-thrift-on-snow-leopard-a-quick-start-guide/
• Setting up a Cassandra cluster
• http://www.onemanclapping.org/2009/11/building-cassandra-thrift-bindings-on.html