Kristian Lunde » Java http://www.klunde.net www.klunde.net Wed, 01 Feb 2012 23:57:16 +0000 en hourly 1 http://wordpress.org/?v= Installing nutch 1.0 on OSX http://www.klunde.net/2009/04/07/installing-nutch-10-on-osx/ http://www.klunde.net/2009/04/07/installing-nutch-10-on-osx/#comments Tue, 07 Apr 2009 21:15:05 +0000 Kristian Lunde http://www.klunde.net/?p=275

Today I started to work on a little project that required a crawler, and Nutch seemed to do most of what I needed. The nutch team conveniently released Nutch 1.0 late in March 2009, so I had a brand new release to test out. Installing nutch 1.0 on a mac is not as straight forward as I thought, I ran into a lot of unexpected issues and here is my cook book description of how to successfully install nutch 1.0 on your mac.

  1. Download the latest source code from the Apache SVN repository http://svn.apache.org/repos/asf/lucene/nutch/. I tried running it from the tarball without success, I also tried to compile the source from the tarball, but a post on the nutch forum clearly states that this will not work.
  2. Set your JAVA_HOME and NUTCH_JAVA_HOME variables, again this is not straight forward, they both need to point to your real installation of Java 1.6 (earlier versions of Java will fail). I sat these variables to: /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home, I could not get the /Library/Java/Home symbolic link to work properly.
  3. Compile the source code using Ant (I built it in Eclipse).
  4. Setup your nutch configuration, by following the tutorial by Peter P. Wang
  5. Run your first crawl with: ./bin/nutch crawl urls -dir crawl -depth 3 -topN 50

Most of the issues I encountered was related to the Java version and the fact that using /Application/Utilities/Java/Java preferences application do not really change the JAVA_HOME directory /Library/Java/Home properly. So make sure you have set both JAVA_HOME and NUTCH_JAVA_HOME, and that your OSX does not fool you when it pretend to be symbolically linking to the 1.6 installation.

Good luck.

]]> http://www.klunde.net/2009/04/07/installing-nutch-10-on-osx/feed/ 2 The View Helper pattern http://www.klunde.net/2008/07/20/the-view-helper-pattern/ http://www.klunde.net/2008/07/20/the-view-helper-pattern/#comments Sun, 20 Jul 2008 21:11:58 +0000 Kristian Lunde http://www.klunde.net/?p=37

Developing MVC (Model View Controller pattern) applications in PHP or any other language often require a lot from the view tier. The view needs to process data received from the model tier and form it into presentable data, it also has to manage user input and form that into data understandable for the model tier.
This might not be a big issue while working with small application, but when it comes to midscale and large applications the view helper pattern can be of great help. The view helper pattern is one of the J2EE core patterns and the documentation can be found on:

http://java.sun.com/blueprints/corej2eepatterns/Patterns/ViewHelper.html

What does the helper pattern do?
First of all the helper pattern adds an extra tier to the system, this tier can be seen as a mid tier which has some understanding of the logic of the system, it knows a little bit about the view and a little bit about the model. Another cool thing is that the view helper pattern makes your code more reusable. When moving complex structures from the view and into a view helper it can with ease be used by other views.

Example:

You are writing a web application where the user writes a review of some product, the application should do auto saving of the user input every 20 second using Ajax functionality. The application should of course also save the user input then the user submits the data. The ajax request and the user submit does almost the same thing but the when the user submits the data the view should also store a rating of the product. This require the ajax request and the user submit to be two different views, or one complex view.

In an ordinary MVC system you would have to implement two views with very much of the similar behavior or one complex view. Using the view helper pattern you extract the storage of the user review in a helper which can be reused by both the ajax request view and the user submit view.

I have written a very simple implementation of the example in PHP. The implementation is not complete at all, but it is meant as a proof of concept that the reusability of code in your application can increase using the view helper pattern.

  1. <?php
  2.  
  3. class ProductReviewHelper
  4. {
  5.  public function __construct(){}
  6.  
  7.  /**
  8.    * save the review
  9.   **/
  10.  public function save($user_input)
  11.  {
  12.   //validate input
  13.   $input = $this->validate($user_input);
  14.  
  15.   //saves the review and return the result of the save
  16.   return $review_manager->save($input['product_id'], $input['review']);
  17.  }
  18.  
  19.  /**
  20.   * validate the input
  21.   **/
  22.  public function validate($user_input)
  23.  {
  24.   $filter_args = array('product_id' => FILTER_VALIDATE_INT,
  25.          'review'   => FILTER_SANITIZE_STRING);
  26.   $input = filter_var_array($user_input, $filter_args);
  27.  
  28.   //do validation
  29.   return $input;
  30.  }
  31. }
  1. <?php
  2.  
  3. //Ajax view
  4. $review_helper = new ProductReviewHelper();
  5. $result = $review_helper->save($_POST);
  6. echo $result;
  7. exit();
  8. ?>
  1. <?php
  2.  
  3. //User submit view
  4. $review_helper = new ProductReviewHelper();
  5. $review_result = $review_helper->save($_POST);
  6.  
  7. $rating_helper = new ProductRatingHelper();
  8. $rating_result = $rating_helper->save($_POST);
  9.  
  10. //manage the result from the helpers
  11. ?>

I am sure that the view helper pattern has helped me to write better and more organized code, which is easily understandable and very reusable.

If you do not use the view helper pattern, and still have solved the problem with reusability of code in the view tier please feel free to leave me a comment describing your solution.

]]> http://www.klunde.net/2008/07/20/the-view-helper-pattern/feed/ 0 My first introduction to java and databases http://www.klunde.net/2008/05/05/my-first-introduction-to-java-and-databases/ http://www.klunde.net/2008/05/05/my-first-introduction-to-java-and-databases/#comments Mon, 05 May 2008 20:14:57 +0000 Kristian Lunde http://www.klunde.net/?p=9

The other day I did a complete reconfiguration of java on my system. Tonight I started to play around with java and postgresql. I installed PostgreSQL 8.2 (yes I know 8.3 is out there), and downloaded the jdbc driver for PosgreSQL.

PostgreSQL

After installing PostgreSQL, you have to add a user to the authorization file pg_hba.conf located in /etc/postgresql/8.x/main on a ubuntu system, remember to restart the postgreSQL server after editing pg_hba.conf. When that is done add the same user to the postgreSQL server either by using postgreSQL’s createuser function or by logging into the postgreSQL server and doing a SQL CREATE USER, I did the latter one.

  1. CREATE USER kristian WITH PASSWORD 'xxxxxxxx' CREATEDB;

This enables me to do a regular access of the database with:

  1. kristian@Saturn:~$ psql -U kristian crawler
  2. Welcome to psql 8.2.7, the PostgreSQL interactive terminal.</code>
  3.  
  4. Type:  \copyright for distribution terms
  5. \h for help with SQL commands
  6. \? for help with psql commands
  7. \g or terminate with semicolon to execute query
  8. \q to quit
  9.  
  10. crawler=#

My table:

  1. crawler=# \d document;
  2. TABLE "public.document"
  3. COLUMN     |            Type             |                       Modifiers
  4. —————+—————————–+————————
  5. id            | integer                     | NOT NULL DEFAULT NEXTVAL('document_id_seq'::regclass)
  6. url           | text                        | NOT NULL
  7. document      | text                        | NOT NULL
  8. ts_downloaded | timestamp without time zone | DEFAULT now()

Data in the table:

  1. crawler=# select * from document;
  2. id |          url          | document |       ts_downloaded
  3. —-+———————–+———-+—————————-
  4. 1 | http://www.klunde.net | jalla    | 2008-04-19 21:11:11.805482

Java

First thing to get the postgreSQL jdbc driver to work is to add it to the java classpath, and of course copy the actual jar file to its correct location, for instance /usr/lib/jvm/java-6-sun-1.6.0.03/jre/ext/lib/which is my java library path.
My classpath:

  1. /usr/lib/jvm/java-6-sun-1.6.0.03/lib:/usr/lib/jvm/java-6-sun-1.6.0.03/jre/ext/lib/postgresql-8.3-603.jdbc4.jar:.

Now its about time to write some actual code:

  1. import java.sql.Connection;
  2. import java.sql.DriverManager;
  3. import java.sql.PreparedStatement;
  4. import java.sql.ResultSet;
  5. import java.sql.SQLException;
  6. import java.sql.Statement; </code>
  7.  
  8. public class DatabaseTest
  9. {
  10. private Connection connection = null;
  11.  
  12. private static final String USERNAME = "XXXXX";
  13. private static final String PASSWORD = "XXXXX";
  14. private static final String URL = "jdbc:postgresql:";
  15. private static final String DATABASE = "crawler";
  16.  
  17. public DatabaseTest()
  18. {
  19. try
  20. {
  21. connection = DriverManager.getConnection(URL + DATABASE, USERNAME, PASSWORD);
  22. ResultSet rs = doQuery("SELECT * FROM document");
  23.  
  24. rs.next();
  25. System.out.println(rs.getString(2));
  26. }
  27. catch (SQLException e)
  28. {
  29. System.out.println("Database Exception!");
  30. System.out.println(e.toString());
  31. e.printStackTrace();
  32. }
  33.  
  34. }
  35.  
  36. public ResultSet doQuery(String sql)
  37. {
  38. ResultSet rs;
  39. try
  40. {
  41. Statement stmt = connection.createStatement();
  42. rs = stmt.executeQuery(sql);
  43. return rs;
  44. }
  45. catch (SQLException e) {
  46. e.printStackTrace();
  47. return null;
  48. }
  49. }
  50. }

Trying to execute my script:

  1. kristian@Saturn:~/workspace/crawler$  javac DatabaseTest.java
  2. kristian@Saturn:~/workspace/crawler$  java -Djdbc.drivers=org.postgresql.Driver DatabaseTest
  3. http://www.klunde.net

It works :D The installation and initial setup went pretty much by the book, but as a PHP developer I must say that I’m a little bit disappointed, JDBC cannot compare itself to the PHP PDO extension yet. But then again, maybe I just need to get to know the JDBC library better. Anyhow it was fun writing this little java app and refreshing some of my java skills.

]]> http://www.klunde.net/2008/05/05/my-first-introduction-to-java-and-databases/feed/ 2 Java on Ubuntu http://www.klunde.net/2008/05/02/java-on-ubuntu/ http://www.klunde.net/2008/05/02/java-on-ubuntu/#comments Fri, 02 May 2008 08:04:35 +0000 Kristian Lunde http://www.klunde.net/?p=11

The other day I decided that it was about time to refresh my java knowledge. I’ve done some java development at work lately, but besides that It has been about 3 years since I last wrote any decent java apps. I have a few ideas of some small cool apps I want to write, but first of all I had to check the java installation on my machine.

Im currently sitting on a Ubuntu 7.04 installation, so I started out by opening my console and typing

java -version

I was a bit surprised about the result, java version could not be determined, I found out that 4 different versions of java were installed and neither the classpath or java_home were set. The java version I wished to used were not among the installed versions. I used the java installation guide on help.ubuntu.com to install my preferred java version, the java 6 version from Sun. The guide was straight forward and the java installation seemed to work as expected after completing the guide.


kristian@Saturn:/usr/lib$ java -version
java version "1.6.0_03"
Java(TM) SE Runtime Environment (build 1.6.0_03-b05)
Java HotSpot(TM) Client VM (build 1.6.0_03-b05, mixed mode, sharing)

Looking good. So I tried typing:


kristian@Saturn:/usr/lib$ javac -version
javac: error while loading shared libraries: libjli.so: cannot open shared object file: No such file or directory

Not looking that good, something was wrong :P I googled the result and found my mistake. Earlier while I was trying to get it all together with the 4 different java versions I changed the symlink in /usr/bin/ and I forgot to add the -s when I created the link. Removing the symblink and recreating it did the trick.


kristian@Saturn:/usr/bin$ sudo ln -s /etc/alternatives/javac javac

The javac in /etc/alternatives is a symblink to the current java version. And trying the all famous javac -version now result in the anticipated result:


kristian@Saturn:~$ javac -version
javac 1.6.0_03

Now I’m all set to go and try out my java skills :P

]]> http://www.klunde.net/2008/05/02/java-on-ubuntu/feed/ 0