Sunday, December 30, 2012

Couchbase 101: Create views (MapReduce) from your Java application

When you are developing a new applications with Couchbase 2.0, you sometimes need to create view dynamically from your code. For example you may need this when you are installing your application, writing some test, or you can also use that when you are building frameworks, and wants to dynamically create views to query data. This post shows how to do it.

Prerequisites

If you are using Maven you can use the following information in your pom.xml to add the Java Client library:

  
    
      couchbase
      Couchbase Maven Repository
      default
      http://files.couchbase.com/maven2/
      
        false
      
    
  
  
  
    
      couchbase
      couchbase-client
      1.1.0
      jar
    
  
See online at https://gist.github.com/4337172

Create and Manage Views From Java 

The full Maven project is available on Github.

Connect to Couchbase Cluster

The first thing to do when you want to create a view from Java is obviously to connect to the cluster.

import com.couchbase.client.CouchbaseClient;
...
...

    List uris = new LinkedList();
    uris.add(URI.create("http://127.0.0.1:8091/pools"));
    CouchbaseClient client = null;
    try {
        client = new CouchbaseClient(uris, "beer-sample", "");

        // put your code here

        client.shutdown();      

    } catch (Exception e) {
        System.err.println("Error connecting to Couchbase: " + e.getMessage());
        System.exit(0);
    }

...
...

  1. Create a list of URIs to different nodes of the cluster - lines 5-6. (In this example I am working on a single node)
  2. Connect to the bucket, in our case beer-sample -line 9. You can include the password if the bucket is protected ( this is not the case here so I am sending an empty string)
If you are looking for more information about Couchbase and Java, you can read this article from DZone : Hello World with Couchbase and Java.

Let's now talk about Couchbase views. You use views/map-reduce functions to index and query data from Couchbase Server based on the content of the JSON document you store inside Couchbase. For more information about views you can look at the "view basics" chapter of the Couchbase Server Manual.

Create Views from Java

Creating a view from Java is really easy : the Java Client Library contains all the classes and methods to do it. As a concrete use case we will use the Application that is described in the Couchbase Java Tutorial.

When you follow this tutorial, you need to manually create some views, as you can see here. In this example, we will create our map function and directly in our Java code and then store it to Couchbase Server. The tutorial asks you to create the following artifacts:
  • a view named "by_name
  • in the design document named "dev_beer" (development mode)
  • and the map function which looks like the following :
 function (doc, meta) {
   if(doc.type && doc.type == "beer") {
     emit(doc.name, null);
   }
 }


The following code allows you to do it from Java:

import com.couchbase.client.protocol.views.DesignDocument;
import com.couchbase.client.protocol.views.ViewDesign;
...
    DesignDocument designDoc = new DesignDocument("dev_beer");

    String viewName = "by_name";
    String mapFunction =
            "function (doc, meta) {\n" +
            "  if(doc.type && doc.type == \"beer\") {\n" +
            "    emit(doc.name);\n" +
            "  }\n" +
            "}";

    ViewDesign viewDesign = new ViewDesign(viewName,mapFunction);
    designDoc.getViews().add(viewDesign);
    client.createDesignDoc( designDoc );
...


  • Create a design document using the com.couchbase.client.protocol.views.DesignDocument class - line 4.
  • Create a view using com.couchbase.client.protocol.views.ViewDesign class with a name and the map function - line 14.
  • You can add this view to a design document - line 15
  • Finally save the document into the cluster using the CouchbaseClient.createDesignDoc method.
If you need to use a reduce function (built-in or custom) you just need to pass to the ViewDesign constructor as 3rd parameter.

When developing view, from Java or from any other tool/language be sure you understand what are the best practices, and the life cycle of the index. This is why I am inviting you to take a look to the following chapters in the Couchbase documentation:

Using the view

First of all, the view that you just created is in "development mode", and by default the Java client SDK will only access the view when it is in "production mode". This means that when you are calling a view from your application it will search it into the production environment. So before connecting to Couchbase cluster you need to setup the viewmode to development.

This is done using the viewmode environment variable from the Java SDK, that could be set using the following methods:
  • In your code, add this line before the client connects to the cluster : System.setProperty("viewmode", "development");
  • At the command line -Dviewmode=development
  • In a properties file viewmode=development
Once it is done you can call the view using the following code:
import import com.couchbase.client.protocol.views.*;

...
   System.setProperty("viewmode", "development"); // before the connection to Couchbase
...
   View view = client.getView("beer", "by_name");
   Query query = new Query();
   query.setIncludeDocs(true).setLimit(20);
   query.setStale( Stale.FALSE );
   ViewResponse result = client.query(view, query);
   for(ViewRow row : result) {
     row.getDocument(); // deal with the document/data
   }
...

This code queries the view you just created. This means Couchbase Server will generate an index based on your map function, will query the server for results. In this case, we specifically want to set a limit of 20 results and also get the most current results by setting Stale.FALSE.
  • Set the viewmode to development - line 4
  • Get the view using the CouchbaseClient.getView() method -line 6-. As you can see I just use the name beer for the design document (and not dev_beer, Couchbase will know where to search since I am in development mode)
  • Create a query and set a limit (20) and ask the SDK to return the document itself
    setIncludeDocs(true) -line 8- The document will be returned from Couchbase server in the most efficient way
  • Ask the system to update the index before returning the result using query.setStale( Stale.FALSE ); -line 9-. Once again be careful when you use setStale method. Just to be sure here is the documentation about it : Index Updates and the stale Parameter
  • Execute the query - line 10
  • And use the result - lines 11-13

Conclusion

In this article, you have learned:
  • How to create Couchbase views from Java
  • Call this view from Java
  • Configure development/production mode views from Couchbase Java Client Library
This example is limited to the creation of a view, you can take a look to the other methods related to design documents and views if you want to manage your design documents : getDesignDocument(), deleteDesignDocument(), ... .


Wednesday, December 26, 2012

What to do if your Couchbase Server does not start?

Working with the Couchbase 2.0.0 release you may have issues when trying to access the Web Admin Console or simply starting the server. This is due to the way Couchbase Server uses the IP address/hostname during the installation process. So when you have one of the following errors :
  • On Windows, Server is not working at all, even after installation. You can access the sever on port 8092 (Couchbase API port), but cannot on port 8091
  • You have the following error when accessing the console
    "[10:02:02] IP address seems to have changed. Unable to listen on 'ns_1@10.0.2.15'" 


  • When you try to restart the server it does not start and you have the following error message in the error log :
    "Configured address '10.0.2.15' seems to be invalid. Will refuse to start for safety reasons" 
Some of these issues are related to a known issue on Windows ( see MB-7417 that will be fixed in 2.0.1) or the fact that Couchbase server does not support change of the IP address after installation.  This is documented in the section “Using Couchbase in the Cloud: Handling Changes in IP Addresses” of the Couchbase Server Manual. This article explains what should be done when configuring Couchbase Server on Windows, but you can do equivalent steps on any platform using the shell scripts available on Linux and/or Mac OS X.

Once you have installed Couchbase, you can see in the console that the IP address of your server is used :


Typically the address 192.168.0.97 is stored in the configuration of Couchbase. If your server receives a new address from the DHCP server, Couchbase will not work anymore. In this article you will see how you can configure Couchbase to use another IP address or Hostname.

Important: The steps that follow will completely destroy any data and configuration from the node, so it is best to start with a fresh Couchbase install. If you can not, you should backup your data using the file based backup-restore documented here.

Setting up a hostname in hosts file
The best practice is to register Couchbase using a hostname instead of an IP Address. For this you will need to associate this hostname to an IP address in the hosts file.

Since the hosts file is part of the system, you need to edit it as administrator. You have different approaches to achieve this:

The following steps explain the easiest way to do it: 
  1. Click “Start Menu”
  2. Navigate to “All Programs > Accessories”
  3. “Right Click” on “Notepad” (or your favorite text editor)
  4. Click “Run as Administrator”

You can now open the file C:\Windows\System21\drivers\etc\hosts

Add a new entry with a host name that will be used by Couchbase Server for example something like :
127.0.0.1    couchbase-node1.mycompany.com

Here I am using the local address (127.0.0.1) like that I won't have to change it even when my IP address changes. (This is useful when you are working in a single node mode)

Configure Couchbase to use the hostname or new IP address

During the installation Couchbase has been registered as a Windows Service. To be able to associate Couchbase to the new hostname (or IP address) the service needs to re-configured and reinstalled.

This could be done using the scripts provided with the product. To run the scripts you need to do it as administrators, you can do it with one of the following methods:
  • Search for the file and right click and select  Run as Administrator  (documented below)
  • Run the terminal as administrator and run all the command from there (documented below)
  • Search for the file and run it using Ctrl+Shift+Enter

Option 1 : Using the Start Menu and Search Program

Stop Couchbase Server Windows Service
The first thing to do, is to stop this service that is automatically started after the installation:
  1. Click Start Menu
  2. Type Services in the Search Program form
  3. Click on Services
  4. In the Services Application navigate to CouchbaseServer
  5. Right Click and Click on Stop 
  6. Couchbase is now stopped. 
Edit the Service Register script
Note: Due to a small formatting issue (See MB-7322), Notepad could not be used, a solution is to take Notepad++ or any other advanced editing tool.
  1. As Administrator, open the
    C:\Program Files\Couchbase\Server\bin\service_register.bat file with your favorite editor. To open the editor as Administrator you can use the approach described in the previous step.
  2. Edit the line 9 to replace %IP_ADDR% by your hostname, the line should look like:
    set NS_NAME=ns_1@couchbase-node1.mycompany.com
  3. Save the file
Delete existing configuration and logs
  1. Using the file explorer, go into:
     C:\Program Files\Couchbase\Server\var\lib\couchbase\mnesia
  2. Delete its content (Select All and Right Click)
Register the new Configuration as Service
  1. Using the file explorer, go into:
    C:\Program Files\Couchbase\Server\bin
  2. Right Click on service_reregister.bat
  3. Click on Run as Administrator
This script recreates the Couchbase Server Windows Service and starts it automatically.

Check the configuration
  1. Launch your Internet Browser
  2. Go to http://localhost:8091
  3. Follow the Couchbase Installation Steps
  4. Once install connect to the console
  5. Go to Server Nodes tab
  6. Check that the server name is now couchbase-node1.mycompany.com
Your Couchbase node is now configured to use the hostname of your server.



Option 2 : Using the Command Line

Launch Command Prompt as Administrator
  1. Click Start Menu
  2. Type Command Prompt in the Search program form
  3. Type Ctrl+Shift+Enter
  4. Go to C:\Program Files\Couchbase\Server\bin (or other location if you have chosen another location during installation)
You are now ready to do the administration tasks.
  1. Execute the service_stop.bat
  2. Edit the Service Register script
    1. Open service_register.bat
    2. Edit the line 9 to replace %IP_ADDR% by your hostname (or your IP address), the line should look like:
      set NS_NAME=ns_1@couchbase-node1.mycompany.com
    3. Save the file
  3. Delete the content of:
     C:\Program Files\Couchbase\Server\var\lib\couchbase\mnesia
  4. Execute the service_reregister.bat
    This script recreates the Couchbase Server Windows Service and starts it automatically.

Check the configuration
  1. Launch your Internet Browser
  2. Go to http://localhost:8091
  3. Follow the Couchbase Installation Steps
  4. Once install connect to the console
  5. Go to Server Nodes tab
  6. Check that the server name is now couchbase-node1.mycompany.com

Your Couchbase node is now configured to use the hostname of your server.


Saturday, December 1, 2012

New Adventure...

Yesterday was my last day at eXo... I have been working at eXo since 2008, and we have achieved many exciting things such as building eXo Platform, the open source social platform, and the Cloud-IDE allowing developers to build, test, and deploy applications online.

It was a great experience for me, but it is time for me to jump into a new adventure...

I am joining Couchbase as a Technical Evangelist for the EMEA Region. When I have started to work with NoSQL engines (starting with Google BigTable for Resultri) I really enjoyed the experience and the flexibility that it gives to the developers. Then I choose to work on a documented oriented database because it looks very natural to me, and evaluate the clustering capabilities. This is how I discovered Couchbase and its community.

This new job is a great opportunity for me to do the things that I really like about software:

  • Coding applications using different languages and frameworks
  • Understanding the sysops/devops related challenges when using a new product/technology
  • And finally probably the most important part; sharing it with others!
I look forward to sharing what I like about NoSQL and Couchbase and discuss with others about their experience or needs around NoSQL. 

See you soon online and in real life during conferences and meetups!

Couchbase Logo