[Share Video] HDFS - Hadoop distributed file system, Introduction

I came across this simple and short video on HDFS a introduction

[Solution] .bat file don't proceed after mvn clean install

Executing .bat does not proceed after mvn clean install.

mvn -o clean install
echo $classpath  -- This never executes

Solution :

call mvn -o clean install 
echo $classpath  

Reason: .bat file, halts after calling another program.  CALL command is invoke another program from the current and proceed further

Reference: http://www.computerhope.com/call.htm

Spring : context:component-scan vs context:annotation-config

Whats the difference between component-scan and annotation-config

• <context:annotation-config/>  -   Looks for @Autowired and JSR annotation, It does not identify the @component, @configuration annotations.  It just autowires existing beans
  
• whereas <context:component-scan  base-package="org.first" />  Does recoginize @Autowired, @component and other annotations also.
  
• <context:component-scan> also does  <context:annotation-config/>  and also scans packages to find registeredbeans

---

Example  1 :  Let's define the classes in beans.xml and also annotate with @Component and @Autowired

package org.first;

@Component
public class FirstServiceImpl implements FirstService{

    public Name userName;

    public FirstServiceImpl(){
        System.out.println("new FirstServiceImpl()");
    }


    @Autowired
    public void setUserName(final Name userName){
        System.out.println("setUserName(final Name userName)");
        this.userName = userName;
    }



   //getters, setters, other constructors

}

package org.first;

@Component

public class Name{

    String name = "new";

    public Name(){
        super();

        System.out.println("new Name()");
    }

   //getters, setters, other constructors
}

beans.xml

<beans><!-- Add Namesapece here -->

    <bean name="userName" class="org.first.Name">
        <property name="name" value="Vidhya"></property>
    </bean>

    <bean name="firstService" class="org.first.FirstServiceImpl">
        <property name="userName" ref="userName"></property>
    </bean>
</beans>

OUTPUT : This will instantiate the userName and firstService classes, because both classes are defined in the xml

new Name()
new FirstServiceImpl()
setUserName(final Name userName)

Example  2 :  Lets try to use <context-annotation-config> :  

package org.first;
@Component
public class FirstServiceImpl implements FirstService{

    public Name userName;

    public FirstServiceImpl(){
        System.out.println("new FirstServiceImpl()");
    }


    @Autowired
    public void setUserName(final Name userName){
        System.out.println("setUserName(final Name userName)");
        this.userName = userName;
    }



   //getters, setters, other constructors

}


package org.first;

public class Name{
    String name = "new";

    public Name(){
        super();
        System.out.println("new Name()");
    }
   //getters, setters, other constructors
}




beans.xml

 <beans><!-- Add Namesapece here -->
    <context:annotation-config/>
</beans>










OUTPUT : This will NOT do anything. NO OUTPUT. Why? The annotation-config cannot know about the beans unless defined. 

How to fix this? We need to add bean definitions. Lets add the bean definitions in the beans.xml





Example  3 :  Lets use <context-annotation-config> and bring back the bean definitions.



beans.xml

 <beans><!-- Add Namesapece here -->
    <context:annotation-config/>


    <bean name="userName" class="org.first.Name">
        <property name="name" value="Vidhya"></property>
    </bean>

    <bean name="firstService" class="org.first.FirstServiceImpl">
        <property name="userName" ref="userName"></property>
    </bean>
</beans>













OUTPUT : This will instantiate the userName and firstService classes, because both classes are defined in the xml

new Name()
new FirstServiceImpl()
setUserName(final Name userName)








Example  4 :  So, what's the use of annotation-config? We saw that annotation-config cannot load @component, Now lets see if it can autowire, with same class as in Example 2.



beans.xml


 <beans><!-- Add Namesapece here -->
    <context:annotation-config/>


    <bean name="userName" class="org.first.Name">
        <property name="name" value="Vidhya"></property>
    </bean>

    <bean name="firstService" class="org.first.FirstServiceImpl">
        <property name="userName" ref="userName"></property>
    </bean>
</beans>













OUTPUT : 


new Name()
new FirstServiceImpl()
setUserName(final Name userName)





Looks like annotation-config can autowire, but cannot instantiate @Component

Example  5 :  Lets use <context:component-scan> now. As per definition, it can read all@component classes and also autowire. We will check that

package org.first;

@Component
public class FirstServiceImpl implements FirstService{
    public Name userName;


    public FirstServiceImpl(){
        System.out.println("new FirstServiceImpl()");
    }


    @Autowired
    public void setUserName(final Name userName){
        System.out.println("setUserName(final Name userName)");
        this.userName = userName;
    }



   //getters, setters, other constructors

}


package org.first;
@Component
public class Name{

    String name = "new";

    public Name(){
        super();
        System.out.println("new Name()");
    }
   //getters, setters, other constructors
}




beans.xml

 <beans><!-- Add Namespace here -->
     <context:component-scan base-package="org.first" />
</beans>

OUTPUT : There you go.. Now we achieve with single line in beans.xml, which all @Component and @Autowired annotations.













new FirstServiceImpl()

new Name()

setUserName(final Name userName)







SUMMARY:
1. <context:component-scan> is one stop solution for @Autowired and @Component 
2.<context:annotation-config> can only do @Autowired.
3. I am readying the annotation-config is very useful in the web context with DispatcherServlet. But need to try that. Probably will write about it later when I try it.



References
http://docs.spring.io/spring/docs/3.0.0.M3/reference/html/ch04s11.html



http://www.tutorialspoint.com/spring/spring_annotation_based_configuration.htm
http://stackoverflow.com/questions/7414794/difference-between-contextannotation-config-vs-contextcomponent-scan

Example: Spring configuration using AnnotationConfigApplicationContext without XML

Below is the example for Bean configuration using AnnotationConfigApplicationContext.

AnnotationConfigApplicationContext(<Package to scan>) or
AnnotationConfigApplicationContext(@Configuration class with @bean declaration)

We have to make sure that the same beans are not marked @component and also defined in @configuration class

---

---

Class with @component annotation

---

package org.first;

import org.springframework.stereotype.Component;

@Component
public class FirstServiceImpl implements FirstService
{

    public FirstServiceImpl()
    {
    }

    public String printMessage()
    {
        final String s = "--printMessage()---";
        System.out.println(s);
        return s;

    }

    public String messagePrint()
    {
        final String s = "--messagePrint()---";
        System.out.println(s);
        return s;

    }

    public FirstServiceImpl getfirstService()
    {
        return new FirstServiceImpl();
    }
}

---

Test code

---

import org.first.FirstService;
import org.junit.Test;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.AnnotationConfigApplicationContext;

public class AOPTest
{

    ApplicationContext ctx = new AnnotationConfigApplicationContext("org.first");

    @Test
    public void testContext()
    {
        final FirstService service = ctx.getBean(FirstService.class);
        service.printMessage();

    }
}

Example : Spring context:component-scan and @Component

Automatic Component scanning is useful in defining beans and autowiring them without defining them in an .xml file


Below is the example  with xml and  and beans annotated

---

beans.xml

---

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:aop="http://www.springframework.org/schema/aop" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:context="http://www.springframework.org/schema/context"
    xsi:schemaLocation="http://www.springframework.org/schema/beans 
        http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
        http://www.springframework.org/schema/aop
        http://www.springframework.org/schema/aop/spring-aop-3.0.xsd 
        http://www.springframework.org/schema/context
        http://www.springframework.org/schema/context/spring-context-3.0.xsd">

    <context:component-scan  base-package="org.first" />
</beans>

---

Class with @component annotation

---

package org.first;

import org.springframework.stereotype.Component;

@Component
public class FirstServiceImpl implements FirstService
{

    public FirstServiceImpl()
    {
    }

    public String printMessage()
    {
        final String s = "--printMessage()---";
        System.out.println(s);
        return s;

    }

    public String messagePrint()
    {
        final String s = "--messagePrint()---";
        System.out.println(s);
        return s;

    }

    public FirstServiceImpl getfirstService()
    {
        return new FirstServiceImpl();
    }
}

---

Test code

---

import org.first.FirstService;
import org.junit.Test;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;

public class AOPTest
{

    ApplicationContext ctx = new ClassPathXmlApplicationContext("beans.xml");

    @Test
    public void testContext()
    {
        final FirstService service = ctx.getBean(FirstService.class);
        service.printMessage();

    }
}

Spring Controller with Hibernate using ContextLoaderListener

• Below is a simple example to show how to instantiate the spring context in ContextLoaderListener and use in a HTTPServlet.
  
• There are other ways of directly using Dispatcher servlet, but the is the basic way for a beginner.
• Eclipse project for this example can be found here: spring-web-contextloader.zip
   
• The given codebase does not contain the database scripts. You may want to create ur own tables and change the hibernate.cfg.xml and beans.xml with the database connection details
  
• This code has been tested in a jetty webserver. But should work in any webserver

---

Web.xml -

---

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://java.sun.com/xml/ns/javaee" xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
    xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" id="WebApp_ID" version="2.5">

    <!-- Spring app context using a listener  -->
    <listener>
        <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
    </listener>

    <context-param>
        <param-name>contextConfigLocation</param-name>
        <param-value>/WEB-INF/beans.xml</param-value>
    </context-param>



    <display-name>spring-web</display-name>
    <welcome-file-list>
        <welcome-file>index.jsp</welcome-file>
    </welcome-file-list>

    <!-- Trying the simple Servlet type -->
    <servlet>
        <description></description>
        <display-name>EmployeeServlet</display-name>
        <servlet-name>EmployeeServlet</servlet-name>
        <servlet-class>org.controllers.EmployeeServlet</servlet-class>
    </servlet>
    <servlet-mapping>
        <servlet-name>EmployeeServlet</servlet-name>
        <url-pattern>*.action</url-pattern>
    </servlet-mapping>

</web-app> 

---

beans.xml

---

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:context="http://www.springframework.org/schema/context"
    xmlns:mvc="http://www.springframework.org/schema/mvc" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.springframework.org/schema/beans 
              http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
              http://www.springframework.org/schema/context
              http://www.springframework.org/schema/context/spring-context-3.0.xsd
              http://www.springframework.org/schema/mvc 
              http://www.springframework.org/schema/mvc/spring-mvc-3.0.xsd">

    <bean name="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
        <property name="driverClassName" value="org.hsqldb.jdbc.JDBCDriver" />
        <property name="url" value="jdbc:hsqldb:hsql://localhost/section1" />
        <property name="username" value="sa" />
        <property name="password" value="" />
    </bean>

    <bean name="sessionFactory" class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
        <property name="dataSource" ref="dataSource"></property>
        <property name="packagesToScan" value="org.entity" />
        <property name="configLocation" value="classpath:hibernate.cfg.xml" />
    </bean>


    <bean name="employeeDAO" class="org.dao.EmployeeDAO">
        <property name="sessionFactory" ref="sessionFactory" />
    </bean>

</beans>

---

hibernate.cfg.xml

---

<?xml version='1.0' encoding='utf-8'?>
<!DOCTYPE hibernate-configuration PUBLIC
"-//Hibernate/Hibernate Configuration DTD//EN"
"http://hibernate.sourceforge.net/hibernate-configuration-3.0.dtd">

<hibernate-configuration>
    <session-factory>
        <property name="hibernate.connection.driver_class">org.hsqldb.jdbc.JDBCDriver</property>
        <property name="hibernate.connection.url">jdbc:hsqldb:hsql://localhost/section1</property>
        <property name="hibernate.connection.username">sa</property>
        <property name="hibernate.connection.password"></property>
        <property name="hibernate.hbm2ddl.auto">update</property>

        <!-- property name="hibernate.connection.pool_size">1</property -->
        <property name="hibernate.show_sql">true</property>
        <property name="hibernate.format_sql">true</property>
        <property name="hibernate.dialect">org.hibernate.dialect.HSQLDialect</property>


        <!-- mapping class="org.entity.Product" / -->
        <mapping class="org.entity.Employee" />
    </session-factory>
</hibernate-configuration>

---

Servlet - Manual Controller

---

package org.controllers;

import java.io.IOException;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.dao.EmployeeDAO;
import org.entity.Employee;
import org.springframework.context.ApplicationContext;
import org.springframework.web.context.support.WebApplicationContextUtils;

public class EmployeeServlet extends HttpServlet
{
    ApplicationContext ctx = null;

    @Override
    protected void service(final HttpServletRequest req, final HttpServletResponse resp)
            throws ServletException, IOException
    {
        if (ctx == null)
        {
            ctx = WebApplicationContextUtils.getWebApplicationContext(getServletContext());
        }
        final String uri = req.getRequestURI();
        System.out.println("---------------" + uri);
        if (uri.contains("employee.action"))
        {
            getEmployee(req, resp);
        }

    }

    public void getEmployee(final HttpServletRequest req, final HttpServletResponse resp)
            throws ServletException, IOException
    {
        try
        {
            final EmployeeDAO dao = ctx.getBean("employeeDAO", EmployeeDAO.class);
            final Employee emp = dao.getEmployee(null);
            req.setAttribute("emp", emp);
            req.getRequestDispatcher("employee.jsp").forward(req, resp);
        }
        catch (final Exception e)
        {
            e.printStackTrace();
        }

        req.getRequestDispatcher("errors.jsp").forward(req, resp);
    }
}

---

DAO

---

package org.dao;

import org.entity.Employee;
import org.hibernate.SessionFactory;

public class EmployeeDAO
{

    private SessionFactory sessionFactory;

    public void setSessionFactory(final SessionFactory sessionFactory)
    {
        this.sessionFactory = sessionFactory;
    }

    public Employee getEmployee(Integer id)
            throws Exception
    {
        try
        {

            if (id == null)
            {
                id = 1;
            }
            final Employee emp = (Employee) sessionFactory.openSession().get(Employee.class, id);
            System.out.println("--sessionFactory.get(Employee.class, " + id + ")   ---> " + emp);
            return emp;

        }
        catch (final Exception e)
        {
            e.printStackTrace();
            throw e;
        }
    }
}

---

Entity

---

package org.entity;

import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;

@Entity
public class Employee {
 @Id
 @GeneratedValue(strategy = GenerationType.IDENTITY)
 private Integer employeeId;
 private String firstName;
 private String lastName;
 private String jobTitle;
 private Double salary;

 public Employee() {
 }

 public Employee(Integer employeeId, String firstName, String lastName,
   String jobTitle, Double salary) {
  this.employeeId = employeeId;
  this.firstName = firstName;
  this.lastName = lastName;
  this.jobTitle = jobTitle;
  this.salary = salary;
 }
.....................
Please pick up the code from the zip file
}

---

employee.jsp

---

<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>
<%@ page language="java" contentType="text/html; charset=ISO-8859-1"
    pageEncoding="ISO-8859-1"%>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<title>Insert title here</title>
</head>
<body>
Employee : <%=request.getAttribute("emp") %>
<br/>
Employee using taglig : <c:out value="${requestScope.emp}"></c:out>
</body>
</html>

---

pom.xml

---

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>spring-web</groupId>
    <artifactId>spring-web</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <packaging>war</packaging>
    <properties>
        <!-- Spring -->
        <spring-framework.version>3.2.3.RELEASE</spring-framework.version>

        <!-- Hibernate / JPA -->
        <hibernate.version>4.2.1.Final</hibernate.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-webmvc</artifactId>
            <version>3.2.0.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-jdbc</artifactId>
            <version>3.2.0.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-orm</artifactId>
            <version>3.2.0.RELEASE</version>
        </dependency>
        <!-- Hibernate -->
        <dependency>
            <groupId>org.hibernate</groupId>
            <artifactId>hibernate-entitymanager</artifactId>
            <version>${hibernate.version}</version>
        </dependency>
        <!-- db -->
        <dependency>
            <groupId>org.hsqldb</groupId>
            <artifactId>hsqldb</artifactId>
            <version>2.2.8</version>
        </dependency>

    </dependencies>
    <build>
        ....................

        <!-- Please see attached zip containing the code --> 

    </build>
</project>

OUTPUT

Elasticsearch shards and replicas

Shards and replicas : Here is a good explanation on this topic http://stackoverflow.com/a/15705989

Routing  clustering and index internals: http://exploringelasticsearch.com/advanced_techniques.html#ch-advanced

Cluster join example : http://thediscoblog.com/blog/2013/09/03/effortless-elasticsearch-clustering/

HibernateTemplate not advised starting Hibernate 4

Here is the simple example for spring and hibernate integration
 spring_hibernate.zip

The sessionFactory is injected into the DAO with all definitions in the beans.xml with datasource wired.

---

beans.xml 

---

<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">

    <bean name="dataSource"
        class="org.springframework.jdbc.datasource.DriverManagerDataSource">
        <property name="driverClassName" value="org.hsqldb.jdbc.JDBCDriver" />
        <property name="url" value="jdbc:hsqldb:hsql://localhost/section1" />
        <property name="username" value="sa" />
        <property name="password" value="" />
    </bean>

    <bean name="sessionFactory"
        class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
        <property name="dataSource" ref="dataSource"></property>
        <property name="packagesToScan" value="org.entity"/>
    </bean>


    <bean name="EmployeeDAO" class="org.dao.EmployeeDAO">
        <property name="sessionFactory" ref="sessionFactory"/>
    </bean>

</beans>
 

---

org.entity.Employee

---

package org.entity;

import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;

@Entity
public class Employee {
 @Id
 @GeneratedValue(strategy = GenerationType.IDENTITY)
 private Integer employeeId;
 private String firstName;
 private String lastName;
 private String jobTitle;
 private Double salary;

..............

..............

}

---

DAO

---

package org.dao;

import org.entity.Employee;
import org.hibernate.SessionFactory;

public class EmployeeDAO {

 private SessionFactory sessionFactory;

 public void setSessionFactory(SessionFactory sessionFactory) {
  this.sessionFactory = sessionFactory;
 }

 public void getEmployee() throws Exception {
  try {
   Employee emp = (Employee) sessionFactory.openSession().get(Employee.class, 1);
   System.out.println("--sessionFactory.get(Employee.class, 1)   ---> " + emp);

  } catch (Exception e) {
   e.printStackTrace();
   throw e;
  }

 }
}

---

Test Code

---

package org;

import org.dao.EmployeeDAO;
import org.junit.Test;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;

public class EmployeeDAOTest {

 ApplicationContext ctx = new ClassPathXmlApplicationContext("beans.xml");
 
 @Test
 public void testGetEmployeeWithHibernate() throws Exception{
 
  EmployeeDAO dao = (EmployeeDAO) ctx.getBean("EmployeeDAO");
  dao.getEmployee();
 }

}
 

Spring Database connection with HSQLDB using datasource

Below is a simple example to try JDBC connection in Spring.  This example does not have the complete code

---

DAO

---

import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.ResultSet;

import javax.sql.DataSource;

public class EmployeeDAO {

    private DataSource datasource;

    public EmployeeDAO() {
        
    }
    
    public EmployeeDAO(DataSource datasource) {
        super();
        this.datasource = datasource;
    }


    public DataSource getDatasource() {
        return datasource;
    }


    public void setDatasource(DataSource datasource) {
        this.datasource = datasource;
    }


    public void countEmployee() throws Exception{
        Connection con = null;
        try {
            con = datasource.getConnection();
            PreparedStatement ps = con.prepareStatement("select count(*) from employeebimap");
            ResultSet rs = ps.executeQuery();
            if(rs.next())
            {
                System.out.println("select count(*) from employeebimap ---> "+ rs.getString(1));
            }

        } catch (Exception e) {
            e.printStackTrace();
            throw e;
        } finally {
            if (con != null)
                con.close();
        }

    }
}

---

database-beans.xml in classpath

---

<beans br="" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.springframework.org/schema/beans" xsi:schemalocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">

    <bean br="" class="org.springframework.jdbc.datasource.DriverManagerDataSource" name="datasource">
        <property name="driverClassName" value="org.hsqldb.jdbc.JDBCDriver">
        <property name="url" value="jdbc:hsqldb:hsql://localhost/section1">
        <property name="username" value="sa">
        <property name="password" value="">
    </property></property></property></property></bean>

    <bean class="EmployeeDAO" name="EmployeeDAO">
        <property name="datasource" ref="datasource">
    </property></bean>

    <bean autowire="constructor" br="" class="EmployeeDAO" name="EmployeeDAOAutowired">
</bean>
</beans>

---

EmployeeDAOTest

---

import EmployeeDAO;
import org.junit.Test;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;

public class EmployeeDAOTest {

    ApplicationContext ctx = new ClassPathXmlApplicationContext("database-beans.xml");
    
    @Test
    public void testEmployee() throws Exception{
    
        EmployeeDAO dao = (EmployeeDAO) ctx.getBean("EmployeeDAO");
        dao.countEmployee();
    }
    
    @Test
    public void testEmployeeAutowiredConstructor() throws Exception{
    
        EmployeeDAO dao = (EmployeeDAO) ctx.getBean("EmployeeDAOAutowired");
        dao.countEmployee();
    }

}

OUTPUT:

select count(*) from employeebimap ---> 2
select count(*) from employeebimap ---> 2

buildSessionFactory() after Hibernate 4

Configuration cfg = new Configuration().configure();
cfg.buildSessionFactory()

cfg.buildSessionFactory() is deprecated starting hibernate 4 onwards.


Following code can be used to create session factory starting from hibernate 4

Configuration cfg = new Configuration().configure();
ServiceRegistry registry = new StandardServiceRegistryBuilder().applySettings(cfg.getProperties()).build();
factory = cfg.buildSessionFactory(registry);

[Book Review] Mastering Elasticsearch - Not for Begineers

Book Details

Language : English
Paperback : 386 pages [ 235mm x 191mm ]
Release Date : October 2013
ISBN : 178328143X
ISBN 13 : 9781783281435
Author(s) : Rafał Kuć, Marek Rogoziński
Topics and Technologies : All Books, Big Data and Business Intelligence, Open Source



Review :  This books starts with introduction to lucene. The authors have organized the first few topics so well, that it gives very good insight for the people who are new to Lucene. This book is for advanced Elasticsearch users who are interested to know more about ES architecture and turning the performance. This book is NOT for beginners.

Jenkins - Setup to schedule your repetitive scripts

What is Jenkins & when to use It?

When you have bunch of repeated jobs or scripts that you want to run, scheduled or sometimes manually triggered. Go for jenkins.

Its as easy as 

•    Step 1: download
•    Step 2: Install
•    Step 3: Install Basic Plugins (optional)
•    Step 4: Setup security
•    Step 5: Create new job(s)
•    Run

Jenkins is also used these days in the case of continuous integration, working with the Source Control, Review Boards, and other quality monitoring tools.

Step 1 : Download  : http://jenkins-ci.org/

Based on your environment, you can download the installer as shown below and install it.

Step 2 : Install   - Follow instructions.

• Follow the installer instructions.
• On Linux, firewall to be disabled if it blocks 8080 port
  service iptables off
• After installation accessible at  http://<hostname>:8080  . Example http://localhost:8080

You will see the below screen after successful installation & start-up

Step 3 : Install  Plugins (Optional)

There are some default plugins. If you need more, pick and choose from the plugin list as shown below.

For Manual installation.

• Download plugins from https://updates.jenkins-ci.org/download/plugins/
• copy the *.hpi/*.jpi file into the $JENKINS_HOME/plugins directory. 
• Restart Jenkins (many containers let you do this without restarting the container)

I picked up few extra plugins Greenballs, XShell, setenv

Step 4 : Enable Security

1. Manage Jenkins -> Configure Global Security -> Enable Security.  

2. Configure as shown in the below picture.

1. Enable Security
2. Save  in jenkins database with Signup Enabled. This will enable you to add users later.
3. Add privileges for admin user (which you will add through signup process)
4. anonymous will have no privileges. Admin user - Select all checkboxes

1. 5. Save, which will take you to the Login page as shown in the next image.

3. Signup for admin user from the Login Page. 

You are done with the basic security. You can go ahead and add other privileges using the admin user that is created.

Step 5 : Create new job

The dashboard now shows the job that was created.  and can now be run to see the progress

Some special settings when running on Linux Environment.

Jenkins in linux is created as a service and can be started using service jenkins start

The service will not be able access scripts or files within /root user  Say java -jar /root/test.jar will give you permission denied, even after setting the chmod appropriately. 

To solve this problem, the sudoers file has to be edited as mentioned below and execute the command prefixed with sudo. 

sudo java -jar /root/test.jar

Changes in the sudoers file

• requiretty to be disabled for the jenkins service.  Add the Defaults:jenkins !requiretty  afterDefaults requiretty
• visudo .(edit sudoers files)
• Defaults requiretty
• Defaults:jenkins !requiretty
• add this line jenkins        ALL=(ALL)       NOPASSWD: ALL at the end

[Solution] Elasticsearch 0.90 insallation with .rpm and logging Errors

Problem : 

I downloaded the latest ES rpm and installed on centos. When tried running the script in foreground, i see that the logging is not enabled.

tried both /etc/init.d/elasticsearch -f and   /usr/share/elasticsearch/bin/ -f both. 

log4j:WARN No appenders could be found for logger (node).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

and nothing gets logged.

Tried the following in vain : 

• export ES_CLASSPATH=/etc/elasticsearch/logging.yml 
• changed patch.conf, path.data in elasticsearch.yml file

But, Nothing seems to fix the logging problem

Solution:

edit /usr/share/elasticsearch/bin/elasticsearch  

give absolute paths for the home, and all files.

This file should be changed, so "any" startup of elasticsearch will work in foreground without errors

ES_JAVA_OPTS="-Des.config=/etc/elasticsearch/elasticsearch.yml -Des.path.conf=/etc/elasticsearch/ -Des.path.home=/usr/share/elasticsearch -Des.path.logs=/var/log/elasticsearch -Des.path.data=/var/lib/elasticsearch -Des.path.work=/tmp/elasticsearch -Des.path.plugins=/usr/share/elasticsearch/plugins"