In this section, you can run neptune on multiple machines(Linux).

System Requirements

In order to run or compile neptune you need to have following software already installed on your system

  • Operating System: Linux
  • Machine: more than three
  • Java Virtual Machine: JDK6 or higher
  • File system: Hadoop 0.17 or higher, other all file systems which can implement NeptuneFileSystem interface. 
  • JavaCC: https://javacc.dev.java.net/
  • Ant: http://ant.apache.org/
  • Zookeeper: http://hadoop.apache.org/zookeeper/

Server configuration

time sync.

All nodes in neptune cluster must have same time.
[root@master] crontab -e  (add time sync cron job)
0 4 * * * rdate -s time server && hwclock -w

increase "open files" parameter

On linux system, there is a limit set in the kernel on how many open file descriptors are allowed on the system. Default value is 1024.
Neptune openes many sockets. In java program, a socket is needed 2 file descriptors, so you must increase "open files" parameter.
[root@master] ulimit -a
core file size (blocks) unlimited
data seg size (kbytes) unlimited
file size (blocks) unlimited
max locked memory (kbytes) unlimited
max memory size (kbytes) unlimited
open files 1024
pipe size (512 bytes) 8
stack size (kbytes) unlimited
cpu time (seconds) unlimited
max user processes 4095
virtual memory (kbytes) unlimited

[root@master] vi /etc/security/limits.conf  (add next tow lines)
neptune_admin_id   soft   nofile     10240
neptune_admin_id   hard   nofile     10240

configure SSH

Neptune requires SSH access to manage its nodes. you have SSH up and running on your machine and configured it to allow SSH public key authentication.

generate ssh key
[neptune@master] ssh-keygen -t rsa (Use empty responses for each prompt)
  Enter passphrase (empty for no passphrase):
  Enter same passphrase again:
  Your identification has been saved in
  /home/neptune/.ssh/id_rsa.
Your public key has been saved in /home/neptune/.ssh/id_rsa.pub.
The key fingerprint is:||||a6:5c:c3:eb:18:94:0b:06:a1:a6:29:58:fa:80:0a:bc neptune@localhost
copy ssh key to all nodes

[neptune]scp /home/neptune/.ssh/id_rsa.pub
neptune@devcluster01:/home/neptune/.ssh/authorized_keys

Download

  • Release Versions: can be found here http://www.openneptune.com/download.html
  • Subversion: can get the latest with subversion
    svn checkout --username anonsvn https://dev.naver.com/svn/neptune
    The password is 'anonsvn'

Configure Neptune

${NEPTUNE_HOME}/conf/neptune-site.xml

<?xml version="1.0"?>
<configuration>
    <property>
        <name>neptune.root</name>
        <value>/user/neptune</value>
        <description>root path for data file in DFS(Hadoop)</description>
    </property>
    <property>
      <name>neptune.local.temp</name>
      <value>/home/user_home/tmp/neptune-${user.name}</value>
      <description>A base for other temporary directories.</description>
    </property>
    <property>
      <name>changelog.data.dir</name>
      <value>$USER_HOME$/changelog/</value>
      <description>path where change logs are stored. local machine</description>
    </property>      
    <property>
      <name>changelog.image.dir</name>
      <value>/user/neptune/data/changelog/image</value>
      <description>dfs dir</description>
    </property>   
    <property>
      <name>neptune.cluster.zkservers</name>
      <value>zookeeper_hostname:2181</value>
      <description>zookeeper server hostname(comma separator)</description>
    </property>     
</configuration>
Most configuration variable is directory for data directory on distributed file system or local temporary directory and zookeeper servers.

${NEPTUNE_HOME}/conf/slaves

tabletserver_host01
tabletserver_host02
tabletserver_host03

${NEPTUNE_HOME}/conf/changelog_servers

changelogserver_host01
changelogserver_host02
changelogserver_host03

${NEPTUNE_HOME}/conf/neptune-env.sh

# Set Neptune-specific environment variables here.
# JAVA_HOME, NEPTUNE_HOME, HADOOP_HOME required

heapsize() {
  echo " -Xms$1""m"" -Xmx$2""m "
}

#JAVA HOME dir
export JAVA_HOME=/usr/java/jdk1.6.0_06

#Neptune home dir
export NEPTUNE_HOME=/home/neptune

#Neptune conf dis
export NEPTUNE_CONF_DIR="${NEPTUNE_HOME}/conf"

# JVM Heap Size (MB) of each Neptune component
export NEPTUNE_MASTER_HEAPSIZE=`heapsize 512 1024`
export NEPTUNE_TABLETSERVER_HEAPSIZE=`heapsize 2048 2048`
export NEPTUNE_CHANGELOGSERVER_HEAPSIZE=`heapsize 512 512`

# TABLET_SERVER JMX Socket Port
export TABLETSERVER_JMX_PORT=58004

# Extra Java CLASSPATH elements.  Optional.
#export NEPTUNE_CLASSPATH=

# The directory where pid files are stored. /tmp by default.
export NEPTUNE_PID_DIR=~/.neptune_pids

# A string representing this instance of neptune. $USER by default.
# export NEPTUNE_IDENT_STRING=$USER

#ZooKeeper dir
#set neptune home dir
export ZOOKEEPER_HOME=/home/neptune
export ZOOKEEPER_CONF=${ZOOKEEPER_HOME}/conf/zoo.cfg

# Hadoop Home dir
export HADOOP_HOME=/home/hadoop

# Hadoop Conf dir
export HADOOP_CONF_DIR="${HADOOP_HOME}/conf"

# JVM Options of each Neptune component
export NEPTUNE_MASTERNODE_OPTS="$NEPTUNE_MASTER_HEAPSIZE -Dcom.sun.management.jmxremote"
export NEPTUNE_TABLETSERVER_OPTS="-XX:+UseConcMarkSweepGC -XX:+ExplicitGCInvokesConcurrent -XX:+CMSClassUnloadingEnabled -XX:CMSInitiatingOccupancyFraction=50 -server -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.port=$TABLETSERVER_JMX_PORT $NEPTUNE_TABLETSERVER_HEAPSIZE"
export NEPTUNE_CHANGELOG_OPTS="-XX:+UseConcMarkSweepGC -XX:+ExplicitGCInvokesConcurrent -XX:+CMSClassUnloadingEnabled -XX:CMSInitiatingOccupancyFraction=50 -server $NEPTUNE_CHANGELOGSERVER_HEAPSIZE"

Configure zookeeper

Zookeeper is distributed coordinating service. It's like goolge's chubby. Neptune uses zookeeper for master election, TabletServer failover, etc.
If you have already zookeeper service, set ZOOKEEPER_HOME and ZOOKEEPER_CONF variable in conf/neptune-env.sh. And If not use shared zookeeper service(use dedicated zookeeper serivce for neptune), config NEPTUNE_HOME/conf/zoo.cfg
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
dataDir=/home/neptune/zookeeper/data
# the port at which the clients will connect
clientPort=2181      
			

Deploy to all nodes

Copy ${NEPTUNE_HOME}/conf, ${NEPTUNE_HOME}/lib, ${NEPTUNE_HOME}/bin, ${NEPTUNE_HOME}/neptune*.jar to all nodes.
[neptune@master]chmod 755 ./bin/*       
[neptune@master]chmod 755 ./conf/*.sh
[neptune@master]tar cvf neptune.tar conf lib bin neptune*.jar
[neptune@master]scp neptune.tar tabletserver_host01:/home/neptune/neptune.tar
[neptune@master]scp neptune.tar tabletserver_host02:/home/neptune/neptune.tar
...
[neptune@master]ssh tabletserver_host01
[neptune@tabletserver_host01]tar xf neptune.tar
[neptune@tabletserver_host01]exit
...
[neptune@master]ssh tabletserver_host02
[neptune@tabletserver_host02]tar xf neptune.tar

Start/Stop

When neptune starts up, several daemons start in the following order
  • zookeeper server on master node(or other nodes for failover)
  • ChangeLogserver on slave nodes
  • NeptuneMaster on master(or other nodes for failover)
  • TabletServer on on slave nodes
Run ${NEPTUNE_HOME}/bin/zookeeper-start.sh to start zookeeper server
[neptune@master]./bin/zookeeper-start.sh
			
Run ${NEPTUNE_HOME}/bin/start.sh to start neptune cluster and run ${NEPTUNE_HOME}/bin/stop.sh to stop neptune cluster
#start cluster
[neptune@master]./bin/start.sh

#format and start cluster 
[neptune@master]./bin/start.sh -format

#stop cluster
[neptune@master]./bin/stop.sh
You can check server status this url "http://neptune_master:57000". Also you can see server logs in ${NEPTUNE_HOME}/logs directory.
Not to run entire cluster but to run NeptuneMaster or TabletServer on a particular node, the following command is used.
#Run NeptuneMaster
[neptune@master]./bin/neptune-daemon.sh start neptunemaster

#Run TabletServer
[neptune@tabletserver_host01]./bin/neptune-daemon.sh start tabletserver