Elasticsearch: Cluster Setup

Elasticsearch requires very little configuration. We can run it as stand alone, just by installing it without any config change. But in case of cluster setup we need to do minor config changes.

Elastic.co provides comprehensive setup instruction which can be found here

Before we start:

Master node: Responsible for cluster management like discovering healthy nodes and adding or removing them from a cluster.

Data node: Stores data and runs search and aggregation.

node name / cluster name: Elasticsearch will assign node / cluster name automatically, but its better to change it, for better visibility and understanding.

Configuring cluster:

Lets setup cluster with 3 nodes, 1 master and two data nodes. Let assume master node IP as 192.168.100.10 and two data nodes IP’s are 192.168.100.20, 192.168.100.30 receptively.

On Master node:

Sudo vi /etc/elasticsearch/elasticsearch.yml

 cluster.name: "sit_env"
 #node name, please change it as required.
 node.name: "sit_node_1" 
 #path where data will be saved
 path.data: /var/lib/elasticsearch
 #path for log file, can be changed as required.
 path.logs: /var/log/elasticsearch
 node.master: true
 node.data: false
 #default number of shards
 index.number_of_shards:6
 #default number of replicas 1 ( total data nodes - 1)
 index.number_of_replicas:1
 #IP address of host.
 network.host: 192.168.100.10
 #port
 http.port:9200
 #path for backup snapshot to be saved.
 path.repo: /mnt/elasticsearch/elasticsearch_backup
 #IP of all nodes
 discovery.zen.ping.unicast.hosts: ["192.168.100.10","192.168.100.20", "192.168.100.30"]

save and exit

sudo service elasticsearch restart

Data node 1:

 cluster.name: "sit_env"
 #node name, please change it as required.
 node.name: "sit_node_2" 
 #path where data will be saved
 path.data: /var/lib/elasticsearch
 #path for log file, can be changed as required.
 path.logs: /var/log/elasticsearch
 node.master: true
 node.data: true
 #default number of shards
 index.number_of_shards:6
 #default number of replicas 1 ( total data nodes - 1)
 index.number_of_replicas:1
 #IP address of host.
 network.host: 192.168.100.20
 #port
 http.port:9200
 #path for backup snapshot to be saved.
 path.repo: /mnt/elasticsearch/elasticsearch_backup
 #IP of all nodes
 discovery.zen.ping.unicast.hosts: ["192.168.100.10","192.168.100.20", "192.168.100.30"] 

save and exit

sudo service elasticsearch restart

Data node 2:

 cluster.name: "sit_env"
 #node name, please change it as required.
 node.name: "sit_node_3" 
 #path where data will be saved
 path.data: /var/lib/elasticsearch
 #path for log file, can be changed as required.
 path.logs: /var/log/elasticsearch
 node.master: true
 node.data: true
 #default number of shards
 index.number_of_shards:6
 #default number of replicas 1 ( total data nodes - 1)
 index.number_of_replicas:1
 #IP address of host.
 network.host: 192.168.100.30
 #port
 http.port:9200
 #path for backup snapshot to be saved.
 path.repo: /mnt/elasticsearch/elasticsearch_backup
 #IP of all nodes
 discovery.zen.ping.unicast.hosts: ["192.168.100.10","192.168.100.20", "192.168.100.30"]  

save and exit

sudo service elasticsearch restart

Advertisements

Postfix: Forward all outgoing emails to one email address on Centos / Redhat

Assuming the system is Centos / Redhat system.

Steps:

Step 1: 

Create virtual file lets say “all_email_forwarder”  for email forwarding inside /etc/postfix/

vi /etc/postfix/all_email_forwarder

To forward all outgoing emais, lets say ” myemail@domain.com”, please do the following:

/.+@.+/  myemail@domain.com

Step 2:

Edit /etc/postfix/main.cf file and add /edit virtual_maps variable with the following:

virtual_maps = regexp:/etc/postfix/all_email_forwarder

Step 3:

restart postfix service:

service postfix restart