Tuesday, September 26, 2017

Robotic Process Automation

What is RPA(Robotic Process Automation?

RPA is the automation of back and front office processes that are largely rules based, structures and repetitive. It is a first step and necessary foundation in the enterprise digital operation journey, before implementing cognitive, chat bots and AI. It frees your human resources from low value activities to focus on building the business.
Configuration that automate manual, repeatable tasks
Algorithms that solve specific problem
Software 'robots' that plug into, and access, existing business software.
Work flow enabled interaction.

Level1 (Basic Automation)
Level2 ( Structured Rules)
Level3 (Pattern based Decisions)
Level4  (Multiple Decision making)

Key Objective of RPA
Cost Saving
Improve customer Experience
Improve accuracy/control
identified average cost savings

How it is works?
It uses demonstrative steps, programming skills not required to train these virtual bots.
SMEs and Process exports have to demonstrate the jobs to these software bots (Knowledge transition phase). Once it gets trained it will do the tasks 24/7 without taking coffee/lunch breaks :)

1 Robot FTE costs approximately $3,000 per year in average  licensing costs.  (depends on application & license package).

Major RPA Service Provider
.



 ref:
http://www.everestgrp.com


Sunday, September 10, 2017

Rancher- Container OS

Rancher OS is a light weight  container Operating system, specially designed  to run Dockerized applications.


Download Rancher ISO  OS images and install it in bare metal  hardware

In my laptop I have already installed Virtual-box and docker-machine tools.I am going to 
deploying Rancher OS VM using docker-machine command.

C:\Users\Rengs>docker-machine create -d virtualbox --virtualbox-boot2docker-url https://releases.rancher.com/os/latest/rancheros.iso Rancher-test
Running pre-create checks...
(Rancher-test) Boot2Docker URL was explicitly set to "https://releases.rancher.com/os/latest/rancheros.iso" at create time, so Docker Machine cannot upgrade thi
Creating machine...
(Rancher-test) Boot2Docker URL was explicitly set to "https://releases.rancher.com/os/latest/rancheros.iso" at create time, so Docker Machine cannot upgrade thi
(Rancher-test) Downloading C:\Users\Rengs\.docker\machine\cache\boot2docker.iso from https://releases.rancher.com/os/latest/rancheros.iso...
(Rancher-test) 0%....10%....20%....30%....40%....50%....60%....70%....80%....90%....100%
(Rancher-test) Creating VirtualBox VM...
(Rancher-test) Creating SSH key...
(Rancher-test) Starting the VM...
(Rancher-test) Check network to re-create if needed...
(Rancher-test) Waiting for an IP...
Waiting for machine to be running, this may take a few minutes...
Detecting operating system of created instance...
Waiting for SSH to be available...
Detecting the provisioner...
Provisioning with rancheros...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
Checking connection to Docker...
Docker is up and running!
To see how to connect your Docker Client to the Docker Engine running on this virtual machine, 
run: docker-machine env Rancher-test
C:\Users\Rengs>docker-machine ls
NAME           ACTIVE   DRIVER       STATE     URL        SWARM   DOCKER    ERRORS
default        -        virtualbox   Saved                                  Unknown
Rancher-test   -        virtualbox   Running   tcp://192.168.99.100:2376    v1.12.6

C:\Users\Rengs>
Login to the Rancher VM
C:\Users\Rengs>docker-machine ssh Rancher-test
[docker@Rancher-test ~]$

Verifying the Rancher OS version and other details.
[docker@Rancher-test ~]$ uname -a
Linux Rancher-test 4.9.9-rancher #1 SMP Thu Feb 9 13:00:19 UTC 2017 x86_64 GNU/Linux
[docker@Rancher-test /]$ ros -v
ros version v0.8.0
Checking disk usage. It uses maximum 460MB disk space for OS.
[docker@Rancher-test ~]$
[docker@Rancher-test ~]$ df -h
Filesystem          Size      Used Available Use% Mounted on
overlay             19.1G    459.6M     17.7G   2% /
tmpfs               477.5M         0    477.5M   0% /dev
tmpfs               497.0M         0    497.0M   0% /sys/fs/cgroup
/dev/sda            19.1G    459.6M     17.7G   2% /home
none               497.0M    628.0K    496.4M   0% /run
/dev/sda           19.1G    459.6M     17.7G   2% /media
/dev/sda           19.1G    459.6M     17.7G   2% /opt
/dev/sda           19.1G    459.6M     17.7G   2% /mnt
/dev/sda           19.1G    459.6M     17.7G   2% /var/log
/dev/sda           19.1G    459.6M     17.7G   2% /usr/sbin/iptables
devtmpfs           477.5M         0    477.5M   0% /host/dev
shm                64.0M         0     64.0M   0% /host/dev/shm
/dev/sda           19.1G    459.6M     17.7G   2% /etc/docker
/dev/sda           19.1G    459.6M     17.7G   2% /etc/selinux
/dev/sda           19.1G    459.6M     17.7G   2% /usr/lib/modules
none                497.0M    628.0K    496.4M   0% /var/run
/dev/sda            19.1G    459.6M     17.7G   2% /usr/lib/firmware
/dev/sda            19.1G    459.6M     17.7G   2% /var/lib/rancher
/dev/sda            19.1G    459.6M     17.7G   2% /var/lib/docker
/dev/sda            19.1G    459.6M     17.7G   2% /usr/bin/ros
/dev/sda            19.1G    459.6M     17.7G   2% /usr/share/ros
/dev/sda            19.1G    459.6M     17.7G   2% /var/lib/rancher/conf
/dev/sda            19.1G    459.6M     17.7G   2% /etc/ssl/certs/ca-certificates.crt.rancher
/dev/sda            19.1G    459.6M     17.7G   2% /var/lib/rancher/cache
/dev/sda            19.1G    459.6M     17.7G   2% /etc/resolv.conf
/dev/sda            19.1G    459.6M     17.7G   2% /etc/hostname
/dev/sda            19.1G    459.6M     17.7G   2% /etc/hosts
shm                 64.0M         0     64.0M   0% /dev/shm
devtmpfs            477.5M         0    477.5M   0% /dev
shm                 64.0M         0     64.0M   0% /dev/shm
/dev/sda            19.1G    459.6M     17.7G   2% /var/lib/docker/overlay
[docker@Rancher-test ~]$
There are two Docker daemons running in Rancher OS.


 i) System Docker - Where Rancher OS runs System services. System Docker can  only used by root.
ii) Normal Docker - Where our application containers will be deployed

[docker@Rancher-test ~]$ sudo system-docker ps
CONTAINER ID        IMAGE          COMMAND        CREATED    STATUS     PORTS  NAMES
7e27e12d5258  ancher/os-docker:1.12.6  "ros user-docker"   6 min ago  Up 6 min docker
f02bd52571c5 rancher/os-console:v0.8.0 "/usr/bin/ros entrypo" 6 min ago Up 6 min console
dacc85ba24c6  rancher/os-base:v0.8.0  "/usr/bin/ros entrypo"  6 min ago  Up 6 min ntp
e50103a50c6f rancher/os-base:v0.8.0  "/usr/bin/ros entrypo" 6 min ago  Up 6 min  network
777b16ef8d95 rancher/os-base:v0.8.0  "/usr/bin/ros entrypo" 6 min ago Up 6 min   udev
974eb14cc628 rancher/os-base:v0.8.0  "/usr/bin/ros entrypo" 6 min ago Up 6 min  syslog
90478b595064 rancher/os-acpid:v0.8.0  "/usr/bin/ros entrypo" 6 min ago Up 6 min acpid


[docker@Rancher-test /]$ sudo ros engine list
disabled docker-1.10.3
disabled docker-1.11.2
current  docker-1.12.6
disabled docker-1.13.1
[docker@Rancher-test ~]$ sudo docker ps
CONTAINER ID        IMAGE       COMMAND      CREATED      STATUS      PORTS    NAMES
[docker@Rancher-test ~]$ docker --version
Docker version 1.12.6, build 78d1802

 

ROS administration commands

 Changing docker engine version in ROS
sudo ros engine switch docker-1.11.2

Start other docker engine after next reboot
$ sudo ros engine enable docker-1.10.3
Network configuration
$ sudo ros config set rancher.network.interfaces.eth1.address 172.68.1.100/24
$ sudo ros config set rancher.network.interfaces.eth1.gateway 172.68.1.1
$ sudo ros config set rancher.network.interfaces.eth1.mtu 1500
$ sudo ros config set rancher.network.interfaces.eth1.dhcp false

Upgrade ROS
[docker@Rancher-test /]$ sudo ros os version
v0.8.0
[docker@Rancher-test /]$ sudo ros os list
rancher/os:v0.8.0 remote latest running
rancher/os:v0.7.1 remote available
rancher/os:v0.7.0 remote available
rancher/os:v0.6.1 remote available
rancher/os:v0.6.0 remote available
rancher/os:v0.5.0 remote available
rancher/os:v0.4.5 remote available
rancher/os:v0.4.4 remote available
rancher/os:v0.4.3 remote available
rancher/os:v0.4.2 remote available
rancher/os:v0.4.1 remote available
rancher/os:v0.4.0 remote available
[docker@Rancher-test /]$
[docker@Rancher-test /]$ sudo ros os upgrade
Upgrading to rancher/os:v0.8.0
Already at version v0.8.0. Continue anyway [y/N]: N
[docker@Rancher-test /]$

Configuration file

sudo cat /var/lib/rancher/conf/cloud-config.yml

Hands-on Docker 1.12

Docker 1.12: They introduced with many features, mainly docker engine with in-built orchestration. 
When I worked in previous  docker engine the configuration part was very complex. I need to install and configure manually Docker engine, service registry and discovery, load balance, etc.,  Now I no longer have to install an external discovery service (consul, etcd, zookeeper) or a separate scheduling service. TLS is configured end-to-end out of the box, there is no "insecure mode". There is no doubt in my mind that the new Docker Swarm is the quickest way to get a production-ready docker-native cluster up and running. It is very simple and user friendly. 
I tested single master and single node Docker swarm cluster. New Architecture of Docker 1.12


Creating Swarm master 
=======================
root@docker-swarm-master-secondary:~# docker swarm init --listen-addr 192.168.109.5:2377
Swarm initialized: current node (cq27v0fz0ch53z4evtov8rwpt) is now a manager.
root@docker-swarm-master-secondary:~# docker node ls
ID                           NAME                                       MEMBERSHIP  STATUS  AVAILABILITY  MANAGER STATUS
cq27v0fz0ch53z4evtov8rwpt *  docker-swarm-master-secondary.localdomain  Accepted    Ready   Active        Leader

Adding worker node to the cluster
===========================
root@container-node2:~# docker swarm join --listen-addr 192.168.109.8:2377 192.168.109.5:2377
This node joined a Swarm as a worker.

on Master
==========
root@docker-swarm-master-secondary:~# docker node ls
ID                           NAME                                       MEMBERSHIP  STATUS  AVAILABILITY  MANAGER STATUS
600n95at82b1vvwk3bex5fuo5    container-node2.localdomain                Accepted    Ready   Active
cq27v0fz0ch53z4evtov8rwpt *  docker-swarm-master-secondary.localdomain  Accepted    Ready   Active        Leader


Testing Sample WEB APPLICATION
===============================

Creating Webpage Service
----------------------------------------
root@docker-swarm-master-secondary:~# docker service create --name webpage --publish 80:80 sixeyed/docker-swarm-walkthrough
9i9a4kdj56sot50lbyoxxwbbf

root@docker-swarm-master-secondary:~# docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

root@docker-swarm-master-secondary:~# docker service ls
ID            NAME     REPLICAS  IMAGE                             COMMAND
9i9a4kdj56so  webpage  1/1       sixeyed/docker-swarm-walkthrough

Checking Service configuration in readable format
-----------------------------------------------------------------------
root@docker-swarm-master-secondary:~# docker service inspect webpage --pretty
ID:             9i9a4kdj56sot50lbyoxxwbbf
Name:           webpage
Mode:           Replicated
 Replicas:      1
Placement:
 Strategy:      Spread
UpdateConfig:
 Parallelism:   0
ContainerSpec:
 Image:         sixeyed/docker-swarm-walkthrough
Resources:
Reservations:
Limits:
Ports:
 Name =
 Protocol = tcp
 TargetPort = 80
 PublishedPort = 80

Displaying task details of webpage service
--------------------------------------------------------------
root@docker-swarm-master-secondary:~# docker service tasks webpage
ID                         NAME       SERVICE  IMAGE                             LAST STATE         DESIRED STATE  NODE
b5p34exnmp49n7wjzzp1llgdh  webpage.1  webpage  sixeyed/docker-swarm-walkthrough  Running 2 minutes  Running        docker-swarm-master-secondary.localdomain

Creating one more Replica for the service(scaleup)
---------------------------------------------------------------------
root@docker-swarm-master-secondary:~# docker service update --replicas 2 webpage
webpage
root@docker-swarm-master-secondary:~# docker service tasks webpage
ID                         NAME       SERVICE  IMAGE                             LAST STATE           DESIRED STATE  NODE
b5p34exnmp49n7wjzzp1llgdh  webpage.1  webpage  sixeyed/docker-swarm-walkthrough  Running 3 minutes    Running        docker-swarm-master-secondary.localdomain
03bawni3ynq654p7snhucugvj  webpage.2  webpage  sixeyed/docker-swarm-walkthrough  Preparing 9 seconds  Running        container-node2.localdomain

For more details about docker 1.12
https://blog.docker.com/2016/06/docker-1-12-built-in-orchestration/

3 Node Consul and swarm cluster for Docker ennvironment


Main objective of this lab is to test the 3 node service discovery cluster for the Container environment. I configured Consul service discovery cluster for Docker Swarm setup.

1. Docker engine Installation

root@docker-swarm-master-primary:/etc/apt/sources.list.d# ls
docker.list  partner.list  partner.list.save  vchs.list
root@docker-swarm-master-primary:/etc/apt/sources.list.d# mv vchs.list vchs.list.orig
root@docker-swarm-master-primary:/etc/apt/sources.list.d#
root@docker-swarm-master-primary:/etc/apt/sources.list.d# apt-get clean
root@docker-swarm-master-primary:/etc/apt/sources.list.d# apt-get update

root@docker-swarm-master-primary:/etc/apt/sources.list.d# apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --secret-keyring /tmp/tmp.aw5jZzZ527 --trustdb-name /etc/apt/trustdb.gpg --keyring /etc/apt/trusted.gpg --primary-keyring /etc/apt/trusted.gpg --keyring /etc/apt/trusted.gpg.d//vchs-cloudimage-keyring.gpg --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
gpg: requesting key 2C52609D from hkp server p80.pool.sks-keyservers.net
gpg: key 2C52609D: public key "Docker Release Tool (releasedocker) <docker@docker.com>" imported
gpg: no ultimately trusted keys found
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)

Create new file
root@docker-swarm-master-primary:/etc/apt/sources.list.d# more docker.list
deb https://apt.dockerproject.org/repo ubuntu-precise main

Verify APT is pulling from the right repository
root@docker-swarm-master-primary:/etc/apt/sources.list.d# apt-cache policy docker-engine
docker-engine:
  Installed: (none)
  Candidate: 1.11.1-0~precise
  Version table:
     1.11.1-0~precise 0
        500 https://apt.dockerproject.org/repo/ ubuntu-precise/main amd64 Packages
     1.11.0-0~precise 0
        500 https://apt.dockerproject.org/repo/ ubuntu-precise/main amd64 Packages
     1.10.3-0~precise 0
        500 https://apt.dockerproject.org/repo/ ubuntu-precise/main amd64 Packages
     1.10.2-0~precise 0
        500 https://apt.dockerproject.org/repo/ ubuntu-precise/main amd64 Packages
     1.10.1-0~precise 0
        500 https://apt.dockerproject.org/repo/ ubuntu-precise/main amd64 Packages
     1.10.0-0~precise 0
        500 https://apt.dockerproject.org/repo/ ubuntu-precise/main amd64 Packages
     1.9.1-0~precise 0
        500 https://apt.dockerproject.org/repo/ ubuntu-precise/main amd64 Packages
     1.9.0-0~precise 0
        500 https://apt.dockerproject.org/repo/ ubuntu-precise/main amd64 Packages
     1.8.3-0~precise 0
        500 https://apt.dockerproject.org/repo/ ubuntu-precise/main amd64 Packages
     1.8.2-0~precise 0
        500 https://apt.dockerproject.org/repo/ ubuntu-precise/main amd64 Packages
     1.8.1-0~precise 0
        500 https://apt.dockerproject.org/repo/ ubuntu-precise/main amd64 Packages
     1.8.0-0~precise 0
        500 https://apt.dockerproject.org/repo/ ubuntu-precise/main amd64 Packages
     1.7.1-0~precise 0
        500 https://apt.dockerproject.org/repo/ ubuntu-precise/main amd64 Packages
     1.6.2-0~precise 0
        500 https://apt.dockerproject.org/repo/ ubuntu-precise/main amd64 Packages
     1.6.1-0~precise 0
        500 https://apt.dockerproject.org/repo/ ubuntu-precise/main amd64 Packages
     1.6.0-0~precise 0
        500 https://apt.dockerproject.org/repo/ ubuntu-precise/main amd64 Packages
     1.5.0-0~precise 0
        500 https://apt.dockerproject.org/repo/ ubuntu-precise/main amd64 Packages
root@docker-swarm-master-primary:/etc/apt/sources.list.d#

Install both required and optional packages
root@docker-swarm-master-primary:/etc/apt/sources.list.d# apt-get install linux-image-generic-lts-trusty
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
  linux-image-3.13.0-86-generic
Suggested packages:
  fdutils linux-lts-trusty-doc-3.13.0 linux-lts-trusty-source-3.13.0 linux-lts-trusty-tools linux-headers-3.13.0-86-generic
The following NEW packages will be installed:
  linux-image-3.13.0-86-generic
The following packages will be upgraded:
  linux-image-generic-lts-trusty
1 upgraded, 1 newly installed, 0 to remove and 99 not upgraded.
Need to get 52.6 MB of archives.
After this operation, 198 MB of additional disk space will be used.
Do you want to continue [Y/n]? y
Get:1 http://archive.ubuntu.com/ubuntu/ precise-updates/main linux-image-3.13.0-86-generic amd64 3.13.0-86.131~precise1 [52.6 MB]
Get:2 http://archive.ubuntu.com/ubuntu/ precise-updates/main linux-image-generic-lts-trusty amd64 3.13.0.86.78 [2,340 B]
Fetched 52.6 MB in 8s (6,074 kB/s)
Selecting previously unselected package linux-image-3.13.0-86-generic.
(Reading database ... 52292 files and directories currently installed.)
Unpacking linux-image-3.13.0-86-generic (from .../linux-image-3.13.0-86-generic_3.13.0-86.131~precise1_amd64.deb) ...
Done.
Preparing to replace linux-image-generic-lts-trusty 3.13.0.44.38 (using .../linux-image-generic-lts-trusty_3.13.0.86.78_amd64.deb) ...
Unpacking replacement linux-image-generic-lts-trusty ...
Setting up linux-image-3.13.0-86-generic (3.13.0-86.131~precise1) ...
Running depmod.
update-initramfs: deferring update (hook will be called later)
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 3.13.0-86-generic /boot/vmlinuz-3.13.0-86-generic
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 3.13.0-86-generic /boot/vmlinuz-3.13.0-86-generic
update-initramfs: Generating /boot/initrd.img-3.13.0-86-generic
run-parts: executing /etc/kernel/postinst.d/update-notifier 3.13.0-86-generic /boot/vmlinuz-3.13.0-86-generic
run-parts: executing /etc/kernel/postinst.d/x-grub-legacy-ec2 3.13.0-86-generic /boot/vmlinuz-3.13.0-86-generic
Searching for GRUB installation directory ... found: /boot/grub
Searching for default file ... found: /boot/grub/default
Testing for an existing GRUB menu.lst file ... found: /boot/grub/menu.lst
Searching for splash image ... none found, skipping ...
Found kernel: /boot/vmlinuz-3.13.0-86-generic
Found kernel: /boot/vmlinuz-3.13.0-44-generic
Found kernel: /boot/vmlinuz-3.2.0-75-virtual
Found kernel: /boot/memtest86+.bin
Replacing config file /run/grub/menu.lst with new version
Updating /boot/grub/menu.lst ... done

run-parts: executing /etc/kernel/postinst.d/zz-update-grub 3.13.0-86-generic /boot/vmlinuz-3.13.0-86-generic
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-3.13.0-86-generic
Found initrd image: /boot/initrd.img-3.13.0-86-generic
Found linux image: /boot/vmlinuz-3.13.0-44-generic
Found initrd image: /boot/initrd.img-3.13.0-44-generic
Found linux image: /boot/vmlinuz-3.2.0-75-virtual
Found initrd image: /boot/initrd.img-3.2.0-75-virtual
Found memtest86+ image: /boot/memtest86+.bin
done
Setting up linux-image-generic-lts-trusty (3.13.0.86.78) ...
root@docker-swarm-master-primary:/etc/apt/sources.list.d#
root@docker-swarm-master-primary:/etc/apt/sources.list.d#
root@docker-swarm-master-primary:/etc/apt/sources.list.d# reboot

Configuring Docker Daemon
--------------------------------------------
root@docker-swarm-master-tertiary:/etc/apt/sources.list.d# apt-get install docker-engine
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
  aufs-tools cgroup-lite git git-man libapparmor1 liberror-perl libltdl7
Suggested packages:
  git-daemon-run git-daemon-sysvinit git-doc git-el git-arch git-cvs git-svn git-email git-gui gitk gitweb
The following NEW packages will be installed:
  aufs-tools cgroup-lite docker-engine git git-man libapparmor1 liberror-perl libltdl7
0 upgraded, 8 newly installed, 0 to remove and 100 not upgraded.
Need to get 26.9 MB of archives.
After this operation, 87.9 MB of additional disk space will be used.
Do you want to continue [Y/n]? y
Get:1 http://archive.ubuntu.com/ubuntu/ precise/main libltdl7 amd64 2.4.2-1ubuntu1 [37.6 kB]
Get:2 http://archive.ubuntu.com/ubuntu/ precise/universe aufs-tools amd64 1:3.0+20111101-1ubuntu1 [98.8 kB]
Get:3 http://archive.ubuntu.com/ubuntu/ precise-updates/main libapparmor1 amd64 2.7.102-0ubuntu3.10 [37.3 kB]
Get:4 http://archive.ubuntu.com/ubuntu/ precise/main liberror-perl all 0.17-1 [23.8 kB]
Get:5 http://archive.ubuntu.com/ubuntu/ precise-updates/main git-man all 1:1.7.9.5-1ubuntu0.3 [632 kB]
Get:6 https://apt.dockerproject.org/repo/ ubuntu-precise/main docker-engine amd64 1.11.1-0~precise [19.9 MB]
Get:7 http://archive.ubuntu.com/ubuntu/ precise-updates/main git amd64 1:1.7.9.5-1ubuntu0.3 [6,103 kB]
Get:8 http://archive.ubuntu.com/ubuntu/ precise-updates/main cgroup-lite all 1.1.5 [4,098 B]
Fetched 26.9 MB in 5s (5,050 kB/s)
Selecting previously unselected package libltdl7.
(Reading database ... 52292 files and directories currently installed.)
Unpacking libltdl7 (from .../libltdl7_2.4.2-1ubuntu1_amd64.deb) ...
Selecting previously unselected package aufs-tools.
Unpacking aufs-tools (from .../aufs-tools_1%3a3.0+20111101-1ubuntu1_amd64.deb) ...
Selecting previously unselected package libapparmor1.
Unpacking libapparmor1 (from .../libapparmor1_2.7.102-0ubuntu3.10_amd64.deb) ...
Selecting previously unselected package docker-engine.
Unpacking docker-engine (from .../docker-engine_1.11.1-0~precise_amd64.deb) ...
Selecting previously unselected package liberror-perl.
Unpacking liberror-perl (from .../liberror-perl_0.17-1_all.deb) ...
Selecting previously unselected package git-man.
Unpacking git-man (from .../git-man_1%3a1.7.9.5-1ubuntu0.3_all.deb) ...
Selecting previously unselected package git.
Unpacking git (from .../git_1%3a1.7.9.5-1ubuntu0.3_amd64.deb) ...
Selecting previously unselected package cgroup-lite.
Unpacking cgroup-lite (from .../cgroup-lite_1.1.5_all.deb) ...
Processing triggers for man-db ...
Processing triggers for ureadahead ...
Setting up libltdl7 (2.4.2-1ubuntu1) ...
Setting up aufs-tools (1:3.0+20111101-1ubuntu1) ...
Setting up libapparmor1 (2.7.102-0ubuntu3.10) ...
Setting up docker-engine (1.11.1-0~precise) ...
docker start/running, process 5560
Setting up liberror-perl (0.17-1) ...
Setting up git-man (1:1.7.9.5-1ubuntu0.3) ...
Setting up git (1:1.7.9.5-1ubuntu0.3) ...
Setting up cgroup-lite (1.1.5) ...
cgroup-lite start/running
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place
root@docker-swarm-master-tertiary:/etc/apt/sources.list.d# service docker start
start: Job is already running: docker
root@docker-swarm-master-tertiary:/etc/apt/sources.list.d# docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
4276590986f6: Pull complete
a3ed95caeb02: Pull complete
Digest: sha256:4f32210e234b4ad5cac92efacc0a3d602b02476c754f13d517e1ada048e5a8ba
Status: Downloaded newer image for hello-world:latest

Hello from Docker.
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker Hub account:
 https://hub.docker.com

For more examples and ideas, visit:
 https://docs.docker.com/engine/userguide/

root@docker-swarm-master-tertiary:/etc/apt/sources.list.d# docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
root@docker-swarm-master-tertiary:/etc/apt/sources.list.d# docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                     PORTS               NAMES
4956aaf7d7b6        hello-world         "/hello"            8 seconds ago       Exited (0) 7 seconds ago                       condescending_hypatia
root@docker-swarm-master-tertiary:/etc/apt/sources.list.d#

CONSUL CLUSTER setup

NODE1 Configuration
root@docker-swarm-master-primary:~# docker run -d -h consul1 -v /mnt:/data \
    -p 192.168.109.4:8300:8300 \
    -p 192.168.109.4:8301:8301 \
    -p 192.168.109.4:8301:8301/udp \
    -p 192.168.109.4:8302:8302 \
    -p 192.168.109.4:8302:8302/udp \
    -p 192.168.109.4:8400:8400 \
    -p 192.168.109.4:8500:8500 \
    -p 192.168.109.4:53:53/udp \
    progrium/consul -server -advertise 192.168.109.4 -bootstrap-expect 3
root@docker-swarm-master-primary:~# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                                                                                                                                                          NAMES
f2ce90b8ac5a        progrium/consul     "/bin/start -server -"   7 seconds ago       Up 6 seconds        192.168.109.4:53->53/udp, 192.168.109.4:8300-8302->8300-8302/tcp, 192.168.109.4:8400->8400/tcp, 53/tcp, 192.168.109.4:8301-8302->8301-8302/udp, 192.168.109.4:8500->8500/tcp   elated_colden

NODE2 Configuration
root@docker-swarm-master-secondary:/# docker run -d -h node2 -v /mnt:/data  \
    -p 192.168.109.5:8300:8300 \
    -p 192.168.109.5:8301:8301 \
    -p 192.168.109.5:8301:8301/udp \
    -p 192.168.109.5:8302:8302 \
    -p 192.168.109.5:8302:8302/udp \
    -p 192.168.109.5:8400:8400 \
    -p 192.168.109.5:8500:8500 \
    -p 192.168.109.5:53:53/udp \
    progrium/consul -server -advertise 192.168.109.5 -join 192.168.109.4
root@docker-swarm-master-secondary:/# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                                                                                                                                                          NAMES
f05014b5588e        progrium/consul     "/bin/start -server -"   4 seconds ago       Up 2 seconds        192.168.109.5:53->53/udp, 192.168.109.5:8300-8302->8300-8302/tcp, 192.168.109.5:8400->8400/tcp, 53/tcp, 192.168.109.5:8301-8302->8301-8302/udp, 192.168.109.5:8500->8500/tcp   elegant_golick

NODE3 Configuration
root@docker-swarm-master-tertiary:/# docker run -d -h consul3 -v /mnt:/data  \
    -p 192.168.109.6:8300:8300 \
    -p 192.168.109.6:8301:8301 \
    -p 192.168.109.6:8301:8301/udp \
    -p 192.168.109.6:8302:8302 \
    -p 192.168.109.6:8302:8302/udp \
    -p 192.168.109.6:8400:8400 \
    -p 192.168.109.6:8500:8500 \
    -p 192.168.109.6:53:53/udp \
    progrium/consul -server -advertise 192.168.109.6 -join 192.168.109.4

root@docker-swarm-master-tertiary:/# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                                                                                                                                                          NAMES
f6a7d3bb73d5        progrium/consul     "/bin/start -server -"   6 seconds ago       Up 3 seconds        192.168.109.6:53->53/udp, 192.168.109.6:8300-8302->8300-8302/tcp, 192.168.109.6:8400->8400/tcp, 53/tcp, 192.168.109.6:8301-8302->8301-8302/udp, 192.168.109.6:8500->8500/tcp   cocky_mirzakhani
root@docker-swarm-master-tertiary:/# docker logs f6a7d3bb73d5
==> WARNING: It is highly recommended to set GOMAXPROCS higher than 1
==> Starting raft data migration...
==> Starting Consul agent...
==> Starting Consul agent RPC...
==> Joining cluster...
    Join completed. Synced with 1 initial agents
==> Consul agent running!
         Node name: 'consul3'
        Datacenter: 'dc1'
            Server: true (bootstrap: false)
       Client Addr: 0.0.0.0 (HTTP: 8500, HTTPS: -1, DNS: 53, RPC: 8400)
      Cluster Addr: 192.168.109.6 (LAN: 8301, WAN: 8302)
    Gossip encrypt: false, RPC-TLS: false, TLS-Incoming: false
             Atlas: <disabled>

==> Log data will now stream in as it occurs:

    2016/05/31 18:18:47 [INFO] serf: EventMemberJoin: consul3 192.168.109.6
    2016/05/31 18:18:47 [INFO] serf: EventMemberJoin: consul3.dc1 192.168.109.6
    2016/05/31 18:18:47 [INFO] raft: Node at 192.168.109.6:8300 [Follower] entering Follower state
    2016/05/31 18:18:47 [INFO] agent: (LAN) joining: [192.168.109.4]
    2016/05/31 18:18:47 [INFO] consul: adding server consul3 (Addr: 192.168.109.6:8300) (DC: dc1)
    2016/05/31 18:18:47 [INFO] consul: adding server consul3.dc1 (Addr: 192.168.109.6:8300) (DC: dc1)
    2016/05/31 18:18:47 [INFO] serf: EventMemberJoin: node2 192.168.109.5
    2016/05/31 18:18:47 [INFO] serf: EventMemberJoin: consul1 192.168.109.4
    2016/05/31 18:18:47 [INFO] agent: (LAN) joined: 1 Err: <nil>
    2016/05/31 18:18:47 [ERR] agent: failed to sync remote state: No cluster leader
    2016/05/31 18:18:47 [INFO] consul: adding server node2 (Addr: 192.168.109.5:8300) (DC: dc1)
    2016/05/31 18:18:47 [INFO] consul: adding server consul1 (Addr: 192.168.109.4:8300) (DC: dc1)
    2016/05/31 18:18:48 [INFO] consul: New leader elected: consul1
    2016/05/31 18:18:50 [INFO] agent: Synced service 'consul'

SWARM CLUSTER Configuration


SWARM MASTER
root@docker-swarm-master-primary:/# docker run -d -p 5500:5500 swarm manage -H :5500 --replication --advertise 192.168.109.4:5500 consul://192.168.109.4:8500
Unable to find image 'swarm:latest' locally
latest: Pulling from library/swarm
51436fd4bb0d: Pull complete
c31a5390266f: Pull complete
e40019be13ea: Pull complete
a3ed95caeb02: Pull complete
Digest: sha256:02fdbfa3740ac923dc0c9a579ba04efaa0d39b0c05eb933358fe7421fdaf17b7
Status: Downloaded newer image for swarm:latest
2f52a0c5c73cfd7c171fdb3aedfac76df18ad9100bdb6c393de4d651cbb66757
root@docker-swarm-master-primary:/# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                                                                                                                                                                                                                      NAMES
2f52a0c5c73c        swarm               "/swarm manage -H :55"   6 seconds ago       Up 3 seconds        2375/tcp, 0.0.0.0:5500->5500/tc                              p                                                                                                                                                                             gigantic_kilby
19bd02b79b48        progrium/consul     "/bin/start -server -"   3 minutes ago       Up 3 minutes        192.168.109.4:53->53/udp, 192.1                              68.109.4:8300-8302->8300-8302/tcp, 192.168.109.4:8400->8400/tcp, 53/tcp, 192.168.109.4:8301-8302->8301-8302/udp, 192.168.109.4:8500->850                              0/tcp   awesome_wescoff

Node 2
root@docker-swarm-master-secondary:/# docker run -d -p 5500:5500 swarm manage -H :5500 --replication --advertise 192.168.109.5:5500 consul://192.168.109.5:8500
Unable to find image 'swarm:latest' locally
latest: Pulling from library/swarm
51436fd4bb0d: Pull complete
c31a5390266f: Pull complete
e40019be13ea: Pull complete
a3ed95caeb02: Pull complete
Digest: sha256:02fdbfa3740ac923dc0c9a579ba04efaa0d39b0c05eb933358fe7421fdaf17b7
Status: Downloaded newer image for swarm:latest
dafb2d2d815e401868b1a037cb8ed6048ae9e3d76a8f2767e558c8da7dadf315

root@docker-swarm-master-secondary:/# docker -H tcp://192.168.109.5:5500 info
Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 0
Server Version: swarm/1.2.3
Role: replica
Primary: 192.168.109.4:5500
Strategy: spread
Filters: health, port, containerslots, dependency, affinity, constraint
Nodes: 0
Plugins:
 Volume:
 Network:
Kernel Version: 3.13.0-86-generic
Operating System: linux
Architecture: amd64
CPUs: 0
Total Memory: 0 B
Name: dafb2d2d815e
Docker Root Dir:
Debug mode (client): false
Debug mode (server): false
WARNING: No kernel memory limit support

Node3
root@docker-swarm-master-tertiary:/# docker run -d -p 5500:5500 swarm manage -H :5500 --replication --advertise 192.168.109.6:5500 consul://192.168.109.5:8500
Unable to find image 'swarm:latest' locally
latest: Pulling from library/swarm
51436fd4bb0d: Pull complete
c31a5390266f: Pull complete
e40019be13ea: Pull complete
a3ed95caeb02: Pull complete
Digest: sha256:02fdbfa3740ac923dc0c9a579ba04efaa0d39b0c05eb933358fe7421fdaf17b7
Status: Downloaded newer image for swarm:latest
c471cb29cc2033439c1ed39b30c8aa67b94493b90b8b0e7ba898abe4761c4442

root@docker-swarm-master-tertiary:/# docker logs c471cb29cc20
time="2016-06-01T17:52:50Z" level=info msg="Initializing discovery without TLS"
time="2016-06-01T17:52:50Z" level=info msg="Listening for HTTP" addr=":5500" proto=tcp
time="2016-06-01T17:52:50Z" level=info msg="Leader Election: Cluster leadership lost"
time="2016-06-01T17:52:50Z" level=info msg="New leader elected: 192.168.109.4:5500"

root@docker-swarm-master-tertiary:/# docker -H tcp://192.168.109.6:5500 info
Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 0
Server Version: swarm/1.2.3
Role: replica
Primary: 192.168.109.4:5500
Strategy: spread
Filters: health, port, containerslots, dependency, affinity, constraint
Nodes: 0
Plugins:
 Volume:
 Network:
Kernel Version: 3.13.0-44-generic
Operating System: linux
Architecture: amd64
CPUs: 0
Total Memory: 0 B
Name: c471cb29cc20
Docker Root Dir:
Debug mode (client): false
Debug mode (server): false
WARNING: No kernel memory limit support

SWARM node

root@docker-swarm-master-primary:/# docker -H=tcp://192.168.109.8:2375 run -d --restart=unless-stopped swarm join --advertise=192.168.109.8:2375 consul://192.168.109.4:8500
f461384ca0177029173b4880979eb25643e06615b7ef7022d636cf121cb51c6a
root@docker-swarm-master-primary:/# docker -H :5500 info
Containers: 1
 Running: 1
 Paused: 0
 Stopped: 0
Images: 1
Server Version: swarm/1.2.3
Role: primary
Strategy: spread
Filters: health, port, containerslots, dependency, affinity, constraint
Nodes: 1
 container-node2.localdomain: 192.168.109.8:2375
  └ ID: CEPC:XXMY:N5PB:6PS7:CA6E:QNOA:KN3E:EYBU:IL3Q:SKCK:MLXV:7NQL
  └ Status: Healthy
  └ Containers: 1
  └ Reserved CPUs: 0 / 1
  └ Reserved Memory: 0 B / 2.052 GiB
  └ Labels: executiondriver=, kernelversion=3.13.0-44-generic, operatingsystem=Ubuntu precise (12.04.5 LTS), storagedriver=aufs
  └ UpdatedAt: 2016-06-17T19:34:29Z
  └ ServerVersion: 1.11.2
Plugins:
 Volume:
 Network:
Kernel Version: 3.13.0-86-generic
Operating System: linux
Architecture: amd64
CPUs: 1
Total Memory: 2.052 GiB
Name: master1
Docker Root Dir:
Debug mode (client): false
Debug mode (server): false
WARNING: No kernel memory limit support

root@docker-swarm-master-primary:~# docker -H=tcp://192.168.109.7:2375 run -d --restart=unless-stopped swarm join --advertise=192.168.109.7:2375 consul://192.168.109.4:8500
fe810770276b2c1a56ed1703a03dfb1ee5485465d7021e244c9d3dfd22849312

root@docker-swarm-master-primary:~# docker -H :5500 info
Containers: 2
 Running: 2
 Paused: 0
 Stopped: 0
Images: 2
Server Version: swarm/1.2.3
Role: primary
Strategy: spread
Filters: health, port, containerslots, dependency, affinity, constraint
Nodes: 2
 container-node1.localdomain: 192.168.109.7:2375
  └ ID: 2PMY:LKZP:JGI7:ATXH:RL65:FKXH:4RWU:UDFR:LHG4:XSOJ:56ZM:TCGP
  └ Status: Healthy
  └ Containers: 1
  └ Reserved CPUs: 0 / 1
  └ Reserved Memory: 0 B / 2.052 GiB
  └ Labels: executiondriver=, kernelversion=3.13.0-86-generic, operatingsystem=Ubuntu precise (12.04.5 LTS), storagedriver=aufs
  └ UpdatedAt: 2016-06-17T19:40:24Z
  └ ServerVersion: 1.11.2
 container-node2.localdomain: 192.168.109.8:2375
  └ ID: CEPC:XXMY:N5PB:6PS7:CA6E:QNOA:KN3E:EYBU:IL3Q:SKCK:MLXV:7NQL
  └ Status: Healthy
  └ Containers: 1
  └ Reserved CPUs: 0 / 1
  └ Reserved Memory: 0 B / 2.052 GiB
  └ Labels: executiondriver=, kernelversion=3.13.0-44-generic, operatingsystem=Ubuntu precise (12.04.5 LTS), storagedriver=aufs
  └ UpdatedAt: 2016-06-17T19:39:46Z
  └ ServerVersion: 1.11.2
Plugins:
 Volume:
 Network:
Kernel Version: 3.13.0-86-generic
Operating System: linux
Architecture: amd64
CPUs: 2
Total Memory: 4.105 GiB
Name: master1
Docker Root Dir:
Debug mode (client): false
Debug mode (server): false
WARNING: No kernel memory limit support

root@docker-swarm-master-primary:~#

AWS Autoscaling demo lab Configuring VPC, Public/Private subnets, Internet Gateway (IGW), NAT gateway, etc.,   Crea...