PuppetAgileInfrastructure < Main

Main Web>TWikiUsers>SimaolhodaBaymani>PuppetAgileInfrastructure (2016-08-08, unknown)

Cloud Infrastructure

sbaymani@aiadm060 ~]$ openstack
(openstack) help

Shell commands (type help <topic>):
===================================
cmdenvironment  edit  hi       l   list  pause  r    save  shell      show
ed              help  history  li  load  py     run  set   shortcuts

Undocumented commands:
======================
EOF  eof  exit  q  quit

[sbaymani@aiadm060 ~]$ openstack keypair create ai-keypair > ai-keypair.pem # will pop up in the web-GUI
[sbaymani@aiadm060 ~]$ cat ai-keypair.pem 

[sbaymani@aiadm060 ~]$ env | grep OS
OS_PROJECT_DOMAIN_ID=default
HOSTNAME=aiadm060.cern.ch
OS_PROJECT_NAME=Personal sbaymani # see drop down in web-GUI for projects I have access to. This env can be changed to any of those
OS_IDENTITY_API_VERSION=3
OS_AUTH_TYPE=v3kerberos
OS_AUTH_URL=https://keystone.cern.ch/krb/v3
LCG_GFAL_INFOSYS=lcg-bdii.cern.ch:2170
[sbaymani@aiadm060 ~]$

Web-GUI

CC7 Base vs Extra - base, no afs, no kerberos, needs puppet to manage and configure, extra is an server-minimal with domain access, kerberos and other things prepared

Base: can only enter using single user mode. There is no kerberos so no user data base -> can't login!! Only using ssh and logging in as root. See below. Extra: creator of machine is also in sudoers list

[sbaymani@aiadm060 ~]$ openstack image list
[sbaymani@aiadm060 ~]$ openstack flavor list
[sbaymani@aiadm060 ~]$ openstack keypair list
[sbaymani@aiadm060 ~]$ openstack server create --image 'CC7 Base - x86_64 [2015-06-12]' --flavor m1.tiny --key-name ai-keypair sbaymani-AI-02
+--------------------------------------+-----------------------------------------------------------------------+
| Field                                | Value                                                                 |
+--------------------------------------+-----------------------------------------------------------------------+
| OS-DCF:diskConfig                    | MANUAL                                                                |
| OS-EXT-AZ:availability_zone          | nova                                                                  |
| OS-EXT-STS:power_state               | 0                                                                     |
| OS-EXT-STS:task_state                | scheduling                                                            |
| OS-EXT-STS:vm_state                  | building                                                              |
| OS-SRV-USG:launched_at               | None                                                                  |
| OS-SRV-USG:terminated_at             | None                                                                  |
| accessIPv4                           |                                                                       |
| accessIPv6                           |                                                                       |
| addresses                            |                                                                       |
| config_drive                         |                                                                       |
| created                              | 2015-10-02T07:40:47Z                                                  |
| flavor                               | m1.tiny (1)                                                           |
| hostId                               |                                                                       |
| id                                   | 4c7e6d04-9d03-4311-908d-5335b0b2db14                                  |
| image                                | CC7 Base - x86_64 [2015-06-12] (217fdcb1-19ca-42b0-946c-5d51fe57c500) |
| key_name                             | ai-keypair                                                            |
| name                                 | sbaymani-AI-02                                                        |
| os-extended-volumes:volumes_attached | []                                                                    |
| progress                             | 0                                                                     |
| properties                           |                                                                       |
| security_groups                      | [{u'name': u'default'}]                                               |
| status                               | BUILD                                                                 |
| tenant_id                            | 3f71284d-b2f7-4a4f-bace-7f0ae12604f8                                  |
| updated                              | 2015-10-02T07:40:47Z                                                  |
| user_id                              | sbaymani                                                              |
+--------------------------------------+-----------------------------------------------------------------------+

[sbaymani@aiadm060 ~]$ openstack server list
+--------------------------------------+----------------------+--------+--------------------------------------------------------+
| ID                                   | Name                 | Status | Networks                                               |
+--------------------------------------+----------------------+--------+--------------------------------------------------------+
| 4c7e6d04-9d03-4311-908d-5335b0b2db14 | sbaymani-AI-02       | BUILD  |                                                        |
| d489e367-4d88-43f6-90c8-17cb39f717d7 | sbaymani-AI-01       | BUILD  |                                                        |
| 39790693-b267-4239-a5cb-628729e4fa69 | sbaymani-germain     | ACTIVE | CERN_NETWORK=128.142.132.105, 2001:1458:301:97::100:5a |
| 727d6cf3-563e-40b4-b795-acf92487db5b | sbaymani-du-chatelet | ACTIVE | CERN_NETWORK=128.142.128.198, 2001:1458:301:d9::100:b8 |
+--------------------------------------+----------------------+--------+--------------------------------------------------------+

[sbaymani@aiadm060 ~]$ openstack server list -c Name # filter on column name - lots of other options for filtering
+----------------------+
| Name                 |
+----------------------+
| sbaymani-AI-02       |
| sbaymani-AI-01       |
| sbaymani-germain     |
| sbaymani-du-chatelet |
+----------------------+

[sbaymani@aiadm060 ~]$ ssh -i ai-keypair.pem root@sbaymani-AI-01 # this is a Base machine
The authenticity of host 'sbaymani-ai-01 (188.184.129.30)' can't be established.
RSA key fingerprint is 39:14:0a:c6:59:31:98:75:cb:03:02:1e:f3:33:a7:b6.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'sbaymani-ai-01,188.184.129.30' (RSA) to the list of known hosts.
[root@sbaymani-ai-01 ~]# 

[root@sbaymani-ai-01 ~]# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1        10G  1.1G  9.0G  11% /
devtmpfs        228M     0  228M   0% /dev
tmpfs           245M     0  245M   0% /dev/shm
tmpfs           245M  4.3M  241M   2% /run
tmpfs           245M     0  245M   0% /sys/fs/cgroup
tmpfs           245M  4.3M  241M   2% /etc/machine-id

# create persistent storage. Uses CEPH, which has replication of 3. Volume names are not unique.
[sbaymani@aiadm060 ~]$ openstack volume create --size 1 AI01vol
+---------------------+--------------------------------------+
| Field               | Value                                |
+---------------------+--------------------------------------+
| attachments         | []                                   |
| availability_zone   | nova                                 |
| bootable            | false                                |
| created_at          | 2015-10-02T07:56:00.876168           |
| display_description | None                                 |
| display_name        | AI01vol                              |
| encrypted           | False                                |
| id                  | bc9850a4-739f-4b51-ba50-b69e090790f6 |
| multiattach         | false                                |
| properties          |                                      |
| size                | 1                                    |
| snapshot_id         | None                                 |
| source_volid        | None                                 |
| status              | creating                             |
| type                | standard                             |
+---------------------+--------------------------------------+
[sbaymani@aiadm060 ~]$ 

[sbaymani@aiadm060 ~]$ openstack volume list
+--------------------------------------+--------------+-----------+------+----------+
| ID                                   | Display Name | Status    | Size | Attached |
+--------------------------------------+--------------+-----------+------+----------+
| bc9850a4-739f-4b51-ba50-b69e090790f6 | AI01vol      | available |    1 |          |
+--------------------------------------+--------------+-----------+------+----------+
[sbaymani@aiadm060 ~]$ 
[sbaymani@aiadm060 ~]$ openstack server add volume
usage: openstack server add volume [-h] [--device <device>] <server> <volume>
openstack server add volume: error: too few arguments
[sbaymani@aiadm060 ~]$ openstack server add volume sbaymani-AI-01 AI01vol
[sbaymani@aiadm060 ~]$ 
[sbaymani@aiadm060 ~]$ openstack volume list
+--------------------------------------+--------------+--------+------+----------+
| ID                                   | Display Name | Status | Size | Attached |
+--------------------------------------+--------------+--------+------+----------+
| bc9850a4-739f-4b51-ba50-b69e090790f6 | AI01vol      | in-use |    1 |          |
+--------------------------------------+--------------+--------+------+----------+
[sbaymani@aiadm060 ~]$ 

[sbaymani@aiadm060 ~]$ openstack server show sbaymani-AI-01
+--------------------------------------+-----------------------------------------------------------------------+
| Field                                | Value                                                                 |
+--------------------------------------+-----------------------------------------------------------------------+
| OS-DCF:diskConfig                    | MANUAL                                                                |
| OS-EXT-AZ:availability_zone          | cern-geneva-b                                                         |
| OS-EXT-STS:power_state               | 1                                                                     |
| OS-EXT-STS:task_state                | None                                                                  |
| OS-EXT-STS:vm_state                  | active                                                                |
| OS-SRV-USG:launched_at               | 2015-10-02T07:42:51.000000                                            |
| OS-SRV-USG:terminated_at             | None                                                                  |
| accessIPv4                           |                                                                       |
| accessIPv6                           |                                                                       |
| addresses                            | CERN_NETWORK=188.184.129.30, 2001:1458:301:dc::100:11c                |
| config_drive                         |                                                                       |
| created                              | 2015-10-02T07:34:28Z                                                  |
| flavor                               | m1.tiny (1)                                                           |
| hostId                               | 2d13c7ccdb3cbf4d2ce13960f55945cef15c905a388bec8795b0bcbf              |
| id                                   | d489e367-4d88-43f6-90c8-17cb39f717d7                                  |
| image                                | CC7 Base - x86_64 [2015-06-12] (217fdcb1-19ca-42b0-946c-5d51fe57c500) |
| key_name                             | ai-keypair                                                            |
| name                                 | sbaymani-AI-01                                                        |
| os-extended-volumes:volumes_attached | [{u'id': u'bc9850a4-739f-4b51-ba50-b69e090790f6'}]                    |
| progress                             | 0                                                                     |
| properties                           |                                                                       |
| security_groups                      | [{u'name': u'default'}]                                               |
| status                               | ACTIVE                                                                |
| tenant_id                            | 3f71284d-b2f7-4a4f-bace-7f0ae12604f8                                  |
| updated                              | 2015-10-02T07:42:52Z                                                  |
| user_id                              | sbaymani                                                              |
+--------------------------------------+-----------------------------------------------------------------------+
[sbaymani@aiadm060 ~]$ 

[root@sbaymani-ai-01 ~]# mkfst -t ext4 /dev/vdb
-bash: mkfst: command not found
[root@sbaymani-ai-01 ~]# mkfs -t ext4 /dev/vdb
mke2fs 1.42.9 (28-Dec-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
65536 inodes, 262144 blocks
13107 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=268435456
8 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks: 
   32768, 98304, 163840, 229376

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

[root@sbaymani-ai-01 ~]# mount /dev/vdb /mnt
[root@sbaymani-ai-01 ~]# 

[sbaymani@aiadm060 ~]$ openstack server remove volume sbaymani-AI-01 AI01vol
[sbaymani@aiadm060 ~]$ 

[sbaymani@aiadm060 ~]$ openstack server add volume sbaymani-AI-02 AI01vol
[sbaymani@aiadm060 ~]$ 
[sbaymani@aiadm060 ~]$ ssh -i ai-keypair.pem root@sbaymani-AI-02
The authenticity of host 'sbaymani-ai-02 (128.142.128.105)' can't be established.
RSA key fingerprint is 21:3b:1b:70:76:d6:78:80:c3:57:aa:80:a1:44:2d:84.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'sbaymani-ai-02,128.142.128.105' (RSA) to the list of known hosts.
[root@sbaymani-ai-02 ~]# 
[root@sbaymani-ai-02 ~]# 
[root@sbaymani-ai-02 ~]# 
[root@sbaymani-ai-02 ~]# fdisk -l

Disk /dev/vda: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000a010c

   Device Boot      Start         End      Blocks   Id  System
/dev/vda1   *        2048    20970332    10484142+  83  Linux

Disk /dev/vdb: 1073 MB, 1073741824 bytes, 2097152 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

[root@sbaymani-ai-02 ~]# mount /dev/vdb /mnt
[root@sbaymani-ai-02 ~]# ls /mnt/
lost+found
[root@sbaymani-ai-02 ~]# 

[sbaymani@aiadm060 ~]$ openstack volume list
+--------------------------------------+--------------+--------+------+----------+
| ID                                   | Display Name | Status | Size | Attached |
+--------------------------------------+--------------+--------+------+----------+
| bc9850a4-739f-4b51-ba50-b69e090790f6 | AI01vol      | in-use |    1 |          |
+--------------------------------------+--------------+--------+------+----------+
[sbaymani@aiadm060 ~]$ openstack server remove volume sbaymani-AI-02 AI01vol
[sbaymani@aiadm060 ~]$ openstack volume list
+--------------------------------------+--------------+-----------+------+----------+
| ID                                   | Display Name | Status    | Size | Attached |
+--------------------------------------+--------------+-----------+------+----------+
| bc9850a4-739f-4b51-ba50-b69e090790f6 | AI01vol      | available |    1 |          |
+--------------------------------------+--------------+-----------+------+----------+
[sbaymani@aiadm060 ~]$ openstack volume delete AI01vol
[sbaymani@aiadm060 ~]$ openstack volume list

[sbaymani@aiadm060 ~]$

Shared disks - Manila, they are looking into this. You can simulate shared disks using NFS, where one machine mounts the volume and the others connect using NFS.

You can create a machine that boots from a volume, where the image gets copied onto the volume. That way, if the machine goes down for some reason, the volume is retained and you only have to create a new machine that boots from the old volume. Also, this enables you to create machines with larger disk, if the image itself gets too big.

Resizing works like this: a new machine is created and the image is copied over. Moving an image from one project to another is difficult. Use a snapshot and share it with the other project. This is a cold move, i.e. you have to turn off the machine -> downtime. Images in personal projects are deleted when you leave CERN. IT prefers that you create production images in a non-personal project, because they don't know if you have personal information on it.

Configuration Management

How to provision, configure and monitor servers in an automated way. Puppet, Foreman, git.

Write code for managing your servers. Test your configuration before deploying them.

Hostgroups - subhostgroups. For grouping nodes into management chunks, for the same service.
Machines created using Nova are completely unmanaged.
If you have a managed machine and you have defined that AFS should run, the AFS module will always be up to date.
After a puppet run, a report is sent to Foreman.
Foreman is a link between the Puppet master and the machine, telling Puppet which group the machine belongs to.
Hiera for storing configuration parameters.
A hostgroup must always represent a service.
ACLs(foreman and git) are at the hostgroup level
Each hostgroup has it\s own git repository and email address it-hostgroup-HOSTGROUP
The git repo contains folders
- code/
- data/

hg=webstuff
class=hg_webstuff
file= code/manifests/iit.pp
Configurations are applied top-down.
Modules = libraries. If you want to install MySQL, just "include mysql" in your manifest.
shared = common stuff
individual = ??
Each module has also its own git repo.

Hiera has also a hostgroup hierarchy. Most specific wins.

Environments allow you to test stuff. All git repos has always production/ qa/ for production and testing. Put machine into qa env to test and then move to prod when ready. Create an environment that includes the changed manifest. Create a topic branch, move a machine into the newly created environment and test it. Then merge your change into qa and request that your module is tested.

Secrets -tbag.
You can specify which hostgroups and which machines that can access a secret

/mnt/puppet...

Chat: conference.jabber.cern.ch

http://configdocs.web.cern.ch/configdocs/ - follow tutorial here

create new playground
playground/sbaymani

sometimes the puppet run fails the first run, when you have just created the machine grey box =??

git clone https://:@git.cern.ch/kerberos/it-puppet-hostgroup-playground # expects you to be logged on to aiadm

Otherwise go to gitlab.cern.ch (?) it-puppet-hostgroup-playground

ai-bs-vm will register in all services your machine

[sbaymani@aiadm060 ~]$ ai-bs-vm --foreman-hostgroup playground/sbaymani --nova-image "SLC6 Server - x86_64 [2015-02-10]" --nova-sshkey ai-keypair sbaymani-ai-04.cern.ch
Trying to bootstrap 'sbaymani-ai-04.cern.ch'...
VM flavor: m1.small
Booting from image: SLC6 Server - x86_64 [2015-02-10]
VM tenant: Personal sbaymani
VM SSH key name: ai-keypair
Foreman environment: production
Foreman hostgroup: playground/sbaymani
Puppet master: it-puppet-masters-public.cern.ch
Certmgr server: baby02.cern.ch
Certmgr port: 8008
Roger server: woger.cern.ch
Roger port: 8201
Preparing dynamic user data...
Using '/usr/share/ai-tools/userdata/puppetinit' as userdata script template to init Puppet
Adding host 'sbaymani-ai-04.cern.ch' to Foreman...
Host 'sbaymani-ai-04.cern.ch' created in Foreman
Staging host 'sbaymani-ai-04.cern.ch' on Certmgr...
Host 'sbaymani-ai-04.cern.ch' staged
Using auth plugin: v3kerberos
Creating virtual machine 'sbaymani-ai-04'...
Request to create VM 'sbaymani-ai-04' sent
Adding 'sbaymani-ai-04.cern.ch' to Roger
----------------------------------------------------------------------
* Your machine is booting and the network is being configured right now,
  Puppet will run immediately after a successful boot process.
* It typically takes around 30 minutes between this command is
  executed and the first Puppet report arrives to Foreman:
  https://judy.cern.ch/hosts/sbaymani-ai-04.cern.ch/reports
  (although this depends a lot on the complexity of your configuration)
* After the initial configuration, if you've set rootegroups or
  rootusers in Foreman or Hiera you should be able to log in as
  root using your Kerberos credentials. The LANDB responsible
  has also root access by default.
* You can check the status of the VM creation request by running:
  'openstack server show sbaymani-ai-04'
* In case of problems, if you provided a SSH key when creating the VM
  use it to log into the box and take a look at /var/log/firstboot.log.
  Console log can be retrieved by using 'openstack console log show'.
----------------------------------------------------------------------
[sbaymani@aiadm060 ~]$ 

It will take a while before the machine turns up in foreman. You can search for it. It will need around 3 Puppet runs before being up to date.

There are flags for using latest SLC6, CC7 etc. E.g. --slc6

Set AI env: ai-rc "IT Cloud" e.g. for setting project.
ai-rc -s lxplus001 # to set the environment based on the server, i.e. which project that server belongs to

[sbaymani@aiadm060 ~]$ git config --global user.name "Sima Baymani"
[sbaymani@aiadm060 ~]$ git config --global user.email "sima.baymani@cern.ch"
[sbaymani@aiadm060 ~]$ git config --global branch.autosetuprebase always
[sbaymani@aiadm060 ~]$ mkdir ~/.git_template
[sbaymani@aiadm060 ~]$ git clone https://github.com/AlbertoPeon/puppet-git-hooks.git ~/.git_template/hooks
Initialized empty Git repository in /afs/cern.ch/user/s/sbaymani/.git_template/hooks/.git/
remote: Counting objects: 304, done.
remote: Total 304 (delta 0), reused 0 (delta 0), pack-reused 304
Receiving objects: 100% (304/304), 69.05 KiB, done.
Resolving deltas: 100% (168/168), done.
[sbaymani@aiadm060 ~]$ git config --global init.templatedir '~/.git_template'
[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ touch code/manifests/sbaymani.pp
[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ vim code/manifests/sbaymani.pp
class hg_playground::sbaymani {
        notify{"Hello Puppet World":}
}

[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ git commit -m "My first manifest!"
-bash: !": event not found
[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ git commit -m 'My first manifest!'
Checking puppet manifest syntax for code/manifests/sbaymani.pp...
[master 23e0512] My first manifest!
 1 files changed, 3 insertions(+), 0 deletions(-)
 create mode 100644 code/manifests/sbaymani.pp
[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ 

[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ git pull --rebase origin master && git push origin master
remote: Counting objects: 24, done.
remote: Compressing objects: 100% (20/20), done.
remote: Total 20 (delta 8), reused 0 (delta 0)
Unpacking objects: 100% (20/20), done.
From https://git.cern.ch/kerberos/it-puppet-hostgroup-playground
 * branch            master     -> FETCH_HEAD
First, rewinding head to replay your work on top of it...
Applying: My first manifest!
Counting objects: 8, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (5/5), 481 bytes, done.
Total 5 (delta 2), reused 0 (delta 0)
To https://:@git.cern.ch/kerberos/it-puppet-hostgroup-playground
   29eb23e..21ee631  master -> master
[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ 

[sbaymani@aiadm060 ~]$ ssh -i ai-keypair.pem root@sbaymani-ai-04
* ********************************************************************
* Welcome to sbaymani-ai-04.cern.ch, SLC, 6.7
* Archive of news is available in /etc/motd-archive
* Reminder: You have agreed to comply with the CERN computing rules
* http://cern.ch/ComputingRules
* Puppet environment: production
* Puppet hostgroup: playground/sbaymani
* This is for short-term playing and testing. Machines in here should have no expectation of being stable and may be deleted without warning.
* ********************************************************************
[root@sbaymani-ai-04 ~]# 
[root@sbaymani-ai-04 ~]# 
[root@sbaymani-ai-04 ~]# puppet agent -t -v
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
...
[root@sbaymani-ai-04 ~]# less /var/log/messages # search for Hello

The module should have been loaded by puppet because the class defined the exact hostgroup my machine belongs to. The notification should turn up both in the web-GUI report and in /var/log/messages

Puppet reports:

Green = nothing was done
Blue = changes were made, everything is fine
Red = errors

[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ git diff --cached
diff --git a/code/manifests/sbaymani.pp b/code/manifests/sbaymani.pp
index eb00da7..f934c9a 100644
--- a/code/manifests/sbaymani.pp
+++ b/code/manifests/sbaymani.pp
@@ -1,3 +1,5 @@
 class hg_playground::sbaymani {
        notify{"Hello Puppet World":}
+       include afs
+       include sssd
 }
[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ git commit -m "Adding afs and ssd to sbaymani manifest"

Checking puppet manifest syntax for code/manifests/sbaymani.pp...

[master c9d5e6e] Adding afs and ssd to sbaymani manifest
 1 files changed, 2 insertions(+), 0 deletions(-)
[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ git pull --rebase origin master && git push origin master

AFS needs two runs with Puppet to get ready.

[root@sbaymani-ai-04 ~]# puppet agent -t -v
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for sbaymani-ai-04.cern.ch
Info: Applying configuration version '1443785013'
Notice: /Stage[main]/Afs::Install/Package[openafs]/ensure: created
Notice: /Stage[main]/Afs::Install/Package[cern-wrappers]/ensure: created

If you want to play around on a machine withouth having Puppet disturbing, you can disable the agent. It is also possible to disable it for a time interval

[root@sbaymani-ai-04 ~]# puppet agent --disable
[root@sbaymani-ai-04 ~]# puppet agent --enable
[root@sbaymani-ai-04 ~]# puppet agent --help

[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ git diff --cached
diff --git a/data/hostgroup/playground/sbaymani.yaml b/data/hostgroup/playground/sbaymani.yaml
new file mode 100644
index 0000000..65bf7ee
--- /dev/null
+++ b/data/hostgroup/playground/sbaymani.yaml
@@ -0,0 +1,3 @@
+sssd::interactiveallowusers:
+        - sbaymani
+
[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ git commit -m "Giving sbaymani interactice access"
Checking yaml syntax for data/hostgroup/playground/sbaymani.yaml...
[master 960e8fb] Giving sbaymani interactice access
 1 files changed, 3 insertions(+), 0 deletions(-)
 create mode 100644 data/hostgroup/playground/sbaymani.yaml
[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ git pull --rebase origin master && git push origin master

[root@sbaymani-ai-04 ~]# puppet agent -t -v
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for sbaymani-ai-04.cern.ch
Info: Applying configuration version '1443786212'
Notice: /Stage[main]/Sssd::Config/File[/etc/sssd/sssd.conf]/content: 
--- /etc/sssd/sssd.conf   2015-10-02 13:08:00.087986236 +0200
+++ /tmp/puppet-file20151002-7331-13cyhev-0   2015-10-02 13:44:47.019583263 +0200
@@ -4,7 +4,7 @@
 # A comma delimited list of ldap servers.
 # ldaps =  xldaplb01.cern.ch,xldaplb02.cern.ch,xldaplb11.cern.ch,xldaplb12.cern.ch
 # A comma delimited list of groups and users to allow.
-# allow = 
+# allow = sbaymani
 # A comma delimited list of groups to deny.
 # deny = LxPlus-Denied-Users
 # LDAP user search base
@@ -62,10 +62,15 @@
 ldap_group_object_class = group
 
 
+# allow users
 
-# no user/group is allowed interactive access
-#ldap_access_filter = 
+# Allowing users sbaymani : (&(objectClass=user) (cn=sbaymani))
 
+# egroup filter of allow, deny e-groups, also disable disabled accounts.
+# Denying e-group LxPlus-Denied-Users : (!(memberOf:1.2.840.113556.1.4.1941:=CN=LxPlus-Denied-Users,OU=e-groups,OU=Workgroups,DC=cern,DC=ch))
+# Always deny disabled accounts : (!(userAccountControl:1.2.840.113556.1.4.803:=2))
+
+ldap_access_filter = (&(|(|(&(objectClass=user) (cn=sbaymani))))(!(memberOf:1.2.840.113556.1.4.1941:=CN=LxPlus-Denied-Users,OU=e-groups,OU=Workgroups,DC=cern,DC=ch))(!(userAccountControl:1.2.840.113556.1.4.803:=2)))
 
 
ldap_force_upper_case_realm = True

Info: Computing checksum on file /etc/sssd/sssd.conf
Info: /Stage[main]/Sssd::Config/File[/etc/sssd/sssd.conf]: Filebucketed /etc/sssd/sssd.conf to puppet with sum 5db2cca4f0b82635cc36e0e1bea829af
Notice: /Stage[main]/Sssd::Config/File[/etc/sssd/sssd.conf]/content: content changed '{md5}5db2cca4f0b82635cc36e0e1bea829af' to '{md5}8a846bf5bd41eb9e97b042fce27c238a'
Info: /Stage[main]/Sssd::Config/File[/etc/sssd/sssd.conf]: Scheduling refresh of Service[sssd]
Info: /Stage[main]/Sssd::Config/File[/etc/sssd/sssd.conf]: Scheduling refresh of Exec[authconfig]
Notice: /Stage[main]/Sssd::Config/Exec[authconfig]: Triggered 'refresh' from 1 events
Info: /Stage[main]/Sssd::Config/Exec[authconfig]: Scheduling refresh of Service[sssd]
Notice: Hello Puppet World
Notice: /Stage[main]/Hg_playground::Sbaymani/Notify[Hello Puppet World]/message: defined 'message' as 'Hello Puppet World'
Notice: /Stage[main]/Sssd::Service/Service[sssd]: Triggered 'refresh' from 2 events
Notice: Finished catalog run in 51.15 seconds
[root@sbaymani-ai-04 ~]#

Parameters for sssd:

https://git.cern.ch/web/it-puppet-module-sssd.git/blob/a45106db7bf8e88f63e264ffc819bdacbeefb665:/code/manifests/params.pp

[root@sbaymani-ai-04 ~]# puppet agent -t -v
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for sbaymani-ai-04.cern.ch
Info: Applying configuration version '1443786965'
Notice: /Stage[main]/Hg_playground::Sbaymani/Package[training-sensor]/ensure: created
Notice: Hello Puppet World
Notice: /Stage[main]/Hg_playground::Sbaymani/Notify[Hello Puppet World]/message: defined 'message' as 'Hello Puppet World'
Notice: Finished catalog run in 73.43 seconds
[root@sbaymani-ai-04 ~]# rpm -qa training-sensor
training-sensor-1-2.el6.noarch
[root@sbaymani-ai-04 ~]# 

diff --git a/code/manifests/sbaymani.pp b/code/manifests/sbaymani.pp
index 67c9b9b..3ee3118 100644
--- a/code/manifests/sbaymani.pp
+++ b/code/manifests/sbaymani.pp
@@ -1,4 +1,4 @@
-class hg_playground::sbaymani {
+class hg_playground::sbaymani ($myenv_variable = "default"){
        notify{"Hello Puppet World":}
        include afs
        include sssd
@@ -6,4 +6,11 @@ class hg_playground::sbaymani {
        package{'training-sensor':
                ensure => present
        }       
+       
+       file{'/etc/profile.d/mystuff_sbaymani.sh':
+               owner   =>      root,
+               group   =>      root,
+               mode    =>      0644,
+               content =>      template('hg_playground/mystuff_sbaymani.erb')
+       }
 }
diff --git a/code/templates/mystuff_sbaymani.erb b/code/templates/mystuff_sbaymani.erb
new file mode 100644
index 0000000..33b71f1
--- /dev/null
+++ b/code/templates/mystuff_sbaymani.erb
@@ -0,0 +1,2 @@
+export MYENV="<%= @myenv_variable -%>"
+
diff --git a/data/hostgroup/playground/sbaymani.yaml b/data/hostgroup/playground/sbaymani.yaml
index 65bf7ee..2a5fac2 100644
--- a/data/hostgroup/playground/sbaymani.yaml
+++ b/data/hostgroup/playground/sbaymani.yaml
@@ -1,3 +1,4 @@
 sssd::interactiveallowusers:
         - sbaymani
+hg_playground::sbaymani::myenv_variable: "test"


[root@sbaymani-ai-04 ~]# puppet agent -t -v
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for sbaymani-ai-04.cern.ch
Info: Applying configuration version '1443789812'
Notice: /Stage[main]/Hg_playground::Sbaymani/File[/etc/profile.d/mystuff_sbaymani.sh]/ensure: defined content as '{md5}d35a46b72388d36dc200aa4eaf3db02b'
Notice: Hello Puppet World
Notice: /Stage[main]/Hg_playground::Sbaymani/Notify[Hello Puppet World]/message: defined 'message' as 'Hello Puppet World'
Notice: Finished catalog run in 47.36 seconds
[root@sbaymani-ai-04 ~]# ls /etc/profile.d/
colorls.csh  cvs.csh  glib2.csh  lang.csh  less.csh  mystuff_sbaymani.sh  qt.sh    vim.sh     zz_sue.csh
colorls.sh   cvs.sh   glib2.sh   lang.sh   less.sh   qt.csh               vim.csh  which2.sh  zz_sue.sh
[root@sbaymani-ai-04 ~]# cat /etc/profile.d/
cat: /etc/profile.d/: Is a directory
[root@sbaymani-ai-04 ~]# cat /etc/profile.d/mystuff_sbaymani.sh 
export MYENV="test"
[root@sbaymani-ai-04 ~]# 
[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ git diff
diff --git a/code/manifests/sbaymani.pp b/code/manifests/sbaymani.pp
index 0090849..cfdf925 100644
--- a/code/manifests/sbaymani.pp
+++ b/code/manifests/sbaymani.pp
@@ -12,5 +12,17 @@ class hg_playground::sbaymani($myenv_variable = "default") {
                group   => root,
                mode    => 0644,
                content => template('hg_playground/mystuff_sbaymani.erb')
-  }
+       }
+       
+       yumrepo{"rpmforge":
+               baseurl =>      "http://linuxsoft.cern.ch/rpmforge/redhat/el6/en/x86_64/rpmforge/",
+               descr   =>      "RPM forge repository",
+               enabled =>      1,
+               gpgcheck        =>      0,
+       }
+
+       package{'vim-puppet':
+               ensure  =>      present,
+               require =>      Yumrepo['rpmforge']
+       }
 }
[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ git add code/manifests/sbaymani.pp
[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ git commit -m "Adding RPM forge repo"

Reached 3.10 of Puppet tutorial

Monitoring

Tutorial here

http://itmon-docs.web.cern.ch/itmon-docs/

[sbaymani@aiadm060 ~]$ roger update --all_alarms false sbaymani-ai-04 
[sbaymani@aiadm060 ~]$ roger show sbaymani-ai-04 
[
    {
        "app_alarmed": false,
        "appstate": "build",
        "expires": "",
        "hostname": "sbaymani-ai-04.cern.ch",
        "hw_alarmed": false,
        "message": "",
        "nc_alarmed": false,
        "os_alarmed": false,
        "update_time": "1443792230",
        "update_time_str": "Fri Oct  2 15:23:50 2015",
        "updated_by": "sbaymani",
        "updated_by_puppet": false
    }
]
[sbaymani@aiadm060 ~]$

http://metricmgr.cern.ch/

[root@sbaymani-ai-04 ~]# rpm -qa | grep training-sensor
training-sensor-1-2.el6.noarch
[root@sbaymani-ai-04 ~]#

http://lemon-monitoring.web.cern.ch/pythonapi#SensorAPI

[root@sbaymani-ai-04 ~]# cd /usr/libexec/sensors/
[root@sbaymani-ai-04 sensors]# ls
AFScm.pm                lemon-sensor-facts.pyc  lemon-sensor.py   puppetd.pm          training_sensor.pyc
edg-fmon-sensor-file    lemon-sensor-facts.pyo  lemon-sensor.pyc  sensorAPI.py        training_sensor.pyo
edg-fmon-sensor-linux   lemon-sensor-file       lemon-sensor.pyo  sensorAPI.pyc
lemon-sensor-exception  lemon-sensor-linux      lib               sensorAPI.pyo
lemon-sensor-facts.py   lemon-sensor.pl         ParseLog.pm       training_sensor.py
[root@sbaymani-ai-04 sensors]# 

registerMetric(
   "training_metricclass_sbaymani",
   "check puppet execution time",
   "training_sensor.check_puppet")

[root@sbaymani-ai-04 sensors]# python lemon-sensor.py training_sensor
INI 13326 training_metricclass_sbaymani
GET 13326

self.log(SENSOR_LOG_INFO, "Hello logging world!")

[root@sbaymani-ai-04 sensors]# python lemon-sensor.py training_sensor
INI 13326 training_metricclass_sbaymani
GET 13326
LOG 13326 INFO Hello logging world!

self.storeSample01(puppet_time)

[root@sbaymani-ai-04 sensors]# python lemon-sensor.py training_sensor
INI 13326 training_metricclass_sbaymani
GET 13326
LOG 13326 INFO Hello logging world!
PUT 01 13326 1443794466 45.13

[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ git branch
* master

Manifest defines hostgroup! "write it down in your hostgroup" == write it down in your manifest
By adding the metric to the manifest, you ask puppet to install it and be used automatically

[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ vim code/manifests/sbaymani.pp
[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ git diff --cached
diff --git a/code/manifests/sbaymani.pp b/code/manifests/sbaymani.pp
index cfdf925..e33a308 100644
--- a/code/manifests/sbaymani.pp
+++ b/code/manifests/sbaymani.pp
@@ -25,4 +25,6 @@ class hg_playground::sbaymani($myenv_variable = "default") {
                ensure  =>      present,
                require =>      Yumrepo['rpmforge']
        }
+
+       lemon::metric{'13326':}
 }
[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ 

[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ git commit -m "Adding new metric"
Checking puppet manifest syntax for code/manifests/sbaymani.pp...
[master b5be8a3] Adding new metric
 1 files changed, 2 insertions(+), 0 deletions(-)
[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ 
[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ 
[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ git pull --rebase origin master && git push origin master
remote: Counting objects: 121, done.
remote: Compressing objects: 100% (97/97), done.
remote: Total 97 (delta 51), reused 0 (delta 0)
Unpacking objects: 100% (97/97), done.
From https://git.cern.ch/kerberos/it-puppet-hostgroup-playground
 * branch            master     -> FETCH_HEAD
First, rewinding head to replay your work on top of it...
Applying: Adding new metric
Counting objects: 9, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (5/5), 456 bytes, done.
Total 5 (delta 3), reused 0 (delta 0)
To https://:@git.cern.ch/kerberos/it-puppet-hostgroup-playground
   d57eadd..372890d  master -> master
[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ 

[root@sbaymani-ai-04 sensors]# puppet agent -t -v
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for sbaymani-ai-04.cern.ch
Info: Applying configuration version '1443795306'
Notice: /Stage[main]/Hg_playground::Sbaymani/Lemon::Metric[13326]/Lemon::Sensor[training_sensor_sbaymani]/File[/etc/lemon/agent/sensors/training_sensor_sbaymani.conf]/ensure: defined content as '{md5}d726b53e6bb81ba6950042f964de6307'
Info: /Stage[main]/Hg_playground::Sbaymani/Lemon::Metric[13326]/Lemon::Sensor[training_sensor_sbaymani]/File[/etc/lemon/agent/sensors/training_sensor_sbaymani.conf]: Scheduling refresh of Class[Lemon::Service]
Notice: /Stage[main]/Hg_playground::Sbaymani/Lemon::Metric[13326]/File[/etc/lemon/agent/metrics/13326-training_metric_sbaymani.conf]/ensure: defined content as '{md5}6c5d8738ee946ca8663780ec9ea2e35c'
Info: /Stage[main]/Hg_playground::Sbaymani/Lemon::Metric[13326]/File[/etc/lemon/agent/metrics/13326-training_metric_sbaymani.conf]: Scheduling refresh of Class[Lemon::Service]
Notice: Hello Puppet World
Notice: /Stage[main]/Hg_playground::Sbaymani/Notify[Hello Puppet World]/message: defined 'message' as 'Hello Puppet World'
Notice: /Stage[main]/Hg_playground::Sbaymani/Lemon::Metric[13326]/Lemonfwd::Metrictags[13326]/Concat::Fragment[lemonfwd_tags_fragment_13326]/File[/var/lib/puppet/concat/_etc_lemon_forwarder_metric_metadata.yml/fragments/100_lemonfwd_tags_fragment_13326]/ensure: defined content as '{md5}e772129d8b9e6863e46a75712c7879bb'
Info: /Stage[main]/Hg_playground::Sbaymani/Lemon::Metric[13326]/Lemonfwd::Metrictags[13326]/Concat::Fragment[lemonfwd_tags_fragment_13326]/File[/var/lib/puppet/concat/_etc_lemon_forwarder_metric_metadata.yml/fragments/100_lemonfwd_tags_fragment_13326]: Scheduling refresh of Exec[concat_/etc/lemon/forwarder/metric_metadata.yml]
Info: Class[Lemon::Service]: Scheduling refresh of Service[lemon-agent]
Notice: /Stage[main]/Lemon::Service/Service[lemon-agent]: Triggered 'refresh' from 1 events
Notice: /Stage[main]/Lemonfwd::Config/Concat[/etc/lemon/forwarder/metric_metadata.yml]/Exec[concat_/etc/lemon/forwarder/metric_metadata.yml]/returns: executed successfully
Notice: /Stage[main]/Lemonfwd::Config/Concat[/etc/lemon/forwarder/metric_metadata.yml]/Exec[concat_/etc/lemon/forwarder/metric_metadata.yml]: Triggered 'refresh' from 1 events
Notice: /Stage[main]/Lemonfwd::Config/Concat[/etc/lemon/forwarder/metric_metadata.yml]/File[/etc/lemon/forwarder/metric_metadata.yml]/content: 
--- /etc/lemon/forwarder/metric_metadata.yml   2015-10-02 13:28:41.774018122 +0200
+++ /tmp/puppet-file20151002-16300-mzfe4q-0   2015-10-02 16:16:46.928148089 +0200
@@ -524,6 +524,18 @@
           type: CHAR
 
 
+# training_metric_sbaymani
+13326:
+    name: training_metric_sbaymani
+    class: training_metricclass_sbaymani
+    fields:
+        - index: 1
+          is_key: false
+          name: puppet_exec_time
+          scale: 1.0
+          type: FLOAT
+
+
 # LoadAvg
 20002:
     name: LoadAvg

Info: Computing checksum on file /etc/lemon/forwarder/metric_metadata.yml
Info: /Stage[main]/Lemonfwd::Config/Concat[/etc/lemon/forwarder/metric_metadata.yml]/File[/etc/lemon/forwarder/metric_metadata.yml]: Filebucketed /etc/lemon/forwarder/metric_metadata.yml to puppet with sum ef612794e8d001c347de70db54ff6d57
Notice: /Stage[main]/Lemonfwd::Config/Concat[/etc/lemon/forwarder/metric_metadata.yml]/File[/etc/lemon/forwarder/metric_metadata.yml]/content: content changed '{md5}ef612794e8d001c347de70db54ff6d57' to '{md5}3d02211bec15dac78136595ff95c5793'
Info: Concat[/etc/lemon/forwarder/metric_metadata.yml]: Scheduling refresh of Class[Lemonfwd::Service]
Info: Class[Lemonfwd::Service]: Scheduling refresh of Service[lemon-forwarder]
Notice: /Stage[main]/Lemonfwd::Service/Service[lemon-forwarder]: Triggered 'refresh' from 1 events
Notice: Finished catalog run in 49.69 seconds
[root@sbaymani-ai-04 sensors]# 

[root@sbaymani-ai-04 sensors]# cat /etc/lemon/forwarder/metric_metadata.yml | grep -A14 13326
13326:
    name: training_metric_sbaymani
    class: training_metricclass_sbaymani
    fields:
        - index: 1
          is_key: false
          name: puppet_exec_time
          scale: 1.0
          type: FLOAT


# LoadAvg
20002:
    name: LoadAvg
    class: system.loadAvg
[root@sbaymani-ai-04 sensors]# 

[root@sbaymani-ai-04 sensors]# lemon-cli -m 13326 --start 15m
[INFO] lemon-cli version 1.2.0 started by root at Fri Oct  2 16:17:52 2015 on sbaymani-ai-04.cern.ch
[VERB] Nodes:      sbaymani-ai-04 
[VERB] Metrics:      13326 
[VERB] Start:      Fri Oct  2 16:02:52 2015 - 1443794572
[VERB] End:      Fri Oct  2 16:17:52 2015 - 1443795472 (difference: 900 seconds)
[VERB] Method:      Historical
[VERB] 
[INFO] local:   sbaymani-ai-04   13326   Fri Oct  2 16:16:59 2015   49.69
[VERB] 
[VERB] Total: 1 result
[root@sbaymani-ai-04 sensors]# 


[root@sbaymani-ai-04 sensors]# meter-cli -m 13326 --start 15m
Took: 80ms
Showing 1 out of 1 values
Found results for the following metrics: 13326

== Results for metric 13326 ==

+----------------+--------------------------+------------------+
|         Entity |                Timestamp | puppet_exec_time |
+----------------+--------------------------+------------------+
| sbaymani-ai-04 | 2015-10-02 16:16:59 CEST |            49.69 |
+----------------+--------------------------+------------------+

[root@sbaymani-ai-04 sensors]# 

[root@sbaymani-ai-04 sensors]# meter-cli -m 13326 --start 30m
Took: 24ms
Showing 2 out of 2 values
Found results for the following metrics: 13326

== Results for metric 13326 ==

+----------------+--------------------------+------------------+
|         Entity |                Timestamp | puppet_exec_time |
+----------------+--------------------------+------------------+
| sbaymani-ai-04 | 2015-10-02 16:18:57 CEST |            49.69 |
| sbaymani-ai-04 | 2015-10-02 16:16:59 CEST |            49.69 |
+----------------+--------------------------+------------------+

[root@sbaymani-ai-04 sensors]#

So the metric will be updated every 120 s.

https://meter.cern.ch/public/_plugin/kibana/#/dashboard/temp/itmon::demo_training

@fields.metric_id:"13326" AND @fields.entity:"sbaymani-ai-04"

Notification
_Correlation: <_metric_id>:1 > 1
Format: Metric_id:field_id
First '1' is for field number, we have only one field, which is the first field. Second one is for the comparison

[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ vim code/manifests/sbaymani.pp
[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ git diff
diff --git a/code/manifests/sbaymani.pp b/code/manifests/sbaymani.pp
index e33a308..e26bae4 100644
--- a/code/manifests/sbaymani.pp
+++ b/code/manifests/sbaymani.pp
@@ -27,4 +27,5 @@ class hg_playground::sbaymani($myenv_variable = "default") {
        }
 
        lemon::metric{'13326':}
+       lemon::metric{'33689':}
 }
[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ 

[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ git add code/manifests/sbaymani.pp
[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ git commit -m "Adding new notification"
Checking puppet manifest syntax for code/manifests/sbaymani.pp...
[master b18d7f1] Adding new notification
 1 files changed, 1 insertions(+), 0 deletions(-)
[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ git pull --rebase origin master && git push origin master
remote: Counting objects: 79, done.
remote: Compressing objects: 100% (66/66), done.
remote: Total 66 (delta 38), reused 0 (delta 0)
Unpacking objects: 100% (66/66), done.
From https://git.cern.ch/kerberos/it-puppet-hostgroup-playground
 * branch            master     -> FETCH_HEAD
First, rewinding head to replay your work on top of it...
Applying: Adding new notification
Counting objects: 9, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (5/5), 462 bytes, done.
Total 5 (delta 3), reused 0 (delta 0)
To https://:@git.cern.ch/kerberos/it-puppet-hostgroup-playground
   c561d68..cf7532a  master -> master
[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ 


[root@sbaymani-ai-04 sensors]# puppet agent -tv
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for sbaymani-ai-04.cern.ch
Info: Applying configuration version '1443797075'
Notice: /Stage[main]/Hg_playground::Sbaymani/Lemon::Metric[33689]/File[/etc/lemon/agent/metrics/33689-exception.training_sbaymani.conf]/ensure: defined content as '{md5}081dab7810065b0fd7fdfeefeac67657'
Info: /Stage[main]/Hg_playground::Sbaymani/Lemon::Metric[33689]/File[/etc/lemon/agent/metrics/33689-exception.training_sbaymani.conf]: Scheduling refresh of Class[Lemon::Service]
Notice: Hello Puppet World
Notice: /Stage[main]/Hg_playground::Sbaymani/Notify[Hello Puppet World]/message: defined 'message' as 'Hello Puppet World'
Notice: /Stage[main]/Hg_playground::Sbaymani/Lemon::Metric[33689]/Lemonfwd::Metrictags[33689]/Concat::Fragment[lemonfwd_tags_fragment_33689]/File[/var/lib/puppet/concat/_etc_lemon_forwarder_metric_metadata.yml/fragments/100_lemonfwd_tags_fragment_33689]/ensure: defined content as '{md5}30c29c12e9f3ba45e3d146cc3479573e'
Info: /Stage[main]/Hg_playground::Sbaymani/Lemon::Metric[33689]/Lemonfwd::Metrictags[33689]/Concat::Fragment[lemonfwd_tags_fragment_33689]/File[/var/lib/puppet/concat/_etc_lemon_forwarder_metric_metadata.yml/fragments/100_lemonfwd_tags_fragment_33689]: Scheduling refresh of Exec[concat_/etc/lemon/forwarder/metric_metadata.yml]
Info: Class[Lemon::Service]: Scheduling refresh of Service[lemon-agent]
Notice: /Stage[main]/Lemon::Service/Service[lemon-agent]: Triggered 'refresh' from 1 events
Notice: /Stage[main]/Lemonfwd::Config/Concat[/etc/lemon/forwarder/metric_metadata.yml]/Exec[concat_/etc/lemon/forwarder/metric_metadata.yml]/returns: executed successfully
Notice: /Stage[main]/Lemonfwd::Config/Concat[/etc/lemon/forwarder/metric_metadata.yml]/Exec[concat_/etc/lemon/forwarder/metric_metadata.yml]: Triggered 'refresh' from 1 events
Notice: /Stage[main]/Lemonfwd::Config/Concat[/etc/lemon/forwarder/metric_metadata.yml]/File[/etc/lemon/forwarder/metric_metadata.yml]/content: 
--- /etc/lemon/forwarder/metric_metadata.yml   2015-10-02 16:16:47.686054509 +0200
+++ /tmp/puppet-file20151002-19130-1ldupnz-0   2015-10-02 16:46:12.799212336 +0200
@@ -1627,6 +1627,37 @@
         - m_snow: 0
 
 
+# exception.training_sbaymani
+33689:
+    name: exception.training_sbaymani
+    class: alarm.exception
+    fields:
+        - index: 1
+          is_key: false
+          name: exceptstate
+          scale: 1.0
+          type: INTEGER
+        - index: 2
+          is_key: false
+          name: code
+          scale: 1.0
+          type: CHAR
+        - index: 3
+          is_key: false
+          name: freetxt
+          scale: 1.0
+          type: CHAR
+    tags:
+        all:
+        - notification_type: app
+        - producer_source: host
+        - troubleshooting: http://this.is.my.documentation.site.com
+        open:
+        - fe_name: ""
+        - m_email: 0
+        - m_snow: 0
+
+
 # sshd
 4006:
     name: sshd

Info: Computing checksum on file /etc/lemon/forwarder/metric_metadata.yml
Info: /Stage[main]/Lemonfwd::Config/Concat[/etc/lemon/forwarder/metric_metadata.yml]/File[/etc/lemon/forwarder/metric_metadata.yml]: Filebucketed /etc/lemon/forwarder/metric_metadata.yml to puppet with sum 3d02211bec15dac78136595ff95c5793
Notice: /Stage[main]/Lemonfwd::Config/Concat[/etc/lemon/forwarder/metric_metadata.yml]/File[/etc/lemon/forwarder/metric_metadata.yml]/content: content changed '{md5}3d02211bec15dac78136595ff95c5793' to '{md5}c540ddf2e97c986eb1df17300da38550'
Info: Concat[/etc/lemon/forwarder/metric_metadata.yml]: Scheduling refresh of Class[Lemonfwd::Service]
Info: Class[Lemonfwd::Service]: Scheduling refresh of Service[lemon-forwarder]
Notice: /Stage[main]/Lemonfwd::Service/Service[lemon-forwarder]: Triggered 'refresh' from 1 events
Notice: Finished catalog run in 50.54 seconds
[root@sbaymani-ai-04 sensors]# 

[root@sbaymani-ai-04 sensors]# cat /etc/lemon/forwarder/metric_metadata.yml | grep -A14 33689
33689:
    name: exception.training_sbaymani
    class: alarm.exception
    fields:
        - index: 1
          is_key: false
          name: exceptstate
          scale: 1.0
          type: INTEGER
        - index: 2
          is_key: false
          name: code
          scale: 1.0
          type: CHAR
        - index: 3
[root@sbaymani-ai-04 sensors]# 

[root@sbaymani-ai-04 sensors]# lemon-cli -m 33689 --start 15m
[INFO] lemon-cli version 1.2.0 started by root at Fri Oct  2 16:51:18 2015 on sbaymani-ai-04.cern.ch
[VERB] Nodes:      sbaymani-ai-04 
[VERB] Metrics:      33689 
[VERB] Start:      Fri Oct  2 16:36:18 2015 - 1443796578
[VERB] End:      Fri Oct  2 16:51:18 2015 - 1443797478 (difference: 900 seconds)
[VERB] Method:      Historical
[VERB] 
[INFO] local:   sbaymani-ai-04   33689   Fri Oct  2 16:46:53 2015   1 000 sbaymani-ai-04:13326:1[50.54]_>_1
[INFO] local:   sbaymani-ai-04   33689   Fri Oct  2 16:48:53 2015   1 005 sbaymani-ai-04:13326:1[50.54]_>_1
[INFO] local:   sbaymani-ai-04   33689   Fri Oct  2 16:50:53 2015   1 005 sbaymani-ai-04:13326:1[50.18]_>_1
[VERB] 
[VERB] Total: 3 results
[root@sbaymani-ai-04 sensors]# lemon-host-check --show-all
[INFO] lemon-host-check version 1.3.8 started by root at Fri Oct  2 16:51:39 2015 on sbaymani-ai-04.cern.ch
[VERB] 33689
[VERB]       Name:      exception.training_sbaymani
[VERB]       Reason:      sbaymani-ai-04:13326:1[50.18] > 1
[VERB] -------------------------------------------------------------------------------------------------------
[VERB] Active Exceptions: 1 --- Running Actuators: 0 ---
[VERB] Defined Exceptions: 36 --- Disabled Exceptions: 0 --- Silent Exceptions: 2 --- Local Exceptions: 0
[VERB] -------------------------------------------------------------------------------------------------------
[root@sbaymani-ai-04 sensors]#

https://gni.cern.ch/

body.metadata.metric_name:"exception.training_sbaymani"
body.metadata.metric_id:"33689"

Overwriting notifications
Useful if you are using other people's notifications but you want to change them, e.g. if the original notification happens too often.

[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ vim data/hostgroup/playground/sbaymani.yaml
[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ git diff
diff --git a/data/hostgroup/playground/sbaymani.yaml b/data/hostgroup/playground/sbaymani.yaml
index 1e830ac..a783414 100644
--- a/data/hostgroup/playground/sbaymani.yaml
+++ b/data/hostgroup/playground/sbaymani.yaml
@@ -1,3 +1,6 @@
 sssd::interactiveallowusers:
         - sbaymani
 hg_playground::sbaymani::myenv_variable: "test"
+lemon_33689:
+        params:
+                Correlation: "13326:1 > 60"
[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ git commit -a -m "Overwriting existing notification"
Checking yaml syntax for data/hostgroup/playground/sbaymani.yaml...
[master 299e8ae] Overwriting existing notification
 1 files changed, 3 insertions(+), 0 deletions(-)
[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ git pull --rebase origin master && git push origin master
remote: Counting objects: 81, done.
remote: Compressing objects: 100% (59/59), done.
remote: Total 59 (delta 28), reused 0 (delta 0)
Unpacking objects: 100% (59/59), done.
From https://git.cern.ch/kerberos/it-puppet-hostgroup-playground
 * branch            master     -> FETCH_HEAD
First, rewinding head to replay your work on top of it...
Applying: Overwriting existing notification
Counting objects: 11, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (6/6), done.
Writing objects: 100% (6/6), 609 bytes, done.
Total 6 (delta 2), reused 0 (delta 0)
To https://:@git.cern.ch/kerberos/it-puppet-hostgroup-playground
   14516b9..1553449  master -> master
[sbaymani@aiadm060 it-puppet-hostgroup-playground]$ 

[root@sbaymani-ai-04 sensors]# puppet agent -tv
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for sbaymani-ai-04.cern.ch
Info: Applying configuration version '1443798387'
Notice: /Stage[main]/Hg_playground::Sbaymani/Lemon::Metric[33689]/File[/etc/lemon/agent/metrics/33689-exception.training_sbaymani.conf]/content: 
--- /etc/lemon/agent/metrics/33689-exception.training_sbaymani.conf   2015-10-02 16:45:56.483156125 +0200
+++ /tmp/puppet-file20151002-22135-11rlkfg-0   2015-10-02 17:08:06.065289759 +0200
@@ -4,7 +4,7 @@
    MetricClass   alarm.exception
    Timing      120   30
    Parameters
-      Correlation   13326:1 >1
+      Correlation   13326:1 > 60
    Smoothing
       CacheAll   1
       Index   0

Info: Computing checksum on file /etc/lemon/agent/metrics/33689-exception.training_sbaymani.conf
Info: /Stage[main]/Hg_playground::Sbaymani/Lemon::Metric[33689]/File[/etc/lemon/agent/metrics/33689-exception.training_sbaymani.conf]: Filebucketed /etc/lemon/agent/metrics/33689-exception.training_sbaymani.conf to puppet with sum 081dab7810065b0fd7fdfeefeac67657
Notice: /Stage[main]/Hg_playground::Sbaymani/Lemon::Metric[33689]/File[/etc/lemon/agent/metrics/33689-exception.training_sbaymani.conf]/content: content changed '{md5}081dab7810065b0fd7fdfeefeac67657' to '{md5}934fa36d0e65ed866d613d625d154b5a'
Info: /Stage[main]/Hg_playground::Sbaymani/Lemon::Metric[33689]/File[/etc/lemon/agent/metrics/33689-exception.training_sbaymani.conf]: Scheduling refresh of Class[Lemon::Service]
Notice: Hello Puppet World
Notice: /Stage[main]/Hg_playground::Sbaymani/Notify[Hello Puppet World]/message: defined 'message' as 'Hello Puppet World'
Info: Class[Lemon::Service]: Scheduling refresh of Service[lemon-agent]
Notice: /Stage[main]/Lemon::Service/Service[lemon-agent]: Triggered 'refresh' from 1 events
Notice: Finished catalog run in 49.99 seconds
[root@sbaymani-ai-04 sensors]# 

[root@sbaymani-ai-04 sensors]# cat /etc/lemon/agent/metrics/33689-exception.training_sbaymani.conf 
# Managed by puppet with metric.conf.erb file.
33689
   MetricName   exception.training_sbaymani
   MetricClass   alarm.exception
   Timing      120   30
   Parameters
      Correlation   13326:1 > 60
   Smoothing
      CacheAll   1
      Index   0
      Maxtime   36000
      OnValue   0 000 (null)
      Type   string

# End of template for metric 33689

[root@sbaymani-ai-04 sensors]# 


[root@sbaymani-ai-04 sensors]# lemon-cli -m 33689 --start 15m
[INFO] lemon-cli version 1.2.0 started by root at Fri Oct  2 17:09:28 2015 on sbaymani-ai-04.cern.ch
[VERB] Nodes:      sbaymani-ai-04 
[VERB] Metrics:      33689 
[VERB] Start:      Fri Oct  2 16:54:28 2015 - 1443797668
[VERB] End:      Fri Oct  2 17:09:28 2015 - 1443798568 (difference: 900 seconds)
[VERB] Method:      Historical
[VERB] 
[INFO] local:   sbaymani-ai-04   33689   Fri Oct  2 16:54:53 2015   1 005 sbaymani-ai-04:13326:1[50.18]_>_1
[INFO] local:   sbaymani-ai-04   33689   Fri Oct  2 16:56:53 2015   1 005 sbaymani-ai-04:13326:1[50.18]_>_1
[INFO] local:   sbaymani-ai-04   33689   Fri Oct  2 16:58:53 2015   1 005 sbaymani-ai-04:13326:1[50.18]_>_1
[INFO] local:   sbaymani-ai-04   33689   Fri Oct  2 17:00:53 2015   1 005 sbaymani-ai-04:13326:1[50.18]_>_1
[INFO] local:   sbaymani-ai-04   33689   Fri Oct  2 17:02:53 2015   1 005 sbaymani-ai-04:13326:1[50.18]_>_1
[INFO] local:   sbaymani-ai-04   33689   Fri Oct  2 17:04:53 2015   1 005 sbaymani-ai-04:13326:1[50.18]_>_1
[INFO] local:   sbaymani-ai-04   33689   Fri Oct  2 17:06:53 2015   1 005 sbaymani-ai-04:13326:1[50.18]_>_1
[VERB] 
[VERB] Total: 7 results
[root@sbaymani-ai-04 sensors]# lemon-host-check --show-all
[INFO] lemon-host-check version 1.3.8 started by root at Fri Oct  2 17:09:45 2015 on sbaymani-ai-04.cern.ch
[VERB] 33689
[VERB]       Name:      exception.training_sbaymani
[VERB]       Reason:      (null)
[VERB]       Notes:      possible false exception (cacheAll: enabled)
[VERB] -------------------------------------------------------------------------------------------------------
[VERB] Active Exceptions: 0 --- Running Actuators: 0 ---
[VERB] Defined Exceptions: 36 --- Disabled Exceptions: 0 --- Silent Exceptions: 2 --- Local Exceptions: 0
[VERB] -------------------------------------------------------------------------------------------------------
[root@sbaymani-ai-04 sensors]#

-- SimaolhodaBaymani - 2015-10-02

Topic revision: r3 - 2016-08-08 - unknown

Main

Webs

Public webs

Welcome Guest

- Cern Search
- TWiki Search
- Google Search
Main All webs

Copyright &© 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback