October 3, 2017 - Reading time: ~1 minute

Today I had a machine that had no swap space left which in return triggered a monitoring alert. Since the pressure was already gone but the swap wasn't cleared fast enough I searched and found this script as a nice solution. The script was a bit older and was made for a version of free that seperated buffers and caches. In newer versions they are combined as "buff/cache" and a new field is introduced that shows the combined available memory. Which gave me a chance to simplify the script.

clearswap.sh

#!/bin/bash

free_mem="$(free | grep 'Mem:' | awk '{print $7}')"
used_swap="$(free | grep 'Swap:' | awk '{print $3}')"

echo -e "Free memory:\t$free_mem kB ($((free_mem / 1024)) MiB)\nUsed swap:\t$used_swap kB ($((used_swap / 1024)) MiB)"
if [[ $used_swap -eq 0 ]]; then
    echo "Congratulations! No swap is in use."
elif [[ $used_swap -lt $free_mem ]]; then
    echo "Freeing swap..."
    sudo swapoff -a
    sudo swapon -a
else
    echo "Not enough free memory. Exiting."
    exit 1
fi

 Thx Scott Severance for the initial script

September 12, 2017 - Reading time: ~1 minute

I had to test if all hosts were able to connect to a certain port on a certain server. Ansible was the perfect tool, but since not all machines have nc or nmap installed I had to make a workaround using Python. I will be checking if TCP port nagios.company.com:5666 is open.

the script, nagios.py (will be copied and executed on the remote host by ansible). This only works for TCP, changing SOCK_STREAM to SOCK_DGRAM will always return 0.

#!/usr/bin/python
import socket;
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
result = sock.connect_ex(('nagios.company.com',5666))
if result == 0:
   print "OK"
else:
   print "Not OK"

the playbook, nagios.yml

- name: Nagios connectivity test
  hosts: all
  tasks:
    - name: script
      script: /tmp/nagios.py
      register: nagios
    - debug: msg="{{ nagios.stdout }}"

the run command to filter the hosts that can't connect

ansible-playbook /tmp/nagios.yml | grep -B1 Not\ OK

August 23, 2017 - Reading time: 4 minutes

After an upgrade from RedHat the template for the password is changed and uses a variable that OpenScap doesn't read. This makes that our test fails. On top of that the test checks for the use of common administrator account names like root, admin or administrator. This update solves the issue and from now we use a user instead of root for Grub2.

Today I had to update and verify that we have an entry password for Grub on all machines. We needed to do this to comply with the Certified Cloud Service Provider's OpenScap benchmark.

This only prevents a person with physical access to boot in single user mode! The machine can still be booted without the need of a password.

RHEL6 machines all use legacy boot where on RHEL7 we also make a difference between EFI and non-EFI machines. 

First generate the hashes (on a RHEL6 and on a RHEL7 node)

RHEL6

grub-crypt --sha-512

RHEL7

grub2-mkpasswd-pbkdf2

And to finish... Here are the Ansible lines:

playbook lines:

#GRUB
- name: "grub v1 | add password"
  lineinfile: dest=/etc/grub.conf regexp='^password ' state=present line='password --encrypted {{ grub_password_v1_passwd }}' insertafter='^timeout'
  when: rhel6
  tags: grub-password

- stat: path=/sys/firmware/efi/efivars/
  register: grub_efi
  when: rhel7
  tags: grub-password

- name: remove unwanted grub.cfg on EFI systems
  file:
    state: absent
    path: /boot/grub2/grub.cfg
  when: rhel7 and grub_efi.stat.exists == True
  tags: grub-password

- name: Install user template to make sure grub2-mkconfig doesn't mess up the config
  template:
    src: 01_users.j2
    dest: /etc/grub.d/01_users
    owner: root
    group: root
    mode: '0700'
  notify:
     - grub2-mkconfig EFI
     - grub2-mkconfig MBR
  when: rhel7
  tags: grub-password

- name: "grub v2 EFI | add password"
  lineinfile: dest=/etc/grub2-efi.cfg regexp="^password_pbkdf2 {{ grub_user }} " state=present insertafter=EOF line='password_pbkdf2 {{ grub_user }} {{ grub_password_v2_passwd }}'
  when: rhel7 and grub_efi.stat.exists == True
  tags: grub-password

- name: "grub v2 MBR | add password"
  lineinfile: dest=/etc/grub2.cfg regexp="^password_pbkdf2 {{ grub_user }} " state=present insertafter=EOF line='password_pbkdf2 {{ grub_user }} {{ grub_password_v2_passwd }}'
  when: rhel7 and grub_efi.stat.exists == False

vars:

grub_password_v1_passwd: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
grub_password_v2_passwd: grub.pbkdf2.sha512.10000.xxxxxxxxxxxxxxxxxxx
grub_user: loginuser

 Handlers:

- name: grub2-mkconfig EFI
  command: grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg
  when: grub_efi.stat.exists == True

- name: grub2-mkconfig MBR
  command: grub2-mkconfig -o /boot/grub2/grub.cfg
  when: grub_efi.stat.exists == False

01_users.j2:

#!/bin/sh -e

cat << "EOF"
set superusers="{{ grub_user }}"
export superusers
password_pbkdf2 {{ grub_user }} {{ grub_password_v2_passwd }}
EOF

August 22, 2017 - Reading time: ~1 minute

"This page is taking longer to load than usual. Give it a few moments, then try refreshing. Still having issues? Contact your Confluence admin" 

This is what I got after upgrading Confluence to a version above 6 (6.3.3 in my case), this is an issue caused by a too old version of httpd/apache as a reverse proxy and since we don't need the collaborative editing function we needed to find out how to disable it to get our system going again.

I know there is a python script for the rest API and you can use to web GUI and so on, but if you tried those you prabably already found out you can't kill the damn Synchrony process or get it disabled no matter how hard you try...

The easy solution is to go to Manage add-ons -> Select the view for the system add-ons -> And disable these 2:

Synchrony Interop Bootstrap Plugin

Confluence Collaborative Editor Plugin

And restart confluence to make sure it is permanent.

August 21, 2017 - Reading time: 4 minutes

I was trying to flash a new firmware to my BT headset (Sena SMH5-FM) and found out that the device reconnects several times during the flash process. Since KVM forces the use of BUS and ID rather than VendorID:DeviceID and the ID increments by one every reconnect this never works more than one time. To work around this I made a solution using UDEV to force a connect/reconnect of the device in KVM everytime it is being connected/disconnected to/from the host.

First find out the name of your KVM machine, you can try this command to get only the name portion

ps -ef | grep kvm |sed -n -e 's/^.*\(-name \)/\1/p'| cut -f2 -d" "

Mine is called win2k12r2, albeit a Win10

Stop the VM and create the following file (if you already tried to flash and you have a device like mine that also changes the DeviceID a couple of times make more entries) -> replace Sena with a portion of the name or brand of your device (see output lsusb)

grep -B3 Sena /var/log/syslog | sed -n -e 's/^.*\(idProduct\)/\1/p' | sort | uniq

I got this as a result (I suppose they just change it twice and that the logic is xxxx becomes yxxx and one time they use a generic ffff)

idProduct=4d65
idProduct=cd65
idProduct=ffff

The VendorID is always the same and for Sena it is 092b

Next I created a USB device xml for everyone of them under

/etc/libvirt/qemu/hostdev-092b:ffff.xml

<hostdev mode='subsystem' type='usb'>
  <source>
    <vendor id='0x092b'/>
    <product id='0xffff'/>
  </source>
</hostdev>

and a UDEV rule to connect and disconnect them

/etc/udev/rules.d/10-local.rules

ACTION=="add", ATTRS{idVendor}=="092b", ATTRS{idProduct}=="ffff", RUN+="/usr/bin/virsh attach-device win2k12r2 /etc/libvirt/qemu/hostdev-092b:ffff.xml"
ACTION=="remove", ATTRS{idVendor}=="092b", ATTRS{idProduct}=="ffff", RUN+="/usr/bin/virsh detach-device win2k12r2 /etc/libvirt/qemu/hostdev-092b:ffff.xml"
ACTION=="add", ATTRS{idVendor}=="092b", ATTRS{idProduct}=="cd65", RUN+="/usr/bin/virsh attach-device win2k12r2 /etc/libvirt/qemu/hostdev-092b:cd65.xml"
ACTION=="remove", ATTRS{idVendor}=="092b", ATTRS{idProduct}=="cd65", RUN+="/usr/bin/virsh detach-device win2k12r2 /etc/libvirt/qemu/hostdev-092b:cd65.xml"
ACTION=="add", ATTRS{idVendor}=="092b", ATTRS{idProduct}=="4d65", RUN+="/usr/bin/virsh attach-device win2k12r2 /etc/libvirt/qemu/hostdev-092b:4d65.xml"
ACTION=="remove", ATTRS{idVendor}=="092b", ATTRS{idProduct}=="4d65", RUN+="/usr/bin/virsh detach-device win2k12r2 /etc/libvirt/qemu/hostdev-092b:4d65.xml"

After this you can start your VM and try to flash your device, if it still fails check again to see if there are no extra DeviceID's in the syslog

June 27, 2017 - Reading time: 3 minutes

We started to implement security headers in all our public web services. This article has some examples of how I implemented them in F5 and/or Apache.

For HTTP Strict Transport Security (HSTS) I used an iRule. (current max-age is 181 days)

when RULE_INIT {
set static::max_age 15638400
}
when HTTP_RESPONSE {
  #HSTS
  HTTP::header insert Strict-Transport-Security "max-age=$static::max_age; includeSubDomains"
}

If you don't have an F5 you can implement the HSTS header in apache as well like this:

Header append Strict-Transport-Security "max-age=63072000; includeSubdomains;"

The other Security Headers are in Apache if possible.

RequestHeader set X-Forwarded-Proto "https"

Header append X-Frame-Options SAMEORIGIN
Header append X-XSS-Protection "1; mode=block"
Header append X-Content-Type-Options nosniff
Header append Content-Security-Policy "frame-ancestors 'self' https://*.cloudfront.net https://*.googleapis.com https://fonts.gstatic.com https://*.youtube.com https://www.googleadservices.com https://googleads.g.doubleclick.net https://s.ytimg.com https://www.google.com https://www.google.be https://usage.trackjs.com https://i.ytimg.com"
Header append x-webkit-csp "frame-ancestors 'self' https://*.cloudfront.net https://*.googleapis.com https://fonts.gstatic.com https://*.youtube.com https://www.googleadservices.com https://googleads.g.doubleclick.net https://s.ytimg.com https://www.google.com https://www.google.be https://usage.trackjs.com https://i.ytimg.com"
Header append X-Content-Security-Policy "frame-ancestors 'self' https://*.cloudfront.net https://*.googleapis.com https://fonts.gstatic.com https://*.youtube.com https://www.googleadservices.com https://googleads.g.doubleclick.net https://s.ytimg.com https://www.google.com https://www.google.be https://usage.trackjs.com https://i.ytimg.com"
Header append Referrer-Policy "strict-origin-when-cross-origin"

Or for Java based applications we also use an iRule:

when HTTP_REQUEST { 
HTTP::header insert X-Forwarded-Proto "https"
}
when HTTP_RESPONSE { 
#X-XSS-Protection 
HTTP::header insert X-XSS-Protection "1; mode=block" 
#X-Frame-Options 
HTTP::header insert X-Frame-Options "SAMEORIGIN" 
#X-Content-Type-Options 
HTTP::header insert X-Content-Type-Options "nosniff" 
#CSP 
HTTP::header insert Content-Security-Policy "frame-ancestors 'self'" 
#CSP for IE 
HTTP::header insert X-Content-Security-Policy "frame-ancestors 'self'" 
#CSP 
HTTP::header insert x-webkit-csp "frame-ancestors 'self'" 
#referrere policy 
HTTP::header insert Referrer-Policy "strict-origin-when-cross-origin"
}

Some info about Security Headers:

And to verify your Security Headers you can use this online tool

And the blog of the guy who made the test. He seems well informed and his blog has a lot of useful information and examples

And of course to verify that this site got an A rating!