Using bare variables in Ansible 2.8

Ansible 2.8 was released in May 2019 and later in May came to Fedora 30 in package form. So the first tests could be done to see what needed to be done to switch from 2.7 to 2.8 and don’t generate a lot of stopped GitLab CI-jobs due to new warnings and errors. So let start with one warning that needs to be resolved before the 2.12 release and also is given on many third-party roles.

- name: Enable EPEL repository
  package:
    name: epel-release
    state: present
  when: platform_repo_epel

The example code above is simple enough to get the warning about CONDITIONAL_BARE_VARS. We could opt for disabling the warning in ansible.cfg and move forward, but as this is technical debt we don’t want to get more and resolve the current debt as quickly as possible.

TASK [role.platform : Enable EPEL repository] *******************************
[DEPRECATION WARNING]: evaluating platform_repo_epel as a bare variable, this 
behaviour will go away and you might need to add |bool to the expression in the
 future. Also see CONDITIONAL_BARE_VARS configuration toggle.. This feature 
will be removed in version 2.12. Deprecation warnings can be disabled by 
setting deprecation_warnings=False in ansible.cfg.

First we try to resolve this technical debt in the traditional way and making it a Boolean comparison and this stops Ansible from complaining as it is not a bare variable anymore.

- name: Enable EPEL repository
  package:
    name: epel-release
    state: present
  when: platform_repo_epel == True

Now Ansible lint starts to give a notification, added in version 4.0.0, as you shouldn’t do a Boolean comparison this was. And while it is technical correct, we also want this linting notification gone to pass the CI-pipeline.

[601] Don't compare to literal True/False
/path/to/ansible/project/roles/platform/tasks/main:6
  when: platform_repo_epel == True

In original message from Ansible there was already a hit on how to resolve this and by adding a Boolean filter both Ansible keeps on running correctly and Ansible lint is also happy.

- name: Enable EPEL repository
  package:
    name: epel-release
    state: present
  when: platform_repo_epel|bool

While these kind of modifications seem non-trivial and a test in your CI-pipeline could easily be set to “allow_failure=true”, but it makes code more readable for yourself and others.

Redirecting to mobile Wikipedia

Wikipedia both has a traditional and progressive website that is shown on mobile devices. After years the progressive website is still not shown on desktops sadly enough, but with a browser plugin a redirect can be triggered to the mobile site in most browsers. This as the Redirector plugin works at least in Mozilla Firefox, Google Chrome and Chromium, and will most likely also work in future when Microsoft Edge switches to the Chromium engine.

The configuration in JSON-format below can be imported to setup the Redirector plugin. After enabling the redirect rules, the browser should redirect the Wikipedia to the mobile Wikipedia website.

{
    "createdBy": "Redirector v3.2",
    "createdAt": "2019-04-05T16:30:43.187Z",
    "redirects": [
        {
            "description": "Wikipedia",
            "exampleUrl": "https://en.wikipedia.org/wiki/Wikipedia:About",
            "exampleResult": "https://en.m.wikipedia.org/wiki/Wikipedia:About",
            "error": null,
            "includePattern": "https://([a-z]{2}).(wikipedia|wiktionary|wikiquote|wikisource|wikibooks|wikiversity|wikinews|wikivoyage).org/wiki/(.+)",
            "excludePattern": "\\.m\\.wikipedia\\.org",
            "patternDesc": "",
            "redirectUrl": "https://$1.m.$2.org/wiki/$3",
            "patternType": "R",
            "processMatches": "noProcessing",
            "disabled": false,
            "appliesTo": [
                "main_frame"
            ]
        }
    ]
}

For now the configuration should cover the main Wikipedia websites and a most of the sub-projects in all languages.

Update 2019-05-25: Adding redirect for Twitter.

{
    "createdBy": "Redirector v3.2",
    "createdAt": "2019-05-24T20:05:14.345Z",
    "redirects": [
        {
            "description": "Twitter",
            "exampleUrl": "https://twitter.com/notifications",
            "exampleResult": "https://mobile.twitter.com/notifications",
            "error": null,
            "includePattern": "https://twitter.com/(.+)",
            "excludePattern": "",
            "patternDesc": "",
            "redirectUrl": "https://mobile.twitter.com/$1",
            "patternType": "R",
            "processMatches": "noProcessing",
            "disabled": false,
            "appliesTo": [
                "main_frame"
            ]
        }
    ]
}

Using YUM history to see package changes

When you install or update packages on your system, then changes may occur that were not expected. A recent security update on a server and left Nagios in a failed state, but what exactly happend and can it be traced back as yum-cron installs all required security updates? Luckily YUM keeps a history database of all actions and with yum history can you list all transactions.

$ sudo yum history list all
Loaded plugins: fastestmirror
ID     | Login user               | Date and time    | Action(s)      | Altered
-------------------------------------------------------------------------------
    15 | root <root>              | 2019-02-02 07:30 | Update         |    2   
    14 | root <root>              | 2019-01-05 07:52 | Update         |   50   
    13 | System ... <sysadmin>    | 2018-11-04 20:45 | I, U           |   62   
    12 | Ansible ... <ansible>    | 2018-11-04 01:36 | Install        |    4   
    11 | root <root>              | 2018-10-20 04:21 | Update         |    2   
    10 | root <root>              | 2018-10-06 07:45 | Update         |    2   
     9 | System ... <sysadmin>    | 2018-09-15 08:06 | I, U           |    9   
     8 | System ... <sysadmin>    | 2018-09-12 03:19 | Install        |    1   
     7 | Ansible ... <ansible>    | 2018-09-09 13:19 | Install        |    1   
     6 | Ansible ... <ansible>    | 2018-09-09 13:14 | Install        |   29   
     5 | Ansible ... <ansible>    | 2018-09-06 14:11 | I, U           |   81   
     4 | Ansible ... <ansible>    | 2018-09-06 13:21 | Install        |    1   
     3 | Ansible ... <ansible>    | 2018-09-06 13:20 | Install        |   51   
     2 | Ansible ... <ansible>    | 2018-09-06 13:14 | Install        |    1   
     1 | System <unset>           | 2018-09-06 03:17 | Install        |  275   
history list

As transaction 15 was the latest and only transaction before the defect occurred it is the one to look into. With yum history info the details of the transaction can be shown. It show when and who triggered the transaction, but also with which version of RPM, YUM and which plugins for YUM were used. Most important it also shows which package were updated with versions used and from which repository. This narrows the search down to the packages shown as updated and see what they changed on the system.

$ sudo yum history info 15
Loaded plugins: fastestmirror
Transaction ID : 15
Begin time     : Sat Feb  2 07:30:58 2019
Begin rpmdb    : 450:5f24b4b6a7aaef9f42874d6c8643385133020181
End time       :            07:31:04 2019 (6 seconds)
End rpmdb      : 450:246b0b638aa8b6b851529eb1b040714b7149d0e9
User           : root <root>
Return-Code    : Success
Transaction performed with:
    Installed     rpm-4.11.3-32.el7.x86_64                        @anaconda
    Installed     yum-3.4.3-158.el7.centos.noarch                 @anaconda
    Installed     yum-plugin-fastestmirror-1.1.31-46.el7_5.noarch @updates
Packages Altered:
    Updated nagios-4.3.4-5.el7.x86_64        @epel
    Update         4.4.3-1.el7.x86_64        @epel
    Updated nagios-common-4.3.4-5.el7.x86_64 @epel
    Update                4.4.3-1.el7.x86_64 @epel
history info

Red Hat Linux 8 will be using dnf instead of yum like Fedora 18 and later, but you don’t have to relearning anything as you can use dnf in the same way as yum and with the same parameters for now.

Check HTTP with telnet

HTTPS may become the standard quickly, but HTTP is still the base and understanding how to verify an HTTP server without a web browser can be very useful. A lot of situations simply don’t allow you to install a web browser or gives only a blank page.

As HTTP is a plain-text protocol you can simulate a connection with telnet on the command line. So let connect to fresh Linux machine with Apache running and see what happens. After connecting you type in “GET /index.html HTTP/1.1” to tell webserver which file you want to get and in this case the file in /index.html. The second line tells the webserver for which website you make the request which is 192.168.121.7.xip.io in the example. And finally you give an additional enter to tell your request is complete and can be processed after which you get the response.

$ telnet 192.168.121.7.xip.io 80
Trying 192.168.121.7...
Connected to 192.168.121.7.
Escape character is '^]'.
GET /index.html HTTP/1.1
Host: 192.168.121.7.xip.io

HTTP/1.1 404 Not Found
Date: Sun, 06 Jan 2019 01:27:00 GMT
Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips PHP/7.3.0
Content-Length: 208
Content-Type: text/html; charset=iso-8859-1

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL /index.html was not found on this server.</p>
</body></html>
Connection closed by foreign host.

The response in the example tells that the file index.html doesn’t exist on the webserver, which is correct for this example. It also give additional metadata about the server and the form the content is served which can be handy to see if the mimetype matches or the response size is correct.

PHP 7.3 and forbidden functions

Last month PHP 7.3.0 was released and with that a lot of functions or aliases were deprecated that may lead to issues down the road. While Xdebug still needs to be released for PHP 7.3 an automated test with GitLab isn’t possible yet as the build phase of Xdebug fails. Luckily I’m using PHP Code Sniffer and extending phpcs.xml.dist with the lines below make the build already fail if any of the forbidden functions are being used in the code.

    <!-- Ban some functions -->
    <rule ref="Generic.PHP.ForbiddenFunctions">
        <properties>
            <property name="forbiddenFunctions" type="array">
                <!-- Deprecated Features 7.0, https://secure.php.net/manual/en/migration70.deprecated.php -->
                <element key="ldap_sort" value="null"/>
                <!-- Deprecated Features 7.1, https://secure.php.net/manual/en/migration71.deprecated.php -->
                <!-- Deprecated Features 7.2, https://secure.php.net/manual/en/migration72.deprecated.php -->
                <element key="create_function" value="null"/>
                <element key="each" value="null"/>
                <element key="gmp_random" value="null"/>
                <element key="read_exif_data" value="exif_read_data"/>
                <element key="png2wbmp" value="null"/>
                <element key="jpeg2wbmp" value="null"/>
                <element key="__autoload" value="null"/>
                <!-- Deprecated Features 7.3, https://secure.php.net/manual/en/migration73.deprecated.php -->
                <!-- Searching Strings for non-string Needle -->
                <element key="strpos" value="chr"/>
                <element key="strrpos" value="chr"/>
                <element key="stripos" value="chr"/>
                <element key="strstr" value="chr"/>
                <element key="stristr" value="chr"/>
                <element key="strchr" value="chr"/>
                <element key="strrchr" value="chr"/>
                <!-- Strip-Tags Streaming -->
                <element key="fgetss" value="fgets"/>
                <element key="gzgets" value="gzgets"/>
                <!-- Image Processing and GD -->
                <element key="image2wbmp" value="imagewbmp"/>
                <!-- Multibyte String -->
                <element key="mbregex_encoding" value="mb_regex_encoding"/>
                <element key="mbreg" value="mb_ereg"/>
                <element key="mbregi" value="mb_eregi"/>
                <element key="mbreg_replace" value="mb_ereg_replace"/>
                <element key="mbregi_replace" value="mb_eregi_replace"/>
                <element key="mbsplit" value="mb_split"/>
                <element key="mbreg_match" value="mb_ereg_match"/>
                <element key="mbreg_search" value="mb_ereg_search"/>
                <element key="mbreg_search_post" value="mb_ereg_search_post"/>
                <element key="mbreg_search_regs" value="mb_ereg_search_regs"/>
                <element key="mbreg_search_init" value="mb_ereg_search_init"/>
                <element key="mbreg_search_getregs" value="mb_ereg_search_getregs"/>
                <element key="mbreg_search_getpos" value="mb_ereg_search_getpos"/>
                <element key="mbreg_search_setpos" value="mb_ereg_search_setpos"/>
            </property>
        </properties>
    </rule>

Hopefully PHP Code Sniffer will be extended to check on deprecated constants as well, but for now all code running on PHP 7.2 can be checked to run smoothly on PHP 7.3 and later.