Andy Crouch - Code, Technology & Obfuscation ...

Why A Professional Approach To Testing Is Critical To Your Product

02 February 2021

Photo: Unsplash - Cookie the Pom

This week I read that the Texas Department of Public Safety had to apologise after sending an Amber Alert out featuring Chucky. An Amber Alert is a notification system used in the USA to alert the public about missing or abducted children. Amber Alerts are SMS’d to public cell phones, sent to radio and TV stations and posted to the internet. It is an application that is time sensitive with a high impact outcome.

The message sent by the DPS featured Chucky’s fictional friend Glen as the abducted child. The listed suspect was Chucky and they included a picture of him holding a knife. Oh, and they set his race as “Other: Doll”. You can see more in the tweet below.

In any other context or system, the humour of using these characters might have been seen as funny. Within the context of the Amber Alert system, people were rightly annoyed and dismayed. This is a notification system which in its first 17 years rescued over 600 missing children. So, how did the department responsible explain the Alert?

It was a test malfunction.

A Real Word Story - What Happened

These types of incidents are often reported. Usually with only the embarrassment of the situation as the result. That isn’t always the case. Over ten years ago I was working on an Incident Management Control application. The software allowed clients to manage information and resources at a time of crisis from one, central, system. We had various clients that spanned all kinds of industries. We also had a codebase that was updated and deployed with the frequency of CI/CD without the automated processes and tooling. A large subsystem of the application related to notifications. We could handle everything from pagers (not joking) through email, text to speech and SMS. At the point I joined, the code was never designed to disable or allow for testing of the notification subsystem. After each deployment, the database config had to be updated to disable the functionality (using MS SQL Management Studio no less) in the test environment.

I think you can see where this is going?

A QA we had working on the team was testing the system and ran a well-documented incident scenario involving two of our clients. Unfortunately, the technical lead had rolled out a set of changes to the test site and had not updated the notification settings.

At the point of which the notifications started to be sent the QA called and told me what had happened. I immediately got on to the CEO to damage control the situation but it was too later. Even in that small timeframe, both companies had convened their Incident teams. Those teams included the whole C level team on both sides. Key staff had been sent to the incident management centres. In case you are wondering, one of the clients was an airline.

A Real Word Story - How Did It Happen

There were a number of failures that led to this event happening.

The first was a process failure. The code had been acquired as part of a larger acquisition by the parent group. The original infrastructure for the project was a single Production environment. The source code control lived on a machine in the original developers living room. As the business grew rapidly around the software no one looked at the “how” things were being done. Manual deploys (even back then) were causing a time drain and the process had to be taught to new hires. It was multistep and allowed for a lot of human error.

The second was the failure to develop a codebase that allowed for testing. As a side note, there were no automated tests. That is a whole other post in itself. It indicated the approach of the original developer who would turn off options at the database level. There was no clear configuration layer in the application that made it easy for a human (or automation process) to control.

The third was not listening to the users. In this case, the user was the QA that had been assigned to the project. They had asked for a testing environment to be created which was done but which they had flagged some issues. The first was that it was hosted on the same domain as Production, mapped to a /test suffix. They felt that without due care it was too easy to end up in the wrong environment. The second was that the test system had no visual cue showing it was the test system. The third was they could not check or control application options from the UI.

Think about that for a minute. A user gave clear and concise feedback to address a serious issue and they were ignored. The outcome could have not only cost the company a considerable amount of money but brand and reputational damage as well.

Needless to say, the points raised by the QA were addressed and there was an overhaul of how the application was developed and deployed. This was a good thing but it shouldn’t have taken the failure of a test system to drive it.

How To Approach Testing For Your Project

There are some key takeaways from both my own experience and the “The Chucky Alert”. Keep in mind four main reasons why testing is key to software development:

  • Time And Cost Saving. The sooner you find issues, the cheaper it is to fix them, both financially and in resources.
  • Product Quality. Your application works. It provides the value that your customers expect and functions as advertised.
  • Security. Testing will prevent most known, documented, security issues and reduce vulnerabilities.
  • Customer Satisfaction. By ensuring your application works well, as specified, your customers are more engaged and satisfied.

When starting a new project, you need to consider how you are going to prioritise testing. Think about the use of Feature Flags to enable or disable functionality without the redeployment of an application. Make these accessible to both humans and automation tools. Think about how you design your integrations with various subsystems and APIs. Put these behind a Facade pattern and develop testing fake alternative integrations that prove a system is working. Instead of sending SMS messages to an API have a plugin that allows you to write them to a file.

While it may be tempting to bake in test user logic (I have seen this first hand) into your application, this is a bad idea. One application I worked on used a numeric ID for users. If the number started with a 9 it was a test user. It meant that the code was littered with tests for 9 as the first digit of the user id and then provided a different implementation. (I would have gone for the use of Generics as it was a .Net app but again, a whole other post).

What this all boils down to is that you should have a separate environment for testing your application. There should be no shared resources, databases or API access. With modern CI/CD automation, containers and very cheap resources on hosted platforms, there is no excuse not to.

One final thought, use realistic and professional test data. If everything fails and for some reason your users see something that is an actual test then don’t repeat the Chucky situation. Apologising for sending a test message is one thing but having to explain why you are using potentially offensive test data has a terrible impact on the way your users view your product and team.

If you have any thoughts or comments about how your team approaches testing then let me know via twitter or email.

Configuring Multiple Colour Palettes In Tailwind CSS

05 January 2021

Photo: Unsplash - Jessica Ruscello

I recently wrote about the Tailwind CSS framework and how productive I have found it on my recent projects. One of those projects decided to do a partial redesign to test a new landing page layout. The changes included a brand new colour palette and given how Tailwind defines colours in the config I wondered how I could support both colour palettes.

It turns out tha Tailwind made this really easy.

Basic Tailwind Color Configuration

Tailwind comes with a full colour palette defined which you can use by adding the following line to your tailwind.config.js file:

const colors = require("tailwindcss/colors");

This provides the following colour shades for you to use in your project:

You can access any of the shades above in your utility classes by specifying the colour and variant, as follows:

<div class="border border-red-400 bg-blue-400 text-yellow-400">A clash of colour</div>

Extending The Tailwind Colour Palette

You can define your own colour palette by adding your configuration to the theme section within the tailwind.css.js file:

module.exports = {
  theme: {
    colors: {
      // Define your colour settings here
    },
  },
};

By defining your own palette as part of your theme configuration, you can define your own shades to override the default colours that Tailwind provides:

module.exports = {
  theme: {
    colors: {
      blue: {
        300: "#ced8df",
        400: "#b5c4ce",
        500: "#849dae",
        600: "#637683",
      },
    },
  },
};

Here I am overriding the shades of blue for shades 300, 400, 500 and 600. I can continue to use the remaining predefined blue shades that Tailwind offers without having to override them.

You can take this further. If you want to limit yourself to a subset of shades per colour, you can provide custom identifiers for the shades you want instead of sticking to the number identifiers. For example:

module.exports = {
  theme: {
    colors: {
      blue: {
        light: "#ced8df",
        default: "#849dae",
        dark: "#637683",
      },
      yellow: {
        light: "#ffd774",
        default: "#ffca46",
        dark: "#ffbd18",
      },
    },
  },
};

You would then access them in your utility classes as follows:

<div class="border border-blue-default bg-blue-dark text-yellow-dark">A clash of colour</div>

All this flexibility provided a really easy way to define a second colour palette, as I needed for my project. I had already defined a colour palette as part of the original implementation using the standard 50 - 900 shade mechanism. All I had to do was define the new colour palette and use a custom naming convention to segregate the shades.

module.exports = {
  theme: {
    colors: {
      gray: {
        100: "#cfd3d6",
        homePage100: "#9fa6ad",
        300: "#707a83",
        homePage300: "#404d5a",
        500: "#102131",
        homePage500: "#0c1925",
      },
    },
  },
};

The Tailwind Purge Configuration Option

One of the really useful things that Tailwind does is to optimise the generated CSS file to only include the utility classes you have used in your site. This means that if you do not apply blue-500 to any element then no blue-500 css classes will be produced. This functionality is enabled by the purge configuration option as shown below:

// tailwind.config.js
module.exports = {
  purge: {
    enabled: true,
    content: ["./src/**/*.html", "./src/**/*.html", "./src/**/*.jsx", "./src/**/*.js"],
  },
};

The content field is used to specify the file types to parse when determining the classes to generate.

I mention this mainly for those of us that are using frameworks such as React or Vue. If you take advantage of the “purge” functionality (and why wouldn’t you?) you can not use string interpolation to build dynamic style attributes. So the following wouldn’t work:

const colour = "blue-500";
<div className={`bg-${colour} text-white`}>This will have no background colour</div>;

Where as this will:

const colour = "blue";
<div className={`${colour === "blue" ? `bg-blue-500` : `bg-none`} text-white`}>This will have a blue background colour</div>;

This is down to the way Tailwind parses the utility classes to remove using PurgeCSS. I will write more on the impact of this soon.

Wrapping Up

Tailwind provides a very flexible configuration mechanism to allow you to define your theme and colour palette as you need.

I would advise against adding a secondary palette unless you have a real need (as in my case) as you then weaken two of Tailwinds great strengths, simplicity and intention, I have already migrated the remainder of the project to the new colour palette and removed the secondary palette. If you do add multiple colour palettes I would recommend you use specific identifiers in your config to associate the shades to particular pages or areas of your site. Make it obvious to other developers why the additional palettes exist and it’s intention.

If you have any questions about Tailwind or configuring the colour settings then let me know via twitter or email.

Building Serverless React Forms Using React Hook Form & Getform.io

01 December 2020

Photo: Unsplash - NordWood Themes

The speed of development you can achieve using serverless architectures and services can be mind-boggling. For small projects, having the ability to connect several services, each having a specific purpose, is very powerful. Recently, I built an MVP that used GetForm.io and Zapier to connect a GatsbyJS web app to Mailchimp. The whole process took just a couple of hours.

The aim of the project was to build a long contact form which collected a lot of information from the user. As the web project was built in GatsbyJS, I was building a React form. Small forms in React are straightforward and it is trivial to handle validation. When you build larger forms with many fields, each of which has different validation rules, it is easy for your code to run away from you.

React-Hook-Form

After a search to see if there were any packages that would help make it easier to build the form, I found React Hook Form. Its strapline was “Performant, flexible and extensible forms with easy-to-use validation.” It sounded exactly what I needed. I was able to build the form using standard HTML controls. React Hook Form can register and track the form fields using a registration mechanism using the ref attribute of your controls. They provide easy access to validation errors and as a nice touch, you can “watch” your field values in the console while building and debugging your forms.

To get started, add react-hook-form to your project:

$ npm install react-hook-form

A very basic example form component would look similar to this:

import * as React from "react";
import { useForm } from "react-hook-form";

export default () => {
  const { handleSubmit, register, watch, errors } = useForm();

  const onSubmit = (data) => console.info(data);

  console.info(watch("firstName"));
  console.info(watch("lastName"));
  console.info(watch("age"));

  return (
    <form onSubmit={handleSubmit(onSubmit)}>
      <input name="firstName" ref={register({ required: true })} />
      {errors.firstName && <span>First Name Is Required</span>}

      <input name="lastName" ref={register({ required: true })} />
      {errors.lastName && <span>Last Name Is Required</span>}

      <input name="age" defaultValue="21" ref={register()} />

      <input type="submit" />
    </form>
  );
};

As you can see in the above example:

  • We import react-hook-form.
  • We define the methods we need using useForm().
  • We define an onSubmit handler to fire when a user submits the form. This will receive a data parameter from the hook and you can still define an e parameter to access the target form if you need to.
  • We then define the form. For the onSubmit form handler, we pass in the handleSubmit hook method, passing in our own onSubmit event handle that we have defined.
  • We then just define our form fields as we would but we register them by applying a ref attribute which is used to register the field to the form. This is done by calling the register method passing in any validation rules that need to be applied. All fields need to have a unique name attribute applied.
  • Validation errors are handled via the errors object and each registered field becomes a property to the object. You can see in the first and last name fields I have made them mandatory and if the form is submitted without them being completed the associated validation message are displayed.

When the onSubmit handler fires all of the data in your form is passed as the data parameter. This means you are free to process it as you need. The watch method I have used above is a useful utility to allow you to track and test your code as you develop it. By passing in the name of the field you want to watch, you can see in real-time the changes as you type them in your form in your browser dev tools.

You should read the documentation in full here to find out what else is possible using React Hook Form.

Getform.io

GetForm.io is a really simple service that provides a flexible way to store form data from your site. I have used it in multiple projects for collecting email addresses or contact form data. It works by allowing you to set up a form which has a unique endpoint. You apply the endpoint using the action attribute on your form and ensuring that the method attribute is set to “POST”. This will then post you form data straight to your form in GetForm.io. You can then export your data or set up integrations to automatically sync it with other applications or databases. It is so simple they have an example on their Home Page.

They also have an API which can be used to fire your data into your form which is what I opted to use in my lengthy contact form. The code below shows how this would work in a scaled down manner.

import axios from "axios";
import * as React from "react";
import { useForm } from "react-hook-form";

export default () => {
  const [serverState, setServerState] = useState({ submitting: false, status: null });

  const handleServerResponse = (ok, msg, form) => {
    setServerState({ submitting: false, status: { ok, msg } });

    if (ok) {
      // show success message
      // handle success next steps
      return;
    }

    // show failure message
    // handle failure next steps
    return;
  };

  const onSubmit = (data, e) => {
    const formData = new FormData();

    for (var field in data) {
      formData.append(field, data[field]);
    }

    setServerState({ submitting: true });

    // replace the url parameter with your own GetForm.io form id/url
    axios({
      method: "post",
      url: "https://getform.io/f/11111111-aaaa-1111-aaaa-111111111111",
      data: formData,
    })
      .then((r) => {
        e.target.reset();
        handleServerResponse(true, "Form Submitted", e);
      })
      .catch((r) => {
        handleServerResponse(false, r.response.data.error, e);
      });
  };

  const { handleSubmit, register, watch, errors } = useForm();

  console.info(watch("firstName"));
  console.info(watch("lastName"));
  console.info(watch("age"));

  return (
    <form onSubmit={handleSubmit(onSubmit)}>
      <input name="firstName" ref={register({ required: true })} />
      {errors.firstName && <span>First Name Is Required</span>}

      <input name="lastName" ref={register({ required: true })} />
      {errors.lastName && <span>Last Name Is Required</span>}

      <input name="age" defaultValue="21" ref={register()} />

      <input type="submit" />
    </form>
  );
};

The changes here are very straightforward:

  • I define a state variable serverState to track when the form is waiting for the server to respond. This is useful to control UI elements when posting your data.
  • I update the onSubmit function to receive the e parameter to be able to access the form. I also create a new FormData object and populate it with the fields passed in the data parameter.
  • I then use an axios promise to post the form data to my GetForm.io form (remember to use the correct url as specified in your form settings) and handle the response accordingly.
  • In the example, I define a handleServerResponse function which takes some params for me to handle the axios response. In my real-world example I used this function to show a notification to the user for example. The important thing to note is to ensure you update the serverState variable. You can also see I use the e param in the then promise handler to reset the form back to its original state.

Zapping GetForm.io To Mailchimp

Once the form was working and the data was saving to GetForm.io all that was left was to ensure that when a user submitted the contact form that they received a Thank You email. This was already set up on Mailchimp which was being used for automated email workflows. I set up a Zapier Zap to connect my GetForm.io form and the Mailchimp account. This approach is really flexible and allows you to select the fields that you want to sync with your Mailchimp audience.

Wrapping up

I really like React Hook Form as it makes stuff super simple and form building can be a horrible task no matter what your tech stack. I still have ASP related nightmares even after all of this time.

I am really just scratching the surface of what you can do with serverless architecture, there is a so much more available. The power for me in using these services is being able to quickly prototype and get running functionality that previously could have taken a few days with a couple of developers.

If you have any questions or further tips for using React Hook Form, GetForm.io, Zapier or Mailchimp then let me know via twitter or email.

Manjaro October 2020 Stable Update Nvidia Driver Issue

24 November 2020

Photo: Unsplash - Pierre Bamin

Since I went consulting, 14 months ago, I switched to running Manjaro Linux on a ThinkPad p52 as my daily driver. It has been rock solid and I have used it for everything from general admin to building projects using various tech stacks. I haven’t found one thing that I couldn’t do and when I use a non-Linux based OS I find them slow and hard to work with.

There is one thing that could be easier, that is the proprietary video card. The P52 has a combined Intel graphics card and an Nvidia Quadro P1000. When I got the laptop I set up the Nvidia drivers using the video-nvidia-440xx driver. Once I had Optimus Manager installed all was well.

A couple of weeks ago I ran the update tool in Manjaro and found that my external monitors were failing to work. I started to dig into the issue and found that I had actually updated Manjaro to the latest stable release which had been rolled out in October. A little bit of digging found various error messages about the Nvidia card not being detected. This made sense as the external monitors are connected to the Nvidia card. I soon discovered that the -440 series drives were not compatible with newly updated Linux 5.9 kernel.

To solve the issue you need to upgrade to the video-nvidia-450 driver family. The only issue is that the -440 and -450 utils packages are in conflict. I found the following steps solved the issue and let me upgrade (keep in mind I use i3 and Lightdm so you may need to handle other packages for other desktop environments):

  • To list the Nvidia drivers you are running execute the following

$ mhwd --listinstalled

  • Note the kernel version you are running and uninstall all packages that relate to the -440 series driver

$ pamac remove nvidia-440xx-utils linux$$-nvidia-440xx

(You should replace $$ with the kernel version you are running).

  • Some users reportedly had to uninstall the following (especially if you are running KDE)

$ pamac remove cuda nvtop

  • You will need to repeat the above steps for all kernels you have installed. If you want to check, you can list the kernels using

$ uname -r

(Or use the Manjaro Settings Manager, select “kernel”).

You should now be able to install the Nvidia driver using your preferred method. I use the Manjaro Settings Manager as it is quick and easy. Once completed, you can reboot.

While this wasn’t too hard to fix it was yet another small and annoying issue caused by the Nvidia card. Hopefully, the steps above will be of use to others. If you have any questions about upgrading to the -450 series driver then let me know via twitter or email.

A High Level Introduction To Tailwind CSS

17 November 2020

Photo: Unsplash - Branko Stancevic

The last few projects I have built have been either React or Gatsby based. When setting up these projects I wanted a fast and simple way to apply a consistent and maintainable style system for them. There are different ways to define styles when building components, from inline styles and CSS stylesheets through to CSS-in-JS. On top of these foundations has grown a wide array of libraries and tools to help developers define and manage their project styles. From Sass which extends the CSS language through to frameworks such as Bootstrap. I could have gone with any of these for my projects but I didn’t.

Instead, I have gone to a happy place by using Tailwind CSS.

First, Some History

CSS is easy to write and even easier to allow to grow into a mess. On small projects with limited resources, it is not so much of a problem but when the codebase grows it can be hard to keep your CSS under control. It’s a problem that seems to become linearly harder based on the number of developers on the project. There have been many attempts from projects trying to govern how CSS should be written and applied in code. It would seem that any developer that has written on the subject has at least 3 suggestions about how to solve the problems that CSS causes.

The most logical of the previously suggested approaches was BEM. This stands for Block, Element and Modifier. This is a great way to structure your HTML and CSS but it requires you to be consistent and does not provide much in the way of tooling to help you. This brings us to the biggest issue with almost all styling approaches, humans. On large projects knowledge transfer is hard, developers will struggle to maintain convention-based code that a compiler will ignore. You will end up with a lot of duplicate code and implementing a consistent UI/UX style becomes way more work than it should be.

What Is Tailwind?

Tailwind CSS is a utility first CSS framework which aims to provide an API for your design system.

The main difference between Tailwind and other frameworks is that it provides a default set of helper style classes and an adaptable configuration approach but does not provide ready to use components. Instead, by working at a lower level, you apply the helper classes directly to your elements to quickly build your own components. Tailwind is completely unopinionated and its configuration is highly flexible to allow you to build and include only the features that you need.

Getting Started

Tailwind has brilliant documentation. Probably some of the best I have seen across both Open Source and commercial products. Follow their installation guide for your project type to get Tailwind set up and ready to use.

Once installed, you will have created a tailwind.css.js configuration file which sits at the root of your project. This will be used to configure the styles and elements to include at build time.

You will also need to include the following in your CSS file. These are placeholders for the build process which will generate the required styles and inject them into your CSS file.

@tailwind base;
@tailwind components;
@tailwind utilities;

Using Tailwind

Tailwinds utility classes can be used directly within the HTML as shown on the snippet below:

<div class="container mx-auto border border-blue-500">
  <h1 class="text-2xl font-bold text-red-400">Tailwind CSS</h1>
  <button class="w-16 text-white bg-blue-500 rounded shadow-lg hover:bg-yellow-500 hover:text-black ">
    Click Me
  </button>
</div>

So what are we doing here? Lets first look at the div definition.

<div class="container mx-auto border border-blue-500">

We are defining a div element to act as a container and we are applying the container utility class. This sets the div’s width to a fixed size based on the current screen breakpoint. To centre the fixed size container, we apply the mx-auto utility class. m prefixed utility classes control margin’s and the x means we want to control the horizontal margins. By using the auto modifier we are providing an equal margin on the left and the right of the container so that it becomes centred in the total screen area. Finally, we are applying a full border to the container using the border utility class and specifying the colour of the border using the border-blue-500 utility class.

<h1 class="text-2xl font-bold text-red-400">Tailwind CSS</h1>

Tailwind provides several utility classes that make working with text simple. In the h1 snippet above we are using the text prefixed utility classes to increase the size of the text to xx-large and to make the text red. The –500 modifier is used to specify the shade of red to apply based on your configuration. The colour pallet used in Tailwind can be fully customised and provides 9 shades for each colour which can be accessed by the -100 to -900 modifiers. The font prefixed modifier is used to make the text bold.

<button class="w-16 text-white bg-blue-500 rounded shadow-lg hover:bg-yellow-500 hover:text-black ">
   Click Me
</button>

Tailwind is designed to support responsiveness out of the box and fully supports variants as a way to work with pseudo-classes. This can be seen in the button snippet above. We start by using the w utility class to specify a width for the button. Tailwind provides a large number of predefined widths and heights (using the h prefixed modifiers). We then define the background and text colours using the text and bg-blue-500 class variants. These are the standard styles applied to the element. To change the colours of the background and text when we hover over the button element we define secondary colour classes and prefix them with the hover variant modifier.

The Tailwind documents show what a broad selection of utility classes they provide.

Really Useful Resources For Tailwind

The following are a bunch of really useful rearouses and links I found when picking up and building things with Tailwind:

Wrapping Up

Tailwind is quick to pick up and it is so intuitive that when you switch back to a project that is not using it, you really miss it. It is not an infallible system and I am sure that some projects may still struggle with their CSS by using it. But, by mixing it with other tools available (such as Emotion or Storybook), developing a design system has become a lot easier to implement and manage. I will be pushing for Tailwind at the start of all projects going forward so I can stay in my happy place.

If you have any questions or tips in using Tailwind then let me know via twitter or email.