
In Node.js, buffers are a special type of object that can store raw binary data. A buffer represents a chunk of memory - typically RAM - allocated in your computer. Once set, the size of a buffer cannot be changed.

A buffer stores **bytes**. A byte is a sequence of eight **bits**. Bits are the most basic unit of storage on your computer, they can hold the value of either 0 or 1.

Node.js exposes the `Buffer` class in the global scope (you don't need to import or require it like other modules). With this API, you get a series of functions and abstractions to manipulate raw binaries.

A buffer in Node.js looks like this:

```jsx
<Buffer 61 2e 71 3b 65 2e 31 2f 61 2e>
```

In this example, you can see 10 pairs of letters and numbers. Each pair represents a byte stored in the buffer. The total size of this particular buffer is 10.

You might be asking yourself: "if these are bits and bytes, where are the 0s and 1s?"

That's because Node.js displays bytes using the **hexadecimal** system. This way, every byte can be represented using just two digits - a pair of numbers and letters from 0-9 and "a" to "f".

**Why buffers?** Before buffers were introduced, there was no easy way of handling binary data in JavaScript. You would have to resort to primitives such as strings, which are slower and have no specialized tools to handle binaries. Buffers were created to provide a proper set of APIs to manipulate bits and bytes in an easy and performant way.

## Working with buffers

Let's see some of the things we can do with buffers.

You will notice that handling buffers is a bit similar to the way we handle arrays in JavaScript. For example, you can `.slice()`, `.concat()` and get the `.length` of a buffer. Buffers are also iterable and can be used within constructs such as `for-of`.

If you're following the examples on your computer, keep in mind that the `Buffer` class is exposed globally. You don't need to import or require it as a separate module.

### Creating buffers

Buffers are created using these three methods:

- Buffer.from()
- Buffer.alloc()
- Buffer.allocUnsafe()

> In the past, buffers were created using the Buffer class constructor (e.g., `new Buffer()`). This syntax is now deprecated.

**Buffer.from()**

This method is the most straightforward way to create a buffer. It accepts a string, an array, an `ArrayBuffer`, or another buffer instance. Depending on which params you pass, `Buffer.from()` will create a buffer in a slightly different way.

When passing a string, a new buffer object will be created containing that string. By default, it will parse your input using **utf-8** as the enconding (see [here](https://nodejs.org/api/buffer.html#buffer_buffers_and_character_encodings) all enconding types supported):

```jsx
// Creates a new buffer with the string 'heya!'
// If no enconding is passed in the second parameter, defaults to 'utf-8'.
Buffer.from("heya!");

// Creates the same buffer as the above, but passes 'heya!' as a hex encoded string
Buffer.from("6865796121", "hex");
```

You can also pass an array of bytes to `Buffer.from()`. Here I am passing the same string as before (_"heya!"_), but represented as an array of hexadecimal characters:

```jsx
// Also writes 'heya!' to the buffer, but passes a array of bytes
Buffer.from([0x68, 0x65, 0x79, 0x61, 0x21]);
```

> If you're not familar with the `0xNN` syntax, it means that the characters after `0x` should be interpreted as hexadecimal values.

When passing a buffer to `Buffer.from()`, Node.js will copy that buffer into the current one. The new buffer is allocated in a different area of memory, so you can modify it independently:

```js
const buffer1 = Buffer.from("cars");

// Creates a buffer from `buffer1`
const buffer2 = Buffer.from(buffer2);

// Modify `buffer2`
buffer2[0] = 0x6d; // 0x6d is the letter "m"

console.log(buffer1.toString()); // --> "cars"
console.log(buffer2.toString()); // --> "mars"
```

These should cover most of the cases where you use `Buffer.from()`. Refer to the [docs](https://nodejs.org/api/buffer.html) for other ways to use it.

**Buffer.alloc()**

The `.alloc()` method is useful when you want to create empty buffers, without necessarily filling them with data. By default, it accepts a number and returns a buffer of that given size filled with 0s:

```jsx
Buffer.alloc(6);
// --> <Buffer 00 00 00 00 00 00>
```

You can later on fill the buffer with any data you want:

```jsx
// Creates a buffer of size 1 filled with 0s (<Buffer 00>)
const buff = Buffer.alloc(1);

// Fill the first (and only) position with content
buff[0] = 0x78 // 0x78 is the letter "x"

console.log(buff.toString('utf-8');
// --> 'x'
```

You can also fill the buffer with other content than 0 and a given enconding:

```jsx
Buffer.alloc(6, "x", "utf-8");
// --> <Buffer 78 78 78 78 78 78>
```

**Buffer.allocUnsafe()**

With `.allocUnsafe()` , the process of sanitizing and filling the buffer with 0s is skipped. The buffer will be allocated in a area of memory that may contain old data (that's where the "unsafe" part comes from). For example, the following code will most likely always print some random pieces of data every time you run it:

```jsx
// Allocates a random area of memory with size 10000
// Does not sanitizes it (fill with 0) so it may contain old data
const buff = Buffer.allocUnsafe(10000);

// Prints loads of random data
console.log(buff.toString("utf-8"));
```

A good use case for `.allocUnsafe()` is when you are copying a buffer that was safely allocated. Since you will completely overwrite the copied buffer, all the old bytes will be replaced by predictable data:

```jsx
// Creates a buffer from a string
const buff = Buffer.from("hi, I am a safely allocated buffer");

// Creates a new empty buffer with `allocUnsafe` of the same
// length as the previous buffer. It will be initally filled with old data.
const buffCopy = Buffer.allocUnsafe(buff.length);

// Copies the original buffer into the new, unsafe buffer.
// Old data will be overwritten with the bytes
// from 'hi, I am a safely allocated buffer' string.
buff.copy(buffCopy);

console.log(buffCopy.toString());
// --> 'hi, I am a safely allocated buffer'
```

In general, `.allocUnsafe()` should only be used if you have a good reason (e.g., performance optimizations). Whenever using it, make sure you never return the allocated buffer without completely filling it with new data, otherwise you could be potentially leaking sensitive information.

### Writing to buffers

The way to write data into buffers is using `Buffer.write()`. By default, it will write a string encoded in `utf-8` with no offset (starts writing from the first position of the buffer). It returns a number, which is the number of bytes that were written in the buffer:

```jsx
const buff = Buffer.alloc(9);

buff.write("hey there"); // returns 9 (number of bytes written)

// If you write more bytes than the buffer supports,
// your data will truncated to fit the buffer.
buff.write("hey christopher"); // retuns 9 (number of bytes written)

console.log(buff.toString());
// -> 'hey chris'
```

Keep in mind that not all characters ocuppy a single byte in the buffer (!):

```jsx
const buff = Buffer.alloc(2);

// The copyright symbol ('©') occupies two bytes,
// so the following operation will completely fill the buffer.
buff.write("©"); // returns 2

// If the buffer is too small to store the character, it will not write it.
const tinyBuff = Buffer.alloc(1);

tinyBuff.write("©"); // returns 0 (nothing was written)

console.log(tinyBuff);
// --> <Buffer 00> (empty buffer)
```

Also notice that 2 is **not** the highest number of bytes a character can have. For example, the `utf-8` enconding type supports characters with up to 4 bytes. Since you cannot modify the size of the buffer, you always need to be mindful of what you are writing and how much space it will take (size of the buffer vs. size of your content).

Another way to write into buffers is throguh an array-like syntax, where you add bytes to a specific position of the buffer. It's important to notice that any data with more than 1 byte needs to be broken down and set on each position of the buffer:

```jsx
const buff = Buffer.alloc(5);

buff[0] = 0x68; // 0x68 is the letter "h"
buff[1] = 0x65; // 0x65 is the letter "e"
buff[2] = 0x6c; // 0x6c is the letter "l"
buff[3] = 0x6c; // 0x6c is the letter "l"
buff[4] = 0x6f; // 0x6f is the letter "o"

console.log(buff.toString());
// --> 'hello'

// ⚠️ Warning: if you try setting a character with more than 2 bytes
// to a single position, it will fail:
buff[0] = 0xc2a9; // 0xc2a9 is the symbol '©'

console.log(buff.toString());
// --> '�ello'

// But if you write each byte separately...
buff[0] = 0xc2;
buff[1] = 0xa9;

console.log(buff.toString());
// --> '©llo'
```

While it's appreciated that you can write to buffers using an array-like syntax, I suggest sticking to `Buffer.from()` when you can. Managing the length of inputs can be a hard task and will bring complexity to your code. With `.from()`, you can write things in a buffer worry-free and handle the cases where the input is too large by checking if nothing was written (when it returns 0).

### Iterating over buffers

You can use modern JavaScript constructs to iterate over a buffer the same way you would with an array. For example, with `for-of`:

```jsx
const buff = Buffer.from("hello!");

for (const b of buff) {
  // `.toString(16)` returns the content in hexadecimal format.
  console.log(b.toString(16));
}

// Prints:
// --> 68
// --> 65
// --> 6c
// --> 6c
// --> 6f
// --> 21
```

Other iterator helpers such as `.entries()`, `.values()` and `.keys()` are also available for buffers. For example, using `.entries()`:

```js
const buff = Buffer.from("hello!");
const copyBuff = Buffer.alloc(buff.length);

for (const [index, b] of buff.entries()) {
  copyBuff[index] = b;
}

console.log(copyBuff.toString());
// -> 'hello!'
```

## Going further: **Buffers and TypedArrays**

In JavaScript (I mean JavaScript in general, not restricted to Node.js), memory can be allocated using the special `ArrayBuffer` class. We rarely manipulate `ArrayBuffer` objects directly. Instead, we use a set of "view" objects which reference the underlying array buffer. Some of the view objects are:

`Int8Array`, `Uint8Array`, `Uint8ClampedArray`, `Int16Array`, `Uint16Array`, `Int32Array`, etc. See the full list [here](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray#typedarray_objects).

And then there is `TypedArray`, which is an umbrella term to refer to all of these view objects listed above. All view objects inherit methods from `TypedArray` via prototypes. The `TypedArray` constructor is **not** exposed globally, you always have to use one of the view methods. If you see some tutorial or documentation using `new TypedArray()`, it means it's using any of the view objects (Uint8Array, Float64Array, etc).

In Node.js, objects created from the `Buffer` class are also instance of `Uint8Array`. There are a few small differences between them, which you can read [here](https://nodejs.org/api/buffer.html#buffer_buffers_and_typedarrays).

## Conclusion

As a beginner, buffers were a topic in Node.js that got me very confused (another one was streams, but that deserves its own post). Hopefully I was able to demystify some of the concepts around buffers and give an overview of the Buffer API.

For questions, you can DM me on Twitter! Thanks for reading 👋

---



The newest major release of [npm](https://npmjs.com/), launched in October 2020, came out with a very anticipated feature (at least for me). The 7th version of the package manager introduced **Workspaces**.

The name may sound familiar. Other package managers such as [Yarn](https://classic.yarnpkg.com/en/) and [pnmp](https://pnpm.io/) already ship with Workspaces for quite a while now. In fact, npm is not trying to reinvent the wheel. You can find similarities between all three Workspace implementations.

But what are Workspaces for? Workspaces help us managing repositories with multiple packages - more than one `package.json` file. In projects like this, you usually have a complex dependency tree, with many packages depending on each other. This special type of repository is known as a **monorepo**.

In this post, you will see how npm Workspaces work, how to get started, and a comparison with other Workspace implementations. I've also provided a [repository](https://github.com/ruanmartinelli/npm-workspaces-demo) on GitHub with some sample code from the examples.

## What changes with Workspaces?

Now, when you run `npm install` in a multi-package repository, [npm's dependency tree manager](https://github.com/npm/arborist) is smart enough to scan your folders looking for all dependencies to install.

Dependencies are hoisted, meaning they get installed in the root `node_modules` folder. This is done for performance reasons: if a dependency is shared by multiple packages, it gets saved only once in the root.

Your packages (the ones you created) also get symlinked in the root `node_modules` folder. If two of your packages depend on each other, they get the reference from there.

For example, if you have this structure:

```bash
.
├── package.json
└── packages
    ├── package-a
    │   └── package.json # Dependencies: `lodash`
    └── package-b
        └── package.json # Dependencies: `lodash`, `package-a`
```

After running `npm install`, your `node_modules` folder will look like this:

```bash
.
├── node_modules
│   ├── lodash # `lodash` is installed in the root `node_modules/`
│   ├── package-a # package-a is symlinked
│   └── package-b # package-b is symlinked
├── package.json
└── packages
    └── # ...

```

If you have packages using the same dependency but on different versions, npm will create a `node_modules` folder inside of one of the packages. For example, suppose `package-a` uses `lodash@4` while `package-b` is still on `lodash@3`:

```bash
.
├── package.json
└── packages
    ├── package-a
    │   └── package.json # Dependencies: `lodash@4`
    └── package-b
        └── package.json # Dependencies: `lodash@3`
```

When running `npm install`, you will get the following result:

```bash
.
├── node_modules
│   ├── lodash # This will be `lodash` version 4.x
│   └── # ...
├── package.json
└── packages
    ├── package-a
    │   └── package.json
    └── package-b
        │   node_modules # `node_modules` is created inside the package
        │      └── lodash # This is `lodash` version 3.x
        └── package.json
```

These are just a few examples to illustrate how the new dependency resolution works on Workspaces, there's certainly a lot more happening under the hood.

## Getting started

### Setting up

You can try out Workspaces today by updating your npm to version 7. To update, run this command on your terminal:

```
npm install -g npm@7
```

If you install Node.js 15 today, it should already come with npm 7.

You can use a similar structure as the ones showed before as a starting point for playing around with workspaces. For example:

```
.
├── package.json
└── packages
    ├── package-a
    │   └── package.json
    └── package-b
        └── package.json
```

Notice you still need a `package.json` file in the root of your repository, even if it has no dependencies. In fact, it's on that file that you define the location where your packages live.

Edit the root `package.json` to reference your packages:

```json
// ./package.json
{
  // ...
  "workspaces": ["./packages/*"]
}
```

Now, when you run `npm install` in the root of your repository, npm will be smart enough to install `package-a` and `package-b`'s dependencies.

I made a simple GitHub repository with this example in case you wanna check it in more detail. Here’s the link: [npm-workspaces-demo](https://github.com/ruanmartinelli/npm-workspaces-demo).

### Using Workspaces

The npm CLI also introduced the `--workspace` and `--workspaces` flags. These flags can be added to many of the existing npm commands to run them in your sub-packages, instead of your root package.

For example, imagine `package-a` and `package-b` both have a script named "test" defined in their `package.json` files:

```json
// packages/package-a/package.json
{
  "name": "package-a",
  "scripts": {
    "test": "echo 'all package-a tests passed!'"
  }
  // ...
}

// packages/package-b/package.json
{
  "name": "package-b",
  "scripts": {
    "test": "echo 'all package-b tests passed!'"
  }
  // ...
}
```

You can run all the "test" scripts at once by adding the `--workspaces` (plural) to your `npm run` command:

```bash
# Run "test" script on all packages
npm run test --workspaces

# Tip - this also works:
npm run test  -ws
```

That will give you the following output:

```bash
> package-a@1.0.0 test
> echo 'all package-a tests passed!'

all package-a tests passed!

> package-b@1.0.0 test
> echo 'all package-b tests passed!'

all package-b tests passed!
```

To run a command for a specific package, add the `--workspace` (singular) flag:

```bash
# Runs "test" only on package-a
npm run test --workspace package-a

# Tip - this also works:
npm run test -w package-a
```

The `install` command also accepts the `--workspace` flag:

> Important! This was only introduced on npm@7.14.0

```bash
# Install `lodash` on `package-a`
npm install lodash --workspace package-a

# Install `tap` on `package-b` as a dev dependency
npm install tap --workspace package-b --save-dev

# Install `package-a` on `package-b`
npm install package-a --workspace package-b

# Install `eslint` in all packages
npm install eslint --workspaces
```

## npm Workspaces vs. Yarn Workspaces

Yarn is the second biggest package manager for JavaScript, so it might be fair to make a comparison.

Yarn Workspaces is around for much longer. It was launched somewhere around 2017. The `yarn workspaces` interface includes some extra tooling that npm is still catching up on. For example, an equivalent of `yarn workspace <workspace> add <dependency>` (adding a dependency to a workspace) <strike>is still in the works</strike> **Update: this feature was added on v7.14.0!**.

You can also see some minor differences in the CLI (e.g. `npm --workspaces` vs. `yarn workspaces`), but the overall concepts remain the same. Yarn is constantly cited as prior art in the [RFCs](https://github.com/npm/rfcs). I would be surprised to see big disparities between both CLIs.

Despite the differences, npm Workspaces is iterating fast. Since I first wrote this post in November 2020, many features have been released and RFCs have been accepted. It shouldn't take long for npm to reach feature parity with Yarn and other package managers.

## Using npm Workspaces today

_"I'm starting a new monorepo project today, can I use npm Workspaces?"_

You can, but keep reading.

If you are building a small monorepo project and need npm to just resolve your dependencies in a smart way, npm Workspaces alone should do the trick. You will get a similar experience as if you were using Yarn Workspaces.

But if you're working on a bigger project, with multiple packages and a complex dependency tree, you might want to combine npm with a tool like [Lerna](https://github.com/lerna/lerna).

Lerna has many [useful commands](https://lerna.js.org/#commands) to manage monorepos. Yarn has [stated before](https://classic.yarnpkg.com/en/docs/workspaces/#toc-how-does-it-compare-to-lerna) that the goal of Yarn Workspaces is to provide low-level primitives for tools such as Lerna to use, not to compete with them. While npm hasn't given a similar statement, I suspect they will follow the same approach as Yarn.

## What's next?

As I mentioned, npm Workspaces is still under active development and new features are being shipped fast. This is the second time I am updating this post, and I suspect it won't be the last.

If you want to keep up-to-date with Workspaces development, there are a few things you can do:

- Watch out for [accepted RFCs](https://github.com/npm/rfcs/tree/latest/accepted) to find out what's coming next;
- Subscribe to npm releases on [GitHub](https://github.com/npm/cli/releases).
- Follow me on [Twitter](https://twitter.com/ruanmartinelli), I'll post anything interesting I can find. :)



[Supabase](https://supabase.io) is an open source managed back-end platform. It's a direct alternative to Firebase, which is owned by Google and closed source.

Supabase comes with features such as authentication, object storage, and managed databases. Everything is built on top of open source tools, such as [PostgREST](https://postgrest.org/en/stable/) and [GoTrue](https://github.com/netlify/gotrue). If you want, you can also [self-host your own instance](https://supabase.io/docs/guides/self-hosting) of Supabase. As of today, Supabase is in public Beta.

In this tutorial you will learn how to build an a simple React application with authentication using [Create React App](https://github.com/facebook/create-react-app) (CRA). Supabase will serve as the back-end for the authentication part. The application will include sign in, sign up, and a private route than can only be accessed with valid credentials.

If you wanna jump straight to the code, you can check the [GitHub repository](https://github.com/ruanmartinelli/supabase-auth-react).

## Setting up Supabase

Visit [Supabase's website](https://supabase.com) to create a new account. Click the "Start your project" button and sign in with your GitHub account.

After signing in to the dashboard, hit the green "New Project" button. A modal like this should appear:

<img src="/images/supabase-auth-1.png" class="shadow-lg rounded-sm mb-4 mx-auto" />
<figcaption>1. Create a new project on Supabase</figcaption>

Choose a name for your project and a region close to you. It's required that you set a database password too, but we won't be using any on this tutorial.

It will take a few minutes for the project to be fully created. After it's done, go to _Settings_ > _API_ and copy the **URL** & **Public Anonymous API key**. Save the values somewhere, you will need them later on.

<img src="/images/supabase-auth-2.png" class="shadow-lg rounded-sm mb-4 mx-auto" />
<figcaption>2. Copy public credentials</figcaption>

> 💡 Tip: To make it easier to test the sign up flow, you can also disable email confirmations on _Authentication_ > _Settings_ > _Disable Email Confirmations_.

## Setting up the project

Create a new project using Create React App:

```bash
npx create-react-app supabase-auth-react
```

<figcaption>3. Create a new CRA project</figcaption>

I usually do some cleanup on new CRA projects before start developing. Here's how the project structure looks like after moving files around and deleting a few imports:

```bash
.
├── package.json
├── .env.local
└── src
    ├── components
    │   ├── App.js # Moved from src/
    │   ├── Dashboard.js
    │   ├── Login.js
    │   ├── PrivateRoute.js
    │   └── Signup.js
    ├── contexts
    │   └── Auth.js
    ├── index.js # Already created by CRA
    └── supabase.js
```

<figcaption>4. Overview of the folder structure after cleaning up CRA files</figcaption>

Feel free to recreate the same file structure. Don't worry about adding any code or trying to make sense of all the components just yet, we will go through everything later.

The `src/index.js` and `src/components/App.js` were already created by CRA. Here's how they look after cleaning up:

```jsx
// src/components/App.js

export function App() {
  return (
    <div>
      <h1>supabase-auth-react</h1>
    </div>
  )
}
```

<figcaption>5. Updated App.js component</figcaption>

```jsx
// src/index.js

import React from 'react'
import ReactDOM from 'react-dom'

import { App } from './components/App'

ReactDOM.render(
  <React.StrictMode>
    <App />
  </React.StrictMode>,
  document.getElementById('root')
)
```

<figcaption>6. Updated root component</figcaption>

## Setting up the Supabase Client Library

First, install the Supabase JavaScript client library on your project:

```bash
npm install @supabase/supabase-js
```

<figcaption>7. Install the Supabase client</figcaption>

Now add the code to initialize Supabase on `src/supabase.js`:

```jsx
// src/supabase.js

import { createClient } from '@supabase/supabase-js'

const supabase = createClient(
  process.env.REACT_APP_SUPABASE_URL,
  process.env.REACT_APP_SUPABASE_PUBLIC_KEY
)

export { supabase }
```

<figcaption>8. Supabase client initalization</figcaption>

In your `.env.local` file, add the URL and Public Anonymous API Key saved from the first step:

```bash
# .env.local

REACT_APP_SUPABASE_URL="https://YOUR_SUPABASE_URL.supabase.co"
REACT_APP_SUPABASE_PUBLIC_KEY="eyJKhbGciOisJIUzI1Nd2iIsInR5cCsI6..."
```

<figcaption>9. Set Supabase configuration variables</figcaption>

## Create authentication pages

Let's write the code for the `Signup`, `Login` and `Dashboard` components. These will be the three main pages of the application.

For now, let's just focus on writing a boilerplate for those components, without any authentication logic. Start by writing the `Signup` component:

```jsx
// src/components/Signup.js

import { useRef, useState } from 'react'

export function Signup() {
  const emailRef = useRef()
  const passwordRef = useRef()

  async function handleSubmit(e) {
    e.preventDefault()

    // @TODO: add sign up logic
  }

  return (
    <>
      <form onSubmit={handleSubmit}>
        <label htmlFor="input-email">Email</label>
        <input id="input-email" type="email" ref={emailRef} />

        <label htmlFor="input-password">Password</label>
        <input id="input-password" type="password" ref={passwordRef} />

        <br />

        <button type="submit">Sign up</button>
      </form>
    </>
  )
}
```

<figcaption>10. Initial code for the Signup component</figcaption>

The `Login` component looks very similar to `Signup`, with a few differences:

```jsx
// src/components/Login.js

import { useRef, useState } from 'react'

export function Login() {
  const emailRef = useRef()
  const passwordRef = useRef()

  async function handleSubmit(e) {
    e.preventDefault()

    // @TODO: add login logic
  }

  return (
    <>
      <form onSubmit={handleSubmit}>
        <label htmlFor="input-email">Email</label>
        <input id="input-email" type="email" ref={emailRef} />

        <label htmlFor="input-password">Password</label>
        <input id="input-password" type="password" ref={passwordRef} />

        <br />

        <button type="submit">Login</button>
      </form>
    </>
  )
}
```

<figcaption>11. Initial code for the Login component</figcaption>

The `Dashboard` is a simple component that displays a greeting message and offers the user to sign out:

```jsx
// src/components/Dashboard.js

export function Dashboard() {
  async function handleSignOut() {
    // @TODO: add sign out logic
  }

  return (
    <div>
      <p>Welcome!</p>
      <button onClick={handleSignOut}>Sign out</button>
    </div>
  )
}
```

<figcaption>12. Initial code for the Dashboard component</figcaption>

## Routing components with React Router

So far the components are isolated. There is no routing between the `Signup`, `Login` and `Dashboard` pages.

Let's work on that by adding [React Router](https://reactrouter.com/) to the project:

```bash
npm install react-router-dom
```

<figcaption>13. Install React Router</figcaption>

In `src/components/App.js`, declare a route for each of the components created before:

```jsx
// src/components/App.js

import { BrowserRouter as Router, Switch, Route } from 'react-router-dom'

import { Signup } from './Signup'
import { Login } from './Login'
import { Dashboard } from './Dashboard'

export function App() {
  return (
    <div>
      <h1>supabase-auth-react</h1>

      {/* Add routes here👇 */}
      <Router>
        <Switch>
          <Route exact path="/" component={Dashboard} />
          <Route path="/signup" component={Signup} />
          <Route path="/login" component={Login} />
        </Switch>
      </Router>
    </div>
  )
}
```

<figcaption>14. Declare routes in the App component</figcaption>

Let's also add links to navigate between the `Signup` and `Login` components:

```jsx
// src/components/Signup.js

import { Link } from 'react-router-dom'

export function Signup() {
  // ...

  return (
    <>
      <form onSubmit={handleSubmit}>{/* ... */}</form>

      <br />

      {/* Add this 👇 */}
      <p>
        Already have an account? <Link to="/login">Log In</Link>
      </p>
    </>
  )
}
```

<figcaption>15. Add link between Signup and Login pages</figcaption>

```jsx
// src/components/Login.js

import { Link } from 'react-router-dom'

export function Login() {
  // ...

  return (
    <>
      <form onSubmit={handleSubmit}>{/* ... */}</form>

      <br />

      {/* Add this 👇 */}
      <p>
        Don't have an account? <Link to="/signup">Sign Up</Link>
      </p>
    </>
  )
}
```

<figcaption>16. Add link between Signup and Login pages</figcaption>

You can test the navigation between components by running the project and clicking on the links or changing the URL in the navigation bar:

<img src="/images/supabase-auth-3.gif" class="shadow-lg rounded-sm mb-4 mx-auto" />

<figcaption>17. Demo of the navigation working</figcaption>

> Shameless plug: you may notice that the HTML looks a bit different, that's because I am using [axist](https://ruanmartinelli.github.io/axist/), a tiny drop-in CSS library that I built.

## Adding the authentication logic

To set up the authentication logic for the app, we're going to use React's Context API.

The Context API allows sharing data to a tree of components without explicitly passing props through every level of the tree. It's used to share data that is considered "global" (within that component tree).

You can read more about React Context in the [official documentation](https://reactjs.org/docs/context.html).

In this tutorial, we will use Context to share data associated with the user and the authentication operations. All this information will come from Supabase and will be needed on multiple parts of the app.

Let's start by adding code on `src/contexts/Auth.js`. First, create a Context object:

```jsx
// src/contexts/Auth.js

import React, { useContext, useState, useEffect } from 'react'
import { supabase } from '../supabase'

const AuthContext = React.createContext()

// ...
```

<figcaption>18. Create the Context object</figcaption>

Now, in the same file, create a Provider component called `AuthProvider`:

```jsx
// src/contexts/Auth.js

// ...

export function AuthProvider({ children }) {
  const [user, setUser] = useState()
  const [loading, setLoading] = useState(true)

  useEffect(() => {
    // Check active sessions and sets the user
    const session = supabase.auth.session()

    setUser(session?.user ?? null)
    setLoading(false)

    // Listen for changes on auth state (logged in, signed out, etc.)
    const { data: listener } = supabase.auth.onAuthStateChange(
      async (event, session) => {
        setUser(session?.user ?? null)
        setLoading(false)
      }
    )

    return () => {
      listener?.unsubscribe()
    }
  }, [])

  // Will be passed down to Signup, Login and Dashboard components
  const value = {
    signUp: (data) => supabase.auth.signUp(data),
    signIn: (data) => supabase.auth.signIn(data),
    signOut: () => supabase.auth.signOut(),
    user,
  }

  return (
    <AuthContext.Provider value={value}>
      {!loading && children}
    </AuthContext.Provider>
  )
}

// ...
```

<figcaption>19. Create the AuthProvider</figcaption>

The `AuthProvider` does three things:

- Calls `supabase.auth.session` to find out the current state of the user and update the user object.
- Listens for changes in the authentication state (user signed in, logged out, created a new account, etc.) by subscribing to `supabase.auth.onAuthStateChange` function.
- Prepares the object that will be shared by its children components (`value` prop). In this case, any components down the tree will have access to the `signUp`, `signIn`, `signOut` functions and the `user` object. They will be used by the `Signup`, `Login` and `Dashboard` components later on.

The `loading` state property will make sure the child components are not rendered before we know anything about the current authentication state of the user.

Now, create a `useAuth` function to help with accessing the context inside the children components:

```jsx
// src/contexts/Auth.js

// ...

export function useAuth() {
  return useContext(AuthContext)
}
```

<figcaption>20. Create the useAuth function</figcaption>

You can check how the `src/contexts/Auth.js` looks after all the changes on the [GitHub repository](https://github.com/ruanmartinelli/supabase-auth-react/blob/main/src/contexts/Auth.js).

Lastly, we need to wrap the `Signup`, `Login` and `Dashboard` components with the `AuthProvider`:

```jsx
// src/components/App.js

// ...

import { AuthProvider } from '../contexts/Auth'

export function App() {
  return (
    <div>
      <h1>supabase-auth-react</h1>
      <Router>
        {/* Wrap routes in the AuthProvider 👇 */}
        <AuthProvider>
          <Switch>
            <PrivateRoute exact path="/" component={Dashboard} />
            <Route path="/signup" component={Signup} />
            <Route path="/login" component={Login} />
          </Switch>
        </AuthProvider>
      </Router>
    </div>
  )
}
```

<figcaption>21. Add the AuthProvider to App.js</figcaption>

## Adding authentication to the components

Remember the `@TODO`s you left earlier in the components? Now it's time to, well, do them.

The functions needed by the components - `signUp`, `signIn` and `signOut` - as well as the `user` object are available through the Context. We can now get those values using the `useAuth` function.

Let's start by adding the sign up logic to the `Signup` component:

```jsx
// src/components/Signup.js

import { useRef, useState } from 'react'
import { useHistory, Link } from 'react-router-dom'

import { useAuth } from '../contexts/Auth'

export function Signup() {
  const emailRef = useRef()
  const passwordRef = useRef()

  // Get signUp function from the auth context
  const { signUp } = useAuth()

  const history = useHistory()

  async function handleSubmit(e) {
    e.preventDefault()

    // Get email and password input values
    const email = emailRef.current.value
    const password = passwordRef.current.value

    // Calls `signUp` function from the context
    const { error } = await signUp({ email, password })

    if (error) {
      alert('error signing in')
    } else {
      // Redirect user to Dashboard
      history.push('/')
    }
  }

  return (
    <>
      <form onSubmit={handleSubmit}>{/* ... */}</form>

      <br />

      <p>
        Already have an account? <Link to="/login">Log In</Link>
      </p>
    </>
  )
}
```

<figcaption>22. Add authentication logic to the Signup component</figcaption>

The `Login` component will look very similar. The main difference is that you will call `signIn` instead of `signUp`:

```jsx
// src/components/Login.js

import { useRef, useState } from 'react'
import { useHistory, Link } from 'react-router-dom'

import { useAuth } from '../contexts/Auth'

export function Login() {
  const emailRef = useRef()
  const passwordRef = useRef()

  // Get signUp function from the auth context
  const { signIn } = useAuth()

  const history = useHistory()

  async function handleSubmit(e) {
    e.preventDefault()

    // Get email and password input values
    const email = emailRef.current.value
    const password = passwordRef.current.value

    // Calls `signIn` function from the context
    const { error } = await signIn({ email, password })

    if (error) {
      alert('error signing in')
    } else {
      // Redirect user to Dashboard
      history.push('/')
    }
  }

  return (
    <>
      <form onSubmit={handleSubmit}>{/* ... */}</form>

      <br />

      <p>
        Don't have an account? <Link to="/signup">Sign Up</Link>
      </p>
    </>
  )
}
```

<figcaption>23. Add authentication logic to the Login component</figcaption>

Lastly, change the `Dashboard` so the user can sign out of the application. You can also display some basic information together with the greeting message, such as the user ID:

```jsx
// src/components/Dashboard.js

import { useHistory } from 'react-router'
import { useAuth } from '../contexts/Auth'

export function Dashboard() {
  // Get current user and signOut function from context
  const { user, signOut } = useAuth()

  const history = useHistory()

  async function handleSignOut() {
    // Ends user session
    await signOut()

    // Redirects the user to Login page
    history.push('/login')
  }

  return (
    <div>
      {/* Change it to display the user ID too 👇*/}
      <p>Welcome, {user?.id}!</p>
      <button onClick={handleSignOut}>Sign out</button>
    </div>
  )
}
```

<figcaption>24. Add authentication logic to the Dashboard component</figcaption>

## Protecting routes

Currently, all the authentication logic is in place, but the `Dashboard` component remains publicly accessible. Anyone who happens to fall on _locahost:3000_ would see a broken version of the dashboard.

Let's fix that by protecting the route. If a user that is not authenticated tries to access it, they will be redirected to the login page.

Start by creating a `PrivateRoute` component:

```jsx
// src/components/PrivateRoute.js

import React from 'react'
import { Route, Redirect } from 'react-router-dom'

import { useAuth } from '../contexts/Auth'

export function PrivateRoute({ component: Component, ...rest }) {
  const { user } = useAuth()

  return (
    <Route
      {...rest}
      render={(props) => {
        // Renders the page only if `user` is present (user is authenticated)
        // Otherwise, redirect to the login page
        return user ? <Component {...props} /> : <Redirect to="/login" />
      }}
    ></Route>
  )
}
```

<figcaption>25. Create the PrivateRoute component</figcaption>

The `PrivateRoute` wraps the `Route` component from React Router and passes down the props to it. It only renders the page if the `user` object is not null (the user is authenticated).

If the `user`object is empty, a redirect to the login page will be made by `Redirect` component from React Router.

Finally, update the dashboard route in the `App` component to use a `PrivateRoute` instead:

```diff
// src/components/App.js

- <Route exact path="/" component={Dashboard} />
+ <PrivateRoute exact path="/" component={Dashboard} />
```

<figcaption>26. Update the Dashboard component to use PrivateRoute</figcaption>

Done! The dashboard is only available for authenticated users.

## Final result

This is how the final version of the application should look like:

<img src="/images/supabase-auth-4.gif" class="shadow-lg rounded-sm mb-4 mx-auto" />

<figcaption>26. Final demo of the application</figcaption>

You can see the sign up, login, and sign out working. The dashboard page is also protected, attempting to access it by changing the URL redirects the user to the login page. Notice the user ID showing there too.

## Going further

There are a few things that we could add for a more complete authentication flow:

**Password reset**. I intentionally left it out for simplicity, but [Supabase supports password reset](https://supabase.io/docs/reference/javascript/reset-password-email) through their API. It does all the heavy-lifting for you, including sending the email to the user with the reset instructions.

**Authentication providers**. When logging in, you can also specify different authentication providers like Google, Facebook and GitHub. [Check out the docs](https://supabase.io/docs/reference/javascript/auth-signin).

**User operations**. You can also add metatada to the authenticated user using the [update function](https://supabase.io/docs/reference/javascript/auth-update). With this, you could build for example a basic user profile page.

Thanks for reading this far!



[Cloud Run](https://cloud.google.com/run) is a serverless platform from [Google Cloud](https://cloud.google.com/) to deploy and run containers. It's fully managed, autoscallable, and has a generous [free tier](https://cloud.google.com/run/pricing). Cloud Run can be used to serve Restful web APIs, WebSocket applications, or microservices connected by gRPC. It also integrates well with other Google Cloud solutions such as Cloud Tasks, Cloud Scheduler, and Pub/Sub.

[Terraform](https://www.terraform.io/) is a popular open-source tool for running infrastructure as code. Terraform lets you manage and deploy infrastructure from multiple providers, one of them being Google Cloud. If this is your first time reading about Terraform, you might wanna check [this introduction](/posts/introduction-to-terraform) first.

In this article, you will see how to deploy a Cloud Run service to Google Cloud using Terraform. All the infrastructure will be written using HCL, the native syntax for Terraform's language. By the end of the tutorial, you should have a service up and running on Cloud Run and a URL to access it.

## Prerequisites

To follow this tutorial you will need:

- Terraform CLI. I recommend using the latest version, currently v0.14. Instructions to download and install Terraform can be found [here](https://www.terraform.io/downloads.html).
- Google Cloud SDK. The most recent version should also work well for this tutorial. Installation instructions [here](https://cloud.google.com/sdk/docs/install).
- A Google Cloud account. If you don't have one, create it [here](https://console.cloud.google.com/getting-started).

## Initial setup

Start by authenticating the SDK to Google Cloud:

```bash
# Authenticate to Google Cloud
gcloud auth application-default login
```

Follow the web flow to obtain the access credentials. By the end of this step, you will be able to execute commands on the SDK similar to a service account.

Create a new project where your Cloud Run service will be deployed. Replace `PROJECT_ID` and `PROJECT_NAME` with the desired values:

```bash
gcloud projects create "PROJECT_ID" --name="PROJECT_NAME"
```

Create a folder with a HCL file inside:

```bash
mkdir my-service && cd my-service

touch main.tf
```

For simplicity, all the Terraform code in the next steps will be added to `main.tf`.

## Creating your first service

Begin by adding the requirements for Terraform and the Google provider on `main.tf`:

```ruby
# main.tf

terraform {
  required_version = ">= 0.14"

  required_providers {
    # Cloud Run support was added on 3.3.0
    google = ">= 3.3"
  }
}
```

This will require the Terraform version to be the latest and the Google provider to be at least on version `3.3` - when Cloud Run support was added.

Now add the Google provider configuration. Replace `PROJECT_ID` with the value from the previous step:

```ruby
# main.tf

provider "google" {
  # Replace `PROJECT_ID` with your project
  project = "PROJECT_ID"
}
```

The Cloud Run API doesn't come enabled on projects by default. Add the following resource to enable it:

```ruby
# main.tf

# Enables the Cloud Run API
resource "google_project_service" "run_api" {
  service = "run.googleapis.com"

  disable_on_destroy = true
}
```

Now create the Cloud Run service in the `us-central1` region:

```ruby
# main.tf

# Create the Cloud Run service
resource "google_cloud_run_service" "run_service" {
  name = "app"
  location = "us-central1"

  template {
    spec {
      containers {
        image = "gcr.io/google-samples/hello-app:1.0"
      }
    }
  }

  traffic {
    percent         = 100
    latest_revision = true
  }

  # Waits for the Cloud Run API to be enabled
  depends_on = [google_project_service.run_api]
}
```

Let's stop for a while and check what the code above is doing:

- `name`: the name of your service. It will be displayed in the public URL.
- `location`: the region where your service will run. See all the options [here](https://cloud.google.com/run/docs/locations).
- `image`: The Docker image that will be used to create the container. Cloud Run has direct support for images from the [Container Registry](https://cloud.google.com/container-registry) and [Artifact Registry](https://cloud.google.com/artifact-registry).
- `traffic`: controls the traffic for this revision. The `percent` property indicates how much traffic will be redirected to this revision. `latest_revision` specifies that this traffic configuration needs to be used for the latest revision.
- `depends_on`: waits for a resource to be ready, in this case, the Cloud Run API.

## Invoking the service

By default, Cloud Run services are private and secured by [IAM](https://cloud.google.com/iam). To access them, you would need valid credentials with at least the _Cloud Run Invoker_ permission set.

Let's change that and make the service publicly available through an HTTP endpoint.

> There are other ways than HTTP requests to trigger a service. For example, they can be accessed by gRPC requests, WebSockets, and other Google Cloud products like Cloud Scheduler.

Add the following code on `main.tf` to expose your service:

```ruby
# main.tf

# Allow unauthenticated users to invoke the service
resource "google_cloud_run_service_iam_member" "run_all_users" {
  service  = google_cloud_run_service.run_service.name
  location = google_cloud_run_service.run_service.location
  role     = "roles/run.invoker"
  member   = "allUsers"
}
```

The resource above is adding the permission to invoke the service to anyone on the internet. The `allUsers` identifier is a special value that represents authenticated and unauthenticated users.

To display the service URL in the Terraform command output, add this output to the configuration:

```ruby
# main.tf

# Display the service URL
output "service_url" {
  value = google_cloud_run_service.run_service.status[0].url
}
```

## Deploying the infrastructure

At this point, you have all it takes to deploy the infrastructure to Google Cloud using Terraform.

Start by initializing the configuration. Run the following command in your terminal:

```bash
# Initialize Terraform configuration
terraform init
```

Run `terraform plan` to verify the changes that will be applied:

```bash
# Plan the changes
terraform plan
```

If everything is correct, you will see that **3 resources** will be created and the service URL will be displayed.

```bash
Plan: 3 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + service_url = (known after apply)
```

Run `terraform apply` to apply all the changes:

```bash
# Apply all the changes
terraform apply
```

If everything goes well, you will see this at the end of the output:

```bash
Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

Outputs:

service_url = https://app-cvmew7554a-uc.a.run.app
```

You can check if the service is running using `curl`:

```bash
curl https://app-cvmew7554a-uc.a.run.app

Hello, world!
Version: 1.0.0
Hostname: localhost
```

## Updating the service

Terraform can be used not just to push your initial infrastructure to Cloud Run, but also to update it.

Cloud Run works with _revisions_. When a configuration is changed or a new image is added, a new revision is created as a result. You can then redirect all the traffic to the new revision and start serving your updated application.

To update your service, simply change the value in the `image` property and pass it a new image:

```ruby
# main.tf

resource "google_cloud_run_service" "run_service" {
  name = "app"

  # ...

  template {
    spec {
      containers {
        # Change `hello-app:1.0` to `hello-app:2.0` 👇
        image = "gcr.io/google-samples/hello-app:2.0"
      }
    }
  }

  # ...
}
```

With this, Terraform will create a new revision on Cloud Run. The previous revision is preserved, but because of the `traffic` options defined previously, it won't recieve any traffic.

Run `terraform apply` to deploy the changes:

```bash
# Apply all the changes
terraform apply
```

You can check that the new image is live using `curl`:

```bash
curl https://app-cvmew7554a-uc.a.run.app

Hello, world!
Version: 2.0.0 # Now serving 2.0.0
Hostname: localhost
```

## Cleaning up

To delete all resources created with Terraform, run the following command and confirm the prompt:

```bash
# Destroy all the infrastructure created by Terraform
terraform destroy
```

This will disable the Cloud Run API, delete the Cloud Run service and its permissions.

The project was created using the `gcloud` CLI tool, so you will need to delete it manually. For that, you can run:

```bash
# Remove the Google Cloud project
gcloud projects delete PROJECT_ID
```

--

Thanks for reading this far!



<img src="/images/container.jpg" class="shadow-lg rounded-sm mx-auto pt-5 mb-4" />

<figcaption>Did you know? All articles about Docker need a picture of shipping container</figcaption>

In this post, you will see how to use Docker Compose to set up a local development environment for your project or application.

If you want to follow along, you will need to install Docker Compose first. The steps can be found [here](https://docs.docker.com/compose/install/). All the code was executed using the `1.27.4` version.

The examples listed are in the context of APIs and server side applications. If you are building a different type or application, fear not! The same concepts and ideas can be applied to other types of development.

## The problem

When building applications, it’s often a requirement to integrate with multiple tools, services, or external software in general.

For example, a Node.js application can connect to a database, a Redis instance, the Firebase API, or even a second application (thinking of a microservices scenario).

The more _dependency services_ you have, the harder it gets to configure and manage a development workflow on your computer. To list a few issues:

- **Complex setup**. The "getting started" section of your project can become an extensive list of software to install. Onboarding new developers takes more time as the project grows.
- **Version management**. If a project uses `v1` or a dependency service and another one is on `v3` for example, it can be hard to work on both projects simultaneously and switch between versions.
- **Complex initialization**. In a microservices scenario for example, running all services can require a lot of coordination. It’s easy to get lost between multiple terminal windows and commands to run.

## Meet Docker Compose

Docker Compose is a command-line tool for running applications with multiple Docker containers.

Using the YAML syntax, you can declare a list of services and run them with a single command — `docker-compose up`. Services can be created from a local build or a Docker image from a remote repository.

Docker Compose solves all the problems listed above. It works in a declarative way — you specify the exact environment needed for your project, and it will take care of turning it into reality. The environment created by Docker Compose is completely isolated and can be torn down with a single command.

Below you will see how to use Docker Compose on different scenarios to build reproducible development environments.

## Example #1: Node.js application with a Postgres database

To get started, all you need is a `docker-compose.yaml` file. The only required field is the `version` field.

```diff
# docker-compose.yaml

version: '3.8'

# ...more code to come...
```

<figcaption>1. Basic Docker Compose file, without any services.</figcaption>

This should be enough to make the `docker-compose up` command work (but it's not doing anything yet).

In the same file, create a service from the Postgres image:

```diff
# docker-compose.yaml

version: '3.8'

+ services:
+  my-database:
+    image: postgres:11.4-alpine
+    environment:
+      POSTGRES_USER: root
+      POSTGRES_PASSWORD: password
+      POSTGRES_DB: db
+    ports:
+      - 5432:5432
```

<figcaption>2. Docker Compose file with a Postgres database service.</figcaption>

In the code above, `my-database` is the name of the service being created.

The values listed in `environment` are related to the Postgres image. They specify how the database should be configured. See other options [here](https://hub.docker.com/_/postgres).

The field `ports` opens up the port `5432` of the container to the host — your computer. This makes the database accessible on `localhost:5432`.

Run the following command in your terminal to create and start the service:

```bash
$ docker-compose up --detach
```

Notice I am running `docker-compose up` with an additional `--detach` flag. This will run the containers in the background and give you back the control of your terminal.

The output will look like this:

```bash
Creating networkapp_default with the default driver
Creating app_my-database_1 ... done
```

This means that the Postgres database is running on your machine. Thanks to the magic of containers, this will work even if you never installed Postgres on your machine! How cool is that?

You can test the setup above with a simple Node.js application:

```js
// index.js

const knex = require('knex') // `knex` and `pg` needs to be installed from npm!

;(async () => {
  const connection = knex({
    client: 'pg',
    connection: 'postgresql://root:password@localhost:5432/db',
  })

  const result = await connection.raw('select 1+1 as sum')

  console.log(`The result is: ${result.rows[0].sum}`)
})()
```

<figcaption>3. Simple Node.js script to test the connection to the database created with Docker Compose.</figcaption>

To stop and remove the containers created, run:

```bash
$ docker-compose down
```

## Example #2: Postgres and Redis

Docker Compose can also run multiple services at once. Let's extend the previous example with a Redis database:

```diff
# docker-compose.yaml
version: '3.8'

services:

  my-database:
    image: postgres:11.4-alpine
    environment:
      POSTGRES_USER: root
      POSTGRES_PASSWORD: password
      POSTGRES_DB: db
    ports:
      - 5432:5432

+  redis:
+    image: redis:6.0.10
+    ports:
+      - 6379:6379
```

<figcaption>4. Docker Compose file with a Redis and Postgres database.</figcaption>

The code above follows the same logic from the first example. There is no need to pass any values on `environment` since that was specific to the Postgres image.

Now, when running `docker-compose up`, it will start both the Redis and the Postgres instance in one go. You can connect to the Redis database using `localhost:6739`.

## Example #3: Local builds

Services can be created from a local Dockerfile instead of external images.

This is useful if you want to start applications alongside your dependency services (or just applications, without any dependency). For example, you could have a database _and_ a server both starting with the `docker-compose up` command.

The YAML file looks like this:

```diff
# docker-compose.yaml
version: '3.8'

services:
  my-api:
    build:
      context: .
      dockerfile: Dockerfile
    environment:
      DATABASE_URL: postgresql://root:password@localhost:5432/db
    depends_on: [my-database]
    ports:
      - '8080:8080'

  my-database:
    image: postgres:11.4-alpine
    environment:
      POSTGRES_USER: root
      POSTGRES_PASSWORD: password
      POSTGRES_DB: db
    ports:
      - 5432:5432
```

<figcaption>5. Docker Compose file with a Postgres database and a local application.</figcaption>

The syntax is slightly different for local builds. Instead of `image`, we are using the `build` property. It lets you specify the path or a directory where your Dockerfile is. By default, it looks for a file named `Dockerfile`, but this behaviour can be changed using the `dockerfile` property.

In the `my-api` service, the `DATABASE_URL` environment variable matches the configuration passed to the database service. The property `depends_on` tells Docker Compose that `my-api` should only be started after `my-database` is started.

> About `depends_on`: there is a difference between _started_ and _ready_. The property `depends_on` only waits for the first. If you need to wait for a service to be _ready_, there are [other ways](https://docs.docker.com/compose/startup-order/) to control the startup order with Docker Compose.

But there is a problem with this approach. Because of how Docker works, any changes to your code require the container to be rebuilt. Even with a good image caching strategy, this process can slow down development a lot.

To solve this, you can mount the path of the host machine to the container using the `volumes` property:

```diff
# docker-compose.yaml
version: "3.8"

services:
  my-api:
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - "8080:8080"
+    volumes:
+      - .:/app

# ...rest of the code
```

<figcaption>6. Using `volumes` to mount your local code in the container.</figcaption>

This will mount the code in the current repository to `/app` inside the container. Any changes will be available on the fly inside the container, without having to rebuild the image.

## Conclusion

The biggest advantage of integrating Docker Compose into your workflow is **reproducibility**. You will get the same local environment whether you run it on your computer, a coworker's laptop, or a CI/CD virtual machine.

Using Docker Compose for local development workflows is listed as one of the use cases on [Docker's webpage](https://docs.docker.com/compose/#development-environments), but I suspect it's still not a widely known technique. It requires some basic understanding of containers and Docker, which are not straightforward concepts. Hopefully I was able to clarify things a bit!

Thanks for reading! Follow me on [Twitter](https://twitter.com/ruanmartinelli) for more updates!



This year, [Fastify](https://fastify.io) became my go-to framework for building Node.js APIs.

If the word sounds new to you, Fastify is a **web framework for Node.js**. It is used to build APIs and services the same way Express does.

Fastify comes with great features that really speed up the process of making applications. Among those features, my favorite one is the fact that the framework is _schema-based_ (I'll explain).

In this post, I will share a few tricks on how you can leverage Fastify's schema capabilities to build APIs at a fast clip.

## Schemas

Fastify adopts the [JSON Schema](https://json-schema.org/) format on its core. Many of its features and libraries are built around the popular standard. [Ajv](https://ajv.js.org/), a library to compile and validate JSON Schemas, is a direct dependency of the framework.

By adopting JSON Schema, Fastify opens doors to an entire ecosystem of tools built around it. Below, let's see how to combine all these tools and libraries together with the framework.

### 1. Validation

One of the ways Fastify uses JSON Schema is to validate data coming from clients. It lets you add _input schemas_ to your routes. For example:

```js
// Schema for `POST /movie` body
const PostMovieBody = {
  type: 'object',
  properties: {
    title: { type: 'string' },
    releaseYear: { type: 'integer', minimum: 1878 },
  },
}

app.post('/movie', {
  schema: {
    // Refence the schema here
    body: PostMovieBody,
  },
  handler: createMovie,
})
```

<figcaption>1. Using input schemas to validate data in the router.</figcaption>

In this example, any incoming data to `POST /movie` that doesn't conform to the `PostMovieBody` schema will throw a validation error.

This way, we are making sure that the handler function doesn't process any invalid or unexpected payloads.

Invalid objects will result in a validation error that looks like this:

```bash
POST /movie
{ releaseYear: 2020 } # The `title` parameter was not sent

{
  "statusCode": 400,
  "error": "Bad Request",
  "message": "body should have required property 'title'"
}
```

<figcaption>2. Example of Fastify's default error response.</figcaption>

> Note: the content on `message` comes from Ajv.

### 2. Serialization

Serialization is the process of converting an object to a format that can be transferred over a network.

With Fastify, you can also define _output schemas_ for JSON payloads. When you do so, any data returned to clients will be serialized _and_ validated according to that definition.

More specifically, defining output schemas helps you in two ways:

- Fastify serializes the data with [fast-json-stringify](https://github.com/fastify/fast-json-stringify). In many cases, it is faster than `JSON.stringify`.
- Ajv validates the response. This will prevent sensitive fields from being exposed.

When declaring output schemas in your routes, each possible status code accepts a definition. For example, you can have schemas defined for `200` and `204` responses.

> Tip: to use the same schema for a family of status codes, you can use the `'2xx'` notation.

Here's how to define an output schema to responses with a `200` status code:

```js
// Generic `Movie` schema
const Movie = {
  type: 'object',
  properties: {
    id: { type: 'integer' },
    title: { type: 'string' },
    releaseYear: { type: 'integer', minimum: 1878 },
  },
}

app.post('/movie', {
  schema: {
    response: {
      // Payloads will be serialized according to the `Movie` schema
      200: Movie,
    },
  },
  // ...
})
```

<figcaption>3. Using output schemas to serialize and validate data in the router.</figcaption>

In this example, any object returned by the handler that doesn't match the `Movie` schema will result in an error. By default, the client receives a `400` response - similar to the example #2.

### 3. Documentation

Documentation is an essential piece in any REST API.

There are many ways to document your application. One of them is manually, where you write routes and definitions by hand in a common format like YAML or JSON.

You can already guess this approach has many problems: outdated schemas, inconsistent validations, type discrepancies, etc.

Another approach is automating your documentation. A tool will automatically generate all the routes and definitions based on an existing schema.

One popular specification for writing documentation is [Swagger](https://swagger.io/). Thanks to the official [fastify-swagger](https://github.com/fastify/fastify-swagger) plugin, you can transform your existing JSON Schema definitions into Swagger ones and expose a beautiful documentation page in a flick.

Adding `fastify-swagger` to a Fastify application should be straightforward:

```js
const fastify = require('fastify')()

// Register the plugin before your routes
fastify.register(require('fastify-swagger'), {
  exposeRoute: true,
  routePrefix: '/documentation',
  swagger: {
    info: { title: 'movie-api' },
    // Add more options to get a nicer page ✨
  },
})

// Declare your routes here...
```

<figcaption>4. Registering the `fastify-swagger` plugin.</figcaption>

Now, when you start your Fastify application and navigate to `/documentation` in a browser this page will pop up:

<img src="/images/swagger-fastify.jpg" class="shadow-lg rounded-sm mb-4" />

<figcaption>5. The documentation page generated by `fastify-swagger`.</figcaption>

> Note: The more metadata you add to your routes, the more complete your page will look. Check out the [route options documentation](https://www.fastify.io/docs/latest/Routes/#options).

### 4. Mocking

When testing services or endpoints, many times you will need to provide a fake or simulated input. These inputs are called _mock objects_. They replicate the structure and behavior of real objects.

You can create mock objects dynamically with the schemas you already have by using [json-schema-faker](https://github.com/json-schema-faker/json-schema-faker). The library converts existing JSON Schemas into dummy objects that you can use in your tests. Let's see an example.

First, create a helper function (just a wrapper for `json-schema-faker`):

```js
const jsf = require('json-schema-faker')

/**
 * Creates an object from a JSON Schema. Example:
 * schemaToObject(Movie)
 * => { id: 823, title: 'unicorn', releaseYear: 1942 }
 */
function schemaToObject(schema) {
  return jsf.resolve(schema)
}
```

<figcaption>6. Creating the `schemaToObject` helper function. </figcaption>

The `schemaToObject` function does exactly what the name says: given a JSON Schema definition, it returns a matching mock object.

Now let's put it to use. You can call this function whenever you need to create fake objects for your tests. For example, when sending requests to routes:

```js
it('should create a movie', async () =
	// Create a mock object for the request
	const payload = await schemaToObject(PostMovieBody)

	// Calls the POST /movie
	const response = await request.post('/movie', payload)

	expect(response.status).toBe(200)
})
```

<figcaption>7. Using `schemaToObject` to generate fake payloads.</figcaption>

In this example, we are creating a mock object, `POST`-ing it to the `POST /movie` route, and checking the status code.

The `schemaToObject` function gives you a nice and clean way to test the "happy path" in your tests (when everything meets the expectations).

### 5. Jest

[Jest](https://jestjs.io/) is a testing framework for JavaScript. One of its features is the possibility to create or import custom matchers.

One of these matchers is [jest-json-schema](https://github.com/americanexpress/jest-json-schema#readme). This package adds a new assertion to Jest: `toMatchSchema`. It lets you validate an object against an existing JSON Schema definition - it's like Ajv was integrated to Jest.

> Note: If you are using another testing framework, there is likely an equivalent to `jest-json-schema` for it.

Instead of manually asserting the values of each property in an object like this:

```js
it('should create a movie', async () => {
  // ...
  expect(response.title).toBeString()
  expect(response.releaseYear).toBePositive()
})
```

<figcaption>8. Manually asserting each property in an object.</figcaption>

You can simplify things using `toMatchSchema`:

```js
import { matchers } from 'jest-json-schema'
import { Movie } from './schemas'

expect.extend(matchers)

it('should create a movie', async () => {
  // ...
  expect(response).toMatchSchema(Movie)
})
```

<figcaption>9. Asserting all properties using a schema.</figcaption>

Notice I am using the `Movie` schema defined in example #3.

Of course, this is just simplifying type-checking in your tests. There are still other aspects of your code that need to be tested. Still, based on how easy it is to implement, I believe it's a good addition.

## Putting it all together

Let's do a quick recap.

In examples #1 and #3, we have declared two schemas using the JSON Schema format - `PostMovieBody` and `Movie`. These schemas are used for:

1. Validating objects sent to the route.
2. Serializing _and_ validating objects returned to the clients.
3. Generating documentation.
4. Creating mock objects.
5. Asserting objects on tests.

Now here's the fun part!

Suppose you need to start tracking a new property in your movie objects. For example, you need to save and display _the movie poster URL_. Let's name the new field `posterUrl`.

If you were not using a schema-based framework, you would need to go through all your code and update the existing objects to include the new property. This is far from ideal. The chances of missing an assertion in your tests or forgetting to update the documentation are high.

But thanks to the magic of schemas, this process is a breeze. **Your definitions are your source of truth**. Anything based on the schemas will change once the schema changes.

So, now let's see how we can add the `posterUrl` property.

The first step is to change the input schema (`PostMovieBody`) to include the new property:

```diff
const PostMovieBody = {
  type: 'object',
  properties: {
    title: { type: 'string' },
    releaseYear: { type: 'integer', minimum: 1878 },
+   posterUrl: { type: 'string' }
  }
}
```

<figcaption>10. Adding `posterUrl` to the input schema.</figcaption>

Now, since `posterUrl` must also be serialized and returned to the client, we also add it to the output schema (`Movie`):

```diff
const Movie = {
  type: 'object',
  properties: {
    id: { type: 'integer' },
    title: { type: 'string' },
    releaseYear: { type: 'integer', minimum: 1878 }
+   posterUrl: { type: 'string' }
  }
}
```

<figcaption>11. Adding `posterUrl` to the output schema.</figcaption>

And that's pretty much it!

Here is what will happen once you restart your server:

1. Fastify will start checking for `posterUrl` in the `POST /movie` route.
2. The Swagger file will be updated. The `posterUrl ` property will start showing on the documentation page.
3. Mock objects in your tests will start being generated with a string value for `posterUrl`.
4. Tests using the `toMatchSchema` matcher will start checking for the `posterUrl` property.

...and you got all that just by changing two lines in your code. How cool is that?

### Honorable mention: fluent-schema

If you are used to libraries like [Joi](https://github.com/sideway/joi) or [Yup](https://github.com/jquense/yup), writing schemas using raw JavaScript objects might feel like a step back.

To overcome that feeling, you can use [fluent-schema](https://github.com/fastify/fluent-schema). It gives you the same compact and programmable interface present in other tools.

For example, we could rewrite the `Movie` schema in example #3 using `fluent-schema`:

```js
const S = require('fluent-schema')

const Movie = const schema = S.object()
  .prop('title', S.string())
  .prop('releaseYear', S.number().minimum(1878))
```

<figcaption>12. Example of the `Movie` schema defined with fluent-schema.</figcaption>

Looks neat, huh?

And that's a wrap! I hope you have enjoyed it. Stay tuned for more Fastify articles. ✌️



In this tutorial you are going to deploy a simple Node.js API to Google Cloud Functions using Terraform.

**Cloud Functions** is a compute solution from [Google Cloud Platform (GCP)](https://cloud.google.com/) . It provides functions as a service (FaaS), which is a way to run your code "on-demand", without managing any servers.

For deployment we choose **Terraform**, a command-line tool to build and deploy infrastructure using code. Terraform will help create a predictable and reproducible environment to run your code.

Since it’s not the main focus of this tutorial, we will go with a super simple **Node.js API** using Fastify. Feel free to use any other languages supported by Cloud Functions on this part.

After finished, you will have an up and running API with an URL that you can make requests to.

## Prerequisites

To follow this guide you will need:

- Terraform 0.13 or later. You can find the installation instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli);
- Google Cloud SDK. Any recent version should be fine. Installation instructions [here](https://cloud.google.com/sdk/docs/install);
- Node.js 12 or later. If you don’t have Node.js installed, I recommend using [nvm](https://github.com/nvm-sh/nvm) for that.

## 1. Setting up the GCP account

If you are using the Google Cloud SDK for the first time, you will need to authenticate with your Google Account. You can run the following command:

```bash
# Authenticate with GCP
gcloud auth application-default login
```

Now create the project on GCP:

```bash
# Create a GCP project
gcloud projects create PROJECT_ID --name="My App"
```

> Note: replace `PROJECT_ID` with an unique project identifier (e.g. "my-app-2847162").

Set the project you just created as the default one. This will make it easier to run the subsequent commands.

```bash
# Set the project as the default one
gcloud config set project PROJECT_ID
```

Many features on GCP require a billing account linked to the project, Cloud functions is one of them. For this step, you will need to visit the dashboard:

[Create a billing account on GCP](https://cloud.google.com/billing/docs/how-to/manage-billing-account).

> Note: even though Google asks for your credit card, **this tutorial should not cost any money to run**. The first 2 million invocations of a Cloud Function are free. You will also learn how to shutdown and destroy all the resources created here.

After setting up billing, the account will be listed when you run the following command:

```bash
# List billing accounts available
gcloud beta billing accounts list
```

The output will look something like this:

```bash
ACCOUNT_ID            NAME         OPEN  MASTER_ACCOUNT_ID
87PPT3-QECCDL-9OLSSQ  my-account   True
```

Copy the account ID and run the following command to link the billing account to your project:

```bash
# Link a billing account to project
gcloud beta billing projects link PROJECT_ID --billing-account=BILLING_ACCOUNT_ID
```

Now you are going to structure the project.

## 2. Structuring the project

Create the files listed below so your repository looks like this:

```haskell
.
├── terraform
│ 	├── modules
│   │   └── function
│   │       ├── main.tf
│   │       ├── outputs.tf
│   │       └── variables.tf
│ 	├── main.tf
│ 	├── backend.tf
│ 	├── outputs.tf
│ 	└── variables.tf
└── src
     └── index.js
```

Don’t worry about adding any content now. We will do that on the next step.

> Note: This is an opinionated, but very common structure for Terraform projects. Terraform doesn’t require your files to be in a certain disposition or have a specific name. Although not recommended, you _can_ build an entire system within a single `main.tf` file.

The `terraform/` folder contain files related to Terraform.

The `src` folder hosts the code for the Node.js API. Remember, the API code will be very simple, only a single `index.js` file is enough.

## 3. Writing the API

Let’s write the API using Fastify.

If you are following this tutorial with a different language, you can add your custom code at this point.

> Note: If you never heard about Fastify before, it has a very similar API to other Node.js frameworks like Express.

First, initialize the project with npm and install `fastify` as a dependency:

```bash
# Initialize project
npm init

# Install fastify
npm install fastify
```

Add this content to the `src/index.js` file:

```js
// src/index.js

const fastify = require('fastify')
const app = fastify({ logger: true })

app.get('/', async (req, res) => {
  return { works: true }
})

exports.app = async (req, res) => {
  await app.ready()
  app.server.emit('request', req, res)
}
```

Update the entry point for your code in the `package.json` file:

```diff
// package.json

{
-   "main": "index.js",
+   "main": "src/index.js"
  // ...
}
```

This will tell Cloud Functions where your API is located. Now let’s jump to the `terraform/` folder and start writing the infrastructure code.

## 4. Writing the infrastructure code

At this point, you already have all the files and folders created inside your `terraform/` folder.

Before start adding code to them, let’s take a look at each file’s responsibility:

- `backend.tf`. Declares which [Terraform backend](https://www.terraform.io/docs/backends/index.html) you will use.
- `main.tf`. Where you will write the logic for creating resources or invoking modules.
- `variables.tf`. Lists the variables and their values that will be used on `main.tf`.
- `outputs.tf`. Lists the values your Terraform code will return.
- `modules/`. A place for your [Terraform modules](https://www.terraform.io/docs/modules/index.html). In this case, there will be just one named `function`.

> Note: The name `function` was chosen for the module because it creates a Cloud Function. Feel free to use a different name.

> Note: If you are not familiar with modules in Terraform, think of them as a function or component. It’s something that you write once and then can reuse on other parts of your code.

Begin by declaring which Terraform backend you want to use - where you want to store your Terraform state files.

Let's choose the "local" backend for now, meaning the state files will be stored on your local repository.

```ruby
# terraform/backend.tf

terraform {
  backend "local" {}
}
```

Now add the following variables to your `terraform/variables.tf` file:

```ruby
# terraform/variables.tf

variable "project" {
	default = "PROJECT_ID"
}
variable "region" {
	default = "us-central1" # Choose a region
}
```

In `terraform/main.tf`, declare the provider Terraform will connect to. In your case, the Google Cloud Platform provider (named `"google"`) .

The Google provider has two required parameters, _project_ and _region_. We can reference the values declared on the step above by accessing the properties in the `var` object.

```ruby
# terraform/main.tf

provider "google" {
  project = var.project
  region  = var.region
}

# ⚠️ More code here soon
```

You will go back to this file soon to add more configuration.

### Creating the `function` module

In order to create a Cloud Function on GCP you will need to combine a few resources together:

- A storage bucket, to store the code that will be executed by the function
- The function itself, to run the code you wrote
- An IAM policy, to allow users to invoke the function

These resources will be grouped in the Terraform module you are about to create.

This will make it easier if you want to deploy a second environment (e.g. development & staging) or create multiple functions - you can just invoke the module again with different parameters.

On `variables.tf` inside `terraform/modules/function`, add the arguments needed by the module. All arguments are required, so don’t add default values.

```ruby
# terraform/modules/function/variables

variable "project" {}
variable "function_name" {}
variable "function_entry_point" {}
```

Proceeding to `terraform/modules/function/main.tf`, add the logic to create the function and all resources needed.

> ⚠️ This file is a bit dense. Follow through the comments in it to get a better idea of what’s happening.

```ruby
# terraform/modules/function/main.tf

locals {
  timestamp = formatdate("YYMMDDhhmmss", timestamp())
	root_dir = abspath("../")
}

# Compress source code
data "archive_file" "source" {
  type        = "zip"
  source_dir  = local.root_dir
  output_path = "/tmp/function-${local.timestamp}.zip"
}

# Create bucket that will host the source code
resource "google_storage_bucket" "bucket" {
  name = "${var.project}-function"
}

# Add source code zip to bucket
resource "google_storage_bucket_object" "zip" {
  # Append file MD5 to force bucket to be recreated
  name   = "source.zip#${data.archive_file.source.output_md5}"
  bucket = google_storage_bucket.bucket.name
  source = data.archive_file.source.output_path
}

# Enable Cloud Functions API
resource "google_project_service" "cf" {
  project = var.project
  service = "cloudfunctions.googleapis.com"

  disable_dependent_services = true
  disable_on_destroy         = false
}

# Enable Cloud Build API
resource "google_project_service" "cb" {
  project = var.project
  service = "cloudbuild.googleapis.com"

  disable_dependent_services = true
  disable_on_destroy         = false
}

# Create Cloud Function
resource "google_cloudfunctions_function" "function" {
  name    = var.function_name
  runtime = "nodejs12" # Switch to a different runtime if needed

  available_memory_mb   = 128
  source_archive_bucket = google_storage_bucket.bucket.name
  source_archive_object = google_storage_bucket_object.zip.name
  trigger_http          = true
  entry_point           = var.function_entry_point
}

# Create IAM entry so all users can invoke the function
resource "google_cloudfunctions_function_iam_member" "invoker" {
  project        = google_cloudfunctions_function.function.project
  region         = google_cloudfunctions_function.function.region
  cloud_function = google_cloudfunctions_function.function.name

  role   = "roles/cloudfunctions.invoker"
  member = "allUsers"
}
```

This file is dealing with all the logic of compressing the source code, storing it in a bucket, creating the Cloud Function and setting the necessary permissions to it.

### Using your module

Now that you have your `function` module ready, you can invoke it in other parts of your Terraform code.

Go back to the entry point file on `terraform/main.tf` and add the following:

```diff
# terraform/main.tf

provider "google" {
  project = var.project
  region  = var.region
}

+ module "my_function" {
+   source               = "./modules/function"
+   project              = var.project
+   function_name        = "my-function"
+   function_entry_point = "app"
+ }
```

When running the file above, Terraform will look for a `main.tf` file on the path declared in the `source` parameter and run the code there along with the other variables.

> Note: The `function_entry_point` must match the name of the exported variable in your Node.js code. You will find `exports.app = …` on the bottom of `src/index.js`.

In the `terraform/outputs.tf` file, add the return values from the module you want to use. Since the module only returns one output value, your file should look like this:

```ruby
# terraform/outputs.tf

output "function_url" {
	# Access the module output with module.<module_name>.<output_name>
	value = module.my_function.function_url
}
```

Now let’s see how to deploy all the resources with the Terraform CLI.

## 5. Deploying

The hard work is already done! Creating the infrastructure should be an easier step.

Run the following commands on the root for your repository to create all the resources and deploy your code:

```bash
# Make sure you are on the terraform folder
cd terraform

# Initialize your configuration
terraform init

# Plan the configuration
terraform plan

# Create all the resources
terraform apply
```

If everything works well, you will see a similar output in your terminal:

```bash
Apply complete! Resources: 6 added, 0 changed, 0 destroyed.

Outputs:

function_url = https://us-central1-my-project-1234567.cloudfunctions.net/my-function
```

You can verify that it works with a simple `curl` command. Remember to replace the URL with your own URL.

```bash
curl https://us-central1-my-project-1234567.cloudfunctions.net/my-function
{"works":true}
```

### Updating the function

Your first deploy is never final. Eventually, you will want to deploy new versions of the code that runs in the Cloud Function.

After changing and testing your code, you can simply run `terraform apply` in your terminal. Terraform will compress your source files, store them in the Cloud Storage bucket and update the function with the new code.

### Destroying the function

You can clean up all the resources created by running `terraform destroy`.

The project won’t be deleted this way (it wasn’t created by Terraform). For that, you can run:

```bash, data-test=1
# Delete the project
gcloud projects delete PROJECT_ID
```

## 6. Going further

This tutorial provides a quick way to get started. Many other good practices can be incorporated to build a more robust application:

**Remote Terraform backend**. If you check your repository you will notice that a state file was created by Terraform. It’s a good practice to store this file in a remote storage. You can change the backend from "local" to a Cloud Storage bucket, for example. See the list of available backends [here](https://www.terraform.io/docs/backends/index.html).

**Multiple environments**. You might want to deploy the same infrastructure here under a different environment (e.g. development & production). There are many ways to do it with Terraform and you will find lots of tutorials around.

**Continuous deployment**. Ideally, you shouldn’t be running `terraform plan` and `terraform apply` from your local machine. This should be done as part of the automation process of a CI/CD solution such as Cloud Build or GitHub Actions.

This whole tutorial, plus some of the things are implemented on [this repository on GitHub](https://github.com/ruanmartinelli/terraform-cloud-function-api). Check it out!



First, let’s talk about **Infrastructure as Code** (IaC).

Like the name suggests, Infrastructure as Code is the process of provisioning infrastructure using code. It is a way to automate the creation of the topology needed to run your application.

Think of _Infrastructure_ as a broad term. It can be anything from virtual machines to Kubernetes clusters to storage buckets. _Code_ in this context refers to the language used for writing the automation. The most common ones are JSON, YAML and HashiCorp Language Format (HCL).

## IaC: the good parts

Automating your infrastructure has great advantages. Some of my favorites are:

**Consistency**. Consistency is the name of the game on IaC. Ad-hoc changes to infrastructure can cause a mismatch between development, staging and production environments.

**Dynamic provisioning**. Infrastructure is transient. When it’s automated, it becomes easier to provision and deprovision resources according to load.

**Reproducible environments**. Spinning up a sandbox for tests with a certain configuration usually takes a few minutes and has very little human intervention.

**Documented infrastructure**. The IaC code works as an always up-to-date documentation of your infrastructure. Tools like Git will also give you an audit trail and bring collaboration aspects to it.

## Meet Terraform

[Terraform](https://www.terraform.io) is an open source command line tool by [HashiCorp](https://www.hashicorp.com).

It gives you the power to manage - build, update and destroy - infrastructure using IaC principles.

Terraform uses the HCL language. With HCL, you declare a blueprint of how you want your infrastructure to look like. Terraform will read that blueprint and provision all the resources you declared.

This is how an HCL file looks like:

```ruby
provider "aws" {
  version = "~> 3.0"
  region  = "us-east-1"
}

resource "aws_s3_bucket" "bucket" {
  bucket = "my-private-bucket"
  acl    = "private"

  versioning {
    enabled = true
  }
}
```

<figcaption>1. Example HCL file.</figcaption>

In the example above we are informing Terraform that Amazon Web Services (AWS) is our provider of choice. We are also declaring a resource: a private S3 bucket.

## Providers

One very important concept from Terraform are _providers_.

Providers are the glue between the Terraform CLI and other systems. They work like plugins and live separately from Terraform.

With providers, Terraform is able to connect to cloud vendors (e.g. Google Cloud Platform, AWS), PaaS solutions (e.g. Heroku, Kubernetes) and SaaS applications (e.g. Fastly, GitHub).

## State

Terraform keeps track of your infrastructure though a **state file**.

The state file is the source of truth for your architectural elements. It contains all the resources created by Terraform, plus any metadata needed to make Terraform work.

When you update your configuration - your HCL files - you are declaring a new **desired state**.

Terraform’s job is to turn that desired state into real-world. It is going to match your current state - state file - and your desired state - your configuration - and come up with a plan of execution.

The plan contains information about what needs to be created, updated or deleted on the infrastructure. Dependencies between resources are solved using a [dependency graph](https://en.wikipedia.org/wiki/Dependency_graph). With everything in place, Terraform will then call all the APIs necessary to build and manage all the resources.

## Example

Let’s go back to our first example.

Imagine we change the bucket ACL from `private` to `public-read`. Here’s what Terraform would do:

1. Compares the current state with the desired state;
2. Sees that the ACL of the bucket has changed;
3. Creates a plan - 1 resource needs to be updated;
4. Calls the AWS S3 endpoints to update the bucket ACL.

Will it simply update the ACL to `public-read`? Will it delete the bucket and create a new one?

That depends on which type of resource is modified. In general, Terraform is smart enough to only destroy and recreate resources when absolutely needed.

## Is Terraform a good choice for my company?

..or startup, or side project?

Terraform does bring additional complexity to your workflow. HCL is a new language to learn. The command-line tool is still fairly new compared to other IaC solutions and it still shows a few rough edges.

That being said, once you master Terraform, it becomes an indispensable tool. Managing infrastructure with code can actually feel easier and safer than poking around on buttons in a cloud provider dashboard.

The variety of providers on the [Terraform Registry](https://registry.terraform.io) also shows how versatile it is. Once you learn it, you can apply the same knowledge on any cloud provider, service, or [pizza delivery chain](https://github.com/ndmckinley/terraform-provider-dominos).

That’s it for an introduction, thanks for reading! More Terraform posts will come, so stay tuned.



Notes about Technical Writing, most content and examples are from [Google's Technical Writing course](https://developers.google.com/tech-writing/).

## Words

**Define new or existing terms**

If the term is new, introduce it. If the term already exists, link to a good explanation.
If the document is introducing too many terms, consider using a glossary.

**Use terms consistently**

When naming things, apply the same word or term consistently throughout your document.

> 🚫 **Protocol Buffers** provide their own definition language. […] And that’s why **protobufs** have won so many county fairs.

> ✅ **Protocol Buffers** (or **protobufs** for short) provide their own definition language. […] And that’s why protobufs have won so many county fairs.

**Acronyms**

When using an acronym for the first time, always spell out the full term followed by the acronym in parenthesis. You can then use the acronym going forward.

> This document is for engineers who are new to the **Telekinetic Tactile Network** (**TTN**) or need to understand how to order TTN replacement parts through finger motions

Choose only one. Do not cycle back-and-forth between the acronym and the expanded version.

**When to use acronyms**

You should use acronyms if:

- The acronym is significantly shorter than the expanded term.
- The acronym/term will be referenced many times in the document.

**Popular acronyms**

After being used so much, acronyms develop their own identity. It’s OK to use popular acronyms such as _HTML_ or _CSS_ without the expanding the full term for the first time.

**Disambiguate pronouns**

Pronouns point to a previously introduced noun, which can sometimes cause ambiguity. In the examples below, can you tell what **It** and **their** refer to?

> 🚫 Python is interpreted, while C++ is compiled. **It** has an almost cult-like following.

> 🚫 Be careful when using Frambus or Carambola with HoobyScooby or BoiseFram because a bug in **their** core may cause accidental mass unfriending.

Some guidelines to use pronouns are:

- Only use a pronoun after you introduced the noun.
- Place the pronoun as close as possible to the noun.
- If you introduce a second noun between your noun and its pronoun, reuse your noun instead of using a pronoun.

<hr/>

## Active voice

Most sentences in technical writing should be in **active voice**.

**Active voice vs. Passive voice**

In an active voice sentence, an actor acts on a target: `actor + verb + target`

A passive voice sentence reverses the formula: `target + verb + actor`

**Active Voice Example**

> The cat sat on the mat.

- actor: The cat
- verb: sat
- target: the mat

**Passive Voice Example**

> The mat was sat on by the cat.

- target: The mat
- passive verb: was sat
- actor: the cat

Passive verbs typically have the following formula:

`passive verb = form of be + past participle verb`

Examples:

- was interpreted
- is generated
- was formed
- is frozen

**Why use Active voice**

Use active voice most of the time and passive voice sparingly. Active voice is better because:

- Readers mentally convert passive voice to active anyway, so better use Active voice straight up
- Passive voice reports an action indirectly and obfuscates the main idea
- Active voice is shorter than passive voice

<hr/>

## Clear sentences

**Choose strong verbs**

Choose precise, strong and specific verbs to engage readers. Imprecise, weak and generic verbs should be avoided. Examples:

> Weak verbs: _occur, happen, is, are (forms of be)_

Weak Verbs x Strong Verbs:

> 🚫 The error **occurs** when clicking the Submit button.<br/>
> ✅ Clicking the Submit button **triggers** the error.

> 🚫 This error message **happens** when…<br/>
> ✅ The system **generates** this error message when…

> 🚫 We **are** very careful to ensure…<br/>
> ✅ We carefully **ensure**…

**Reduce there is/there are**

**There is** and **There are** weakens the sentence and usually associate a generic noun to a generic verb. Sometimes you can just delete the **There is** or **There are** to make the sentence more clear:

> 🚫 There is a variable called `met_trick` that stores the current accuracy.<br/>
> ✅ A variable named `met_trick` stores the current accuracy.<br/>
> 💯 The `met_trick` variable stores the current accuracy.

You can sometimes repair a **There is** or **There are** sentence by moving the true subject and true verb from the end of the sentence to the beginning:

> 🚫 There are two disturbing facts about Perl **you** should know.<br/>
> ✅ **You** should know two disturbing facts about Perl.

And if no subject exists, consider creating one:

> 🚫 There is no guarantee that the updates will be received in sequential order. <br/>
> ✅ **Clients** might not receive the updates in sequential order.

<hr/>

## Short sentences

Each sentence should focus on a single idea. Breaking a long sentence into a succession of single-idea sentences makes it clearer. Example:

> 🚫 The late 1950s was a key era for programming languages because IBM introduced Fortran in 1957 and John McCarthy introduced Lisp the following year, which gave programmers both an iterative way of solving problems and a recursive way.<br/>
> ✅ The late 1950s was a key era for programming languages. IBM introduced Fortran in 1957. John McCarthy invented Lisp the following year. Consequently, by the late 1950s, programmers could solve problems iteratively or recursively.

**Convert some long sentences to lists**

When you see the conjunction **or** in a long sentence, consider refactoring that sentence into a bulleted list. For example:

> To alter the usual flow of a loop, you may use either a **break** statement (which hops you out of the current loop) or a **continue** statement (which skips past the remainder of the current iteration of the current loop).

Can be converted to:

> To alter the usual flow of a loop, call one of the following statements:<br/>
>
> - **break**, which hops you out of the current loop.<br/>
> - **continue**, which skips past the remainder of the current iteration of the current loop.

**Eliminate or reduce extraneous words**

Remove words that only consume space and have no added value to the sentence:

> 🚫 An input value greater than 100 **causes the triggering** of logging.<br/>
> ✅ An input value greater than 100 **triggers** logging.

> 🚫 This design document **provides a detailed description of** Project Frambus.<br/>
> ✅ This design document **details** Project Frambus.

A few more common bloated phrases:

- at this point in time —> now
- determine the location of —> find
- is able to —> can

---

## Lists and tables

**Bulleted list vs. numbered list**

If you rearrange the items on the list and the meaning don’t change, you should use a bulleted list. If the meaning changes, the list should be numbered.

**Embedded list**

A list where items are all stuffed within a sentence. For example:

> The llamacatcher API enables callers to create and query llamas, analyze alpacas, delete vicugnas, and track dromedaries.

When you can, try to convert embedded lists to bulleted or numbered lists.

**Keep list items parallel**

In parallel lists, all items look like they “belong” together and match a pattern. The first item in a parallel list establishes a pattern that readers expect to see repeated in subsequent items.

On the example below, all items are plural nouns (**grammar**), edible (**logical category**), lower case (**capitalization**), and without periods or commas (**punctuation**)

> - carrots
> - potatoes
> - cabbages

By contrast, a nonparallel list could look like this:

> - carrots
> - potatoes
> - The summer light obscures all memories of winter.

**Start numbered list items with imperative verbs**

Imperative verbs are commands, such as **open** or **start**. Consider starting all items in a numbered list with an imperative verb. For example:

> ✅
>
> 1. **Download** the Frambus app from Google Play or iTunes.
> 2. **Configure** the Frambus app’s settings.
> 3. **Start** the Frambus app.

> 🚫
>
> 1. **Instantiate** the Froobus class.
> 2. **Invoke** the Froobus.Salmonella() method.
> 3. The process stalls.

**Introduce each list and table**

It’s good practice to introduce lists and tables. A good practice is putting the word **following** into the introductory sentence. Examples:

> The following list identifies key performance parameters:

> Take the following steps to install the Frambus package:

> The following table summarizes our product’s features against our key competitors’ features:

---

## Paragraphs

Good opening sentences establish the paragraph’s central point. For example, the following opening sentence establishes the theme of the paragraph as an introduction to loops:

> A loop runs the same block of code multiple times. For example, suppose you wrote a block of code that detected whether an input line ended with a period. To evaluate a million input lines, create a loop that runs a million times.

**Focus each paragraph on a single topic**

A paragraph should represent an independent unit of logic. Restrict each paragraph to the current topic. Don’t describe what will happen in a future topic or what happened in a past topic. When revising, ruthlessly delete (or move to another paragraph) any sentence that doesn’t directly relate to the current topic.

**Don’t make paragraphs too long or too short**

Long paragraphs are visually intimidating. _Very_ long paragraphs form a dreaded “wall of text” that readers ignore. Readers generally welcome paragraphs containing three to five sentences, but will avoid paragraphs containing more than about seven sentences.

**Answer what, why, and how**

1. **What** are you trying to tell your reader?
2. **Why** is it important for the reader to know this?
3. **How** should the reader use this knowledge. Alternatively, how should the reader know your point to be true?

Example:

[**What:**] The `garp()` function returns the delta between a dataset’s mean and median. [**Why:**] Many people believe unquestioningly that a mean always holds the truth. However, a mean is easily influenced by a few very large or very small data points. [**How:**] Call `garp()` to help determine whether a few very large or very small data points are influencing the mean too much. A relatively small `garp()` value suggests that the mean is more meaningful than when the `garp()` value is relatively high.

---

## Audience

**Define your audience**

Things to consider when defining your audience:

- Are you writing the document to software engineers? Product managers? Both?
- What’s the proximity to the knowledge? Example: Engineers who worked on project Frambus know things about project Dingus, but nothing about project Carambola.

**Fit documentation to your audience**

- Be mindful of proximity. The people on your team probably understand your team’s abbreviations, but do people on other teams understand those same abbreviations?
- Unless you are writing specifically for other experienced members of your team, you typically must explain more than you expect.
- **Curse of knowledge**. As experts, it is easy to forget that novices don’t know what you already know.
- Prefer simple words over complex words. English is not the native language of a significant percentage of technical readers.
- Keep your writing culturally neutral. Avoid things like baseball metaphors (common in American English) or other localised idioms. Examples: “piece of cake”, “Bob’s your uncle”, “off the bat”.

---

## Summary

- Use terms consistently.
- Avoid ambiguous pronouns.
- Prefer active voice to passive voice.
- Pick specific verbs over vague ones.
- Focus each sentence on a single idea.
- Convert some long sentences to lists.
- Eliminate unneeded words.
- Use a numbered list when ordering is important and a bulleted list when ordering is irrelevant.
- Keep list items parallel.
- Start numbered list items with imperative words.
- Introduce lists and tables appropriately.
- Create great opening sentences that establish a paragraph’s central point.
- Focus each paragraph on a single topic.
- Determine what your audience needs to learn.
- Fit documentation to your audience.
- Establish your document’s key points at the start of the document.



> Notes about Docker.

**Containers** are an isolated area of an OS with resource usage limits applied.

Two kernel low level constructs make containers possible: **Control groups** and **Namespaces**. Working with these constructs directly is hard if you are not a Kernel engineer. The Docker engine does its “magic” of creating containers using those constructs.

---

Namespaces are responsible for **isolation**. Each container has its own process tree, root file system, eth0 interface, root user, etc. It acts like an isolated OS but it shares the Kernel with the host (and other containers). Containers are not aware of each other.

Some of the Namespaces present on each container: `pid`, `net`, `mnt`,`ipc`, `uts`, `user`.

Control groups (`cgroups`) are responsible for **setting limits**. They polish the consumption of system resources.

Problem: in a multi-container setup, one of the containers steals all the resources from the host OS, leaving only a few to the other containers.

---

The **Docker Engine** interfaces with the Kernel (namespaces and croups).

Before having the engine, Docker was built on top of [LXC](https://en.wikipedia.org/wiki/LXC). LXC has stability problems so they had to switch to something more stable and that they could control, so `libcontainer` was created. Later on more things like the registry, HTTP server, registry, compose, orchestration, etc. were added. This monoltith - called the **deamon** - became huge and bloated.

Docker started refactoring the daemon to make it more modular. By the same time the [Open Container Initiative](https://opencontainers.org/) (OCI) started developing standards for containers.

Today: the docker daemon is mostly implementing the REST API. Under the hood it calls `containerd` which calls `runc`.

---

- [LXC vs. libcontainer](https://stackoverflow.com/questions/34152365/difference-between-lxc-and-libcontainer)
- [runc vs. containers vs. docker](https://medium.com/@alenkacz/whats-the-difference-between-runc-containerd-docker-3fc8f79d4d6e)

---

When you run `docker container run <params>` on Linux...

1. The docker client posts an API request (`POST /vX.X/containers/create HTTP/1.1`) to the endpoint in the daemon;
2. The daemon doesn’t have any code to execute or run containers (after the refactor) so it calls `containerd` via GRPC on a local Unix socket. (`client.newContainer(context,…`);
3. Despite the name, `containerd`actually doesn’t create the containers. The logic to interact with namespaces and control groups is implemented by the OCI implementation (`runc`on Linux by default);
4. `runc`creates the container, gives it back to `containerd`and exits (short-lived process). `containerd`sticks around managing the containers created by `runc`(long-lived process, daemon).

---

Images are read-only templates for creating containers. They hold all the code and supporting files to run an application - OS files and objects, app files and a manifest. Images are sharable through a Registry.

> You can think of images as a stopped container and containers as a running image.

---

Images are a collection of layers stacked. Example:

- (3) Updates
- (2) Redis and Application code
- (1) Ubuntu OS (Base layer)

The manifest JSON file describes things such as the ID, the tags, when it was created and a list of the layers that were stacked.

---

What happens in a `docker image pull ...`?

1. Calls the docker registry API (defaults to Docker Hub) to fetch the **fat manifest**;
2. Fetch the **image manifest** based on your architecture;
3. Fetch the layers present on the image manifest.

Fat manifest (a.k.a. Manifest list) : a list of manifests for each architecture.
Image manifest: the actual manifest for the image but taking into consideration the architecture (ARM, etc.)

```
☁  ~ docker image pull redis
Using default tag: latest
latest: Pulling from library/redis
8559a31e96f4: Pull complete
85a6a5c53ff0: Pull complete
b69876b7abed: Pull complete
a72d84b9df6a: Pull complete
5ce7b314b19c: Pull complete
04c4bfb0b023: Pull complete
Digest: sha256:800f2587bf3376cb01e6307afe599ddce9439deafbd4fb8562829da96085c9c5
Status: Downloaded newer image for redis:latest
docker.io/library/redis:latest
```

---

Images are hosted on Registries. They are represented by `<registry>/<repository>:<version`.

When pulling an image, docker defaults to the Docker Hub registry (https://docker.io) and the `latest` version of the image. These two commands are equivalent:

`docker pull image redis`
`docker pull image docker.io/redis:latest`

---

[Content hashes vs. Distribution hashes](https://stackoverflow.com/a/50737616)

---

Best practices:

1. Use official images when possible;
2. Be explicit when addressing image versions (do not use the `latest` tag).



AWS Lambda Layers was announced during 2018's re:Invent as a new way bundle dependencies into functions. It allows us to include libraries, binaries and even runtimes into Lambdas.

Without Layers, dependencies need to be bundled together with the function code, inside the `.zip` file that is pushed to Amazon servers. This results on large deployment packages and a slow development pipeline. Some dependencies can't even be bundled with ease, since they exceed Lambda's current 50 MB package size limit.

Layers allows us to package and deploy dependencies independently from our functions. Every Layer also gets an unique identifier that can be shared and re-used between other Lambdas.

This tutorial shows how to publish a function and a Layer using the Serverless Framework.

## Creating a Screenshot service with Chromium and Puppeteer

Let's see how we can deploy a Lambda Function that receives an URL, takes a screenshot of the page and returns it. [Puppeteer](https://pptr.dev/) will be the library of choice to handle page loading and screenshots. Since Puppeteer relies on a compiled version of Chromium to deal with pages, we also need to provide it as a dependency by using Layers.

> Tip: The process for including other prebuilt binaries - such as FFmpeg or Imagemagick - should be very similar.

### Setting up the project

First, let's create an empty repository and add the project files:

```bash
# Create the project repository
mkdir screenshot-page

# Enter the repository
cd screenshot-page

# Create a minimal package.json
yarn init -y

# Create a file for the function code
touch handler.js

# Create a Serverless configuration file
touch serverless.yml

# Create a folder where we'll keep the dependencies (in this case, the Chromium binary)
mkdir layer
```

Run the commands below to install the dependencies needed:

```bash
# Install "puppeteer-core" for taking screenshots
yarn add puppeteer-core

# Install "serverless" to manage and deploy our function
yarn add -D serverless
```

> Tip: The `pupeteer` npm package downloads Chromium by default when installed. This would result in a huge deployment package. Since we'll be providing the Chromium binary via Layer we can only install `puppeteer-core`, which doesn't include Chromium. See more on [puppeteer vs. puppeteer-core](https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#puppeteer-vs-puppeteer-core).

> Tip: If you are using a global installation of `serverless`, make sure the version is higher than v1.34.0 (when support for Layers was added)

### Downloading Chromium

Now let's add Chromium to the project. For convenience, let's grab one of the prebuilt binaries from the `serverless-chrome` repository [here](https://github.com/adieuadieu/serverless-chrome/releases).

Download the file, extract the binary and add it to the `layer` folder:

```bash
# Still inside the project repository...

# Download the zip file
curl -L0 https://github.com/adieuadieu/serverless-chrome/releases/download/v1.0.0-55/stable-headless-chromium-amazonlinux-2017-03.zip -o chromium.zip

# Unzip it
unzip chromium.zip

# Move the binary into the "layer" folder
mv headless-chromium ./layer/headless-chromium

# Delete the zip file
rm chromium.zip
```

> Tip: You can add the `layer` folder (or whatever you call it) to your `.gitignore` and avoid pushing large files to your Git server.

After following these steps you should end up with this structure on your project:

```
.
├── handler.js
├── layer
│   └── headless-chromium
├── package.json
├── serverless.yml
└── yarn.lock
```

### Adding code and configuration

Now let's add code to our files. Below is an example of how your `serverless.yml` configuration should look like.

Note that we have a new top-level property called `layers`. It contains the path to the folder where we store our dependencies - in this case, Chromium's prebuilt binary.

To attach the Layer to our function, we can simply reference it on the function declaration and it will be available during the function execution.

```yaml
service: screenshot-page

provider:
  name: aws
  runtime: nodejs8.10
  memorySize: 512
  timeout: 30

# Here we define the name of the layer and its path.
# Everything that's inside of /layer will be published
# in a separate package.
layers:
  chromium:
    path: layer

functions:
  screenshot:
    handler: handler.screenshot
    events:
      - http:
          method: get
          path: /screenshot
    # Here we reference the layer used by this function.
    # The Ref name is generated by TitleCasing the layer name & appending "LambdaLayer".
    layers:
      - { Ref: ChromiumLambdaLayer }
```

On `handler.js` we add the Puppeteer code to screenshot a page:

```js
const puppeteer = require("puppeteer-core");

module.exports.screenshot = async function (event) {
  // Get the url from parameters
  const { url } = event.queryStringParameters;

  // Launch a puppeteer instance
  const browser = await puppeteer.launch({
    headless: true,
    args: ["--no-sandbox", "--disable-gpu", "--single-process"],
    // Layer dependencies will be available on "/opt"
    // during the Lambda execution.
    executablePath: "/opt/headless-chromium",
  });

  const page = await browser.newPage();

  await page.goto(url);

  // Screenshots the page
  const image = await page.screenshot({ encoding: "base64" });

  return {
    body: `<img src="data:image/png;base64,${image}" />`,
    headers: { "Content-Type": "text/html" },
    statusCode: 200,
  };
};
```

Note that, because we are using `puppeteer-core`, we need to provide the path where the Chromium binary is hosted using the `executablePath` option.

Another important thing about Layers is that they **extract the dependencies on the `/opt` folder** during the function execution.

### Deploying the function

At this point we are ready to deploy our function and the Layer containing the Chromium binary. Just run the command below in your terminal:

```bash
yarn sls deploy
```

The output will look like this:

```bash
Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Excluding development dependencies...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Uploading service .zip file to S3 (790.45 KB)...
Serverless: Uploading service .zip file to S3 (47.74 MB)...
Serverless: Validating template...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
...................
Serverless: Stack update finished...
Service Information
service: screenshot-page
stage: dev
region: us-east-1
stack: screenshot-page-dev
api keys:
  None
endpoints:
  GET - https://km8mqcwfvk.execute-api.us-east-1.amazonaws.com/dev/screenshot
functions:
  screenshot: screenshot-page-dev-screenshot
layers:
  chromium: arn:aws:lambda:us-east-1:372768542448:layer:chromium:8
Serverless: Removing old service artifacts from S3...
Done in 295.39s.
```

Here we can see that the Layer and the function code got split into two separate `.zip` files and were uploaded independently.

The Layer ARN is also present on the output:

```
layers:
  chromium: arn:aws:lambda:us-east-1:372768542448:layer:chromium:8
```

If we ever need to use Chromium on any other function we can simply reference this Layer's ARN on the AWS Dashboard.

> Tip: You can also publish the Layer without any function with Serverless. Just remove the `functions` section on your `serverless.yml` and run the deployment command normally.

To test the function just visit the url:

[https://km8mqcwfvk.execute-api.us-east-1.amazonaws.com/dev/screenshot?url=https://news.ycombinator.com](https://km8mqcwfvk.execute-api.us-east-1.amazonaws.com/dev/screenshot?url=https://news.ycombinator.com)

And you will see the screenshot:

![HN Screenshot](https://i.imgur.com/75LWZhP.png)

## Conclusion

Layers is a very welcome addition to the AWS Lambda toolset. Bundling dependencies on Lambdas used to be a difficult task, specially for large binaries. Now with Layers it became easier to manage, share and keep dependencies updated.

Serverless also provides a very simple and straightforward way to use and deploy Lambda Layers to AWS.

## References

- [Official Documentation](https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html)
- [How to publish and use AWS Lambda Layers with the Serverless Framework](https://serverless.com/blog/publish-aws-lambda-layers-serverless-framework/)



The following is a rough draft of links and notes I have put together while researching property-based testing.

## Example-based testing

- Most common form of testing.
- Tests if a code works for a pre-defined set of examples.
- Very easy and cheap to write.
- The number of test cases can grow exponentially depending on how many paths a program can take.

Example:

```js
test("sum(a, b)", () => {
  expect(sum(1, 3)).toBe(4)
  expect(sum(1, 0)).toBe(1)
  expect(sum(0, 0)).toBe(0)
})
```

## Property-based testing

- Shifts the focus to the properties of a program.
- Tests the boundaries of the outputs instead of a limited set of instances.
- Example based vs. property-based - "There-exists" vs. "For-all"

Some definitions:

> Property based testing is the construction of tests such that, when these tests are fuzzed, failures in the test reveal problems with the system under test that could not have been revealed by direct fuzzing of that system. - David Maclver

> Property based testing works like this: first, you describe the arguments of your program, then you describe the result that you expect from those arguments. After that the computer does multiple attempts to prove your code wrong. - Someone on the internet

> In property based testing, rather than checking the results with specific input, properties are asserted - “for any possible input, [some condition] should hold” - and a test runner searches for counter-examples. - NarendraC

## Properties

- Predicates that have to hold true for all inputs generated.
- Invariants - won't change given the input.
  - They follow the same idea of [mathematical properties](https://en.wikipedia.org/wiki/Addition#Properties).
- Coming up with properties is the hard part of property based testing. Coming up with examples is quite easy.

Example - testing the `sum` function with `jsverify`:

```js
describe("sum(a, b)", () => {
  jsc.property(
    "sum is commutative (a + b = b + a)",
    jsc.integer,
    jsc.integer,
    (a, b) => sum(a, b) === sum(b, a)
  )
})
```

## Patterns

_Update: I've implemented some of the patterns mentioned below. See it [here](https://github.com/ruanmartinelli/property-based-testing-patterns)._

Choosing properties is hard, but we have some patterns to help:

- Inverse functions
  - `decode(encode(x)) === x`
- Fuzzing
  - Looks for unexpected crashes (500 status code, unexpected exceptions, etc.)
- Test Oracle
  - Use an alternative version of implementation to check the result.
  - `myCode() === oracleCode()`

More patterns can be found on the links:

- [Choosing properties for property-based testing](https://fsharpforfunandprofit.com/posts/property-based-testing-2/)
- [Property-based Testing is a Mindset](https://youtu.be/p84DMv8TQuo?t=824)
- [Patterns to Find Good Properties](https://blog.johanneslink.net/2018/07/16/patterns-to-find-properties/)
- [Property-based Testing in Practice](https://www.slideshare.net/InfoQ/propertybased-testing-in-practice)

## Benefits

Some benefits of property-based testing:

- Forces you to reason about your code in a way you are not used to.
- Good for finding and tracking corner cases that weren't considered.
- Good for finding bad inputs of a program.

## Tooling

QuickCheck:

- Invented in 2000 by John Hughes and Koen Clasessen.
- First tool created to test the properties of programs.
- [QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs](https://www.cs.tufts.edu/~nr/cs257/archive/john-hughes/quick.pdf)
- Ported to many languages since then.

JavaScript:

- [jsverify](https://www.npmjs.com/package/jsverify)
- [fast-check](https://www.npmjs.com/package/fast-check)

Shrinking:

- A mechanism to simplify failing inputs of a test case.
- Finds the minimal reproducible case of a failing test.

Generators/Seed:

- A mechanism to produce random inputs from a type.
- Random values are generated from a seed.
  - You can use the seed to reproduce failing tests later with the same input generated before.

## Who's using?

- [Volvo](http://www.quviq.com/volvo-quickcheck/)
  - Used for testing third-party car components.
- [LevelDB](http://www.quviq.com/google-leveldb/)
  - Found a bug in a sequence of 17 operations with stateful testing.
- Clojure

## Fuzzing vs. Property based testing

Definition of Fuzzing:

> Fuzzing is feeding a piece of code (function, program, etc.) data from a large corpus, possibly dynamically generated, possibly dependent on the results of execution on previous data, in order to see whether it fails.

- Fuzz testing is a form of property-based testing.
- Tests the property "it doesn't crash":
  - No 500 status code returned from API.
  - All responses are valid JSON.
  - No unexpected exceptions are thrown.
- [What's the difference between fuzz-testing and property-based testing?](https://sqa.stackexchange.com/questions/18751/whats-the-difference-between-fuzz-testing-and-property-based-testing)

## Stateful Properties

- Complex systems have state!!
  - E.g. a key-value database.
- With Stateful Property testing, you define a set of possible actions and the framework will try to find sequences of those actions that result in a failure.
- [Stateful Properties](https://www.propertesting.com/book_stateful_properties.html)

## Misc

Local vs. CI:

- Good scenario: use a few iterations on local development and bump the number on CI.
  - E.g. 25 iterations on local environments and 500 on CI.

Unit vs. Property:

- They both go well together.
- Unit tests are still better for doing regressions.

## Links & References

- [Property-Based Testing for Better Code](https://www.youtube.com/watch?v=shngiiBfD80)
- [What is Property Based Testing?](https://hypothesis.works/articles/what-is-property-based-testing/)
- [Better Testing With Less Code: Property Based Testing With Python](https://www.youtube.com/watch?v=jvwfDdgg93E)


Hi — I'm Ruan.

Categories

Most Popular

Getting Started with npm Workspaces
1 Jun 2021 • 6 min read • 18.1k

Deploying to Cloud Functions with Terraform
30 Nov 2020 • 10 min read • 4.2k

How to Use Schemas on Fastify for Fun and Profit
10 Dec 2020 • 8 min read • 2.9k

Hi — I'm Ruan.

Categories

Most Popular

Getting Started with npm Workspaces1 Jun 2021 • 6 min read • 18.1k

Deploying to Cloud Functions with Terraform30 Nov 2020 • 10 min read • 4.2k

How to Use Schemas on Fastify for Fun and Profit10 Dec 2020 • 8 min read • 2.9k

Getting Started with npm Workspaces
1 Jun 2021 • 6 min read • 18.1k

Deploying to Cloud Functions with Terraform
30 Nov 2020 • 10 min read • 4.2k

How to Use Schemas on Fastify for Fun and Profit
10 Dec 2020 • 8 min read • 2.9k