Securing Webhook Payload Delivery in Phoenix

Some of the most interesting and valuable behaviors we can build as web developers are through connecting different web applications together. Often times this comes in the form of webhooks, where one web app will POST events to your own web app. Keeping that communication secure is important and today we will take a peak into how you can secure webhook payloads in a Phoenix application.

Webhook Security 101

Webhook might sound like some scary, exotic term but at the end of the day webhooks are nothing more that a web server endpoint configured to receive HTTP POST requests in some agreed upon format. The concern however is, without any additional security, anyone could discover the endpoint and start sending their own malicious payloads to your web application.

To help secure a webhook, when you first set it up you’ll give the third party some generated secret. As an example, here is where you can store the secret while configuring a GitHub webhook:

GitHub Webhook configuration screen, showing a field for “secret”.

When GitHub then sends the HTTP POST request they will include, in the headers of that request, a signature of the payload using the previously shared secret. As you process the HTTP POST request you can then generate your own signature, based on the incoming payload and the known secret, to verify the payload’s authenticity. If the secrets match then you have much more trust that the payload came from the correct third party.

Validating a Payload Signature in Phoenix

To get this to work in Phoenix we’ll need a few core behaviors.

First, we will want the ability to generate a signature based on a payload and a secret.

  def generate_payload_signature(_, nil) do
    {:error, :missing_app_secret}
  end

  def generate_payload_signature(payload, app_secret) do
    {:ok, :crypto.hmac(:sha256, app_secret, payload) |> Base.encode64()}
  end

Second we’ll need a way to pull out the signature from the connection’s request headers, if present:

  defp signature_from_req_headers(req_headers) do
    case List.keyfind(req_headers, "x-hub-signature", 0) do
      {"x-hub-signature", full_signature} ->
        "sha256=" <> signature = full_signature
        signature

      _ ->
        nil
    end
  end

Finally we’ll connect the dots creating a simple call site function where someone can pass in a Plug.Conn and a payload and ask if it is authentic.

  def is_authentic_payload?(%Conn{req_headers: req_headers}, payload) do
    case signature_from_req_headers(req_headers) do
      nil ->
        false

      signature ->
        is_payload_signature_valid?(signature, payload)
    end
  end

  defp is_payload_signature_valid?(payload_signature, payload) do
    case generate_payload_signature(payload, webhook_secret()) do
      {:ok, ^payload_signature} -> true
      _ -> false
    end
  end

  defp webhook_secret do
    Keyword.fetch!(Application.fetch_env!(:webhook_signature, :github), :webhook_secret)
  end

There are a few ways you could go about getting this check into your controller code. The most challenging part is that out of the box Phoenix does not make the raw body of an HTTP POST request available to you through the Plug.Conn struct. Instead a default Phoenix app uses parsers and converts such payloads into native Elixir value types.

One way to turn off this behavior is to edit your Phoenix app’s endpoint.ex file and remove the auto-parsing behavior for all routes of that endpoint. You can find some more info about doing this in the Plug docs in a section titled “Custom body reader”.

Since this parsing behavior is helpful for non-webhook requests in our example app, we will relocate the parsing configurations out of the endpoint.ex file and into the pipeline definition on the router.ex file. This way we can keep the parsing behavior working for the rest of the example app and do something custom for our webhook endpoint.

defmodule WebhookSignatureWeb.Endpoint do
  # CONDENSED FOR DEMONSTRATION

-  plug Plug.Parsers,
-    parsers: [:urlencoded, :multipart, :json],
-    pass: ["*/*"],
-    json_decoder: Phoenix.json_library()

end
defmodule WebhookSignatureWeb.Router do
  # CONDENSED FOR DEMONSTRATION

  pipeline :browser do
    plug :accepts, ["html"]
    plug :fetch_session
    plug :fetch_flash
    plug :protect_from_forgery
    plug :put_secure_browser_headers

+    plug Plug.Parsers,
+      parsers: [:urlencoded, :multipart, :json],
+      pass: ["*/*"],
+      json_decoder: Phoenix.json_library()
  end

end

Our next question is where do we this verification check? In our example app, we will keep the controller code focused on the behavior of processing the payload and put the behaviors for signature verification inside some plugs as part of the router’s pipeline for this specific endpoint.

The first plug will read the contents of the Plug.Conn body into a new assigns value :raw_body.

defmodule WebhookSignatureWeb.Plugs.RawBodyPassthrough do
  @moduledoc """
  This plug will read the body for `POST` and PUT` request and store it into a
  new assigns key `:raw_body`.

  This plug is used on certain routes in preference to the default Phoenix
  behaviors that would automatically decode the params and request body into
  native elixir values for a controller. It is a required choice since the body
  of a `Plug.Conn` can only be read from once.
  """

  import Plug.Conn
  alias Plug.Conn

  @spec init(Keyword.t()) :: Keyword.t()
  def init(options), do: options

  @spec call(Conn.t(), term()) :: Conn.t()
  def call(%Conn{method: method} = conn, opts) when method == "POST" or method == "PUT" do
    case Conn.read_body(conn, opts) do
      {:ok, body, _conn_details} ->
        Conn.assign(conn, :raw_body, body)

      {:more, _partial_body, _conn_details} ->
        conn
        |> send_resp(413, "PAYLOAD TOO LARGE")
        |> halt
    end
  end

  def call(conn, _opts), do: Conn.assign(conn, :cached_body, %{})
end

The second plug will pull out the signature from the HTTP request headers and then verify it matches our expectations.

defmodule WebhookSignatureWeb.Plugs.RequirePayloadSignatureMatch do
  @moduledoc """
  This plug will verify that the payload from a webhook request matches the 
  accompanying header signature, based on a previously shared `webhook_secret`.

  When the payload is verified the connection continues as normal.

  When the payload is unverifiable the connection is halted with a 403 response.
  """

  import Plug.Conn
  alias Plug.Conn
  alias WebhookSignature.PayloadValidator

  @spec init(Keyword.t()) :: Keyword.t()
  def init(options), do: options

  @spec call(Conn.t(), term()) :: Conn.t()
  def call(%Conn{method: method} = conn, _opts) when method == "POST" or method == "PUT" do
    case PayloadValidator.is_authentic_payload?(conn, conn.assigns.raw_body) do
      true ->
        conn

      false ->
        conn
        |> put_resp_content_type("application/json")
        |> send_resp(403, "{\"error\":\"PAYLOAD SIGNATURE FAILED\"}")
        |> halt
    end
  end

  def call(conn, _opts), do: Conn.assign(conn, :cached_body, %{})
end

Finally we’ll bring it all together, defining an :api pipeline and using that pipeline for our GitHubWebhookController route.

defmodule WebhookSignatureWeb.Router do
  # CONDENSED FOR DEMONSTRATION

  pipeline :api do
    plug :accepts, ["json"]
    plug WebhookSignatureWeb.Plugs.RawBodyPassthrough, length: 4_000_000

    # It is important that this comes after 
    # `WebhookSignatureWeb.Plugs.RawBodyPassthrough` as it relies
    # on the `:raw_body` being inside the `conn.assigns`.
    plug WebhookSignatureWeb.Plugs.RequirePayloadSignatureMatch
  end

  scope "/github", WebhookSignatureWeb do
    pipe_through :api

    post "/webhook", GitHubWebhookController, :webhook
  end
end

If the signature match is successful the controller will be allowed to do its processing of the payload values.

If the signature match is not successful we return a 403 response since “The server understood the request but refuses to authorize it.”

Show Me The Code

There is an example project with the behaviors expressed in this post on GitHub:

https://github.com/phoenix-by-example/webhook_signature

Further Reading

For more on webhooks, you can check out GitHub’s documentation.