Horizon3.ai - Automated Pen Testing as a Service

Documentation

Horizon3.ai API

The Horizon3.ai API provides programmatic access to a subset of functionality available through the Horizon3.ai portal. At a a high level, this API allows for:

  • scheduling an automated pentest, known as an operation
  • monitoring the status of the operation while it is running
  • retrieving an operation report after it’s complete

The API can be used for a variety of use cases such as periodically scheduling assessments of your environment or kicking off an operation as part of a continuous integration build pipeline.

  • Pre-Requisites
    • API Access
    • Client Infrastructure
      • Host/VM Requirements
      • Outbound Network Access
      • Inbound Network Access
      • Container Privileges
    • Create an Operation Template
  • API Specification
    • Authentication
    • Scheduling an Operation
    • Launching an Operation
    • Monitoring the Status of an Operation
      • Operation Life Cycle
      • Querying the Operation State
    • Reporting
  • Example Script

Pre-Requisites

Prior to using the Horizon3.ai API, make sure to complete the following steps:

  • Getting an API Key
  • Setting up a host/VM to host the Horizon3.ai Docker container, called NodeZero
  • Creating an operation template These steps are described in greater detail below.

API Access

An API key is required to access the API. The key is tied to an existing user account with Org Admin privileges in the Horizon3.ai portal.

At this time, API keys are manually provisioned by Horizon3.ai. If you don’t have an API key, please contact your Horizon3.ai representative to acquire one.

In addition, if you have an API key and need to revoke it and/or generate a new one, please contact your Horizon3.ai representative Please download the PDF to view it

Client Infrastructure

Running a Horizon3.ai operation requires running a Docker container, called NodeZero, on a Linux VM or host. The NodeZero container receives instructions from the Horizon3.ai cloud in order carry out the operation.

The NodeZero container may open ports on the host/VM in order to conduct certain types of attacks. The container automatically shuts down after the operation is complete.

A NodeZero container is used only once per operation. Once an operation is complete, the NodeZero container can be removed.

api_img_01.png

Host/VM Requirements

  • Linux:
    • Ubuntu >= 16.04
    • Redhat >= 7
  • 2 CPUs
  • 4GB RAM
  • 10GB free disk space
  • Docker >= v17.0.7

Outbound Network Access

  • Uninterrupted network access to the following endpoints over TCP 443 during the entire operation
    • cognito-identity.us-east-2.amazonaws.com
    • us-east-2.queue.amazonaws.com
    • *.s3.us-east-2.amazonaws.com

It is possible to run NodeZero through a proxy if necessary. Contact your Horizon3.ai representative for more details.

Inbound Network Access

The following ports should be opened on the NodeZero host/VM to allow traffic in:

  • TCP 21, 23, 25, 53, 80, 88, 110, 135, 139, 143, 389, 443, 445, 587, 1433, 3389, 8080
  • UDP 69

This is required on the NodeZero host. This does not pertain to perimeter firewalls.

Container Privileges

The NodeZero container runs with the following elevated privileges:

  • root user inside the container
  • --network=host privileges
  • --cap-add=SYS_ADMIN
  • --security-opt apparmor=unconfined

These privileges are required to carry out certain types of automated attacks.

Create an Operation Template

Prior to scheduling an operation, the user to whom the API key belongs must first create an operation template in the Horizon3.ai portal. An operation template describes the scope that the client wants to scan and the operation configuration. The scope consists of IP addresses and subnets that the client wants to scan. The configuration describes which attack behaviors are turned on or off during the operation.

See Template Docs for details

API Specification

The API is hosted at api.horizon3ai.com. It provides endpoints for authentication, scheduling an op, monitoring the status of an op, and reporting. The current API version is v1, and all endpoints in the v1 API are accessible at api.horizon3ai.com/v1.

At a high level, to run an operation, the following sequence of steps should be followed:

  1. Authenticate using an API key.
  2. Schedule an operation using the operation template you configured earlier.
  3. Download a launch script.
  4. Run the launch script on the NodeZero Linux host to launch the operation. This is done outside of the API.
  5. While the operation is running, its state can be monitored using the API. The operation is complete when its state is “done.”
  6. Download an operation report.

api_img_02.png

These steps are described in further detail below.

Authentication

A client first authenticates by using its API key against the auth endpoint. The auth API validates the API key and returns a JSON Web Token (JWT) that clients use to invoke other APIs, as described below.

The JWT expires after an hour. Clients must retrieve a new JWT from the auth endpoint after an hour.

Example Request


curl \
  -X POST https://api.horizon3ai.com/v1/auth \
  -H "Content-Type: application/json" \
  -d @- <<HERE 
{
    "key": "<API key>"
}
HERE

Example response


{"token": "<token>"}

Status Codes

  • 200: Success: The response contains the JWT in the token field.
  • 400: Bad Request: Missing Content-Type header
  • 401: Unauthorized: Malformed request payload or unauthorized API key
  • 5xx: Internal Server Error

Launching an Operation

After a client schedules an operation, the Horizon3.ai service begins the process of provisioning cloud infrastructure to support the operation. This process can take a few minutes. When the provisioning process completes, the client should download an operation launch script using the graphql query shown below. The nodezero_script_url field in the response returns a short-lived download URL that should be used by the client to directly download a NodeZero launch script. The URL is valid for up to 12 hours.

The launch script is a bash script that should be run on the NodeZero host. The script validates NodeZero system requirements, downloads the NodeZero Docker image, and runs the NodeZero container.

When an operation is initially scheduled, the download URL for the launch script will not be immediately available. The client can poll periodically on the graphql endpoint until a download URL is returned. Alternatively, the client can poll for the operation status as described in the next section. The script will always be available by the time an operation enters the “ready” state.

Example Request

curl \
 -X POST https://api.horizon3ai.com/v1/graphql \
 -H "Content-Type: application/json" \
 -H "Authorization: Bearer <token>"
 -d @- <<HERE
{
   "query": "
                 query {
                             op(op_id:\"e11b0a7c-fd24-4d48-b4c9-de8220f99847\") {
                                    nodezero_script_url
                             }
                   }
       "
}
HERE

Example Response

{
     "data": {
             "op": {
                   "nodezero_script_url": "<link>"
               }
       }
}

Status Codes

  • 200: Success. The nodezero_script_url field in the response will contain a link to download the script using an HTTPS GET. If the operation is not yet ready, or if the operation is already running or complete, the nodezero_script_url field in the response will have a value of null.
  • 400: Bad Request: Malformed request. The errors field inside the response will contain a descriptive message.
  • 401: Unauthorized: Invalid or expired JWT
  • 403: Forbidden: op_id in the GET request is invalid or not authorized for access. The errors field inside the response will contain a descriptive message.
  • 5xx: Internal Server Error

Monitoring the Status of an Operation

Operation Life Cycle

It’s helpful to understand the life cycle of an operation. An operation always starts off in an initial state of “scheduled”, and it proceeds until it reaches a terminal state of either “Done”, “Canceled”, or “Error.” In between the initial state and a terminal state, there are other states that an operation can assume. These are described in the table below, roughly in the order that the operation proceeds.

api_img_03.png

State Description
scheduled This is the first state an operation enters, Immediately after an API client or end-user schedules an operation.
provisioning The Horizon3.ai cloud service is actively setting up infrastructure to support the operation.
ready The operation is ready to begin. The Horizon3.ai cloud service is waiting for the NodeZero container to start. The operation launch script is available for client download.
running The NodeZero container has connected to the Horizon3.ai cloud service, and the operation is in progress.
complete The Horizon3.ai service has finished executing all attack sequences against the client environment via NodeZero.
destroying The Horizon3.ai service is actively tearing down infrastructure that was provisioned for the operation.
post-processing The Horizon3.ai service is processing the results of the operation and preparing an operation report.
done The operation report is available in the portal or via the API.
canceled An end-user has manually canceled the operation from the Horizon3.ai portal. An operation can only be canceled while it is in the “ready” or “running” state. After it is canceled, the operation moves into the “complete” state and ends at “canceled” instead of “done.” The operation report for everything that was collected prior to cancellation is available in the portal or via the API.
error Unspecified error

Querying the Operation State

The following graphql query against the graphql API endpoint can be used to retrieve the state of an operation. Knowing the state of an operation is helpful for several reasons:

  • If a client has just scheduled an operation, a client can poll on the operation state until it reaches “ready.” When the operation is “ready”, the client can get a link to the launch script to start the operation.
  • While an operation is “running”, a client can poll the operation state until it reaches “done” or “canceled” to know when the operation report is available for download.

A successful response contains the status of the operation in the op_state field.

Example Request

curl \
  -X POST https://api.horizon3ai.com/v1/graphql \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <token>"
  -d @- <<HERE 
{
    "query": "
        query { 
            op(op_id:\"e11b0a7c-fd24-4d48-b4c9-de8220f99847\") {
                op_state
            }
        }
    "
}
HERE

Example Response

{
    "data": {
        "op": {
            "op_state": "done"
        }
    }
}

Status

  • 200: Success. The response will contains the operation status in the op_state field.
  • 400: Bad Request: Malformed request. The errors field inside the response will contain a descriptive message.
  • 401: Unauthorized: Invalid or expired JWT
  • 403: Forbidden: op_id in the GET request is invalid or not authorized for access. The errors field inside the response will contain a descriptive message.
  • 5xx: Internal Server Error

Reporting

After an operation reaches the “done” or “canceled” state, a client can download a summary report using the following graphql query against the graphql API endpoint. The endpoint returns a set of summary statistics containing the number of weaknesses found, credentials discovered, and hosts scanned.

Example Request

curl \
  -X POST https://api.horizon3ai.com/v1/graphql \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <token>"
  -d @- <<HERE 
{
    "query": "
        query { 
            op(op_id:\"e11b0a7c-fd24-4d48-b4c9-de8220f99847\") {
                op_name
                op_state
                scheduled_timestamp_iso
                launched_timestamp_iso
                completed_timestamp_iso
                duration_hms
                in_scope_hosts_count
                confirmed_weaknesses_count
                confirmed_credentials_count
            }
        }
    "
}
HERE

Example Response

{
    "data": {
        "op": {
            "completed_timestamp_iso": "2020-11-13T23:23:43",
            "duration_hms": "00:21:45",
            "in_scope_hosts_count": 10,
            "launched_timestamp_iso": "2020-11-13T23:01:59",
            "op_name": "my op name",
            "op_state": "done",
            "confirmed_credentials_count": 11,
            "confirmed_weaknesses_count": 7,
            "scheduled_timestamp_iso": "2020-11-13T22:14:17"
        }
    }
}

Status

-200: Success. The response contains summary data for the operation. If the operation was never launched or is not yet done, certain fields inside the response may have null values.

  • 400: Bad Request: Malformed request. The errors field inside the response will contain a descriptive message.
  • 401: Unauthorized: Invalid or expired JWT
  • 403: Forbidden: op_id in the GET request is invalid or not authorized for access. The errors field inside the response will contain a descriptive message.
  • 5xx: Internal Server Error

Example Script

The following bash script demonstrates all of the above API actions, from scheduling an operation to retrieving the operation report after the operation is done.

NOTE: This script does not handle error conditions or token renewal if the operation extends past an hour.

#!/bin/bash
#
# requires: curl, jq
#

# User defined parameters, set in the environment
API_KEY="$API_KEY"
OPERATION_TEMPLATE_NAME="$OP_TEMPLATE"
OPERATION_NAME="$OP_NAME"

# Static
API_URL="https://api.horizon3ai.com"
AUTH_URL="$API_URL"/v1/auth
GQL_URL="$API_URL"/v1/graphql
CONTENT_TYPE_HEADER="Content-Type: application/json"

function authenticate {
    curl -s -S \
     -X POST "$AUTH_URL" \
     -H "$CONTENT_TYPE_HEADER" \
     -d "{\"key\": \"$API_KEY\"}"
}

function schedule_op_template {
    curl -s -S \
     -X POST "$GQL_URL" \
     -H "$CONTENT_TYPE_HEADER" \
     -H "$AUTHZ_HEADER" \
     -d @- <<EOF 
{
    "query": "
        mutation { 
            schedule_op_template(
                op_template_name:\"$OPERATION_TEMPLATE_NAME\"
                op_name:\"$OPERATION_NAME\"
            ) { 
                op {
                    op_id
                }
            }
        }
    "
}
EOF
}

function fetch_op_state {
    curl -s -S \
      -X POST "$GQL_URL" \
      -H "$CONTENT_TYPE_HEADER" \
      -H "$AUTHZ_HEADER" \
      -d @- <<EOF
{
    "query": "
        query { 
            op(op_id:\"$OP_ID\") {
                op_state
            }
        }
    "
}
EOF
}

function fetch_nodezero_script_url {
    curl -s -S \
      -X POST "$GQL_URL" \
      -H "$CONTENT_TYPE_HEADER" \
      -H "$AUTHZ_HEADER" \
      -d @- <<EOF 
{
    "query": "
        query { 
            op(op_id:\"$OP_ID\") {
                nodezero_script_url
            }
        }
    "
}
EOF
}

function fetch_op_report {
    curl -s -S \
      -X POST "$GQL_URL" \
      -H "$CONTENT_TYPE_HEADER" \
      -H "$AUTHZ_HEADER" \
      -d @- <<EOF 
{
    "query": "
        query {
            op(op_id:\"$OP_ID\") {
                op_name
                op_state
                scheduled_timestamp_iso
                launched_timestamp_iso
                completed_timestamp_iso
                duration_hms
                in_scope_hosts_count
                confirmed_weaknesses_count
                confirmed_credentials_count
            }
        }
    "
}
EOF
}

function poll_op_for_status {
    while [ 1 ]; do
        op_state=`fetch_op_state | jq -r .data.op.op_state`
        if [ "$op_state" = "$WAIT_STATE" ]; then
            echo "op_state $op_state == $WAIT_STATE! breaking loop."
            break
        fi
        echo "op_state $op_state != $WAIT_STATE ..."
        sleep 15
    done
}

# Step 1: Authenticate to API to get short-lived session token
TOKEN=`authenticate | jq -r .token`
echo "Authenticated to API"

export AUTHZ_HEADER="Authorization: Bearer $TOKEN"

# Step 2: Schedule an operation
export OP_ID=`schedule_op_template | jq -r .data.schedule_op_template.op.op_id`
echo "Scheduled operation id: $OP_ID"

# Step 3: Wait for op to hit "ready" state
export WAIT_STATE="ready"
poll_op_for_status
echo "Operation is ready"

# Step 4: Get the link to the NodeZero launch script
SCRIPT_URL=`fetch_nodezero_script_url | jq -r .data.op.nodezero_script_url`
echo "Launching NodeZero..."

# Step 5: Launch the NodeZero container
curl "$SCRIPT_URL" | bash &

# Step 6: Wait til the op is "done"
export WAIT_STATE="done"
poll_op_for_status
echo "Operation is done"

# Step 7: Download the op report
REPORT=`fetch_op_report | jq -r .data.op`
echo "Operation report: $REPORT"