Understanding APIs: REST

 

Security orchestration, automation and response (SOAR) platforms rely heavily on APIs (application programming interfaces) to drive orchestration of disparate security tools (products) and invoke desired responses in the form of actions. Besides SOAR products, APIs are commonplace among almost all services, tools, and products used by technical workers.

Even though APIs are extremely common, you may not have experience using them or even know that a service has one when interacting with it. For example, Facebook uses an API framework called Graph API.

“The Graph API is the primary way for apps to read and write to the Facebook social graph.”

APIs come in many different forms, but no matter which API framework is used for a service, it enables technical professionals to interface with a system or service. APIs enable us to rapidly develop and connect one system to another, whether that is strictly internal to a single application or on a larger scale like Swimlane.

The use of APIs provides individuals (and organizations) across the world a way to share and interact with information quickly, securely (if implemented properly of course) and in a defined (described) format. Without APIs automation would not exist as it stands today.

In part one of this two-part series, I am going to provide an overview of REST (Representable State Transfer) APIs, as well as the key points to help jump-start your understanding of APIs.

Authentication

REST APIs typically require some mechanism of authentication to interact with the API. This is standard when interacting with a paid product or service, but not necessarily with open-source intelligence tools (OSINT) or other free services on the internet.

There are several standard and custom authentication methods that APIs use. It is impossible to list them all, but you will commonly see a few of these types of authentication methods:

Type	Example	Description
No Authentication	Querying ThreatMiner’s API and lookup up the domain google.com	An API that does not require you to add any additional information when requesting data from an API
Username & Password	Querying Microsoft Exchange on-premises API	An API that requires basic authentication using your username and password
Token	Querying haveibeenpwned.com, which requires that you create a free account and API token.	An API that requires you to have a unique token to use the API service. Typically this token must be provided in the header of your request.
Delegation	Using Microsoft Graph API, which requires OAuth2 Authentication	An API which works in conjunction with an authentication service which provides you with temporary tokens to use the API.

These are just a few examples of the different authentication mechanisms that a service can use. Depending on the product and company, they may choose a standard like the ones above or they may choose to use their own.

The main purpose of these authentication mechanisms is to ensure you have the correct access rights for the service and to also ensure that you are staying within their prescribed usage limits for the service.

If you are interested in a deep-dive into Microsoft’s implementation of OAuth2, then checkout this three part series I wrote which explains more about this specific authentication mechanism: Part 1, Part 2, and Part 3.

Versions

Going forward, I will be using a fictitious service called Josh’s Threat Intelligence Service and the URL for this service is: https://joshsthreatintel.example.

You may visit a site like this and interact with it as you normally would any other website, but behind it contains an API that allows you to access information programmatically to drive automation.

A good API will have very good documentation which outlines the details on how to authenticate and use the service. A common standard used is Swagger, but we don’t need to understand what swagger is (it’s a standard which enables faster development of APIs and standardization when it comes to documentation of them).

At https://joshsthreatintel.example our API is versioned and can be accessed by adding /v1 to our URL. If you access this folder on our website, it could be blank, you may receive some notification that you are not authenticated, or it may be a page that shows you the documentation for the API. Either way, the root of our API will be accessed by using this URL: https://joshsthreatintel.example/v1/.

Each API you will come across may have one or more versions. For example havibeenpwned.com has three:

1. https://haveibeenpwned.com/API/v1
2. https://haveibeenpwned.com/API/v2
3. https://haveibeenpwned.com/API/v3

Methods

REST-based APIs are intended to be interacted with by specific methods defined within their source code. These methods respond differently based on the type used and the data sent to the API. The typical methods you will see when interacting with an API are listed in the table below:

Method	Term Description	Description
GET	Retrieve or Read	GET methods are used to retrieve information from an API
POST	Create or Add or Invoke	POST methods are used to create or add data, or used to invoke an action
PUT	Patch or Update	PUT methods are used to update data or modify it in some way
DELETE	Destroy or Remove	DELETE methods are used to remove data or to stop an action

The most common methods you will come across are GET and POST. Depending on the type of API (REST or SOAP) and service, you may have to supply additional information when making an API request in the headers of your request.

Typically, in a GET request, you will send an http request to an endpoint (path on a URL) to retrieve specific information from the API. For example, imagine our fictitious threat intelligence service has an endpoint called ip and requires that you provide an IP address to this endpoint. This endpoint may look like /ip/123.123.123.123, which we append to our root URL: https://joshsthreatintel.example/v1/ip/123.123.123.123.

As part of our request, we specify in our call to this endpoint that we are performing a GET request, and we are wanting information about the IP address 123.123.123.123.

Here are two examples of doing this in Python and PowerShell Core:

• Python:
  • import requests
  • response = requests.get(‘https://joshsthreatintel.example/v1/ip/123.123.123.123’)
• PowerShell:

   

  • $response = Invoke-RestMethod -Uri ‘https://joshsthreatintel.example/v1/ip/123.123.123.123’ -Method GET

A different endpoint may be able available that allows you to POST a specific URL to our fictitious service (Josh’s Threat Intelligence Service) so that it can be scanned for malicious activity. This endpoint is called url and would be appended to our root URL: https://joshsthreatintel.example/v1/url.

In additional to the normal http request, when we are posting to this fake service to scan a URL, we must provide in the body of our request the URL we want Josh’s Threat Intelligence Service to scan. An example of this in Python is:

import requests
body = { 'url': 'http://some.malicious.website.com' }
response = requests.post('https://joshsthreatintel.example/v1/url', data=body}

You will find that the above example is common, and preferred by most APIs, but this endpoint could also allow you to GET a URL using query parameters.

A query parameter, at its basic level, is a way to filter or select information from an API. Query parameters are commonly seen when a service allows the user to influence the results returned from an API call, but they can be used to invoke an action; like scanning a URL instead of providing attributes in the body of an http request.

Query parameters are typically appended to each other, which does reduce readability (in my opinion). Here is an example you may see when using query parameters with APIs:

https://joshsthreatintel.example/v1/url?url=some.malicious.website.com 
{website path to api}/{endpoint}?{param=value}

As you can see from the above example, we have the root path of our API and then the endpoint; which is the URL in this case. After the URL endpoint, we can see our query parameter. Our query parameter starts with a ? and is then followed by a parameter name and the value we are using for that parameter.

An API endpoint may allow you to specify several query parameters. For example, here is the URL syntax for retrieving information from RIPE’s (Réseaux IP Européens) REST API for a specific IP address:

https://rest.db.ripe.net/search?source=ripe&query-string={querystring} 
{website path to api}/{endpoint}?{param=value}&{param=value}

You may have noticed the addition of an &, which indicates that there is another query parameter. This also contains a parameter name and a value we are using for this parameter. An API could allow you to have several of these appended to each other. You will need to look at the APIs documentation to see what is allowed for a specific endpoint.

You will find that real life APIs will choose to use query parameters or require that you provide a key value pair of data to the body of a request.

Headers

When using APIs, you may be required to add additional information to a GET or POST or another method to the headers of an http request. This may be an authentication token, specifying the content type, or providing other information.

In basic terms, a header value is metadata used to establish a connection with an API. This metadata is used by the API to authenticate or set the environment for the type of action/request to perform.

It is extremely common that APIs which require authentication will require that you provide a token in your http request headers. This token can be a long term token, or in the case of a delegation based (e.g. OAuth2) authentication scheme a temporal token that must be refreshed after a period of time. Either way, providing this authentication is common for APIs.

You may also need to specify Accept or Content-Type headers along with an Authorization header. In simplest terms, a header is typically in the format of a key value pair. Here is an example in Python of what these two headers may look like:

custom_headers = { 
'Accept': 'application/json', 
'Authorization': 'Bearer {SOME_AUTHENTICATION_TOKEN}' }

These headers are then sent along with our http request to the server. Headers may or may not be required when using an API, and if it is required by an API it will typically be required for all methods used in your http requests.

I hope this helps with understanding the basics of REST APIs. In the next post, I will break down SOAP (Simple Object Access Protocol) APIs which are much more complex than REST APIs.