Child pages
  • A Quick Primer on Writing API Scripts
Skip to end of metadata
Go to start of metadata


This article documents how you should approach developing a custom script that uses the Squore REST API introduced in Squore 19.0.

Concept

Our objective is to export all findings from all projects in a specific group into a CSV file.

This operation is not available from the web interface, which currently restricts Findings export to project level at most.


The Findings tab is inactive at group level in the Explorer

The current workaround involved clicking on every project in the C group to export the findings for each project manually. This operation can now be scripted with the use of the Squore REST API.

Finding the API Documentation

Before writing your script, you'll need to get familiar with the API endpoints available. 

The API documentation is available directly from the Squore web interface from ? > API Documentation

The link to the API Documentation in the Help menu


The API endpoints in the API Documentation


For each API endpoint, the documentation lists:

  • A brief description of the information that can be retrieved
  • The supported parameters supported by the request
  • The possible response codes and sample response data

Additionally, you can test each API endpoint directly on your server from the documentation page using the Try it out button. You'll have to create an API token to test the requests, which we'll cover later.

The documentation of the GET /projects API endpoint

Planning Your Script's Workflow


Based on the available API endpoints, building a script that exports all findings for projects in a specific group can be modelled as shown below:


Testing Out API Requests


Before you can test individual requests to the REST API, you'll need a valid token to test out with data from your Squore Server.

Go to your account settings to create a token: specify a name and click Add to create your API token.


Creating an API token for the user demo

Copy the generated token (it can only be retrieved at this time) and go to the API documentation to click the Authorize button.

The Authorize button for interacting with the API on the Squore Server

Click Authorize to enter your API token and confirm.

The token generated earlier is used t authorise your API requests

If the token is accepted, you can close the autorisation popup.

You are now authenticated and can play with the Squore API!



You can now test each API endpoint and get real data back when you click the Try it out button in the documentation

Here is an example of the data shown when testing the GET /projects endpoint, which retrieves basic information about existing projects on the server.


You can try out the other API endpoints to see how to retrieve project artefacts (GET /projets/{id}/artefacts) or artefact findings (GET /artefacts/{id}/findings) on the same page.


Putting it All Together


For the purpose of this example, we will write a shell script that queries the API with curl, receives data in JSON and converts it to CSV. jq is a simple JSON command line processor that can help us parse responses from the API and convert the data in the expected format.

jq has a powerful filter and query management syntax, which you can try out at https://jqplay.org/


Your script will use curl commands that will pass the following headers for every request:

  • the API token
  • The response content type

These are declared as variables at the top of the script:

#!/bin/sh

# Set some variables
API_URL="http://localhost:8180/SQuORE_Server/api"
API_TOKEN="Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1aWQiOjIsImlhdCI6MTU1MDIyMzc5MjU2Nn0=.YlvjJszAo6jf85BtlkGgcx5TnT5qegdW-RXHJ-HBlcQ="
CONTENT_TYPE="accept: application/json"


# Clear any existing findings export file
cat /dev/null > findings.csv


Retrieving all the projects is a simple call to GET /projects, whose output is piped to jq to keep the ones in the C group and extract their IDs into a variable:

# Build a list of projects you will export findings from.
# This is done by getting information about all projects from the Squore API, 
# piping the output to jq to filter the ones in the "C/" group,
# and storing their IDs in a $PROJECT_IDS variable

PROJECT_IDS=`curl -s -H "$CONTENT_TYPE" -H "$API_TOKEN" -X GET $API_URL/projects | jq '.content | .[] | select(.group == "C/") | .id'`



For each project IDs, you can export all findings (GET /artefacts/{id}/findings) after finding the ID of the root artefact (GET /projects/{id}):

# Iterate over the project IDs to export findings for each project in the "C/" group
# First, get the ID of the root artefact of the project (the APPLICATION node)
# Then, export findings for the root artefact and its descendants,
# pipe content into jq to retrieve selected information,
# iterating over each occurrence (i.e. finding location),
# and finally convert JSON data to CSV.

for projectId in $(echo "$PROJECT_IDS"); do
	_exportFindings() {
		ROOT_ARTEFACT_ID=`curl -s -H "$CONTENT_TYPE" -H "$API_TOKEN" -X GET "$API_URL/projects/$projectId" | jq '.artefactId'`
		curl -s -H "$CONTENT_TYPE" -H "$API_TOKEN" -X GET "$API_URL/artefacts/$ROOT_ARTEFACT_ID/findings" | jq --raw-output '.content[] |[._links.self, .tool, .definition.id, .definition.name, .definition.description, .status] + (.locations[] | [.artefact.type.id, .artefact.name, .location, .artefact.path]) | @csv'
	}
	echo $(_exportFindings >> findings.csv)
done


Improving The Script


You may have noticed that the initial request to GET /projects actually returns the root artefact ID. You can therefore simplify your script to retrieve the artifact Id in the first curl command and save one API call per project:

#!/bin/sh

API_URL="http://localhost:8180/SQuORE_Server/api"
API_TOKEN="Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1aWQiOjIsImlhdCI6MTU1MDIyMzc5MjU2Nn0=.YlvjJszAo6jf85BtlkGgcx5TnT5qegdW-RXHJ-HBlcQ="
CONTENT_TYPE="accept: application/json"

cat /dev/null > findings.csv

ROOT_ARTEFACT_IDS=`curl -s -H "$CONTENT_TYPE" -H "$API_TOKEN" -X GET $API_URL/projects | jq '.content | .[] | select(.group == "C/") | .artefactId'`

for rootArtefactId in $(echo "$ROOT_ARTEFACT_IDS"); do
	_exportFindings() {
		curl -s -H "$CONTENT_TYPE" -H "$API_TOKEN" -X GET "$API_URL/artefacts/$rootArtefactId/findings" | jq --raw-output '.content[] |[._links.self, .tool, .definition.id, .definition.name, .definition.description, .status] + (.locations[] | [.artefact.type.id, .artefact.name, .location, .artefact.path]) | @csv'
	}
	echo $(_exportFindings >> findings.csv)
done


The final CSV contains all 5000 findings in CSV format, with a URL reference to get the full information:

"http://localhost:8180/SQuORE_Server/api/findings/41","SQUORE","R_DYNMEMALLOC","Dynamic Memory Allocation shall not be used","Dynamic heap memory allocation shall not used.","LEGACY_SYSTEM","C_FUNCTION","machine_read_file()","Line: 12","apps/machine.c"
"http://localhost:8180/SQuORE_Server/api/findings/45","SQUORE","R_COMPOUNDIFELSE","Missing compound if","An if (expression) construct shall be followed by a compound statement.The else keyword shall be followed by either a compound statement, or another if statement.","OPEN","C_FUNCTION","consistent()","Line: 77","core/util.c"
...
"http://localhost:8180/SQuORE_Server/api/findings/247","SQUORE","R_NOCFTC_MODULES","Cloned Algorithmic","[No description] Cloned Algorithmic: There shall be no algorithmic cloning","OPEN","C_FUNCTION","help(int*)","Line: 98","apps/player.c"
"http://localhost:8180/SQuORE_Server/api/findings/247","SQUORE","R_NOCFTC_MODULES","Cloned Algorithmic","[No description] Cloned Algorithmic: There shall be no algorithmic cloning","OPEN","C_FUNCTION","help_robot(int*)","Line: 98","apps/robot.c"
...
"http://localhost:8180/SQuORE_Server/api/findings/7893","SQUORE","R_COMPOUNDIFELSE","Missing compound if","An if (expression) construct shall be followed by a compound statement.The else keyword shall be followed by either a compound statement, or another if statement.","RELAXED_DEROGATION","C_FUNCTION","machine_plays()","Line: 370","apps/machine.c"
"http://localhost:8180/SQuORE_Server/api/findings/8137","SQUORE","R_COMPOUNDIFELSE","Missing compound if","An if (expression) construct shall be followed by a compound statement.The else keyword shall be followed by either a compound statement, or another if statement.","OPEN","C_FUNCTION","find_digit(char[7])","Line: 151","core/base.c"





Integrating Your Work in the Squore UI

Working from the command line works fine if you want to retrieve information during a continuous integration job or schedule the export, but for some users, it may be more convenient to launch this export directly from the Squore web interface.

This can be done by writing a custom export definition, which will be available from the Reports tab in the Explorer.

The Reports tab displays your custom export definitions

To learn more about creating a custom API definitions that reuses the logic of the API requests from your bash script consult the Getting Started Guide.

  • No labels