Uploading Files Programmatically

You can upload files programmatically if you want to create an integration, or you need to upload files up to 300 MB in size.

How to upload files programmatically

Generating a pre-signed URL

To generate a pre-signed URL, first hit the following endpoint: https://sandbox1-api.unit21.com/v1/imports/pre-signed-url/create in the following format. Note that you must specify the stream name (specified in the setup above) and the file you wish to upload.

curl --request POST \
     --url https://sandbox1-api.unit21.com/v1/imports/pre-signed-url/create \
     --header 'accept: application/json' \
     --header 'content-type: application/json' \
     --header 'u21-key: my_favorite_u21_api_key' \
     --data '
{
  "stream_name": "stream_name",
  "file_name": "file_you_want_to_upload.csv"
}
'

Your response will look something like this:

{
  "file_id":"unit21_file_id",
  "form_fields":
   {"AWSAccessKeyId":"access_key_id",
    "key":"pointer_to_file_location_in_s3",
    "policy":"string_indicating_policy",
    "signature":"your_unique_signature"
   },
 "url":"https://file-uploads.s3.amazonaws.com/"
}

Uploading a file using the pre-signed URL

You can then take information from the response: AWSAccessKeyId, key, policy, signature, url and upload your file. Your request will look like this:

curl --request POST \
     --url https://file-uploads.s3.amazonaws.com/ \
     --header 'content-type: multipart/form-data' \
     --form key='pointer_to_file_location_in_s3' \
     --form AWSAccessKeyId='access_key_id' \
     --form policy='string_indicating_policy' \
     --form signature='your_unique_signature' \
     --form file=@file_you_want_to_upload.csv

Once you've uploaded and received a successful response (HTTP Code 204), you can check the status of the file in the UI or via API query.

Getting the status of an uploaded file

You can get the uploaded file status. Note that you must retain the file id returned while generating your pre-signed URL: unit21_file_id. Your request will look like this:

curl --request GET \
     --url https://sandbox1-api.unit21.com/v1/imports/unit21_file_id \
     --header 'accept: application/json' \
     --header 'u21-key: my_favorite_u21_api_key'

You'll get a descriptive JSON response indicating the status of file, including the following details:

file_id, file_name, file_type, date_created, process_start_time, process_end_time, successful_count, failure_count, status, failure_code

The possible statuses are:

PENDING_UPLOAD   # pre-signed URL has been generated but file has not been uploaded yet
ADDED   # file has been added manually through the UI
QUEUED  # file is waiting to be picked up for processing 
PROCESSING  # file is being processed
FINISHED  # file has finished processing
FAILED  # file failed to process
DELETED  # file has been deleted

Getting a list of uploaded files

You can get a list of uploaded files. Note that you can also include additional filters, such as created_before, created_after, statuses, and stream_names to in the JSON body to narrow your search.

The endpoint also supports offset and pagination, specified by offset and limit. By default, the limit is 1000.

  • created_before and created_after are type UNIX_TIMESTAMP.
  • statuses and stream_names are type list.

A sample request will look like this:

curl --request POST \
     --url https://sandbox1-api.unit21.com/v1/imports/list \
     --header 'accept: application/json' \
     --header 'content-type: application/json' \
     --header 'u21-key: my_favorite_u21_api_key' \
     --data '
{
  "created_after": UNIX_TIMESTAMP,
  "created_before": UNIX_TIMESTAMP,
  "statuses": statuses as a list, example: ["QUEUED", "FINISHED"],
  "stream_names": stream names as a list, example: ["stream1", "stream2"],
  "offset": 1,
  "limit": 1000
}
'

You'll get a descriptive JSON response similar to what's listed in Getting the status of an uploaded file, and also get the count of relevant files.