Separate from our API file upload pathway , Unit21 also allows you to connect an S3 bucket with our data ingestion system. Once configured, you will be able to link each stream to a folder within the bucket, and files uploaded to that folder will be loaded and processed by the associated stream automatically.
The file size limit is 300 MB for this upload channel.
S3 bucket name restriction
For Unit21 to recognize your S3 bucket, it must be named with the appropriate prefix. This will look like
unit21-ffip-{environment}
The environment field is determined by which Unit21 environment will be receiving the data, some common environments are
- US Sandbox1 -> sandbox1
- US Prod1 -> prod1
- US Prod2 -> prod2
If you don't see your intended environment above, please reach out to your customer success manager.
Permissions required
Unit21 must be able to read data from your S3 bucket. To that end, we ask that you grant the following permissions for your S3 bucket to the Unit21 AWS account, ARN: arn:aws:iam::837700644609:root
s3:ListBuckets3:GetObject
How to set up a bucket
Once you've created a bucket according to the naming convention and have granted the appropriate permissions to Unit21, you can connect your bucket in the Unit21 UI. Under the Data Managementtab, navigate to the Direct Uploads button in the top right corner.
Next enter your S3 bucket's name into the prompt, and click + Connect. When clicked, Unit21 will check if we have the appropriate permissions to your bucket, and if permissions are correct, you should see the bucket appear in the panel
Example for a successful connection:
If you choose the Edit stream option for any stream
There will be a new section for Direct uploads, with a drop down to select the S3 bucket previously registered.
In this section, select the appropriate bucket, and the appropriate folder containing files that should be processed by that particular stream. Finish by clicking the Update Stream button in the corner.
For example, if the files for the currently configured stream are located under exports/data/entity, then the file path should be exports/data/entity/. (Note the trailing forward slash).
If you've configured the bucket with the proper permissions and the file path, then files should be automatically start copying from your bucket within the hour.
Notifications
You can also configure notifications to notify yourself when an FFIP copy job fails.
This can be found under Profile -> Notifications -> General Notifications.
Common reasons why the copy job fails includes, but are not limited to:
- AWS being down or having an outage
- Missing proper permissions to the Unit21 role
