Storage

Storage connectors let Umnai read source data from an Amazon S3 bucket during ingestion.

A connector points to a bucket in your account. Ingestion jobs use the connector together with a data path, schema, ingestion configuration, and compute configuration to create or update datasources.

Grant read-only access

Grant Umnai read-only access to the S3 bucket that contains your source data.

The platform only needs permission to list bucket contents and read objects.

1. In the AWS Console, open S3. Ensure your region is eu-west-1 (Ireland).

2. Select your bucket and open the Permissions tab.

3. Under Bucket policy, select Edit, then paste this policy, replacing <YOUR_BUCKET_NAME> with your bucket’s name:

   1
   {
   2
     "Version": "2012-10-17",
   3
     "Statement": [
   4
       {
   5
         "Sid": "AllowObjectControl",
   6
         "Effect": "Allow",
   7
         "Principal": {
   8
           "AWS": "arn:aws:iam::794038224470:root"
   9
         },
   10
         "Action": ["s3:GetObject"],
   11
         "Resource": "arn:aws:s3:::<YOUR_BUCKET_NAME>/*"
   12
       },
   13
       {
   14
         "Sid": "AllowListBucket",
   15
         "Effect": "Allow",
   16
         "Principal": {
   17
           "AWS": "arn:aws:iam::794038224470:root"
   18
         },
   19
         "Action": ["s3:ListBucket"],
   20
         "Resource": "arn:aws:s3:::<YOUR_BUCKET_NAME>"
   21
       }
   22
     ]
   23
   }

4. Save. You should see a confirmation banner that the policy was updated.

Create a storage connector

Create a storage connector after the bucket policy is in place.

1
curl -X POST https://api.hi.umnai.com/storage-connectors \
2
     -H "Authorization: Bearer <token>" \
3
     -H "Content-Type: application/json" \
4
     -d '{
5
  "bucket_name": "sample-storage-connector-bucket",
6
  "name": "Sample Storage Connector"
7
}'

1
{
2
  "id": "02910f90b30a5b94ad505efe83280dca",
3
  "name": "Sample Storage Connector",
4
  "bucket_name": "sample-storage-connector-bucket",
5
  "account": {
6
    "id": "e268443e43d93dab7ebef303bbe9642f",
7
    "name": "Sample Account"
8
  },
9
  "created_at": "2026-03-12T13:55:14.315928Z",
10
  "created_by": {
11
    "id": "dad46b2058752ffde5eaf02f1eb6abd5",
12
    "name": "Sample User"
13
  },
14
  "updated_at": null
15
}

Save the returned connector id. You will use it when retrieving an inferred schema and when creating ingestion jobs.

Retrieve storage connectors

Retrieve storage connectors when you need to reuse an existing connector.

1
curl https://api.hi.umnai.com/storage-connectors \
2
     -H "Authorization: Bearer <token>"

1
{
2
  "data": [
3
    {
4
      "id": "ea8b745ead39ec442fc3b4fba359bf98",
5
      "name": "Sample Organisation",
6
      "created_at": "2026-03-11T16:04:19.749000Z",
7
      "updated_at": null,
8
      "status": "ACTIVE"
9
    }
10
  ],
11
  "pagination": {
12
    "total_items": 1,
13
    "items_per_page": 10,
14
    "current_page": 1,
15
    "total_pages": 1
16
  }
17
}

To inspect a specific connector, retrieve it by id.

1
curl https://api.hi.umnai.com/storage-connectors/{storageConnectorId} \
2
     -H "Authorization: Bearer <token>"

1
{
2
  "data": [
3
    {
4
      "id": "ea8b745ead39ec442fc3b4fba359bf98",
5
      "name": "Sample Organisation",
6
      "status": "ACTIVE",
7
      "created_at": "2026-03-11T16:04:19.749000Z",
8
      "updated_at": null
9
    }
10
  ],
11
  "pagination": {
12
    "total_items": 1,
13
    "items_per_page": 10,
14
    "current_page": 1,
15
    "total_pages": 1
16
  }
17
}

Update a storage connector

Update a storage connector when you need to rename it.

1
curl -X PATCH https://api.hi.umnai.com/storage-connectors/storageConnectorId \
2
     -H "Authorization: Bearer <token>" \
3
     -H "Content-Type: application/json" \
4
     -d '{
5
  "name": "Sample Storage Connector"
6
}'

1
{
2
  "id": "02910f90b30a5b94ad505efe83280dca",
3
  "name": "Sample Storage Connector",
4
  "bucket_name": "sample-storage-connector-bucket",
5
  "account": {
6
    "id": "e268443e43d93dab7ebef303bbe9642f",
7
    "name": "Sample Account"
8
  },
9
  "created_at": "2026-03-12T13:55:14.315928Z",
10
  "created_by": {
11
    "id": "dad46b2058752ffde5eaf02f1eb6abd5",
12
    "name": "Sample User"
13
  },
14
  "updated_at": "2026-03-12T13:55:14.231528Z"
15
}

Retrieve an inferred schema

After creating a connector, you can retrieve an inferred data schema from a path inside the bucket.

Use this when you want Umnai to inspect source data and suggest the schema to refine before ingestion.

1
curl https://api.hi.umnai.com/storage-connectors/{storageConnectorId}/schema \
2
     -H "Authorization: Bearer <token>"

1
{
2
  "data": [
3
    {
4
      "id": "ea8b745ead39ec442fc3b4fba359bf98",
5
      "name": "Sample Organisation",
6
      "status": "ACTIVE",
7
      "created_at": "2026-03-11T16:04:19.749000Z",
8
      "updated_at": null
9
    }
10
  ],
11
  "pagination": {
12
    "total_items": 1,
13
    "items_per_page": 10,
14
    "current_page": 1,
15
    "total_pages": 1
16
  }
17
}

Delete a storage connector

Delete a storage connector when it should no longer be used for ingestion.

1
curl -X DELETE https://api.hi.umnai.com/storage-connectors/storageConnectorId \
2
     -H "Authorization: Bearer <token>"