Source storage | Umnai | Documentation

Storage connectors let Umnai read source data from an Amazon S3 bucket during ingestion.

A connector points to a bucket in your AWS account. Ingestion jobs use the connector together with a data path, schema, ingestion configuration, and compute configuration to create or update datasources.

Source storage and account storage have different roles. A source connector only requires read access to the files you want to ingest. Account storage contains platform-created resources and requires broader permissions.

Both roles can use the same S3 bucket, but dedicated buckets are strongly recommended to keep source files and platform-created resources isolated. See Account storage for account-storage options and permissions.

Grant read-only access

For a bucket used only as source storage, grant Umnai read-only access.

The platform only needs permission to list bucket contents and read objects. If you reuse a provided account-storage bucket, it already requires broader permissions; confirm its policy with Umnai support rather than replacing it with this read-only policy.

In the AWS Console, open S3. Ensure that the selected AWS Region matches the region agreed with Umnai during onboarding.
Select the bucket that contains your source data and open the Permissions tab.
Under Bucket policy, select Edit, then paste the following policy.
Obtain the Umnai platform AWS account ID for your environment from Umnai support.

Replace <ACCOUNT-ID> with the account ID provided to you and <YOUR-BUCKET-NAME> with your bucket’s name:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowSourceObjectRead",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<ACCOUNT-ID>:root"
      },
      "Action": [
        "s3:GetObject"
      ],
      "Resource": "arn:aws:s3:::<YOUR-BUCKET-NAME>/*"
    },
    {
      "Sid": "AllowSourceBucketList",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<ACCOUNT-ID>:root"
      },
      "Action": [
        "s3:ListBucket"
      ],
      "Resource": "arn:aws:s3:::<YOUR-BUCKET-NAME>"
    }
  ]
}

Select Save changes. You should see confirmation that the bucket policy was updated.

Create a storage connector

Create a storage connector after the bucket policy is in place.

POST

/storage-connectors

curl -X POST https://api.hi.umnai.com/storage-connectors \
     -H "Authorization: Bearer <token>" \
     -H "Content-Type: application/json" \
     -d '{
  "bucket_name": "sample-storage-connector-bucket",
  "name": "Sample Storage Connector"
}'

Response

{
  "id": "02910f90b30a5b94ad505efe83280dca",
  "name": "Sample Storage Connector",
  "bucket_name": "sample-storage-connector-bucket",
  "account": {
    "id": "e268443e43d93dab7ebef303bbe9642f",
    "name": "Sample Account"
  },
  "created_at": "2026-03-12T13:55:14.315928Z",
  "created_by": {
    "id": "dad46b2058752ffde5eaf02f1eb6abd5",
    "name": "Sample User"
  }
}

Save the returned connector id. You will use it when retrieving an inferred schema and when creating ingestion jobs.

Retrieve storage connectors

Retrieve storage connectors when you need to reuse an existing connector.

GET

/storage-connectors

curl https://api.hi.umnai.com/storage-connectors \
     -H "Authorization: Bearer <token>"

Response

{
  "data": [
    {
      "id": "c8d98a209dbe25e77ed54e7e12b4fd37",
      "name": "Sample Storage Connector",
      "bucket_name": "sample-storage-connector-bucket",
      "account": {
        "id": "729c654903c0aee752e82c44d02f5918",
        "name": "Sample Account"
      },
      "created_at": "2026-03-12T13:55:14.315928Z",
      "created_by": {
        "id": "eca422469dc3270625bf646df3472a45",
        "name": "Sample User"
      }
    }
  ],
  "pagination": {
    "total_items": 1,
    "items_per_page": 100,
    "current_page": 1,
    "total_pages": 1
  }
}

To inspect a specific connector, retrieve it by id.

GET

/storage-connectors/:storageConnectorId

curl https://api.hi.umnai.com/storage-connectors/storageConnectorId \
     -H "Authorization: Bearer <token>"

Response

{
  "id": "02910f90b30a5b94ad505efe83280dca",
  "name": "Sample Storage Connector",
  "bucket_name": "sample-storage-connector-bucket",
  "account": {
    "id": "e268443e43d93dab7ebef303bbe9642f",
    "name": "Sample Account"
  },
  "created_at": "2026-03-12T13:55:14.315928Z",
  "created_by": {
    "id": "dad46b2058752ffde5eaf02f1eb6abd5",
    "name": "Sample User"
  }
}

Update a storage connector

Update a storage connector when you need to rename it.

PATCH

/storage-connectors/:storageConnectorId

curl -X PATCH https://api.hi.umnai.com/storage-connectors/storageConnectorId \
     -H "Authorization: Bearer <token>" \
     -H "Content-Type: application/json" \
     -d '{
  "name": "Sample Storage Connector"
}'

Response

{
  "id": "02910f90b30a5b94ad505efe83280dca",
  "name": "Sample Storage Connector",
  "bucket_name": "sample-storage-connector-bucket",
  "account": {
    "id": "e268443e43d93dab7ebef303bbe9642f",
    "name": "Sample Account"
  },
  "created_at": "2026-03-12T13:55:14.315928Z",
  "created_by": {
    "id": "dad46b2058752ffde5eaf02f1eb6abd5",
    "name": "Sample User"
  },
  "updated_at": "2026-03-12T13:55:14.231528Z"
}

Retrieve an inferred schema

After creating a connector, you can retrieve an inferred data schema from a path inside the bucket.

Use this when you want Umnai to inspect source data and suggest the schema to refine before ingestion.

GET

/storage-connectors/:storageConnectorId/schema

curl -G https://api.hi.umnai.com/storage-connectors/storageConnectorId/schema \
     -H "Authorization: Bearer <token>" \
     -d path=path

Response

{
  "fields": [
    {
      "data_type": {
        "data_type": "NUMBER",
        "format": "INT64"
      },
      "designation": "FEATURE",
      "name": "Sample Field",
      "semantic_type": "CONTINUOUS",
      "vector": false
    },
    {
      "data_type": {
        "data_type": "STRING",
        "format": null
      },
      "designation": "FEATURE",
      "name": "Sample Field",
      "semantic_type": "CATEGORICAL",
      "vector": false
    },
    {
      "data_type": {
        "data_type": "STRING",
        "format": null
      },
      "designation": "FEATURE",
      "name": "Sample Field",
      "semantic_type": "CATEGORICAL",
      "vector": false
    },
    {
      "data_type": {
        "data_type": "STRING",
        "format": null
      },
      "designation": "FEATURE",
      "name": "Sample Field",
      "semantic_type": "CATEGORICAL",
      "vector": false
    },
    {
      "data_type": {
        "data_type": "STRING",
        "format": null
      },
      "designation": "FEATURE",
      "name": "Sample Field",
      "semantic_type": "CATEGORICAL",
      "vector": false
    },
    {
      "data_type": {
        "data_type": "STRING",
        "format": null
      },
      "designation": "FEATURE",
      "name": "Sample Field",
      "semantic_type": "CATEGORICAL",
      "vector": false
    },
    {
      "data_type": {
        "data_type": "STRING",
        "format": null
      },
      "designation": "FEATURE",
      "name": "Sample Field",
      "semantic_type": "CATEGORICAL",
      "vector": false
    },
    {
      "data_type": {
        "data_type": "STRING",
        "format": null
      },
      "designation": "FEATURE",
      "name": "Sample Field",
      "semantic_type": "CATEGORICAL",
      "vector": false
    },
    {
      "data_type": {
        "data_type": "NUMBER",
        "format": "INT64"
      },
      "designation": "FEATURE",
      "name": "Sample Field",
      "semantic_type": "CONTINUOUS",
      "vector": false
    },
    {
      "data_type": {
        "data_type": "NUMBER",
        "format": "INT64"
      },
      "designation": "FEATURE",
      "name": "Sample Field",
      "semantic_type": "CONTINUOUS",
      "vector": false
    },
    {
      "data_type": {
        "data_type": "NUMBER",
        "format": "INT64"
      },
      "designation": "FEATURE",
      "name": "Sample Field",
      "semantic_type": "CONTINUOUS",
      "vector": false
    },
    {
      "data_type": {
        "data_type": "STRING",
        "format": null
      },
      "designation": "FEATURE",
      "name": "Sample Field",
      "semantic_type": "CATEGORICAL",
      "vector": false
    },
    {
      "data_type": {
        "data_type": "NUMBER",
        "format": "INT64"
      },
      "designation": "FEATURE",
      "name": "Sample Field",
      "semantic_type": "CONTINUOUS",
      "vector": false
    }
  ]
}

Delete a storage connector

Delete a storage connector when it should no longer be used for ingestion.

DELETE

/storage-connectors/:storageConnectorId

curl -X DELETE https://api.hi.umnai.com/storage-connectors/storageConnectorId \
     -H "Authorization: Bearer <token>"

Deleting a storage connector does not change your S3 bucket policy. If you also want to revoke bucket access, remove the corresponding S3 bucket policy statement in AWS.