S3 resource

S3 resource

Direct access to an existing S3-compatible bucket for object reads/writes and the project S3 authorization surface. Choose storage instead when files need project PostgreSQL metadata, owners, audit evidence and managed download links.

Verified against stable Viat 1.1.0. Examples use synthetic values/provider stubs; they were not executed, connected to live services or used to send messages. Start with the resource overview.

Prerequisites and permissions

Create the bucket and narrowly scoped access credentials first. Allow API-node access. Grant ListBucket for Test/list and the required GetObject/PutObject/DeleteObject operations for your prefix. Platform access also needs applicable rules on the project authorization type s3 (resource/key and read/write/delete actions); old permission-string examples are not the current authorization model.

Complete configuration fields

Required describes structural parsing; conditional provider/backend requirements, runtime defaults and limitations follow below. Use null only where the type allows it. Numeric storage bounds do not imply every value is operationally valid. Do not rely on unknown-key dropping to detect typos.

plain

Field	Type / allowed values	Required	Omitted default	Constraints	Meaning
plain.allowed_authorizers	array<string>	no; may be conditional	no explicit default; see rules	—	Deprecated plain S3 compatibility field; ignored by current request authorization.
plain.bucket	string	yes	—	—	Existing bucket name only, not a URL/path.
plain.endpoint	string / null	no; may be conditional	null	—	Configured service endpoint; exact base-vs-operation and normalization rules depend on kind.
plain.path_style	boolean	no; may be conditional	false	—	Use endpoint/bucket/key rather than bucket.endpoint/key.
plain.region	string	yes	—	—	Signing/operation region; provider rules may require it.

secret

Field	Type / allowed values	Required	Omitted default	Constraints	Meaning
secret.access_key	string	yes	—	—	S3-compatible access-key identifier.
secret.secret_key	string	yes	—	—	S3-compatible signing secret.
secret.session_token	string / null	no; may be conditional	null	—	Optional temporary STS token; rotate before expiry, no automatic refresh.

Runtime rules, defaults and limits

bucket is a nonblank name only, at most 63 characters; region is nonblank and is the signing region, not necessarily a geographic AWS name (compatible providers may use auto or oss-cn-hangzhou). access_key and secret_key are nonblank. endpoint is a service origin, not a bucket-specific domain: virtual-host style refuses a host already starting with bucket. Input trims bucket/region, adds https:// to a schemeless endpoint and removes trailing slashes; blank endpoint becomes absent. Absent endpoint resolves to the regional AWS service endpoint. path_style defaults false; enable it for a provider such as MinIO that needs endpoint/bucket/key. session_token is only for temporary STS credentials; it is never auto-refreshed.

plain.allowed_authorizers is a deprecated compatibility field: it is still shape-validated (nonblank, case-insensitively unique names) but ignored at request time. Do not confuse it with repo-file top-level allowed_authorizers, which belongs to access synchronization. Do not build new access policy with either an old permission string or this deprecated plain field.

Permissions, identity and save semantics

Management API requires platform authentication and project membership. List/show require membership; create, PUT, PATCH, Test, refresh, copy and delete require an effective project Owner/Admin/Developer role; the lower-privilege operator role cannot write. Management access grants neither upstream privileges nor runtime s3/file/row-policy access. MCP exposes only its actual listed tools; a resource schema is not a full MCP CRUD interface.

• Create: POST /projects/{slug}/resources, body {name,folder?,kind,plain,secret}. name is 1–64 bytes without /; system is reserved. Omitted/empty folder means root; otherwise [A-Za-z0-9_-]+ segments, at most 8 levels/255 bytes, no leading/trailing/repeated /.
• Identity is project-local (folder,name), not a global bare name. Duplicate names may exist in different folders. Use /resources/{name}?folder=team to locate detail routes explicitly; CLI uses team/name. Script bare names resolve own folder, root, then globally unique name; qualify cross-folder references, using Python getattr(ctx.resources, "team/name") or TypeScript ctx.resources["team/name"], not a property chain.
• PUT fully replaces: resend correct kind, complete plain and all secret values to retain; show masks cannot recover them. PATCH fixes kind; omitted plain preserves it, supplied plain replaces the entire document (not a deep merge); secret overlays only supplied fields, retaining omitted fields. Folder moves recheck references at the new anchor.
• Clear an optional secret by explicit null (only nullable fields) or empty string (only fields allowing empty). Required/conditionally required fields cannot be cleared. To clear optional plain fields, send complete plain without them so defaults/absence take effect. Null is not universal and *** is never an old-secret value.
• $keep is a PATCH/repo sentinel only at top-level plain fields, retaining an existing server value even for numeric/boolean fields. It cannot initialize a new resource or missing field. Nested $keep is not a deep-merge instruction.
• Repo file resources/<folder>/<name>.yaml carries folder through its path, with a bare name inside. viat pull writes string $secret placeholders for secret fields; viat push removes them (also accepts legacy !secret tags) and PATCHes existing resources, never using markers as credentials. Safely supply real required secrets before creating a new resource; a placeholder cannot create credentials. Top-level allowed_authorizers is repo access-sync metadata, not something to insert into plain arbitrarily.

The $secret placeholder differs from a credential reference: {"$secret":"name"} names another secret resource in supported config/cloud positions; HTTP/GraphQL string headers use the entire "$secret:name" value. These are not plaintext credentials. Save/read/export retain references; use resolves them. Targets resolve relative to the consuming resource folder and must already exist with kind secret. No substring interpolation.

Public responses contain resource identifiers, kind, folder, plain and a secret summary, never secret values. Summary *** denotes declared fields, not proof every optional credential is populated. Plain is readable/exportable; secret uses the deployment encrypted backend. Published script/dependency code follows snapshot rules: uses is frozen unless (live), not blanket live on every run. Resource projection, broker operations and explicit live secret IO are different boundaries.

Script secret boundary

1.1.0 compatibility/security limitation: active projection still includes S3 access_key/secret_key/session_token fields. Prefer the brokered operations below, which preserve platform policy; do not extract keys into application code or log/return the resource. Every script dependency must be trusted with this exposure. Managed storage avoids exposing backend credentials.

Minimal configuration: management API / CLI file

This create-request JSON is also valid YAML for resource.yaml and the real CLI command below. Domains/credentials are fixtures; replace them and satisfy prerequisites before real use. Never commit a file containing actual secrets.

{
  "name": "demo",
  "folder": "",
  "kind": "s3",
  "plain": {
    "bucket": "demo-fixtures",
    "region": "us-east-1",
    "endpoint": "https://s3-stub.example.test",
    "path_style": true
  },
  "secret": {
    "access_key": "fixture-access",
    "secret_key": "fixture-secret-not-a-credential"
  }
}

viat project select <slug>
viat resource create --file resource.yaml
viat resource show demo --json

Equivalent API: POST /projects/{slug}/resources, relative to the instance management API root, with the body above. Expect the saved public resource and masked summary, not echoed secrets; an existing fixture name conflicts with 409. For repo sync omit folder and put it in resources/demo.yaml; viat push synchronizes complete plain and non-placeholder secret fields. Saving does not prove every remote operation works.

SDK operations and examples

Python: get(key) → bytes; get_text(key,encoding=“utf-8”) → text; put(key,data,content_type=None); head(key) → {exists,size,content_type,etag}; delete(key); list(prefix=None,max_keys=None). TypeScript uses get/getText/put/head/delete/list, with contentType as put’s third argument. get/put are capped by VIAT_S3_HELPER_MAX_BYTES (default 8 MiB), because data crosses the JSON pipe in base64. download(key,dest_path)/upload(key,src_path,content_type=None) stream through server/worker-shared files without that helper body cap; TypeScript uses destPath/srcPath/contentType. List is one provider page, not an automatic full-bucket walk. Rust’s generic call wrapper does not encode the top-level s3_op envelope; use the binary worker protocol for S3 rather than inventing a typed Rust method.

Both complete scripts declare the same root resource demo. Use only an approved fixture run. Writes, sends, builds and deletion are not dry-run.

Python

# @resource demo
from viat import ctx

def main():
    return ctx.resources.demo.head("reference/hello.txt")

TypeScript

// @resource demo
import { ctx } from "viat";

export async function main() {
  return await ctx.resources.demo.head("reference/hello.txt");
}

Shell has JSON environment projection, not method-decorated ctx.resources; these methods are not shell functions. Binary programs can implement corresponding worker-protocol RPC, not automatically Python/TypeScript methods. The sql lane supports the four SQL kinds; viat_gql only PostgreSQL/MySQL/SQLite; graphql targets remote graphql; llm targets llm. Unsupported lane/kind combinations are not automatically adapted.

Expected result

With an authorized fixture object reference/hello.txt, head reports exists:true plus metadata without downloading it; missing objects report exists:false. This page does not create or contact the fixture bucket.

Test behavior and side effects

Makes a signed ListObjectsV2 GET with max-keys=1, not the outdated bucket HEAD described in some source comments. It needs list permission and can incur a request charge, but does not write an object. Uses a no-proxy client so signed headers are preserved.

Management API: POST /projects/{slug}/resources/{name}/test?folder=; CLI: viat resource test demo. Inspect body ok: connection failures normally still return HTTP 200 {ok:false,error:"…"}, possibly with schema_warnings. Status alone is insufficient; CLI exits nonzero on failure. Run only after approving network/filesystem effects.

Common failures and recovery

resource.s3.empty_bucket/empty_region/empty_access_key/empty_secret_key: supply fields; bad_bucket_length: use the bucket name; endpoint_includes_bucket: use the service origin. HTTP 403: check policy, signing region and expired session_token; 404: check bucket/key. A helper-size error calls for streaming download/upload, not a larger in-memory retry. Platform forbidden requires s3 authz rules, not broader cloud credentials.

Management errors normally use {error,message}. DTO resource.<kind>.* identifiers often occur in the message of 400 bad_request, not the top-level error; they are not a universal enum returned by every runtime helper. For 401 restore platform auth, 403 check effective project role/runtime rules, 404 check folder/name, 409 resolve duplicate identity or resource_in_use references. Distinguish upstream refusals from platform refusals and determine whether work might already have applied before retrying.

Production pitfalls and cleanup

Plan object retention, versioning and orphan cleanup at the bucket. Rotate STS material before expiry. Direct clients using injected credentials bypass the platform authorization/audit path, so prefer broker helpers. Deleting the resource does not delete the bucket or objects; it removes s3 rules naming the resource to prevent accidental grant inheritance.

General removal: disable callers, proxy/CRUD targets and related triggers, migrate dependencies and retain required evidence; then use viat resource delete demo or DELETE on the detail route. For 409 resource_in_use rebind/remove named endpoint holders first; do not bypass references. Removing an asset is not revoking third-party credentials, deleting external data or undoing prior side effects.

Related resources and maintenance evidence

• Managed storage resource
• Cloud API resource
• Resource overview

Pinned contract: contracts/1.1.0/resources.json → resource.s3。

Reviewed implementation (product-repository-relative paths): viat-core/src/types/resource.rs (KindSpec/Plain/Secret); viat-api/src/dto.rs (normalization/validation); viat-api/src/handlers/resources.rs (CRUD/Test); viat-runtime/src/resource_io.rs, resources_json.rs, python.rs, typescript.rs; viat-api/src/runner/broker.rs; viat-cli/src/main.rs and repo/resources.rs; viat-core/src/types/snapshot.rs.