Reference · built on requestConnectorFiles, documents & spreadsheets

Google Cloud Storage

Buckets and objects in Google Cloud Storage through the JSON API, with Pub/Sub notifications for object changes.

Request accessPricing ↗

About

Google Cloud Storage is Google's object store. The Cloud Storage JSON API at storage.googleapis.com/storage/v1/ exposes buckets, objects, access controls, notification configurations, HMAC keys, and managed folders. An object can be up to 5 TiB. A notification configuration publishes OBJECT_FINALIZE, OBJECT_DELETE, OBJECT_ARCHIVE, and OBJECT_METADATA_UPDATE messages to a Pub/Sub topic, with at-least-once delivery.

Through Fibric, an operator subscribes to those messages, reads the object's metadata, and proposes an upload, a metadata patch, or a copy into another bucket. You approve it. A receipt records what changed, why, and how to undo it.

This is a reference listing. It documents what Fibric would read from Google Cloud Storage and what it could propose, based on the vendor's published interfaces. Fibric builds it under a managed deployment when you request it; selecting it here installs nothing.

Inputs

  • Objects under a bucket or prefix through GET /b/{bucket}/o, with generation and metageneration on each object
  • Object metadata and custom metadata through GET /b/{bucket}/o/{object}, up to 8 KiB of custom keys and values
  • Bucket metadata and IAM policy through the Buckets resource
  • Pub/Sub messages for OBJECT_FINALIZE, OBJECT_DELETE, OBJECT_ARCHIVE, and OBJECT_METADATA_UPDATE, with bucketId, objectId, objectGeneration, and eventTime attributes
  • Notification configurations on a bucket, with their topic, event_types, object_name_prefix, and payload_format
  • Managed folders and HMAC keys as the JSON API lists them

Proposed actions

  • Target capability: propose an upload through POST /upload/storage/v1/b/{bucket}/o, with the destination object named first
  • Target capability: propose a metadata change through PATCH /b/{bucket}/o/{object}, guarded by the object's metageneration
  • Target capability: propose a copy or rewrite into another bucket, with the source generation shown
  • Target capability: propose a notification configuration through the Notifications resource's insert method, naming the topic and event types

Proposed actions are target capabilities. Every action runs propose-first and needs a validated deployment and the appropriate permissions.

What you can build

  • Know when a feed lands late

    The operator subscribes to OBJECT_FINALIZE on the prefix a nightly export lands in, compares eventTime to the expected window, and proposes a notice when the file is missing or late.

    With Data Freshness

  • Prove a backup was written

    The operator lists the backup prefix each morning, checks each object's size and generation against the previous day, and proposes a record of what landed and what did not.

    With Backup Proof

  • Watch a bucket grow

    The operator sums object sizes by prefix from the Objects list, tracks the change week over week, and proposes a cleanup for your approval.

    With Storage Growth

Requirements

  • A Google Cloud project with the Cloud Storage JSON API enabled and the buckets you want read
  • A service account granted roles/storage.objectViewer on those buckets, or roles/storage.objectUser where uploads are approved
  • For change notifications, a Pub/Sub topic and a notification configuration on each bucket
Authentication
A Google Cloud service account holding an IAM role such as roles/storage.objectViewer or roles/storage.objectAdmin, sending OAuth 2.0 tokens scoped to devstorage.read_only or devstorage.read_write.

Limits

  • One write per second to the same object, and one metadata update per second; faster writes can return throttling errors
  • At most 100 notification configurations per bucket and 10 per event type
  • Notifications are at-least-once, arrive without an SLA, and undelivered messages can be dropped after 7 days
  • Custom metadata is capped at 8 KiB per object; larger records belong in the object body or a database

Access and pricing

Reference listing. Fibric builds the connector under a managed deployment when you request it. Your quote covers the build, capabilities, usage, and support.

Request Google Cloud Storage ↗

Questions and answers

Which events can Cloud Storage publish to Pub/Sub?
OBJECT_FINALIZE when an object is created or overwritten, OBJECT_METADATA_UPDATE when metadata changes, OBJECT_DELETE when an object is deleted, and OBJECT_ARCHIVE when a live version becomes noncurrent in a versioned bucket. OBJECT_INITIALIZE applies to zonal buckets. Each message carries bucketId, objectId, objectGeneration, and eventTime attributes.
Does Fibric need the account's HMAC keys?
No. The JSON API accepts OAuth 2.0 tokens, so a service account with an IAM role on the bucket is enough. HMAC keys are credentials for the XML API and are not used by this connector.
How large can an object be, and how fast can it change?
An object can be up to 5 TiB. The same object accepts about one write per second and one metadata update per second. A bucket initially supports roughly 1,000 object writes and 5,000 object reads per second before it scales.
Ask about Google Cloud Storage

Ask about the capabilities and requirements in this listing.

For project-specific requirements, contact Fibric.

This connector listing is developed, published, and supported by Fibric. It describes integration with Google Cloud Storage through published interfaces. Third-party names and logos identify the systems an integration connects to; they are the property of their respective owners, who are not affiliated with Fibric and do not sponsor or endorse this listing. Trademark policy