GCS Downloader

A GCSDownloader is an object which handles connections to an HTTPS-enabled collection and single file downloads over HTTPS.

It primarily features two APIs:

  1. Initialization and use as a context manager

  2. GCSDownloader.read_file() to get a single file by URL

class globus_sdk.experimental.gcs_downloader.GCSDownloader(app, *, gcs_client=None, transfer_client=None, transport=None)[source]

An object which manages connection and authentication state to enable HTTPS downloads from a specific Globus Connect Server collection.

The initial request to read a file features support for determining authentication requirements dynamically, and subsequent requests will reuse that authentication data.

Using a single GCSDownloader to access distinct collections is not supported. A separate downloader should be used for each collection.

Downloaders may be used as context managers, in which case they automatically call their close() method on exit:

>>> with GCSDownloader(app) as downloader:
>>>     print(downloader.read_file(url))
Parameters:

  • app (globus_sdk.GlobusApp) – The GlobusApp used to authenticate calls to this server.

  • gcs_client (GCSCollectionClient | None) – The underlying client used for the file read request. Typically omitted. When not provided, one will be constructed on demand by the downloader.

  • transfer_client (globus_sdk.TransferClient | None) – A client used when detecting collection information. Typically omitted. When not provided, one will be constructed on demand by the downloader.

  • transport (globus_sdk.transport.RequestsTransport | None) – A transport for the downloader, used for authentication sniffing operations. When a client is built by the downloader it will inherit this transport.

close()[source]

Close all resources which are owned by this downloader.

read_file(file_uri: str, *, as_text: Literal[True]) → str[source]
read_file(file_uri: str, *, as_text: Literal[False]) → bytes
read_file(file_uri: str) → str

Given a file URI on a GCS Collection, read the data.

Parameters:

  • file_uri – The full URI of the file on the collection which is being downloaded.

  • as_text – When True, the file contents are decoded into a string. Set to False to retrieve data as bytes.

Caution

The file read is done naively as a GET request. This may be unsuitable for very large files.

Example Usage

In this example, a GCSDownloader with a GlobusApp completely handles the authentication process for an HTTPS file download, which is then printed to stdout.

import globus_sdk
from globus_sdk.experimental.gcs_downloader import GCSDownloader

# SDK Tutorial Client ID - <replace this with your own client>
CLIENT_ID = "61338d24-54d5-408f-a10d-66c06b59f6d2"

# this example is a path on the Globus Tutorial Collections
FILE_URL = (
    "https://m-d3a2c3.collection1.tutorials.globus.org/home/share/godata/file2.txt"
)

with globus_sdk.UserApp("gcs-downloader-demo", client_id=CLIENT_ID) as app:
    with GCSDownloader(app) as downloader:
        print(downloader.read_file(FILE_URL))

Note

This example will require you to log in twice the first time you run it. This is normal behavior and is a consequence of needing to be logged in to discover authentication requirements.