Initiating a Transfer¶

Moving data within the Globus Ecosystem is performed by submitting a Transfer Task against the Globus Transfer service.

The below examples demonstrate how to do that using a globus sdk TransferClient. They are split into two categories:

Transferring data between two known collections - both source and destination collections are known in advance and are likely be hardcoded into your script.
Transferring data where at least one collection is unknown - either the source or destination collection will be determined at runtime (e.g. by script argument).

We differentiate these examples because certain collections have special auth requirements which must either be defined up front or fixed reactively if omitted. Certain collections (mapped non-high assurance ones) require that a special scope (“data_access”) to be attached to the transfer request to grant Transfer access to that collection’s data. If both collections are known this can be done proactively with a call to the add_app_data_access_scope method. If, however, one or more collections are unknown, the script must reactively solve the ConsentRequired error that is raised when the transfer is submitted.

Transferring data between two known collections¶

Note

The script references two globus hosted “tutorial” collections. Replace these ids & paths with your own collection ids and paths to move your own data.

Note

Some collections require you to attach a “data_access” scope to your transfer request. You should evaluate whether this is necessary for both your source and destination collections and omit the transfer_client.add_app_data_access_scope calls as needed.

A collection requires “data_access” if it is (1) a mapped collection and (2) is not high assurance.

submit_transfer_collections_known.py [download]¶

import globus_sdk
from globus_sdk.globus_app import UserApp

# Tutorial Client ID - <replace this with your own client>
NATIVE_CLIENT_ID = "61338d24-54d5-408f-a10d-66c06b59f6d2"

# Globus Tutorial Collection 1
# https://app.globus.org/file-manager/collections/6c54cade-bde5-45c1-bdea-f4bd71dba2cc
SRC_COLLECTION = "6c54cade-bde5-45c1-bdea-f4bd71dba2cc"
SRC_PATH = "/share/godata/file1.txt"

# Globus Tutorial Collection 2
# https://app.globus.org/file-manager/collections/31ce9ba0-176d-45a5-add3-f37d233ba47d
DST_COLLECTION = "31ce9ba0-176d-45a5-add3-f37d233ba47d"
DST_PATH = "/~/example-transfer-script-destination.txt"


def main():
    with UserApp("my-simple-transfer", client_id=NATIVE_CLIENT_ID) as app:
        with globus_sdk.TransferClient(app=app) as client:
            submit_transfer(client)


def submit_transfer(transfer_client: globus_sdk.TransferClient):
    # Comment out each of these lines if the referenced collection is either
    #   (1) A guest collection or (2) high assurance.
    transfer_client.add_app_data_access_scope(SRC_COLLECTION)
    transfer_client.add_app_data_access_scope(DST_COLLECTION)

    transfer_request = globus_sdk.TransferData(SRC_COLLECTION, DST_COLLECTION)
    transfer_request.add_item(SRC_PATH, DST_PATH)

    task = transfer_client.submit_transfer(transfer_request)
    print(f"Submitted transfer. Task ID: {task['task_id']}.")


if __name__ == "__main__":
    main()

Transferring data where at least one collection is unknown¶

In the case where your script does not know the full set of collections that it will be interacting with, you may need to reactively respond to ConsentRequired errors instead of proactively attaching the “data_access” scope.

This script demonstrates how to do that by:

Enabling auto-redrive of GAREs on the GlobusApp.
Submitting the transfer without any “data_access” scopes.

If the API signals that we did need a data_access scope, the script will prompt the user to login again with the proper consent & re-attempt the transfer submission.

Note

The script references two globus hosted “tutorial” collections. Replace these ids & paths with your own collection ids and paths to move your own data.

Warning

Given that this script reactively fixes auth states, it could involve two user login interactions instead of the one guaranteed by the above proactive approach.

submit_transfer_collections_unknown.py [download]¶

import globus_sdk
from globus_sdk import GlobusAppConfig, UserApp

# Tutorial Client ID - <replace this with your own client>
NATIVE_CLIENT_ID = "61338d24-54d5-408f-a10d-66c06b59f6d2"

# Globus Tutorial Collection 1
# https://app.globus.org/file-manager/collections/6c54cade-bde5-45c1-bdea-f4bd71dba2cc
SRC_COLLECTION = "6c54cade-bde5-45c1-bdea-f4bd71dba2cc"
SRC_PATH = "/share/godata/file1.txt"

# Globus Tutorial Collection 2
# https://app.globus.org/file-manager/collections/31ce9ba0-176d-45a5-add3-f37d233ba47d
DST_COLLECTION = "31ce9ba0-176d-45a5-add3-f37d233ba47d"
DST_PATH = "/~/example-transfer-script-destination.txt"


def main():
    with UserApp(
        "my-simple-transfer",
        client_id=NATIVE_CLIENT_ID,
        config=GlobusAppConfig(auto_redrive_gares=True),
    ) as app:
        with globus_sdk.TransferClient(app=app) as client:
            submit_transfer(client)


def submit_transfer(transfer_client: globus_sdk.TransferClient):
    transfer_request = globus_sdk.TransferData(SRC_COLLECTION, DST_COLLECTION)
    transfer_request.add_item(SRC_PATH, DST_PATH)

    task = transfer_client.submit_transfer(transfer_request)
    print(f"Submitted transfer. Task ID: {task['task_id']}.")


if __name__ == "__main__":
    main()