Transferring Data with Globus
A how-to on large file transfer into KBase.
Transfer large files (~20GB and more) to KBase using Globus. Drag & drop from your local computer works for many files, but there is a size limit dependent on your computer and browser.
Globus is a data management and file transfer system that can facilitate bulk transfer of data (large data files or a large number of files) between two endpoints. The endpoints that apply here are KBase, JGI, and your local computer. The KBase endpoint is called “KBase Bulk Share,” and JGI has their own way to link to Globus. To do any transfer using Globus, you will need a Globus account.
Your local computer can become a Globus endpoint with a Globus Connect Personal Endpoint. This is necessary for transfers to or from your local computer.
Once you have a Globus account, link to your KBase account to facilitate file transfers. See Linking Accounts for further guidance.
A quick 4-min video on using Globus with KBase
To initiate a transfer using Globus, click 'Or upload to this staging area by using Globus Online' in the Import tab of the Data Browser below the Staging Area.
This brings you to a page that looks like this:
The panel on the right should be set to “KBase Bulk Share,” which directs to the Staging Area. In the example image above, the share point is at the root directory, and the user needs to click on their individual directory. If needed, Globus can be used to create new directories in your Staging Area for your data transfer and to delete staging files when you are done importing into KBase.
Set the left panel to the location of the data you want to transfer (i.e. you could choose the JGI endpoint they send via email). On the KBase endpoint, if you don’t see a directory with your name as in the example above, add /username to the Path for the KBase endpoint (/user/ is used in the example above).
You are then be able to drag data files or folders from the left endpoint to the right one, inside the Globus interface, and the files will appear in your KBase Staging Area.
Your transfer request will be submitted to Globus. If there are network or other problems, Globus will retry several times before giving up. Globus will send an email when the transfer is complete. In the KBase Staging Area, you may need to click on the refresh icon (two encircling arrows to the left of the user name) to update the Import tab.
Some government institutions have a firewall between users and the outside world. In an effort to increase security, many web links can be blocked, including Globus traffic. If you see a message that ends with “Details: 500 Command failed….”, a firewall may be the cause.
Another common error seen when using Globus shows and "No ACL rules" error on the KBase Bulk Share endpoint in the Globus UI. The most common cause for this error is multiple Globus accounts where the wrong account is linked to KBase. Be sure to log out of all Globus accounts, then link the correct account at https://narrative.kbase.us/#/auth2/account. If you still get this error, contact the help desk.
If you are a JGI user, you can transfer public genome reads and assemblies (and your private data and annotated genomes) from JGI to your KBase account—see this page for instructions.
Last modified 6mo ago