Transfer Data from JGI

The Data Transfer Service (DTS) connects KBase and the Joint Genome Institute (JGI) to seamlessly import data from JGI into your Narratives.

The first version of the DTS has been implemented in a beta state and there may be bugs. If you have trouble with transfers, please contact us at [email protected].

Currently, the DTS only supports isolate genomes which were sequenced by JGI.

Before transferring data from JGI to KBase, you must:

• Get a JGI account and link it to ORCID

• Get a KBase account and link it to ORCID (see the section account linking instructions here for help)

• Use either Chrome, Firefox, or Safari browser

Tutorial on Data Transfer Service

Selecting data

JGI lets users select data from the JGI Genome Portal's sequencing projects and make them transferable to KBase, all in a few simple steps.

The DTS operates as a push from IMG to KBase. You can find genome data to move to KBase using the normal IMG/MER Data Portal. Once you have a selction of genomes to push, add them to your cart.

For more on finding JGI data, see their series of webinar tutorials.

Transferring Data from JGI to KBase

When you have selected your genome(s) of interest from IMG, you can access the data through the Genome Cart.

Select the genomes you want to transfer to KBase using the checkboxes in the IMG Genome Cart. When you have selected the genomes, go to the "Upload & Export & Save" tab.

Provide a name and description for the transfer job, then click "Push to KBase" and it will begin the transfer.

Some JGI data is on tape storage and not immediately accessible. If you choose archived genomes there may be a significant delay before the files are transferred.

When the transfer is complete you will get a confirmation that the files have successfully transferred to KBase Staging Area.

To view the status of your transfer jobs, you can either click the link to "job status" in the confirmation, or go to "My Jobs" under the "My IMG" dropdown.

Importing to KBase

Open the Narrative where you want to import the data. Each transfer job that successfully moves files into KBase will have a folder with a name starting with dts-. Navigate into the folder you want to import.

The top level will contain a file manifest.json and a folder img, if the transfer was successful. You can examine the contents of img to verify you have all the files you wish to import. The file manifest.json is formatted to be usable by the staging service for bulk import. Choose the type "Data Transfer Service Manifest" from the "Import As" dropdown, ensure the file is checked, and click "Import Selected."

The staging service will read the manifest and create a bulk import cell. From here, imports and data management proceed as normally. See Bulk Import Guide for more information.