Batch or bulk uploading files into CORE is really no different that uploading a single file or two with the exception of processing time.
In this article:
Recommendations and Best Practices for Bulk Uploads
Size of Batches or Large Files
There's no one combination for the best way to bulk upload into CORE, but consider the following:
Batch sizes should be kept to a maximum of 400-500 files per upload batch and not exceed a total of 20-25gb.
Files upwards of 100gb should be uploaded individually.
If you have 10 files that are 20gb or more in size, break them down into 2-3 batches.
If you have slow internet (below 100mbps) at your location, break batches down into smaller groups totaling no more than 20gb per batch. Files will transfer; however, anticipate that this could take several hours for the files to transfer depending on your internet bandwidth.
If you have slow internet and very large files (20gb+), upload the files individually.
Batches of files can be uploaded in quick succession of each other, so that the uploads progress concurrently.
When uploading large files that are individually 40gb+, please alert [email protected], so they can monitor your uploads and ensure your system's resources are scaling properly.
Upload Tools: How to Decide Between CORE Browser or Aspera Connect with CORE
CORE Browser Upload (HTTPS / Simple)
HTTPS uploads via your computer browser are best suited for batches under 3gb or less. Anything above, you should use Aspera.
Aspera Connect Uploads with CORE
Aspera is a content transfer manager using a technology that adaptively leverages the bandwidth of your computer and your internet in order to maximize the transfer speeds of your files.
All large files and large batches of files should be transferred using Aspera.
Aspera has a maximum upload cap of 5,000 files at a time.
Upload a Batch of Files
To bulk upload a set of files,
Choose your upload method -- HTTPS or Aspera -- by selecting the More menu next to the Upload icon and clicking your desired tool.
Then, select the Upload icon.
A File dialog window will open. Select as many files as you would like to upload.
To select multiple files, hold Ctrl (Command on macOS) as you click.
To select a range of files, hold Shift as you click to extend your selection.
To select all files, hold Ctrl or Command + A
Once your files are selected, choose Open and the files will begin their upload into CORE.
Alternatively, you can drag and drop your large individual assets or file batches into existing Asset Groups. When you drag and drop:
The upload will upload via the tool currently selected - either HTTPS or Aspera
With the UI may take a moment to respond and expand the drag and drop areas of nearby Asset Groups at the same time, so take a moment before dropping the files into an asset group to ensure you're dropping them into the right one.
Batch Tagging Files
As your files upload, you can tag them and submit them to File Search even before they've completed uploading and processing. There are a couple ways to batch tag files.
Batch Tag All Files in Your Upload With the Same Tags
In this case, you'd follow the standard Tagging process - How to Tag Files in CORE:
Select the Domain, if one isn't already selected
Fill out the Required Tag Panel
Enter your own tag values as needed
Fill out additional fields in the other Tag Groups, if it's relevant
Submit your tags
From there you can go to File Search, or share your files as a package, etc.
Apply Different Tag Sets to the Bulk Upload
To apply different tags to different files in the upload, you follow the same steps with a couple additional ones:
Select the first set of files you want to tag via the checkbox to the left of each file
Next, go through the tagging steps:
Select the Domain
Fill out the Required Tags
Add tags from any other Tag Groups
Hit Submit
Now, you'll see an additional button called Categorize Files. Select that to be taken back to the tagging panel.
Unselect your previously tagged files. NOTE: If you choose the select all checkbox at the top and then unselect all in the same checkbox, you can quickly clear out your selected files.
Now select files that you want to tag in a different way, and go through the same steps.
Repeat until all your files are tagged.
When all your files are tagged, the Categorize Files button will disappear after hitting Submit, and you can proceed to File Search, Sharing, or your Inbox.
Batch Tagging Hacks & Tips
If you're uploading massive batches of files that require different tags, it will be faster to upload the files as separate batches and tag the entire group with one set of tags.
If you are tagging with different sets of tags, you can see which files you've already tagged by their Categorized status of Pending or Categorized.
If you're bulk uploading files to the same Asset Group (using the same Structural tags each time), use the Load Last Categorized feature or the Save Tags Template feature to quickly load a set of tags.
Troubleshooting Batch Uploads and Bulk Tagging
If you've uploaded a large batch of files or just a big file and you're not seeing the upload progress, give it a few minutes. Sometimes a large file looks like it's not uploading when it actually is. If you still see no movement after several minutes, contact [email protected]
If you accidentally hit the Exit button after selecting your first group of tags and you're not done tagging all the files in your upload, go to the Transfers page and Uploads. Select the link on your batch that says # of files require categorization. This will reopen the upload page with tagging panel.
You dragged and dropped your files into an Asset Group, but you're not seeing them there. Check nearby Asset Groups, meaning the most recent asset group in your system. It's possibly you dropped your files into the wrong Asset Group.
To update your files, go into that Asset Group.
Open the Asset Details panel.
Check all the files you need to move.
Select Edit Details.
Tag your files with the proper tags.
Select Save or Submit.
Refresh the page. Your files should no longer appear in this asset group.
Go out to File Search and find the correct Asset Group for your files.