Instructions for how to access bulk data from Analytic Index
Data Availabiliity
Location: Google Cloud Storage
The data for the prior month will be available all month long to access again if needed. At the end of the month the data will be updated to the new previous month’s data. Any historical data persistence would need to be downloaded and stored by the client.
Example:
The January 1 - 31st data will be available during the month of February and will be replaced with February data starting March 1.
Bulk data can be accessed from Google Cloud Storage. See below for connection information:
Prerequisites
- A Google Cloud Platform account
- A Project with billing setup
- 1+ users assigned to the project
Google CLI Tools Setup
- Send list of authorized users to your account manager to be set up in the bulk data access system
- Configure your Google CLI tools
- gcloud installation documentation
- gcloud setup documentation
Accessing the Analytic Index Bulk Data
Listing the available objects:
gsutil -u [project-ID] ls gs://[SUBSCRIBED_BUCKET_NAME]/
Copying objects:
gsutil -u [project-ID] cp gs://[SUBSCRIBED_BUCKET_NAME]/path/to/object [destination]
Additional gsutil Documentation
Requestor Pays Bucket Policy
Due to the nature of the bulk data, we have chosen not to control how many times files are downloaded. We have configured our storage system with the requestor pays policy to prevent cost overruns.
We recommend downloading the data one time into your system and using a storage policy that best fits your needs. In most cases, the data transfer fees for each individual client will fall within the free tier limits or will be very minimal.