The Gnip Historical Power Track product is a job-based system and usage is reliant on a multiple-request API. This API is solely intended for retrieving Historical data, and is not to be used as a platform for generating volume estimates. Generation of data via the Historical Power Track product requires two distinct steps: requesting a job quote/estimate (including projected cost, data volume, and time) and accepting or rejecting a job quote/estimate.
The process is initiated with a POST request that creates a job and kicks off an estimation process. Following such a request for a quote, the job status can be monitored with GET requests until a quote/estimate is available. Once a quote is generated, a subsequent PUT request can either accept the quote and initiate data generation or reject the quote and delete it from the queue. Once job execution has begun, the job’s status can once again be monitored for completion using GET requests.
Historical data are available for 15 days after the job has been accepted.
The API is structured in a way that it is easy to integrate into a customer’s existing workflow / tools. Additionally, we have separated the quoting of and acceptance of a quote into distinct steps as we recognize the price of a Historical Power Track job can be significant and for many of our customers, those requesting data may not have the authority within their organization to spend this money.
- Each Historical Power Track job supports up to 1000 rules.
- Historical Power Track supports the same rules and operators as its real-time counterpart. Details about Power Track rules and its Twitter operators are available at:
- For help with creating PowerTrack rules using the operators referenced above, please see THIS ARTICLE.
Create / Estimate Job
Creates a job and initiates estimation of cost, time to generate, and data volume. Note: this estimate is to be used as a general indicator of the parameters of the job you are about to run so that you may reject jobs which potentially fall far outside your expectations with regard to volume and time to completion. It is not to be used as a tool to estimate volumes for jobs you might decide to run in the future. See here for documentation.
Monitor Job Status
Monitor the status of a job be it generation of a quote/estimate or job execution/data generation. See here for documentation.
Accept / Reject Job (Execute/Delete)
Accept a job estimate / quote and kickoff the data generation process or reject it and remove it from your queue. Acceptance of a job estimate equates to signing a contract and you will be billed the amount of the quote. See here for documentation.
At this time, Gnip Enrichments including Klout scores, language classification and url expansion, are only available back to 3/26/2012. As a result, all operators reliant on these enrichments are not supported for jobs with a timeframe prior to this date.
Geo-tagged data is suppressed from all Tweets prior to 9/1/2011 for Twitter Compliance reasons. As a result, all operators reliant on this geo data will not be supported for jobs with a timeframe prior to this date.
- The url_contains operator will still function prior to this date, but will only match against URLs as they are entered by a user into a Tweet and not the fully resolved URL (i.e. if a bit.ly URL is entered in the Tweet it can only match against the bit.ly and not the URL that has been shortened by bit.ly)
User Profile Data
All data prior to 1/1/2011 contains user profile information as it appeared in that user’s profile in September 2011. (e.g @jack’s very first Tweet in March 2006 contains his bio data from September 2011 that references his position as CEO at Square, which was not in existence at the time of the Tweet)
Example Code Snippets
The example code snippets at the following link are not officially supported by Gnip and only represent a basic framework of how to interact with the historical API. We encourage you to use it as a starting point and reference for building your own app.