This page is designed to help you migrate from version 1.0 to 2.0 of our PowerTrack API. Below you’ll find a summary of the changes, new feature list, as well as version differences to help with the transition.

Note: As we make changes to this API, we will update the Change Log found on the PowerTrack API 2.0 home page.


Summary of Changes 

  • New Endpoint URLs
  • New Features and Operators
  • Enrichments Support
  • Version Differences (changes in behavior)
    • Note: New parameter values: "streamType":"track_v2" and "dataFormat":"activity_streams"

New Endpoint URLs 

  • Create:
    POST https://gnip-api.gnip.com/historical/powertrack/accounts/{ACCOUNT_NAME}/publishers/twitter/jobs.json

  • Accept/Reject:
    PUT https://gnip-api.gnip.com/historical/powertrack/accounts/{ACCOUNT_NAME}/publishers/twitter/jobs/{JOB_UUID}.json

  • Status of All Jobs:
    GET https://gnip-api.gnip.com/historical/powertrack/accounts/{ACCOUNT_NAME}/publishers/twitter/jobs.json

  • Individual Job Status:
    GET https://gnip-api.gnip.com/historical/powertrack/accounts/{ACCOUNT_NAME}/publishers/twitter/jobs/{JOB_UUID}.json

  • File URLs for Download:
    GET https://gnip-api.gnip.com/historical/powertrack/accounts/{ACCOUNT_NAME}/publishers/twitter/jobs/{JOB_UUID}/results.json


New Features 

Below is a comprehensive list of the new features available in 2.0.

Feature: Details:
Enhanced URL Expansion Enrichment Tweets returned by PowerTrack 2.0 now include the enhanced URL Expansion enrichment including the URL’s Title and Meta Description. This data is part of the Tweet payload and available for filtering.
Expanded language tokenization Support for tokenization of additional languages besides Latin and Japanese including Korean, Chinese, Arabic and more!
Enrichments in Original Format Original format streams can include any enrichment and the “matching rules” element. Latency only introduced if url expansion enrichment enabled.
Unique Rule IDs When creating new PowerTrack rules, they will be assigned a unique ID that will be provided in the response and the matching_rule object of all matching tweets

New Operators 

Below is a comprehensive list of the new operators available in 2.0.

Operator: Details:
has:images operator Query for only tweets with native images
has:videos operator Query for only tweets with native videos
has:symbols operator Also known as cashtags, has:symbols support querying for only tweets that have symbols/cashtags
$ operator (e.g. $AAPL) Two new operators have been added to match against Tweets that mention stock ticker symbols. The $ operator, used like $TWTR, matches against Tweets that mention a specific ticker symbol. The "has:symbols" operator matches against all Tweets that mention any stock ticker symbol.
Emoji Filtering PowerTrack rules can contain emoji characters, allowing Tweets to be filtered based on emoji in the Tweet text, just like a keyword. You can also use emoji characters with the contains: operator with, for example ```contains:"🍕🍕"```.
bio: Performs a tokenized match on the contents of a user’s bio string
bio_name: Performs a tokenized match on the contents of a user’s “name” field
url operator Performs a tokenized (keyword/phrase) match on the expanded URLs of a tweet (similar to url_contains)
url_title operator Performs a keyword/phrase match on the new page title metadata available through the enhanced URL Expansion operator
url_description Performs a keyword/phrase match on the new page description metadata available through the enhanced URL Expansion operator
@ operator supports UserIDs The @mention operator now supports filtering using UserIDs instead of only Usernames/handles

Enrichment Support 

  • Deprecating Gnip’s Language Classification enrichment in favor of Twitter’s language classification which supports far more languages. (associated operator changes outlined above)
  • Profile Geo is available in Historical PowerTrack 2.0
  • URL Expansion is available in Historical PowerTrack 2.0 (see ‘New Features’ table above for more details) (enhancements only available after 2016-07-29)
  • Klout 2.0 is supported in Historical PowerTrack 2.0 (enhancements only available after 2016-07-29)

Product Differences (changes in behavior) 

The table below contains a list of operators and other product differences in version 2.0.

Difference: Details:
lang: operator lang: Uses Twitter’s per-tweet language classification rather than Gnip’s language classification
place: This operator will perform a keyword/phrase match rather than an exact match as it does today
bio_location This operator will perform a keyword/phrase match rather than an exact match as it does today
POST /jobs - New parameter values
  • "streamType":"track" => "streamType":"track_v2"
  • "dataFormat":"activity-streams" => "dataFormat":"activity_streams"
profile_region, profile_locality, profile_subregion These operators will perform a keyword/phrase match rather than an exact match as it does today
Tokenization
  • Subtle Japanese differences
  • Will collapse multiple white spaces in phrase matches (e.g. “hello world” will match “hello world”)
  • May be other subtle differences - please let us know what you find

Deprecated Operators 

The table below contains operators that have been deprecated in version 2.0 and the suggested replacement operator (if available).

Operator Replacement Operator (if exists) Future State
twitter_lang: lang: Gnip’s language classification is being deprecated and replaced with Twitter’s language classification. With this change, the more commonly used “lang” operator will now be applied to the Twitter language classification, rendering the twitter_lang operator unnecessary and redundant.
has:lang -lang:und The Twitter language classification field is present for all tweets and unclassified tweets have a value of “und” (undefined), so a query for only tweets with a language classification would now be -lang:und (i.e. NOT language=undefined).
klout_score: none Operator has minimal usage that does not justify support at this time
klout_topic: none Operator has minimal usage that does not justify continued support
klout_topic_contains: none Operator has minimal usage that does not justify continued support
klout_topic_id: none Operator has minimal usage that does not justify support at this time
bio_lang: none Operator has minimal usage that does not justify continued support. The bio language field of the payload is a poor representation of a user’s language. The majority of users do not change this setting from the English default and in a great number of cases, does not correlate to the language a user tweets in or their first language. Twitter’s language classification is a much more accurate and better representation. Further, operator has minimal usage that does not justify continued support
country_code: place_country The functionality of the country_code operator will ultimately be replaced and made available through a new place_country operator
profile_country_code: profile_country The functionality of the profile_country_code operator will ultimately be replaced and made available through a new profile_country operator
bio_location_contains: bio_location: The bio_location operator is being changed to perform a keyword/phrase match as opposed to an exact match. Once introduced and supported on PowerTrack, the bio_location_contains operator will be unnecessary
place_contains: place: The place operator is being changed to perform a keyword/phrase match as opposed to an exact match. Once introduced and supported on PowerTrack, the place_contains operator will be unnecessary
has:profile_geo_region:, has:profile_geo_subregion:, has:profile_geo_locality: n/a Operator has minimal usage that does not justify continued support
profile_region_contains:, profile_locality_contains:, profile_subregion_contains: profile_region:, profile_locality:, profile_subregion: The profile_region, profile_locality, profile_subregion operators are being changed to perform a tokenized match as opposed to an exact match. Once introduced these operators will be unnecessary
bio_name_contains: bio_name: A new “bio_name” operator has been introduced that will perform a keyword/phrase match on a user’s name. With introduction of the “bio_name” operator, bio_name_contains becomes unnecessary.
bio_contains: bio: A new “bio” operator has been introduced that will perform a keyword/phrase match on a user’s bio. With introduction of the “bio” operator, bio_contains will be unnecessary.

New Payload Samples 

Below are sample PowerTrack 2.0 payloads in both Original and Activity Streams format.

Miscellaneous 

Create Responses

Below are the API responses provided for both a successful Job creation and when Job creation fails due to invalid rule syntax.

Successful Job Creation

{
  "title": "customer_job_3",
  "account": "your_account_name",
  "publisher": "twitter",
  "streamType": "track_v2",
  "format": "original",
  "fromDate": "201606150000",
  "toDate": "201606270000",
  "requestedBy": "your_email@domain.com",
  "requestedAt": "2016-06-27T19:38:13Z",
  "status": "opened",
  "statusMessage": "Waiting on quote from Gnip.",
  "jobURL": "https:\/\/historical.staging.gnip.com:443\/accounts\/your_account_name\/publishers\/twitter\/historical\/track_v2\/jobs\/4w1j7sbrd6.json",
  "rules": {
    "summary": {
      "created": 1,
      "not_created": 0
    },
    "detail": [
      {
        "rule": {
          "value": "cats has:videos",
          "tag": "staging_test",
          "id": 8.4086625583897e+18
        },
        "created": true
      }
    ],
    "sent": "2016-06-27T19:38:15.965Z"
  }
}

Failed Job Creation due to bad rules

{
  "status": "error",
  "statusMessage": "Invalid Rules, see details.",
  "rules": {
    "summary": {
      "created": 0,
      "not_created": 1
    },
    "detail": [
      {
        "rule": {
          "value": "from:mcman_s has:video",
          "tag": "staging_test"
        },
        "created": false,
        "message": "Unrecognized 'has' value: 'video', must be from the list: [geo, hashtags, images, links, media, mentions, profile_geo, symbols, videos] (at position 18)\n"
      }
    ],
    "sent": "2016-06-29T16:08:34.887Z"
  }
}

Rule Management Enhancements

  • Upon rule creation, rules are given a unique ID, that will be included in the Rule create response (see below)

Note: Rules created during the beta period prior to 4/1/2016 should be re-loaded (deleted and re-added) to recieve associated Gnip generated rule IDs.

    {"value":"gnip lang:en","tag":"test","id":708372975137861632}
  • The “matching rule” element of the Tweet payload will now include a rule’s unique ID (see below)
        "matching_rules": [
            {
                "tag": "test",
                "id": 708372975137861632
            }
        ]
  • The Rules API now provides visibility into what rules were successfully added or deleted and for those that were not, a reason why
  • The Rules API now has a fully supported validation endpoint that will check the validity of a rule without actually creating it.