Twitter Data Format

Important Notes:

Note that the Twitter Original/Native format will be supported long term, with all new metadata included in it, and is the preferred Tweet JSON format for new customers.
Be sure to check out this documentation on long/extended Tweets (“Doing more with 140”) HERE.

The following documentation describes the normalized Gnip Activity Streams format for Twitter data. For detailed documentation of Twitter’s original data format, please see documentation or skip to the Sample Payloads section to view a side-by-side comparison of Original and Activity Stream format.

Tweet Activities

Tweets, Retweets, Quote Tweets

The following table describes the root-level data structures for Tweet activities (Tweet, Retweet, Quote Tweet). For fields with multiple levels of sub-fields, click the links provided to reveal details about the sub-fields.

A unique IRI for the tweet. In more detail, "tag" is the scheme, "search.twitter.com" represents the domain for the scheme, and 2005 is when the scheme was derived.

When storing Tweets, this should be used as the unique identifier or primary key.

"id": "tag:search.twitter.com,2005:347769243409977344"

actor

An object representing the twitter user who tweeted. The Actor Object refers to a Twitter User, and contains all metadata relevant to that user.
Show Sub-field Details

Property	Description
objectType	"person" See here for more detailed information. "objectType": "person"
id	IRI for the twitter user "id": "id:twitter.com:277184168"
link	permalink "link": "http:\/\/www.twitter.com\/KidCodo"
displayName	the user's name "displayName": "Zach Codo"
image	the url of the user's icon. "image": "https:\/\/si0.twimg.com\/profile_images\/3664410292\/1d75c213a572873bf6797c5591475da5_normal.jpeg"
summary	the user's bio "summary": null
postedTime	the user account's creation date. "postedTime": "2011-04-04T21:31:20.000Z"
links[0].href	the user's website. "links": [ { "href": null, "rel": "me" } ]
location	The user provided location. May be a Twitter Place, with a displayName and objectType, or a simple String. "location": { "objectType": "place", "displayName": "Naperville, IL" }
utcOffset	The user's timezone offset from utc in seconds "utcOffset": "-21600"
preferredUsername	The user's screen name "preferredUsername": "KidCodo"
languages[0]	The user's default language "languages": [ "en" ]
twitterTimeZone	The user's timezone's name. "twitterTimeZone": "Central Time (US & Canada)"
friendsCount	Number of people the user follows. "friendsCount": 64
followersCount	Number of followers the user has. "followersCount": 207
listedCount	Number of lists the user is in "listedCount": 1
statusesCount	Number of tweets the user has tweeted "statusesCount": 11207
verified	A true/false condition indicating whether Twitter has classified the posting user's account as "verified" "verified": false

 "actor":
 {
    "objectType": "person",
    "id": "id:twitter.com:277184168",
    "link": "http:\/\/www.twitter.com\/KidCodo",
    "displayName": "Zach Codo",
    "postedTime": "2011-04-04T21:31:20.000Z",
    "image": "https:\/\/si0.twimg.com\/profile_images\/3664410292\/1d75c213a572873bf6797c5591475da5_normal.jpeg",
    "summary": null,
    "links":
    [
       {
          "href": null,
          "rel": "me"
       }
    ]
    "friendsCount": 64,
    "followersCount": 207,
    "listedCount": 1,
    "statusesCount": 11207,
    "twitterTimeZone": "Central Time (US & Canada)",
    "verified": false,
    "utcOffset": "-21600",
    "preferredUsername": "KidCodo",
    "languages":
    [
       "en"
    ],
    "location":
    {
       "objectType": "place",
       "displayName": "Naperville, IL"
    },
       "favoritesCount": 1123
 }

verb

The type of action being taken by the user.

Tweets, "post"
Retweets, "share"
Deleted Tweets, "delete"

The verb is the proper way to distinguish between a Tweet and a true Retweet. However, this only applies to true retweets, and not modified or quoted Tweets, which don't use Twitter's Retweet functionality. For a description of AS verbs click here.

For Deletes, note that only a limited number of fields will be included, as shown in the sample payload below.

 "verb": "post"

generator

An object representing the utility used to post the Tweet. This will contain the name ("displayName") and a link ("link") for the source application generating the Tweet.

"generator":
{
   "displayName": "Twitter for iPhone",
   "link": "http:\/\/twitter.com\/download\/iphone"
}

provider

A JSON object representing the provider of the activity. This will contain an objectType ("service"), the name of the provider ("displayName"), and a link to the provider's website ("link").

"provider":
{
   "objectType": "service",
   "displayName": "Twitter",
   "link": "http:\/\/www.twitter.com"
}

inReplyTo

A JSON object referring to the Tweet being replied to, if applicable. Contains a link to the Tweet.

"inReplyTo":
{
   "link": "http:\/\/twitter.com\/GOP\/statuses\/349573991561838593"
}

location

A JSON object representing the Twitter "Place" where the tweet was created. This is an object passed through from the Twitter platform.
Show Sub-field Details

Property	Description
objectType	"place" See here for more detailed information. "objectType": "place"
displayName	The full name of the place. "displayName": "Alsip, IL"
link	A link to the full Twitter JSON representation of the place. "link": "http:\/\/api.twitter.com\/1\/geo\/id\/9fdc3d1edf51a0a0.json"
geo	The GeoJSON bounding box provided by Twitter. "geo": { "type": "Polygon", "coordinates": [ [ [ -87.778572, 41.651233 ], [ -87.778572, 41.690948 ], [ -87.694909, 41.690948 ], [ -87.694909, 41.651233 ] ] ] }
streetAddress	The Street address of the place if available.
name	place.name from Twitter's JSON format "name": "Alsip"

"location": 
    {
    "objectType": "place",
    "displayName": "Boulder, CO",
    "name": "Boulder",
    "country_code": "United States",
    "twitter_country_code": "US",
    "link": "https://api.twitter.com/1.1/geo/id/fd70c22040963ac7.json",
    "geo": {
        "type": "Polygon",
        "coordinates":
        [
            [
                [
                    -105.3017759,
                    39.953552
                ],
                [
                    -105.3017759,
                    40.094411
                ],
                [
                    -105.183597,
                    40.094411
                ],
                [
                    -105.183597,
                    39.953552
                ]
            ]
        ]
    },
    "twitter_place_type": "city"
},

geo

Point location where the Tweet was created.

"geo":
{
   "type": "Point",
   "coordinates":
   [
      41.68291626,
      -87.77269682
   ]
}

twitter_entities

The entities object from Twitter's data format which contains lists of urls, mentions and hashtags. Please reference the Twitter documentation on Entities here Note that in Retweets, Twitter may truncate the values of entities that it extracts at the root level. So, for Retweets, your app should look at object.twitter_entities to ensure that you are using non-truncated values.
Show Sub-field Details

Property	Example
hashtags	"hashtags": [ { "text": "snow", "indices": [ 66, 71 ] }, { "text": "skiing", "indices": [ 73, 80 ] } ]
symbols	"symbols": [ { "text": "TWTR", "indices": [ 82, 87 ] }, { "text": "GOOG", "indices": [ 82, 87 ] } ]
urls	"urls": [ { "url": "http:\/\/t.co\/tPLOXjoafD", "expanded_url": "http:\/\/goo.gl\/nE2Uqa", "display_url": "goo.gl\/nE2Uqa", "indices": [ 87, 109 ] } ]
user_mentions	"user_mentions": [ { "screen_name": "TweetMaker", "name": "Maker of tweets", "id": 1234567890, "id_str": "1234567890", "indices": [ 3, 15 ] } ]
media	"media": [ { "id": 4.8198342372925e+17, "id_str": "481983423729254400", "indices": [ 105, 127 ], "media_url": "http:\/\/pbs.twimg.com\/media\/BrBZXscCQBAYC3I.jpg", "media_url_https": "https:\/\/pbs.twimg.com\/media\/BrBZXscCQAASC3I.jpg", "url": "http:\/\/t.co\/JwO4Y11lpG", "display_url": "pic.twitter.com\/JwO4Y51lpG", "expanded_url": "http:\/\/twitter.com\/TweetMaker\/status\/481933421943172096\/photo\/1", "type": "photo", "sizes": { "small": { "w": 340, "h": 358, "resize": "fit" }, "medium": { "w": 600, "h": 631, "resize": "fit" }, "large": { "w": 913, "h": 960, "resize": "fit" }, "thumb": { "w": 150, "h": 150, "resize": "crop" } } } ]

  "twitter_entities": {
    "hashtags": [
      {
        "text": "snow",
        "indices": [
          66,
          71
        ]
      },
      {
        "text": "skiing",
        "indices": [
          73,
          80
        ]
      }
    ],
    "symbols": [
      {
        "text": "THIS",
        "indices": [
          82,
          87
        ]
      },
      {
        "text": "THAT",
        "indices": [
          82,
          87
        ]
      }
    ],
    "urls": [
      {
        "url": "http:\/\/t.co\/tPLOXjoafD",
        "expanded_url": "http:\/\/goo.gl\/nE2Uqa",
        "display_url": "goo.gl\/nE2Uqa",
        "indices": [
          87,
          109
        ]
      }
    ],
    "user_mentions": [
      {
        "screen_name": "TweetMaker",
        "name": "Maker of tweets",
        "id": 1234567890,
        "id_str": "1234567890",
        "indices": [
          3,
          15
        ]
      }
    ],
    "media": [
      {
        "id": 4.8198342372925e+17,
        "id_str": "481983423729254400",
        "indices": [
          105,
          127
        ],
        "media_url": "http:\/\/pbs.twimg.com\/media\/BrBZXscCQBAYC3I.jpg",
        "media_url_https": "https:\/\/pbs.twimg.com\/media\/BrBZXscCQAASC3I.jpg",
        "url": "http:\/\/t.co\/JwO4Y11lpG",
        "display_url": "pic.twitter.com\/JwO4Y51lpG",
        "expanded_url": "http:\/\/twitter.com\/TweetMaker\/status\/481933421943172096\/photo\/1",
        "type": "photo",
        "sizes": {
          "small": {
            "w": 340,
            "h": 358,
            "resize": "fit"
          },
          "medium": {
            "w": 600,
            "h": 631,
            "resize": "fit"
          },
          "large": {
            "w": 913,
            "h": 960,
            "resize": "fit"
          },
          "thumb": {
            "w": 150,
            "h": 150,
            "resize": "crop"
          }
        }
      }
    ]
  }

twitter_extended_entities

An object from Twitter's native data format containing "media". This will be present for any Tweet where the twitter_entities object has data present in the "media" field, and will include multiple photos where present in the post. Note that this is the correct location to retrieve media information for multi-photo posts.

Multiple photos are represented by comma-separated JSON objects within the "media" array.

"twitter_extended_entities": {
    "media": [
      {
        "id": 5.0475663942184e+17,
        "id_str": "504756639421837312",
        "indices": [
          64,
          86
        ],
        "media_url": "http:\/\/pbs.twimg.com\/media\/BwFBfT7CMAA-Jct.jpg",
        "media_url_https": "https:\/\/pbs.twimg.com\/media\/BwFBfT7CMAA-Jct.jpg",
        "url": "http:\/\/t.co\/VPXUMLqPKI",
        "display_url": "pic.twitter.com\/VPXUMLqPKI",
        "expanded_url": "http:\/\/twitter.com\/RedSox\/status\/504756640550514688\/photo\/1",
        "type": "photo",
        "sizes": {
          "medium": {
            "w": 600,
            "h": 800,
            "resize": "fit"
          },
          "large": {
            "w": 645,
            "h": 860,
            "resize": "fit"
          },
          "thumb": {
            "w": 150,
            "h": 150,
            "resize": "crop"
          },
          "small": {
            "w": 340,
            "h": 453,
            "resize": "fit"
          }
        }
      },
      {
        "id": 5.0475663942185e+17,
        "id_str": "504756639421853696",
        "indices": [
          64,
          86
        ],
        "media_url": "http:\/\/pbs.twimg.com\/media\/BwFBfT7CcAAYBhR.jpg",
        "media_url_https": "https:\/\/pbs.twimg.com\/media\/BwFBfT7CcAAYBhR.jpg",
        "url": "http:\/\/t.co\/VPXUMLqPKI",
        "display_url": "pic.twitter.com\/VPXUMLqPKI",
        "expanded_url": "http:\/\/twitter.com\/RedSox\/status\/504756640550514688\/photo\/1",
        "type": "photo",
        "sizes": {
          "large": {
            "w": 773,
            "h": 579,
            "resize": "fit"
          },
          "medium": {
            "w": 600,
            "h": 449,
            "resize": "fit"
          },
          "thumb": {
            "w": 150,
            "h": 150,
            "resize": "crop"
          },
          "small": {
            "w": 340,
            "h": 254,
            "resize": "fit"
          }
        }
      }
    ]
  }

link

A Permalink for the tweet.

"link": "http:\/\/twitter.com\/KidCodo\/statuses\/347769243409977344"

body

The tweet text.

In Retweets, note that Twitter modifies the value of the body at the root level by adding "RT @username" at the beginning, and by truncating the original text and adding an ellipsis at the end. Thus, for Retweets, your app should look at the object.body to ensure that it is extracting the non-modified text of the original Tweet (being retweeted).

"body": "With Cardiff, Crystal Palace, and Hull City joining the EPL from the Championship it will be a great relegation battle at the end."

objectType

"activity"

"objectType": "activity"

object

An object representing tweet being posted or shared.

For Retweets, this will contain an entire "activity", with the pertinent fields described in this schema.

For Original Tweets, this will contain a "note" object, with the fields described here.

Show Sub-field Details

Property	Description
objectType	"note" See http://activitystrea.ms/head/activity-schema.html#note for more detailed information. "objectType": "note"
id	An IRI for the tweet. "id": "object:search.twitter.com,2005:349575936838082561"
summary	The text of the tweet. "summary": "\u201c@GOP: .@RNCResearch Obama\u2019s lack of leadership landing him in a \u2018dead zone presidency\u2019 http:\/\/t.co\/Eygp4TRqkF\u201d"
link	The permalink to the tweet. "link": "http:\/\/twitter.com\/chuckleslong\/statuses\/349575936838082561"
postedTime	The creation time of the tweet. "postedTime": "2013-06-25T17:12:52.000Z"

 "object":
 {
    "objectType": "note",
    "id": "object:search.twitter.com,2005:347769243409977344",
    "summary": "With Cardiff, Crystal Palace, and Hull City joining the EPL from the Championship it will be a great relegation battle at the end.",
    "link": "http:\/\/twitter.com\/KidCodo\/statuses\/347769243409977344",
    "postedTime": "2013-06-20T17:33:43.000Z"
 }

postedTime

The time the action occurred, e.g. the time the Tweet was posted.

"postedTime": "2013-06-25T17:12:52.000Z"

Sample Payloads

Be sure to check out this documentation on long/extended Tweets (“Doing more with 140”) HERE.

Original Format
Activity Streams Format

Data Format

Tweet Activities

Tweets, Retweets, Quote Tweets

Sample Payloads

Tweet

Retweet

Quote Tweet

Quote Count and Reply Count (Original Format Only - Lines 109 and 110)

Image Alt-Text (Original Format Only - Lines 106, 147, and 184)

Extended Tweet

Extended Tweet 280 characters

Status Delete

User Delete

User Undelete

Scrub Geo

User Protect

User Unprotect

User Suspend

User Unsuspend

User Withheld

Status Withheld

APIs

Sources

Enrichments

Resources