All Articles
Choosing a Historical API
Introduction Both Historical PowerTrack and Full-Archive Search provide access to any publicly available Tweet, starting with the first Tweet from March 2006. Deciding which product better suits your use... keep reading
20 June 2017
Full-Archive Search API - metadata and filtering timeline
Introduction How Twitter evolved as a platform, and how that affected the JSON used to encode Tweets, is discussed here. That article also begins the discussion of how these... keep reading
12 June 2017
Historical PowerTrack API - metadata and filtering timeline
Introduction How Twitter evolved as a platform, and how that affected the JavaScript Object Notation (JSON) used to encode Tweets, is discussed here. That article also begins the discussion of... keep reading
02 June 2017
Tweet Metadata Timeline
Introduction At its core, Twitter is a public, real-time, and global communication network. Since 2006, Twitter’s evolution has been driven by both user use-patterns and conventions and new product... keep reading
25 May 2017
Rule Limits
Overview Gnip will now begin to enforce long-held contractual limits for the number of rules that a customer is able to add to their stream by enforcing rule limits on... keep reading
18 January 2017
Migrating Code to Gnip 2.0
Introduction Realtime PowerTrack, Historical PowerTrack, the 30-Day Search API, and Volume Streams (such as the Decahose product) are now available on the Gnip 2.0 platform. These new, enhanced versions of... keep reading
22 August 2016
Migrating PowerTrack Rules from Version 1.0 to 2.0
Introduction As announced on the Twiter Data blog, Gnip 2.0 is here. This means a new version of Gnip’s realtime PowerTrack, version 2.0, is now available. PowerTrack 2.0 provides a... keep reading
03 August 2016
Updating Twitter Hosebird Client for PowerTrack 2.0
Introduction Twitter’s Hosebird client (HBC) has been a popular streaming consumer app for many years. Originally built around the Twitter (public) Streaming API, it was extended a couple of... keep reading
02 August 2016
Consuming Streaming Data
Overview PowerTrack, Volume, and Replay streams utilize Streaming HTTP protocol to deliver data through an open, streaming API connection. Rather than delivering data in batches through repeated requests by your... keep reading
14 June 2016
Usage Monitoring and Management
As a Gnip customer, you’re probably familiar with the fact that monthly invoices are largely based on the level of usage of your Gnip products. Given that, it is imperative... keep reading
30 March 2016
Converting Data from JSON to CSV
Converting Data from JSON to CSV json2csv Customers often ask us about converting Tweet JSON into comma-separated values (CSV). These customers have received Twitter data from a Gnip Product such... keep reading
30 November 2015
Disconnections Explained
Disconnects from your PowerTrack stream can happen for a handful of reasons - whether they proactively planned or unplanned. Regardless of whether or not they were planned, any sort of... keep reading
08 May 2015
Storing Dynamic and Static Metadata - Part 3
Storing Dynamic and Static Metadata While much of Twitter metadata is dynamic in nature, changing tweet-by-tweet, other metdata can change more slowly or stay completely static for long periods of... keep reading
26 March 2015
Storing Twitter Metadata Arrays - Part 2
Storing Metadata Arrays Twitter data is dynamic in nature, and includes several types of metadata that are in arrays of variable length. For example, tweets can consist of multiple hashtags,... keep reading
03 March 2015
Storing Twitter Data in Relational Databases - Part 1
Introduction There are many ways to store social media data, such as in files, NoSQL-type datastores, and relational databases. This is an introduction to a series of articles on storing... keep reading
25 February 2015
Example Relational Database Schemas
Some Example Schemas Below are some example schemas that provide a starting place for specifying your database schema. We start with a single-table schema that represents the most simple option.... keep reading
01 January 2015
Visualizing Twitter Geo Data
This article describes the processes involved in plotting geotagged tweets on a map built in d3, which is useful for web based visualization. For an in depth overview of the... keep reading
03 December 2014
Consuming, Parsing, and Processing Tweets with Python
Introduction Python is a powerful, easily readable, and well-documented scripting language that is well suited for data exploration and analysis. The purpose of this article is to explore a Python... keep reading
27 October 2014
Data and Rule Management with jq
If you’ve spent time working with Gnip APIs, then you’re already familiar with the formatting used for PowerTrack Rules, and possibly also the JSON data (normalized Activity Streams or original... keep reading
24 September 2014
Honoring User Intent on Twitter
We believe that respecting the privacy and intent of Twitter users is critically important to the long term health of one of the largest public, real-time information platforms in the... keep reading
17 August 2014
Planning for High-Volume Social Data Events
Major national and global events are often accompanied by dramatic spikes in user activity across social media platforms. Sometimes these events are known about in advance, like the Super Bowl,... keep reading
04 June 2014
Downloading Historical PowerTrack Files
Introduction Historical PowerTrack (HPT) provides an API wrapper around a process that filters and outputs data from a tweet archive. Data are gathered in a multi-step process where the... keep reading
20 May 2014
Translating Plain Language to PowerTrack Rules
Using Gnip PowerTrack rules to filter social data firehoses is key to ingesting just the right data for your platform. Whether your app can generate meaningful analysis or insights depends... keep reading
17 April 2014
PowerTrack Video Walkthrough
If you are getting ready to use PowerTrack for the first time, the following video provides an overview of what you need to know, and how to get connected quickly.... keep reading
16 April 2014
Capturing Tweets with Photos and Videos
Gnip’s PowerTrack filtering language gives you the ability to filter the Twitter Firehose, and a number of other sources’ firehoses, for data that is relevant to you and your brand.... keep reading
15 April 2014
Search API Video Walkthrough
If you are getting acquainted with the Search API, check out the following video. You’ll be introduced to the high-level functionality, and get everything you need to connect quickly. This... keep reading
25 March 2014
Data Collector Video Walkthrough
Just getting started with the Data Collector? The following video provides an overview of the Data Collector and how to use it to get started using public API data. Step-by-step... keep reading
24 March 2014
Twitter Geo-Referencing: An Example Use-case
The “Tweeting in the Rain” series on the Gnip blog discussed looking for a Twitter “signal” around rain gauges in six areas of the US including San Diego, Las Vegas,... keep reading
21 March 2014
Gnip Overview
Gnip is the world’s largest and most trusted provider of social data. With the most sources, the most customers, and the most robust infrastructure to deliver it all, Gnip is... keep reading
13 March 2014
cURL on Windows
cURL is a handy command-line utility for making HTTP requests. cURL is so useful you will notice that we provide sample cURL commands on the “API Help” tab of the... keep reading
07 March 2014
Twitter Geographical Metadata
Gnip’s customers often build products that need to use the location of a Tweet, or the user who posted it. For example, a customer may be interested in public opinion... keep reading
12 February 2014
Identifying and Understanding Retweets & Quote Tweets
Gnip customers often want to know the specifics around identifying and integrating Retweets into their products, but can run into a few common roadblocks. If you’re looking for the best... keep reading
07 February 2014
Unix Commands on Windows with CygWin
When you’re first beginning to use one of Gnip’s products, one major benefit of working in a Linux/Unix environment is the ability to utilize simple command-line tools to interact with... keep reading
31 January 2014
Deploying Ruby/Tk Applications as Windows 7 Executables
Since Gnip first released the Historical PowerTrack (HPT) API, we have had many academics and other researchers come to us for data to support their projects. Not long after, we... keep reading
28 January 2014
Activity Streams Intro
Activity streams is an open format used to represent activities performed by actors (usually people). Gnip uses an implementation of Activity Streams to provide a normalized structure to the various... keep reading
14 January 2014
Types of Data Sources
Gnip provides access to two general types of data sources through its APIs – Complete Access Sources, and Public API Access Sources. The differences in these types of access dictate... keep reading
03 January 2014
Filtering Twitter by Location
Activity Locations (Tweet-specific) Twitter enables users to specify a location for individual Tweets. PowerTrack offers multiple ways to filter for Tweets by Tweet-specific location data through its various operators (see... keep reading
14 December 2013