All Articles

Choosing a Historical API

Introduction   Both Historical PowerTrack and Full-Archive Search provide access to any publicly available Tweet, starting with the first Tweet from March 2006. Deciding which product better suits your use... keep reading

20 June 2017

Full-Archive Search API - metadata and filtering timeline

Introduction   How Twitter evolved as a platform, and how that affected the JSON used to encode Tweets, is discussed here. That article also begins the discussion of how these... keep reading

12 June 2017

Historical PowerTrack API - metadata and filtering timeline

Introduction  How Twitter evolved as a platform, and how that affected the JavaScript Object Notation (JSON) used to encode Tweets, is discussed here. That article also begins the discussion of... keep reading

02 June 2017

Tweet Metadata Timeline

Introduction   At its core, Twitter is a public, real-time, and global communication network. Since 2006, Twitter’s evolution has been driven by both user use-patterns and conventions and new product... keep reading

25 May 2017

Rule Limits

Overview  Gnip will now begin to enforce long-held contractual limits for the number of rules that a customer is able to add to their stream by enforcing rule limits on... keep reading

18 January 2017

Migrating Code to Gnip 2.0

Introduction  Realtime PowerTrack, Historical PowerTrack, the 30-Day Search API, and Volume Streams (such as the Decahose product) are now available on the Gnip 2.0 platform. These new, enhanced versions of... keep reading

22 August 2016

Migrating PowerTrack Rules from Version 1.0 to 2.0

Introduction  As announced on the Twiter Data blog, Gnip 2.0 is here. This means a new version of Gnip’s realtime PowerTrack, version 2.0, is now available. PowerTrack 2.0 provides a... keep reading

03 August 2016

Updating Twitter Hosebird Client for PowerTrack 2.0

Introduction   Twitter’s Hosebird client (HBC) has been a popular streaming consumer app for many years. Originally built around the Twitter (public) Streaming API, it was extended a couple of... keep reading

02 August 2016

Consuming Streaming Data

Overview  PowerTrack, Volume, and Replay streams utilize Streaming HTTP protocol to deliver data through an open, streaming API connection. Rather than delivering data in batches through repeated requests by your... keep reading

14 June 2016

Usage Monitoring and Management

As a Gnip customer, you’re probably familiar with the fact that monthly invoices are largely based on the level of usage of your Gnip products. Given that, it is imperative... keep reading

30 March 2016

Converting Data from JSON to CSV

Converting Data from JSON to CSV json2csv Customers often ask us about converting Tweet JSON into comma-separated values (CSV). These customers have received Twitter data from a Gnip Product such... keep reading

30 November 2015

Disconnections Explained

Disconnects from your PowerTrack stream can happen for a handful of reasons - whether they proactively planned or unplanned. Regardless of whether or not they were planned, any sort of... keep reading

08 May 2015

Storing Dynamic and Static Metadata - Part 3

Storing Dynamic and Static Metadata While much of Twitter metadata is dynamic in nature, changing tweet-by-tweet, other metdata can change more slowly or stay completely static for long periods of... keep reading

26 March 2015

Storing Twitter Metadata Arrays - Part 2

Storing Metadata Arrays Twitter data is dynamic in nature, and includes several types of metadata that are in arrays of variable length. For example, tweets can consist of multiple hashtags,... keep reading

03 March 2015

Storing Twitter Data in Relational Databases - Part 1

Introduction There are many ways to store social media data, such as in files, NoSQL-type datastores, and relational databases. This is an introduction to a series of articles on storing... keep reading

25 February 2015

Example Relational Database Schemas

Some Example Schemas Below are some example schemas that provide a starting place for specifying your database schema. We start with a single-table schema that represents the most simple option.... keep reading

01 January 2015

Visualizing Twitter Geo Data

This article describes the processes involved in plotting geotagged tweets on a map built in d3, which is useful for web based visualization. For an in depth overview of the... keep reading

03 December 2014

Consuming, Parsing, and Processing Tweets with Python

Introduction Python is a powerful, easily readable, and well-documented scripting language that is well suited for data exploration and analysis. The purpose of this article is to explore a Python... keep reading

27 October 2014

Data and Rule Management with jq

If you’ve spent time working with Gnip APIs, then you’re already familiar with the formatting used for PowerTrack Rules, and possibly also the JSON data (normalized Activity Streams or original... keep reading

24 September 2014

Honoring User Intent on Twitter

We believe that respecting the privacy and intent of Twitter users is critically important to the long term health of one of the largest public, real-time information platforms in the... keep reading

17 August 2014

Planning for High-Volume Social Data Events

Major national and global events are often accompanied by dramatic spikes in user activity across social media platforms. Sometimes these events are known about in advance, like the Super Bowl,... keep reading

04 June 2014

Downloading Historical PowerTrack Files

Introduction   Historical PowerTrack (HPT) provides an API wrapper around a process that filters and outputs data from a tweet archive. Data are gathered in a multi-step process where the... keep reading

20 May 2014

Translating Plain Language to PowerTrack Rules

Using Gnip PowerTrack rules to filter social data firehoses is key to ingesting just the right data for your platform. Whether your app can generate meaningful analysis or insights depends... keep reading

17 April 2014

PowerTrack Video Walkthrough

If you are getting ready to use PowerTrack for the first time, the following video provides an overview of what you need to know, and how to get connected quickly.... keep reading

16 April 2014

Capturing Tweets with Photos and Videos

Gnip’s PowerTrack filtering language gives you the ability to filter the Twitter Firehose, and a number of other sources’ firehoses, for data that is relevant to you and your brand.... keep reading

15 April 2014

Search API Video Walkthrough

If you are getting acquainted with the Search API, check out the following video. You’ll be introduced to the high-level functionality, and get everything you need to connect quickly. This... keep reading

25 March 2014

Data Collector Video Walkthrough

Just getting started with the Data Collector? The following video provides an overview of the Data Collector and how to use it to get started using public API data. Step-by-step... keep reading

24 March 2014

Twitter Geo-Referencing: An Example Use-case

The “Tweeting in the Rain” series on the Gnip blog discussed looking for a Twitter “signal” around rain gauges in six areas of the US including San Diego, Las Vegas,... keep reading

21 March 2014

Gnip Overview

Gnip is the world’s largest and most trusted provider of social data. With the most sources, the most customers, and the most robust infrastructure to deliver it all, Gnip is... keep reading

13 March 2014

cURL on Windows

cURL is a handy command-line utility for making HTTP requests. cURL is so useful you will notice that we provide sample cURL commands on the “API Help” tab of the... keep reading

07 March 2014

Twitter Geographical Metadata

Gnip’s customers often build products that need to use the location of a Tweet, or the user who posted it. For example, a customer may be interested in public opinion... keep reading

12 February 2014

Identifying and Understanding Retweets & Quote Tweets

Gnip customers often want to know the specifics around identifying and integrating Retweets into their products, but can run into a few common roadblocks. If you’re looking for the best... keep reading

07 February 2014

Unix Commands on Windows with CygWin

When you’re first beginning to use one of Gnip’s products, one major benefit of working in a Linux/Unix environment is the ability to utilize simple command-line tools to interact with... keep reading

31 January 2014

Deploying Ruby/Tk Applications as Windows 7 Executables

Since Gnip first released the Historical PowerTrack (HPT) API, we have had many academics and other researchers come to us for data to support their projects. Not long after, we... keep reading

28 January 2014

Activity Streams Intro

Activity streams is an open format used to represent activities performed by actors (usually people). Gnip uses an implementation of Activity Streams to provide a normalized structure to the various... keep reading

14 January 2014

Types of Data Sources

Gnip provides access to two general types of data sources through its APIs – Complete Access Sources, and Public API Access Sources. The differences in these types of access dictate... keep reading

03 January 2014

Filtering Tweets by Location

Introduction When working with Tweet data, there are two classes of geographical metadata: Tweet location - Available when user shares location at time of Tweet. Account Location - Based on... keep reading

14 December 2013