Methods 

Method Description
GET /replay/:stream_type/:stream Connect to the data stream. Stream types include: 'powertrack', 'decahose', 'compliance', 'firehose', and 'mentions'.
POST /rules Add rules to a PowerTrack Replay stream. PowerTrack Replay only.
POST /rules _method=delete Delete rules from a PowerTrack Replay stream. PowerTrack Replay only.
POST /rules _method=get Get specified rules from a PowerTrack Replay stream by ID. PowerTrack Replay only.
GET /rules Retrieve the rules currently in place on a PowerTrack Replay stream. PowerTrack Replay only.

Authentication 

All requests to the Replay API must use HTTP Basic Authentication, constructed from a valid email address and password combination used to log into your account at console.gnip.com. Credentials must be passed as the Authorization header for each request.


GET /replay/:stream_type/:stream 

Establishes a persistent connection to the Replay data stream, through which the Tweet data will be delivered.

Please see HERE for details on consuming streaming data after the connection is established.

Request Method HTTP GET
Connection Type Keep-Alive

This should be specified in the header of the request.
URL Found on the stream's API Help page of your dashboard, the URL is built with Stream Type, Account Name and Stream Label tokens. For realtime PowerTrack, the Stream Type is 'powertrack'. For Volume Streams, Stream Types include 'sample10' (i.e. decahose), 'firehose', 'mentions', and 'compliance'.

Replay URLs have the following pattern:

https://gnip-stream.gnip.com/replay/{STREAM_TYPE}/accounts/{ACCOUNT_NAME}/publishers/twitter/{STREAM_LABEL}.json
            

For example, the Replay URL for realtime PowerTrack has the following pattern:

https://gnip-stream.gnip.com/replay/powertrack/accounts/{ACCOUNT_NAME}/publishers/twitter/{STREAM_LABEL}.json
            

For example, the Replay URL for Decahose has the following pattern:

https://gnip-stream.gnip.com/replay/sample10/accounts/{ACCOUNT_NAME}/publishers/twitter/{STREAM_LABEL}.json
            
Compression Gzip. To connect to the stream using Gzip compression, simply send an Accept-Encoding header in the connection request. The header should look like the following:

Accept-Encoding: gzip
Character Encoding UTF-8
Response Format JSON. The header of your request should specify JSON format for the response.
Rate Limit 5 requests per 5 minutes.
fromDate The oldest (starting) UTC timestamp from which the activities will be provided, must be in 'YYYYMMDDHHMM' format. Timestamp is in minute granularity and is inclusive (i.e. 12:00 includes the 00 minute). Valid times must be within the last 5 days, UTC time, and no more recent than 31 minutes before the current point in time.
toDate The latest (ending) UTC timestamp to which the activities will be provided, must be in 'YYYYMMDDHHMM' format. Timestamp is in minute granularity and is exclusive (i.e. 12:30 does not include the 30th minute of the hour). Valid times must be within the last 5 days, UTC time, and no more recent than 30 minutes before the current point in time.
Read Timeout Set a read timeout on your client, and ensure that it is set to a value beyond 30 seconds.


Responses

The following responses may be returned by the API for these requests. Most error codes are returned with a string with additional details in the body. For non-200 responses, clients should attempt to reconnect.

Status Text Description
200 Success The connection was successfully opened, and new activities will be sent through until the end of the requested time period is reached, and a "Replay Request Completed" message is sent.
401 Unauthorized HTTP authentication failed due to invalid credentials. Log in to console.gnip.com with your credentials to ensure you are using them correctly with your request.
406 Not Acceptable Generally, this occurs where your client either fails to properly include the headers to accept gzip encoding from the stream, or specifies an unacceptable fromDate or toDate.

Will contain a JSON message indicating the issue -- e.g. "This connection requires compression. To enable compression, send an 'Accept-Encoding: gzip' header in your request and be ready to uncompress the stream as it is read on the client end." or "Invalid date for query parameter 'toDate'. Can't ask for tweets from within the past 30 minutes."
429 Rate Limited Your app has exceeded the limit on connection requests.
503 Service Unavailable Gnip server issue. Reconnect using an exponential backoff pattern. If no notice about this issue has been posted on status.gnip.com, email support@gnip.com.


“Request Completed” Message

Once a request has been completed, Gnip will display a "Replay Request Completed" message on the "Overview" tab of the dashboard. Gnip will also send a corresponding message through the stream connection prior to disconnecting inside a "info" JSON message. If your stream is disconnected prior to receiving this message, the request was not completed, and you will need to re-run the missing portion of the request.

This may occur especially where your client is not consuming activities quickly enough. In this scenario, Gnip may send the “Completed” message, but the connection may close prior to your client receiving it due to the slow rate of consumption. In this scenario, your client should re-request the end-portion of the data to ensure completeness, based on the timestamps of the last Tweets received.

The "info" JSON message has the following structure:

{
  "info": {
    "message": "Replay Request Completed",
    "sent": "2016-05-27T22:15:50+00:00",
    "activity_count": 8874
  }
}

If any errors are associated with a completed Replay request, the "info" message will indicate that errors occurred and also list the minutes that were effected in the "minutes_failed" field. Here is an example:

{
  "info": {
    "message": "Replay Request Completed with Errors",
    "sent": "2016-05-27T16:00:02+00:00",
    "activity_count": 56333,
    "minutes_failed": [
      "2013-02-20T00:05:00+00:00",
      "2013-02-20T00:06:00+00:00"
    ]
  }
}

Users (or their client applications) should monitor for complete success of the Replay stream, and submit new Replay requests for any minutes that failed.

“Request Failed to Complete” Message

If a Replay request fails to complete, the "info" message will indicate the failure and also list the time range was was not processed. Here is an example:

{
  "info": {
    "message": "Replay Request Failed to Complete",
    "sent": "2016-06-27T16:37:13+00:00",
    "unprocessed_range": {
      "fromDate": "2016-06-26T00:00:00+00:00",
      "toDate": "2016-06-26T00:01:00+00:00"
    },
    "activity_count": 1822
  }
}

If this message is received another Replay request should be made based on the “fromDate” and “toDate” included in the “unprocessed_range” attribute.

Example curl Request

The following example request is accomplished using cURL on the command line, and requests the first hour of data from June 1, 2016. However, note that these requests may also be sent with the programming language of your choice.

curl --compressed -v -uexample@customer.com "https://gnip-stream.gnip.com/replay/powertrack/accounts/{ACCOUNT_NAME}/publishers/twitter/{STREAM_LABEL}json?fromDate=201606010000&toDate=201606010100"



POST /rules 

Adds one or many rules to a PowerTrack Replay stream’s ruleset. This is not available on non-PowerTrack Replay streams.

Request Specifications

Request Method HTTP POST
Content Type "application/json". The request should specify this as the "Content-type".
URL Found on the API Help page, and uses the following structure:

https://gnip-api.twitter.com/rules/powertrack-replay/accounts/{ACCOUNT_NAME}/publishers/twitter/{STREAM_LABEL}.json
            
Character Encoding UTF-8
Request Body Format JSON
Request Body Size Limit 1 MB
Rate Limit 5 requests per 5 seconds, aggregated across all requests to /rules endpoint for the specific stream's API (POST, DELETE, and GET).


Request Body Content

Your request should provide rule data in the following format:

Content-type: "application/json"
{
    "rules":
    [
        {"value":"rule1","tag":"tag1"},
        {"value":"rule2"}
    ]
}


Example curl Request

The following example request demonstrates how to add rules using cURL on the command line, using JSON rules.

curl -v -X POST -uexample@customer.com "https://gnip-api.twitter.com/rules/powertrack-replay/accounts/{ACCOUNT_NAME}/publishers/twitter/{STREAM_LABEL}.json" -d '{"rules":[{"value":"rule1","tag":"tag1"},{"value":"rule2"}]}'


Responses

The following responses may be returned by the API for these requests. Non-200 responses should be retried after making any necessary modifications in the rules.

Status Text Description
201 Created The rule or rules were successfully added to your PowerTrack Replay stream's ruleset.
400 Bad Request Generally relates to poorly formatted JSON, and includes an "Invalid JSON" message in the response.
401 Unauthorized HTTP authentication failed due to invalid credentials. Log in to console.gnip.com with your credentials to ensure you are using them correctly with your request.
406 Not Acceptable This generally occurs due to a request to add or delete rules while there is an existing connection to the stream. This will be accompanied by an appropriate message like "Rules on replay streams can not be updated while a replay stream is connected. You have currently (1) active replay connections."
422 Unprocessable Entity Generally occurs due to an invalid rule, based on the PowerTrack rule restrictions. Requests fail or succeed as a batch. For these errors, each invalid rule and the reason for rejection is included in a JSON message in the response. Catch the associated exception to expose this message.
429 Rate Limited Your app has exceeded the limit on requests to add, delete, or list rules for this stream.
503 Service Unavailable Gnip server issue. Reconnect using an exponential backoff pattern. If no notice about this issue has been posted on status.gnip.com, email support@gnip.com.


Example Responses

HTTP/1.1 201 Created
HTTP/1.1 400 Bad Request
{"error":{"message":"Rule '-estrule' is invalid.  Rules must contain a non-negation term (at position 1)\nRules must contain at least one positive, non-stopword clause (at position 1)\n","sent":"2013-08-16T00:50:00+00:00"}}



POST /rules _method=delete 

Removes the specified rules from a PowerTrack Replay stream’s ruleset. This is not available on non-PowerTrack Replay streams.

Request Specifications

Request Method HTTP POST
Authentication Basic Authentication. Your login credentials must be included in the header of the request.
Content Type "application/json". The request should specify this as the "Content-type".
URL Found on the API Help page, and uses the following structure:

https://gnip-api.twitter.com/rules/powertrack-replay/accounts/{ACCOUNT_NAME}/publishers/twitter/{STREAM_LABEL}.json?_method=delete
            
Character Encoding UTF-8
Request Body Format JSON


Request Body Content

Your request should provide rule data in the following format:

Content-type: "application/json"
{
    "rules":
    [
        {"value":"rule1"},
        {"value":"rule2"}
    ]
}


Example curl Request

The following example request demonstrates how to add rules using cURL on the command line.

curl -v -X POST -uexample@customer.com "https://gnip-api.twitter.com/rules/powertrack-replay/accounts/{ACCOUNT_NAME}/publishers/twitter/{STREAM_LABEL}.json?_method=delete" -d '{"rules":[{"value":"rule1"},{"value":"rule2"}]}'


Responses

The following responses may be returned by the API for these requests. Non-200 responses should be retried following any necessary modifications to the rules being deleted.

Status Text Description
200 OK Indicates that the rule data supplied with the request consisted of valid JSON. However, note that if no rules are found in the ruleset for the PowerTrack Replay stream based on a case-sensitive search, no rules will be deleted.
400 Bad Request Generally relates to poorly formatted JSON, and includes an "Invalid JSON" message in the response.
401 Unauthorized HTTP authentication failed due to invalid credentials. Log in to console.gnip.com with your credentials to ensure you are using them correctly with your request.
406 Not Acceptable This generally occurs due to a request to add or delete rules while there is an existing connection to the stream. This will be accompanied by an appropriate message like "Rules on replay streams can not be updated while a replay stream is connected. You have currently (1) active replay connections."
429 Rate Limited Your app has exceeded the limit on requests to add, delete, or list rules for this stream.
503 Service Unavailable Gnip server issue. Reconnect using an exponential backoff pattern. If no notice about this issue has been posted on status.gnip.com, email support@gnip.com.


Example Responses

HTTP/1.1 200 OK
HTTP/1.1 400 Bad Request
{"error":{"message":"Invalid JSON. The body must be in the format {\"rules\":[{\"value\":\"rule1\", \"tag\":\"tag1\"}, {\"value\":\"rule2\"}]}","sent":"2013-08-16T00:50:00+00:00"}}


Deleting Rules with POST

This method is also available by sending a POST request to the same URL with the URL query parameter "_method=delete" (e.g. for GAE and other environments that don't allow HTTP method 'DELETE' method on HTTP requests).



POST /rules _method=get 

Retrieves requested existing rules for a stream.

Request Specifications

Request Method HTTP POST
URL Found on the API Help page, and uses the following structure:

https://gnip-api.twitter.com/rules/powertrack/accounts/{gnip_account_name}/publishers/twitter/{stream_label}.json?_method=get
            
Character Encoding UTF-8
Request Body Format JSON
Request Body Size Limit 5 MB
Rate Limit 60 requests per minute, aggregated across all requests to /rules endpoint for the specific stream's API (POST and GET).
Compression Gzip compression is supported, but not required for these requests.


Example curl Request

The following example request demonstrates how to add rules using cURL on the command line.

curl -v -X POST -uexample@customer.com "https://gnip-api.twitter.com/rules/powertrack-replay/accounts/{gnip_account_name}/publishers/twitter/{stream_label}.json?_method=get" -d '{"rule_ids":[734938587381145604,734938587381174273]}"


Response

The following responses may be returned by the API for these requests. Non-200 responses should be retried, utilizing an exponential backoff for subsequent requests.

Status Text Description
200 OK The request was successful, and the current ruleset is returned in JSON format.
400 Bad Request Generally relates to poorly formatted JSON, and includes an "Invalid JSON" message in the response.
401 Unauthorized HTTP authentication failed due to invalid credentials. Log in to console.gnip.com with your credentials to ensure you are using them correctly with your request.
429 Rate Limited Your app has exceeded the limit on requests to add, delete, or list rules for this stream.
503 Service Unavailable Gnip server issue. Reconnect using an exponential backoff pattern. If no notice about this issue has been posted on status.gnip.com, email support@gnip.com.


Example Response

HTTP/1.1 200 OK
{
	"rules": [{
		"value": "from:furiouscamper",
		"tag": null,
		"id": 734938587381145604
	}, {
		"value": "fish \uD83D\uDC20",
		"tag": null,
		"id": 734938587381174273
	}],
	"sent": "2016-06-01T15:52:37.341Z"
}
	{
    "error": {
        "message": "Invalid JSON. The body must be in the format {\"rules\":[{\"value\":\"rule1\", \"tag\":\"tag1\"}, {\"value\":\"rule2\"}]} or {\"rule_ids\": [rule_id1, rule_id2, rule_id3, rule_id4, rule_id5]}",
        "sent": "2013-08-16T00:50:00+00:00"
    }
}



GET /rules 

Retrieves all existing rules for a PowerTrack Replay stream. This is not available on non-PowerTrack Replay streams.

Request Specifications

Request Method HTTP GET
URL Found on the API Help page, and uses the following structure:

hhttps://gnip-api.twitter.com/rules/powertrack-replay/accounts/{ACCOUNT_NAME}/publishers/twitter/{STREAM_LABEL}.json
            
Request Body Format N/A
Compression Gzip compression is supported, but not required for these requests.


Response

The following responses may be returned by the API for these requests. Non-200 responses should be retried, utilizing an exponential backoff for subsequent requests.

Status Text Description
200 OK The request was successful, and the current ruleset is returned in JSON format.
400 Bad Request Generally relates to poorly formatted JSON, and includes an "Invalid JSON" message in the response.
401 Unauthorized HTTP authentication failed due to invalid credentials. Log in to console.gnip.com with your credentials to ensure you are using them correctly with your request.
429 Rate Limited Your app has exceeded the limit on requests to add, delete, or list rules for this stream.
503 Service Unavailable Gnip server issue. Reconnect using an exponential backoff pattern. If no notice about this issue has been posted on status.gnip.com, email support@gnip.com.


Example Response

HTTP/1.1 200 OK
{
	"rules": [{
		"value": "from:jondee_test",
		"tag": null,
		"id": 735163830813134848
	}, {
		"value": "fish \uD83D\uDC20",
		"tag": null,
		"id": 738034522583769088
	}, {
		"value": "Pizza \uD83C\uDF55",
		"tag": null,
		"id": 738034522579554304
	}, {
		"value": "eggplant \uD83C\uDF46",
		"tag": null,
		"id": 738034522579570689
	}],
	"sent": "2016-06-01T15:52:37.341Z"
}