Firehose API Reference


Methods 

Method Description
GET /firehose/:stream Connect to the data stream

Authentication 

All requests to Firehose stream APIs must use HTTP Basic Authentication, constructed from a valid email address and password combination used to log into your account at console.gnip.com. Credentials must be passed as the Authorization header for each request.

GET /firehose/:stream 

Establishes a persistent connection to the Firehose stream, through which the realtime data will be delivered.

Please see HERE for details on consuming streaming data after the connection is established.

Request Specifications

Request Method HTTP GET
Connection Type Keep-Alive

This should be specified in the header of the request.
URL Found on the stream's API Help page of your dashboard, using the following structure:

https://stream.gnip.com:443/accounts/{ACCOUNT_NAME}/publishers/{DATA_SOURCE}/streams/firehose/{STREAM_LABEL}.json
		    
Compression Gzip. To connect to the stream using Gzip compression, simply send an Accept-Encoding header in the connection request. The header should look like the following:

Accept-Encoding: gzip
Character Encoding UTF-8
Response Format JSON. The header of your request should specify JSON format for the response.
Rate Limit 10 requests per 60 seconds.
client Parameter If you have purchased a stream with Backfill enabled, encoding the "client" parameter into your API URL implements it. See here for more information on Backfill.
Read Timeout Set a read timeout on your client, and ensure that it is set to a value beyond 30 seconds.

Responses

The following responses may be returned by the API for these requests. Most error codes are returned with a string with additional details in the body. For non-200 responses, clients should attempt to reconnect.

Status Text Description
200 Success The connection was successfully opened, and new activities will be sent through as they arrive.
401 Unauthorized HTTP authentication failed due to invalid credentials. Log in to console.gnip.com with your credentials to ensure you are using them correctly with your request.
406 Not Acceptable Generally, this occurs where your client fails to properly include the headers to accept gzip encoding from the stream, but can occur in other circumstances as well.

Will contain a JSON message similar to "This connection requires compression. To enable compression, send an 'Accept-Encoding: gzip' header in your request and be ready to uncompress the stream as it is read on the client end."
429 Rate Limited Your app has exceeded the limit on connection requests.
503 Service Unavailable Gnip server issue. Reconnect using an exponential backoff pattern. If no notice about this issue has been posted on status.gnip.com, email support@gnip.com.

Example curl Request

The following example request is accomplished using cURL on the command line. However, note that these requests may also be sent with the programming language of your choice.

curl --compressed -v -uexample@customer.com "https://stream.gnip.com:443/accounts/{ACCOUNT_NAME}/publishers/{DATA_SOURCE}/streams/firehose/{STREAM_LABEL}.json"