a blog for those who code

Monday 7 September 2015

How to fetch trending tweets in Nodejs

In this post we are going to discuss about fetching trending tweets in Nodejs. Many online entities like twitter, facebook etc. format their response either in JSON or XML when exposing the APIs to the developers. In this post we will create an application in Nodejs where we will make requests to Twitter's Rest Service to get the current trends and tweets.

Before going through the code I strongly recommend to go through these two posts : How to convert an object to XML in Nodejs and How to convert an object to JSON in Nodejs, which will give you a basic idea of how to manipulate JSON and XML in Nodejs application.

At first you need to create an application in Twitter. Go to http://dev.twitter.com/apps and login to your twitter account, you will see something like below.
It will ask you to create a new Twitter App. Once created you can see screen containing OAuth settings. We will need Consumer Key (API Key) and Consumer Secret (API Secret) from this page. At first we need to create a authorization file to grab the access token and save it to a settings file (oauth.json).

var https = require('https'),
oauthJsonFile = require('fs').createWriteStream('oauth.json');
var gettingAccess = {
  consumerKey: '<Your_Consumer_Key>',
  consumerSecretKey: '<Your_Consumer_Secret_Key>' 

var request = https.request({
  method: 'POST',
  host: 'api.twitter.com',
  path: '/oauth2/token',
  headers: {
    'User-Agent': 'Coding Defined',
    Authorization: 'Basic ' + Buffer((encodeURIComponent(gettingAccess.consumerKey) + ':' + encodeURIComponent(gettingAccess.consumerSecretKey))).toString('base64'),
    'Content-type': 'application/x-www-form-urlencoded;charset=UTF-8',
    'Content-length': 29


request.on('response', function(response) {
  if(response.statusCode !== 200) {
    return console.log('Error ' + response.statusCode);

If you go through the above code, we have created a POST request with headers like user-agent, authorization, content-type and content-length. When we are getting a response with the status code of 200 ,we are adding the response to the oauthJsonFile (oauth.json). The oauth.json file will look like below


The next step is to fetch the Trending Tweets from Twitter. We will create a file called trends.js. At first we will create a header for connecting to twitter using our authorization file. Here the authorization will be the access_token which we have got from the above code.

var https = require('https');
var headers = {
  'User-Agent': 'Coding Defined',
  Authorization: 'bearer ' + require('./oauth.json').access_token

Next we will create a function called jsonHandler() which will receive the response stream and convert JSON to object.

function jsonHandler(response, callback) {
  var json = '';
  if(response.statusCode === 200) {
    response.on('data', function(chunk) {
      json += chunk;
    }).on('end', function() {
  } else {
    console.log('Error : ' + reseponse.statusCode);

Then we will create two objects trendOptions and tweetDetails which will provide settings relevant to the Twitter API. Both these objects holds the reference to our headers object which contains the User-Agent and Authorization properties which are essential HTTP headers for a successful Twitter API request.

var trendOptions = {
  host: 'api.twitter.com',
  path: '/1.1/trends/place.json?id=1', // id = 1 for global trends
  headers: headers

var tweetDetails = {
  maxresults: 10,
  resultType: 'recent', // options are mixed, popular and recent
  options: {
    host: 'api.twitter.com',
    headers: headers,

Then we will create a function called fullTweetPath() which will take a query parameter and build the final Search API path. It then injects this path into the options object of tweetDetails.

function fullTweetPath(query) {
  var path = '/1.1/search/tweets.json?q=' + query
  + '&count=' + tweetDetails.maxResult
  + '&include_entities=true&result_type=' + tweetDetails.resultType;

  tweetDetails.options.path = path; 

Then we will create a function called callTwitter() which will combine the GET request and JSON handling and provides a callback with the single parameter which is data returned by Twitter.

function callTwitter(options, callback){
  https.get(options, function(response) {
    jsonHandler(response, callback);
  }).on('error', function(e) {
    console.log('Error : ' + e.message);

Now we will write a logic to get the trending tweets from twitter. As you can see in the below code we are calling callTwitter two times. In outer callTwitter we will use trendsArray to locate the query and then pass it along to the fullTweetPath which will build the the final Search API path. In inner callTwitter we have a tweetObj which contains the tweet data. Then we loop through the tweetObj to get each tweet separately. 

callTwitter(trendOptions, function(trendsArray) {
  callTwitter(tweetDetails.options, function(tweetObj) {
    tweetObj.statuses.forEach(function(tweet) {
    console.log('\n' + tweet.user.screen_name + ' : ' + tweet.text);

When we run our trends.js using command node trends.js we will get the below output

You can download the project from GitHub. Please Like and Share CodingDefined.com Blog, if you find it interesting and helpful.

No comments:

Post a Comment