javascript - Node js redis socket.io pubsub realtime updates - Stack Overflow

admin管理员组

文章数量:1404927

Hi I am building a "Twitter clone" for my school project.

I want to implement a publish subscribe pattern for realtime updates.

• Users can "follow" other users
• When a user is online, and a "follower" posts a new message, the user should get a realtime notification.

I am using Node.js, Socket.io, Redis and MySql as database provider. Should I use a message queue, and whatfor are people using message queue's?

Thanks for help and answers

Hi I am building a "Twitter clone" for my school project.

I want to implement a publish subscribe pattern for realtime updates.

• Users can "follow" other users
• When a user is online, and a "follower" posts a new message, the user should get a realtime notification.

I am using Node.js, Socket.io, Redis and MySql as database provider. Should I use a message queue, and whatfor are people using message queue's?

Thanks for help and answers

• javascript
• node.js
• socket.io
• real-time
• publish-subscribe

Share Improve this question Follow asked Jul 30, 2012 at 9:38 onlineracoononlineracoon 2,97055 gold badges4949 silver badges6666 bronze badges

Add a ment  | 

2 Answers 2

Sorted by: Reset to default 7

Update

The problem is not there when you are small. But when you get big the fanout(forwarding message to all followers is going to be expensive and you want to do this offline using a MQ. Like twitter you store all active tweets in memory. When a tweet is posted you put(set) that tweet in memory @key(unique). You could use something like Twitter's snowflake for that.

Next the fanout process happens. For every user you need to put that unique key(tweet id) in their list so that they can retrieve the tweets from memory. When your site is small I guess you could do this without a message queue, but when you need to distribute a message from a user like for example scoble with 274,776 followers and who tweets a lot this can get pretty expensive.

A lot of users are offline so these tweets do not need to get delivered to the user immediately. You design your system like this because you need to keep everything in memory. I think that is the only way to do this effectively.

---

You should use a MQ just like twitter does. They have even open-sourced their own MQ: Kestrel. The High Scalability blog has a really interesting article: Scaling Twitter: Making Twitter 10000 Percent Faster. I advice you to study at least hot articles at High Scalability blog to learn how the big players scale their website. Some other links explaining how Twitter scales:

• http://highscalability./blog/2009/10/13/why-are-facebook-digg-and-twitter-so-hard-to-scale.html
• http://highscalability./blog/2011/12/19/how-twitter-stores-250-million-tweets-a-day-using-mysql.html
• http://highscalability./blog/2009/4/20/some-things-about-memcached-from-a-twitter-software-develope.html

I also assume you have read:

• http://redis.io/topics/twitter-clone

Also I would have a look at all the projects Twitter has open-sourced:

• https://github./twitter

I would have a look at the popular MQs like for example:

• Redis
• Beanstalkd
• Gearman.

I recently worked on a similar use case, and I used nodejs, socketio and redis pubsub.

The code is available at https://github./roshansingh/realtime-notifications.

Now ing back to your questions:

• Users can "follow" other users
• When a user is online, and a "follower" posts a new message, the user should get a realtime notification.

You can achieve both by creating rooms using socketio and a channel with same name in redis pubsub.

The flow can be something like this: You can make user join socketio rooms(say John, Dan etc) as soon as they login for which you will save all their subscribed rooms in database. And that the same time you will subscribe to redis pubsub with these channel names (like John). These updates when received can then be broadcasted to the rooms, and hence to all the online users.

You will have to publish John's activities on the same channel name(John) to redis.

Please read the code on the link pasted above. Let me know if you need any help.

本文标签： javascriptNode js redis socketio pubsub realtime updatesStack Overflow