Tutorial: rabbitmq

RabbitMQ is a open source message broker software that implements the AMQP protocol (Advanced Message Queuing Protocol).

If you are not sure what you might do with a message broker you should read their getting started guide.

It is more than just a publish & subscribe pattern that can be implemented through RabbitMQ.

If you need high availability and a high flexibility for routing messages you should consider RabbitMQ as your messaging plattform.

Time to install the server.

1. Add key of RabbitMQ repro:

wget http://www.rabbitmq.com/rabbitmq-signing-key-public.asc
apt-key add rabbitmq-signing-key-public.asc

2. Add sources:

nano /etc/apt/sources.list

Content:

deb http://www.rabbitmq.com/debian/ testing main

3. Install server

apt-get update && apt-get install rabbitmq-server

4. Add addons:

rabbitmq-plugins enable rabbitmq_management
rabbitmq-plugins enable rabbitmq_web_stomp

5. Login into web fronend

Start your browser and call "http://ip-of-your-server:15672"

The GUI is great.

Use the login "guest" with the password "guest" to login.

Please change the login afterwards.

rabbitmq1.JPG

You can manage the whole server through the GUI but you can use the console tool "rabbitmqctl" too.

Some common use cases are:

  • rabbitmqctl list_users
    List all users and their attached roles.
  • rabbitmqctl add_vhost /test
    Add a new vhost to the server.
    Vhosts are used to separate queues and users.
    You can e.g. limit the access rights.
  • rabbitmqctl add_user username password
    Add a user.
  • rabbitmqctl set_permissions -p /vhost username "." "." ".*"
    Grand access to a vhost.

5. Clustering

If you want to cluster RabbitMQ you have to sync the hosts file and the "erlang cookie":

nano /etc/hosts

Check that each hostname does have the right external ip address.

#master
cat /var/lib/rabbitmq/.erlang.cookie

#slaves
echo "the-cookie" > /var/lib/rabbitmq/.erlang.cookie

You have to stop RabbitMQ and copy the conent of the master cookie to all slave cookies

Double check the value because this file should not contain any whitespaces or carriage returns.

And join each slave to the cluster

rabbitmqctl stop_app
rabbitmqctl join_cluster [email protected]
rabbitmqctl start_app

You can check the status of the cluster by running following command

rabbitmqctl cluster_status

6. Clients

And there are tons of clients - from C++ to PHP or Javascript.

I have wrote a little chat application that is based on the JS client lib and using websockets for the communication.

My main usage of the RabbitMQ server are:

  • Managing of payloads
    I do have two masters that are generating jobs for my twitter/web archive.
    The RabbitMQ server is distributing the jobs through all workers.
    SSH/keys are not needed because if a worker is ready it is connecting to the RabbitMQ server.
    So master and slave do not know each other.
    I can add and remove slaves at will and ip changes do not mean anything.
    RabbitMQ is persisting the messages and routing the job messages to free workers (load balancing) too.
  • Log handling
    If things go wrong I get a lot of log entries about services/routings/domains/websites that are not available.
    So my RabbitMQ server works as a buffer to ensure that my database or my email servers are not overloaded.
    Perfect tool to queue and load balancing stuff.

If you want to play around you can look at the PHP based master and slave examples (python: M/S).