Apollo 1.0-beta4 User Manual
Creating a Broker
A broker instance is the directory containing all the configuration and runtime data such as logs and data files associated with a broker process. It is recommended that you do not create the instance directory under the directory where the Apollo distribution is installed.
On unix systems, it is a common convention to store this kind of runtime data
under the /var/lib
directory. For example, to create an instance at
'/var/lib/mybroker', run:
cd /var/lib
apollo create mybroker
A broker instance directory will contain the following sub directories:
bin
: holds execution scripts associated with this instance.etc
: hold the instance configuration filesdata
: holds the data files used for storing persistent messageslog
: holds rotating log filestmp
: holds temporary files that are safe to delete between broker runs
At this point you may want to adjust the default configuration located in etc directory.
Broker Configuration
Each broker instance can be tuned and configured by editing one of the following files.
bin/apollo-broker
: You can update the start script to control JVM level configuration options like JVM memory sizing.etc/apollo.xml
: The primary configuration file for the broker. It controls the opened ports, the queues, security, virtual host settings and more.etc/log4j.properties
: This is a standard log4j configuration file which controls the broker's logging.etc/keystore
: A Java key store used to hold cryptographic security keys and certificates. It is only need for brokers using SSL.etc/login.conf
: A standard JAAS [login.conf] configuration file used to define the JAAS authentication domains available to the broker.etc/users.properties
: Holds userid/password mappings of users that can access the broker. Referenced by theetc/login.conf
file.etc/groups.properties
: Holds groups to users mappings so that you can simplify access control lists (ACL) by using group instead listing individual users.black-list.txt
: A list of IP address which are banned from connecting to the broker.
Automatic Configuration Reloading
Once a broker is started, you can edit any of the configuration files in
the etc
directory and your changes will be automatically reloaded. The
configuration update will be applied in the least non-disruptive way possible.
For example, if you removed a connector, the port that connector was listening
on will be released and no new connections will be accepted. Connections that
were previously accepted by that connector will continue to operate normally.
Adjusting JVM Settings
You can define the following environment variables in the bin/apollo-broker
start script to customize the JVM settings:
JAVACMD
: The path to the java executable to useJVM_FLAGS
: The first JVM flags passed. Defaults to-server -Xmx1G
, you may want to lower or raise the maximum memory based on your actual usage.APOLLO_OPTS
: Additional JVM options you can addAPOLLO_DEBUG
: Set to true to enabled debugging on port 5005APOLLO_PROFILE
: Set to true to YourKit based profilingJMX_OPTS
: The JMX JVM options used, defaults to -Dcom.sun.management.jmxremote
Make sure you define the variables before the apollo-broker
script
executes apollo
and that the variables get exported in the case of the
unix script.
Understanding the apollo.xml
File
There are many XSD aware XML editors which make editing XML configuration files less error prone. If your using one of these editors, you can configure it to use this apollo.xsd file.
The simplest valid apollo.xml
defines a single virtual host and a
single connector.
<broker xmlns="http://activemq.apache.org/schema/activemq/apollo">
<virtual_host id="default">
<host_name>localhost</host_name>
</virtual_host>
<connector id="tcp" bind="tcp://0.0.0.0:61613"/>
</broker>
The broker, virtual host, and connector are assigned an id which which is used to by the REST based administration console to identify the corresponding runtime resource. The virtual host will not persist any messages sent to it since it does not have a configured messages store.
Brokers can be configured with multiple virtual hosts and connectors.
When a broker is first started up, it will validate the configuration
file against the the XSD Schema and report any
errors/warnings it finds but it will continue to start the broker even
it finds problems. You would want to the broker to abort starting up
if any issues are found with the schema validation you should set
the broker
element's validation
attribute to strict
. Example:
<broker validation="strict"
xmlns="http://activemq.apache.org/schema/activemq/apollo">
...
</broker>
Connectors
A broker connector is used to accept new connections to the broker.
A connector
element can be configured with the following attributes
enabled
: if set to false, then the connector host will be disabled.bind
: The transport that the connector will listen on, it includes the ip address and port that it will bind to. Transports are specified using a URI syntax.connection_limit
: The maximum number of concurrently open connections this connector will accept before it stops accepting additional connections. If not set, then there is no limit.protocol
: Defaults tomulti
which means that any of the broker's supported protocols can connect via this transport.
Furthermore, the connector element may contain protocol specific
configuration elements. For example, to have the broker set the user_id
header of messages to the id of user that sent the message, you would
use the following configuration:
<connector id="tcp" bind="tcp://0.0.0.0:61613">
<stomp add_user_header="user_id"/>
</connector>
TCP Transports
The TCP transport uses the tcp://
URI scheme. It uses the URI host
and port to determine to which local interfaces to bind. For example:
tcp://0.0.0.0:61613
binds to all IPv4 interfaces on port 61613tcp://[::]:61613
binds to all IPv4 and IPv6 interfaces on port 61613tcp://127.0.0.1:0
binds to the loopback interface on a dynamic port
The TCP URI also supports several query parameters to fine tune the settings used on the socket. The supported parameters are:
receive_buffer_size
: Sets the size of the internal socket receive buffer. Defaults to 65536 (64k)send_buffer_size
: Sets the size of the internal socket send buffer.
Defaults to 65536 (64k)traffic_class
: Sets traffic class or type-of-service octet in the IP header for packets sent from the transport. Defaults to8
which means the traffic should be optimized for throughput.max_read_rate
: Sets the maximum bytes per second that this transport will receive data at. This setting throttles reads so that the rate is not exceeded. Defaults to 0 which disabled throttling.max_write_rate
: Sets the maximum bytes per second that this transport will send data at. This setting throttles writes so that the rate is not exceeded. Defaults to 0 which disabled throttling.
Example which uses a couple of options:
<connector id="tcp" bind="tcp://0.0.0.0:61613?receive_buffer_size=1024&max_read_rate=65536"/>
Note that &
was used to separate the option values instead of just &
since the
URI being written within an XML file.
Virtual Hosts
A virtual hosts allows Apollo to support multi tenant style configurations. Each virtual host is highly isolated each with it's own persistence, security, and runtime constraints configuration.
Protocols like STOMP 1.1, inform the broker of which host the client is
attempting to connect with. The broker will search it's list of virtual hosts
to find the first host who has a configured host_name
that matches.
Protocols which do NOT support virtual hosts, will just connect to the first
virtual host defined in the configuration file.
host_name
: a host name that the virtual host is known as. This element should be repeated multiple times if the host has many host names.
A virtual_host
element may be configured with the following attributes:
enabled
: if set to false, then the virtual host will be disabled.purge_on_startup
: if set to true, the persistent state of the broker will be purged when the broker is started up.
The virtual_host
can also define multiple topic
, queue
, and
dsub
elements to secure or tune how message delivery works
for different topics or queues. If none are defined, then sensible default
settings are used which allows destinations to be auto created as they get
accessed by applications.
Finally virtual_host
configuration should also include a message store
configuration element to enable message persistence on the virtual host.
Queues
When a new queue is first created in the broker, it's configuration will be
determined by the first queue
element which matches the queue being
created. The attributes matched against are:
id
: The name of the queue, you can use wild cards to match multiple or don't set to match all queues.
If the queue definition is not using a wild card in the id, then the queue will be created when the broker first starts up.
A queue
element may be configured with the following attributes:
unified
: If set to true, then routing then there is no difference between sending to a queue or topic of the same name. See the Unified Destinations documentation for more details.queue_buffer
: The amount of memory buffer space allocated for each queue.producer_buffer
: The amount of memory buffer space allocated to each producer for receiving messages.consumer_buffer
: The amount of memory buffer space allocated to each subscription for receiving messages.persistent
: If set to false, then the queue will not persistently store it's message.swap
: If set to false, then the queue will not swap messages out of memory.flush_range_size
: The number max number of flushed queue entries to load from the store at a time. Note that Flushed entires are just reference pointers to the actual messages. When not loaded, the batch is referenced as sequence range to conserve memory.
Topics
When a new topic is first created in the broker, it's configuration will be
determined by the first topic
element which matches the topic being
created. The attributes matched against are:
id
: The name of the topic, you can use wild cards to match against multiple or don't set to match all topics.
If the topic definition is not using a wild card in the id, then the topic will be created when the broker first starts up.
A topic
element may be configured with the following attributes:
slow_consumer_policy
: Valid values areblock
andqueue
. Defaults toblock
. This setting defines how topic subscriptions are handled which affects slow consumer scenarios. If set toqueue
then each subscription gets a temporary queue which can swap messages to disk when you have a slow consumer so that produces do not slow down to the speed of the slowest consumer. If set toblock
, the producers block on slow consumers which makes producers only as fast as the slowest consumer on the topic.
Durable Subscriptions
When a new durable subscription is first created in the broker, it's
configuration will be determined by the first dsub
element
which matches the durable subscription being created. The attributes matched
against are:
id
: The name of the subscription.id_regex
: A regular expression used to match against the subscription id
If you want to create the durable subscription when the broker first starts up,
then you must set the topic
attribute and optionally the selector
attribute.
topic
: The topic which the durable subscription will subscribe to.selector
: A selector expression which filters out messages
A dsub
element may be configured with all the
attributes available on the queue
element.
Unified Destinations
Unified destinations can be used so that you can mix queue and topic
behavior on one logical destination. For example, lets assumed foo
is configured as a unified destination and you have 2 subscribers
on queue foo
and 2 subscribers on topic foo
, then when you publish to
queue foo
or topic foo
, the 2 queue subscribers will load balance
their messages and the 2 topic subscribers will each get a copy of the message.
It is important to note that the unified subscription will not start retaining it's messages in a queue until a queue subscriber subscribes from it.
Message Stores
A message store is used to implement reliable messaging and message swapping which are both optional features that disabled if no message store is configured on the virtual host. If no message store is configured, then all message routing is performed in memory and queue will quickly “fill up” when you have slow or no consumers since the messages cannot get swapped to disk.
Apollo supports multiple message store implementations. The implementations currently supported are:
- BDB Store : is a file based message store implemented using the Sleepycat BDB library. This is the most stable implementation.
- JDBM2 Store : is a file based message store implemented using the JDBM2 library. This is the most stable implementation.
BDB Store
Apache cannot redistribute the BDB library due to the terms of it's
license, but you can easily get a free copy directly from Oracle. Before
you can start using the BDB Store you must first download it from Oracle
at je-4.1.10.jar
and then copy it into the ${APOLLO_HOME}/lib
directory.
For those of you with curl installed, you can just run:
curl http://download.oracle.com/maven/com/sleepycat/je/4.1.10/je-4.1.10.jar > ${APOLLO_HOME}/lib/je-4.1.10.jar
Once that is done, you can enable the store by adding a bdb_store
element
inside your virtual_host
. Example:
...
<virtual_host id="default">
...
<bdb_store directory="${apollo.base}/data"/>
..
</virtual_host>
...
A bdb_store
element may be configured with the following attributes:
directory
: The directory which the store will use to hold it's data files. The store will create the directory if it does not already exist.flush_delay
: The flush delay is the amount of time in milliseconds that a store will delay persisting a messaging unit of work in hopes that it will be invalidated shortly thereafter by another unit of work which would negate the operation.read_threads
: The number of concurrent read threads to use when accessing the store. The value defaults to 10.
JDBM2 Store
The JDBM2 store implementation is redistributable by apache so it can be used out of the box without having to install any additional software. The problem with this store is that it's single threaded in nature and subsequently has several unwanted performance problems. For example, the store periodically will compact itself to reclaim disk space but the compaction is a slow process which pauses access to the store while the compaction is running.
You can enable the store by adding a jdbm2_store
element
inside your virtual_host
. Example:
...
<virtual_host id="default">
...
<jdbm2_store directory="${apollo.base}/data"/>
..
</virtual_host>
...
A jdbm2_store
element may be configured with the following attributes:
directory
: The directory which the store will use to hold it's data files. The store will create the directory if it does not already exist.flush_delay
: The flush delay is the amount of time in milliseconds that a store will delay persisting a messaging unit of work in hopes that it will be invalidated shortly thereafter by another unit of work which would negate the operation.compact_interval
: how often the store is compacted in seconds. If this attribute is not set, it will default to 60 seconds. Set to -1 to disable compaction altogether.
Security
The SSL/TLS Transport
Apollo supports SSL/TLS for transport level security to avoid 3rd
parties listening in on the communications between the broker and it's
clients. To enable it, you just need to add a connector which binds using
on of the secure transports such as ssl://
. It also requires having a
key_storage
configuration element under the broker
to configure the where
the encryption keys and certificates are stored.
Example:
...
<key_storage
file="${apollo.base}/etc/keystore"
password="password"
key_password="password"/>
<connector id="stomp-secure" bind="ssl://0.0.0.0:61614"/>
...
The connector
element's bind
attribute controls which secure transport
algorithm gets used by the sever. Supported values are:
ssl://
- Supports some version of SSLsslv2://
- Supports SSL version 2 or highersslv3://
- Supports SSL version 3tls://
- Supports some version of TLStlsv://
- Supports RFC 2246: TLS version 1.0tls1.1://
- Supports RFC 4346: TLS version 1.1
The attributes that you can configure on the key_storage
element are:
file
: Path to where the key store is located.password
: The key store passwordkey_password
: The password to the keys in the key store.store_type
: The type of key store, defaults toJKS
.trust_algorithm
: The trust management algorithm, defaults toSunX509
.key_algorithm
: The key management algorithm, defaults toSunX509
.
The SSL/TLS transport is an extension of the TCP transport and as such it supports all the same URI options which the TCP transport supports.
Authentication
The first step to securing the broker is authenticating users. The default
Apollo configurations use file based authentication. Authentication
is performed using JAAS who's config file is located in the instance
directory under etc/login.conf
. JAAS configuration files can define
multiple named authentication domains. The broker
element and
virtual_host
elements can be configured to authenticate against these
domains.
The authentication
element defined at the broker level will get used to
authenticate broker level administration functions and to authenticate
any virtual hosts which did not define an authentication
element. If you
want to disable authentication in a virtual host, you set the enable
attribute
to false
.
<broker xmlns="http://activemq.apache.org/schema/activemq/apollo">
<authentication domain="internal"/>
<virtual_host id="wine.com">
<authentication domain="external"/>
<host_name>wine.com</host_name>
</virtual_host>
<virtual_host id="internal.wine.com">
<host_name>internal.wine.com</host_name>
</virtual_host>
<virtual_host id="test">
<authentication enabled="false"/>
<host_name>cheeze.com</host_name>
</virtual_host>
<connector id="tcp" bind="tcp://0.0.0.0:61613"/>
</broker>
The above example uses 2 JAAS domains, internal
and external
. Broker
The wine.com
host will use the external domain, the internal.wine.com
host will use the internal domain and the test
host will not authenticate
users.
Using Custom Login Modules
Apollo uses JAAS to control against which systems users
authenticate. The default Apollo configurations use file based
authentication but with a simple change of the JAAS configuration, you can be
authenticating against local UNIX account or LDAP. Please reference
the JAAS documentation
for more details on how to edit the etc/login.conf
file.
Since different JAAS login modules produce principals of different class types, you may need to configure which of those class types to recognize as the user principal and the principal used to match against the access control lists (ACLs).
The default user principal classes recognized are
org.apache.activemq.jaas.UserPrincipal
and
javax.security.auth.x500.X500Principal
. You can change the default by
adding user_principal_kind
elements under the authentication
element.
The first principal who's type matches this list will be selected as the
user's identity for informational purposes.
Similarly, default acl principal class recognized is
org.apache.activemq.jaas.GroupPrincipal
. You can configure it by adding
acl_principal_kinds elements under the
authentication` element. The ACL
entries which do not have an explicit kind will default to using the the
kinds listed here.
Example of customizing the principal kinds used:
...
<authentication domain="apollo">
<user_principal_kind>com.sun.security.auth.UnixPrincipal</user_principal_kind>
<user_principal_kind>com.sun.security.auth.LdapPrincipal</user_principal_kind>
<acl_principal_kind>com.sun.security.auth.UnixPrincipal</acl_principal_kind>
<acl_principal_kind>com.sun.security.auth.LdapPrincipal</acl_principal_kind>
</authentication>
...
</broker>
Authorization
User authorization to broker resources is accomplished by configuring an
access control list using an acl
element on the broker
, connector
,
virtual_host
, topic
, dsub
, or queue
resources. The
acl defines which principals are allowed or denied access to perform actions
against the resources. An example of acl
is shown below:
<acl>
<send deny="guest"/>
<send allow="*"/>
<receive allow="app1"/>
</acl>
If a configuration resource does not have an acl
element defined within
it, then the resource allows access if the containing resource would allow
access to the action. If the action is not defined in the containing
resource then it allows anyone to access. The acl
is made up of a list of authorization rule entries. Each entry defines
that action the rule applies to and if the rule is allowing or denying
access to a user principal.
Users can have many principals of many different kinds associated with
them. The rules will only match up against principals of type
org.apache.activemq.jaas.GroupPrincipal
since that is the default
setting of the acl_principal_kind
of the authentication
domain.
If you want the rule to match against more/different kinds of principals,
you should update the authentication
element's configuration or you
explicitly state the kind you want to match against in your rule
definition. Example:
<acl>
<send deny="chirino" kind="org.apache.activemq.jaas.UserPrincipal"/>
<send allow="*"/>
</acl>
Wildcards
Wild cards can be used in the deny
, allow
, and kind
attributes to match
multiple values. Two types of wildcards are supported:
*
: Matches any value on zero or more principles.+
: Matches any value on one or more principles.
Examples of using the *
wild card:
<acl>
<connect allow="*"/>
</acl>
The previous example allows anyone to connect even if the subject they authenticated with has no principles associated with it.
Examples of using the +
wild card:
<acl>
<connect allow="+" kind="org.apache.activemq.jaas.UserPrincipal"/>
</acl>
The previous example allows an UserPrincipal
principal to connect. It
would reject the connection if subject that has no UserPrincipals
.
You can also use the wildcard on the kind attribute. When the wild
card is used on the kind attribute, then *
acts like the +
wild
card and only matches for one or more principles.
For example:
<acl>
<connect allow="Hiram" kind="*"/>
</acl>
The previous example allows a subject with at least one hiram
to
principal connect. The principal can be of any type..
Ordering
The order in which rule entries are defined are significant when the user matches multiple entries. The first entry the user matches determines if he will have access to the action. For example, lets say a user is groups 'blue' and 'red', and you are matching against an ACL list defined as:
<acl>
<send deny="blue"/>
<send allow="red"/>
</acl>
Then the user would not be allowed to send since <send deny="blue"/>
was
defined first. If the order in the ACL list were reversed, like
so:
<acl>
<send allow="red"/>
<send deny="blue"/>
</acl>
Then the user would be allowed access to the resource since the first rule
which matches the user is <send allow="red"/>
.
The type of resource being secured determines the types of actions that can be secured by the acl rule entries. Here is listing of which actions can be secured on which resources:
broker
,virtual_host
,topic
,queue
, anddsub
admin
: use of the administrative web interfacemonitor
: read only use of the administrative web interfaceconfig
: use of the administrative web interface to access and change the broker configuration.
connector
andvirtual_host
connect
: allows connections to the connector or virtual host
topic
,queue
anddsub
create
: allows creationdestroy
: allows destructionsend
: allows the user to send to the destinationreceive
: allows the user to send to do non_destructive read from the destination
queue
anddsub
consume
: allows the user to do destructive reads against the queue.
Encrypting Passwords in the Configuration
The etc/apollo.xml
file supports using ${<property-name>}
style
syntax. You can use any system properties and if the
etc/apollo.xml.properties
file exists, then any of the properties
defined there. Any of the properties values in the
etc/apollo.xml.properties
can be replaced with encrypted versions by
using the apollo encrypt
command.
Lets say you your current key_storage
contains plain text passwords that
need to be replaced with encrypted versions:
...
<key_storage
file="${apollo.base}/etc/keystore"
password="open"
key_password="sesame"/>
...
Lets first find out what the encrypted versions of the passwords would be.
Apollo encrypts and decrypts values using the password stored in
the APOLLO_ENCRYPTION_PASSWORD
environment variable.
The following is an example of how you can encrypt the previous passwords:
$ export APOLLO_ENCRYPTION_PASSWORD='keepmesafe'
$ apollo encrypt open
ENC(6r7HKCib0H8S+OuSfV+muQ==)
$ apollo encrypt sesame
ENC(FP+H2FIg++sSaOxg/ISknw==)
Once you have the encrypted passwords, you can add them to the
etc/apollo.xml.properties
file. Example:
store.pass=ENC(6r7HKCib0H8S+OuSfV+muQ==)
key.pass=ENC(FP+H2FIg++sSaOxg/ISknw==)
Finally the last step of securing the configuration is to replace the plain text passwords with variable references to the corresponding property names:
...
<key_storage
file="${apollo.base}/etc/keystore"
password="${store.pass}"
key_password="${key.pass}"/>
...
When you use encrypted passwords in your configuration, you MUST make
sure that the APOLLO_ENCRYPTION_PASSWORD
environment variable is set
to the proper value before starting the broker.
Web Based Administration
Apollo starts a web based administration interface on
http://127.0.0.1:61680
and
https://127.0.0.1:61681
by default. Note
that it binds to the loopback interface so that only local web
browsers can access the interface.
If the broker has authentication enabled and has defined an ACL configuring the admins of the broker, then the web interface will perform basic authentication and will only grant access to those users which are in the admin ACL.
If you want to disable the web the interface then you should remove
the web_admin
configuration elements. If you want to allow
remote administration, you should update the configuration so
it bind either the 0.0.0.0
or [::]
address.
For example:
<broker xmlns="http://activemq.apache.org/schema/activemq/apollo">
...
<web_admin bind="http://0.0.0.0:61680"/>
<web_admin bind="https://0.0.0.0:61681"/>
...
</broker>
A web_admin
element may be configured with the following attributes:
bind
: The address and port to bind the web interface on in URL syntax.
Managing Brokers
The rest of this section's example assume that you have created a broker
instance under the /var/lib/mybroker
directory or c:\mybroker
directory
if your on windows.
Running a Broker Instance in the Foreground
To start the broker instance in the foreground simply execute
bin/apollo-broker run
. Example:
/var/lib/mybroker/bin/apollo-broker run
To stop it, press Ctrl-C
to send the termination signal
to the process.
Managing a Background Broker Instance
On Linux/Unix
If you are on Unix, you can use the bin/apollo-broker-service
script
to manage the broker service. This script is compatible with the
/etc/init.d
style scripts most Unix/Linux systems use to control
background services.
On a Ubuntu OS, you install the service and have it run on start up by running:
sudo ln -s /var/lib/mybroker/bin/apollo-broker-service /etc/init.d/apollo
sudo update-rc.d apollo defaults
On a Redhat OS, you install the service and have it run on start up by running:
sudo ln -s /var/lib/mybroker/bin/apollo-broker-service /etc/init.d/apollo
sudo chkconfig apollo --add
On other Unixes, you install the service and have it run on start up by running:
sudo ln -s /var/lib/mybroker/bin/apollo-broker-service /etc/init.d/apollo
sudo ln -s /etc/init.d/apollo /etc/rc0.d/K80apollo
sudo ln -s /etc/init.d/apollo /etc/rc1.d/K80apollo
sudo ln -s /etc/init.d/apollo /etc/rc3.d/S20apollo
sudo ln -s /etc/init.d/apollo /etc/rc5.d/S20apollo
sudo ln -s /etc/init.d/apollo /etc/rc6.d/K80apollo
You can directly use the script to perform the following functions:
- starting:
apollo-broker-service start
- stopping:
apollo-broker-service stop
- restarting:
apollo-broker-service restart
- checking the status:
apollo-broker-service status
When the broker is started in the background, it creates
a data/apollo.pid
file which contains the process id of
the process executing the broker. This file is typically
used to integrate with an external watch dog process
such as Monit.
On Windows
Windows users can install the broker as background service using the
bin\apollo-broker-service
executable.
To install as a background service you would execute:
c:\mybroker\bin\apollo-broker-service install
Uninstalling the service is done using:
c:\mybroker\bin\apollo-broker-service uninstall
You can now use the standard Windows service tools to control starting and stopping the broker or you can also use the same executable to:
- start the service:
apollo-broker-service start
- stop the service:
apollo-broker-service stop
- check the status:
apollo-broker-service restart
If you want to customize the JVM options use to start the background
service, you will need to edit the bin\apollo-broker-service.xml
file. All service console and event logging perform by the background
service are stored under log\apollo-broker-service.*
.
Viewing Broker State
Apollo provides a web based interface for administrators to inspect
the runtime state of the Broker. If your running the broker on your local
machine, just open your web browser to http://localhost:61680
or https://localhost:61681
.
The web interface will display the status of the the connectors and show attached connections. It will also allow you to drill into each configured virtual host and view the topics and queues being used.
Please see the Management API documentation for more information on how to use the web based interface as a RESTful API.
Exporting/Importing Stores
Exporting compresses all the data in a virtual host's message store in a zip archive. Importing reverses the export process and restore the exported data back to a virtual host's message store. Exporting/Importing is typically used to:
- backup a message store
- migrate between different message store implementations
The stores which support exporting and importing are:
The broker must be stopped before using the import or export commands. Be careful when importing an archive, since it will first purge the message store of any data!
Use the apollo-broker store-export
command to export the data. For example:
/var/lib/mybroker/bin/apollo-broker store-export myarchive.zip
The above command will load the mybroker
's configuration and export the
first virtual host's messages store to the myarchive.zip
. You can use the
--virtual-host
command line option to be more specific of which virtual
host you wish to export.
Use the apollo-broker store-import
command to import the data. For example:
/var/lib/mybroker/bin/apollo-broker store-import myarchive.zip
Just like in the case of the store-export
command, it will load the
mybroker
's configuration and import the archive into the first virtual
host's message store.
Using the STOMP Protocol
Clients can connect to Apollo using the STOMP protocol. STOMP provides an interoperable messaging wire level protocol that allows any STOMP clients can communicate with any STOMP message broker. It aims to provide easy and widespread messaging interoperability among many languages, platforms and brokers.
Apollo supports the following versions of the STOMP specification:
The specification is short and simple to read, it is highly recommend that users to get familiar with it before using one of the many available client libraries.
Stomp Protocol Options
You can use the stomp
configuration element within the connector
element
in the apollo.xml
configuration file to change the default settings used
in the STOMP protocol implementation. The stomp
element supports the
following configuration attributes:
add_user_header
: Name of the header which will be added to every received message received. The value of the header will be set to the id of user that sent the message. Not set by default.max_header_length
: The maximum allowed length of a STOMP header. Defaults to 10240 (10k).max_headers
: The maximum number of allowed headers in a frame. Defaults to 1000.max_data_length
: The maximum size of the body portion of a STOMP frame.
Defaults to 104857600 (100 megs).
The stomp configuration element can also be used to control how the destination headers are parsed and interpreted. The supported attributes are:
queue_prefix
: Defaults to/queue/
topic_prefix
: Defaults to/topic/
path_separator
: Defaults to.
destination_separator
: Defaults to,
any_child_wildcard
: Defaults to*
regex_wildcard_start
: Defaults to{
regex_wildcard_end
: Defaults to}
any_descendant_wildcard
: Defaults to**
It also supports nested add_user_header
elements to more finely control how
user headers are added to received STOMP messages. The add_user_header
element
should contain the name of the header to set on the STOMP message. It also
supports the following attributes:
separator
: If user has multiple principles which match, this separator will be used to delimit them in the header. If not set, then only the first matching principle will be set in the header.kind
: The principle kind to look for. Defaults to*
(matches all principle kinds)
Example:
<connector id="tcp" bind="tcp://0.0.0.0:61613">
<stomp max_header_length="10000">
<add_user_header separator=",">user</add_user_header>
</stomp>
</connector>
Client Libraries
There are many open source STOMP clients for different platforms and languages. You can find a full listing of available clients at:
The Apollo distribution ships with an examples
directory
where you can find some simple examples of how to use some of those
clients to send and receive messages from a broker instance.
This section will focus on clarifying the parts of the STOMP specification which allow servers to choose their behavior and to describe how to access Apollo specific features.
Connecting
The default broker configuration secures access to the broker so that only
the admin
user can connect. The default password for the admin
user
is password
.
Example STOMP frame to connect to the broker:
CONNECT
login:admin
passcode:password
^@
STOMP 1.0 clients do specify which virtual host they are connecting to so
the broker connects those clients to the first virtual host defined in
it's configuration. STOMP 1.1 clients do specify a virtual host when they
connect. If no configured virtual host host_name
matches the client's
requested host, the connection is terminated with an ERROR. Therefore,
it is critical that the virtual hosts configuration define all the
possible host names that clients may connect to host with.
Destination Types
Apollo supports three types of destinations, queues, topics, and durable subscriptions.
The difference between queues and topics is how messages are delivered to consumers. A queue will load balance it's messages over the connected subscribers so that only one subscriber gets a message. Topics replicate every message sent to it to all the connected subscribers. Queues hold on to unconsumed messages even when there are no subscriptions attached, while a topic will drop messages when there are no connected subscriptions.
A durable subscription allows you to create a subscription against a topic which will queue messages even after the client disconnects. Clients can reconnect and consume the queued message originating from the topic at a later time.
If you want to send or subscribe to a queue, topic, or durable
subscription the STOMP destination should be prefixed with /queue/
,
/topic/
or /dsub/
respectively.
Example STOMP frame sending to a queue:
SEND
destination:/queue/a
hello queue a
^@
Reliable Messaging
Apollo supports reliable messaging by allowing messages to be persisted so that they can be recovered if there is failure which kills the broker. Processing persistent messages has orders of magnitude more overhead than non-persistent variety. You should only use it if your application really needs it.
To have a message be persistent, the sender must add the persistent:true
header to the SEND
STOMP frame. Furthermore, if you are not sending the
message in a transaction, it is recommend that you also add a receipt
header. Once the broker responds with a RECEIPT
frame correlated to
the send, you are guaranteed the broker will not drop the message even if
a failure occurs.
Example:
SEND
destination:/queue/a
receipt:001
hello queue a
^@
The client should not consider the send to have succeeded until he receives
the correlated RECEIPT
frame from the server. Example:
RECEIPT
receipt-id:001
^@
It is important to note that to do reliable messaging, your system will
be prone to receive duplicate messages. For example, if your sending
application dies before it receives the RECEIPT
frame, then when it starts
up again will probably try to send the same message again which may
cause a duplicate message to get placed on the broker.
You should only use subscribers which use the client
or client-individual
ack mode to consume reliable messages. Any messages on a queue delivered to a
client which have not been acked when the client disconnects will get
redelivered to another subscribed client.
Message Expiration
Apollo supports expiring old messages. Unconsumed expired messages
are automatically removed from the queue. You just need to specify when
the message expires by setting the expires
message header. The expiration
time must be specified as the number of milliseconds since the Unix epoch.
Example:
SEND
destination:/queue/a
expires:1308690148000
this message will expire on Tue Jun 21 17:02:28 EDT 2011
^@
Topic Durable Subscriptions
A durable subscription is a queue which is subscribed to a topic so that even if the client which created the durable subscription is not online, he can still get a copy of all the messages sent to the topic when he comes back online. Multiple clients can subscribe to the same durable subscription and since it's backed by a queue, those subscribers will have the topic's messages load balanced across them.
To create or reattach to a a durable subscription with STOMP, you uniquely name
the durable subscription using the id
header on the `
SCRIBE` frame and
also adding a persistent:true
header. Example:
SUBSCRIBE
id:mysub
persistent:true
destination:/topic/foo
^@
A standard UNSUBSCRIBE
frame does not destroy the durable subscription, it
only disconnects the client from the durable subscription. To destroy a
durable subscription, you must once again add persistent:true
header
to the UNSUBSCRIBE
frame. Example:
UNSUBSCRIBE
id:mysub
persistent:true
^@
If the durable subscription already exists you can address it directly
using /dsub/
prefix on the destination
header. For example,
send a message to the previously created mysub
durable subscription,
you send the following STOMP frame:
SEND
destination:/dsub/mysub
hello durable sub!
^@
Similarly, you can also subscribe to the subscription in the same way:
SUBSCRIBE
id:0
destination:/dsub/mysub
^@
Unlike typical STOMP subscriptions id's which are local to the STOMP
client's connection, the durable subscription id's are global across
a virtual host. If two different connections use the same durable
subscription id, then messages from the subscription will get load
balanced across the two connections. If the second connection uses
a different destination
or selector
header, then updates
the original subscription, and the original connection will
subsequently only receive messages matching the updated
destination or selector.
Browsing Subscriptions
A normal subscription on a queue will consume messages so that no other
subscription will get a copy of the message. If you want to browse all the
messages on a queue in a non-destructive fashion, you can create browsing
subscription. Browsing subscriptions also works with durable subscriptions
since they are backed by a queue. To make a a browsing subscription, just add the
browser:true
header to the SUBSCRIBE
frame. For example:
SUBSCRIBE
id:mysub
browser:true
destination:/queue/foo
^@
Once the broker sends a browsing subscription the last message in the queue,
it will send the subscription a special “end of browse” message to indicate
browsing has completed and that the subscription should not expect any more
messages. The “end of browse” message will have a browser:end
header set.
Example:
MESSAGE
subscription:mysub
browser:end
^@
Exclusive Subscriptions
We maintain the order of messages in queues and dispatch them to subscriptions in order. However if you have multiple subscriptions consuming from the same queue, you will loose the guarantee of processing the messages in order; since the messages may be processed concurrently on different subscribers.
Sometimes it is important to guarantee the order in which messages are processed. e.g. you don't want to process the update to an order until an insert has been done; or to go backwards in time, overwriting an newer update of an order with an older one etc.
So what folks have to do in clusters is often to only run one consumer process in a cluster to avoid loosing the ordering. The problem with this is that if that process goes down, no one is processing the queue any more, which can be problem.
Apollo supports exclusive subscriptions which avoids the end user having to restrict himself to only running one process. The broker will pick a single subscription to get all the messages for a queue to ensure ordering. If that subscription fails, the broker will auto failover and choose another subscription.
An exclusive subscription is created by adding a exclusive:true
header
to the SUBSCRIBE
frame. Example:
SUBSCRIBE
id:mysub
exclusive:true
destination:/queue/foo
^@
Destination Wildcards
We support destination wildcards to easily subscribe to multiple destinations with one subscription. This concept has been popular in financial market data for some time as a way of organizing events (such as price changes) into hierarchies and to use wildcards for easy subscription of the range of information you're interested in.
.
is used to separate names in a path*
is used to match any name in a path{regex}
is used to match a path name against a regular expression.**
is used to recursively match path names
For example imagine you are sending price messages from a stock exchange feed. You might use some kind of destination naming conventions such as:
/topic/PRICE.STOCK.NASDAQ.IBM
to publish IBM's price on NASDAQ and/topic/PRICE.STOCK.NYSE.SUNW
to publish Sun's price on the New York Stock Exchange
A subscriber could then use exact destinations to subscribe to exactly the prices it requires. Or it could use wildcards to define hierarchical pattern matches to the destinations to subscribe from.
For example using the example above, these subscriptions are possible
/topic/PRICE.**
: Any price for any product on any exchange/topic/PRICE.STOCK.**
: Any price for a stock on any exchange/topic/PRICE.STOCK.NASDAQ.*
: Any stock price on NASDAQ/topic/PRICE.STOCK.*.IBM
: Any IBM stock price on any exchange/topic/PRICE.STOCK.*.I*
: Any stock price starting with 'I' on any exchange/topic/PRICE.STOCK.*.*{[0-9]}
: Any stock price that ends in a digit on any exchange
Destination wildcards can only be used in a SUBSCRIBE frame.
Composite Destinations
You can use composite destinations to send or subscribe to multiple
destinations at one time. You use separator of ,
between destination
names. For example, to send one message to 2 queues and 1 topic:
SEND
destination:/queue/a,/queue/b,/topic/c
Composites rock!
^@
Message Selectors
Message selectors allow a subscription to only receive a subset of the messages sent to a destination. The selector acts like a filter which is applied against the message properties and only those messages which match pass through to the subscription.
Selectors are defined using SQL 92 syntax and typically apply to message headers; whether the standard properties available on a JMS message or custom headers you can add via the JMS code.
Here is an example:
type = 'car' AND color = 'blue' AND weight > 2500
To create a subscription with a message selector, you set the selector
header in the STOMP SUBSCRIBE
frame to the desired selector. Example:
SUBSCRIBE
id:sub0
selector:type = 'car' AND color = 'blue' AND weight > 2500
destination:/topic/foo
^@
Destination Name Restrictions
Destination names are restricted to using the characters a-z
, A-Z
, 0-9
,
_
, -
%
, ~
, or .
in addition to composite separator ,
and the wild
card *
.