Salt Reactor System

What is the reactor system?

The reactor system provided by SaltStack is the ability to execute commands based on detected events in the environment. This system binds our normal state files to event tags observed on a Salt Master. When the Salt Master sees an event tag it runs the associated state file. Easy!

As a quick aside, we briefly touched the reactor system in my post about Using Salt-Cloud in AWS EC2.

Anyway, back to the reactor. In order to use the reactor system we first need to enable it in the salt-master's configuration file. This is done by adding a reactor section, which can then take arguments in the form of tag, state file. Here's a quick example:

reactor:
  - 'event/tag/path/argument/1':
    - '/srv/reactor/standard.sls'

SaltStack uses a local ZeroMQ interface to receive events from clients. As mentioned above the event system fires events with a very specific format. Every event has a tag (tags allow for fast filtering). Every tag has a data structure. This data structure is a Python dict, which contains relevant information about the event.

Here's a more comprehensive example of configuring the reactor system either through /etc/salt/master, or /etc/salt/master.d/reactor.conf.

reactor:                            # Master config section "reactor"
  • ‘salt/minion//start’: # Match tag “salt/minion//start”

    • /srv/reactor/start.sls # Things to do when a minion starts
    • /srv/reactor/monitor.sls # Other things to do
    • /srv/reactor/cleanup.sls # Other things to do
  • ‘salt/cloud/*/destroyed’: # Globs can be used to matching tags

    • /srv/reactor/destroy/*.sls # Globs can be used to match file names
  • ‘myco/custom/event/tag’: # React to custom event tags

    • salt://reactor/mycustom.sls # Put reactor files under file_roots

In this example we can start to see how this could be useful for reacting to events in our environment... a minion starting up, first time a minion joins, when an EC2 instance is terminated, or when a minion joins a cluster (like Cassandra, or Memcache...).

The Reactor sends commands down to minions in the exact same way Salt's CLI interface does. It calls a function locally on the master that sends the name of the function as well as a list of any arguments and a dictionary of any keyword arguments that the minion should use to execute that function.

And here's an example of what a reactor state file could look like, /srv/reactor/cleanup.sls:



clean_tmp: local.cmd.run: - tgt: ‘’ - arg: - rm -rf /tmp/

Or an example showing how to run a highstate on newly made minions:

/etc/salt/master.d/reactor

reactor:

  • ‘salt/cloud/*/created’:
    • ‘/srv/reactor/startup_highstate.sls’


/srv/reactor/startup_highstate.sls

reactor_highstate: cmd.state.highstate: - tgt: {{ data[‘name’] }}

Or a useful HAProxy example:

/etc/salt/master.d/reactor.conf

‘salt/key’ tag is used when a key is accepted.

reactor:

  • ‘salt/key’:
    • /srv/salt/haproxy/react_new_minion.sls


/srv/salt/haproxy/react_new_minion.sls

Pass new minion id to the event state file via inline pillar.

{% if data[‘act’] == ‘accept’ and data[‘id’].startswith(‘web’) %} add_new_minion_to_pool: local.state.sls: - tgt: ‘haproxy*’ - arg: - haproxy.refresh_pool - kwarg: pillar: new_minion: {{ data[‘id’] }} {% endif %}

/srv/salt/haproxy/refresh_pool.sls:

Refresh haproxy hosts list, but leave new minion as disabled.

{% set new_minion = salt‘pillar.get’ %}

listen web :80 balance source {% for server,ip in salt[‘mine.get’](‘web’, ‘network.interfaces’, [‘eth0’]).items() %} {% if server == new_minion %} server {{ server }} {{ ip }}:80 disabled {% else %} server {{ server }} {{ ip }}:80 check {% endif %} {% endfor %}

I highly recommend taking a look at the official documentation, available here.