Erlang's pool module provides a very easy to use load-balancing pooling mechanism implementing a master-slave pooling paradigm with one master and many slaves. By starting the pool, the master tries to log in to all slave machines and start the slave (see slave(3)). At that point the pool is set up and ready to use - basically. Stey by step: First of all a useful pool needs at least one additional slave node (with the pool module the master incorporates a slave node at the same time). The full hostnames must be listed as erlang atoms in the file .hosts.erlang, which resides either in the current working directory or in your home directory. Example:
'foo.bar.priv'. 'blubb.bar.priv'. (newline)
Make sure that it is possible to log in to all machines without a password prompt. ~/.ssh/authorized_keys and ~/.ssh/config might be of help here. If you get an error involving "ssh-askpass" later, try to log in manually first. In order to let the erlang nodes communicate with each other their cookies must be synced. This can be done by setting the ~/.erlang.cookie or by passing the command line argument -setcookie COOKIE. That's all for the basic setup. To try it out we could start erlang like so:
erl -pa boo -setcookie pooltest000 -name pooltest@`hostname` -rsh ssh
and start the pool by
pool:start(pooltest, lists:concat(["-setcookie ", erlang:get_cookie()])).
The argument -pa boo adds boo to the code search path and -rsh ssh tells pool to use ssh instead of rsh. Next, it would be nice to automatically distribute our local code base to all the slave nodes. Luckily the code module provides a simple way to do this:
... {_Module, Binary, Filename} = code:get_object_code(Module), rpc:call(Node, code, load_binary, [Module, Filename, Binary]), ...
Distributed processes can be easily created using pool:pspawn/3.
A complete example:
Continue reading "Pooling and Automated Code Distribution with..." »