NAME

  rq v1.0.0

SYNOPSIS

  rq (queue | export RQ_Q=q) mode [mode_args]* [options]*

DESCRIPTION

  ruby queue (rq) is a tool used to create instant linux clusters by managing
  sqlite databases as nfs mounted priority work queues.  multiple instances of
  rq running from multiples hosts can work from these queues to
  distribute processing load to n nodes - bringing many dozens of otherwise
  powerful cpus to their knees with a single blow.  clearly this software should
  be kept out of the hands of free radicals, seti enthusiasts, and j. safran.

  the central concept of rq is that n nodes work in isolation to pull jobs from
  an central nfs mounted priority work queue in a synchronized fashion.  the
  nodes have absolutely no knowledge of each other and all communication if done
  via the queue meaning that, so long as the queue is available via nfs and a
  single node is running jobs from it, the system will continue to process jobs.
  there is no centralized process whatsoever - all nodes work to take jobs from
  the queue and run them as fast as possible.  this creates a system which load
  balances automatically and is robust in face of node failures.

  the first argument to any rq command is the name of the queue.  this
  name may be omitted if, and only if, the environment variable RQ_Q has been
  set to contain the absolute path of target queue.

  rq operates in one of the modes create, submit, list, status,
  delete, update, query, execute, configure, snapshot, lock, backup, help, or
  feed.  depending on the mode of operation and the options used the meaning of
  'mode_args' may change.

MODES

  the following mode abbreviations exist

    c  => create
    s  => submit
    l  => list
    ls => list
    t  => status
    d  => delete
    rm => delete
    u  => update
    q  => query
    e  => execute
    C  => configure
    S  => snapshot
    L  => lock
    b  => backup
    h  => help
    f  => feed

create, c :

    create a queue.  the queue must be located on an nfs mounted file system
    visible from all nodes intended to run jobs from it.

    examples :

      0) to create a queue
          ~ > rq /path/to/nfs/mounted/q create
        or simply
          ~ > rq /path/to/nfs/mounted/q c

submit, s :

    submit jobs to a queue to be proccesed by a feeding node.  any 'mode_args'
    are taken as the command to run.  note that 'mode_args' are subject to shell
    expansion - if you don't understand what this means do not use this feature
    and pass jobs on stdin.

    when running in submit mode a file may by specified as a list of commands to
    run using the '--infile, -i' option.  this file is taken to be a newline
    separated list of commands to submit, blank lines and comments (#) are
    allowed.  if submitting a large number of jobs the input file method is
    MUCH, more efficient.  if no commands are specified on the command line rq
    automatically reads them from STDIN.  yaml formatted files are also allowed
    as input (http://www.yaml.org/) - note that the output of nearly all rq
    commands is valid yaml and may, therefore, be piped as input into the submit
    command.

    when submitting the '--priority, -p' option can be used here to determine
    the priority of jobs.  priorities may be any whole number - zero is the
    default.  note that submission of a high priority job will NOT supplant
    currently running low priority jobs, but higher priority jobs WILL always
    migrate above lower priority jobs in the queue in order that they be run as
    soon as possible.  constant submission of high priority jobs may create a
    starvation situation whereby low priority jobs are never allowed to run.
    avoiding this situation is the responsibility of the user.  the only
    guaruntee rq makes regarding job execution is that jobs are
    executed in an 'oldest highest priority' order and that running jobs are
    never supplanted.

    examples :

      0) submit the job ls to run on some feeding host

        ~ > rq q s ls

      1) submit the job ls to run on some feeding host, at priority 9

        ~ > rq -p9 q s ls

      2) submit 42000 jobs (quietly) from a command file.

        ~ > wc -l cmdfile
        42000
        ~ > rq q s -q < cmdfile

      3) submit 42 priority 9 jobs from a command file.

        ~ > wc -l cmdfile
        42
        ~ > rq -p9 q s < cmdfile

      4) submit 42 priority 9 jobs from a command file, marking them as
         'important' using the '--tag, -t' option.

        ~ > wc -l cmdfile
        42
        ~ > rq -p9 -timportant q s < cmdfile

      5) re-submit all the 'important' jobs (see 'query' section below)

        ~ > rq q query tag=important | rq q s

      6) re-submit all jobs which are already finished (see 'list' section
         below)

        ~ > rq q l f | rq q s

list, l, ls :

    list mode lists jobs of a certain state or job id.  state may be one of
    pending, running, finished, dead, or all.  any 'mode_args' that are numbers
    are taken to be job id's to list.

    states may be abbreviated to uniqueness, therefore the following shortcuts
    apply :

      p => pending
      r => running
      f => finished
      d => dead
      a => all

    examples :

      0) show everything in q
          ~ > rq q list all
        or
          ~ > rq q l all
        or
          ~ > export RQ_Q=q
          ~ > rq l

      1) show q's pending jobs
          ~ > rq q list pending

      2) show q's running jobs
          ~ > rq q list running

      3) show q's finished jobs
          ~ > rq q list finshed

      4) show job id 42
          ~ > rq q l 42

status, t :

    status mode shows the global state the queue.  there are no 'mode_args'.
    the meaning of each state is as follows:

      pending  => no feeder has yet taken this job
      running  => a feeder has taken this job
      finished => a fe