backtype.storm
Class Config

java.lang.Object
  java.util.AbstractMap<K,V>
      java.util.HashMap<java.lang.String,java.lang.Object>
          backtype.storm.Config

All Implemented Interfaces:

java.io.Serializable, java.lang.Cloneable, java.util.Map<java.lang.String,java.lang.Object>

public class Config
extends java.util.HashMap<java.lang.String,java.lang.Object>

Topology configs are specified as a plain old map. This class provides a convenient way to create a topology config map by providing setter methods for all the configs that can be set. It also makes it easier to do things like add serializations.

This class also provides constants for all the configurations possible on a Storm cluster and Storm topology. Default values for these configs can be found in defaults.yaml.

Note that you may put other configurations in any of the configs. Storm will ignore anything it doesn't recognize, but your topologies are free to make use of them by reading them in the prepare method of Bolts or the open method of Spouts. .

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class java.util.AbstractMap

java.util.AbstractMap.SimpleEntry<K,V>, java.util.AbstractMap.SimpleImmutableEntry<K,V>

Field Summary

static java.lang.String	DEV_ZOOKEEPER_PATH The path to use as the zookeeper dir when running a zookeeper server via "storm dev-zookeeper".
static java.lang.String	DRPC_INVOCATIONS_PORT This port on Storm DRPC is used by DRPC topologies to receive function invocations and send results back.
static java.lang.String	DRPC_PORT This port is used by Storm DRPC for receiving DPRC requests from clients.
static java.lang.String	DRPC_REQUEST_TIMEOUT_SECS The timeout on DRPC requests within the DRPC server.
static java.lang.String	DRPC_SERVERS List of DRPC servers so that the DRPCSpout knows who to talk to.
static java.lang.String	JAVA_LIBRARY_PATH This value is passed to spawned JVMs (e.g., Nimbus, Supervisor, and Workers) for the java.library.path value.
static java.lang.String	NIMBUS_CHILDOPTS This parameter is used by the storm-deploy project to configure the jvm options for the nimbus daemon.
static java.lang.String	NIMBUS_CLEANUP_INBOX_FREQ_SECS How often nimbus should wake the cleanup thread to clean the inbox.
static java.lang.String	NIMBUS_FILE_COPY_EXPIRATION_SECS During upload/download with the master, how long an upload or download connection is idle before nimbus considers it dead and drops the connection.
static java.lang.String	NIMBUS_HOST The host that the master server is running on.
static java.lang.String	NIMBUS_INBOX_JAR_EXPIRATION_SECS The length of time a jar file lives in the inbox before being deleted by the cleanup thread.
static java.lang.String	NIMBUS_MONITOR_FREQ_SECS How often nimbus should wake up to check heartbeats and do reassignments.
static java.lang.String	NIMBUS_REASSIGN Whether or not nimbus should reassign tasks if it detects that a task goes down.
static java.lang.String	NIMBUS_SUPERVISOR_TIMEOUT_SECS How long before a supervisor can go without heartbeating before nimbus considers it dead and stops assigning new work to it.
static java.lang.String	NIMBUS_TASK_LAUNCH_SECS A special timeout used when a task is initially launched.
static java.lang.String	NIMBUS_TASK_TIMEOUT_SECS How long without heartbeating a task can go before nimbus will consider the task dead and reassign it to another location.
static java.lang.String	NIMBUS_THRIFT_PORT Which port the Thrift interface of Nimbus should run on.
static java.lang.String	STORM_CLUSTER_MODE The mode this Storm cluster is running in.
static java.lang.String	STORM_ID The id assigned to a running topology.
static java.lang.String	STORM_LOCAL_DIR A directory on the local filesystem used by Storm for any local filesystem usage it needs.
static java.lang.String	STORM_LOCAL_HOSTNAME The hostname the supervisors/workers should report to nimbus.
static java.lang.String	STORM_LOCAL_MODE_ZMQ Whether or not to use ZeroMQ for messaging in local mode.
static java.lang.String	STORM_SCHEDULER A global task scheduler used to assign topologies's tasks to supervisors' wokers.
static java.lang.String	STORM_ZOOKEEPER_AUTH_PAYLOAD A string representing the payload for Zookeeper authentication.
static java.lang.String	STORM_ZOOKEEPER_AUTH_SCHEME The Zookeeper authentication scheme to use, e.g.
static java.lang.String	STORM_ZOOKEEPER_CONNECTION_TIMEOUT The connection timeout for clients to ZooKeeper.
static java.lang.String	STORM_ZOOKEEPER_PORT The port Storm will use to connect to each of the ZooKeeper servers.
static java.lang.String	STORM_ZOOKEEPER_RETRY_INTERVAL The interval between retries of a Zookeeper operation.
static java.lang.String	STORM_ZOOKEEPER_RETRY_TIMES The number of times to retry a Zookeeper operation.
static java.lang.String	STORM_ZOOKEEPER_ROOT The root location at which Storm stores data in ZooKeeper.
static java.lang.String	STORM_ZOOKEEPER_SERVERS A list of hosts of ZooKeeper servers used to manage the cluster.
static java.lang.String	STORM_ZOOKEEPER_SESSION_TIMEOUT The session timeout for clients to ZooKeeper.
static java.lang.String	SUPERVISOR_CHILDOPTS This parameter is used by the storm-deploy project to configure the jvm options for the supervisor daemon.
static java.lang.String	SUPERVISOR_ENABLE Whether or not the supervisor should launch workers assigned to it.
static java.lang.String	SUPERVISOR_HEARTBEAT_FREQUENCY_SECS how often the supervisor sends a heartbeat to the master.
static java.lang.String	SUPERVISOR_MONITOR_FREQUENCY_SECS How often the supervisor checks the worker heartbeats to see if any of them need to be restarted.
static java.lang.String	SUPERVISOR_SCHEDULER_META the metadata configed on the supervisor
static java.lang.String	SUPERVISOR_SLOTS_PORTS A list of ports that can run workers on this supervisor.
static java.lang.String	SUPERVISOR_WORKER_START_TIMEOUT_SECS How long a worker can go without heartbeating during the initial launch before the supervisor tries to restart the worker process.
static java.lang.String	SUPERVISOR_WORKER_TIMEOUT_SECS How long a worker can go without heartbeating before the supervisor tries to restart the worker process.
static java.lang.String	TASK_HEARTBEAT_FREQUENCY_SECS How often a task should heartbeat its status to the master.
static java.lang.String	TASK_REFRESH_POLL_SECS How often a task should sync its connections with other tasks (if a task is reassigned, the other tasks sending messages to it need to refresh their connections).
static java.lang.String	TOPOLOGY_ACKER_EXECUTORS How many executors to spawn for ackers.
static java.lang.String	TOPOLOGY_AUTO_TASK_HOOKS A list of task hooks that are automatically added to every spout and bolt in the topology.
static java.lang.String	TOPOLOGY_DEBUG When set to true, Storm will log every message that's emitted.
static java.lang.String	TOPOLOGY_DISRUPTOR_WAIT_STRATEGY Configure the wait strategy used for internal queuing.
static java.lang.String	TOPOLOGY_ENABLE_MESSAGE_TIMEOUTS True if Storm should timeout messages or not.
static java.lang.String	TOPOLOGY_EXECUTOR_RECEIVE_BUFFER_SIZE The size of the Disruptor receive queue for each executor.
static java.lang.String	TOPOLOGY_EXECUTOR_SEND_BUFFER_SIZE The size of the Disruptor send queue for each executor.
static java.lang.String	TOPOLOGY_FALL_BACK_ON_JAVA_SERIALIZATION Whether or not to use Java serialization in a topology.
static java.lang.String	TOPOLOGY_KRYO_DECORATORS A list of classes that customize storm's kryo instance during start-up.
static java.lang.String	TOPOLOGY_KRYO_REGISTER A list of serialization registrations for Kryo ( http://code.google.com/p/kryo/ ), the underlying serialization framework for Storm.
static java.lang.String	TOPOLOGY_MAX_SPOUT_PENDING The maximum number of tuples that can be pending on a spout task at any given time.
static java.lang.String	TOPOLOGY_MAX_TASK_PARALLELISM The maximum parallelism allowed for a component in this topology.
static java.lang.String	TOPOLOGY_MESSAGE_TIMEOUT_SECS The maximum amount of time given to the topology to fully process a message emitted by a spout.
static java.lang.String	TOPOLOGY_NAME Name of the topology.
static java.lang.String	TOPOLOGY_OPTIMIZE Whether or not the master should optimize topologies by running multiple tasks in a single thread where appropriate.
static java.lang.String	TOPOLOGY_RECEIVER_BUFFER_SIZE The maximum number of messages to batch from the thread receiving off the network to the executor queues.
static java.lang.String	TOPOLOGY_SKIP_MISSING_KRYO_REGISTRATIONS Whether or not Storm should skip the loading of kryo registrations for which it does not know the class or have the serializer implementation.
static java.lang.String	TOPOLOGY_SLEEP_SPOUT_WAIT_STRATEGY_TIME_MS The amount of milliseconds the SleepEmptyEmitStrategy should sleep for.
static java.lang.String	TOPOLOGY_SPOUT_WAIT_STRATEGY A class that implements a strategy for what to do when a spout needs to wait.
static java.lang.String	TOPOLOGY_STATE_SYNCHRONIZATION_TIMEOUT_SECS The maximum amount of time a component gives a source of state to synchronize before it requests synchronization again.
static java.lang.String	TOPOLOGY_STATS_SAMPLE_RATE The percentage of tuples to sample to produce stats for a task.
static java.lang.String	TOPOLOGY_TASKS How many instances to create for a spout/bolt.
static java.lang.String	TOPOLOGY_TICK_TUPLE_FREQ_SECS How often a tick tuple from the "__system" component and "__tick" stream should be sent to tasks.
static java.lang.String	TOPOLOGY_TRANSACTIONAL_ID This config is available for TransactionalSpouts, and contains the id ( a String) for the transactional topology.
static java.lang.String	TOPOLOGY_TRANSFER_BUFFER_SIZE The size of the Disruptor transfer queue for each worker.
static java.lang.String	TOPOLOGY_WORKER_CHILDOPTS Topology-specific options for the worker child process.
static java.lang.String	TOPOLOGY_WORKER_SHARED_THREAD_POOL_SIZE The size of the shared thread pool for worker tasks to make use of.
static java.lang.String	TOPOLOGY_WORKERS How many processes should be spawned around the cluster to execute this topology.
static java.lang.String	TRANSACTIONAL_ZOOKEEPER_PORT The port to use to connect to the transactional zookeeper servers.
static java.lang.String	TRANSACTIONAL_ZOOKEEPER_ROOT The root directory in ZooKeeper for metadata about TransactionalSpouts.
static java.lang.String	TRANSACTIONAL_ZOOKEEPER_SERVERS The list of zookeeper servers in which to keep the transactional state.
static java.lang.String	UI_CHILDOPTS Childopts for Storm UI Java process.
static java.lang.String	UI_PORT Storm UI binds to this port.
static java.lang.String	WORKER_CHILDOPTS The jvm opts provided to workers launched by this supervisor.
static java.lang.String	WORKER_HEARTBEAT_FREQUENCY_SECS How often this worker should heartbeat to the supervisor.
static java.lang.String	ZMQ_LINGER_MILLIS How long a connection should retry sending messages to a target host when the connection is closed.
static java.lang.String	ZMQ_THREADS The number of threads that should be used by the zeromq context in each worker process.

Constructor Summary

Config()

Method Summary

void	registerDecorator(java.lang.Class<? extends IKryoDecorator> klass)
void	registerSerialization(java.lang.Class klass)
void	registerSerialization(java.lang.Class klass, java.lang.Class<? extends Serializer> serializerClass)
void	setDebug(boolean isOn)
void	setFallBackOnJavaSerialization(boolean fallback)
void	setMaxSpoutPending(int max)
void	setMaxTaskParallelism(int max)
void	setMessageTimeoutSecs(int secs)
void	setNumAckers(int numExecutors)
void	setNumWorkers(int workers)
void	setOptimize(boolean isOn)
void	setSkipMissingKryoRegistrations(boolean skip)
void	setStatsSampleRate(double rate)

Methods inherited from class java.util.HashMap

clear, clone, containsKey, containsValue, entrySet, get, isEmpty, keySet, put, putAll, remove, size, values

Methods inherited from class java.util.AbstractMap

equals, hashCode, toString

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface java.util.Map

equals, hashCode

Field Detail

STORM_ZOOKEEPER_SERVERS

public static java.lang.String STORM_ZOOKEEPER_SERVERS

A list of hosts of ZooKeeper servers used to manage the cluster.

STORM_ZOOKEEPER_PORT

public static java.lang.String STORM_ZOOKEEPER_PORT

The port Storm will use to connect to each of the ZooKeeper servers.

STORM_LOCAL_DIR

public static java.lang.String STORM_LOCAL_DIR

A directory on the local filesystem used by Storm for any local filesystem usage it needs. The directory must exist and the Storm daemons must have permission to read/write from this location.

STORM_SCHEDULER

public static java.lang.String STORM_SCHEDULER

A global task scheduler used to assign topologies's tasks to supervisors' wokers. If this is not set, a default system scheduler will be used.

STORM_CLUSTER_MODE

public static java.lang.String STORM_CLUSTER_MODE

The mode this Storm cluster is running in. Either "distributed" or "local".

STORM_LOCAL_HOSTNAME

public static java.lang.String STORM_LOCAL_HOSTNAME

The hostname the supervisors/workers should report to nimbus. If unset, Storm will get the hostname to report by calling InetAddress.getLocalHost().getCanonicalHostName(). You should set this config when you dont have a DNS which supervisors/workers can utilize to find each other based on hostname got from calls to InetAddress.getLocalHost().getCanonicalHostName().

STORM_LOCAL_MODE_ZMQ

public static java.lang.String STORM_LOCAL_MODE_ZMQ

Whether or not to use ZeroMQ for messaging in local mode. If this is set to false, then Storm will use a pure-Java messaging system. The purpose of this flag is to make it easy to run Storm in local mode by eliminating the need for native dependencies, which can be difficult to install. Defaults to false.

STORM_ZOOKEEPER_ROOT

public static java.lang.String STORM_ZOOKEEPER_ROOT

The root location at which Storm stores data in ZooKeeper.

STORM_ZOOKEEPER_SESSION_TIMEOUT

public static java.lang.String STORM_ZOOKEEPER_SESSION_TIMEOUT

The session timeout for clients to ZooKeeper.

STORM_ZOOKEEPER_CONNECTION_TIMEOUT

public static java.lang.String STORM_ZOOKEEPER_CONNECTION_TIMEOUT

The connection timeout for clients to ZooKeeper.

STORM_ZOOKEEPER_RETRY_TIMES

public static java.lang.String STORM_ZOOKEEPER_RETRY_TIMES

The number of times to retry a Zookeeper operation.

STORM_ZOOKEEPER_RETRY_INTERVAL

public static java.lang.String STORM_ZOOKEEPER_RETRY_INTERVAL

The interval between retries of a Zookeeper operation.

STORM_ZOOKEEPER_AUTH_SCHEME

public static java.lang.String STORM_ZOOKEEPER_AUTH_SCHEME

The Zookeeper authentication scheme to use, e.g. "digest". Defaults to no authentication.

STORM_ZOOKEEPER_AUTH_PAYLOAD

public static java.lang.String STORM_ZOOKEEPER_AUTH_PAYLOAD

A string representing the payload for Zookeeper authentication. It gets serialized using UTF-8 encoding during authentication.

STORM_ID

public static java.lang.String STORM_ID

The id assigned to a running topology. The id is the storm name with a unique nonce appended.

NIMBUS_HOST

public static java.lang.String NIMBUS_HOST

The host that the master server is running on.

NIMBUS_THRIFT_PORT

public static java.lang.String NIMBUS_THRIFT_PORT

Which port the Thrift interface of Nimbus should run on. Clients should connect to this port to upload jars and submit topologies.

NIMBUS_CHILDOPTS

public static java.lang.String NIMBUS_CHILDOPTS

This parameter is used by the storm-deploy project to configure the jvm options for the nimbus daemon.

NIMBUS_TASK_TIMEOUT_SECS

public static java.lang.String NIMBUS_TASK_TIMEOUT_SECS

How long without heartbeating a task can go before nimbus will consider the task dead and reassign it to another location.

NIMBUS_MONITOR_FREQ_SECS

public static java.lang.String NIMBUS_MONITOR_FREQ_SECS

How often nimbus should wake up to check heartbeats and do reassignments. Note that if a machine ever goes down Nimbus will immediately wake up and take action. This parameter is for checking for failures when there's no explicit event like that occuring.

NIMBUS_CLEANUP_INBOX_FREQ_SECS

public static java.lang.String NIMBUS_CLEANUP_INBOX_FREQ_SECS

How often nimbus should wake the cleanup thread to clean the inbox.

See Also:

NIMBUS_INBOX_JAR_EXPIRATION_SECS

NIMBUS_INBOX_JAR_EXPIRATION_SECS

public static java.lang.String NIMBUS_INBOX_JAR_EXPIRATION_SECS

The length of time a jar file lives in the inbox before being deleted by the cleanup thread. Probably keep this value greater than or equal to NIMBUS_CLEANUP_INBOX_JAR_EXPIRATION_SECS. Note that the time it takes to delete an inbox jar file is going to be somewhat more than NIMBUS_CLEANUP_INBOX_JAR_EXPIRATION_SECS (depending on how often NIMBUS_CLEANUP_FREQ_SECS is set to).

See Also:

NIMBUS_CLEANUP_FREQ_SECS

NIMBUS_SUPERVISOR_TIMEOUT_SECS

public static java.lang.String NIMBUS_SUPERVISOR_TIMEOUT_SECS

How long before a supervisor can go without heartbeating before nimbus considers it dead and stops assigning new work to it.

NIMBUS_TASK_LAUNCH_SECS

public static java.lang.String NIMBUS_TASK_LAUNCH_SECS

A special timeout used when a task is initially launched. During launch, this is the timeout used until the first heartbeat, overriding nimbus.task.timeout.secs.

A separate timeout exists for launch because there can be quite a bit of overhead to launching new JVM's and configuring them.

NIMBUS_REASSIGN

public static java.lang.String NIMBUS_REASSIGN

Whether or not nimbus should reassign tasks if it detects that a task goes down. Defaults to true, and it's not recommended to change this value.

NIMBUS_FILE_COPY_EXPIRATION_SECS

public static java.lang.String NIMBUS_FILE_COPY_EXPIRATION_SECS

During upload/download with the master, how long an upload or download connection is idle before nimbus considers it dead and drops the connection.

UI_PORT

public static java.lang.String UI_PORT

Storm UI binds to this port.

UI_CHILDOPTS

public static java.lang.String UI_CHILDOPTS

Childopts for Storm UI Java process.

DRPC_SERVERS

public static java.lang.String DRPC_SERVERS

List of DRPC servers so that the DRPCSpout knows who to talk to.

DRPC_PORT

public static java.lang.String DRPC_PORT

This port is used by Storm DRPC for receiving DPRC requests from clients.

DRPC_INVOCATIONS_PORT

public static java.lang.String DRPC_INVOCATIONS_PORT

This port on Storm DRPC is used by DRPC topologies to receive function invocations and send results back.

DRPC_REQUEST_TIMEOUT_SECS

public static java.lang.String DRPC_REQUEST_TIMEOUT_SECS

The timeout on DRPC requests within the DRPC server. Defaults to 10 minutes. Note that requests can also timeout based on the socket timeout on the DRPC client, and separately based on the topology message timeout for the topology implementing the DRPC function.

SUPERVISOR_SCHEDULER_META

public static java.lang.String SUPERVISOR_SCHEDULER_META

the metadata configed on the supervisor

SUPERVISOR_SLOTS_PORTS

public static java.lang.String SUPERVISOR_SLOTS_PORTS

A list of ports that can run workers on this supervisor. Each worker uses one port, and the supervisor will only run one worker per port. Use this configuration to tune how many workers run on each machine.

SUPERVISOR_CHILDOPTS

public static java.lang.String SUPERVISOR_CHILDOPTS

This parameter is used by the storm-deploy project to configure the jvm options for the supervisor daemon.

SUPERVISOR_WORKER_TIMEOUT_SECS

public static java.lang.String SUPERVISOR_WORKER_TIMEOUT_SECS

How long a worker can go without heartbeating before the supervisor tries to restart the worker process.

SUPERVISOR_WORKER_START_TIMEOUT_SECS

public static java.lang.String SUPERVISOR_WORKER_START_TIMEOUT_SECS

How long a worker can go without heartbeating during the initial launch before the supervisor tries to restart the worker process. This value override supervisor.worker.timeout.secs during launch because there is additional overhead to starting and configuring the JVM on launch.

SUPERVISOR_ENABLE

public static java.lang.String SUPERVISOR_ENABLE

Whether or not the supervisor should launch workers assigned to it. Defaults to true -- and you should probably never change this value. This configuration is used in the Storm unit tests.

SUPERVISOR_HEARTBEAT_FREQUENCY_SECS

public static java.lang.String SUPERVISOR_HEARTBEAT_FREQUENCY_SECS

how often the supervisor sends a heartbeat to the master.

SUPERVISOR_MONITOR_FREQUENCY_SECS

public static java.lang.String SUPERVISOR_MONITOR_FREQUENCY_SECS

How often the supervisor checks the worker heartbeats to see if any of them need to be restarted.

WORKER_CHILDOPTS

public static java.lang.String WORKER_CHILDOPTS

The jvm opts provided to workers launched by this supervisor. All "%ID%" substrings are replaced with an identifier for this worker.

WORKER_HEARTBEAT_FREQUENCY_SECS

public static java.lang.String WORKER_HEARTBEAT_FREQUENCY_SECS

How often this worker should heartbeat to the supervisor.

TASK_HEARTBEAT_FREQUENCY_SECS

public static java.lang.String TASK_HEARTBEAT_FREQUENCY_SECS

How often a task should heartbeat its status to the master.

TASK_REFRESH_POLL_SECS

public static java.lang.String TASK_REFRESH_POLL_SECS

How often a task should sync its connections with other tasks (if a task is reassigned, the other tasks sending messages to it need to refresh their connections). In general though, when a reassignment happens other tasks will be notified almost immediately. This configuration is here just in case that notification doesn't come through.

TOPOLOGY_ENABLE_MESSAGE_TIMEOUTS

public static java.lang.String TOPOLOGY_ENABLE_MESSAGE_TIMEOUTS

True if Storm should timeout messages or not. Defaults to true. This is meant to be used in unit tests to prevent tuples from being accidentally timed out during the test.

TOPOLOGY_DEBUG

public static java.lang.String TOPOLOGY_DEBUG

When set to true, Storm will log every message that's emitted.

TOPOLOGY_OPTIMIZE

public static java.lang.String TOPOLOGY_OPTIMIZE

Whether or not the master should optimize topologies by running multiple tasks in a single thread where appropriate.

TOPOLOGY_WORKERS

public static java.lang.String TOPOLOGY_WORKERS

How many processes should be spawned around the cluster to execute this topology. Each process will execute some number of tasks as threads within them. This parameter should be used in conjunction with the parallelism hints on each component in the topology to tune the performance of a topology.

TOPOLOGY_TASKS

public static java.lang.String TOPOLOGY_TASKS

How many instances to create for a spout/bolt. A task runs on a thread with zero or more other tasks for the same spout/bolt. The number of tasks for a spout/bolt is always the same throughout the lifetime of a topology, but the number of executors (threads) for a spout/bolt can change over time. This allows a topology to scale to more or less resources without redeploying the topology or violating the constraints of Storm (such as a fields grouping guaranteeing that the same value goes to the same task).

TOPOLOGY_ACKER_EXECUTORS

public static java.lang.String TOPOLOGY_ACKER_EXECUTORS

How many executors to spawn for ackers.

If this is set to 0, then Storm will immediately ack tuples as soon as they come off the spout, effectively disabling reliability.

TOPOLOGY_MESSAGE_TIMEOUT_SECS

public static java.lang.String TOPOLOGY_MESSAGE_TIMEOUT_SECS

The maximum amount of time given to the topology to fully process a message emitted by a spout. If the message is not acked within this time frame, Storm will fail the message on the spout. Some spouts implementations will then replay the message at a later time.

TOPOLOGY_KRYO_REGISTER

public static java.lang.String TOPOLOGY_KRYO_REGISTER

A list of serialization registrations for Kryo ( http://code.google.com/p/kryo/ ), the underlying serialization framework for Storm. A serialization can either be the name of a class (in which case Kryo will automatically create a serializer for the class that saves all the object's fields), or an implementation of com.esotericsoftware.kryo.Serializer. See Kryo's documentation for more information about writing custom serializers.

TOPOLOGY_KRYO_DECORATORS

public static java.lang.String TOPOLOGY_KRYO_DECORATORS

A list of classes that customize storm's kryo instance during start-up. Each listed class name must implement IKryoDecorator. During start-up the listed class is instantiated with 0 arguments, then its 'decorate' method is called with storm's kryo instance as the only argument.

TOPOLOGY_SKIP_MISSING_KRYO_REGISTRATIONS

public static java.lang.String TOPOLOGY_SKIP_MISSING_KRYO_REGISTRATIONS

Whether or not Storm should skip the loading of kryo registrations for which it does not know the class or have the serializer implementation. Otherwise, the task will fail to load and will throw an error at runtime. The use case of this is if you want to declare your serializations on the storm.yaml files on the cluster rather than every single time you submit a topology. Different applications may use different serializations and so a single application may not have the code for the other serializers used by other apps. By setting this config to true, Storm will ignore that it doesn't have those other serializations rather than throw an error.

TOPOLOGY_MAX_TASK_PARALLELISM

public static java.lang.String TOPOLOGY_MAX_TASK_PARALLELISM

The maximum parallelism allowed for a component in this topology. This configuration is typically used in testing to limit the number of threads spawned in local mode.

TOPOLOGY_MAX_SPOUT_PENDING

public static java.lang.String TOPOLOGY_MAX_SPOUT_PENDING

The maximum number of tuples that can be pending on a spout task at any given time. This config applies to individual tasks, not to spouts or topologies as a whole. A pending tuple is one that has been emitted from a spout but has not been acked or failed yet. Note that this config parameter has no effect for unreliable spouts that don't tag their tuples with a message id.

TOPOLOGY_SPOUT_WAIT_STRATEGY

public static java.lang.String TOPOLOGY_SPOUT_WAIT_STRATEGY

A class that implements a strategy for what to do when a spout needs to wait. Waiting is triggered in one of two conditions: 1. nextTuple emits no tuples 2. The spout has hit maxSpoutPending and can't emit any more tuples

TOPOLOGY_SLEEP_SPOUT_WAIT_STRATEGY_TIME_MS

public static java.lang.String TOPOLOGY_SLEEP_SPOUT_WAIT_STRATEGY_TIME_MS

The amount of milliseconds the SleepEmptyEmitStrategy should sleep for.

TOPOLOGY_STATE_SYNCHRONIZATION_TIMEOUT_SECS

public static java.lang.String TOPOLOGY_STATE_SYNCHRONIZATION_TIMEOUT_SECS

The maximum amount of time a component gives a source of state to synchronize before it requests synchronization again.

TOPOLOGY_STATS_SAMPLE_RATE

public static java.lang.String TOPOLOGY_STATS_SAMPLE_RATE

The percentage of tuples to sample to produce stats for a task.

TOPOLOGY_FALL_BACK_ON_JAVA_SERIALIZATION

public static java.lang.String TOPOLOGY_FALL_BACK_ON_JAVA_SERIALIZATION

Whether or not to use Java serialization in a topology.

TOPOLOGY_WORKER_CHILDOPTS

public static java.lang.String TOPOLOGY_WORKER_CHILDOPTS

Topology-specific options for the worker child process. This is used in addition to WORKER_CHILDOPTS.

TOPOLOGY_TRANSACTIONAL_ID

public static java.lang.String TOPOLOGY_TRANSACTIONAL_ID

This config is available for TransactionalSpouts, and contains the id ( a String) for the transactional topology. This id is used to store the state of the transactional topology in Zookeeper.

TOPOLOGY_AUTO_TASK_HOOKS

public static java.lang.String TOPOLOGY_AUTO_TASK_HOOKS

A list of task hooks that are automatically added to every spout and bolt in the topology. An example of when you'd do this is to add a hook that integrates with your internal monitoring system. These hooks are instantiated using the zero-arg constructor.

TOPOLOGY_EXECUTOR_RECEIVE_BUFFER_SIZE

public static java.lang.String TOPOLOGY_EXECUTOR_RECEIVE_BUFFER_SIZE

The size of the Disruptor receive queue for each executor. Must be a power of 2.

TOPOLOGY_RECEIVER_BUFFER_SIZE

public static java.lang.String TOPOLOGY_RECEIVER_BUFFER_SIZE

The maximum number of messages to batch from the thread receiving off the network to the executor queues. Must be a power of 2.

TOPOLOGY_EXECUTOR_SEND_BUFFER_SIZE

public static java.lang.String TOPOLOGY_EXECUTOR_SEND_BUFFER_SIZE

The size of the Disruptor send queue for each executor. Must be a power of 2.

TOPOLOGY_TRANSFER_BUFFER_SIZE

public static java.lang.String TOPOLOGY_TRANSFER_BUFFER_SIZE

The size of the Disruptor transfer queue for each worker.

TOPOLOGY_TICK_TUPLE_FREQ_SECS

public static java.lang.String TOPOLOGY_TICK_TUPLE_FREQ_SECS

How often a tick tuple from the "__system" component and "__tick" stream should be sent to tasks. Meant to be used as a component-specific configuration.

TOPOLOGY_DISRUPTOR_WAIT_STRATEGY

public static java.lang.String TOPOLOGY_DISRUPTOR_WAIT_STRATEGY

Configure the wait strategy used for internal queuing. Can be used to tradeoff latency vs. throughput

TOPOLOGY_WORKER_SHARED_THREAD_POOL_SIZE

public static java.lang.String TOPOLOGY_WORKER_SHARED_THREAD_POOL_SIZE

The size of the shared thread pool for worker tasks to make use of. The thread pool can be accessed via the TopologyContext.

TOPOLOGY_NAME

public static java.lang.String TOPOLOGY_NAME

Name of the topology. This config is automatically set by Storm when the topology is submitted.

TRANSACTIONAL_ZOOKEEPER_ROOT

public static java.lang.String TRANSACTIONAL_ZOOKEEPER_ROOT

The root directory in ZooKeeper for metadata about TransactionalSpouts.

TRANSACTIONAL_ZOOKEEPER_SERVERS

public static java.lang.String TRANSACTIONAL_ZOOKEEPER_SERVERS

The list of zookeeper servers in which to keep the transactional state. If null (which is default), will use storm.zookeeper.servers

TRANSACTIONAL_ZOOKEEPER_PORT

public static java.lang.String TRANSACTIONAL_ZOOKEEPER_PORT

The port to use to connect to the transactional zookeeper servers. If null (which is default), will use storm.zookeeper.port

ZMQ_THREADS

public static java.lang.String ZMQ_THREADS

The number of threads that should be used by the zeromq context in each worker process.

ZMQ_LINGER_MILLIS

public static java.lang.String ZMQ_LINGER_MILLIS

How long a connection should retry sending messages to a target host when the connection is closed. This is an advanced configuration and can almost certainly be ignored.

JAVA_LIBRARY_PATH

public static java.lang.String JAVA_LIBRARY_PATH

This value is passed to spawned JVMs (e.g., Nimbus, Supervisor, and Workers) for the java.library.path value. java.library.path tells the JVM where to look for native libraries. It is necessary to set this config correctly since Storm uses the ZeroMQ and JZMQ native libs.

DEV_ZOOKEEPER_PATH

public static java.lang.String DEV_ZOOKEEPER_PATH

The path to use as the zookeeper dir when running a zookeeper server via "storm dev-zookeeper". This zookeeper instance is only intended for development; it is not a production grade zookeeper setup.

Constructor Detail

Config

public Config()

Method Detail

setDebug

public void setDebug(boolean isOn)

setOptimize

public void setOptimize(boolean isOn)

setNumWorkers

public void setNumWorkers(int workers)

setNumAckers

public void setNumAckers(int numExecutors)

setMessageTimeoutSecs

public void setMessageTimeoutSecs(int secs)

registerSerialization

public void registerSerialization(java.lang.Class klass)

registerSerialization

public void registerSerialization(java.lang.Class klass,
                                  java.lang.Class<? extends Serializer> serializerClass)

registerDecorator

public void registerDecorator(java.lang.Class<? extends IKryoDecorator> klass)

setSkipMissingKryoRegistrations

public void setSkipMissingKryoRegistrations(boolean skip)

setMaxTaskParallelism

public void setMaxTaskParallelism(int max)

setMaxSpoutPending

public void setMaxSpoutPending(int max)

setStatsSampleRate

public void setStatsSampleRate(double rate)

setFallBackOnJavaSerialization

public void setFallBackOnJavaSerialization(boolean fallback)