weka.knowledgeflow.steps

## Class MakeResourceIntensive

- java.lang.Object
- - weka.knowledgeflow.steps.BaseStep
  - - weka.knowledgeflow.steps.MakeResourceIntensive

- All Implemented Interfaces:
    java.io.Serializable, BaseStepExtender, Step

      @KFStep(name="MakeResourceIntensive",
              category="Flow",
              toolTipText="Makes downstream connected steps resource intensive (or not). This shifts processing of such steps between the main step executor<br>service and the high resource executor service or vice versa.",
              iconPath="weka/gui/knowledgeflow/icons/DiamondPlain.gif")
      public class MakeResourceIntensive
      extends BaseStep

  A Step that makes downstream steps that are directly connected to this step resource intensive (or not). This overrides whatever the downstream step may (or may not) have declared in it's `KFStep` class annotation with regards to whether it is resource intensive (cpu or memory). The Knowledge Flow execution environment uses two executor services - a primary one to execute batch processing for steps; and a secondary one for executing `StepTask`s (which are assumed to be resource intensive by default) or for executing batch processing for a Step when it declares itself to be resource intensive. This secondary executor service uses a limited (typically `<= num cpu cores`) number of threads. Steps that involve potentially intensive (cpu/memory) processing should declare themselves resource intensive so that less taxing steps (and the UI) get cpu cycles. E.g. the Classifier Step is resource intensive so that processing cross-validation folds in parallel for a large data set or computationally intensive classifier does not blow out memory or bog the system down.
  Version:
    $Revision: $
  Author:
    Mark Hall (mhall{[at]}pentaho{[dot]}com)
  See Also:
    Serialized Form

- - ### Constructor Summary

    Constructors
    Constructor and Description
    `MakeResourceIntensive()`

  - ### Method Summary

    All Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `java.util.List<java.lang.String>` | `getIncomingConnectionTypes()` Get a list of incoming connection types that this step can accept.
    `boolean` | `getMakeResourceIntensive()` Get whether downstream steps are to be made resource intensive
    `java.util.List<java.lang.String>` | `getOutgoingConnectionTypes()` Get a list of outgoing connection types that this step can produce.
    `void` | `processIncoming(Data data)` Process incoming data
    `void` | `setMakeResourceIntensive(boolean resourceIntensive)` Set whether downstream steps are to be made resource intensive or not
    `void` | `stepInit()` Initialize the step

    - ### Methods inherited from class weka.knowledgeflow.steps.BaseStep

      `environmentSubstitute, getCustomEditorForStep, getDefaultSettings, getInteractiveViewers, getInteractiveViewersImpls, getName, getStepManager, globalInfo, isResourceIntensive, isStopRequested, outputStructureForConnectionType, setName, setStepIsResourceIntensive, setStepManager, setStepMustRunSingleThreaded, start, stepMustRunSingleThreaded, stop`

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Constructor Detail

    - #### MakeResourceIntensive

          public MakeResourceIntensive()

  - ### Method Detail

    - #### setMakeResourceIntensive

          @OptionMetadata(displayName="Make downstream step(s) high resource",
                          description="<html>Makes downstream connected steps resource intensive (or not)<br>This shifts processing of such steps between the main step executor service and the high resource executor service or vice versa.</html>")
          public void setMakeResourceIntensive(boolean resourceIntensive)

      Set whether downstream steps are to be made resource intensive or not
      Parameters:
        `resourceIntensive` - true if the downstream connected steps are to be made resource intensive

    - #### getMakeResourceIntensive

          public boolean getMakeResourceIntensive()

      Get whether downstream steps are to be made resource intensive
      Returns:
        true if downstream connected steps are to be made resource intensive

    - #### stepInit

          public void stepInit()
                        throws WekaException

      Initialize the step
      Throws:
        `WekaException` - if a problem occurs

    - #### getIncomingConnectionTypes

          public java.util.List<java.lang.String> getIncomingConnectionTypes()

      Get a list of incoming connection types that this step can accept. Ideally (and if appropriate), this should take into account the state of the step and any existing incoming connections. E.g. a step might be able to accept one (and only one) incoming batch data connection.
      Returns:
        a list of incoming connections that this step can accept given its current state

    - #### getOutgoingConnectionTypes

          public java.util.List<java.lang.String> getOutgoingConnectionTypes()

      Get a list of outgoing connection types that this step can produce. Ideally (and if appropriate), this should take into account the state of the step and the incoming connections. E.g. depending on what incoming connection is present, a step might be able to produce a trainingSet output, a testSet output or neither, but not both.
      Returns:
        a list of outgoing connections that this step can produce

    - #### processIncoming

          public void processIncoming(Data data)
                               throws WekaException

      Process incoming data
      Specified by:
        `processIncoming` in interface `BaseStepExtender`
      Specified by:
        `processIncoming` in interface `Step`
      Overrides:
        `processIncoming` in class `BaseStep`
      Parameters:
        `data` - the data to process
      Throws:
        `WekaException`