Interface SupportsFileLocalization

All Superinterfaces:
AutoCloseable, Closeable, Executor, Operator, OperatorPipelineV3, Serializable, org.apache.spark.sql.api.java.UDF1<Iterator<org.apache.spark.sql.Row>,Integer>

@DeveloperApi @FeatureBeforeCall public interface SupportsFileLocalization extends Executor, org.apache.spark.sql.api.java.UDF1<Iterator<org.apache.spark.sql.Row>,Integer>
A mix-in interface for Executor. Executors can implement this interface to support writing input dataframe to local file system for shell script execution. Executors implementing this mix-in interface is a user-defined function (UDF) that takes a DataFrame partition as an input parameter and returns the localization result as an Integer, which 0 means successful. This feature is invoked before applying Executor's call function.
  • Method Summary

    Modifier and Type
    Method
    Description
    Sets the local destination path which the datasets will be written to.

    Methods inherited from interface java.io.Closeable

    close

    Methods inherited from interface com.atgenomix.seqslab.piper.plugin.api.executor.Executor

    init

    Methods inherited from interface com.atgenomix.seqslab.piper.plugin.api.Operator

    getName, getOperatorContext

    Methods inherited from interface org.apache.spark.sql.api.java.UDF1

    call
  • Method Details

    • setLocalPath

      String setLocalPath(String path)
      Sets the local destination path which the datasets will be written to.
      Parameters:
      path - File or directory path
      Returns:
      The same or updated local path