Void function ( JavaRDD < String > arg0 ) { if ( arg0 != null ) { LOG . info ( "srini_string" ) ; arg0 . foreachPartition ( new SavePartitionWithVTI ( spliceJdbcUrl ) ) ; } return null ; }
a rdd is created on the driver for the blocks created during the batchinterval . the blocks generated during the batchinterval are partitions of the rdd . each partition is a task in spark . this takes the incoming rdd 's partitions and for each partition it calls savepartition . the savepartition processes is distributed to the non-driver executors to perform their operations and result in parallelization . to understand this in more detail , read the section ` level of parallelism in data receiving ' in the spark streaming programming guide concode_field_sep String spliceJdbcUrl concode_elem_sep Logger LOG concode_field_sep void writeExternal concode_elem_sep void readExternal