python/ray/data/read_api.py
def from_spark(
    df: "pyspark.sql.DataFrame", *, parallelism: Optional[int] = None
) -> Dataset[ArrowRow]:
    """Create a dataset from a Spark dataframe.

    Args:
        spark: A SparkSession, which must be created by RayDP (Spark-on-Ray).
        df: A Spark dataframe, which must be created by RayDP (Spark-on-Ray).
            parallelism: The amount of parallelism to use for the dataset.
            If not provided, it will be equal to the number of partitions of
            the original Spark dataframe.

    Returns:
        Dataset holding Arrow records read from the dataframe.
    """
    import raydp

    return raydp.spark.spark_dataframe_to_ray_dataset(df, parallelism)
Create a dataset from a Spark dataframe.

    Args:
        spark: A SparkSession, which must be created by RayDP (Spark-on-Ray).
        df: A Spark dataframe, which must be created by RayDP (Spark-on-Ray).
            parallelism: The amount of parallelism to use for the dataset.
            If not provided, it will be equal to the number of partitions of
            the original Spark dataframe.

    Returns:
        Dataset holding Arrow records read from the dataframe.
    
Create a dataset from a Spark dataframe.
Create a dataset from a Spark dataframe .
def from_spark(
    df: "pyspark.sql.DataFrame", *, parallelism: Optional[int] = None
) -> Dataset[ArrowRow]:
    import raydp
    return raydp.spark.spark_dataframe_to_ray_dataset(df, parallelism)
def from_spark ( df : "pyspark.sql.DataFrame" , * , parallelism : Optional [ int ] = None ) -> Dataset [ ArrowRow ] : import raydp return raydp . spark . spark_dataframe_to_ray_dataset ( df , parallelism )
Create a dataset from a Spark dataframe.
Create a dataset from a Spark dataframe .
"""Create a dataset from a Spark dataframe.

    Args:
        spark: A SparkSession, which must be created by RayDP (Spark-on-Ray).
        df: A Spark dataframe, which must be created by RayDP (Spark-on-Ray).
            parallelism: The amount of parallelism to use for the dataset.
            If not provided, it will be equal to the number of partitions of
            the original Spark dataframe.

    Returns:
        Dataset holding Arrow records read from the dataframe.
    """
Dataset holding Arrow records read from the dataframe.
Dataset holding Arrow records read from the dataframe .
df
"pyspark.sql.DataFrame"
A Spark dataframe, which must be created by RayDP (Spark-on-Ray).
parallelism: The amount of parallelism to use for the dataset.
If not provided, it will be equal to the number of partitions of
the original Spark dataframe.
A Spark dataframe which must be created by RayDP ( Spark - on - Ray ) . parallelism : The amount of parallelism to use for the dataset . If not provided it will be equal to the number of partitions of the original Spark dataframe .
parallelism
Optional[int]
spark
A SparkSession, which must be created by RayDP (Spark-on-Ray).
A SparkSession which must be created by RayDP ( Spark - on - Ray ) .