Andrew Ray

With a background in mathematics and graph theory, Andrew is always looking for simple and elegant solutions to complex problems. He enjoys working at the intersection of engineering and data science.

Recent Posts

Structured Streaming in Spark

This post gives you a quick overview of the new structured streaming feature in Spark 2.0, illustrating why it’s an exciting addition.

Reshaping Data with Pivot in Spark

Andrew gives you a deep dive into pivoting data with SparkSQL. This piece was originally posted on the Databricks blog.

Data Day and Graph Day Texas Slides

Check out the slides from our recent presentations at Data Day TX and Graph Day.

Pivoting Data in SparkSQL

Andrew Ray, Senior Data Engineer, contributed to the most recent release of Spark. This post gives examples of how to use his pivot commit in PySpark.



  • Best Practices for Spark in Production

    Principal Engineers Richard Williamson and Andrew Ray will be on Pepperdata’s webinar panel of industry experts, talking about Spark trends and use cases. Sign up here to attend the webinar, or get the recording.

Past Events


  • Graph Day

    Austin, TX

    Join us as Senior Data Engineer Andrew Ray presents a talk called “Intro to Pregel and PowerGraph.”

  • Spark Summit East

    New York, NY

    We’re excited to have Senior Data Engineer Andrew Ray speaking about pivoting data with SparkSQL. Andrew contributed the Pivot Commit to Spark 1.6.0.

  • StampedeCon 2016

    St. Louis, MO

    Senior Data Engineer and Spark contributor Andrew Ray will be at the conference, giving a talk on structured streaming and datasets in Spark 2.0.

Sign up for our newsletter