Spark sql library has limited horizontal scalability based on the scientific proof summerized in this paper. And as speedup is the best scalability measure, we conclude that we can not expect a good speedup on any scala spark application that uses a spark sql component. Please download the full paper in PDF using the following … Continue reading SQL Scalability