We are glad to announce the release of our Automated Model-to-Code service for the NSPM Architecture.
Move to 100% CPU Efficiency in your datalake —Automatically
The industry’s first “YACC for Bigdata” is here. Datalake M2C is an automated engine that transforms your existing datalake based-on relational tables into high-performance, shuffle-free, NSPM based architecture datalake
1. The Power of Automated M2C Synthesis
Datalake M2C performs an automated Model-to-Code Transformation, translating your DBMS metadata into the Novel Spark Performance Model (NSPM) layout.
2. Why Datalake M2C?
Traditional SQL-based Datalake / Architectures are “Feasibility Only”—they work, but they don’t scale. Datalake M2C moves your project into Performance Mode:
- Zero-Shuffle Architecture: By synthesizing “NSPM Blocks,” we eliminate the need for network coordination. 0 GB Shuffle Write.
- 100% CPU Utilization: Stop paying for idle clusters. Our generated code ensures your CPUs are processing data, not waiting for the network.
- Strong Typing: We generate Scala Case Classes for your Wide and NSPM schemas, moving errors from runtime to compile-time.
60% Development Savings: We automate the architecture, the schemas, and the I/O boilerplate. You only write the functional business logic.
3. Scientific Proof: 14% vs. 100%
Based on the TPC-H Q100 benchmark (paper published in this website), the difference is undeniable:
| Feature | Legacy SQL/DBMS Style | Datalake M2C (NSPM) |
| Execution Mode | Feasibility (Sequential) | Performance (Bare Metal) |
| Shuffle Data | 114.1 GB | 0 GB |
| CPU Saturation | 14% | 100% |
| Code Structure | Typeless DataFrames | Strongly Typed Case Classes |
4. How to Use the Datalake M2C Service
- Contact us to get credentials.
- Upload: Provide your datalake table Schema description (DDL/Relations) to the NSPM Portal.
- Synthesize: Our engine generates a complete Scala/Spark project tailored to your data.
- Deploy: Run the generated migration-code to rebuild your Data Lake into NSPM Datalake.
- Scale: Enjoy unlimited horizontal scalability with 0% shuffle.
Everything you need to know about the underlying architecture is available in:
1- the book “Spark Performance”
2-TPC-H Benchmark article,and other articles
All are available on this website.
“Datalake M2C is not a shortcut; it is a Precision Instrument.”
This service is designed for Data Engineers who understand the limitations of the SQL paradigm in bigdata world and are ready to implement the NSPM Bare-Metal Performance standards.
While we automate the structural synthesis of your Data Lake, the integrity of the functional transformation remains in the hands of the engineering team.