In theory, the advantages of a modern Data Lake over a legacy data warehouse are quite obvious. In practice, they are not. Many businesses’ data warehouses have been proven to work with their existing analytics workflows. The promise of improving these workflows with a new strategic approach is met with strong skepticism and detailed questions.
Fortunately, we have answers. We have years of cutting-edge Data Lake experience that includes global scale data solutions based on Microsoft’s Azure cloud platform.
In this blog series, we have compiled our answers to some of the most common questions we have encountered along the way. We hope you will find them useful, as you explore your own Data Lake opportunities.
“Should a Data Lake take data directly from SAP ECC or be precalculated/pre-aggregated from SAP BW?”
Yes, pre-aggregated data can go into the Data Lake, but data lineage may be difficult to trace.
Generally, Data Lakes work best with raw data. But if translating business logics and moving it to the cloud is difficult or even impossible — e.g. in case the team that implemented it does not exist anymore — then pre-aggregated data can also be treated as raw data.
Data Lakes, as a flexible architecture choice, allow for both scenarios. A valid approach is to also perform incremental migration to different technology stacks and to perform a gradual phase-out of the SAP BW solution. Depending on factors such as the level of investment and business complexity already implemented in the SAP environment, the location of the SAP stack vs your desired BI platform (data latency, performance targets), infrastructure, license and staffing costs; different Data Lake sourcing strategies can be chosen. In case a decision is made to preserve the SAP BW stack as part of the target architecture, Date Lakes can also use such data sources.
Depending on the requirements, it is an option to perform integration via BI tools with data blending capabilities or use integration patterns involving tools such as SDA, BODS or even JDBC.