Automate Data Integration and Reporting for a Mid-Sized Manufacturing Company

This project focuses on automating the reporting infrastructure of a mid-sized manufacturing company using modern data engineering practices. The company’s existing manual reporting processes, based on Excel spreadsheets, were error-prone, time-intensive, and unable to meet the growing demands for timely, accurate, and scalable data analysis. To address these challenges, the project implemented an automated data pipeline using Prefect for orchestration and Snowflake for data warehousing, designed around Kimball’s star schema methodology. 

The pipeline extracts raw data from the proALPHA ERP system, transforms it to address multilingual and multicurrency complexities, and loads it into Snowflake for efficient storage and querying. The system integrates Power BI to provide dynamic, interactive dashboards that replace static reports, enabling stakeholders to access near-real-time insights into key metrics such as sales, orders, and profitability. 

The overall objective was to streamline the company’s reporting process by creating a scalable, reliable, and efficient data infrastructure. This infrastructure eliminates manual inefficiencies, improves data quality, and supports consistent corporate reporting. By automating the data pipeline and implementing a reusable architecture, the project positions the company to adapt to future growth, including acquisitions and new datasets. The initiative not only meets the company’s immediate reporting needs but also establishes a foundation for long-term strategic decision-making, ensuring alignment with corporate goals and fostering a culture of data-driven operations.