Senior Analyst - Data Engineer

Main Purpose: ▪Collaborate with data scientists and business stakeholders to design, develop, and maintain efficient data pipelines feeding into the organization's data lake. ▪ Maintain the integrity and quality of the data lake, enabling accurate and actionable insights for data scientists and informed decision-making for business stakeholders. ▪Utilize extensive knowledge of data engineering and cloud technologies to enhance the organization's data infrastructure, promoting a culture of data-driven decision-making. ▪ Apply data engineering expertise to define and optimize data pipelines using advanced concepts to improve the efficiency and accessibility of data storage. ▪Own the development of an extensive data catalog, ensuring robust data governance and facilitating effective data access and utilization across the organization. Knowledge Skills and Abilities, Key Responsibilities: Key Responsibilities •Contribute to the development of scalable and performant data pipelines on Databricks, leveraging Delta Lake, Delta Live Tables (DLT), and other core Databricks components. •Develop data lakes/warehouses designed for optimized storage, querying, and real-time updates using Delta Lake. •Implement effective data ingestion strategies from various sources (streaming, batch, API-based), ensuring seamless integration with Databricks. •Ensure the integrity, security, quality, and governance of data across our Databricks-centric platforms. •Collaborate with stakeholders (data scientists, analysts, product teams) to translate business requirements into Databricks-native data solutions. •Build and maintain ETL/ELT processes, heavily utilizing Databricks, Spark (Scala or Python), SQL, and Delta Lake for transformations. Page •Experience with CI/CD and DevOps practices specifically tailored for the Databricks environment. •Monitor and optimize the cost-efficiency of data operations on Databricks, ensuring optimal resource utilization. •Utilize a range of Databricks tools, including the Databricks CLI and REST API, alongside Apache Spark™, to develop, manage, and optimize data engineering solutions. Key Relationships and Department Overview: Key Relationships •Internal – Data Engineering Manager •Developers across various departments, Managers of Departments in other regional hubs of Puma Energy •External – Platform providers Puma Energy is a leading global energy business which supplies, stores and distributes petroleum products in 48 countries across six continents. Founded in 1997, Puma Energy has its headquarters in Singapore and Geneva and employs over 8,000 people globally with regional hubs in South Africa, Puerto Rico, Australia and Estonia and have recently setup a Global Delivery Center in Mumbai, India. Puma Energy has a network of 100 storage terminals, more than 3,000 retail sites, and a presence at over 80 airports. Our mission is to energize communities to help drive growth and prosperity by sustainably serving our customers' needs in high potential countries around the world