Software Engineer, Data Platform

What you’ll do

You will be part of the team that builds a multitenant data platform to onboard internal and external data sources and perform a central role in data ingestion, consumption, and cataloguing. You will join a team of experts in data pipelines focusing on data streaming and acquisition, storage of the data in multiple forms (raw, normalized, operational vs historical), ELT, transformation, distribution, access layers for different types of technical access and entitlements. Building and enhancing our data platform includes applying data lake components with raw data, joined operational data, a warehouse for history and API, and an access layer, across on prem and cloud technologies and clients. In this role, you will be responsible for leading and providing best practices for the team managing the Data Platform on AWS for our Firm. You will work closely with the architect/business for gathering requirements and delivering. As a Lead Python Engineer, you will be:

• Working together with Engineering Managers and the rest of Leadership team in Poland and US

• Planning projects and successfully delivering them with the team

• Building data ingestion pipelines in the cloud and assisting in extending our AWS cloud offering

• Working extensively in Databricks platform and leveraging all of the Databricks features (pipelines, Unity Catalogue, streaming, etc)

• Improving excellence of the team, processes, and practices within the company

• Working with customers towards a project’s success