As a Data Engineer with Py Spark, your role will involve the following responsibilities:
Utilizing Python, Py Spark, SQL, No-SQL, and DBMS for data processing tasks
Implementing Git for source code versioning and CI/CD
Conducting Exploratory Data Analysis (EDA) and applying Imputation Techniques
Performing Data Linking and Cleansing operations
Engaging in Feature Engineering for data enhancement
Setting up Apache Airflow/Jenkins for scheduling and automation
Leveraging Github and Github Actions for code management
Demonstrating proficiency in writing unit tests using Python
Prior experience in tuning and deploying data pipelines to production
Optimizing Spark jobs for performance enhancement
To qualify for this role, you should have:
Over 10 years of experience in Python, Py Spark, Jupyter, SQL, No-SQL, DBMS, and Git
Previous production experience in deploying data pipelines
Familiarity with Spark job optimization and tuning techniques
If you are interested in this opportunity, please apply. Thank you for considering this role. As a Data Engineer with Py Spark, your role will involve the following responsibilities:
Utilizing Python, Py Spark, SQL, No-SQL, and DBMS for data processing tasks
Implementing Git for source code versioning and CI/CD
Conducting Exploratory Data Analysis (EDA) and applying Imputation Techniques
Engaging in Feature Engineering for data enhancement
Setting up Apache Airflow/Jenkins for scheduling and automation
Leveraging Github and Github Actions for code management
Demonstrating proficiency in writing unit tests using Python
Prior experience in tuning and deploying data pipelines to production
Optimizing Spark jobs for performance enhancement
To qualify for this role, you should have:
Over 10 years of experience in Python, Py Spark, Jupyter, SQL, No-SQL, DBMS, and Git
Previous production experience in deploying data pipelines
Familiarity with Spark job optimization and tuning techniques
If you are interested in this opportunity, please apply. Thank you for considering this role.