Sr. Software Engineer, Data Engineering

Responsibilities

• Build highly scalable, available, fault-tolerant data processing systems using AWS technologies, MapReduce, Hive, Kafka, Spark, Flink and other big data technologies to handle batch and real-time data processing over 10s of terabytes of data ingested every day and a petabyte-sized data warehouse. - Responsible for taking care of some of the most critical data pipelines at Poshmark. - Participate in architecture discussions, influence product roadmap, build best practices, take ownership and responsibility over new projects. - Create WBS (Work breakdown structure) for projects and execute them, taking full ownership, with minimal guidance. ## Desired Skills & Experience

• 3+ years of relevant software engineering experience. - 2+ year of Data Engineering experience. - Excellent technical problem solving using data structures and algorithms, with emphasis on optimization and code quality. - Expertise in architecting and building large-scale data processing systems using Big Data technologies like Hadoop, Hbase, Spark, Kafka, Druid, Flink, DataLake, Redshift etc. - Eagerness to try out newer technologies and adopt them as and when needed. - Object Oriented Patterns, Scala / Java / Python / C++ / SQL / Apache Spark, Flink, Hadoop, AWS S3, Redshift / Postgresql, Kinesis, EMR, Apache Airflow, Jenkins.