Data Engineer at Nielsen in Emeryville, CA
What You Will Be Doing:
Design, develop, and maintain end to end data solutions using open source, modern data lake technologies (Spark, Cassandra, Parquet, AWS Services) Create and maintain optimal data pipeline architecture Proactively identify performance & data quality problems and drive the team to remediate them. Advocate architectural and code improvements to the team to improve execution speed and reliability. Develop best practices around observability, monitoring, logging and data processing Identify, design, and implement internal process improvements:
automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using Spark and AWS 'big data' technologies. Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs. Work with data and analytics experts to strive for greater functionality in our data systems. Required skills:
3
years of experience in developing data driven software applications 3
years experience in any of these:
Java, Scala, Python 2
years development with Apache Spark, Parquet data format, and other big data technologies 2
years using relational database concepts with a working knowledge of SQL, data modeling best principles, and data storage optimizations 1
years of experience in using AWS services especially serverless applications would be a great plus Experience with development best practices, including query optimization, version control, code reviews, and documentation Golang working experience is a plus Working knowledge of message queues, stream processing, and highly scalable data stores Experience supporting and working with cross-functional teams in a dynamic environment Experience analyzing data to identify deliverables, gaps and inconsistencies Experience with schema design and dimensional data modeling
Salary Range:
$80K -- $100K
Minimum Qualification
Data Science & Machine Learning, Database AdministrationEstimated Salary: $20 to $28 per hour based on qualifications.
Design, develop, and maintain end to end data solutions using open source, modern data lake technologies (Spark, Cassandra, Parquet, AWS Services) Create and maintain optimal data pipeline architecture Proactively identify performance & data quality problems and drive the team to remediate them. Advocate architectural and code improvements to the team to improve execution speed and reliability. Develop best practices around observability, monitoring, logging and data processing Identify, design, and implement internal process improvements:
automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using Spark and AWS 'big data' technologies. Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs. Work with data and analytics experts to strive for greater functionality in our data systems. Required skills:
3
years of experience in developing data driven software applications 3
years experience in any of these:
Java, Scala, Python 2
years development with Apache Spark, Parquet data format, and other big data technologies 2
years using relational database concepts with a working knowledge of SQL, data modeling best principles, and data storage optimizations 1
years of experience in using AWS services especially serverless applications would be a great plus Experience with development best practices, including query optimization, version control, code reviews, and documentation Golang working experience is a plus Working knowledge of message queues, stream processing, and highly scalable data stores Experience supporting and working with cross-functional teams in a dynamic environment Experience analyzing data to identify deliverables, gaps and inconsistencies Experience with schema design and dimensional data modeling
Salary Range:
$80K -- $100K
Minimum Qualification
Data Science & Machine Learning, Database AdministrationEstimated Salary: $20 to $28 per hour based on qualifications.
|
|
 | |
