Title: Senior Data Engineer
KA, IN
Job Description
Must Have
You will be responsible for: (Job description)
Lead a team of data analyst/data engineers to design and develop efficient data pipelines to ensure the smooth flow of data from various sources into the data warehouse
Gather requirements and work with stakeholders to analyze business needs around new solutions and optimization
Design, and implement robust data models to support the organization’s evolving business needs while ensuring data integrity and performance
Continuously optimize data pipelines and processes to improve performance, scalability ad reliability, leveraging best practices and emerging technologies
Extracting and combining data from various heterogeneous data sources
Building and migrating the complex ETL pipelines from sources, to S3, Redshift and/or Elastic Map Reduce to make the system grow elastically
Build an understanding of full project life cycle: analysis, design, development, test, release and support procedures
Manipulate, process and extract value from large datasets
Implement and enforce data governance policies and procedures to ensure data quality, security, and compliances with regulatory requirements, while also maintaining lineage and documentation
Provides quality customer service, including interacting with customers, addressing customer queries, manage deliverables and effectively handling client requirements
Understanding of pharma commercial is required, specifically worked in the areas of brand, customer, multi-channel, omnichannel and content performance
Create realistic and actionable metrics for marketing, brand and digital ops managers, including collation and delivery of brand, sales and marketing activity KPI reports
Your impact: Candidate should be able to deliver cross functional projects with highest quality, mentor team to create next layer of leadership.
About you: (Desired profile)
We are seeking a dynamic and experienced Data Engineer to lead our talented team of data engineers/data analyst. In this role, you will be instrumental in shaping the architecture and infrastructure of our data systems, driving innovation, and ensuring the delivery of high-quality solutions. You will play a critical role in designing and implementing scalable data pipelines, optimizing data workflows, and leveraging advanced analytics techniques to drive business value. Pharma background preferred. Team management preferred.
Must have: (Requirements)
Experience with Life Sciences or Pharma data is a significant plus
Demonstrated ability in data modeling, ETL development, and data warehousing
Demonstrated experience manipulating, processing, and extracting value from large datasets
Excellent problem solving, analytical, technical, interpersonal and communication skills and an efficient team player with an ability to take new roles
6+ years of hands-on experience with AWS Services such as AWS S3, Glue ETL, Glue Catalog, Athena, EMR with PySpark, Redshift and Redshift Spectrum, etc.
5+ years of Hands-on experience on components of Hadoop Ecosystem like HDFS, Hive, Spark, Sqoop, Map Reduce and YARN
Hands-on experience with ETL tools such as Talend and Amazon Glue
Good working experience with different databases like MS SQL Server/Oracle/MySQL.
Working Experience with AWS Data Warehousing and database platforms (RedShift, Athena, Aurora).
Hands-on experience in Python Function Programming language
Deep SQL coding experience
Good to have
EQUAL OPPORTUNITY