Title: Senior Data Engineer
KA, IN
Job Description
Must Have
You will be responsible for: (Job description)
- Lead a team of data analyst/data engineers to design and develop efficient data pipelines to ensure the smooth flow of data from various sources into the data warehouse
- Gather requirements and work with stakeholders to analyze business needs around new solutions and optimization
- Design, and implement robust data models to support the organization’s evolving business needs while ensuring data integrity and performance
- Continuously optimize data pipelines and processes to improve performance, scalability ad reliability, leveraging best practices and emerging technologies
- Extracting and combining data from various heterogeneous data sources
- Building and migrating the complex ETL pipelines from sources, to S3, Redshift and/or Elastic Map Reduce to make the system grow elastically
- Build an understanding of full project life cycle: analysis, design, development, test, release and support procedures
- Manipulate, process and extract value from large datasets
- Implement and enforce data governance policies and procedures to ensure data quality, security, and compliances with regulatory requirements, while also maintaining lineage and documentation
- Provides quality customer service, including interacting with customers, addressing customer queries, manage deliverables and effectively handling client requirements
- Understanding of pharma commercial is required, specifically worked in the areas of brand, customer, multi-channel, omnichannel and content performance
- Create realistic and actionable metrics for marketing, brand and digital ops managers, including collation and delivery of brand, sales and marketing activity KPI reports
Your impact: Candidate should be able to deliver cross functional projects with highest quality, mentor team to create next layer of leadership.
About you: (Desired profile)
We are seeking a dynamic and experienced Data Engineer to lead our talented team of data engineers/data analyst. In this role, you will be instrumental in shaping the architecture and infrastructure of our data systems, driving innovation, and ensuring the delivery of high-quality solutions. You will play a critical role in designing and implementing scalable data pipelines, optimizing data workflows, and leveraging advanced analytics techniques to drive business value. Pharma background preferred. Team management preferred.
Must have: (Requirements)
- Experience with Life Sciences or Pharma data is a significant plus
- Demonstrated ability in data modeling, ETL development, and data warehousing
- Demonstrated experience manipulating, processing, and extracting value from large datasets
- Excellent problem solving, analytical, technical, interpersonal and communication skills and an efficient team player with an ability to take new roles
- 6+ years of hands-on experience with AWS Services such as AWS S3, Glue ETL, Glue Catalog, Athena, EMR with PySpark, Redshift and Redshift Spectrum, etc.
- 5+ years of Hands-on experience on components of Hadoop Ecosystem like HDFS, Hive, Spark, Sqoop, Map Reduce and YARN
- Hands-on experience with ETL tools such as Talend and Amazon Glue
- Good working experience with different databases like MS SQL Server/Oracle/MySQL.
- Working Experience with AWS Data Warehousing and database platforms (RedShift, Athena, Aurora).
- Hands-on experience in Python Function Programming language
- Deep SQL coding experience
Good to have
EQUAL OPPORTUNITY