Job Description
Must Have
Role: Senior Data Engineer
Job Description:
We are looking for a Senior Data Engineer to design, build, and optimize our next-generation data architecture. In this role, you will be responsible for the end-to-end orchestration of data, from complex API and batch ingestions to the development of real-time streaming pipelines. You will serve as a technical leader, ensuring that our data warehouse, whether in Snowflake, BigQuery, or Redshift, is modeled for high performance and analytics enablement
Roles & Responsibilities
1. Data Architecture & Pipeline Engineering
End-to-End Orchestration: Design, develop, and maintain scalable ETL/ELT pipelines to ingest data from diverse sources, including complex APIs, microservices, and batch systems.
Stream Processing: Build and optimize real-time streaming data pipelines to support immediate business insights and low-latency analytics.
Infrastructure Optimization: Leverage cloud data warehousing (Snowflake, BigQuery, or Redshift) to build a high-performance "next-generation" data architecture.
2. Data Modeling & Warehouse Management
Schema Design: Architect and implement robust data models using Star and Snowflake schemas to ensure high performance and ease of use for analytics teams.
Performance Tuning: Monitor and optimize query performance, indexing, and partitioning strategies within the cloud data warehouse to manage costs and speed.
Warehouse Administration: Act as the subject matter expert for the data warehouse environment, ensuring appropriate storage and compute scaling.
3. Technical Leadership & Quality Assurance
Standardization: Establish and enforce best practices for data engineering, including version control, documentation, and coding standards in Python and SQL.
Data Integrity: Implement rigorous unit and integration testing for all data assets to ensure accuracy, reliability, and "production-grade" quality.
CI/CD Integration: Work closely with DevOps teams to integrate data pipelines into automated deployment cycles for seamless delivery.
4. Security, Privacy & Compliance
Data Governance: Design and implement data privacy and anonymization protocols to ensure sensitive information is handled according to enterprise security standards.
Access Management: Collaborate with security teams to enforce role-based access to data assets, ensuring audit readiness and compliance with global data regulations.
5. Cross-Functional Collaboration
Stakeholder Alignment: Partner with Data Scientists, Analysts, and Product Managers to translate complex business requirements into technical data specifications.
Analytics Enablement: Ensure data is structured and accessible for advanced analytics, automated reporting, and data-driven decision-making across the organization.
Key Skills: ETL/ELT Pipeline Development, Advanced SQL & Python, Cloud Data Warehousing (Snowflake/BigQuery/Redshift), Data Modeling (Star/Snowflake Schema), Stream Processing, API Integration, and Data Privacy/Anonymization.
Qualification: Bachelor’s or Master’s degree in Computer Science, Information Technology, Data Engineering, or a related quantitative field. Certifications in Cloud Platforms (Azure/GCP/AWS) or Data Tools (Snowflake/dbt) are highly desirable
Experience: 4+ years of experience in data engineering and architecture, with a proven track record of building scalable production-grade data pipelines.