Title: Databricks Architect - Lead
NJ, US
Job Description
We are a technology-led healthcare solutions provider. We are driven by our purpose to enable healthcare organizations to be future-ready. We offer accelerated, global growth opportunities for talent that is bold, industrious and nimble. With Indegene, you gain a unique career experience that celebrates entrepreneurship and is guided by passion, innovation, collaboration and empathy. To explore exciting opportunities at the convergence of healthcare and technology, check out www.careers.indegene.com
What if we told you that you can move to an exciting role in an entrepreneurial organization without the usual risks associated with it?
We understand that you are looking for growth and variety in your career at this point and we would love you to join us in our journey and grow with us. At Indegene, our roles come with the excitement you require at this stage of your career with the reliability you seek. We hire the best and trust them from day 1 to deliver global impact, handle teams and be responsible for the outcomes while our leaders support and mentor you.
We are a profitable rapidly growing global organization and are scouting for the best talent for this phase of growth. With us, you are at the intersection of two of the most exciting industries of healthcare and technology. We offer global opportunities with fast-track careers while working with a team that is fueled by purpose. The combination of these will lead to a truly differentiated experience for you.
If this excites you, then apply below.
Role: Databricks Architect - Lead
Description:
• Architecture Design: Developing and maintaining data architectures using Databricks, including data lakes, data warehouses, and real-time processing systems. This involves creating blueprints for organizing data processes and flows, ensuring alignment with business objectives, as noted in "Data Architecture Insight: Enhance Data Usability"
• Cluster Management: Configuring, managing, and optimizing Databricks clusters to ensure high performance and cost efficiency. This includes understanding clusters, autoscaling, cluster policies, and optimizing performance, as detailed in the Medium article.
• Pipeline Development: Designing and implementing ETL/ELT pipelines using Databricks and Apache Spark to handle large-scale data processing. This involves creating efficient and scalable pipelines, a key aspect of data engineering expertise.
• Data Governance: Implementing and enforcing data governance policies, security measures, and compliance standards within the Databricks environment. This includes managing IAM roles, VPCs, encryption, and using tools like Unity Catalog for governance, as seen in cloud architecture discussions.
• Collaboration: Working closely with data scientists, analysts, and business stakeholders to understand data needs and deliver solutions that drive business value. This requires strong communication skills to convey technical concepts to diverse audiences.
• Integration: Integrating Databricks with other cloud services and tools, such as AWS S3, Azure Data Lake Storage, or Google Cloud Storage, to create a cohesive data ecosystem. This ensures seamless data flow and interoperability with existing systems.
• Innovation: Staying abreast of the latest developments in Databricks and related technologies, and applying best practices to enhance data capabilities. This includes keeping up with updates like Delta Lake, Databricks SQL, and new features in cloud integrations.
EQUAL OPPORTUNITY