Principal Data Engineer/Architect - Remote
About the position Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health equity on a global scale. Join us to start Caring. Connecting. Growing together. You’ll enjoy the flexibility to work remotely from anywhere within the U.S. as you take on some tough challenges. As a senior member of the Cloud Data Engineering team, the candidate should provide: The technical leadership for all the data engineering efforts on public cloud for the Optum Rx Data Estate scrum team. Will design and build scalable and cost-effective public cloud (Azure) native solutions as per the business use cases Should be hands-on engineer with experience in multiple data engineering technologies (Spark/Kafka/ADF/Databricks) Standard dev-ops technologies and processes (CI/CD- Containers, Jenkins etc.) As a tech lead, he should be responsible for the deliverables of the team working closely with the product owners and the delivery leads You’ll be rewarded and recognized for your performance in an environment that will challenge you and give you clear direction on what it takes to succeed in your role as well as provide development for other roles you may be interested in.
Responsibilities
- Provide the technical leadership for all the data engineering efforts on public cloud for the Optum Rx Data Estate scrum team.
- Design and build scalable and cost-effective public cloud (Azure) native solutions as per the business use cases
- Be a hands-on engineer with experience in multiple data engineering technologies (Spark/Kafka/ADF/Databricks)
- Utilize standard dev-ops technologies and processes (CI/CD- Containers, Jenkins etc.)
- As a tech lead, be responsible for the deliverables of the team working closely with the product owners and the delivery leads
Requirements
- Engineering Degree in computer science or equivalent experience
- 10+ years of Data Engineering experience
- 5+ years of Data engineering experience in Spark
- 3+ years on Cloud native data engineering technologies
- Experience using AI/ML
- Experience in Dev Ops and CI/CD – Jenkins, Terraform (Azure) and containers (Docker)
- Work experience on Proof of Concepts for AI/ML
- Hands-on experience on building data engineering pipelines on Spark/ Data bricks (Azure)
- Hands-on experience with stream data ingestion /processing with Spark
- Hands-on experience on other Azure native data technology stack – Azure Data factory, Synapse, Snowflake
- Exposure to providing ETL solutions for Azure native stack and in Spark
- Good exposure on Enterprise Data Lake (Delta Lake) and Data Warehouse concepts
Nice-to-haves
- Experience working with business partners for requirement gathering, scoping, estimation, and execution plan for various capabilities
- Experience working in an Agile environment with multiple Scrum and Kanban teams
Benefits
- comprehensive benefits package
- incentive and recognition programs
- equity stock purchase
- 401k contribution