[Remote] Senior Data Engineer
Note: The job is a remote job and is open to candidates in USA. Tredence Inc. is seeking a Senior Data Engineer to manage end-to-end delivery of data solutions. The role involves developing data pipeline architecture, optimizing data processes, and ensuring data quality while collaborating with cross-functional teams.
Responsibilities
- Manage end to end delivery by Investigating problem areas, working cross-functionally with product manager & other stakeholders
- Follow the Agile development methodology; think strategically and execute methodically
- Develop and manage capacity and growth projection forecasts of the environment within budgets
- Create and maintain optimal data pipeline architecture, assemble large, complex data sets that meet functional / non-functional business requirements
- Drive optimization, testing and tooling to improve quality of solutions
- Manage teams that build and operate high volume distributed systems in a SaaS environment
- Great at devising efficient processes that increase velocity and quality
- Understanding business use cases and deriving codes as per requirements
- Promote the data discovery, availability and quality capabilities and expertise to business unit leaders and educate them in leveraging these capabilities in achieving their business goals
- Accountable for operational effectiveness in performance, uptime, release management of the enterprise data platform
- Responsible for creating and validating end-to-end scalable data pipeline templates
- Responsible on the migrating the streaming data from the S3 to Databricks layers (raw, processed, consumption) and applying required data quality checks and transformations
- Handle slowly changing dimensions, change data capture, dimensions, facts and manage upsets
- Responsible for handling adhoc data analysis requests
- Responsible for handling client communication
- Responsible for the data accuracy, data governance, file handling, error handling, data validation
Skills
- Master's degree (or foreign equivalent) in Business Analytics, Information Systems, Data Science, or a related field
- One (1) year of experience in the job offered or a related position
- Demonstrable knowledge of Data Engineering Pipelines
- Demonstrable knowledge of Data Warehousing
- Demonstrable knowledge of Data Modeling
- Demonstrable knowledge of SQL
- Demonstrable knowledge of Cloud Technology (Google Cloud)
- Demonstrable knowledge of Data Wrangling
- Demonstrable knowledge of Data Analysis
- Ability to travel to unanticipated client locations throughout the U.S., approximately 30%, as required
Company Overview