Sr. Software Engineer

Employer: CDK Global LLC Job Title: Sr. Software Engineer Worksite: 11809 Domain Drive, Suite 200 Austin TX 78758. #LI-DNI Job Description: Design and develop EMR pipelines by using AWS services like SQS QUEUE, EC2 instances, AWS data pipeline, S3 buckets, AWS glue, RDS and others. Create extract-transform-load (ETL) EMR pipelines based on HADOOP, hive, Yarn resource manager, NIFI, spark and python frameworks in AWS. Interpret the data mapping document to identify the source systems like SQL server and develop required spark transformations for ingesting the ETL data into titan platform. Optimize spark jobs using Pyspark after complete analysis of multiple parameters and opportunities to improve the target systems along with data quality checks. Create spark data frames/RDD's and load the data in different formats JSON, Parquet, AVRO, CSV and others. Evaluate new architectures and technologies such as snowflake, Debezium and other tools to improve performance and efficiency of ETL tasks. Responsible for completing the data requests from CDK product customers and help them debug and resolve any data quality issues in a timely manner. Work closely with the CDK customers and provide the feedback to the CDK service team to improve reliability of our products. Work in the scaled agile methodologies to increase the quality of the deliverables. Monitor and resolve production 11/12 issues. 100% Telecommuting. Requirements: Bachelor’s degree or foreign equivalent in Computer Science, Information Technology, Computer Engineering or a related field plus 5 years of professional experience as a Software Developer or related occupation. Alternatively, a Master’s degree or foreign equivalent in Computer Science, Information Technology, Computer Engineering or a related field plus 3 years of professional experience as a Software Developer or related occupation. Additionally, the applicant must have employment experience with: 1) Designing, developing, and migrating data pipelines to latest data frameworks such as Databricks; 2) Creating Extract-Transform-Load (ETL) pipelines based on Hadoop, Hive, Nifi, Spark and Python frameworks in cloud AWS/Snowflake; 3) Map data between source systems and data lake and develop required data transformations for ingesting the source data into data lake (TITAN Platform); 4) Creating spark data frames/RDD's and load the data in different formats such as Json, Parquet, Avro, and CSV; 5) Evaluating new architectures and technologies such as Snowflake, Debezium, Kafka and other tools to improve performance and efficiency of ETL tasks. To Apply: Applicants who are interested in this position visit https://cdk.wd1.myworkdayjobs.com/en-US/CDK or email resume to [email protected] Reference Req# JR9104. #recruit #LI-DNI CDK Global is committed to fair and equitable compensation practices. Compensation packages are based on several factors, including but not limited to skills, experience, certifications, and work location. The total compensation package for this position may also include annual performance bonus, benefits and/or other applicable incentive compensation plans.We offer Medical, dental, and vision benefits in addition to: Paid Time Off (PTO) 401K Matching Program Tuition Reimbursement At CDK, we believe inclusion and diversity are essential in inspiring meaningful connections to our people, customers and communities. We are open, curious and encourage different views, so that everyone can be their best selves and make an impact. CDK is an Equal Opportunity Employer committed to creating an inclusive workforce where everyone is valued. Qualified applicants will receive consideration for employment without regard to race, color, creed, ancestry, national origin, gender, sexual orientation, gender identity, gender expression, marital status, creed or religion, age, disability (including pregnancy), results of genetic testing, service in the military, veteran status or any other category protected by law. Applicants must be legally authorized to work in the United States without the need for current or future employment-based visa sponsorship. The employee must be able to perform the essential functions of the position satisfactorily and, if requested, reasonable accommodations will be made to enable employees with disabilities to perform the essential functions of their job, absent undue hardship. CDK retains the right to change or assign other duties to this position as needed.