Data Engineer (50% Data Engg 50% data Scientist)

Location: Dallas, Texas(Hybrid)

Experience Required: 10+ Years

Responsibilities

• Work with business stakeholders, Business Systems Analysts and Developers to ensure quality delivery of software.
• Interact with key business functions to confirm data quality policies and governed attributes.
• Follow quality management best practices and processes to bring consistency and completeness to integration service testing
• Designing and managing the testing AWS environments of data workflows during development and deployment of data products
• Provide assistance to the team in Test Estimation & Test Planning
• Design, development of Reports and dashboards.
• Analyzing and evaluating data sources, data volume, and business rules.
• Proficiency with SQL, familiarity with Python, Scala, Athena, EMR, Redshift and AWS.
• No SQL data and unstructured data experience.
• Extensive experience in programming tools like Map Reduce to HIVEQL
• Experience in data science platforms like SageMaker/Machine Learning Studio/ H2O.
• Should be well versed with the Data flow and Test Strategy for Cloud/ On Prem ETL Testing.
• Interpret and analyses data from various source systems to support data integration and data reporting needs.
• Experience in testing Database Application to validate source to destination data movement and transformation.
• Work with team leads to prioritize business and information needs.
• Develop complex SQL scripts (Primarily Advanced SQL) for Cloud ETL and On prem.
• Develop and summarize Data Quality analysis and dashboards.
• Knowledge of Data modeling and Data warehousing concepts with emphasis on Cloud/ On Prem ETL.
• Execute testing of data analytic and data integration on time and within budget.
• Work with team leads to prioritize business and information needs
• Troubleshoot & determine best resolution for data issues and anomalies
• Experience in Functional Testing, Regression Testing, System Testing, Integration Testing & End to End testing.
• Has deep understanding of data architecture & data modeling best practices and guidelines for different data and analytic platforms

Requirements:

• Extensive Experience in Data migration is a must ( Teradata to Redshift preferred)
• Extensive testing Experience with SQL/Unix/Linux scripting is a must
• Extensive experience testing Cloud/On Prem ETL (e.g. Abinitio, Informatica, SSIS, Datastage, Alteryx, Glu)
• Extensive experience DBMS like Oracle, Teradata, SQL Server, DB2, Redshift, Postgres and Sybase.
• Extensive experience using Python scripting and AWS and Cloud Technologies.
• Extensive experience using Athena, EMR, Redshift, AWS, and Cloud Technologies
• Experienced in large-scale application development testing – Cloud/ On Prem Data warehouse, Data Lake, Data science
• Experience with multi-year, large-scale projects
• Expert technical skills with hands-on testing experience using SQL queries.
• Extensive experience with both data migration and data transformation testing
• Extensive experience DBMS like Oracle, Teradata, SQL Server, DB2, Redshift, Postgres and Sybase.
• Extensive testing Experience with SQL/Unix/Linux.
• Extensive experience testing Cloud/On Prem ETL (e.g. Abinitio, Informatica, SSIS, Datastage, Alteryx, Glu)
• Extensive experience using Python scripting and AWS and Cloud Technologies.
• Extensive experience using Athena, EMR , Redshift and AWS and Cloud Technologies
• API/Rest Assured automation, building reusable frameworks, and good technical expertise/acumen
• Java/Java Script - Implement core Java, Integration, Core Java and API.
• Functional/UI/ Selenium - BDD/Cucumber, Specflow, Data Validation/Kafka, BigData, also automation experience using Cypress.
• AWS/Cloud - Jenkins/ Gitlab/ EC2 machine, S3 and building Jenkins and CI/CD pipelines, SouceLabs.
• API/Rest API - Rest API and Micro Services using JSON, SoapUI
• Extensive experience in DevOps/Data Ops space.
• Strong experience in working with DevOps and build pipelines.
• Strong experience of AWS data services including Redshift, Glue, Kinesis, Kafka (MSK) and EMR/ Spark, Sage Maker etc…
• Experience with technologies like Kubeflow, EKS, Docker
• Extensive experience using No SQL data and unstructured data experience like MongoDB, Cassandra, Redis, ZooKeeper.
• Extensive experience in Map reduce using tools like Hadoop, Hive, Pig, Kafka, S4, Map R.
• Experience using Jenkins and Gitlab
• Experience using both Waterfall and Agile methodologies.
• Experience in testing storage tools like S3, HDFS
• Experience with one or more industry-standard defect or Test Case management Tools
• Great communication skills (regularly interacts with cross functional team members