Role: Senior Data Engineer

Years of Experience: 5-10

Location: Ahmedabad

Overview:

We are seeking a highly skilled and experienced Senior Software Engineer to join our Data Engineering team. The ideal candidate will have 5 to 10 years of experience in software engineering, with a strong focus on data engineering practices and an awareness of AI/ML and Generative AI technologies. This role demands proficiency in data processing and analysis, with a keen understanding of both the technical and business aspects of data engineering.

Key Responsibilities:

• Design, develop, and maintain data pipelines and architectures.
• Implement and optimize data integration and transformation processes.
• Collaborate with data scientists and machine learning engineers to integrate AI/ML models into data workflows.
• Utilize cloud platforms (AWS, GCP, Azure) to deploy and manage data solutions.
• Ensure high-quality data ingestion and processing through rigorous testing and validation.
• Participate in code reviews to maintain high code quality and standards.
• Troubleshoot and debug data engineering solutions, ensuring reliability and performance.
• Work with version control systems (preferably git) to manage and maintain codebases.

Mandatory Skills (Need Proficiency):

• Experience: 5 to 10 years in software/data engineering.
• Data Technologies: Proficiency in SQL, NoSQL databases (e.g., DynamoDB, MongoDB), ETL tools, and data warehousing solutions.
• Programming Languages: Proficiency in Python is a must.
• Cloud Platforms: Azure, AWS (e.g., EC2, S3, RDS) or GCP.
• Visualization Tools: Experience with data visualization tools (e.g., Tableau, Power BI, Looker).
• Data Governance: Knowledge of data governance and security practices.
• CI/CD: Experience with DevOps practices, including CI/CD pipelines and containerization (Docker, Kubernetes).
• Communication Skills: Excellent verbal and written communication skills in English.
• Agile Methodologies: Experience working in Agile development environments.
• AI/ML Awareness: Understanding of AI and ML concepts, frameworks (e.g., TensorFlow, PyTorch), and practical applications.0
• Generative AI Awareness: Familiarity with Generative AI technologies and their potential use cases.

Good to Have Skills:

• APIs: Understanding of RESTful APIs and their integration into data workflows.
• Advanced Analytics: Knowledge of advanced analytics and statistical techniques.
• Microservices: Familiarity with microservices architecture and its implementation.
• Data Lakes: Understanding of data lakes and their role in modern data architecture.
• Scripting: Proficiency in shell scripting and automation.
• AI/ML Deployment: Experience working with and deploying AI/ML models in production environments.
• Big Data Tools: Experience with Hadoop, Spark, Kafka, and other big data technologies.

Qualifications:

- Education: Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.

- Certifications: Relevant certifications in data engineering, cloud platforms, or AI/ML are a plus.