We are looking to hire an experienced Lead Kafka Engineer who will be responsible for installing, monitoring, troubleshooting, and maintaining Kafka platform, ensuring optimal performance, security, developing new features/automation/integration.

Responsibilities

• Install and provision new Kafka clusters and supporting infrastructure
• Continuously monitor the health and performance of Kafka platforms and data pipelines
• Troubleshoot and resolve issues related to data pipelines, network disruptions, and other infrastructure failures
• Perform regular performance tuning and cost optimization for Kafka components
• Manage the upgrade process for Kafka platforms, including planning, testing, and implementation
• Enforce and manage security protocols including access control, encryption, and conduct regular security audits
• Implement disaster recovery procedures and conduct regular platform backups
• Oversee capacity management and scaling projections
• Document technical procedures, configurations, issue resolutions and share knowledge across teams
• Collaborate with internal and vendor support teams for escalated issue resolutions
• Maintain and enhance Infrastructure as Code (IaC) and Configuration Management (CM) automations
• Develop and refine onboarding and automation scripts for streamlined operations
• Facilitate Kafka setup for application teams including consumers, producers, and connectors
• Respond to team requests efficiently, converting complex issues into CLOUD Tickets when required
• Integrate new vendor features and capabilities in collaboration with relevant stakeholders
  
  
  

Requirements

• Proven experience in the implementation and maintenance of Confluent Platform
• Minimum 8 years of experience in Kafka administration
• Knowledge of Helm and Kubernetes
• Proficiency in deploying Kafka in Kubernetes
• Skills in Python or Shell Scripting
• Background in cloud technologies including AWS, GCP (compute, networking, storage, IAM)