We are seeking a passionate Big Data Platform engineer to join our Global Data Platform team.

Responsibilities

• Operating Global Data Platform components (VM Servers, Kubernetes, Kafka) and applications (Apache stack, Collibra, Dataiku and similar).
• Implement automation of infrastructure, security components, and Continuous Integration & Continuous Delivery for optimal execution of data pipelines (ELT/ETL).
• Develop solutions to build resiliency in data pipelines with platform health checks, monitoring, and alerting mechanisms, quality, timeliness, recency, and accuracy of data delivery are improved.
• Apply DevSecOps & Agile approaches to deliver the holistic and integrated solution in iterative increments.
• Liaison and collaborate with enterprise security, digital engineering, and cloud operations to gain consensus on architecture solution frameworks.
• Review system issues, incidents, and alerts to identify root causes and continuously implement features to improve platform performance.
• Be current on the latest industry developments and technology trends to effectively lead and design new features/capabilities.

Qualifications - Bachelor’s degree in Engineering/Information Technology/Computer Science, or related discipline

Required Skills

• 5-7 years of experience in building or designing large-scale, fault-tolerant, distributed systems, (for example: data lakes, delta lakes, data meshes, data lake houses, data platforms, data streaming solutions…)
• In-depth knowledge and experience in one or more large scale distributed technologies including but not limited to: Hadoop ecosystem, Kafka, Kubernetes, Spark
• Migration experience of storage technologies (e.g. HDFS to S3 Object Storage)
• Integration of streaming and file based data ingestion / consumption (Kafka, Control M, AWA)
• Experience in DevOps, data pipeline development, and automation using Jenkins and Octopus (optional: Ansible, Chef, XL Release, and XL Deploy)
• Expert in Python and Java or another static language like Scala/R, Linux/Unix scripting, Jinja templates, puppet scripts, firewall config rules setup
• VM setup and scaling (pods), K8S scaling, managing Docker with Harbor, pushing Images through CI/CD
• Experience using data formats such as Apache Parquet, ORC or Avro Experience in machine learning algorithms is a plus.
• Hands-on experience in integrating Data Science Workbench platforms (e.g. Dataiku)
• Cloud migration experience might come handy.
• Experience of agile project management and methods (e.g., Scrum, SAFe).
• Knowledge of financial sector and its products is beneficial.

Preferred Skills - Experience in machine learning algorithms is a plus. Hands-on experience in integrating Data Science Workbench platforms (e.g. Dataiku). Cloud migration experience might come handy. Experience of agile project management and methods (e.g., Scrum, SAFe). Knowledge of financial sector and its products is beneficial.

Pay range and compensation package -SGD 9000-10000 per month