Jobs 1 - 20 out of 205 for Big Data Engineer

Own a component of the data platform end to end, including pipelines, tables, quality checks, monitoring, and recovery. Design and build ETL on EMR (Spark) and Airflow, and data ingestion from OLTP sources. Model the data for analytical use cases, ensure data quality, tune runtime and infrastructure...

Design, build, and operate large-scale data processing pipelines and analytics platforms on Hadoop and related big-data ecosystems. Responsible for ingesting, transforming, and analyzing massive volumes of structured and unstructured data to support enterprise analytics, machine learning, and report...

The senior developer's role in this project will involve designing and implementing platform components, ingesting data, processing and normalizing the data, distributing the data to different stakeholders, building and improving common framework including monitoring, CI/CD pipelines, testing, perfo...

As a Principal Big Data Engineer, you will be responsible for architecture, design, and development of our data ingestion and processing platform. You will design and implement different architectural models for scalable data processing and storage, and build tools for data ingestion from multiple s...

Develop and maintain big data solutions, work with technologies such as Java, Docker, Spring Boot, Kubernetes, and AWS. Collaborate with teams to implement data pipelines and ensure data integrity.

Ingest data, process and normalize it, and distribute to stakeholders. Build and improve frameworks including monitoring, CI/CD, testing, and code quality. Maintain and support applications in production. Collect and refine business requirements, design solutions, lead development activities, and su...

The senior developer's role in this project will involve designing and implementing platform components, ingesting data, processing and normalizing the data, distributing the data to different stakeholders, building and improving common framework including monitoring, CI/CD pipelines, testing, perfo...

Develop and optimize complex data pipelines, integrate and optimize data and model pipelines within production environments, collaborate with data science teams to review model-ready datasets, perform data discovery and analysis of raw data sources, and write and maintain model monitoring scripts.

Develop and maintain big data solutions, implement ETL processes, and work with data warehousing technologies. Utilize GCP BigQuery and Java frameworks to build APIs.

Design and implement a scalable infrastructure on AWS using Cassandra, Elasticsearch, Postgres, Kafka, Spark to support terabytes of data. Enhance current infrastructure solutions to scale as data grows. Build monitoring solutions for the infrastructure components. Build scalable solutions on top of...

Build scalable data pipelines, develop ML models, and integrate big data systems. Focus on optimizing data systems, building ETL pipelines, and deploying AI models in cloud environments.

Build and optimize big data pipelines, architectures, and datasets. Develop real-time data pipelines in a large-scale, high-volume production environment. Utilize advanced SQL knowledge and work with unstructured datasets.

Design and develop scalable big data solutions using Hadoop, Spark, and Hive. Architect and implement big data pipelines and workflows, ensuring efficiency, security, and reliability. Deploy big data solutions, leveraging best practices for optimal performance in large-scale environments. Optimize b...

Ingest data, process and normalize it, distribute to stakeholders, build and improve frameworks including monitoring, CI/CD, testing, and performance. Maintain applications in production, design solutions, lead development activities, support continuous improvement, deliver production-ready services...

The Big Data Engineer will design, develop, and maintain the client's big data ecosystem to support advanced analytics and data-driven decision-making across engineering, procurement, and construction workflows. Responsibilities include designing and implementing big data architectures, developing E...

Design, develop, and maintain large-scale data processing pipelines using Big Data technologies. Implement data ingestion, storage, transformation, and analysis solutions. Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions. Optimiz...

Design, develop, and maintain large-scale data processing and analytics platforms. Build and optimize data pipelines for batch and real-time data processing. Develop and manage data lakes, data warehouses, and distributed storage systems. Integrate data from multiple internal and external data sourc...

Develop and implement big data solutions, collaborate with teams to achieve common goals, and build applications using Spark frameworks. Strong hands-on experience in Java/J2EE and Spark is required, along with knowledge of AWS and experience in building Big Data technologies.

Collaborate on the Smartstream cash reconciliation platform to support data integration and transformation workflows. Design and implement ETL processes using tools like AWS Glue, Spark, or Python. Conduct data mapping exercises to align legacy data structures with modernized attributes stored in S3...

Design and maintain scalable, reliable big data pipelines. Optimize Spark/Hadoop workloads for performance, scalability, and cost efficiency. Implement automated testing and data quality validation. Enable analytics and data science teams with high-quality, accessible datasets. Leverage AI-assisted ...