Data Engineer

Krakow, Poland

Contract

For performing data collection, processing, storage and integration. Proficiency in data Extract-Transform-Load (ETL) processes, data pipeline, database and data warehouse technologies on AWS cloud are required. Prior experience with timeseries and unstructured (e.g., image) data is essential.

Job Description:

• Overall, 5-8 years’ experience with expertise in performing data collection, processing, storage and integration.
• Designing and building ETL Pipelines using AWS Glue/Pyspark and familiar with ETL processes
• Standardizing code across all workflows to maintain consistency and scalability.
• Loading initial data into the required workflows and validating data integrity. Conducting unit tests on all converted jobs to ensure accuracy and reliability.
• Implementing a layered architecture in AWS and Snowflake for efficient data consumption.
• Creating comprehensive mapping documents for all existing processes and assets.
• Experience in processing all timeseries and Structured / Unstructured data to support ongoing Data requirements
• Developing additional views and logic to supplement reporting requirements.
• Deploying solutions to QA/UAT environments and providing support for Production migration.
• Knowledge of CICD pipelines and Jenkins.
• Proficient in architecture principles, design patterns, and best practices.
• Hands-on experience with cloud platforms (e.g., AWS) and related services (AWS Glue, AWS Athena, AWS Lambda, AWS Devops, AWS CloudWatch, AWS S3, Event Bridge).
• Proficiency in PySpark, Python and Snowflake.

For more info, please share updated CV at [email protected]