ABOUT THIS FEATURED OPPORTUNITY
All internal teams that require data collection collaborate with this team. The team handles numerous custom reports, with every project needing some adjustments. The engineer will be responsible for updating these reports and making the reporting system configurable and scalable across different use cases. An ideal candidate will be proficient in automation, enabling pipelines to be adaptable and replicable for other internal customers.

This role involves taking over ownership of an existing data pipeline, ensuring its stability and enhancing it to be more accessible and user-friendly for a broader internal audience. The engineer will be expected to document, modularize, and generalize the pipeline so it can be easily adopted and reused by other teams.

You'll be building automation pipelines using Python to manage data flow within the environment. Python library experience is crucial for categorizing and managing large datasets. The data originates from audio recordings, videos, images, and metadata, which provide users with a menu-like interface. Once collected, the files can be selected and stored through the application.

KEY SUCCESS FACTORS (top 3 must haves)

1. 3+ years of experience in building and maintaining data pipelines.
2. Proficiency in Python automation and familiarity with JSON file handling.
3. Strong expertise in AWS S3, with a proven track record of managing and optimizing S3 operations.

NICE TO HAVES

1. Rio, conductor, airflow, bolt, blobby, cube
2. Experience with SQL for data querying and manipulation.