Lead Data Engineer

12 Month Contract

Remote, Located in Charlotte, NC

Overview

Brooksource is seeking an experienced Data Engineer to lead the design and implementation of scalable data pipelines and analytics tools on AWS, helping to revolutionize grid planning through ultra-high-speed, cloud-based analytics. In this role, you will support the development of next-generation tools that reduce grid planning analysis from weeks to minutes by leveraging an in-house Intelligent Grid Services (iGrid) platform and AWS’s expansive cloud infrastructure.

Key Responsibilities

• Provide technical leadership and direction for the design, build, testing, and deployment of scalable data solutions.
• Lead the development of data pipelines, ETL processes, and analytics tools using AWS technologies.
• Collaborate with cross-functional teams including Data Engineers, Software Engineers, Data Scientists, and Technical Test Leads to deliver high-quality solutions.
• Translate business requirements and use cases into technical scope and actionable development plans.
• Guide and mentor development teams, ensuring adherence to best practices, peer reviews, and accountability.
• Work closely with Product Owners to align on delivery goals, manage team backlogs, and prioritize tasks.
• Partner with Data and Solution Architects to define system architecture and ensure alignment with business needs.
• Support the implementation of integrated data quality frameworks and performance optimization strategies.
• Enable real-time data ingestion and event-driven architecture to support advanced analytics and forecasting.
• Ensure robust data security, access control, and compliance with encryption and logging standards.

Qualifications

• Proven experience leading teams in the delivery of complex, cloud-based data products.
• Strong hands-on development skills in AWS data services (e.g., S3, Glue, Lambda, Athena, EMR, Kinesis).
• Proficiency in programming languages such as Python, Java, Scala, and frameworks like Django and Pandas.
• Experience with infrastructure as code tools like Terraform.
• Expertise in building and optimizing big data pipelines and real-time data streams.
• Familiarity with power systems engineering, DER dispatch, and automated fault analysis systems.
• Strong communication skills and ability to engage with business stakeholders.
• Experience with data security practices including encryption, access control, and audit logging.
• Ability to support data scientists in model testing and validation.

Preferred Qualifications

• Experience with Hadoop ecosystem (Hive, HBase, Druid), Spark, PySpark, and SQL.
• Familiarity with workflow automation tools and CI/CD pipelines (e.g., Bitbucket, Concourse).
• Knowledge of API frameworks and DevOps practices.

Eight Eleven Group (Brooksource) provides equal employment opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, national origin, age, sex, citizenship, disability, genetic information, sexual orientation, gender identity, marital status, amnesty or status as a covered veteran in accordance with applicable federal, state, and local laws.