Job Summary
The position involves supporting the development of new system functionalities while also ensuring the maintenance and enhancement of bulk data pipelines and applications within a customer-driven cloud environment. The role requires building and documenting ETL workflows, generating comprehensive reports and dashboards from transformed data logs, evaluating data quality, and working closely with customers and integration partners.
Responsibilities
- Support the implementation of new system features.
- Maintain and improve bulk data pipelines and applications in a cloud setting.
- Construct and document ETL workflows.
- Transform logs and additional data into insightful reports and dashboards.
- Assess and ensure the quality of data.
- Collaborate effectively with customers and integration partners.
Qualifications
- Active security clearance with the requisite polygraph.
- Proven experience in big data processing.
- Proficient in Python, Spark, ETL processes, and SQL databases.
- Experience with both structured and unstructured data.
- Strong background in analytics reporting and data quality analysis.
- Familiarity with data ETL documentation and collaboration with customers.
- Experience working in an Agile development environment.
- Knowledge of Databricks, AWS services, and the ELK stack is preferred.
