Join a dynamic team as a Data Engineer, where you'll work on cutting-edge data platforms and AI solutions. Your expertise in building reliable data pipelines will be crucial in supporting large-scale data operations.
Innovative and tech-focused
As a Data Engineer at PM Connection, you'll play a key role in developing and maintaining data platforms that support large-scale data operations. Your main responsibility will be to build scalable data pipelines for both batch and near-real-time data ingestion. This involves working with various data sources, including APIs, databases, and web sources, to ensure that data is efficiently collected and processed.
In addition to building pipelines, you'll also design and implement large-scale scraping solutions to acquire data. You'll need to clean, validate, and transform this data to make it suitable for analytics and AI applications. Your work will support data lake, warehouse, and lakehouse architectures, ensuring that the data is organized and accessible for various use cases.
The ideal candidate for this role should have proven experience in building production-grade data pipelines and a solid understanding of data quality checks and monitoring. If you are passionate about data engineering and eager to work with advanced AI technologies, this position is a great fit for you.
Key responsibilities include: • Building scalable data pipelines for batch and near-real-time ingestion. • Ingesting data from various sources including APIs, files, and event streams. • Designing and operating large-scale scraping solutions. • Cleaning, validating, and transforming data for analytics and reporting. • Supporting data lake and warehouse architectures.
This role is suited for someone with a strong technical background in data engineering and a desire to work on innovative projects that leverage AI and large-scale data solutions.
You'll be taken to the original listing on PNet to apply.