Data Flow Manager

Working as a Data Flow Manager you will:

• Design, implement, evaluate, and support sophisticated data processing pipelines within Cloudera DataFlow (Apache NiFi), managing everything from data ingestion and transformation to enrichment and routing.

• Develop and fine-tune Change Data Capture (CDC) architectures to enable real-time or near-real-time data processing, utilizing tools like NiFi, Kafka, and Debezium or SQL CDC connectors.

• Establish seamless integrations between internal architectures and external data sources using various protocols, including REST API, JDBC, and Kafka.

• Oversee data structures and lifecycles by maintaining Avro schemas and managing metadata as well as data lineage through Apache Atlas.

• Ensure robust data security and governance by configuring appropriate policies and utilizing tools such as Apache Ranger.

• Actively monitor pipeline performance, troubleshoot stability issues, and drive the full automation of data flow processes.

• Partner closely with business stakeholders, architects, and fellow engineers to define system architecture, draft comprehensive flow documentation, and author Standard Operating Procedures (SOPs) and runbooks.

• Support system enhancements by participating in the upgrade processes for platforms like CDP, NiFi, and Kafka, and evaluating new data connectors.

Chabre IT Services is a global professional IT services provider, building long-lasting relationships with Enterprises. We specialize in the delivery of tailor-made solutions,  smart outsourcing, try&hire, and success fee services. We are a smart IT boutique with unique knowledge, which will deliver your ideas into reality.

Our client is a large international public-sector organization operating in a highly regulated and security-focused environment. The organization is responsible for coordinating complex operations, ensuring operational resilience, and maintaining continuity of critical services across multiple stakeholders and jurisdictions. You will join an institution that values structured governance, strong security standards, operational excellence, and cross-functional collaboration in an international setting.

• Proven professional experience in building, maintaining, and monitoring complex data pipelines and data flow architectures.

• Strong, hands-on proficiency with Cloudera DataFlow and Apache NiFi.

• Solid knowledge of real-time streaming technologies, particularly Apache Kafka.

• Practical experience with Change Data Capture (CDC) mechanisms, including Debezium and SQL CDC.

• Familiarity with integration standards and protocols such as REST API and JDBC.

• Experience in managing data schemas (specifically Avro) and working with metadata tracking tools like Apache Atlas.

• Understanding of data security and governance principles, with the ability to configure policies using Apache Ranger.

• Strong analytical skills with a track record of troubleshooting performance bottlenecks and automating operational tasks.

• Excellent communication skills for collaborating with technical teams and business units to produce high-quality architectural documentation.

• Rate up to 200,00 PLN/h + VAT
• Onsite work (Warsaw)
• Subsidy for peripherals in the amount of 500,00zł
• Working tool (MacBook Pro or Lenovo Legion 5)
• Co-financing of courses related to the position
• Benefits: MultiSport, Medicover