Create Production Ready ETL Pipeline using Python, Docker and Dagster

₹12500-37500 INR

Zavřený

Zveřejněno

před více než 2 roky

₹12500-37500 INR

Zaplaceno při doručení

We need to create a dockerizable ETL Pipeline for our data using Python and Dagster. Our requirement / data flowchart is very clear. The basic flow is read data from CSV, save to Kafka. Apply multiple transforms using functions (function code will be provided by us). Use intermediate kafka queues after each transform. Write final output to Postgres DB. Use Dagster for the data orchestration. We just need to implement it in a robust manner. We need :- 1. Docker-compose to setup the Kafka, Postgres and other services. 2. Create a DAG for orchestration of the pipeline using Dagster (using placeholders for the transform functions which our team will add later).

IČ projektu: 32203643

O projektu

2 nabídky

Vzdálený projekt

Aktivní před 2 roky

Chcete si vydělat nějaké peníze?

E-mailová adresa

Výhody podávání nabídek na Freelancer

Stanovte si rozpočet a časový rámec

Získejte za svou práci zaplaceno

Načrtněte svůj návrh

Registrace a podávání nabídek je zdarma

2 freelanceři nabízejí v průměru ₹27 500 INR za tuto práci

@singhyamini1821

** Can start immediately once you hire me. Hello, I am an experienced mobile app developer with an experience of 5+ years. I’m pretty confident that I will be able to create your app as required. I am an expert in making android iOS as well as web apps with remarkable reviews on my profile. I have knowledge about: Objective C, Swift, Xcode, Script, Android studio, Sqlite, cloud services, firebase, in-app purchase, payment gateways, Google API integration, third-party API integration, UI, UX, PHP, SIP, MNV, MVVC, CBD modeling, XML, Angular JS, Jquery, SQL Server, HTML 5, CSS3, JAVA, API, Backend Admin panel, Cocoa-pods, SDK, NDK, Core and many more. Available for immediate discussion, Let's proceed and work together. Please share more details about your project. Thanks,

₹20 000 INR v 5 dnech