#14884 | 2019-10-04 Vedbæk, Denmark

Data Engineer

For a client, we are looking for a Data Engineer with strong developer skills. Knowledge about the technologies is not enough to be a good match for the assignment. The ideal candidate will have practical experience with the requirements listed and should be able to deliver the task described in the job description.

Assignment task is developing:

  • Data transformation Proof of Concept (PoC)
  • Collecting data from SQL DB and XML files and transformation data via PySpark Data Frames into Paquet files
  • Verify the intended DataBricks based Spark SQL data model will deliver the required analytic capabilities and query performance
  • Verify the intended access authorization security model (Azure AD based) provides the required level of access control
  • Demonstrate results as queries executed through DataBricks Notebooks
  • Data Lake orchestration processes
  • Build ingestion pipeline based on Azure tools such as Data Factory
  • Build Data Lake orchestration based on a combination of DataBricks PySpark, Spark SQL, Parquet, Azure SQL DB and similar technologies

Requirements:

  • 2 years experience developing SQL based data processing
  • 2 years experience developing data processing in Python
  • 1-year experience programming cloud-based code in Azure
  • 1½ years experience developing in DataBricks PySpark and Data Frames
  • 1-year experience developing Azure Data Factory data pipelines
  • 1-year experience developing Azure Power BI Dashboards


Start: 17.10.2019
Duration: 19.12.2019
Work location: Vedbæk, Denmark
Requirements: Min. 5 years of professional IT experience.
Job type: Freelance

Opgaven er lukket

Vi beklager, men vi søger ikke længere konsulenter til dette projekt.

Klik på "Freelance job" for at se listen over aktuelle opgaver og job.

Hvis du er kunde og leder efter en lignende profil, kan du benytte "Få CV på IT-konsulent" formularen på vores andet website.

ProData Consult gemmer data i din browser / enhed ved hjælp af cookies med henblik på statistik og optimering af vores hjemmesider og eventuelt til målrettet annoncering. Ved at acceptere, giver du os dit samtykke til denne brug af cookies. Læs vores privatlivspolitik for mere information. Du kan altid trække dit samtykke tilbage her: Persondata politik & cookies

Webstedet kræver brug af "Nødvendige cookies". Vores nødvendige cookies bruges kun til at levere en korrekt fungerende hjemmeside og webservice.

Udvalgte tredjepartstjenester kan lagre cookies for at placere relevante annoncer, der leveres til dig på tredjepartswebsteder.