Engineers

Azure Data Engineer with Pyspark

Poland

Apply Now
Engineers tablet&mobile
Tasks
  • Implementing data ingestion pipelines from multiple data sources using Azure Data Factory, Azure Databricks and other ETL tools.

  • Developing Big Data and non-Big Data cloud-based enterprise solutions in PySpark and SparkSQL and related frameworks/libraries.

  • Developing scalable and re-usable, self-service frameworks for data ingestion and processing.

  • Integrating end to end data pipelines to take data from data source to target data repositories ensuring the quality and consistency of data.

  • Processing performance analysis and optimization.

  • Bringing best practices in following areas:

  • design & analysis,

  • automation (Pipelining, IaC),

  • testing,

  • monitoring,

  • documentation.

 
Requirements

Essential technical skills:

  • At least over 2 year of experience in Azure development.

  • Having PySpark exposure, or intent to learn it.

  • Experience in building ETL/ Data Warehouse transformation process.

  • Very good knowledge of SQL.

  • Hands-on experience designing and delivering solutions using Azure DataLake.

  • Direct experience of building data pipelines using Azure Data Factory, Azure Databricks or Azure Synapse.

  • Experience working with structured and unstructured data.

  • Experience working in Agile environment (preferably Scrum).

  • Experience in cloud-based solutions.

  • Knowledge of data management principles.

Required softskills:

  • Self-driving, able to seek out new tasks when old ones complete.

  • Able to manage own time, estimate when work will be done, fill gaps with related work.

  • Understands when to provide opinion versus seek specialist domain knowledge.

  • Eager to learn new tools.

  • Able to communicate clearly amongst team members.

Nice to have:

  • Big data related topics (Hadoop stack, HDInsight, Azure Databricks) knowledge.

  • Data Science knowledge (AI, ML).

 

Offer
  • Stable employment. On the market since 2008, 1200+ talents currently on board in 7 global sites.

  • “Office as an option” model. You can choose to work remotely or in the office.

  • Flexibility regarding working hours and your preferred form of contract.

  • Comprehensive online onboarding program with a “Buddy” from day 1.

  • Cooperation with top-tier engineers and experts.

  • Certificate training programs. Lingarians earn 500+ technology certificates yearly.

  • Upskilling support. Capability development programs, Competency Centers, knowledge sharing sessions, community webinars, 110+ training opportunities yearly.

  • Grow as we grow as a company. 76% of our managers are internal promotions.

  • A diverse, inclusive, and values-driven community.

  • Autonomy to choose the way you work. We trust your ideas.

  • Create our community together. Refer your friends to receive bonuses.

  • Activities to support your well-being and health.

  • Plenty of opportunities to donate to charities and support the environment.

  • Modern office equipment. Purchased for you or available to borrow, depending on your location.

Share:

Linkedin-job-offer facebook-job-offer