Azure Data Engineer with Pyspark

• Implementing data ingestion pipelines from multiple data sources using Azure Data Factory, Azure Databricks and other ETL tools.

• Developing Big Data and non-Big Data cloud-based enterprise solutions in PySpark and SparkSQL and related frameworks/libraries.

• Developing scalable and re-usable, self-service frameworks for data ingestion and processing.

• Integrating end to end data pipelines to take data from data source to target data repositories ensuring the quality and consistency of data.

• Processing performance analysis and optimization.

• Bringing best practices in following areas:

  design & analysis,

  automation (Pipelining, IaC),

  testing,

  monitoring.

  documentation.

• At least over 2 year of experience in Azure development.

• Having PySpark exposure, or intent to learn it.

• Experience in building ETL/ Data Warehouse transformation process.

• Very good knowledge of SQL.

• Hands-on experience designing and delivering solutions using Azure DataLake.

• Direct experience of building data pipelines using Azure Data Factory, Azure Databricks or Azure Synapse.

• Experience working with structured and unstructured data.

• Experience working in Agile environment (preferably Scrum).

• Experience in cloud-based solutions.

• Knowledge of data management principles.

Required softskills:

• Self-driving, able to seek out new tasks when old ones complete.

• Able to manage own time, estimate when work will be done, fill gaps with related work.

• Understands when to provide opinion versus seek specialist domain knowledge.

• Eager to learn new tools.

• Able to communicate clearly amongst team members.

• Stable employment. On the market since 2008, 1400+ talents currently on board in 7 global sites.

• “Office as an option” model. You can choose to work remotely or in the office. 

• Flexibility regarding working hours and your preferred form of contract.

• Comprehensive online onboarding program with a “Buddy” from day 1.   

• Cooperation with top-tier engineers and experts. 

• Unlimited access to the Udemy learning platform from day 1.

• Certificate training programs. Lingarians earn 500+ technology certificates yearly. 

• Upskilling support. Capability development programs, Competency Centers, knowledge sharing sessions, community webinars, 110+ training opportunities yearly.

• Grow as we grow as a company. 76% of our managers are internal promotions.

• A diverse, inclusive, and values-driven community.   

• Autonomy to choose the way you work. We trust your ideas. 

• Create our community together. Refer your friends to receive bonuses.

• Activities to support your well-being and health.

• Plenty of opportunities to donate to charities and support the environment.