Data engineering involves the development of solutions for the collection, transformation, storage and management of data to support data-driven decision making and enable efficient data analysis by end users. It focuses on the technical aspects of data processing, integration, and delivery to ensure that data is accurate, reliable, and accessible in a timely manner. It also focuses on the scalability, cost-effectiveness, security, and supportability of the solution. Data engineering encompasses multiple toolsets and architectural concepts across on-premises and cloud stacks, including but not limited to data warehousing, data lakes, lake house, data mesh, and includes extraction, ingestion, and synchronization of structured and unstructured data across the data ecosystem. It also includes processing organization and orchestration, as well as performance optimization of data processing.
Bachelor’s degree or equivalent in Computer Science, Software Engineering, Information Systems, or related field.
Good written and verbal communication skills (English)
5+ years of working experience in building and supporting big data solutions
Strong experience in design, develop/coding, Implement and delivering robust and sustainable both streaming and batch big data solutions built on (AWS) cloud
Strong knowledge of Data Warehouse concepts and Schema modeling
Strong technical expertise with SQL and NoSQL databases and data mapping, as well as ETL tools (AWS cloud native tool preferred)
Strong developer experience in AWS Big Data technologies: S3, Redshift, Glue, Glue PySpark, Athena, EMR, Kinesis, Lambda
Strong developer experience in Open-Source Big Data technologies: Apache Spark, Hadoop, HDFS, HIVE or presto
Strong coding skills in Python, PySpark, Pandas, Boto3, SQL, JSON
Strong troubleshooting skills of big data platforms and data intensive workload (strong SQL query building skills)
Background in DataOps
Candidates need to work in Japan shift time, which is 8 AM to 5 PM (4:30 AM to 1:30 PM IST)
Knowledge of AWS technologies like API Gateway, SNS, AWS security, CloudFormation, node.js, serverless framework, AWS networking, Power-BI Data Visualization Tool. Experience in Azure or GCP as secondary cloud computing skill.
ADVANTAGES: Bash Shell Scripting, DevOps / CICD GitHub, Git Actions, Agile Information Sensitivity: General\Internal
Candidate should be able to design, configure, and manage databases to ensure optimal performance and reliability.
Experience with data integration tools and techniques, such as ETL and ELT Candidate should be able to integrate data from multiple sources and transform it into a format that is suitable for analysis.
The Data Engineer’s role is expected to build a scalable platform to collect, enrich C activate data across multiple channels to deliver data driven marketing. The Data Engineer will collaborate with business and senior technology associates to define the requirements for the data C analytics projects and develop solutions that will enable marketers to efficiently build and use platforms C services that enable data ingestion, enrichment, and activation.
Play a key role in collecting, transforming, curating, and serving data via big data solutions
Deliver actionable insights via reporting tools
Work on a reusable and scalable promotion platform with other team members
Work with marketers, data/business analysts to understand marketing challenges, business questions, processes etc.
Using Agile and Dev/Ops methods, build solutions utilizing technologies including AWS, Power BI C Adobe suite (AEP C ecosystem).
articipate in cross-functional teams to promote technology strategies, analyze, test products, perform proofs-of-concept, and pilot new technologies and/or methods.
Establish and document standards, guidelines, and best practices for teams utilizing the solutions.
Keep current on industry best practices and products that can be leveraged to support the company’s business objectives
We Offer:
Stable employment. On the market since 2008, 1300+ talents currently on board in 7 global sites.
100% remote.
Flexibility regarding working hours.
Full-time position
Comprehensive online onboarding program with a “Buddy” from day 1.
Cooperation with top-tier engineers and experts.
Unlimited access to the Udemy learning platform from day 1.
Certificate training programs. Lingarians earn 500+ technology certificates yearly.
Upskilling support. Capability development programs, Competency Centers, knowledge sharing sessions, community webinars, 110+ training opportunities yearly.
Grow as we grow as a company. 76% of our managers are internal promotions.
A diverse, inclusive, and values-driven community.
Autonomy to choose the way you work. We trust your ideas.
Create our community together. Refer your friends to receive bonuses.
Activities to support your well-being and health.
Plenty of opportunities to donate to charities and support the environment.