January 2023 - Agile Mag Newsletter
2023 begins with a brand new edition of our newsletter: AGILE MAG, where Mag is both Magazine (i.e. a collection of news about Agile, the Data Eng world, Data Mesh, Big Data etc.) and also a Magnet (the place where interesting things converge). We also changed the structure of the newsletter to make it better and clearer for you.
We hope you'll enjoy it!
This edition's highlights:
📊 Top 6 Data Trends for 2023: a report about the emerging trends for Data Management
✒️ Technology Abstraction in the Data Mesh: P. Platter, Agile Lab CTO, talks about technology in Data Mesh implementations
▶️ Conference Insights: some feedback about conferences focused on data
🎖️ AWS Advanced Tier Service Partner: a new partnership level for Agile Lab
📣Open Positions: new opportunities to join our team
Top 6 Data Trends for 2023
The report is out! Our stellar engineers reviewed existing trends that haven't fully materialized for 2022 as well as emerging topics that will be the focus of this year.
Here are the 6 Data Trends we envision for 2023:
1. Productization of Data, to achieve faster delivery of value and ROI in the long term
2. Data Governance, to reduce lock-in and optimize costs
3. Significant Changes to Vertical Markets, to address which are the industries that face the most particular challenges
4. Speed of Delivery, to manage, protect and value data quickly and efficiently
5. Broader Adoption of Data Mesh, to address the challenges given by traditional architectures
6. New Key Skills required, to achieve success in “engineering” data production and delivery.
Technology Abstraction in the Data Mesh
In this article, we address the BIG mistake companies make by staking their Data Mesh implementation on technology when the entire paradigm shift is based on a technology-agnostic approach.
2023 begins with a new achievement: Agile Lab is now an Amazon Web Services (AWS) Advanced Partner. This new partnership level certifies our team's strong knowledge and skills and our customer experience in driving innovation and building solutions on AWS. We'd like to thank all the Agilers for their commitment and effort to achieve this goal.
Elite Data Engineering
Conference Insights: expanding horizons
As Agilers, we get the chance to attend conferences throughout the year to explore the continuously changing world of Data Engineering.
We are pleased to share some insights reported by our team regarding some events attended during the last year. We hope you can find some interest suggestions:
Kafka Summit 2022 discussed the use and challenges of Kafka at a high scale and the future of Kafka. Observability on streaming pipelines, Trino, test containers, Azul, OpenTelemetry, LakeFS were some of the new technologies/best practices/tools discussed.
Kubecon2022 focused on the growing popularity of multi-cloud and the use of Cilium for Multi-Cloud. OpenTelemetry and its use as a central monitoring framework, toxiproxy and the use of eBPF were among the new technologies/best practices/tools discussed. An interesting project using Cilium and Kubernetes in a multi-Cloud solution. link: https://www.youtube.com/watch?v=NGuiizWUuaw.
Domain-Driven Design Europe 2022 discussed Data Mesh, modeling exercises, Technical coaching with the Samman method, and Knowledge crunching without bias. The conference also covered how to apply ubiquitous language to tests and data quality and how to scale company culture and drive transitions and changes. The conference also discussed books like Training From the Back of the Room.
DeepLearn 2022 Summer discussed the benefits of overfitting and the importance of latent representations and kernel machines in modern and effective modeling techniques. XAI (eXplainable AI), image classification/detection and dataset bias were among the new technologies/best practices/tools discussed.
We are always looking for the best talents in the market!
If you are a SRE III or a SOFTWARE ENGINEER IV (Full Stack) or would like to join a top-tier data engineering firm with a remote-first culture and an international mindset, take look at our OPEN POSITIONS!
Feel free to share the newsletter with your team and friends.
Was this email forwarded to you? Sign up HERE.