Agile Lab Services

Explore our extensive services covering data strategy, elite data engineering implementation, advancing AI practices, data platform enabling and much more

Data Strategy

A strategic approach to data shows the value of data-related practices

Overview

Being data-driven has always meant being influenced by information – and its availability – in multiple aspects, ranging from identifying areas of growth to mitigating unpredictable risks.

Nowadays, being data-driven also means having a clear and defined data strategy. Think of data strategy as a main map that indicates how the company defines its objectives regarding data (management, compliance, business), how the company defines practices and processes, and how these factors impact the organizational context.

Use Cases

Cost Savings with Data Strategy

Developing and implementing a data strategy has to take into consideration certain business aspects, such as reducing costs when designing a better service. The challenge of a large European bank was centered on the data strategy around the decoupling of Portfolio Management from Core Banking, located on Mainframe.

Solution

Agile Lab provided advisory services around data strategy by developing a data model aimed at facilitating decoupling from the Core Banking data model, which led to cost reduction (less MIPS consumption) via offloading the mainframe in real time.


Results

  • Easier online banking integration thanks document precomputation

  • Improved response time during business conversation

Benefits

From Cost Savings To Analytics on Risk

Reduction of costs related to MIPS and the solution implementation enabled up-to-date analytics on risk.

Data Strategy: from Data Lake to Verticals

Data Strategy practices can be applied throughout projects relating to data infrastructure as well as specific projects aimed to services and products availability and reliability.

A large worldwide commercial vehicles manufacturer designed and implemented several initiatives, such as: data infrastructure management, projects related to availability of parts for delivery trucks, and projects to ensure reliability in delivery schedule.

Solution

The practice over tools paradigm allowed Agile Lab to intersect data strategy with other practice pillars (Data Engineering, Machine Learning & AI), outlining a technology-independent advisory service, Agile Lab provided advisory services to outline the data strategy, enabling:

  • Established ways of sharing data between different teams

  • Improved engineering practices, orchestrations, and monitoring activities

  • Smooth transition from on-premise to cloud

  • Optimized batch and near-real time analytics use cases

Results

  • Data availability on cloud as another step towards Data Mesh adoption

  • Availability and reliability of data increased on a business context

Benefits

Practices and Technology Independence

The technology-independent approach balanced the use of technology and the enterprise-wide exposed practices.

Machine Learning (ML) & Artificial Intelligence (AI)

Advancing AI practices by solving complexity in real-world challenges 

Overview

The relationship between Machine Learning and Artificial Intelligence is broader than a simple subset placement. Advancements in AI and ML often stem from applied field practices, where new AI technologies enhance the capability of ML systems to, for example, recognize objects in image datasets.

Agile Lab aims to advance ML & AI practices within real-world projects, where factual experiences are accrued in environmental characteristics such as data volumes and complexity challenges.

Use Cases

Accelerated Claims Payment with ML

In crucial sectors like healthcare, it's imperative to balance the dual priorities of safeguarding customer data privacy while ensuring the swift processing of payment claims. A major health insurance company was challenged by the slow payments processing that hampered customer perception of the quality of the overall service.

Solution

Agile Lab created a fast processing platform for health claims extracting data from heterogeneous documents, such as images of medical invoices and medical documents, designing and implementing automated extractions and evaluations processing via a provided Machine Learning cloud service.


Results

  • Significant decrease of processing time for claims that led to increased customer satisfaction

  • Cleansed data from a pool of unstructured documents (text, images, invoices) that was transformed into structured information

Benefits

Decreased Complexity and Costs

Sensitive data is now managed and processed in a structured form, decreasing back-office costs.

ML Enables Object Detection

One of the challenges related to a Power Grid maintenance activity is the ability to process information and images generated by aerial reconnaissance systems, performing inspections along individual electrical transmission lines. The objective of a leading worldwide utility company was related to object detection of the Power Grid assets, characterized by high volumes of data, and improve suboptimal quality of collected images.

Solution

Agile Lab provided a Machine Learning solution to process the images generated and stored in a large dataset, such as:

  • Identify components of the transmission line in each image

  • Identify anomalies in components

  • Deduplicate components that appear in multiple images

Results

  • Accurate asset recognition & anomalies detection

  • Efficient Power Grid 2D mapping

Benefits

Automation Features

  • Enabled predictive maintenance over the Power Grid thanks to image processing performed by Machine Learning

  • Semi-automatic validation of external provider data

Design and Build an Enterprise-Wide AI Platform

A major banking client expressed the need to build an enterprise-wide AI Platform while dealing with specific challenges at both organizational and technical level. Skill balancing, deployment limitations, different environments composition, and lack of standardization/automation were the main themes when building the platform. It enabled the creation and sharing of ML models able to manage a catalogue of resources to be reused by the entire bank.  

Solution

Agile Lab provided advice and services to turn potential issues into guidelines to be adopted in a step-by-step approach, defining:

  • Skill rebalancing (headcount of Data Scientists and Data Engineers) among feature teams, thus preventing the platform team becoming a bottleneck.

  • Existing deployment limitations that could lead to inefficient resource usage

  • Standardized Sandbox and Staging environments to decrease the time spent form PoC to Production by using containerization technology.

  • Meticulous tracking, templating, and standardized data access for reproducibility and of ad-hoc solutions avoidance which would increase the AI Platform TCO

Results

  • Implementation of experiment tracking and model registry

  • Creation of a containerized development environment

  • Design of MLOps workflow

  • Development of reusable templates and components

Benefits

Improving Roles and Responsibilities

  • Introduction of Site Reliability Engineering role, new feature teams composition enabled self-service, and improved the speed and efficiency of the platform team

AI Pushed Over the Edge

The design and implementation requirements of a solution based on Machine Learning and Artificial Intelligence often involve Cloud Computing components in addition to resources already available on-premises.

There is also the option of the distributed intelligence provided by Edge Computing.

Devices at the Edge are powerful enough to solve complex problems in a low-power environment at acceptable costs, capable of leveraging Deep Learning to understand specific features in any observed area.

Solution

Introducing AIM2, Agile Lab's latest innovation designed to carve a niche in AI for Edge Computing, with a laser focus on Computer Vision. This new venture pioneers in deploying Edge AI solutions across distributed, energy-efficient frameworks, offering bespoke algorithms optimized for:

  • Real Time video analysis performed directly on the edge with edge resources

  • Portable Deep Learning algorithms dedicated to computer vision

  • Manage alerts notification, KPI calculation, and secure APIs access.

Results

  • PPE (Personal Protection Equipment) continuous monitoring

  • Large selection of supported PPE, such as masks, goggles, gloves

  • People counting, with alert generated when count changes

  • Access control with biometric and custom PPE

  • Man down recognition

Benefits

From Safety to Security

The Edge Computing AI solutions can perform a wide range of functions at reasonably low costs, including safety and security features

Data Engineering

The progress of Data Engineering as a specialization of software engineering

Overview

Data Engineering has been considered for decades a mechanical operation necessary to collect, cleanse, move, and secure data from a data store to another.

Eventually, this definition evolved rapidly, inspired by practices and processes developed by its older sibling: software engineering. Agile Lab elevated this concept as its own, in a world of constant change where the fate of software engineering and engineering can finally intersect after many years of distance.

Use Cases

Data Engineering in Motion

In the Utilities segment, data volumes and data velocity are closely connected to customers activities and efficiency of the Power Grid. A major worldwide utility enterprise needed to balance whopping volumes of data (Petabyte-scale) with the need to query, analyze data, and make critical decisions as quickly as possible, within a very complex environment.

Solution

Agile Lab enabled a massive data integration process through the design and implementation of a solution based on the analysis of streaming data.

The transition to a Data in Motion approach allowed the customer to overcome complexity and manage big datasets from the analytics perspective by adopting the streaming analytics paradigm. This empowered the company with fast analytics performance independent of dataset size.


Results

  • Implementation of full-fledged Power Grid automation

  • Flexibility in running Streaming Analytics on huge datasets with long-time depth events

Benefits

Guaranteed Compliance

High performance of streaming analytics solution compliant with national regulation directives in the utilities market segment

Real-Time KPI

Fine-grained – and real-time – monitoring at the atomic software level (Pod-based and subsystem-based) enabled the collection and evaluation of the needed performance indicators

Data Engineering for Flexible Offloading

The themes around Mainframe offloading (cost savings associated to MIPS usually processed in Core-Banking) have been around since last century, and were included in wide-spread initiatives as downsizing or rightsizing.

Agile Lab provided critical assistance to an Italian Bank, tackling the inflexibility rooted in their Mainframe Core-Banking systems. This rigidity resulted in unnecessary expenses for even basic tasks (like read operations) and hindered real-time communication across various channels.

Solution

Agile Lab, through its elite Data Engineering practice, studied, designed, and implemented an architecture that ensured Real Time Offloading, adding a CDC (Change Data Capture) component to detect status changes caused by operations on data. This is considered a key feature to achieve cost reduction and pave the way for new business opportunities.

Results

  • Total number of MIPS decreased while maintaining and enhancing existing functionalities

  • Flexible Real Time capabilities available for several use cases and business entities

Benefits

New Business Opportunities

New business opportunities enabled thanks to Real Time offloading capabilities (for instance, Real Time credit scoring using new architecture).

Cost Saving

Valuable MIPS resources saved in production environments, and Real Time capabilities were spread over different channels.

Customer-Centric Hub Built with Data Engineering

A 360 degrees Customer View requires speed and accuracy when building a serving layer for APIs to be consumed by several channels and applications.

An important Insurance customer faced challenges like data fragmentation on different data sources, lack of real-time features, and latency problems on its current architecture.

Solution

Agile Lab designed and created an innovative customer-centric hub, enabling a fast, robust serving layer for mobile and web applications. Customer portfolio data (policies, renewals, coverages) are constantly updated at fixed time intervals to ensure accuracy and APIs are managed by an API Gateway service which exposes the data through a serverless function.

 

Results

  • Easy and uniform data consumption through the implementation of Open API standard

  • API robustness and availability is ensured by modern application release techniques like blue/green deployment

Benefits

Enhanced Customer Experience

The customer-centric hub improved the customer experience, which led to increased customer loyalty and satisfaction through the availability of the portfolio data in mobile and web apps

Cost Reduction

The API lifecycle and fast and accurate data availability solution, independent of the consumption mechanism, lowered the overall operational costs

Performant Data Analytics thanks to Data Engineering

Data Engineering can be considered an enabler for projects that involve Data Design and Data Analysis, especially when volumes are geographically distributed.

Requirements from a multinational energy company were challenging: manage and analyze millions of devices across ten countries, maintaining world-class performances targets by implementing a Network Topology Graph, and standardize the Data Model for the entire company.

Solution

Agile Lab designed and implemented a Network Topology Graph that had absolutely relevant numbers:

  • 240 million vertices with more than 280 labels

  • 333 million relationships made of more than 14 types

The implementation was based on Neo4J graph database, exploiting its ACID properties to enable Data Analytics.

Results

  • Efficient engineering of graph traversing oriented toward performance gains in Data Analytics, designing power grid specific traversing

  • Fast execution of expensive computation, in a matter of milliseconds

  • Rapid enablement of Data Products

Benefits

Analytics Enablement

Data Analytics for Network Topology was enabled for all involved countries, without sacrificing performance at the expense of functionality.

Standardization

The customer adopted a standardized Data Model for Network Topology related information that enhanced the overall efficiency and speed.

Data Platform Enabling

How Agile Lab enables the creation and the evolution of an enterprise data platform 

Overview

The crafting of an enterprise data platform requires balancing the equities of data security, automation and overall practices. The practices around data platform enablement can address themes ranging from compliance to effectiveness and can be seen as tangible objects independent of any underlying technology.

Use Cases

Security & Compliance

Non-compliance with GDPR can cause a concrete risk of considerable fines and loss of consumer and stakeholder confidence. Data Scientists and Data Analysts need to continue using data and information closer to production data to build meaningful models and ensure performance and improvements to products and services.

Solution

Agile Lab designed and implemented a framework and a secure data-sharing practice for non-production environments by adhering to tenets and rules included in GDPR regulation, such as the minimization principle.

Agile Lab developed both logical and physical deletion techniques for production and non-production storage layers using both classical anonymization techniques (generalization, suppression, and data redaction) and advanced encryption methods like Crypto-Shredding and Format Preserving Encryption.

Results

  • Compliance with regulation, risk mitigation, and continuous trust from customers and stakeholders

  • Assurance of data integrity and consistent schemas between data sets

Benefits

Data Usability

Increased secure data usability and consumption for Data Analysts, Data Scientists, and Data Engineers while using and protecting meaningful data and information.

Model Efficiency

Using data closer to production environments while assuring compliance with security and regulation allows the customer to prepare and deploy real-world AI models, improving the quality of services offered to customers.

Data Quality

Customer data are often stored in a number of platforms and systems, causing data fragmentation and putting hurdles in the way when outlining a complete and coherent customer view. Multiple information sources and versions can cause inconsistencies and the lack of real time updates hinder a prompt response to customer needs.

Solution

Being practice-driven, Agile Lab contributed to the architectural design of a Customer Master Data Management project. The customer information are spread on several systems, so the project was equipped with a CDC (Change Data Capture) mechanism. Every operation (inserts, modifications, deletions) was contextually streamed to processes aimed to cleanse, deduplicate, and normalize data to achieve a unified customer view.

Results

  • The customer Golden Record as a single, authoritative source of customer-related information

  • Customer data checked and assured with precision and coherence, in real-time

Benefits

Better decisions with a single customer view

Processing customer information from a single source of truth enhanced decision-making ability by offering targeted services.

Real-Time Customer Updates

Real-Time updates allowed information to propagate with minimal latency, enabling market and customer needs. The customer information updates ensured compliance to governance regulation directives and data accuracy principle (GDPR).

Timing and Data Freshness

One of the most critical factors in Data Platform Enablement is the ability to consider and assure data freshness as a gold standard, especially when dealing with ML/AI, to ensure the needed time-to-market as projected by the business.

The main challenge of a customer was to provide a complete platform for Machine Learning projects, based on Cloud technologies, used by several teams within the company and aligned to the company standards.

Solution

Agile Lab provided consultancy and support when building this new Data Platform oriented toward Machine Learning. It designed an end-to-end cloud Data/ML engineering platform, deployed and maintained using strong automation features such as IaC (Infrastructure as Code).

The Data Freshness goal and the performance expectations were achieved, enabling performance monitoring and allowing the migration of existing on-premise solutions to the Cloud-based platform. The implemented solution sped up the transformation from a Prototype Model to Production-ready Model, ensuring a required customization for each project needs.

 

Results

  • A Data/ML platform, following best practices accrued in MLOps

  • Automation of the entire toolchain including automated data pipelines, model versioning, and containerization

  • Adoption of a Continuous Integration/Continuous Deployment cycle

Benefits

Consistent and Fresh Data

Data Freshness was induced by flexibility and customization built-in within the platform and by the improved model performance due to the reliability and efficiency in the automation defined by the Infrastructure as Code processes.

Enhanced Team collaboration

The solution provided by Agile Lab an enhanced collaboration between the team thanks to the implementation of Reproducibility, Monitoring and Maintenance functions available to all team participants.

Worldwide Scale of Data Mesh

One of the largest utility companies worldwide was presented with drawbacks related to a large number and dimensions of data silos. Prohibitive costs of data integration and unidirectional, point-to-point connections between silos required copying and handling data locally, leading to a weak governance strategy.

Solution

The solution provided by Agile Lab was centered around practices. Alignment between IT and Data executives was required to define a common ground made through a Data mesh conceptualization: principle, values, glossary, architecture, dissemination, matching the customer’s business strategy. The implementation phase was strongly based on standardization, high flexibility and strict compliance with customer’s expectations.

Results

  • Transition from an ‘Under Construction’ Data Mesh Platform to a ‘Stable’ Data Mesh platform, delivered as product and deployed to multiple subsidiaries of the holding company

  • Standardized Data Consumption patterns defined and applied

  • Reached thousands of data products in production across different LOBs

Benefits

Data Mesh network effect

Standardized Data Consumption patterns enabled teams to collect and determine skills necessary to design use cases oriented to business drivers, reusing the standardization approach, growing a Data Mesh network effect.

Standardization to cost-reduction

Agile Lab made an impact in cost reduction as no local data copying and no time and money was needed for data integration. It also made an impact in terms of strategy with clear data ownership and governance enforcement on data.

New Business Opportunities with a Scalable Data Platform

One of the themes of Data Platforms is to use them to find new business. An automotive company managing huge volumes of asset tracking data was forced by existing infrastructure restrictions  to query and manage the data with a static approach. The market and customers demanded processing, analysis, and responses with an eye towards real time capability and the enabling of data in motion, sometimes called ‘fast data’.

Solution

Agile Lab enabled the processing of streaming data coming from black boxes installed on vehicles. This solution helped the customer to repurpose the Data Platform to organically support different customers with the same software infrastructure, therefore lowering the TCO.

Results

  • New business opportunities allowing the company to better understand the behavior of the customer’s fleet

  • Adaptation of the offering/solution to market requirements, expanding market share

Benefits

Lower TCO

The Data in Motion enablement has transformed the Asset Tracking data assets to a multi-purpose, multi-customer Data Platform with optimal TCO, contributing to lower overall costs

Managed Data Platform

Transforming the data platform into a managed service to boost innovation and knowledge

Overview

The transition to managed data platform infrastructure is sometimes seen as an end, not a means. Beyond the promise of cost reduction, and beyond facilitated use, companies are following the path of Managed Data Platforms due to factors considered critical: knowledge of practices and technologies. Agile Lab supports customers facing this transition by combining technology support with the choice and application of practices suited to this change, independently of the deploy model.

Use Cases

Managed Data Platform to Empower Knowledge

One of the biggest companies in Italy selected Agile Lab to modernize its current Data Platform infrastructure and to solve several issues related to the existing architecture and technology resources. The legacy data platform was a key component of the shipments management, obstacles connected with old technology releases, suboptimal efficiency and unexpected downtimes during the regular provision of services.

Solution

Agile Lab provided advisory services centered on the migration from an existing on-premises data platform to CDP (Cloudera Data Platform) based on public cloud. The choice was ideal to accommodate the experienced volume increase of the service, which triggered a new capacity planning initiative to double the size of computational environments dedicated to the service.

Results

  • Generation and consolidation of new knowledge focused on the services management and lifecycle thanks to practices adoption

  • New areas – such as security – considered thanks to the migration to updated technology

Managed Data Platform Supporting Innovation

A major Insurance Company combined the needs for innovation, remarkable data volumes and the benefits of a Managed Data Platform to support a specific project related to ingestion and processing of data in motion coming from vehicle black boxes. Data in motion included a set of geographical and status information to provide addresses, trip and routes calculation and geofencing references to provide the desired quality of service for customers.

Solution

Agile Lab delivers pivotal support for the initiative, engaging both the suite of applications and underlying infrastructure. This encompasses a commitment to ongoing enhancement, adherence to the specified Service Level Agreement (SLA), optimizing service performance, and proactive measures to avert and handle any potential disruptions in service.

Results

  • 24x7 support guaranteed on the Data Platform

  • Tuning and infrastructure configuration of all the technologies included in the platform

A Robust and Fault-Tolerant Data Platform

Integrating data from several business entities and systems is complex and time-consuming. Differences in data formats, technologies, and communication protocols used make it difficult to combine and harmonize information.

Solution

Agile Lab has been engaged in designing, implementing, and supporting an on-premise, integrated data platform, based on Cloudera CDP, to solve underutilized resource issues and unexpected malfunctions while absorbing peak loads. The Data Platform was designed to achieve maximum integration while ensuring a higher degree of fault tolerance, allowing both interactive and batch analytics access.

 

Results

  • Significant reduction of wasted resources during regular operations time

  • Cost/time saving and data processing duration from hours to seconds during the peak phase

Benefits

From Better Analysis to Efficiency

This integrated architecture allowed the customer to make the most of its data, improving the quality of analyses, accelerating decision-making processes and optimizing business operations.

Provisioning of New Services

The Data Platform designed by Agile Lab allowed to create new vertical applications, such as Claims Management and Back Office Automation, and provisioning of new services for customers and for the branches network.

Contact Us

Get in touch if you want to know more about our solutions and projects.