About

The Client is a Texas-based telecom company participating in the federal Lifeline Support Program and providing pre-paid cell phones and service packages to low-income individuals.

Challenge

As a part of the project, VolgoTechnologies analytics team was to design and implement data management and analytics platform to let the Client collect the data from multiple sources and get insights into customer behavior. The Client wanted the platform to analyze historical data and enable forecasting. Access rights were another issue to solve, as the Client planned to provide their tenants with the access to the tenant-related analytics.

Solution

The data analytics platform was gathering raw data (such as user s impressions and click-throughs, tariff plans, device models, apps installed and more) from 10+ sources. To collect this telemetry data and move it into Apache Kafka, VolgoTechnologies big data team suggested the MQTT protocol.

The team also suggested using Amazon Spot Instances to reduce the costs of AWS computing resources. To ensure the analytical system s scalability, they used AWS Application Load Balancers.

Apache Kafka acted as a data streaming platform. There, the raw data was organized for further offload into the landing zone that was running on Amazon Simple Storage Service. For data storage and warehousing, Amazon Redshift was chosen, where the telemetry data from mobile phones running on Android, as well as the information from the Enterprise Resource Planning and the Home Location Register (HLR) was supplied to.

Staging

Data Ware House

Data Ware House

Desktop Application

Results

With VolgoTechnologies big data services, the Client was able to Measure the engagement and identify the preferences of a particular user. Spot trends in the users behavior. Make predictions about how users would behave. Invoice advertisers based on their calculated share. Benefit from insightful data analytics (for example, daily earnings, number of new users, customer service data and more).

Technologies and Tools

Amazon Web Services (Amazon cloud), Apache Kafka (data streaming), the Message Queuing Telemetry Transport Protocol, Amazon Simple Storage Service (persistent storage used for data landing zone), Amazon Redshift (data warehouse), Airbnb Airflow and Python (ETL).