Facilitating the growth of news platform audience with cloud-based data analytics

As a large media group, our client has an online platform that covers news worldwide. In order to grow its popularity and publish trending content as soon as it becomes relevant, our client required effective analytical tools that would allow them to monitor topics that were only partially covered or not presented by their own platform as compared to the competing news outlets. They would then use this information to make corresponding tweaks to present more relevant content to their audience.

Our team has helped the client develop a brand-new solution for analyzing competitors' content and capturing trending topics on the basis of defined hours. After assessing the customer’s needs, our team chose AWS to build this solution as it offers the highest standard of security and provides the best tools for achieving set goals.

The solution uses parsers of competitors' sites that were developed by our team to select new topics and upload the source on Amazon. It then reads the source, identifies keywords in articles, and groups trending topics. We have implemented Amazon API Gateway to allow our client to access the customer system and choose the type of searched topic. After that, the data is transmitted to ElasticSearch, from where it is enriched with statistics.

We have also implemented Amazon S3 and AWS Lambda. The first allows storing parsed articles from competitors while the second makes it possible to determine when a new article would come into the Amazon S3 bucket. The solution uses Amazon SQS to record a queue of articles that have been processed with a cloud enricher, and makes queries from Amazon API Gateway to Amazon ElasticSearch.

Also, our team has used AWS CloudFormation, where we described the desired resources and their dependencies to launch and configure them together as a stack. We have implemented CloudWatch to track metrics and have used Amazon SNS to receive notifications when a new article has entered the bucket.

By utilizing various Amazon Web Services, we have developed a brand-new topic monitoring solution for website content analysis. Easy and user-friendly service allows the client to view popular topics in the competitors’ resources and make corresponding tweaks to present more relevant content to the audience. The new solution brings several substantial benefits to our client:

• Helps grow their news platform audience by presenting relevant and engaging content;
• Allows implementation of effective market and competitor analysis to identify trending topics;
• Enables the client to quickly adapt to changes in trends, raising their flexibility.