Question 11
Domain 2: Security Logging and MonitoringA company wants to create a log analytics solution for logs generated from its on-premises devices. The logs are collected from the devices onto a server on premises. The company wants to use AWS services to perform near real-time log analysis. The company also wants to store these logs for 365 days for pattern matching and substring search capabilities later. Which solution will meet these requirements with the LEAST development overhead?
Correct answer: C
Explanation
Amazon Kinesis Agent can ship on-premises log files to AWS with minimal code, and Amazon Data Firehose provides managed delivery for near real-time ingestion. Amazon OpenSearch Service supports “pattern matching and substring search,” and an Index State Management policy can “delete the data after 365 days,” meeting retention with the least development overhead.
Why each option is right or wrong
A. Install Amazon Kinesis Agent on the on-premises server to send the logs to Amazon DynamoDB. Configure an AWS Lambda trigger on DynamoDB streams to perform near real-time log analysis. Export the DynamoDB data to Amazon S3 periodically. Run Amazon Athena queries for pattern matching and substring search. Set up S3 Ufecycle policies to delete the log data after 365 days.
DynamoDB is a key-value database, not a natural fit for log search analytics.
B. Install Amazon Managed Streaming for Apache Kafka (Amazon MSK) on the on-premises server. Create an MSK cluster to collect the streaming data and analyze the data in real time. Set the data retention period to 365 days to store the logs persistently for pattern matching and substring search.
MSK is a streaming platform, not something installed on an on-premises server for log search.
C. Install Amazon Kinesis Agent on the on-premises server to send the logs to Amazon Data Firehose. Configure Amazon Managed Service for Apache Flink (previously known as Amazon Kinesis Data Analytics) as the destination for real-time processing. Store the logs in Amazon OpenSearch Service for pattern matching and substring search. Configure an OpenSearch Service Index State Management (ISM) policy to delete the data after 365 days.
Amazon Kinesis Agent is the lowest-overhead way to ship existing on-premises log files into AWS because it runs on the source server and streams files directly to Kinesis Data Firehose without custom ingestion code. Firehose can then deliver the stream to Amazon Managed Service for Apache Flink for near real-time processing, while Amazon OpenSearch Service is the AWS service that supports full-text pattern matching and substring search; an Index State Management policy can enforce a 365-day retention period by deleting indices after 365 days.
D. Use Amazon API Gateway and AWS Lambda to write the logs from the on-premises server to Amazon DynamoDB. Configure a Lambda trigger on DynamoDB streams to perform near real-time log analysis. Run Amazon Athena federated queries on DynamoDB data for pattern matching and substring search. Set up TTL to delete data after 365 days.
API Gateway, Lambda, and DynamoDB add custom ingestion complexity and still lack native log search strengths.