Solutions Architect Calculations (detailed)

This list is a starting point for estimates!

• My goal is to provide an easy reference for initial calculations.
• Many metrics have variations. I didn't have time/space to list all of them! Feel free to suggest important additions.
• Some metrics are more useful than others. All have limitations. Customize to your use case.
• This is for technical considerations, not marketing metrics (I did not cover them).

6 User Metrics for Growth/Projections

Most of these user calculation relate to right-sizing the app for the number of users we expect to have.

We need this user information to make sure we have the necessary right-sized resources to support the number of users we expect to have.

You can get the data for these metrics from your analytics tools, such as log analysis, javascript tracing, market analytics/research and/or simulated tests. Other related applications log or usage data that may help.

Remember, the more accurate your projections are, the better you will be able to plan resources effectively for performance, reliability and cost.

1. Daily Active Users (DAU)

The number of unique users who have engaged with the app in a day. Also, it can be a sum of all days in a period and expressed as an average daily number for that period of time.

DAU = Unique users / day

DAU avg = Unique users summed during time period / time period

(if daily avg. for multiple days, then divide the sum of DAUs each day by the number of days)

DAU helps estimate resource requirements. However, it's of limited help for specific usage patterns like spikes, or heavier traffic days vs. low traffic. So it's best as a starting point only and rough estimates.

It can be an average daily value over some period of days. For example looking at 3 days, if an app had 10,000 unique users on Monday, 12,000 unique users on Tuesday, and 7,000 unique users on Wednesday, then the DAU sum for those three days in the week so far would be 29,000, so the DAU average would be 29,000 / 3 = 9,667 DAU.

2. Monthly Active Users (MAU)

The formula for MAUs is the number of unique users who have engaged with the app in a given month.

It's different from DAU in that it's removing duplicate users from each day and is a rolling average over a month.

MAU = Unique monthly users / month

MAU at end of year = MAU at start of year * (1 + growth rate)^number of months

Cumulative MAU Growth projection = (MAU at end of year - MAU at start of year) / MAU at start of year * 100

MAU is useful when you expect each unique user to have a fixed requirement such as storage, or a specific amount of data to be processed.

Make sure to use unique users for MAU. If using marketing analytics data, this number is usually available using cookies and other tracking id means. Also, you can use a javascript tracking tool like Google Analytics or New Relic Browser to get this data.

3. Concurrent Users

The number of concurrent users is the number of users who are using the app at the same time.

Concurrent users = Active user sessions / time period

While this is a useful metric, it can be difficult to get an accurate number/sample for different scenarios/times. You may need to do some log analysis based on time period examples or use other analytics tools to get this data.

This is a highly useful metric if you are trying to estimate capacity to handle traffic spikes, bottlenecks, throughput, service quotas and scalability of the app.

Traffic patterns can greatly affect your app. Think of a special event, or a holiday, or a new product launch. The app may work well with 2,000 concurrent users on a normal day but spiking to 20,000 in a short period of time can cause major problems!

While you may have had enough capacity by DAU/MAU standards, you may not have enough capacity to handle the concurrent traffic spikes.

4. Actions Per User (APU)

Actions Per User (APU) is a metric measuring the average number of actions taken by each user within a specific time period.

APU = Total number of actions / total number of unique users

This calculation is useful for many reasons. For capacity planning it can help determine bandwidth, storage, read/write capacity etc.

You may wish to calculate average cost in resources per action in an effort to determine how to reduce costs. For example, if you have a cost of $0.02 per action, then you can estimate the cost of the app by multiplying the APU by the number of users. Then implement a way to reduce per action costs.

5. Daily User Actions (DUA)

Measures the number of actions taken by users on a website or mobile app within a specific time period, typically per day. It is used to measure engagement and activity on a website or mobile app.

DUA = Total Number of Actions / Total Number of Days

6. Requests Per Second (RPS)

Requests Per Second (RPS) measures the number of requests made to an application or website within a specific time period, typically per second.

It is a good metric to look at for individual service requests, such as a web page request, or a database query.

10 Network Calculations

note: This is assuming a host provider has been selected and some network infrastructure is already in place. These do NOT include all telco/router infrastructure.

If you are building a network from scratch then you will need many more considerations outside the scope of this list.

1. CIDR/Subnet Calculation

This is the subnet calculation for a given CIDR prefix and number of IPs available.

Number of IPs = 2³²-2^prefix

(where prefix is the number after the slash in the CIDR notation)

Examples:

• 10.0.0.0/28 = 2³²-2²⁸ = 2⁴ = 16 ips available
• 10.0.0.0/26 = 2³²-2²⁶ = 2⁶ = 64 ips available
• 10.0.0.0/24 = 2³²-2²⁴ = 2⁸ = 256
• 10.0.0.0/20 = 2³²-2²⁰ = 2¹² = 4,096
• 10.0.0.0/16 = 2³²-2¹⁶ = 2¹⁶ = 65,536
• 10.0.0.0/12 = 2³²-2¹² = 2²⁰ = 1,048,576

Most cloud providers reserve at least the first and last IP in a subnet for internal use. So in that case you would have 62, 254, 4,094, or 65,534 IPs available depending on the prefix.

Also, some providers may further restrict your range for smallest or biggest. For example, AWS may restrict you to a range of /28 to /16. (check for current restrictions)

2. Bandwidth Consumed/Utilization

Depending on your system design this can help with determine necessary capacity of network components such as routers, switches, and firewalls.

Bandwidth consumed = (Total data transferred / Total time)

Bandwidth Utilization = (Data transmitted in a certain time period / Total available bandwidth) * 100%

Accurate bandwidth projections help in monitoring network performance and identifying bottlenecks.

Also this would be very important for high bandwidth activities like streaming video or audio and gaming.

Also it can help in defining Service Level Agreements (SLAs) with service providers.

3. Available Bandwidth

Available bandwidth = (Total network capacity - Total bandwidth consumed)

This can be used to determine how much bandwidth is available for other activities.

Bandwidth can be affected by many factors such as physical infrastructure limits, Quality of Service (QOS) issues, network congestion, network traffic, software/hardware limitations, encryption overhead, provider contraints, and network latency.

4. Data Transmission Rate

This formula calculates the rate at which data is transmitted over a network link. The formula is:

Data Transmission Rate = Data size / Transmission time

This is another one particularly useful for high bandwidth activities like streaming video or audio and gaming.

5. Link Capacity

This formula calculates the capacity of a network link. The formula is:

Link Capacity = Bandwidth * Data transmission rate

6. Network Throughput

There are a couple variations to this formula. The most common is:

Network Throughput = Total Data Transferred / Total Time Elapsed

Network Throughput = Data size / Transmission time

For example, if a system transferred 1000 bytes of data in 2 seconds, the throughput would be:

7. Network Latency

There are several ways to measure this but here are a couple popular ones:

Network latency = Transmission time - Processing time

Network latency = (Transmission time + Propagation time + Queuing delay) / 2

Round Trip Time (RTT) = Latency * 2

Jitter = Maximum Latency - Minimum Latency

8. Network I/O:

The number of bytes sent and received by the instance's network interfaces.

Also could use the network packets: The number of packets sent and received by the instance's network interfaces.

9. Request Error rate

The number of times a request responded with an error .

Also could use other metrics such as Status check failures, that the instance's status check has failed.

10. Error response codes

1. 3x Error Codes: 3xx status codes indicate that the request needs further action from the client in order to be fulfilled. Some common 3xx status codes include:
   • 301 Moved Permanently
   • 302 Found
   • 304 Not Modified
2. 4x Error Codes: 4xx status codes indicate that there is an error with the client's request. Some common 4xx status codes include:
   • 400 Bad Request - missing required parameters
   • 401 Unauthorized - often due to lack of a API/user token
   • 403 Forbidden - often a permissions issue
   • 404 Not Found - the requested resource does not exist
   • 429 Too Many Requests - often due to exceeding rate limits
   • 499 Client Closed Request - often due to a timeout
3. 5x Error Codes: 5xx status codes indicate that there is an error with the server. Some common 5xx status codes include:
   • 500 Internal Server Error - often a server-side issue
   • 501 Not Implemented - the server does not support the functionality required to fulfill the request
   • 502 Bad Gateway - the server was acting as a gateway or proxy and received an invalid response from the upstream server
   • 503 Service Unavailable - the server is currently unavailable (overloaded or down)
   • 504 Gateway Timeout - gateway or proxy timeoutfrom the upstream server

Reliability

Mean Time Between Failures (MTBF)

This metric measures the average time that a system operates without a failure.

MTBF = Total Operating Time / Number of Failures

A high MTBF value indicates a more reliable system.

Mean Time to Repair (MTTR)

The average time that it takes to repair a failed system, calculated as

MTTR = Total Repair Time / Number of Failures.

A low MTTR value indicates a more efficient and effective process for repairing failed systems.

Availability

This metric measures the percentage of time that a system is available for use, calculated as (Total Operating Time / Total Elapsed Time). High availability is important to ensure that your systems are functioning optimally and delivering the required level of service to users.

Availability (as %) = Available for use time / Total time

Availability = MTTF / (MTTF + MTTR)

Availability is usually expressed in % or the shorthand of 9s.

Availability in 9s ("nines")

Availability Time in 9s = amount of time a system is available to users / by the total time * 100%

Availability Time in 9s = ( / (8 hours + 2 hours)) * 100% = 80%

According to Amazon AWS documentation:

Availability	Maximum Unavailability (per year)	Application Categories
99%	3 days 15 hours	Batch processing, data extraction, transfer, and load jobs
99.9%	8 hours 45 minutes	Internal tools like knowledge management, project tracking
99.95%	4 hours 22 minutes	Online commerce, point of sale
99.99%	52 minutes	Video delivery, broadcast workloads

99.999% 5 minutes ATM transactions, telecommunications workloads

Availability with Dependencies

(source: AWS docs) Availability = Availability of App  Availability of Dependency 1 Availability of Dependency 2 * (... Availability of Dependency n)

Calculating availability with redundant components.

(source: AWS docs) Availeffective = AvailMAX − ((100%−Availdependency)×(100%−Availdependency))

Shortcut calculation: If the availabilities of all components in your calculation consist solely of the digit nine, then you can sum the count of the number of nines digits to get your answer. In the above example two redundant, independent components with three nines availability results in six nines.

Service restoration time

The time it takes to restore a service after a failure.

This is important because it helps estimate the impact of a failure on the business.

For example, if a critical server is likely to need rebooting once per quarter

Recovery Time Objective (RTO)

RTO is the maximum acceptable delay between the interruption of service and restoration of service. This determines what is considered an acceptable time window when service is unavailable. RTO is similar to MTTR (Mean Time to Recovery)

Recovery Point Objective (RPO)

RPO is the maximum acceptable amount of time since the last data recovery point. This determines what is considered an acceptable loss of data between the last recovery point and the interruption of service.

• Error Rate

This metric measures the number of errors that occur in a system, calculated as (Number of Errors / Total Requests). High error rates can indicate issues with the system's design or implementation, and it's important to monitor this metric to ensure that your systems are functioning correctly.

Single Point of Failure (SPOF)

This metric measures the likelihood that a single component failure will cause a system to fail, calculated as (Number of Components in the System / Number of Critical Components). A high SPOF value indicates that a system is highly vulnerable to failure, and it's important to design systems with a low SPOF value to ensure high reliability.

Redundancy

This metric measures the level of redundancy in a system, calculated as (Number of Redundant Components / Total Number of Components). A high redundancy value indicates that a system is well-designed to withstand failures, and it's important to design systems with a high level of redundancy to ensure high reliability.

Capacity Utilization

This metric measures the utilization of system resources, calculated as (Total Utilization / Total Available Capacity). High capacity utilization can indicate that a system is overburdened and may not be able to perform optimally, so it's important to monitor this metric to ensure that your systems are functioning correctly.

Compute

IOPS (Input/Output Operations Per Second)

This refers to the number of read and write operations that can be performed by a storage device in one second. This is an important metric for storage arrays used in high-performance computing or database applications.

• Divide the total number of I/O operations performed by the time taken to perform those operations. For example, if a storage system performs 1000 I/O operations in 5 seconds, the IOPS would be 200 (1000 operations / 5 seconds = 200 IOPS). In this example, the units used to express IOPS are operations per second (ops/s). The IOPS value is calculated by dividing the number of I/O operations by the time taken to perform those operations.

Memory utilization: The percentage of available memory that is in use by the instance.

• Divide the amount of memory used by the total amount of memory available in the instance. For example, if an EC2 instance has 4 GB of memory and is currently using 2 GB, the memory utilization would be 50% (2 GB / 4 GB = 50%).

• CPU utilization: The percentage of allocated compute units that are currently in use by the instance.

  • Divide the number of CPU cycles used by the total number of CPU cycles available in the instance. For example, if an EC2 instance has 4 vCPUs and is currently using 2 vCPU hours, the CPU utilization would be 50% (2 vCPU hours / 4 vCPUs = 50%).

• Concurrent connections: The number of concurrent connections to the compute instance.

  • To calculate concurrent connections, simply count the number of active connections established between a client and a server at a given time. For example, if there are 10 active connections established between a client and a server at a given time, the number of concurrent connections would be 10.

• Load average: A metric that indicates the average system load over a specified time period.

  • Load Average: Load average is calculated as the average number of processes waiting for CPU time over a specified time period. Typically, the load average is calculated over a 1-minute, 5-minute, and 15-minute time period. For example, if the 1-minute load average is 1.0, this means that, on average, there was 1 process waiting for CPU time over the past minute.

• CPU credits: The number of CPU credits consumed by the instance.

• CPU balance: The number of CPU credits remaining for the instance.

• Lambda invocations: The number of times that the Lambda function has been invoked.

• Lambda duration: The amount of time that the Lambda function has been running.

• Concurrent executions: The number of executions of your Lambda function that are currently running. This metric can be used to track the load on your function and ensure that it has sufficient capacity to handle incoming requests.

• Throttles: The number of times that the Lambda service has throttled the execution of your function. This can occur when your function exceeds the maximum number of allowed concurrent executions.

• Dead Letter Errors: The number of times that an event was unable to be processed by your function and was sent to the dead-letter queue.

Queue/Event Processing

• Queue depth: The number of messages or events that are currently waiting to be processed in the queue. This metric can be used to monitor the load on your system and ensure that it has sufficient capacity to handle incoming requests.

• Retries: The number of times that a message or event has been retried due to a processing error. This metric can be used to monitor the reliability of your system and ensure that messages or events are being processed successfully.

• Dead letters: The number of messages or events that have been moved to a dead-letter queue due to a processing error. This metric can be used to monitor the reliability of your system and ensure that messages or events are being processed successfully.

• Consumer count: The number of consumers that are currently consuming messages or events from the queue. This metric can be used to monitor the load on your system and ensure that it has sufficient capacity to handle incoming requests.

• Consumer utilization: The percentage of time that consumers are actively processing messages or events. This metric can be used to monitor the utilization of your system's resources and identify any potential performance bottlenecks.

• Consumer Request Rate: The rate at which consumers are reading messages from Kafka topics, measured in messages per second. This metric can be used to monitor the performance of your system and identify any potential bottlenecks.

• Replication Lag (Kafka): The amount of time that it takes for messages to be replicated from the leader broker to the followers, measured in milliseconds. This metric can be used to monitor the performance of your system and ensure that messages are being replicated in a timely manner.

• Offset Lag (Kafka): The difference between the latest offset in a topic and the offset of the latest message consumed by a consumer group. This metric can be used to monitor the progress of consumers and ensure that they are processing messages in a timely manner.

• Under-Replicated Partitions (Kafka): The number of partitions in a Kafka topic that do not have the required number of replicas. This metric can be used to monitor the health of your system and ensure that messages are being replicated correctly for high availability.

• Broker Capacity (Kafka): The utilization of disk, CPU, and memory resources on each Kafka broker. This metric can be used to monitor the utilization of your system's resources and identify any potential performance bottlenecks.

Storage

• Cost per gigabyte: This refers to the cost of storing one gigabyte of data. It is an important metric for evaluating the cost-effectiveness of different storage solutions.

• RAID (Redundant Array of Independent Disks): This refers to a method of using multiple disk drives to provide data protection in case of disk failure. Different RAID configurations provide different levels of data protection and performance.

• Disk read and write operations: The number of read and write operations performed on the instance's attached disks.

• Read and write latency: The amount of time that it takes to perform a read or write operation on the instance's attached disks.

• Read and write throughput: The amount of data that can be read or written to the instance's attached disks in a given time period.

• Read and write IOPS: The number of read and write operations that can be performed by the instance's attached disks in a given time period.

• Data Transfer In: The amount of data that is transferred to the instance.

• Data Transfer Out: The amount of data that is transferred from the instance.

• Disk Space Utilization: The amount of disk space that is being used on your storage system, measured as a percentage. This metric can be used to monitor the utilization of your storage resources and ensure that you have sufficient capacity.

• Disk Space Available: The amount of disk space that is available on your storage system, measured in bytes or gigabytes. This metric can be used to monitor the utilization of your storage resources and ensure that you have sufficient capacity.

• Object Count: The number of objects that are stored in your object storage system. This metric can be used to monitor the utilization of your storage resources and ensure that you have sufficient capacity.

• Object Size: The average size of objects stored in your object storage system, measured in bytes. This metric can be used to monitor the utilization of your storage resources and ensure that you have sufficient capacity.

Caching

• Cache Hit Ratio: The ratio of requests that are served from your CDN cache, as opposed to being fetched from the origin server. This metric can be used to monitor the efficiency of your CDN cache and identify any issues with cache hit rates.

• Origin Latency: The amount of time it takes for your CDN to retrieve content from the origin server, measured in milliseconds. High origin latency can indicate a performance issue with your origin server or network connectivity.

• Edge Latency: The amount of time it takes for your CDN to deliver content to the user, measured in milliseconds. High edge latency can indicate a performance issue with your CDN distribution or network connectivity.

• Invalidation Count: The number of invalidations that have been requested for your CDN distribution. This metric can be used to monitor the efficiency of your CDN cache and identify any issues with cache hit rates.

• Connected Clients (Redis): The number of clients that are connected to your Redis instance. This metric can be used to monitor the utilization of your system's resources

• Operations Per Second (Redis): The number of operations that Redis is processing per second. This metric can be used to monitor the performance of your Redis instance and identify any issues with resource utilization.

• Expired Keys (Redis): The number of keys that have expired and been deleted from Redis. This metric can be used to monitor the performance of your Redis instance and ensure that it is functioning optimally.

Cost

• Resource Utilization: This metric measures the utilization of compute, storage, and network resources in your cloud environment. By monitoring resource utilization, you can identify resources that are underutilized or overutilized, and adjust your architecture accordingly to optimize costs.

• Data Transfer: This metric measures the amount of data transferred in and out of your cloud environment. By monitoring data transfer, you can identify areas where you may be able to optimize your architecture to reduce data transfer costs.

• Storage Costs: This metric measures the cost of storage resources in your cloud environment, including the cost of storing data and backups. By monitoring storage costs, you can identify areas where you may be able to optimize your architecture to reduce storage costs.

  • Storage Utilization: To calculate storage utilization, divide the total amount of storage used by the total amount of storage available in your environment. For example, if your environment has 1 TB of storage and you used 800 GB in a given period, your storage utilization would be 80% (800 GB / 1 TB = 80%).

• Compute Costs: This metric measures the cost of compute resources in your cloud environment, including the cost of running virtual machines, containers, and serverless functions. By monitoring compute costs, you can identify areas where you may be able to optimize your architecture to reduce compute costs.

  • Compute Utilization: To calculate compute utilization, divide the total number of CPU or vCPU hours used by the total number of CPU or vCPU hours available in your environment. For example, if your environment has 10 vCPUs and you used 8 vCPU hours in a given period, your compute utilization would be 80% (8 vCPU hours / 10 vCPUs = 80%).

• Network Costs: This metric measures the cost of network resources in your cloud environment, including the cost of data transfer and the cost of using cloud-based network services such as VPNs and load balancers. By monitoring network costs, you can identify areas where you may be able to optimize your architecture to reduce network costs.

  • Network Utilization: To calculate network utilization, divide the total amount of data transferred by the maximum capacity of your network. For example, if your network has a maximum capacity of 1 Gbps and you transferred 800 Mbps in a given period, your network utilization would be 80% (800 Mbps / 1 Gbps = 80%).

• Usage Trends: This metric measures the usage trends for your cloud environment over time, including the trend for resource utilization, data transfer, storage, compute, and network costs. By monitoring usage trends, you can identify areas where your costs are increasing and take steps to address the underlying issues.

• Cost Allocation: This metric measures the allocation of costs across different departments, teams, or applications within your organization. By monitoring cost allocation, you can ensure that costs are fairly distributed and that each team or application is paying for the resources it uses.

Security

https://docs.aws.amazon.com/whitepapers/latest/aws-security-incident-response-guide/establish-metrics-for-success.html

Mean time to detect Mean time to acknowledge Mean time to respond Mean time to contain Mean time to recover Attacker dwell time Use indicators of compromise (IOCs)

Metrics summary