Scaling and High Availability
Hyper-scalable PAS resources are easily scaled up to provide more processing capability while also providing a highly available environment. As the volume of processing expands, adding additional Web and Background nodes distribute the workload, allowing traffic to be spread out over multiple nodes. Additionally, in the event of a system failure on one of the nodes, the additional nodes in the configuration are available to provide uninterrupted service as the failed node is replaced.
To ensure that your data is always available and that your environment can withstand system failures while optimizing system performance, follow the suggestions in this section.
Scale Your Environment to Balance Your Workload
As the load on your Hyper-scalable PAS installation increases, you may notice slower authentication and Admin Portal responses, or that the time to generate reports and synchronization has increased.
To handle incoming authentication requests and slower Admin Portal responses, you can add more Web nodes to your Installation. To add additional systems, see the following procedures:
- Phase 3: Creating a Deployment Package
- Phase 4: Deploying Hyper-scalable PAS software to Web, Background, and TCP Relay Nodes
- Phase 5: Activating the Deployment
You can use ASP.Net Performance Counters to monitor performance of:
- Current connections
- Requests in application queue
- Processor – % processor time
- Memory – available Mbytes
See https://stackify.com/asp-net-performance-counters/ for information on using ASP.Net.
To evaluate Background node load, look for a line such as the following:
2020-04-12 18:34:38,955 [DevInstance-DevInstance|1d65a33bcaba4f4db97b2e1bfd5038a3|275|(null)|(null)|566070|INFO |(null)] JobMonitor:
CloudFire Metrics: 0 jobs in Queue, plus 1 jobs running.
Average Queue Time: 0; Longest: 0
Longest Running Job: 203997 for Job ID: ABC0123:f729408a-fa46-f373-0c00-2a94a51e6f29, Tenant: ABC0123
In general, you should expect very few jobs to be queued as jobs should be in the running state quickly. If jobs are queued but no jobs are running, you may need to restart the Background nodes. If jobs are running and yet there are multiple queued jobs, adding another Background node allows more jobs to run simultaneously, clearing out the queue.
Provide Uninterrupted Service in the Event of System Failure (High Availability)
Use the Centrify-PAS-WatchLogs command to monitor your environment and watch or capture logs from the Web, and Background nodes. If you detect an error in one of the Web or Background nodes, and your configuration contains more than one Web or Background node, you can easily replace the faulty system without interrupting the service. To avoid a single point of failure in your Hyper-scalable PAS solution, be sure that the other components, such as cache, database, Connector, load balancer are also scaled up.
Note that node monitoring is dependent on your organizations chosen software. For information on replacing a faulty system and disaster recovery, see the following sections:
