Basic monitoring recommendations

• Memory utilization
• CPU load
• Disk space
• Disk I/O activity
• Network traffic
• Clock accuracy

• Memory utilization: If your memory utilization is consistently near 75%, consider increasing the memory.
• CPU load: On healthy web application nodes, we typically see CPU load between 0 and 10 (with 10 being high). In your environment, if the CPU load is consistently above 5, you may want to get some thread dumps by using the jive snap command. Then you need to open a support case with Support. For more information, see Getting started with the new Jive Support Portal on Worx.
• Disk space: On the web application nodes, you need enough disk space for search indexes (which can grow large over time) and for attachment, image, and binary content caching. The default limit for the binstore cache is 512MB, you can configure it at Admin console: SystemSettingsStorage Provider). We recommend starting with 512MB for the binstore cache. Note that you also need space for generated static resources.
• Network traffic: While you may not need a specific alert for this, monitoring this is helpful for collecting datapoints. This monitor can be helpful for understanding when traffic dropped off.
• Clock accuracy: In clustered deployments, ensuring the clocks are accurate between web application nodes is critical. We strongly recommend using NTP to keep all of the server clocks in sync.

• Individual web application server
• Through the load balancer's virtual IP address

WebInject interacts with the web application to verify basic functionality. It provides functional tests beyond just connecting to a listening port. Checking individual servers, as well as the load balancer instance, verifies proper load balancer behavior. We recommend setting these checks every five minutes initially. To minimize false alarms, we require two failures before an alert is sent. If you find that these settings are resulting in too many false alarms, then adjust your settings accordingly.

For an example of WebInject XML code that performs all of the above, see WebInject code example.

• Java Management Extensions (JMX) hooks (heap)
• Disk space (logs)

• Heap: If your heap is consistently near 75%, consider increasing the heap size. For more information, see Adjusting Java Virtual Machine (JVM) settings.

• Stats for:
  • Connections
  • Transactions
  • Longest query time and slow queries
• Verify ETLs are running
• Disk space
• Disk I/O activity

• Connections: More connections require more memory. If you need to increase the maximum number of connections allowed by the Jive installation to the core database, consider adding more memory to the database server while ensuring that the database server has enough memory to handle the database connections. The maximum number of connections to the core database is the maximum number of connections allowed for each web application node times the number of webapp nodes in the Jive installation (For more information, see Getting basic system information). Out-of-the-box settings for the core database connections are 25 minimum, 50 maximum. For high-traffic sites in our hosted environment, we set the core database to 25 and 125. Note that additional nodes should be used instead of more database connections for managing additional traffic.

  Default connection settings for the analytics database are 1 minimum and 15 maximum (you may need to adjust this based on usage and load). For the activity engine database the defaults are 1 minimum and 50 maximum.

• Transactions: If the database provides an easy way to measure this number, it can be helpful for understanding the overall traffic volume. However, this metric is less important than monitoring the CPU, memory, and IO utilization for capacity planning and alerting.
• Longest query time and slow queries: It's helpful to monitor slow query logs for the database server that they're provisioned against. In our hosted (PostgreSQL) deployments, we log all slow queries (queries that take more than 1000ms seconds) to a file and then monitor those to help find any queries that might be causing issues that could be helped by database indexes.
• Verify ETLs are running: This is important only for the Analytics database. The easiest way to monitor this is by querying the jivedw_etl_job table with something like this: select state, start_ts, end_ts from jivedw_etl_job where etl_job_id = (select max(etl_job_id) from jivedw_etl_job); If the state is 1, the ETL is running. If any state is 3, there is a hard failure that you need to investigate. If the difference between start_ts and end_ts is too big, you may need to increase the resources for the Analytics database.
• Disk space: On the web application nodes, you need enough disk space for search indexes (which can grow large over time) and for attachment, image, and binary content caching. The default limit for the binstore cache is 512MB; you can configure it at Admin console: System > Settings > Storage Provider. We recommend starting with 512MB for the binstore cache. Note that you also need space for generated static resources. The most critical place to monitor disk space is on the database server; you should never have less than 50% of your disk available. We recommend setting an alert if you reach more than 50% disk utilization on the database server.
• Disk I/O activity: This is good to record because it can be important if you see slow performance on the web application nodes and excessive wait time.

• Tomcat I/O
• Heap
• Queue statistics (for example, average length and wait times)
• Running OpenOffice service statistics
• Overall conversion success rate for each conversion step

• Activity Engine service
• Java Management Extensions (JMX) hooks (heap) and ports
• Queue statistics (e.g., average length and wait times)

• Heap: If your heap is consistently near 75%, consider increasing the heap size. For more information, see Adjusting Java Virtual Machine (JVM) settings.
• For more information about the queue depths for the Activity Engine, see Configuring Activity Engine.