fixes igniterealtime#409: Prevent loading all archived messages in me…

…mory when indexing

When iterating over all rows of a potentially large table, ensure that the database result set is configured to be 'linear' (forward-only and read-only).

This gives a better chance of the database driver to release all rows that have been iterated over, which prevents the fetch buffer to eventually include all rows (and potentially cause out-of-memory issues).

changelog.html

@@ -47,6 +47,7 @@ <h1>
 <p><b>2.7.0</b> -- To be determined</p>
 <ul>
     <li>Requires Openfire 5.0.0</li>
+    <li>[<a href='https://github.com/igniterealtime/openfire-monitoring-plugin/issues/409'>Issue #409</a>] - Building Lucene index should not load all messages in memory</li>
     <li>[<a href='https://github.com/igniterealtime/openfire-monitoring-plugin/issues/401'>Issue #401</a>] - Fixes: Update Jersey from 2.35 to 2.45</li>
     <li>[<a href='https://github.com/igniterealtime/openfire-monitoring-plugin/issues/398'>Issue #398</a>] - Fixes: Missing translation for system property</li>
     <li>[<a href='https://github.com/igniterealtime/openfire-monitoring-plugin/issues/392'>Issue #392</a>] - Fixes: Compatibility issue with Openfire 5.0.0</li>

plugin.xml

@@ -6,7 +6,7 @@
     <description>Monitors conversations and statistics of the server.</description>
     <author>Ignite Realtime</author>
     <version>${project.version}</version>
-    <date>2024-11-22</date>
+    <date>2024-12-10</date>
     <minServerVersion>5.0.0</minServerVersion>
     <databaseKey>monitoring</databaseKey>
     <databaseVersion>8</databaseVersion>

src/java/com/reucon/openfire/plugin/archive/impl/MucIndexer.java

@@ -116,16 +116,21 @@ private Instant indexMUCMessages( IndexWriter writer, Instant since )
             // The entire process took under 8 seconds.
             // Preventing the driver to collect all results at once depends on auto-commit from being disabled, at
             // least for postgres. Getting a 'transaction' connection will ensure this (if supported).
+            // MSSQL differentiates between client-cursored and server-cursored result sets. For server-cursored result
+            // sets, the fetch buffer and scroll window are the same size (as opposed to fetch buffer containing all
+            // the rows). To hint that a server-cursored result set is desired, it should be configured to be 'forward
+            // only' as well as 'read only'.
             con = DbConnectionManager.getTransactionConnection();
 
             if ( since.equals( Instant.EPOCH ) ) {
-                pstmt = con.prepareStatement(ALL_MUC_MESSAGES);
+                pstmt = con.prepareStatement(ALL_MUC_MESSAGES, ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY);
             } else {
-                pstmt = con.prepareStatement(NEW_MUC_MESSAGES);
+                pstmt = con.prepareStatement(NEW_MUC_MESSAGES, ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY);
                 pstmt.setString(1, StringUtils.dateToMillis(Date.from(since))); // This mimics org.jivesoftware.openfire.muc.spi.MUCPersistenceManager.saveConversationLogBatch
             }
 
             pstmt.setFetchSize(250);
+            pstmt.setFetchDirection(ResultSet.FETCH_FORWARD);
             rs = pstmt.executeQuery();
 
             long progress = 0;