BUG: query logging memory hog

[davidsword]

Describe the bug

Big bulk operations in the WP-CLI context on an environment where WP_DEBUG is true can, in some use cases, result in an unsustainable growth of this query logging:

ElasticPress/includes/classes/Elasticsearch.php

Lines 1738 to 1740 in 72e9dc0

	if ( ( defined( 'WP_DEBUG' ) && WP_DEBUG ) || ( defined( 'WP_EP_DEBUG' ) && WP_EP_DEBUG ) ) {
	$this->queries[] = $query;
	}

A filter should be added to the conditional for this query logging so the logging can be explicitly disabled for these bulk operations without having to change the WP_DEBUG constant, ex:

public function add_elasticpress_version_to_user_agent( $user_agent ) {
	 * @param array $query Query to log.
	 */
	protected function add_query_log( $query ) {
-		if ( ( defined( 'WP_DEBUG' ) && WP_DEBUG ) || ( defined( 'WP_EP_DEBUG' ) && WP_EP_DEBUG ) ) {
+		$disable_logging = apply_filters( 'ep_disable_query_logging', false );
+		if ( ! $disable_logging && ( ( defined( 'WP_DEBUG' ) && WP_DEBUG ) || ( defined( 'WP_EP_DEBUG' ) && WP_EP_DEBUG ) ) ) {
			$this->queries[] = $query;
		}

Steps to Reproduce

This doesn't happen on out of the box EP installs, but can happen in certain use cases. For us at VIP it happens out of the box for how we're using EP (more on this later), to replicate:

• setup local wp site using vip dev-env, ensure "Enable Elasticsearch"
• set vip-config.php to

define( 'WP_DEBUG', true );
define( 'VIP_ENABLE_VIP_SEARCH', true ); // Enable Enterprise Search
define( 'VIP_ENABLE_VIP_SEARCH_QUERY_INTEGRATION', true ); // Integrate search queries with Enterprise Search

• have some posts on the site
• create following wpcli cmd

if ( defined('WP_CLI') && WP_CLI ) {
    class Test_CLI_Command {

        public function ing( $args, $assoc_args ) {
            global $wpdb;

            wp_defer_term_counting( true );
            $counter = 0;
            // loop forever to replicate a huge sites bulk operation
            while ( true ) {
                $posts = get_posts( [
                    'post_type' => 'post',
                    'posts_per_page' => -1,
                    'fields' => 'ids',
                ] );
                foreach ( $posts as $post_id ) {
                    update_post_meta( $post_id, 'test', rand() );

                    \WP_CLI::line( "MEMORY: " . round(memory_get_peak_usage() / 1024 / 1024, 2) . "MB" );
                    
                    $counter++;
                    if ($counter % 100 === 0 ) {
                        \WP_CLI::line("batch of 100 done");

                        // memory cleanup 
	                $wpdb->queries = array();
                    }
                }
            }
        }
    }
    WP_CLI::add_command( 'test', 'Test_CLI_Command' );
}

• run the cmd wp test ing
• notice the slow growth of memory usage (about +0.01mb every handful of posts)
• comment out this

  ElasticPress/includes/classes/Elasticsearch.php

  Line 1739 in 72e9dc0

  $this->queries[] = $query;

• run cmd again, notice no slow growth of memory

This is happening specifically on VIP's use case because:

• updated_post_meta in custom cli script triggers this
• EP does a kill_sync check on that callback
• kill_sync runs this filter
• which VIP taps into here
• where VIP does an index_exists check
• index_exists makes a remote request here
• which then gets unsustainably logged here

We reviewed how we've hooked into this for the index check, but we believe it would be best to have a filter for this query logging so this logging can be disabled entirely in the WP-CLI context without using the wider-use WP_DEBUG constant.

Screenshots, screen recording, code snippet

see above

Environment information

No response

WordPress and ElasticPress information

WP 6.6
EP 4.2.2

Code of Conduct

• I agree to follow this project's Code of Conduct

[felipeelia]

Makes sense, @davidsword. Would you or your team have the availability to craft a PR with that new filter?

[rebeccahum]

@felipeelia Done in #4019