Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Put campaign projects cache in redis instead of saving that in memory #1144

Merged
merged 2 commits into from
Oct 5, 2023
Merged

Put campaign projects cache in redis instead of saving that in memory #1144

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a9ad8f9
Put campaign projects cache in redis instead of saving that in memory
mohammadranjbarz Oct 5, 2023
cafffc9
Change default value of CACHE_PROJECT_CAMPAIGNS_CRONJOB_EXPRESSION
mohammadranjbarz Oct 5, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions config/example.env
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -206,5 +206,5 @@ QF_ROUND_ESTIMATED_MATCHING_CACHE_DURATION=60000
# OPTIONAL - default: Every 10 minutes
PROJECT_CAMPAIGNS_CACHE_DURATION=600000

# OPTIONAL - default: */10 * * * * * ( Every 10 minutes)
PROJECT_CAMPAIGNS_CRONJOB_EXPRESSION=*/10 * * * * *
# OPTIONAL - default: *0 */5 * * * ( Every 5 minutes)
PROJECT_CAMPAIGNS_CRONJOB_EXPRESSION=0 */5 * * *
4 changes: 2 additions & 2 deletions config/test.env
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -179,5 +179,5 @@ QF_ROUND_ESTIMATED_MATCHING_CACHE_DURATION=1
PROJECT_CAMPAIGNS_CACHE_DURATION=1


# OPTIONAL - default: */10 * * * * * ( Every 10 minutes)
PROJECT_CAMPAIGNS_CRONJOB_EXPRESSION=*/10 * * * * *
# OPTIONAL - default: *0 */5 * * * ( Every 5 minutes)
PROJECT_CAMPAIGNS_CRONJOB_EXPRESSION=0 */5 * * *
22 changes: 16 additions & 6 deletions src/services/campaignService.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import { ProjectResolverWorker } from '../workers/projectsResolverWorker';
import { QueryBuilder } from 'typeorm/query-builder/QueryBuilder';
import { findAllActiveCampaigns } from '../repositories/campaignRepository';
import { logger } from '../utils/logger';
import { getRedisObject, setObjectInRedis } from '../redis';

const projectFiltersCacheDuration =
Number(process.env.PROJECT_FILTERS_THREADS_POOL_DURATION) || 60000;
Expand Down Expand Up @@ -41,13 +42,17 @@ const createFetchCampaignProjectsQuery = (

return projectsQueryParams;
};
let projectCampaignCache: { [key: number]: string[] } = {};
const PROJECT_CAMPAIGN_CACHE_REDIS_KEY =
'projectCampaignCache-for-projectBySlug';

export const getAllProjectsRelatedToActiveCampaigns = (): {
export const getAllProjectsRelatedToActiveCampaigns = async (): Promise<{
[key: number]: string[];
} => {
}> => {
const projectCampaignCache = await getRedisObject(
PROJECT_CAMPAIGN_CACHE_REDIS_KEY,
);
// It returns all project and campaigns( excluding manuallySelectedCampaign)
return projectCampaignCache;
return projectCampaignCache || {};
};

export const cacheProjectCampaigns = async (): Promise<void> => {
Expand All @@ -67,10 +72,15 @@ export const cacheProjectCampaigns = async (): Promise<void> => {
: (newProjectCampaignCache[project.id] = [campaign.slug]);
}
}
projectCampaignCache = newProjectCampaignCache;
await setObjectInRedis({
key: PROJECT_CAMPAIGN_CACHE_REDIS_KEY,
value: newProjectCampaignCache,
// cronjob would fill it every 10 minutes so the expiration doesnt matter
expiration: 60 * 60 * 24 * 1, // 1 day
});
logger.debug(
'cacheProjectCampaigns() ended successfully, projectCampaignCache size ',
Object.keys(projectCampaignCache).length,
Object.keys(newProjectCampaignCache).length,
);
};

Expand Down
2 changes: 1 addition & 1 deletion src/services/cronJobs/updateProjectCampaignsCacheJob.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import { CacheProjectCampaignsWorker } from '../../workers/cacheProjectCampaigns
// every 10 minutes
const cronJobTime =
(config.get('CACHE_PROJECT_CAMPAIGNS_CRONJOB_EXPRESSION') as string) ||
'*/10 * * * *';
'0 */5 * * *';

const projectsFiltersThreadPool: Pool<
ModuleThread<CacheProjectCampaignsWorker>
Expand Down
Loading