Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add caching layer to dns resolved ips #3265

Closed
wants to merge 4 commits into from
+36 −12
Closed

chore: add caching layer to dns resolved ips #3265

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
3945114
chore: add caching layer to dns resolved ips
Jayachand Apr 9, 2024
14d3922
chore: include ttl value in dns cache
Jayachand Apr 25, 2024
c1b34c4
Merge branch 'develop' into chore.cache-dns-resolve-ips
Jayachand Apr 25, 2024
ea813af
Merge branch 'develop' into chore.cache-dns-resolve-ips
Jayachand Apr 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 21 additions & 1 deletion src/util/utils.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
const fetch = require('node-fetch');

const util = require('util');
const NodeCache = require('node-cache');
const logger = require('../logger');
const stats = require('./stats');

Expand All @@ -16,11 +17,30 @@
const LOCALHOST_OCTET = '127.';
const RECORD_TYPE_A = 4; // ipv4

const DNS_CACHE_ENABLED = process.env.DNS_CACHE_ENABLED === 'true';
const DNS_CACHE_TTL = process.env.DNS_CACHE_TTL ? parseInt(process.env.DNS_CACHE_TTL, 10) : 300;
const dnsCache = new NodeCache({
useClones: false,
stdTTL: DNS_CACHE_TTL,
checkperiod: DNS_CACHE_TTL,
});

const fetchResolvedIps = async (hostname) => {
let ips = dnsCache.get(hostname);

Check warning on line 29 in src/util/utils.js

View check run for this annotation

Codecov / codecov/patch

src/util/utils.js#L29 

Added line #L29 was not covered by tests
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add metrics around cache hits, misses etc

if (ips === undefined) {
ips = await resolver.resolve4(hostname);

Check warning on line 31 in src/util/utils.js

View check run for this annotation

Codecov / codecov/patch

src/util/utils.js#L31 

Added line #L31 was not covered by tests
if (ips?.length > 0) {
dnsCache.set(hostname, ips);

Check warning on line 33 in src/util/utils.js

View check run for this annotation

Codecov / codecov/patch

src/util/utils.js#L33 

Added line #L33 was not covered by tests
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we consider making the cache ttl dynamic like set lower value of the ttl specified in the dns record and the DNS_CACHE_TTL env

}
}
return ips;

Check warning on line 36 in src/util/utils.js

View check run for this annotation

Codecov / codecov/patch

src/util/utils.js#L36 

Added line #L36 was not covered by tests
};

const staticLookup = (transformerVersionId) => async (hostname, _, cb) => {
let ips;
const resolveStartTime = new Date();
try {
ips = await resolver.resolve4(hostname);
ips = DNS_CACHE_ENABLED ? await fetchResolvedIps(hostname) : await resolver.resolve4(hostname);
} catch (error) {
logger.error(`DNS Error Code: ${error.code} | Message : ${error.message}`);
stats.timing('fetch_dns_resolve_time', resolveStartTime, {
Expand Down
Loading