Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev to Main Sync #2283

Merged
merged 32 commits into from
Dec 7, 2024
Merged
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
589b9b2
chore: add more query to search the logs
vinit717 Oct 29, 2024
2c4438a
Added seperate function for unobfuscated contact details and filtered…
RahulGoyal-tech Nov 16, 2024
f08329c
Merge branch 'develop' into iss2223
RahulGoyal-tech Nov 19, 2024
1974ba8
Merge branch 'develop' into iss2223
RahulGoyal-tech Nov 20, 2024
a6983a2
Removed id filtering for current data
RahulGoyal-tech Nov 20, 2024
3f52848
Merge branch 'develop' into iss2223
lakshayman Nov 21, 2024
3b67aa4
Merge branch 'develop' into iss2223
Achintya-Chatterjee Nov 21, 2024
e2aa779
Updated error message
RahulGoyal-tech Nov 22, 2024
acbae87
Merge branch 'develop' into iss2223
Achintya-Chatterjee Nov 26, 2024
d47204d
Added dev feature flag
RahulGoyal-tech Nov 29, 2024
b60f509
Merge branch 'develop' into fetch-logs
vinit717 Nov 29, 2024
41e6a5c
Feat: Implement API for Tracking Orphaned Tasks. (#2267)
VinuB-Dev Nov 30, 2024
10bc222
Feat: Implement APIs for Tracking Departed Users with Assigned Tasks …
VinuB-Dev Nov 30, 2024
6dbda50
Merge branch 'develop' into iss2223
Achintya-Chatterjee Nov 30, 2024
4e5f5da
Merge pull request #2253 from RahulGoyal-tech/iss2223
lakshayman Dec 1, 2024
e1405a0
chore: fix query and add dev flag
vinit717 Dec 3, 2024
bd65f8c
Merge branch 'fetch-logs' of https://github.com/Real-Dev-Squad/websit…
vinit717 Dec 3, 2024
aaa2de4
removed feature flag (#2281)
VaibhavSingh8 Dec 3, 2024
fccab02
Handling the conflict case where email has been already taken in AWS …
vikhyat187 Dec 3, 2024
74f8324
Add unverified role to db and remove on /verify (#2257)
shobhan-sundar-goutam Dec 3, 2024
9dec2c6
Merge branch 'develop' of https://github.com/Real-Dev-Squad/website-b…
vinit717 Dec 4, 2024
ef38a67
chore: write test for logs query
vinit717 Dec 4, 2024
46bc908
Fix: Handle firestore 'in' operator limitation by chunking queries; u…
VinuB-Dev Dec 4, 2024
8b74a52
Merge branch 'develop' into fetch-logs
vinit717 Dec 4, 2024
9cfad9d
chore: fix test and error message
vinit717 Dec 4, 2024
5fe2c65
Merge branch 'fetch-logs' of https://github.com/Real-Dev-Squad/websit…
vinit717 Dec 4, 2024
5228100
Standardize Logs Collection for Consistent Querying (#2226)
vinit717 Dec 5, 2024
daed096
Standardize logs meta field to use userId (#2248)
vinit717 Dec 5, 2024
1b65791
Merge branch 'develop' into fetch-logs
vinit717 Dec 5, 2024
5dc221c
Remove feature flag from arts/userId?dev=true API Endpoint (#2289)
vikasosmium Dec 5, 2024
978c9e9
Merge branch 'develop' into fetch-logs
vinit717 Dec 6, 2024
70cfd78
Merge pull request #2269 from Real-Dev-Squad/fetch-logs
lakshayman Dec 6, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions constants/logs.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@ export const logType = {
TASKS_MISSED_UPDATES_ERRORS: "TASKS_MISSED_UPDATES_ERRORS",
DISCORD_INVITES: "DISCORD_INVITES",
EXTERNAL_SERVICE: "EXTERNAL_SERVICE",
ADD_UNVERIFIED_ROLE: "ADD_UNVERIFIED_ROLE",
REMOVE_ROLE_FROM_USER_SUCCESS: "REMOVE_ROLE_FROM_USER_SUCCESS",
REMOVE_ROLE_FROM_USER_FAILED: "REMOVE_ROLE_FROM_USER_FAILED",
EXTENSION_REQUESTS: "extensionRequests",
TASK: "task",
TASK_REQUESTS: "taskRequests",
Expand Down
6 changes: 6 additions & 0 deletions controllers/awsAccess.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,12 @@ export const addUserToAWSGroup = async (req, res) => {
if (awsUserId === null){
// We need to create the user in AWS before and then fetch its Id
userCreationResponse = await createUser(userInfoData.user.username, userInfoData.user.email);

if (userCreationResponse.conflict){
return res.status(400).json({
error: `Username or Email is already being used, please use another email / username for creating account in AWS`
})
}
awsUserId = userCreationResponse.UserId;
}

Expand Down
2 changes: 1 addition & 1 deletion controllers/extensionRequests.js
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ const createTaskExtensionRequest = async (req, res) => {
type: "extensionRequests",
meta: {
taskId: extensionBody.taskId,
createdBy: req.userData.id,
userId: req.userData.id,
},
body: {
extensionRequestId: extensionRequest.id,
Expand Down
2 changes: 1 addition & 1 deletion controllers/extensionRequestsv2.ts
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ export const createTaskExtensionRequest = async (req: ExtensionRequestRequest, r
taskId,
requestId: extensionRequest.id,
action: LOG_ACTION.CREATE,
createdBy: requestedBy,
userId: requestedBy,
createdAt: Date.now(),
},
body: extensionBody,
Expand Down
20 changes: 11 additions & 9 deletions controllers/external-accounts.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ const { getDiscordMembers } = require("../services/discordService");
const { addOrUpdate, getUsersByRole, updateUsersInBatch } = require("../models/users");
const { retrieveDiscordUsers, fetchUsersForKeyValues } = require("../services/dataAccessLayer");
const { EXTERNAL_ACCOUNTS_POST_ACTIONS } = require("../constants/external-accounts");
const discordServices = require("../services/discordService");
const removeDiscordRoleUtils = require("../utils/removeDiscordRoleFromUser");
const config = require("config");
const logger = require("../utils/logger");
const { markUnDoneTasksOfArchivedUsersBacklog } = require("../models/tasks");
Expand Down Expand Up @@ -71,14 +71,16 @@ const linkExternalAccount = async (req, res) => {
userId
);

try {
const unverifiedRoleId = config.get("discordUnverifiedRoleId");
await discordServices.removeRoleFromUser(unverifiedRoleId, attributes.discordId, req.userData);
} catch (error) {
logger.error(`Error getting external account data: ${error}`);
return res.boom.internal("Role Deletion failed. Please contact admin.", {
message: "Role Deletion failed. Please contact admin.",
});
const unverifiedRoleId = config.get("discordUnverifiedRoleId");
const unverifiedRoleRemovalResponse = await removeDiscordRoleUtils.removeDiscordRoleFromUser(
req.userData,
attributes.discordId,
unverifiedRoleId
);

if (!unverifiedRoleRemovalResponse.success) {
const message = `User details updated but ${unverifiedRoleRemovalResponse.message}. Please contact admin`;
return res.boom.internal(message, { message });
}

return res.status(204).json({ message: "Your discord profile has been linked successfully" });
Expand Down
2 changes: 1 addition & 1 deletion controllers/invites.ts
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ export const createInviteLink = async (req: InviteBodyRequest, res: CustomRespon
type: logType.DISCORD_INVITES,
meta: {
action: "create",
createdBy: logType.EXTERNAL_SERVICE,
userId: logType.EXTERNAL_SERVICE,
createdAt: Date.now(),
},
body: {
Expand Down
12 changes: 12 additions & 0 deletions controllers/logs.js
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,19 @@ const fetchAllLogs = async (req, res) => {
}
};

const updateLogs = async (req, res) => {
try {
const response = await logsQuery.updateLogs();
return res.json({
response,
});
} catch (error) {
return res.boom.serverUnavailable(SOMETHING_WENT_WRONG);
}
};

module.exports = {
fetchLogs,
fetchAllLogs,
updateLogs,
};
4 changes: 2 additions & 2 deletions controllers/oooRequests.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ export const createOooRequestController = async (req: OooRequestCreateRequest, r
meta: {
requestId: requestResult.id,
action: LOG_ACTION.CREATE,
createdBy: userId,
userId: userId,
createdAt: Date.now(),
},
body: requestResult,
Expand Down Expand Up @@ -84,7 +84,7 @@ export const updateOooRequestController = async (req: UpdateRequest, res: Custom
meta: {
requestId: requestId,
action: LOG_ACTION.UPDATE,
createdBy: userId,
userId: userId,
createdAt: Date.now(),
},
body: requestResult,
Expand Down
4 changes: 2 additions & 2 deletions controllers/taskRequestsv2.ts
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ export const createTaskRequestController = async (req: TaskRequestRequest, res:
meta: {
taskRequestId: updatedRequest.id,
action: "update",
createdBy: req.userData.id,
userId: req.userData.id,
createdAt: Date.now(),
lastModifiedBy: req.userData.id,
lastModifiedAt: Date.now(),
Expand Down Expand Up @@ -150,7 +150,7 @@ export const createTaskRequestController = async (req: TaskRequestRequest, res:
meta: {
taskRequestId: newTaskRequest.id,
action: "create",
createdBy: req.userData.id,
userId: req.userData.id,
createdAt: Date.now(),
lastModifiedBy: req.userData.id,
lastModifiedAt: Date.now(),
Expand Down
37 changes: 36 additions & 1 deletion controllers/tasks.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ const { updateUserStatusOnTaskUpdate, updateStatusOnTaskCompletion } = require("
const dataAccess = require("../services/dataAccessLayer");
const { parseSearchQuery } = require("../utils/tasks");
const { addTaskCreatedAtAndUpdatedAtFields } = require("../services/tasks");
const tasksService = require("../services/tasks");
const { RQLQueryParser } = require("../utils/RQLParser");
const { getMissedProgressUpdatesUsers } = require("../models/discordactions");
const { logType } = require("../constants/logs");
Expand Down Expand Up @@ -134,7 +135,19 @@ const fetchPaginatedTasks = async (query) => {

const fetchTasks = async (req, res) => {
try {
const { status, page, size, prev, next, q: queryString, assignee, title, userFeatureFlag } = req.query;
const {
status,
page,
size,
prev,
next,
q: queryString,
assignee,
title,
userFeatureFlag,
orphaned,
dev,
} = req.query;
const transformedQuery = transformQuery(status, size, page, assignee, title);

if (queryString !== undefined) {
Expand All @@ -159,6 +172,28 @@ const fetchTasks = async (req, res) => {
});
}

const isOrphaned = orphaned === "true";
const isDev = dev === "true";
if (isOrphaned) {
if (!isDev) {
return res.boom.notFound("Route not found");
}
try {
const orphanedTasks = await tasksService.fetchOrphanedTasks();
if (!orphanedTasks || orphanedTasks.length === 0) {
return res.sendStatus(204);
}
const tasksWithRdsAssigneeInfo = await fetchTasksWithRdsAssigneeInfo(orphanedTasks);
return res.status(200).json({
message: "Orphan tasks fetched successfully",
data: tasksWithRdsAssigneeInfo,
});
} catch (error) {
logger.error("Error in getting tasks which were orphaned", error);
return res.boom.badImplementation(INTERNAL_SERVER_ERROR);
}
}

const paginatedTasks = await fetchPaginatedTasks({ ...transformedQuery, prev, next, userFeatureFlag });
return res.json({
message: "Tasks returned successfully!",
Expand Down
4 changes: 2 additions & 2 deletions controllers/tasksRequests.js
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ const addTaskRequests = async (req, res) => {
meta: {
taskRequestId: newTaskRequest.id,
action: "create",
createdBy: req.userData.id,
userId: req.userData.id,
createdAt: Date.now(),
lastModifiedBy: req.userData.id,
lastModifiedAt: Date.now(),
Expand Down Expand Up @@ -206,7 +206,7 @@ const updateTaskRequests = async (req, res) => {
taskRequestId: taskRequestId,
action: "update",
subAction: action,
createdBy: req.userData.id,
userId: req.userData.id,
createdAt: Date.now(),
lastModifiedBy: req.userData.id,
lastModifiedAt: Date.now(),
Expand Down
72 changes: 68 additions & 4 deletions controllers/users.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
const chaincodeQuery = require("../models/chaincodes");
const userQuery = require("../models/users");
const profileDiffsQuery = require("../models/profileDiffs");
const firestore = require("../utils/firestore");
const memberRoleModel = firestore.collection("member-group-roles");
const logsModel = firestore.collection("logs");
const admin = require("firebase-admin");
const logsQuery = require("../models/logs");
const imageService = require("../services/imageService");
const { profileDiffStatus } = require("../constants/profileDiff");
Expand Down Expand Up @@ -30,6 +34,7 @@ const { addLog } = require("../models/logs");
const { getUserStatus } = require("../models/userStatus");
const config = require("config");
const { generateUniqueUsername } = require("../services/users");
const userService = require("../services/users");
const discordDeveloperRoleId = config.get("discordDeveloperRoleId");

const verifyUser = async (req, res) => {
Expand Down Expand Up @@ -191,6 +196,30 @@ const getUsers = async (req, res) => {
}
}

const isDeparted = req.query.departed === "true";

if (isDeparted) {
if (!dev) {
return res.boom.notFound("Route not found");
}
try {
const result = await dataAccess.retrieveUsers({ query: req.query });
const departedUsers = await userService.getUsersWithIncompleteTasks(result.users);
if (departedUsers.length === 0) return res.status(204).send();
return res.json({
message: "Users with abandoned tasks fetched successfully",
users: departedUsers,
links: {
next: result.nextId ? getPaginationLink(req.query, "next", result.nextId) : "",
prev: result.prevId ? getPaginationLink(req.query, "prev", result.prevId) : "",
},
});
} catch (error) {
logger.error("Error when fetching users who abandoned tasks:", error);
return res.boom.badImplementation(INTERNAL_SERVER_ERROR);
}
}

if (transformedQuery?.filterBy === OVERDUE_TASKS) {
try {
const tasksData = await getOverdueTasks(days);
Expand Down Expand Up @@ -595,7 +624,7 @@ const markUnverified = async (req, res) => {
const unverifiedRoleId = config.get("discordUnverifiedRoleId");
const usersToApplyUnverifiedRole = [];
const addRolePromises = [];
const discordDeveloperRoleId = config.get("discordDeveloperRoleId");
const batchPromises = [];

allRdsLoggedInUsers.forEach((user) => {
rdsUserMap[user.discordId] = true;
Expand All @@ -611,11 +640,40 @@ const markUnverified = async (req, res) => {
}
});

const batchSize = 500;
const batches = Array.from({ length: Math.ceil(usersToApplyUnverifiedRole.length / batchSize) }, (_, index) =>
usersToApplyUnverifiedRole.slice(index * batchSize, index * batchSize + batchSize)
);

batches.forEach((batch) => {
const firestoreBatch = firestore.batch();

batch.forEach((id) => {
const memberRoleRef = memberRoleModel.doc(id);
const logRef = logsModel.doc();

firestoreBatch.set(memberRoleRef, {
roleid: unverifiedRoleId,
userid: id,
date: admin.firestore.Timestamp.fromDate(new Date()),
});

firestoreBatch.set(logRef, {
type: logType.ADD_UNVERIFIED_ROLE,
meta: { roleid: unverifiedRoleId, userid: id },
body: { message: "Unverified role added successfully" },
timestamp: admin.firestore.Timestamp.fromDate(new Date()),
});
});

batchPromises.push(firestoreBatch.commit());
});

usersToApplyUnverifiedRole.forEach((id) => {
addRolePromises.push(addRoleToUser(id, unverifiedRoleId));
});

await Promise.all(addRolePromises);
await Promise.all([...addRolePromises, ...batchPromises]);
return res.json({ message: "ROLES APPLIED SUCCESSFULLY" });
} catch (err) {
logger.error(err);
Expand Down Expand Up @@ -653,8 +711,14 @@ const getUserImageForVerification = async (req, res) => {
const updateUser = async (req, res) => {
try {
const { id: profileDiffId, message } = req.body;

const profileDiffData = await profileDiffsQuery.fetchProfileDiff(profileDiffId);
const devFeatureFlag = req.query.dev;
let profileDiffData;
if (devFeatureFlag === "true") {
profileDiffData = await profileDiffsQuery.fetchProfileDiffUnobfuscated(profileDiffId);
} else {
profileDiffData = await profileDiffsQuery.fetchProfileDiff(profileDiffId);
}
Object.freeze(profileDiffData);
if (!profileDiffData) return res.boom.notFound("Profile Diff doesn't exist");

const { approval, timestamp, userId, ...profileDiff } = profileDiffData;
Expand Down
1 change: 1 addition & 0 deletions middlewares/validators/tasks.js
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,7 @@ const getTasksValidator = async (req, res, next) => {
return value;
}, "Invalid query format"),
userFeatureFlag: joi.string().optional(),
orphaned: joi.boolean().optional(),
});

try {
Expand Down
1 change: 1 addition & 0 deletions middlewares/validators/user.js
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,7 @@ async function getUsers(req, res, next) {
filterBy: joi.string().optional(),
days: joi.string().optional(),
dev: joi.string().optional(),
departed: joi.string().optional(),
roles: joi.optional().custom((value, helpers) => {
if (value !== "member") {
return helpers.message("only member role is supported");
Expand Down
2 changes: 1 addition & 1 deletion models/discordactions.js
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@
*
* @param options { Object }: Data of the new role
* @param options.rolename String : name of the role
* @param options.roleId String : id of the role
* @param options.roleid String : id of the role
* @returns {Promise<discordRoleModel|Object>}
*/

Expand Down Expand Up @@ -523,7 +523,7 @@

for (let i = 0; i < nicknameUpdateBatches.length; i++) {
const promises = [];
const usersStatusDocsBatch = nicknameUpdateBatches[i];

Check warning on line 526 in models/discordactions.js

View workflow job for this annotation

GitHub Actions / build (20.11.x)

Variable Assigned to Object Injection Sink
usersStatusDocsBatch.forEach((document) => {
const doc = document.data();
const userId = doc.userId;
Expand Down
Loading
Loading