Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize metadata queries #7013

Merged
merged 17 commits into from
Sep 13, 2024
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,12 @@ import { WorkspaceSchemaFactory } from 'src/engine/api/graphql/workspace-schema.
import { TokenService } from 'src/engine/core-modules/auth/services/token.service';
import { AuthContext } from 'src/engine/core-modules/auth/types/auth-context.type';
import { CoreEngineModule } from 'src/engine/core-modules/core-engine.module';
import { useGraphQLErrorHandlerHook } from 'src/engine/core-modules/graphql/hooks/use-graphql-error-handler.hook';
import { User } from 'src/engine/core-modules/user/user.entity';
import { Workspace } from 'src/engine/core-modules/workspace/workspace.entity';
import { EnvironmentService } from 'src/engine/core-modules/environment/environment.service';
import { ExceptionHandlerService } from 'src/engine/core-modules/exception-handler/exception-handler.service';
import { useSentryTracing } from 'src/engine/core-modules/exception-handler/hooks/use-sentry-tracing';
import { useGraphQLErrorHandlerHook } from 'src/engine/core-modules/graphql/hooks/use-graphql-error-handler.hook';
import { User } from 'src/engine/core-modules/user/user.entity';
import { Workspace } from 'src/engine/core-modules/workspace/workspace.entity';
import { handleExceptionAndConvertToGraphQLError } from 'src/engine/utils/global-exception-handler.util';
import { renderApolloPlayground } from 'src/engine/utils/render-apollo-playground.util';

Expand Down Expand Up @@ -69,13 +69,18 @@ export class GraphQLConfigService
let workspace: Workspace | undefined;

try {
if (!this.tokenService.isTokenPresent(context.req)) {
const { user, workspace, apiKey, workspaceMemberId } = context.req;

if (!workspace) {
return new GraphQLSchema({});
}

const data = await this.tokenService.validateToken(context.req);

return await this.createSchema(context, data);
return await this.createSchema(context, {
user,
workspace,
apiKey,
workspaceMemberId,
});
} catch (error) {
if (error instanceof UnauthorizedException) {
throw new GraphQLError('Unauthenticated', {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import { FindOptionsWhere, ObjectLiteral } from 'typeorm';

import { RecordFilter } from 'src/engine/api/graphql/workspace-query-builder/interfaces/record.interface';

import { FieldMetadataMap } from 'src/engine/api/graphql/graphql-query-runner/utils/convert-object-metadata-to-map.util';
import { FieldMetadataMap } from 'src/engine/api/graphql/graphql-query-runner/utils/get-object-metadata.util';

import { GraphqlQueryFilterFieldParser } from './graphql-query-filter-field.parser';

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import { FindOptionsWhere, Not, ObjectLiteral } from 'typeorm';
import { RecordFilter } from 'src/engine/api/graphql/workspace-query-builder/interfaces/record.interface';
import { FieldMetadataInterface } from 'src/engine/metadata-modules/field-metadata/interfaces/field-metadata.interface';

import { FieldMetadataMap } from 'src/engine/api/graphql/graphql-query-runner/utils/convert-object-metadata-to-map.util';
import { FieldMetadataMap } from 'src/engine/api/graphql/graphql-query-runner/utils/get-object-metadata.util';
import { compositeTypeDefinitions } from 'src/engine/metadata-modules/field-metadata/composite-types';
import { isCompositeFieldMetadataType } from 'src/engine/metadata-modules/field-metadata/utils/is-composite-field-metadata-type.util';
import { CompositeFieldMetadataType } from 'src/engine/metadata-modules/workspace-migration/factories/composite-column-action.factory';
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { OrderByDirection } from 'src/engine/api/graphql/workspace-query-builder/interfaces/record.interface';

import { GraphqlQueryOrderFieldParser } from 'src/engine/api/graphql/graphql-query-runner/graphql-query-parsers/graphql-query-order/graphql-query-order.parser';
import { FieldMetadataMap } from 'src/engine/api/graphql/graphql-query-runner/utils/convert-object-metadata-to-map.util';
import { FieldMetadataMap } from 'src/engine/api/graphql/graphql-query-runner/utils/get-object-metadata.util';
import { FieldMetadataType } from 'src/engine/metadata-modules/field-metadata/field-metadata.entity';

describe('GraphqlQueryOrderFieldParser', () => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import {
GraphqlQueryRunnerException,
GraphqlQueryRunnerExceptionCode,
} from 'src/engine/api/graphql/graphql-query-runner/errors/graphql-query-runner.exception';
import { FieldMetadataMap } from 'src/engine/api/graphql/graphql-query-runner/utils/convert-object-metadata-to-map.util';
import { FieldMetadataMap } from 'src/engine/api/graphql/graphql-query-runner/utils/get-object-metadata.util';
import { compositeTypeDefinitions } from 'src/engine/metadata-modules/field-metadata/composite-types';
import { isCompositeFieldMetadataType } from 'src/engine/metadata-modules/field-metadata/utils/is-composite-field-metadata-type.util';
import { CompositeFieldMetadataType } from 'src/engine/metadata-modules/workspace-migration/factories/composite-column-action.factory';
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { FieldMetadataInterface } from 'src/engine/metadata-modules/field-metadata/interfaces/field-metadata.interface';

import { GraphqlQuerySelectedFieldsParser } from 'src/engine/api/graphql/graphql-query-runner/graphql-query-parsers/graphql-query-selected-fields/graphql-selected-fields.parser';
import { ObjectMetadataMap } from 'src/engine/api/graphql/graphql-query-runner/utils/convert-object-metadata-to-map.util';
import { ObjectMetadataMap } from 'src/engine/api/graphql/graphql-query-runner/utils/get-object-metadata.util';
import { getRelationObjectMetadata } from 'src/engine/api/graphql/graphql-query-runner/utils/get-relation-object-metadata.util';

export class GraphqlQuerySelectedFieldsRelationParser {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import {
GraphqlQueryRunnerExceptionCode,
} from 'src/engine/api/graphql/graphql-query-runner/errors/graphql-query-runner.exception';
import { GraphqlQuerySelectedFieldsRelationParser } from 'src/engine/api/graphql/graphql-query-runner/graphql-query-parsers/graphql-query-selected-fields/graphql-selected-fields-relation.parser';
import { ObjectMetadataMap } from 'src/engine/api/graphql/graphql-query-runner/utils/convert-object-metadata-to-map.util';
import { ObjectMetadataMap } from 'src/engine/api/graphql/graphql-query-runner/utils/get-object-metadata.util';
import { compositeTypeDefinitions } from 'src/engine/metadata-modules/field-metadata/composite-types';
import { isCompositeFieldMetadataType } from 'src/engine/metadata-modules/field-metadata/utils/is-composite-field-metadata-type.util';
import { CompositeFieldMetadataType } from 'src/engine/metadata-modules/workspace-migration/factories/composite-column-action.factory';
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ import { GraphqlQuerySelectedFieldsParser } from 'src/engine/api/graphql/graphql
import {
FieldMetadataMap,
ObjectMetadataMap,
} from 'src/engine/api/graphql/graphql-query-runner/utils/convert-object-metadata-to-map.util';
} from 'src/engine/api/graphql/graphql-query-runner/utils/get-object-metadata.util';

export class GraphqlQueryParser {
private fieldMetadataMap: FieldMetadataMap;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ import {
GraphqlQueryRunnerException,
GraphqlQueryRunnerExceptionCode,
} from 'src/engine/api/graphql/graphql-query-runner/errors/graphql-query-runner.exception';
import { ObjectMetadataMap } from 'src/engine/api/graphql/graphql-query-runner/utils/convert-object-metadata-to-map.util';
import { encodeCursor } from 'src/engine/api/graphql/graphql-query-runner/utils/cursors.util';
import { ObjectMetadataMap } from 'src/engine/api/graphql/graphql-query-runner/utils/get-object-metadata.util';
import { getRelationObjectMetadata } from 'src/engine/api/graphql/graphql-query-runner/utils/get-relation-object-metadata.util';
import { compositeTypeDefinitions } from 'src/engine/metadata-modules/field-metadata/composite-types';
import { FieldMetadataType } from 'src/engine/metadata-modules/field-metadata/field-metadata.entity';
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,11 @@ import {
import { GraphqlQueryParser } from 'src/engine/api/graphql/graphql-query-runner/graphql-query-parsers/graphql-query.parser';
import { ObjectRecordsToGraphqlConnectionMapper } from 'src/engine/api/graphql/graphql-query-runner/orm-mappers/object-records-to-graphql-connection.mapper';
import { applyRangeFilter } from 'src/engine/api/graphql/graphql-query-runner/utils/apply-range-filter.util';
import { decodeCursor } from 'src/engine/api/graphql/graphql-query-runner/utils/cursors.util';
import {
convertObjectMetadataToMap,
getObjectMetadata,
} from 'src/engine/api/graphql/graphql-query-runner/utils/convert-object-metadata-to-map.util';
import { decodeCursor } from 'src/engine/api/graphql/graphql-query-runner/utils/cursors.util';
} from 'src/engine/api/graphql/graphql-query-runner/utils/get-object-metadata.util';
import { TwentyORMGlobalManager } from 'src/engine/twenty-orm/twenty-orm-global.manager';

export class GraphqlQueryFindManyResolverService {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,8 @@ import {
} from 'src/engine/api/graphql/graphql-query-runner/errors/graphql-query-runner.exception';
import { GraphqlQueryParser } from 'src/engine/api/graphql/graphql-query-runner/graphql-query-parsers/graphql-query.parser';
import { ObjectRecordsToGraphqlConnectionMapper } from 'src/engine/api/graphql/graphql-query-runner/orm-mappers/object-records-to-graphql-connection.mapper';
import {
convertObjectMetadataToMap,
getObjectMetadata,
} from 'src/engine/api/graphql/graphql-query-runner/utils/convert-object-metadata-to-map.util';
import { getObjectMetadata } from 'src/engine/api/graphql/graphql-query-runner/utils/get-object-metadata.util';
import { generateObjectMetadataMap } from 'src/engine/metadata-modules/utils/generate-object-metadata-map.util';
import { TwentyORMGlobalManager } from 'src/engine/twenty-orm/twenty-orm-global.manager';

export class GraphqlQueryFindOneResolverService {
Expand All @@ -40,9 +38,10 @@ export class GraphqlQueryFindOneResolverService {
authContext.workspace.id,
objectMetadataItem.nameSingular,
);
const objectMetadataMap = convertObjectMetadataToMap(
const objectMetadataMap = generateObjectMetadataMap(
objectMetadataCollection,
);

const objectMetadata = getObjectMetadata(
objectMetadataMap,
objectMetadataItem.nameSingular,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
import {
GraphqlQueryRunnerException,
GraphqlQueryRunnerExceptionCode,
} from 'src/engine/api/graphql/graphql-query-runner/errors/graphql-query-runner.exception';
import { ObjectMetadataMapItem } from 'src/engine/metadata-modules/utils/generate-object-metadata-map.util';

export const getObjectMetadata = (
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OrThrow?

objectMetadataMap: Record<string, any>,
objectName: string,
): ObjectMetadataMapItem => {
const objectMetadata = objectMetadataMap[objectName];

if (!objectMetadata) {
throw new GraphqlQueryRunnerException(
`Object metadata not found for ${objectName}`,
GraphqlQueryRunnerExceptionCode.OBJECT_METADATA_NOT_FOUND,
);
}

return objectMetadata;
};
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { FieldMetadataInterface } from 'src/engine/metadata-modules/field-metadata/interfaces/field-metadata.interface';

import { ObjectMetadataMap } from 'src/engine/api/graphql/graphql-query-runner/utils/convert-object-metadata-to-map.util';
import { ObjectMetadataMap } from 'src/engine/api/graphql/graphql-query-runner/utils/get-object-metadata.util';
import {
deduceRelationDirection,
RelationDirection,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,13 +56,13 @@ export class WorkspaceSchemaFactory {
);
}

const objectMetadataCollection =
await this.workspaceCacheStorageService.getObjectMetadataCollection(
const objectMetadataMap =
await this.workspaceCacheStorageService.getObjectMetadataMap(
authContext.workspace.id,
currentCacheVersion,
);

if (!objectMetadataCollection) {
if (!objectMetadataMap) {
await this.workspaceMetadataCacheService.recomputeMetadataCache(
authContext.workspace.id,
);
Expand All @@ -72,6 +72,13 @@ export class WorkspaceSchemaFactory {
);
}

const objectMetadataCollection = Object.values(objectMetadataMap).map(
(objectMetadataItem) => ({
...objectMetadataItem,
fields: Object.values(objectMetadataItem.fields),
}),
);

// Get typeDefs from cache
let typeDefs = await this.workspaceCacheStorageService.getGraphQLTypeDefs(
authContext.workspace.id,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@ import { Request, Response } from 'express';

import { RestApiCoreService } from 'src/engine/api/rest/core/rest-api-core.service';
import { cleanGraphQLResponse } from 'src/engine/api/rest/utils/clean-graphql-response.utils';
import { JwtAuthGuard } from 'src/engine/guards/jwt.auth.guard';
import { WorkspaceAuthGuard } from 'src/engine/guards/workspace-auth.guard';

@Controller('rest/batch/*')
@UseGuards(JwtAuthGuard)
@UseGuards(WorkspaceAuthGuard)
export class RestApiCoreBatchController {
constructor(private readonly restApiCoreService: RestApiCoreService) {}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@ import { Request, Response } from 'express';

import { RestApiCoreService } from 'src/engine/api/rest/core/rest-api-core.service';
import { cleanGraphQLResponse } from 'src/engine/api/rest/utils/clean-graphql-response.utils';
import { JwtAuthGuard } from 'src/engine/guards/jwt.auth.guard';
import { WorkspaceAuthGuard } from 'src/engine/guards/workspace-auth.guard';

@Controller('rest/*')
@UseGuards(JwtAuthGuard)
@UseGuards(WorkspaceAuthGuard)
export class RestApiCoreController {
constructor(private readonly restApiCoreService: RestApiCoreService) {}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,16 @@ import { User } from 'src/engine/core-modules/user/user.entity';
import { Workspace } from 'src/engine/core-modules/workspace/workspace.entity';
import { AuthUser } from 'src/engine/decorators/auth/auth-user.decorator';
import { AuthWorkspace } from 'src/engine/decorators/auth/auth-workspace.decorator';
import { JwtAuthGuard } from 'src/engine/guards/jwt.auth.guard';
import { UserAuthGuard } from 'src/engine/guards/user-auth.guard copy';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

syntax: Typo in the import path. Remove 'copy' from the end.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

import { WorkspaceAuthGuard } from 'src/engine/guards/workspace-auth.guard';

@ArgsType()
class GetAISQLQueryArgs {
@Field(() => String)
text: string;
}

@UseGuards(JwtAuthGuard)
@UseGuards(WorkspaceAuthGuard, UserAuthGuard)
@Resolver(() => AISQLQueryResult)
export class AISQLQueryResolver {
constructor(
Expand Down
Original file line number Diff line number Diff line change
@@ -1,21 +1,18 @@
import { Resolver, Mutation, Args, Context } from '@nestjs/graphql';
import { UseGuards } from '@nestjs/common';
import { Args, Context, Mutation, Resolver } from '@nestjs/graphql';

import { Request } from 'express';

import { OptionalJwtAuthGuard } from 'src/engine/guards/optional-jwt.auth.guard';
import { AuthWorkspace } from 'src/engine/decorators/auth/auth-workspace.decorator';
import { AuthUser } from 'src/engine/decorators/auth/auth-user.decorator';
import { Workspace } from 'src/engine/core-modules/workspace/workspace.entity';
import { User } from 'src/engine/core-modules/user/user.entity';
import { EnvironmentService } from 'src/engine/core-modules/environment/environment.service';
import { User } from 'src/engine/core-modules/user/user.entity';
import { Workspace } from 'src/engine/core-modules/workspace/workspace.entity';
import { AuthUser } from 'src/engine/decorators/auth/auth-user.decorator';
import { AuthWorkspace } from 'src/engine/decorators/auth/auth-workspace.decorator';

import { AnalyticsService } from './analytics.service';
import { Analytics } from './analytics.entity';
import { AnalyticsService } from './analytics.service';

import { CreateAnalyticsInput } from './dtos/create-analytics.input';

@UseGuards(OptionalJwtAuthGuard)
@Resolver(() => Analytics)
export class AnalyticsResolver {
constructor(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import {

import { AppToken } from 'src/engine/core-modules/app-token/app-token.entity';
import { CreateAppTokenInput } from 'src/engine/core-modules/app-token/dtos/create-app-token.input';
import { JwtAuthGuard } from 'src/engine/guards/jwt.auth.guard';
import { WorkspaceAuthGuard } from 'src/engine/guards/workspace-auth.guard';

export const appTokenAutoResolverOpts: AutoResolverOpts<
any,
Expand Down Expand Up @@ -34,6 +34,6 @@ export const appTokenAutoResolverOpts: AutoResolverOpts<
one: { disabled: true },
},
delete: { many: { disabled: true }, one: { disabled: true } },
guards: [JwtAuthGuard],
guards: [WorkspaceAuthGuard],
},
];
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ export class BeforeCreateOneAppToken<T extends AppToken>
instance: CreateOneInputType<T>,
context: any,
): Promise<CreateOneInputType<T>> {
const userId = context?.req?.user?.user?.id;
const userId = context?.req?.user?.id;

instance.input.userId = userId;
// FIXME: These fields should be autogenerated, we need to run a migration for this
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,14 @@ import { UpdatePasswordViaResetTokenInput } from 'src/engine/core-modules/auth/d
import { ValidatePasswordResetToken } from 'src/engine/core-modules/auth/dto/validate-password-reset-token.entity';
import { ValidatePasswordResetTokenInput } from 'src/engine/core-modules/auth/dto/validate-password-reset-token.input';
import { AuthGraphqlApiExceptionFilter } from 'src/engine/core-modules/auth/filters/auth-graphql-api-exception.filter';
import { CaptchaGuard } from 'src/engine/core-modules/captcha/captcha.guard';
import { UserService } from 'src/engine/core-modules/user/services/user.service';
import { User } from 'src/engine/core-modules/user/user.entity';
import { Workspace } from 'src/engine/core-modules/workspace/workspace.entity';
import { AuthUser } from 'src/engine/decorators/auth/auth-user.decorator';
import { AuthWorkspace } from 'src/engine/decorators/auth/auth-workspace.decorator';
import { JwtAuthGuard } from 'src/engine/guards/jwt.auth.guard';
import { CaptchaGuard } from 'src/engine/core-modules/captcha/captcha.guard';
import { UserAuthGuard } from 'src/engine/guards/user-auth.guard copy';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

syntax: Typo in import path. Remove 'copy' from the end of the file name.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

import { WorkspaceAuthGuard } from 'src/engine/guards/workspace-auth.guard';

import { ChallengeInput } from './dto/challenge.input';
import { ImpersonateInput } from './dto/impersonate.input';
Expand Down Expand Up @@ -111,7 +112,7 @@ export class AuthResolver {
}

@Mutation(() => TransientToken)
@UseGuards(JwtAuthGuard)
@UseGuards(WorkspaceAuthGuard, UserAuthGuard)
async generateTransientToken(
@AuthUser() user: User,
): Promise<TransientToken | void> {
Expand Down Expand Up @@ -141,7 +142,7 @@ export class AuthResolver {
}

@Mutation(() => AuthorizeApp)
@UseGuards(JwtAuthGuard)
@UseGuards(WorkspaceAuthGuard, UserAuthGuard)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic: Similar guard change here. Verify that WorkspaceAuthGuard and UserAuthGuard provide the same level of security as JwtAuthGuard.

async authorizeApp(
@Args() authorizeAppInput: AuthorizeAppInput,
@AuthUser() user: User,
Expand All @@ -155,7 +156,7 @@ export class AuthResolver {
}

@Mutation(() => AuthTokens)
@UseGuards(JwtAuthGuard)
@UseGuards(WorkspaceAuthGuard, UserAuthGuard)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic: Another instance of guard replacement. Test thoroughly to ensure all affected endpoints still function correctly.

async generateJWT(
@AuthUser() user: User,
@Args() args: GenerateJwtInput,
Expand All @@ -177,7 +178,7 @@ export class AuthResolver {
return { tokens: tokens };
}

@UseGuards(JwtAuthGuard)
@UseGuards(WorkspaceAuthGuard, UserAuthGuard)
@Mutation(() => Verify)
async impersonate(
@Args() impersonateInput: ImpersonateInput,
Expand All @@ -186,7 +187,7 @@ export class AuthResolver {
return await this.authService.impersonate(impersonateInput.userId, user);
}

@UseGuards(JwtAuthGuard)
@UseGuards(WorkspaceAuthGuard)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic: Only WorkspaceAuthGuard is used here. Confirm if UserAuthGuard is intentionally omitted for this mutation.

@Mutation(() => ApiKeyToken)
async generateApiKeyToken(
@Args() args: ApiKeyTokenInput,
Expand Down
Loading
Loading