Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EW-694 Common Cartridge Course Import #4700

Merged
merged 61 commits into from
Feb 23, 2024
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
8aec9b7
EW-694 adding a new package and a common cartridge for testing
psachmann Jan 11, 2024
c99d7a8
EW-694 adding parsing for common cartridge file
psachmann Jan 11, 2024
413599a
EW-694 code coverage
psachmann Jan 11, 2024
f44841c
EW-694 creating course import endpoint
psachmann Jan 16, 2024
20c9cf5
EW-694 adding new feature flag for common cartridge course import
psachmann Jan 17, 2024
fe99609
EW-694 adding tests and improving coverage
psachmann Jan 18, 2024
12632fe
Merge branch 'main' into EW-694
psachmann Jan 19, 2024
b17f7a3
EW-694 comment update
psachmann Jan 22, 2024
d32bff5
Merge branch 'main' into EW-694
psachmann Jan 22, 2024
89abf23
EW-694 skipping lib check
psachmann Jan 22, 2024
83466a3
EW-694 finishing touches for the server
psachmann Jan 23, 2024
1710d81
EW-694 updating the regex for the zip mime type
psachmann Jan 25, 2024
18a80ca
Merge branch 'main' into EW-694
psachmann Jan 26, 2024
24c207a
Merge branch 'main' into EW-694
psachmann Jan 29, 2024
f693c07
EW-694 fixing test coverage
psachmann Jan 29, 2024
e007993
EW-694 adding a custom exception and renaming a field
psachmann Jan 29, 2024
fb1b1a0
EW-694 some renaming
psachmann Jan 29, 2024
d78cadf
EW-694 updating tsconfig.ts
psachmann Jan 30, 2024
c98c998
EW-694 renaming importCourse to createCourse
psachmann Jan 30, 2024
27a7591
Merge branch 'main' into EW-694
psachmann Jan 30, 2024
defe983
EW-694 updating package-lock.json
psachmann Jan 30, 2024
fed8215
EW-694 updating new tsconfig.json
psachmann Jan 30, 2024
27d8c1f
Merge branch 'main' into EW-694
psachmann Jan 30, 2024
f71b8a8
Merge branch 'main' into EW-694
psachmann Jan 31, 2024
bb4c9df
Merge branch 'main' into EW-694
psachmann Feb 1, 2024
2a9155c
EW-694 removing changes from tsconfig.json
psachmann Jan 31, 2024
0d28845
EW-694 adding comment
psachmann Feb 7, 2024
4cfbb9a
EW-694 fixing typos
psachmann Feb 8, 2024
ea5e52b
EW-694 restructuring config and validation
psachmann Feb 8, 2024
94c13f4
EW-694 refactoring
psachmann Feb 9, 2024
810d83a
EW-694 adding antivirus scan to cc course import
psachmann Feb 9, 2024
973a7c2
EW-694 removing antivirus scan
psachmann Feb 13, 2024
93e177c
Merge branch 'main' into EW-694
psachmann Feb 14, 2024
d67efc8
Merge branch 'main' into EW-694
psachmann Feb 14, 2024
377e198
EW-694 updating new dependency
psachmann Feb 14, 2024
6a158b6
Merge branch 'main' into EW-694
psachmann Feb 15, 2024
3c46674
EW-694 skipping lib check
psachmann Feb 15, 2024
01ad220
Merge branch 'main' into EW-694
psachmann Feb 15, 2024
f5b2ad9
EW-694 removing regex check during file upload
psachmann Feb 15, 2024
ed49f19
EW-694 fixing linter errors
psachmann Feb 15, 2024
b7a1887
EW-694 updating feature flag description
psachmann Feb 19, 2024
f9f1ee3
EW-694 improving test structure
psachmann Feb 19, 2024
d4928ef
EW-694 reverting lock file version from 3 to 2
psachmann Feb 19, 2024
178016e
Merge branch 'main' into EW-694
psachmann Feb 19, 2024
d9d4929
E-694 removing skipLibCheck
psachmann Feb 19, 2024
f799a24
EW-694 adding definitions for jsdom
psachmann Feb 19, 2024
777f2d2
Merge branch 'main' into EW-694
psachmann Feb 20, 2024
fec4d91
EW-694 fixing tests
psachmann Feb 20, 2024
1220958
EW-694 working on coverage
psachmann Feb 20, 2024
7d12916
EW-694 fixing return types
psachmann Feb 21, 2024
a283272
Merge branch 'main' into EW-694
psachmann Feb 21, 2024
1419ba8
EW-694 fixing tests and linter issues
psachmann Feb 21, 2024
38c407c
Merge branch 'main' into EW-694
psachmann Feb 21, 2024
c0dee5b
EW-694 fixing review comments
psachmann Feb 21, 2024
a3ae1e0
Merge branch 'main' into EW-694
psachmann Feb 21, 2024
9138e2e
Merge branch 'main' into EW-694
psachmann Feb 21, 2024
3e26ade
Merge branch 'main' into EW-694
psachmann Feb 21, 2024
f138478
EW-694 changing validator pipe
psachmann Feb 22, 2024
93a557c
EW-694 updating index file
psachmann Feb 22, 2024
2417000
EW-694 removed not needed test
psachmann Feb 22, 2024
c82ec5c
Merge branch 'main' into EW-694
psachmann Feb 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
import AdmZip from 'adm-zip';
import { CommonCartridgeFileParser } from './common-cartridge-file-parser';

describe('CommonCartridgeFileParser', () => {
describe('constructor', () => {
describe('when manifest file is found', () => {
const setup = (manifestName: string) => {
const archive = new AdmZip();

archive.addFile(manifestName, Buffer.from('<manifest></manifest>'));

const file = archive.toBuffer();

return { file };
};

it('should use imsmanfiest.xml as manifest', () => {
psachmann marked this conversation as resolved.
Show resolved Hide resolved
const { file } = setup('imsmanifest.xml');
const parser = new CommonCartridgeFileParser(file);

expect(parser.manifest).toBeDefined();
});

it('should use manfiest.xml as manifest', () => {
psachmann marked this conversation as resolved.
Show resolved Hide resolved
const { file } = setup('manifest.xml');
const parser = new CommonCartridgeFileParser(file);

expect(parser.manifest).toBeDefined();
});
});

describe('when manifest file is not found', () => {
const setup = () => {
const archive = new AdmZip();
const file = archive.toBuffer();

return { file };
};

it('should throw', () => {
const { file } = setup();

expect(() => new CommonCartridgeFileParser(file)).toThrow('Manifest file not found');
});
});

describe('when file is not an archive', () => {
SimoneRadtke-Cap marked this conversation as resolved.
Show resolved Hide resolved
const setup = () => {
const file = new AdmZip().toBuffer();

return { file };
};

it('should throw', () => {
const { file } = setup();

expect(() => new CommonCartridgeFileParser(file)).toThrow('Manifest file not found');
});
});
});
});
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
import AdmZip from 'adm-zip';
import { CommonCartridgeManifestParser } from './common-cartridge-manifest-parser';

export class CommonCartridgeFileParser {
public readonly manifest: CommonCartridgeManifestParser;
Fshmit marked this conversation as resolved.
Show resolved Hide resolved

public constructor(file: Buffer) {
const archive = new AdmZip(file);

this.manifest = new CommonCartridgeManifestParser(this.getManifestFileAsString(archive));
}

private getManifestFileAsString(archive: AdmZip): string | never {
const manifest = archive.getEntry('imsmanifest.xml') ?? archive.getEntry('manifest.xml');

if (manifest) {
return archive.readAsText(manifest);
}

throw new Error('Manifest file not found');
Fshmit marked this conversation as resolved.
Show resolved Hide resolved
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
import AdmZip from 'adm-zip';
import { readFile } from 'fs/promises';
import { CommonCartridgeManifestParser } from './common-cartridge-manifest-parser';

describe('CommonCartridgeManifestParser', () => {
const setup = async () => {
const buffer = await readFile('./apps/server/test/assets/common-cartridge/us_history_since_1877.imscc');
const archive = new AdmZip(buffer);
const sut = new CommonCartridgeManifestParser(archive.readAsText('imsmanifest.xml'));

return { sut };
};

describe('getSchema', () => {
describe('when schema is present', () => {
it('should return the schema', async () => {
const { sut } = await setup();
const result = sut.getSchema();

expect(result).toBe('IMS Common Cartridge');
});
});
});

describe('getVersion', () => {
describe('when version is present', () => {
it('should return the version', async () => {
const { sut } = await setup();
const result = sut.getVersion();

expect(result).toBe('1.3.0');
});
});
});

describe('getTitle', () => {
describe('when title is present', () => {
it('should return the title', async () => {
const { sut } = await setup();
const result = sut.getTitle();

expect(result).toBe('201510-AMH-2020-70C-12218-US History Since 1877');
});
});
});
});
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
import { JSDOM } from 'jsdom';

export class CommonCartridgeManifestParser {
private readonly doc: Document;

public constructor(manifest: string) {
this.doc = new JSDOM(manifest).window.document;
}

public getSchema(): string | undefined {
const schema = this.doc.querySelector('manifest > metadata > schema');

return schema?.textContent || undefined;
}

public getVersion(): string | undefined {
const version = this.doc.querySelector('manifest > metadata > schemaversion');

return version?.textContent || undefined;
}

public getTitle(): string | undefined {
SimoneRadtke-Cap marked this conversation as resolved.
Show resolved Hide resolved
const title = this.doc.querySelector('manifest > metadata > lom > general > title > string');

return title?.textContent || undefined;
}
}
1 change: 1 addition & 0 deletions apps/server/src/modules/common-cartridge/import/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export * from './common-cartridge-file-parser';
SimoneRadtke-Cap marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import { INestApplication, StreamableFile } from '@nestjs/common';
import { Test, TestingModule } from '@nestjs/testing';
import { Permission } from '@shared/domain/interface';
import { TestApiClient, UserAndAccountTestFactory, cleanupCollections, courseFactory } from '@shared/testing';
import { readFile } from 'node:fs/promises';

const createStudent = () => {
const { studentUser, studentAccount } = UserAndAccountTestFactory.buildStudent({}, [Permission.COURSE_VIEW]);
Expand Down Expand Up @@ -51,6 +52,7 @@ describe('Course Controller (API)', () => {

return { student, course, teacher };
};

it('should find courses as student', async () => {
const { student, course } = setup();
await em.persistAndFlush([student.account, student.user, course]);
Expand All @@ -65,6 +67,7 @@ describe('Course Controller (API)', () => {
expect(data[0].startDate).toBe(course.startDate);
expect(data[0].untilDate).toBe(course.untilDate);
});

it('should find courses as teacher', async () => {
const { teacher, course } = setup();
await em.persistAndFlush([teacher.account, teacher.user, course]);
Expand Down Expand Up @@ -96,6 +99,7 @@ describe('Course Controller (API)', () => {

return { course, teacher, teacherUnkownToCourse, substitutionTeacher, student1 };
};

it('should find course export', async () => {
const { teacher, course } = setup();
await em.persistAndFlush([teacher.account, teacher.user, course]);
Expand All @@ -114,4 +118,24 @@ describe('Course Controller (API)', () => {
expect(response.header['content-disposition']).toBe('attachment;');
});
});

describe('[POST] /courses/import', () => {
const setup = async () => {
const teacher = createTeacher();
const course = await readFile('./apps/server/test/assets/common-cartridge/us_history_since_1877.imscc');

return { teacher, course };
};

it('should import course', async () => {
const { teacher, course } = await setup();
await em.persistAndFlush([teacher.account, teacher.user]);
em.clear();

const loggedInClient = await testApiClient.login(teacher.account);
psachmann marked this conversation as resolved.
Show resolved Hide resolved
const response = await loggedInClient.postWithAttachment('import', 'file', course, 'us_history_since_1877.imscc');
psachmann marked this conversation as resolved.
Show resolved Hide resolved

expect(response.statusCode).toEqual(201);
});
});
});
57 changes: 54 additions & 3 deletions apps/server/src/modules/learnroom/controller/course.controller.ts
Original file line number Diff line number Diff line change
@@ -1,13 +1,39 @@
import { Authenticate, CurrentUser, ICurrentUser } from '@modules/authentication';
import { Controller, Get, NotFoundException, Param, Query, Res, StreamableFile } from '@nestjs/common';
import {
Controller,
FileTypeValidator,
Get,
MaxFileSizeValidator,
NotFoundException,
Param,
ParseFilePipe,
Post,
Query,
Res,
StreamableFile,
UploadedFile,
UseInterceptors,
} from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import { ApiTags } from '@nestjs/swagger';
import { FileInterceptor } from '@nestjs/platform-express';
import {
ApiBadRequestResponse,
ApiBody,
ApiConsumes,
ApiCreatedResponse,
ApiInternalServerErrorResponse,
ApiOperation,
ApiTags,
} from '@nestjs/swagger';
import { PaginationParams } from '@shared/controller/';
import { config } from '@src/modules/files-storage/files-storage.config';
psachmann marked this conversation as resolved.
Show resolved Hide resolved
import { Response } from 'express';
import { CourseMapper } from '../mapper/course.mapper';
import { CourseImportUc } from '../uc';
import { CourseExportUc } from '../uc/course-export.uc';
import { CourseUc } from '../uc/course.uc';
import { CourseMetadataListResponse, CourseQueryParams, CourseUrlParams } from './dto';
import { CommonCartridgeFileValidator } from '../utils';
import { CourseImportBodyParams, CourseMetadataListResponse, CourseQueryParams, CourseUrlParams } from './dto';

@ApiTags('Courses')
@Authenticate('jwt')
Expand All @@ -16,6 +42,7 @@ export class CourseController {
constructor(
private readonly courseUc: CourseUc,
private readonly courseExportUc: CourseExportUc,
private readonly courseImportUc: CourseImportUc,
private readonly configService: ConfigService
) {}

Expand Down Expand Up @@ -47,4 +74,28 @@ export class CourseController {
});
return new StreamableFile(result);
}

@Post('import')
@UseInterceptors(FileInterceptor('file'))
@ApiOperation({ summary: 'Imports a course from a Common Cartridge file.' })
@ApiConsumes('multipart/form-data')
@ApiBody({ type: CourseImportBodyParams, required: true })
@ApiCreatedResponse({ description: 'Course was successfully imported.' })
@ApiBadRequestResponse({ description: 'Request data has invalid format.' })
@ApiInternalServerErrorResponse({ description: 'Internal server error.' })
public async importCourse(
SimoneRadtke-Cap marked this conversation as resolved.
Show resolved Hide resolved
@CurrentUser() currentUser: ICurrentUser,
@UploadedFile(
new ParseFilePipe({
validators: [
new MaxFileSizeValidator({ maxSize: config().MAX_FILE_SIZE }),
psachmann marked this conversation as resolved.
Show resolved Hide resolved
new FileTypeValidator({ fileType: /application\/(octet-stream|.*zip.*)/ }),
new CommonCartridgeFileValidator(),
],
})
)
file: Express.Multer.File
): Promise<void> {
await this.courseImportUc.importFromCommonCartridge(currentUser.userId, file.buffer);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
import { ApiProperty } from '@nestjs/swagger';

export class CourseImportBodyParams {
@ApiProperty({
type: String,
format: 'binary',
required: true,
description: 'The Common Cartridge file to import.',
})
file!: Express.Multer.File;
}
5 changes: 3 additions & 2 deletions apps/server/src/modules/learnroom/controller/dto/index.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
export * from './course.url.params';
export * from './course-import.body.params';
export * from './course-metadata.response';
export * from './course.query.params';
export * from './course.url.params';
export * from './dashboard.response';
export * from './dashboard.url.params';
export * from './lesson';
Expand All @@ -10,4 +12,3 @@ export * from './patch-visibility.params';
export * from './room-element.url.params';
export * from './room.url.params';
export * from './single-column-board';
export * from './course.query.params';
5 changes: 3 additions & 2 deletions apps/server/src/modules/learnroom/index.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
export * from './learnroom.config';
export * from './learnroom.module';
export {
CommonCartridgeExportService,
CourseCopyService,
CourseService,
RoomsService,
CourseGroupService,
CourseService,
DashboardService,
RoomsService,
} from './service';
2 changes: 2 additions & 0 deletions apps/server/src/modules/learnroom/learnroom-api.module.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import { RoomBoardResponseMapper } from './mapper/room-board-response.mapper';
import {
CourseCopyUC,
CourseExportUc,
CourseImportUc,
CourseUc,
DashboardUc,
LessonCopyUC,
Expand All @@ -33,6 +34,7 @@ import {
CourseCopyUC,
RoomsAuthorisationService,
CourseExportUc,
CourseImportUc,
// FIXME Refactor UCs to use services and remove these imports
{
provide: 'DASHBOARD_REPO',
Expand Down
3 changes: 3 additions & 0 deletions apps/server/src/modules/learnroom/learnroom.config.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
export interface LearnroomConfig {
FEATURE_COMMON_CARTRIDGE_COURSE_IMPORT_ENABLED: boolean;
}
3 changes: 3 additions & 0 deletions apps/server/src/modules/learnroom/learnroom.module.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import {
BoardCopyService,
ColumnBoardTargetService,
CommonCartridgeExportService,
CommonCartridgeImportService,
CourseCopyService,
CourseGroupService,
CourseService,
Expand Down Expand Up @@ -51,6 +52,7 @@ import { ToolConfigModule } from '../tool/tool-config.module';
RoomsService,
CourseService,
CommonCartridgeExportService,
CommonCartridgeImportService,
ColumnBoardTargetService,
CourseGroupService,
CourseGroupRepo,
Expand All @@ -61,6 +63,7 @@ import { ToolConfigModule } from '../tool/tool-config.module';
CourseService,
RoomsService,
CommonCartridgeExportService,
CommonCartridgeImportService,
CourseGroupService,
DashboardService,
],
Expand Down
Loading
Loading