Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EW-694 Common Cartridge Course Import #4700

Merged
merged 61 commits into from
Feb 23, 2024
Merged
Show file tree
Hide file tree
Changes from 52 commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
8aec9b7
EW-694 adding a new package and a common cartridge for testing
psachmann Jan 11, 2024
c99d7a8
EW-694 adding parsing for common cartridge file
psachmann Jan 11, 2024
413599a
EW-694 code coverage
psachmann Jan 11, 2024
f44841c
EW-694 creating course import endpoint
psachmann Jan 16, 2024
20c9cf5
EW-694 adding new feature flag for common cartridge course import
psachmann Jan 17, 2024
fe99609
EW-694 adding tests and improving coverage
psachmann Jan 18, 2024
12632fe
Merge branch 'main' into EW-694
psachmann Jan 19, 2024
b17f7a3
EW-694 comment update
psachmann Jan 22, 2024
d32bff5
Merge branch 'main' into EW-694
psachmann Jan 22, 2024
89abf23
EW-694 skipping lib check
psachmann Jan 22, 2024
83466a3
EW-694 finishing touches for the server
psachmann Jan 23, 2024
1710d81
EW-694 updating the regex for the zip mime type
psachmann Jan 25, 2024
18a80ca
Merge branch 'main' into EW-694
psachmann Jan 26, 2024
24c207a
Merge branch 'main' into EW-694
psachmann Jan 29, 2024
f693c07
EW-694 fixing test coverage
psachmann Jan 29, 2024
e007993
EW-694 adding a custom exception and renaming a field
psachmann Jan 29, 2024
fb1b1a0
EW-694 some renaming
psachmann Jan 29, 2024
d78cadf
EW-694 updating tsconfig.ts
psachmann Jan 30, 2024
c98c998
EW-694 renaming importCourse to createCourse
psachmann Jan 30, 2024
27a7591
Merge branch 'main' into EW-694
psachmann Jan 30, 2024
defe983
EW-694 updating package-lock.json
psachmann Jan 30, 2024
fed8215
EW-694 updating new tsconfig.json
psachmann Jan 30, 2024
27d8c1f
Merge branch 'main' into EW-694
psachmann Jan 30, 2024
f71b8a8
Merge branch 'main' into EW-694
psachmann Jan 31, 2024
bb4c9df
Merge branch 'main' into EW-694
psachmann Feb 1, 2024
2a9155c
EW-694 removing changes from tsconfig.json
psachmann Jan 31, 2024
0d28845
EW-694 adding comment
psachmann Feb 7, 2024
4cfbb9a
EW-694 fixing typos
psachmann Feb 8, 2024
ea5e52b
EW-694 restructuring config and validation
psachmann Feb 8, 2024
94c13f4
EW-694 refactoring
psachmann Feb 9, 2024
810d83a
EW-694 adding antivirus scan to cc course import
psachmann Feb 9, 2024
973a7c2
EW-694 removing antivirus scan
psachmann Feb 13, 2024
93e177c
Merge branch 'main' into EW-694
psachmann Feb 14, 2024
d67efc8
Merge branch 'main' into EW-694
psachmann Feb 14, 2024
377e198
EW-694 updating new dependency
psachmann Feb 14, 2024
6a158b6
Merge branch 'main' into EW-694
psachmann Feb 15, 2024
3c46674
EW-694 skipping lib check
psachmann Feb 15, 2024
01ad220
Merge branch 'main' into EW-694
psachmann Feb 15, 2024
f5b2ad9
EW-694 removing regex check during file upload
psachmann Feb 15, 2024
ed49f19
EW-694 fixing linter errors
psachmann Feb 15, 2024
b7a1887
EW-694 updating feature flag description
psachmann Feb 19, 2024
f9f1ee3
EW-694 improving test structure
psachmann Feb 19, 2024
d4928ef
EW-694 reverting lock file version from 3 to 2
psachmann Feb 19, 2024
178016e
Merge branch 'main' into EW-694
psachmann Feb 19, 2024
d9d4929
E-694 removing skipLibCheck
psachmann Feb 19, 2024
f799a24
EW-694 adding definitions for jsdom
psachmann Feb 19, 2024
777f2d2
Merge branch 'main' into EW-694
psachmann Feb 20, 2024
fec4d91
EW-694 fixing tests
psachmann Feb 20, 2024
1220958
EW-694 working on coverage
psachmann Feb 20, 2024
7d12916
EW-694 fixing return types
psachmann Feb 21, 2024
a283272
Merge branch 'main' into EW-694
psachmann Feb 21, 2024
1419ba8
EW-694 fixing tests and linter issues
psachmann Feb 21, 2024
38c407c
Merge branch 'main' into EW-694
psachmann Feb 21, 2024
c0dee5b
EW-694 fixing review comments
psachmann Feb 21, 2024
a3ae1e0
Merge branch 'main' into EW-694
psachmann Feb 21, 2024
9138e2e
Merge branch 'main' into EW-694
psachmann Feb 21, 2024
3e26ade
Merge branch 'main' into EW-694
psachmann Feb 21, 2024
f138478
EW-694 changing validator pipe
psachmann Feb 22, 2024
93a557c
EW-694 updating index file
psachmann Feb 22, 2024
2417000
EW-694 removed not needed test
psachmann Feb 22, 2024
c82ec5c
Merge branch 'main' into EW-694
psachmann Feb 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
import AdmZip from 'adm-zip';
import { CommonCartridgeFileParser } from './common-cartridge-file-parser';

describe('CommonCartridgeFileParser', () => {
describe('constructor', () => {
describe('when manifest file is found', () => {
const setup = (manifestName: string) => {
const archive = new AdmZip();

archive.addFile(manifestName, Buffer.from('<manifest></manifest>'));

const file = archive.toBuffer();

return { file };
};

it('should use imsmanifest.xml as manifest', () => {
const { file } = setup('imsmanifest.xml');
const parser = new CommonCartridgeFileParser(file);

expect(parser.manifest).toBeDefined();
});

it('should use manifest.xml as manifest', () => {
const { file } = setup('manifest.xml');
const parser = new CommonCartridgeFileParser(file);

expect(parser.manifest).toBeDefined();
});
});

describe('when manifest file is not found', () => {
const setup = () => {
const archive = new AdmZip();
const file = archive.toBuffer();

return { file };
};

it('should throw', () => {
const { file } = setup();

expect(() => new CommonCartridgeFileParser(file)).toThrow('Manifest file not found');
});
});

describe('when file is not an archive', () => {
SimoneRadtke-Cap marked this conversation as resolved.
Show resolved Hide resolved
const setup = () => {
const file = new AdmZip().toBuffer();

return { file };
};

it('should throw', () => {
const { file } = setup();

expect(() => new CommonCartridgeFileParser(file)).toThrow('Manifest file not found');
});
});
});
});
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
import AdmZip from 'adm-zip';
import { CommonCartridgeManifestParser } from './common-cartridge-manifest-parser';
import { CommonCartridgeManifestNotFoundException } from './utils/common-cartridge-manifest-not-found.exception';

export class CommonCartridgeFileParser {
private readonly manifestParser: CommonCartridgeManifestParser;

public constructor(file: Buffer) {
const archive = new AdmZip(file);

this.manifestParser = new CommonCartridgeManifestParser(this.getManifestFileAsString(archive));
}

public get manifest(): CommonCartridgeManifestParser {
return this.manifestParser;
}

private getManifestFileAsString(archive: AdmZip): string | never {
// imsmanifest.xml is the standard name, but manifest.xml is also valid until v1.3
const manifest = archive.getEntry('imsmanifest.xml') || archive.getEntry('manifest.xml');

if (manifest) {
return archive.readAsText(manifest);
}

throw new CommonCartridgeManifestNotFoundException();
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
import AdmZip from 'adm-zip';
import { readFile } from 'fs/promises';
import { CommonCartridgeManifestParser } from './common-cartridge-manifest-parser';

describe('CommonCartridgeManifestParser', () => {
const setupFile = async (loadFile: boolean) => {
if (!loadFile) {
const sut = new CommonCartridgeManifestParser('<manifest></manifest>');

return { sut };
}

const buffer = await readFile('./apps/server/test/assets/common-cartridge/us_history_since_1877.imscc');
const archive = new AdmZip(buffer);
const sut = new CommonCartridgeManifestParser(archive.readAsText('imsmanifest.xml'));

return { sut };
};

describe('getSchema', () => {
describe('when schema is present', () => {
const setup = async () => setupFile(true);

it('should return the schema', async () => {
const { sut } = await setup();
const result = sut.getSchema();

expect(result).toBe('IMS Common Cartridge');
});
});

describe('when schema is not present', () => {
const setup = async () => setupFile(false);

it('should return undefined', async () => {
const { sut } = await setup();
const result = sut.getSchema();

expect(result).toBeUndefined();
});
});
});

describe('getVersion', () => {
describe('when version is present', () => {
const setup = async () => setupFile(true);

it('should return the version', async () => {
const { sut } = await setup();
const result = sut.getVersion();

expect(result).toBe('1.3.0');
});
});

describe('when version is not present', () => {
const setup = async () => setupFile(false);

it('should return undefined', async () => {
const { sut } = await setup();
const result = sut.getVersion();

expect(result).toBeUndefined();
});
});
});

describe('getTitle', () => {
describe('when title is present', () => {
const setup = async () => setupFile(true);

it('should return the title', async () => {
const { sut } = await setup();
const result = sut.getTitle();

expect(result).toBe('201510-AMH-2020-70C-12218-US History Since 1877');
});
});

describe('when title is not present', () => {
const setup = async () => setupFile(false);

it('should return null', async () => {
const { sut } = await setup();
const result = sut.getTitle();

expect(result).toBeUndefined();
});
});
});
});
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
import { JSDOM } from 'jsdom';

export class CommonCartridgeManifestParser {
private readonly doc: Document;

public constructor(manifest: string) {
this.doc = new JSDOM(manifest, { contentType: 'text/xml' }).window.document;
}

public getSchema(): string | undefined {
const result = this.doc.querySelector('manifest > metadata > schema');

return result?.textContent ?? undefined;
}

public getVersion(): string | undefined {
const result = this.doc.querySelector('manifest > metadata > schemaversion');

return result?.textContent ?? undefined;
}

public getTitle(): string | undefined {
SimoneRadtke-Cap marked this conversation as resolved.
Show resolved Hide resolved
const result = this.doc.querySelector('manifest > metadata > lom > general > title > string');

return result?.textContent ?? undefined;
}
}
1 change: 1 addition & 0 deletions apps/server/src/modules/common-cartridge/import/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export * from './common-cartridge-file-parser';
SimoneRadtke-Cap marked this conversation as resolved.
Show resolved Hide resolved
11 changes: 11 additions & 0 deletions apps/server/src/modules/common-cartridge/import/jsdom.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
// This is a workaround for the missing types for jsdom, because the types are not included in the package itself.
// This is a declaration file for the JSDOM class, which is used in the CommonCartridgeManifestParser.
// Currently the JSDOM types are bit buggy and don't work properly with our project setup.

declare module 'jsdom' {
class JSDOM {
constructor(html: string, options?: Record<string, unknown>);

window: Window;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
import { CommonCartridgeManifestNotFoundException } from './common-cartridge-manifest-not-found.exception';

describe('CommonCartridgeManifestNotFoundException', () => {
describe('getLogMessage', () => {
describe('when returning a message', () => {
const setup = () => {
const sut = new CommonCartridgeManifestNotFoundException();

return { sut };
};

it('should contain the type', () => {
const { sut } = setup();

const result = sut.getLogMessage();

expect(result.type).toEqual('WRONG_FILE_FORMAT');
});

it('should contain the stack', () => {
const { sut } = setup();

const result = sut.getLogMessage();

expect(result.stack).toEqual(sut.stack);
});
});
});
});
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
import { BadRequestException } from '@nestjs/common';
import { ErrorLogMessage, Loggable } from '@src/core/logger';

export class CommonCartridgeManifestNotFoundException extends BadRequestException implements Loggable {
constructor() {
super('Manifest file not found.');
}

public getLogMessage(): ErrorLogMessage {
const message: ErrorLogMessage = {
type: 'WRONG_FILE_FORMAT',
stack: this.stack,
};

return message;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import { INestApplication, StreamableFile } from '@nestjs/common';
import { Test, TestingModule } from '@nestjs/testing';
import { Permission } from '@shared/domain/interface';
import { TestApiClient, UserAndAccountTestFactory, cleanupCollections, courseFactory } from '@shared/testing';
import { readFile } from 'node:fs/promises';

const createStudent = () => {
const { studentUser, studentAccount } = UserAndAccountTestFactory.buildStudent({}, [Permission.COURSE_VIEW]);
Expand Down Expand Up @@ -51,6 +52,7 @@ describe('Course Controller (API)', () => {

return { student, course, teacher };
};

it('should find courses as student', async () => {
const { student, course } = setup();
await em.persistAndFlush([student.account, student.user, course]);
Expand All @@ -65,6 +67,7 @@ describe('Course Controller (API)', () => {
expect(data[0].startDate).toBe(course.startDate);
expect(data[0].untilDate).toBe(course.untilDate);
});

it('should find courses as teacher', async () => {
const { teacher, course } = setup();
await em.persistAndFlush([teacher.account, teacher.user, course]);
Expand Down Expand Up @@ -96,6 +99,7 @@ describe('Course Controller (API)', () => {

return { course, teacher, teacherUnkownToCourse, substitutionTeacher, student1 };
};

it('should find course export', async () => {
const { teacher, course } = setup();
await em.persistAndFlush([teacher.account, teacher.user, course]);
Expand All @@ -114,4 +118,27 @@ describe('Course Controller (API)', () => {
expect(response.header['content-disposition']).toBe('attachment;');
});
});

describe('[POST] /courses/import', () => {
const setup = async () => {
const teacher = createTeacher();
const course = await readFile('./apps/server/test/assets/common-cartridge/us_history_since_1877.imscc');
const courseFileName = 'us_history_since_1877.imscc';

await em.persistAndFlush([teacher.account, teacher.user]);
em.clear();

const loggedInClient = await testApiClient.login(teacher.account);
psachmann marked this conversation as resolved.
Show resolved Hide resolved

return { loggedInClient, course, courseFileName };
};

it('should import course', async () => {
const { loggedInClient, course, courseFileName } = await setup();

const response = await loggedInClient.postWithAttachment('import', 'file', course, courseFileName);

expect(response.statusCode).toEqual(201);
});
});
});
Loading
Loading