v0.1.0

What's Changed

• Add CI Github workflows by @elreplicante in #10
• Bump @typescript-eslint/parser from 5.17.0 to 5.18.0 by @dependabot in #13
• Bump prettier from 2.6.1 to 2.6.2 by @dependabot in #11
• Bump mongodb from 4.4.1 to 4.5.0 by @dependabot in #12
• Bump @typescript-eslint/eslint-plugin from 5.17.0 to 5.18.0 by @dependabot in #14
• Bump body-parser from 1.19.2 to 1.20.0 by @dependabot in #15
• Add Sentry integration by @elreplicante in #16
• Add Prometheus Express integration by @elreplicante in #18
• Sugarcube job by @daneryl in #17
• Bump eslint from 8.12.0 to 8.13.0 by @dependabot in #25
• Deploy tweaks by @daneryl in #23
• Gracefully shutdown application on docker compose stop by @elreplicante in #26
• rename project to "Preserve" by @daneryl in #27
• move to new files processor and prom middleware by @daneryl in #28
• docker runs with node user by @daneryl in #29
• docker tweaks and env vars passthrough by @daneryl in #30
• use entrypoint to run the process as non-root by @daneryl in #31
• console.log sugarcube job output by @daneryl in #36
• Bump @typescript-eslint/eslint-plugin from 5.18.0 to 5.19.0 by @dependabot in #33
• Bump @typescript-eslint/parser from 5.18.0 to 5.19.0 by @dependabot in #32
• docker dev environment with ./run start:local by @daneryl in #38
• Bump @typescript-eslint/eslint-plugin from 5.19.0 to 5.20.0 by @dependabot in #40
• abstract away db operations to domain object by @daneryl in #41
• user to the root object, do not return on response by @daneryl in #42
• fix nodemons to watch properly by @daneryl in #43
• Microlink job test by @daneryl in #44
• Add title and date to processed evidences by @daneryl in #45
• api supports greater than for dates by @daneryl in #49
• smoke test production as ci action by @daneryl in #50
• upgrade to lateset node lts image by @daneryl in #51
• Error handling by @elreplicante in #52
• Bump eslint from 8.13.0 to 8.14.0 by @dependabot in #54
• pagination default/max limit, limit query option by @daneryl in #55
• Error handling for job processing by @daneryl in #56
• Bump express from 4.17.3 to 4.18.0 by @dependabot in #59
• Bump @typescript-eslint/parser from 5.20.0 to 5.21.0 by @dependabot in #58
• Bump @sentry/node from 6.19.6 to 6.19.7 by @dependabot in #60
• Bump @sentry/tracing from 6.19.6 to 6.19.7 by @dependabot in #57
• Bump @typescript-eslint/eslint-plugin from 5.20.0 to 5.21.0 by @dependabot in #62
• Bump supertest from 6.2.2 to 6.2.3 by @dependabot in #61
• Bump browserless from 9.3.13 to 9.3.14 by @dependabot in #67
• Bump puppeteer from 13.6.0 to 13.7.0 by @dependabot in #66
• proper dependencies to make puppeteer work by @daneryl in #65
• Bump typescript from 4.6.3 to 4.6.4 by @dependabot in #69
• Bump nodemon from 2.0.15 to 2.0.16 by @dependabot in #68
• Filter evidences by status by @elreplicante in #70
• Copy lockfile into Docker image by @elreplicante in #71
• Return evidences sorted by date ascending by @elreplicante in #73
• add global cors for the moment by @daneryl in #74
• Bump express from 4.18.0 to 4.18.1 by @dependabot in #75
• wait for idle before screenshot, log youtubedl errors instead of throw by @daneryl in #78
• ffmpeg to support more youtubedl extra sources by @daneryl in #80
• Bump @typescript-eslint/parser from 5.21.0 to 5.22.0 by @dependabot in #85
• Bump @typescript-eslint/eslint-plugin from 5.21.0 to 5.22.0 by @dependabot in #84
• Graylog's winston transport for production environments by @elreplicante in #88
• Bump browserless from 9.3.14 to 9.3.17 by @dependabot in #89
• Separate API and worker executions by @elreplicante in #90
• capture page crash error by @daneryl in #91
• provide 2 screenshots, 1 fullPage and one "normal" (2000 px height) by @daneryl in #92
• Download videos with the best resolution by @elreplicante in #93
• Bump eslint from 8.14.0 to 8.15.0 by @dependabot in #97
• ./run start scales worker to 4 instances by @daneryl in #95
• Use yt-dlp as download binary by @elreplicante in #98
• checksum for downloads after the job is complete by @daneryl in #100
• docker-compose top to check that all continers up by @daneryl in #103
• Log execution time for jobs by @elreplicante in #102
• Bump @typescript-eslint/eslint-plugin from 5.22.0 to 5.23.0 by @dependabot in #105
• Bump @typescript-eslint/parser from 5.22.0 to 5.23.0 by @dependabot in #104
• no scale by default on ./run start by @daneryl in #107
• Upgrade to node 18.1.0 by @elreplicante in #113
• Add volume for mongodb container by @elreplicante in #114
• Use SHA512 to checksum evidence files by @elreplicante in #117
• Add Sentry integration for worker by @elreplicante in #118
• Bump mongodb from 4.5.0 to 4.6.0 by @dependabot in #119
• Bump browserless from 9.3.17 to 9.3.18 by @dependabot in #129
• Bump youtube-dl-exec from 2.0.11 to 2.0.12 by @dependabot in #130
• Refactor to actions by @elreplicante in #132
• Trusted timestamp by @daneryl in #134
• Bump @typescript-eslint/eslint-plugin from 5.23.0 to 5.24.0 by @dependabot in #136
• Bump @typescript-eslint/parser from 5.23.0 to 5.25.0 by @dependabot in #137
• Bump @typescript-eslint/eslint-plugin from 5.24.0 to 5.25.0 by @dependabot in #138
• Save evidence website cookies by @elreplicante in #139
• Set page cookies by @elreplicante in #140
• Bump browserless from 9.3.18 to 9.3.20 by @dependabot in #142
• Move types to its own file by @elreplicante in #144
• Upgrade node to 18.2 by @elreplicante in #145
• Increase worker stop timeout by @elreplicante in #146
• date now extraced from TSA timestamp response by @daneryl in #148
• move download folder inside files folder by @daneryl in #150
• use node version on github actions from nvmrc by @daneryl in #152
• remove warnings by providing default env values by @daneryl in #156
• Bump ts-node from 10.7.0 to 10.8.0 by @dependabot in #159
• Bump eslint from 8.15.0 to 8.16.0 by @dependabot in #158
• Preserve PDF URLs by @elreplicante in #162
• Bump @typescript-eslint/parser from 5.25.0 to 5.26.0 by @dependabot in #160
• download html and save it as a file by @daneryl in #163
• Bump @typescript-eslint/eslint-plugin from 5.25.0 to 5.26.0 by @dependabot in #161
• Extract video downloader to its own class by @elreplicante in #164
• Rearrange code by @elreplicante in #165
• Bump typescript from 4.6.4 to 4.7.2 by @dependabot in #166
• Run Github Actions only on PR's by @elreplicante in #169
• Remove microlinkJob function by @daneryl in #168
• Video cookies by @elreplicante in #170
• Increase memory on worker by @elreplicante in #171
• Fix Sentry integration on worker by @elreplicante in #172
• Bump @sentry/node from 6.19.7 to 7.0.0 by @dependabot in #176
• Bump @sentry/tracing from 6.19.7 to 7.0.0 by @dependabot in #175
• Build and push base docker image by @elreplicante in #180
• Tag docker image with "latest" instead of main branch name by @elreplicante in #186
• Bump actions/checkout from 2 to 3 by @dependabot in #185
• Bump actions/cache from 2 to 3 by @dependabot in #184
• Bump @typescript-eslint/parser from 5.26.0 to 5.27.0 by @dependabot in #182
• Bump @typescript-eslint/eslint-plugin from 5.26.0 to 5.27.0 by @dependabot in #183
• generic setup for all specs, file paths by @daneryl in #181
• testing date tweaks, and injectable http to tsa by @daneryl in #179
• Add font packs for non-latin websites by @elreplicante in #188
• Always pull base image starting containers by @elreplicante in #190
• url as default title when title is empty by @daneryl in #191
• force base image download on start:local by @daneryl in #193
• Run tests inside docker container by @elreplicante in #192
• limit videos downloaded to 1 by @daneryl in #195
• Bump puppeteer from 13.7.0 to 14.1.2 by @dependabot in #174
• Add dependabot config for Docker by @elreplicante in #196
• Bump puppeteer from 14.1.2 to 14.2.0 by @dependabot in #198
• catch errors properly on video download and tweak download options to only download 1 video by @daneryl in #199
• Bump puppeteer from 14.2.0 to 14.2.1 by @dependabot in #201
• Bump ts-jest from 28.0.3 to 28.0.4 by @dependabot in #202
• Extract Browser class by @elreplicante in #200
• Preserve PDF snapshots by @elreplicante in #203
• increased sigkill grace period to 5 minutes by @daneryl in #204
• Bump ts-node from 10.8.0 to 10.8.1 by @dependabot in #205
• Bump typescript from 4.7.2 to 4.7.3 by @dependabot in #206
• Bump eslint from 8.16.0 to 8.17.0 by @dependabot in #207
• Bump mongodb from 4.6.0 to 4.7.0 by @dependabot in #209
• Bump @typescript-eslint/eslint-plugin from 5.27.0 to 5.27.1 by @dependabot in #210
• Bump node from 18.2 to 18.3.0 by @dependabot in #208
• Bump @typescript-eslint/parser from 5.27.0 to 5.27.1 by @dependabot in #211
• Bump puppeteer from 14.2.1 to 14.3.0 by @dependabot in #212
• Bump @sentry/node from 7.0.0 to 7.1.1 by @dependabot in #213
• Bump @sentry/tracing from 7.0.0 to 7.1.1 by @dependabot in #214
• Bump browserless from 9.3.20 to 9.3.21 by @dependabot in #215
• Bump browserless from 9.3.21 to 9.4.0 by @dependabot in #216
• Bump @typescript-eslint/parser from 5.27.1 to 5.28.0 by @dependabot in #221
• Bump @typescript-eslint/eslint-plugin from 5.27.1 to 5.28.0 by @dependabot in #219
• Bump prettier from 2.6.2 to 2.7.0 by @dependabot in #220
• Bump ts-jest from 28.0.4 to 28.0.5 by @dependabot in #222
• Bump puppeteer from 14.3.0 to 14.4.0 by @dependabot in #223
• Bump puppeteer from 14.4.0 to 14.4.1 by @dependabot in #226
• Bump prettier from 2.7.0 to 2.7.1 by @dependabot in #224
• Bump eslint from 8.17.0 to 8.18.0 by @dependabot in #228
• Bump express-prom-bundle from 6.4.1 to 6.5.0 by @dependabot in #229
• Bump typescript from 4.7.3 to 4.7.4 by @dependabot in #231
• Bump node from 18.3.0 to 18.4.0 by @dependabot in #227
• Update .nvmrc with same version as base Dockerfile by @elreplicante in #232
• Bump @sentry/tracing from 7.1.1 to 7.2.0 by @dependabot in #225
• Bump @typescript-eslint/parser from 5.28.0 to 5.29.0 by @dependabot in #234
• Bump @typescript-eslint/eslint-plugin from 5.28.0 to 5.29.0 by @dependabot in #233
• Bump nodemon from 2.0.16 to 2.0.18 by @dependabot in #237
• Bump winston from 3.7.2 to 3.8.0 by @dependabot in #236
• Bump @sentry/tracing from 7.2.0 to 7.3.0 by @dependabot in #239
• Bump puppeteer from 14.4.1 to 15.1.1 by @dependabot in #240
• Bump eslint-plugin-prettier from 4.0.0 to 4.1.0 by @dependabot in #242
• Bump @typescript-eslint/eslint-plugin from 5.29.0 to 5.30.0 by @dependabot in #243
• Bump @sentry/tracing from 7.3.0 to 7.3.1 by @dependabot in #245
• Bump @typescript-eslint/parser from 5.29.0 to 5.30.0 by @dependabot in #246
• Token generation via API by @elreplicante in #247
• Bump eslint-plugin-prettier from 4.1.0 to 4.2.1 by @dependabot in #248
• Bump browserless from 9.4.0 to 9.5.0 by @dependabot in #250
• Bump puppeteer from 15.1.1 to 15.2.0 by @dependabot in #249
• Bump browserless from 9.5.0 to 9.5.1 by @dependabot in #251
• Bump winston from 3.8.0 to 3.8.1 by @dependabot in #252
• Bump @sentry/tracing from 7.3.1 to 7.4.1 by @dependabot in #254
• Bump eslint from 8.18.0 to 8.19.0 by @dependabot in #257
• Bump @typescript-eslint/parser from 5.30.0 to 5.30.4 by @dependabot in #255
• Bump @typescript-eslint/eslint-plugin from 5.30.0 to 5.30.4 by @dependabot in #256
• Bump ts-node from 10.8.1 to 10.8.2 by @dependabot in #259
• Bump puppeteer from 15.2.0 to 15.3.0 by @dependabot in #258
• Bump supertest from 6.2.3 to 6.2.4 by @dependabot in #260
• Bump @sentry/tracing from 7.4.1 to 7.5.0 by @dependabot in #261
• Bump @typescript-eslint/parser from 5.30.4 to 5.30.5 by @dependabot in #264
• Bump @typescript-eslint/eslint-plugin from 5.30.4 to 5.30.5 by @dependabot in #263
• Bump puppeteer from 15.3.0 to 15.3.1 by @dependabot in #266
• Bump nodemon from 2.0.18 to 2.0.19 by @dependabot in #265
• Bump @sentry/tracing from 7.5.0 to 7.5.1 by @dependabot in #267
• Bump youtube-dl-exec from 2.0.12 to 2.0.13 by @dependabot in #269
• Bump puppeteer from 15.3.1 to 15.3.2 by @dependabot in #273
• Bump youtube-dl-exec from 2.0.13 to 2.1.0 by @dependabot in #276
• Bump @sentry/tracing from 7.5.1 to 7.6.0 by @dependabot in #274
• Bump node from 18.4.0 to 18.5.0 by @dependabot in #277
• Downloader cookies by @daneryl in #280
• Bump puppeteer from 15.3.2 to 15.4.0 by @dependabot in #281
• Bump mongodb from 4.7.0 to 4.8.0 by @dependabot in #282
• Bump youtube-dl-exec from 2.1.0 to 2.1.2 by @dependabot in #283
• Bump @typescript-eslint/parser from 5.30.5 to 5.30.6 by @dependabot in #279
• Bump ts-node from 10.8.2 to 10.9.1 by @dependabot in #284
• Bump @typescript-eslint/eslint-plugin from 5.30.5 to 5.30.6 by @dependabot in #278
• Fail silently on video download by @elreplicante in #285
• Create cookies folder by @elreplicante in #286
• Add header to cookie jar file by @elreplicante in #287
• Bump @sentry/tracing from 7.6.0 to 7.7.0 by @dependabot in #289
• Bump node from 18.5.0 to 18.6.0 by @dependabot in #290
• Bump youtube-dl-exec from 2.1.2 to 2.1.3 by @dependabot in #293
• Bump eslint from 8.19.0 to 8.20.0 by @dependabot in #292
• Bump ts-jest from 28.0.5 to 28.0.7 by @dependabot in #291
• Bump @typescript-eslint/parser from 5.30.6 to 5.30.7 by @dependabot in #295
• Bump @typescript-eslint/eslint-plugin from 5.30.6 to 5.30.7 by @dependabot in #294
• Bump browserless from 9.5.1 to 9.5.4 by @dependabot in #297
• Bump puppeteer from 15.4.0 to 15.5.0 by @dependabot in #296
• Bump node from 18.6.0 to 18.7.0 by @dependabot in #301
• Bump youtube-dl-exec from 2.1.3 to 2.1.4 by @dependabot in #298
• Bump @typescript-eslint/eslint-plugin from 5.30.7 to 5.31.0 by @dependabot in #300
• Bump @typescript-eslint/parser from 5.30.7 to 5.31.0 by @dependabot in #299
• Bump mongodb from 4.8.0 to 4.8.1 by @dependabot in #302
• Bump @sentry/tracing from 7.7.0 to 7.8.0 by @dependabot in #303
• Zero-downtime deploy by @elreplicante in #305
• Bump eslint from 8.20.0 to 8.21.0 by @dependabot in #308
• Bump @typescript-eslint/eslint-plugin from 5.31.0 to 5.32.0 by @dependabot in #309
• Bump @typescript-eslint/parser from 5.31.0 to 5.32.0 by @dependabot in #310
• Bump @sentry/node and @sentry/trace from 7.8.0 to 7.8.1 by @dependabot in #307
• Bump puppeteer from 15.5.0 to 16.0.0 by @dependabot in #311
• Bump browserless from 9.5.4 to 9.6.0 by @dependabot in #312
• Bump @sentry/tracing from 7.8.1 to 7.9.0 by @dependabot in #314
• Bump puppeteer from 16.0.0 to 16.1.0 by @dependabot in #315
• Bump browserless from 9.6.0 to 9.6.1 by @dependabot in #318
• Bump youtube-dl-exec from 2.1.4 to 2.1.5 by @dependabot in #317
• Bump @typescript-eslint/parser from 5.32.0 to 5.33.0 by @dependabot in #319
• Bump @typescript-eslint/eslint-plugin from 5.32.0 to 5.33.0 by @dependabot in #316
• Bump @sentry/tracing from 7.9.0 to 7.10.0 by @dependabot in #321
• Bump ts-jest from 28.0.7 to 28.0.8 by @dependabot in #322
• Bump @typescript-eslint/parser from 5.33.0 to 5.33.1 by @dependabot in #324
• Bump @typescript-eslint/eslint-plugin from 5.33.0 to 5.33.1 by @dependabot in #325
• Bump eslint from 8.21.0 to 8.22.0 by @dependabot in #326
• Bump puppeteer from 16.1.0 to 16.1.1 by @dependabot in #327
• Bump browserless from 9.6.1 to 9.6.4 by @dependabot in #323
• Bump @sentry/tracing from 7.10.0 to 7.11.1 by @dependabot in #329
• Bump mongodb from 4.8.1 to 4.9.0 by @dependabot in #330
• Bump puppeteer from 16.1.1 to 16.2.0 by @dependabot in #331
• Bump @typescript-eslint/eslint-plugin from 5.33.1 to 5.34.0 by @dependabot in #332
• Bump @typescript-eslint/parser from 5.33.1 to 5.34.0 by @dependabot in #334
• Bump prom-client from 14.0.1 to 14.1.0 by @dependabot in #333
• Bump browserless from 9.6.4 to 9.6.5 by @dependabot in #335
• Bump @typescript-eslint/parser from 5.34.0 to 5.35.1 by @dependabot in #337
• Bump @typescript-eslint/eslint-plugin from 5.34.0 to 5.35.1 by @dependabot in #336
• Bump browserless from 9.6.5 to 9.6.8 by @dependabot in #341
• Bump eslint from 8.22.0 to 8.23.0 by @dependabot in #343
• Bump node from 18.7.0 to 18.8.0 by @dependabot in #338
• Bump typescript from 4.7.4 to 4.8.2 by @dependabot in #340
• Bump puppeteer from 16.2.0 to 17.0.0 by @dependabot in #342
• Bump @typescript-eslint/parser from 5.35.1 to 5.36.1 by @dependabot in #344
• Bump @typescript-eslint/eslint-plugin from 5.35.1 to 5.36.1 by @dependabot in #345
• Bump mongodb from 4.9.0 to 4.9.1 by @dependabot in #347
• conditional workflow for testing ci by @daneryl in #354
• Bump browserless from 9.6.8 to 9.6.9 by @dependabot in #357
• Bump @sentry/node from 7.11.1 to 7.12.1 by @dependabot in #356
• Bump @typescript-eslint/parser from 5.36.1 to 5.36.2 by @dependabot in #362
• Bump puppeteer from 17.0.0 to 17.1.1 by @dependabot in #358
• Bump @typescript-eslint/eslint-plugin from 5.36.1 to 5.36.2 by @dependabot in #361
• Bump puppeteer from 17.1.1 to 17.1.2 by @dependabot in #363
• Bump winston from 3.8.1 to 3.8.2 by @dependabot in #365
• Bump puppeteer from 17.1.2 to 17.1.3 by @dependabot in #364
• Bump eslint from 8.23.0 to 8.23.1 by @dependabot in #368
• Bump typescript from 4.8.2 to 4.8.3 by @dependabot in #366
• Bump node from 18.8.0 to 18.9.0 by @dependabot in #367
• Bump @typescript-eslint/eslint-plugin from 5.36.2 to 5.37.0 by @dependabot in #370
• Bump @typescript-eslint/parser from 5.36.2 to 5.37.0 by @dependabot in #371
• Bump nodemon from 2.0.19 to 2.0.20 by @dependabot in #374
• Bump mongodb from 4.9.1 to 4.10.0 by @dependabot in #375
• Bump browserless from 9.6.9 to 9.6.10 by @dependabot in #376
• Bump @typescript-eslint/parser from 5.37.0 to 5.38.0 by @dependabot in #377
• Bump @sentry/tracing from 7.12.1 to 7.13.0 by @dependabot in #373
• Bump @typescript-eslint/eslint-plugin from 5.37.0 to 5.38.0 by @dependabot in #378
• Bump puppeteer from 17.1.3 to 18.0.3 by @dependabot in #382
• Add blank-state npm command to populate authorization database by @elreplicante in #381
• Bump puppeteer from 18.0.3 to 18.0.4 by @dependabot in #383
• Bump puppeteer from 18.0.4 to 18.0.5 by @dependabot in #384
• Bump eslint from 8.23.1 to 8.24.0 by @dependabot in #385
• Bump @typescript-eslint/parser from 5.38.0 to 5.38.1 by @dependabot in #387
• Bump typescript from 4.8.3 to 4.8.4 by @dependabot in #390
• Bump @typescript-eslint/eslint-plugin from 5.38.0 to 5.38.1 by @dependabot in #386
• Bump @sentry/node from 7.13.0 to 7.14.0 by @dependabot in #392
• Bump node from 18.9.0 to 18.10.0 by @dependabot in #391
• Bump youtube-dl-exec from 2.1.5 to 2.1.6 by @dependabot in #393
• Bump @typescript-eslint/parser from 5.38.1 to 5.39.0 by @dependabot in #396
• Bump @sentry/node from 7.14.0 to 7.14.1 by @dependabot in #394
• Bump browserless from 9.6.10 to 9.6.11 by @dependabot in #399
• Bump supertest from 6.2.4 to 6.3.0 by @dependabot in #397
• Bump @typescript-eslint/eslint-plugin from 5.38.1 to 5.39.0 by @dependabot in #398
• Bump body-parser from 1.20.0 to 1.20.1 by @dependabot in #402
• Bump puppeteer from 18.0.5 to 18.2.1 by @dependabot in #403
• Bump @sentry/node from 7.14.1 to 7.14.2 by @dependabot in #401
• Bump eslint from 8.24.0 to 8.25.0 by @dependabot in #406
• Bump @sentry/tracing from 7.14.2 to 7.15.0 by @dependabot in #405
• Bump @typescript-eslint/parser from 5.39.0 to 5.40.0 by @dependabot in #408
• Bump @typescript-eslint/eslint-plugin from 5.39.0 to 5.40.0 by @dependabot in #407
• Bump puppeteer from 18.2.1 to 19.0.0 by @dependabot in #409
• Bump node from 18.10.0 to 18.11.0 by @dependabot in #410
• Bump @typescript-eslint/eslint-plugin from 5.40.0 to 5.40.1 by @dependabot in #411
• Production docker image by @elreplicante in #413

New Contributors

• @elreplicante made their first contribution in #10
• @dependabot made their first contribution in #13

Full Changelog: https://github.com/huridocs/preserve/commits/v0.1.0