Code

Upgrade to Hadoop 3#298

Code

Draft

anjackson wants to merge 18 commits intomasterukwa/webarchive-discovery:masterfrom hadoop3ukwa/webarchive-discovery:hadoop3Copy head branch name to clipboard

+1,008-209

Commits

Commits on Aug 4, 2022

• Updated to Hadoop3, now compiles and Spark runs okay. Tests not verified yet.

  anjackson

  committed
  4279bd2

Commits on Aug 5, 2022

• Decouple experimental code into modules.

  anjackson

  committed
  34fa42e

• Add initial test command line tool.

  anjackson

  committed
  4ce40b5

Commits on Aug 6, 2022

• Merge branch 'master' into hadoop3

  anjackson

  committed
  b42daf4

• Now runs on H3 with shaded job jar.

  anjackson

  committed
  f2a0e61

Commits on Sep 23, 2022

• Fix CDX test.

  anjackson

  committed
  73574e0

• Fix up tests.

  anjackson

  committed
  a2ccedc

• More test cleanups.

  anjackson

  committed
  e594297

Commits on Sep 24, 2022

• Merge branch 'master' into hadoop3

  anjackson

  committed
  7bb9234

Commits on Sep 25, 2022

• Initial sketch of a DataFrame mapping.

  anjackson

  committed
  40532ca

Commits on Sep 26, 2022

• Tidy up logging, upgrade it etc.

  anjackson

  committed
  4df2ddd

Commits on Sep 27, 2022

• Tidyied up code and populated more fields.

  anjackson

  committed
  982628b

• Updated versions, runs under Java 11.

  anjackson

  committed
  4f843fa

Commits on Sep 28, 2022

• Switched to experimental approach that could support a dynamic schema.

  anjackson

  committed
  62de913

Commits on Sep 29, 2022

• Added separate config, avoid exclusions.

  anjackson

  committed
  8834476

• Merge branch 'master' into hadoop3

  anjackson

  committed
  fabc1eb

Commits on Mar 18, 2023

• Merge branch 'master' into hadoop3

  anjackson

  committed
  c364f1a

Commits on Mar 24, 2023

• Merge branch 'master' into hadoop3

  anjackson

  committed
  1220f9c