Skip to content

Latest commit

 

History

History
467 lines (350 loc) · 37.4 KB

CHANGELOG.md

File metadata and controls

467 lines (350 loc) · 37.4 KB
Raw

Change Log

1.2.2 (2018-12-26)

Full Changelog

Merged pull requests:

1.2.1 (2018-04-28)

Full Changelog

Implemented enhancements:

  • workers: use global scope instead of worker context #236
  • Add tests for machine learning library #233
  • Add tests for ds.save() #232
  • Add tests for sc.textfile() #231

Closed issues:

  • document and test save() to CSV format #241
  • modernize javascript syntax #210

Merged pull requests:

1.2.0 (2017-11-14)

Full Changelog

Closed issues:

  • Import skale-ml, skale-examples, skale-cli #181

0.8.0 (2017-11-14)

Full Changelog

Closed issues:

  • Add missing tests, refactor testing framework #205
  • External modules in worker #203
  • hey dude,I wonder has this program been applied to any application. #199
  • for loop over items used later as index #178

Merged pull requests:

  • rename skale-engine to skale (#181) #230 (mvertes)
  • remove deprecated examples/ml/benchmark #229 (mvertes)
  • remove deprecated ml/examples/logreg #228 (mvertes)
  • ml: document StandardScaler() #227 (mvertes)
  • ml: document classificationMetrics #226 (mvertes)
  • Document sgd.predict() #225 (mvertes)
  • document sc.require(). Fix examples #224 (mvertes)
  • improve docs #223 (mvertes)
  • improve docs #222 (mvertes)
  • fix link #221 (mvertes)
  • fix eslint warnings #220 (mvertes)
  • update doc site #219 (mvertes)
  • Convert DataSet Object and related functions to modern JS #218 (frytyler)
  • avoid using require.resolve() which may fail for old node versions #217 (mvertes)
  • Rename tape-test in test, get rid of mocha, resolves #205 #216 (mvertes)
  • add new tests as per #205 #215 (mvertes)
  • additional tests, as per #205 #214 (mvertes)
  • additional tests, as per #205 #213 (mvertes)
  • tests: migrate to tape, as per #205 #212 (mvertes)
  • Implement dependency injection into workers, resolves #203 #211 (mvertes)
  • docs: modularize, add logo #208 (mvertes)
  • Pass context to worker callbacks, add modules setting as per #203 (in progress) #207 (mvertes)
  • test: start to add tape test cases, as per #205 #206 (mvertes)
  • tests: simplify engine. Use standalone mode #204 (mvertes)
  • update dependencies #202 (mvertes)
  • lint: do not scan site/ #201 (mvertes)
  • doc: fix deploy rule #200 (mvertes)
  • doc: use mkdocs-material for documentation website. Add build recipes. #198 (mvertes)
  • doc: rebuild toc #197 (mvertes)
  • fix ml.KMeans, add example based on classical iris dataset #196 (mvertes)
  • fix sample(). Add takeSample(). #195 (mvertes)
  • now compute area under Precision Recall Curve, improve examples #194 (mvertes)
  • rename binary-classification-metrics to classification-metrics #193 (mvertes)
  • README: add link to code of conduct and API reference #192 (mvertes)
  • Create CODE_OF_CONDUCT.md #191 (mvertes)
  • ml: partition adult dataset, add headers in CSV files #190 (mvertes)
  • ml: rename SGDClassifier in SGDLinearModel, add regression example #189 (mvertes)
  • ml: remove deprecated logistic-regression dependency #188 (mvertes)
  • ml SGDClassifier: include optional intercept fitting #187 (mvertes)
  • ml: add SGDClassifier class #186 (mvertes)
  • contributing: add notes about documentation and coding rules #185 (mvertes)
  • ml: simplify a lot model evaluation metrics. Compute ROC AUC. #184 (mvertes)
  • Fix import of ml #183 (mvertes)
  • import skale-ml, as per #181 #182 (mvertes)
  • update dependencies #180 (mvertes)
  • fixed the use of '===' operator in all sources #177 (vsimko)
  • travis: add osx target #176 (mvertes)
  • Fix debug trace, update dependencies #175 (mvertes)
  • aggregateByKey: fix a bug where undefined key crashes post-shuffle #173 (mvertes)
  • save: add support to CSV format output #172 (mvertes)
  • Automatically forward AWS env variables in context and workers #171 (mvertes)
  • workers: control garbage collect by command line option. Improve debug traces #170 (mvertes)

0.7.2 (2017-06-27)

Full Changelog

Merged pull requests:

  • update dependencies build files #169 (mvertes)
  • update dependencies #168 (mvertes)
  • workers: reduce amount of traces during shuffle #167 (mvertes)
  • worker: manually call garbage collector at end of task. Experimental #166 (mvertes)
  • add interactive REPL shell, supporting async/await #165 (mvertes)
  • Fix and improved tracing of time measurements #164 (mvertes)
  • worker: allow to retry connections using -r <nbretry> option #163 (mvertes)
  • dataset: fix parsing of shuffle in aggregateByKey, clean code #162 (mvertes)
  • performance: block processing for pipelines at source level, part 2 #161 (mvertes)
  • eslint: allow arrow functions and async/await #160 (mvertes)
  • Fix previous task serialization change which impacted sc.range() #159 (mvertes)
  • Clean code, improve traces for performances, set maximum partitions. #158 (mvertes)
  • update dependencies #157 (mvertes)
  • textFile: Fix handling of gzipped files in local filesystem #156 (mvertes)

0.7.1 (2017-05-17)

Full Changelog

Merged pull requests:

0.7.0 (2017-04-04)

Full Changelog

Fixed bugs:

  • textFile: fix a bug where the 1st file in S3 dir was skipped #126 (mvertes)

Closed issues:

  • skale-engine version 0.5.3 regression? #61
  • sizeOf is incomplete and inaccurate #52

Merged pull requests:

  • update dependencies #143 (mvertes)
  • textFile, save: document new protocols and formats #142 (mvertes)
  • save: support direct streaming to S3 #141 (mvertes)
  • textFile: support file globbing (i.e. src/**/*.js) in path argument #140 (mvertes)
  • save: support direct streaming to azure. #139 (mvertes)
  • azure: handle server errors with internal retry policy filter #138 (mvertes)
  • AggregateByKey: faster processing of shuffle files #137 (mvertes)
  • aggregateByKey, coGroup: performance improvements #136 (mvertes)
  • Performance increase of aggregateByKey and coGroup #135 (mvertes)
  • azure: handle errors from storage server: fetch before process and implement retries. #134 (mvertes)
  • distributed mode: pre-fork new workers at end of master #133 (mvertes)
  • textFile: fix compute of partitions for Azure #132 (mvertes)
  • save: upload datasets to azure for urls in wasb:// #131 (mvertes)
  • textFile: support file globbing, azure blobs only for now #130 (mvertes)
  • textFile: support reading from azure storage blobs #129 (mvertes)
  • Simplify top and take actions, and fix out of memory errors for large datasets. #128 (mvertes)
  • Aggregate now guarantees partitions ordering. Improve traces. #127 (mvertes)
  • textFile: read parquet from local and S3 directories #125 (mvertes)
  • Add Parquet support #124 (mvertes)
  • Readme: Fix appveyor badge #123 (mvertes)
  • fix benchmark #122 (mvertes)
  • Fix handling internal dependency on self. #121 (mvertes)

0.6.11 (2017-02-09)

Full Changelog

Merged pull requests:

0.6.10 (2017-02-09)

Full Changelog

Merged pull requests:

  • Add node 6 target for travis, use system default memory settings #119 (mvertes)
  • Allow to run on windows #118 (mvertes)

0.6.9 (2017-01-02)

Full Changelog

Merged pull requests:

  • textFile: add a maxFiles option, to limit the number of files to process #117 (mvertes)
  • Increase number of streams over skale protocol #116 (mvertes)
  • stream() action: add option to terminate context on stream end #115 (mvertes)
  • doc: Fix TOC link #114 (mvertes)

0.6.8 (2016-12-14)

Full Changelog

Fixed bugs:

  • In distributed mode, temporary files are not deleted at end of program #110

Merged pull requests:

  • Advertise benchmark in README #113 (mvertes)
  • server, worker-controller: add current statistics #112 (mvertes)
  • Fix bug #110 where temporary files were not deleted at end of task in… #111 (mvertes)
  • use SKALE_WORKERS to set number of workers in distributed mode, as in… #109 (mvertes)
  • update dependencies #108 (mvertes)
  • Doc: add a section on core concepts, label shuffle transforms #107 (mvertes)
  • Refactor internal Task API, to propagate env and dependencies. In pro… #106 (mvertes)

0.6.7 (2016-11-22)

Full Changelog

Merged pull requests:

  • Document standalone and distributed modes #105 (mvertes)
  • Distributed mode: Implement peer-to-peer through HTTP for shuffle data transfer #104 (mvertes)
  • Improve traces and file transfer. #103 (mvertes)
  • Improve task scheduling #102 (mvertes)
  • Fix mocha dependency. Skip yarn files. #101 (mvertes)

0.6.6 (2016-11-04)

Full Changelog

Merged pull requests:

  • Task serialization is now correct in all cases. #100 (mvertes)
  • Optimize task data transfer by sending sparse datasets. #99 (mvertes)
  • Preliminary work for optimization of task data transfer #98 (mvertes)
  • improve getReadStream #97 (mvertes)
  • distributed mode: fixes in compressed transfers #96 (mvertes)
  • Improve distributed mode #95 (mvertes)
  • distributed mode: protocol debug traces enabled with SKALE_DEBUG=3 #94 (mvertes)
  • sc.textFile: fix handling of S3 directories #93 (mvertes)
  • Fix log in distributed worker #92 (mvertes)

0.6.5 (2016-10-23)

Full Changelog

Merged pull requests:

  • bin/worker.js: fix typo in dependencies #91 (mvertes)
  • Debug traces sent to stderr instead of stdout #90 (mvertes)
  • save to S3: extend http timeout to 1h #89 (mvertes)

0.6.4 (2016-10-11)

Full Changelog

Merged pull requests:

  • dataset stream and save: preserve structure #88 (mvertes)
  • textFile: add capability to handle single AWS S3 files, gzipped or not. #87 (mvertes)

0.6.3 (2016-10-08)

Full Changelog

Merged pull requests:

  • Add new stream action, which allows to stream out a dataset #86 (mvertes)

0.6.2 (2016-09-14)

Full Changelog

Closed issues:

  • bin/server doesn't use nworker parameter #82

Merged pull requests:

  • Fix save() action to export a dataset to filesystem or S3 #84 (mvertes)
  • Remove unused nworker command line parameter #83 (mark-bradshaw)
  • Minor grammar update. #80 (mark-bradshaw)
  • Add streaming source from AWS S3 #78 (mvertes)
  • local worker: handle master disconnect #77 (mvertes)
  • Increase buffer size of zlib from 16kB to 64kB. Better performances #76 (mvertes)
  • textFile now supports directory as dataset source #75 (mvertes)
  • Coding style change. We now use the same as NodeJS core. #74 (mvertes)

0.6.1 (2016-07-05)

Full Changelog

Merged pull requests:

  • Important improvements in task serialization, scheduling and debugging #73 (mvertes)
  • save: reset file at init #72 (mvertes)
  • Add SKALE_MEMORY env variable to set worker max memory #71 (mvertes)
  • spillToDisk: do not duplicate memory when writing to disk #70 (mvertes)

0.6.0 (2016-06-24)

Full Changelog

Merged pull requests:

  • stop testing node-0.10 #69 (mvertes)
  • add new source gzipFile() to process gzipped text files #68 (mvertes)
  • New action Dataset#save() to save dataset content in text file. #67 (mvertes)
  • Pass user options and worker context to reduce(), aggregate() and forEach() callbacks. #66 (mvertes)
  • Serverless version, direct IPCs between workers and master #65 (mvertes)
  • code cleaning #64 (mvertes)
  • improve tmp dir handling. Add env SKALE_TMP #63 (mvertes)
  • remove spurious test file #62 (mvertes)

0.5.3 (2016-05-17)

Full Changelog

Closed issues:

  • skale-engine can not be used as a dependency in a Meteor project #56
  • Get rid of direct eval #53
  • Use mkdirp.sync instead of lib/mkdir.js #51

Merged pull requests:

  • rename lib/sizeof.js in lib/rough-sizeof.js as discussed in #52 #60 (mvertes)
  • Use indirect eval, strenghten regexp which match arrow function definition; fix #53 #59 (mvertes)
  • remove lib/mkdir.js and use external mkdirp module. Fix #51 #58 (mvertes)
  • Ensure compatibility with node back to 0.10, fix #56 #57 (mvertes)
  • better use of console.log() #50 (mvertes)
  • ds.filter(): rename internal filter member into _filter to avoid coll… #49 (mvertes)

0.5.2 (2016-05-04)

Full Changelog

Merged pull requests:

  • Fix previous changes incompatible with node4-LTS #48 (mvertes)

0.5.1 (2016-05-04)

Full Changelog

Merged pull requests:

0.5.0 (2016-05-02)

Full Changelog

Merged pull requests:

  • benchmark: update to new action syntax #43 (mvertes)
  • Remove toArray(). Fix examples #42 (mvertes)
  • All actions take an optional callback and return promises instead of … #41 (mvertes)
  • simplify JS benchmark code to make it almost identical to python version #40 (mvertes)
  • add benchmark #38 (mvertes)

0.4.5 (2016-04-25)

Full Changelog

Merged pull requests:

  • Provide a faster sizeof, without external dependency #37 (mvertes)
  • code cleaning #36 (mvertes)
  • fix broken arrow function example #35 (mvertes)
  • Command line option to set max memory per worker #34 (mvertes)
  • master: set process title to ease monitoring #33 (mvertes)

0.4.4 (2016-04-20)

Full Changelog

Merged pull requests:

0.4.3 (2016-04-20)

Full Changelog

Merged pull requests:

  • doc: various fixes #31 (mvertes)
  • Dataset: provide a better RNG. Expose Random and Poisson classes. #30 (mvertes)
  • On workers: evict a partition if going out of memory #29 (mvertes)
  • Handle connection error to terminate a master if a worker fails #28 (mvertes)

0.4.2 (2016-04-17)

Full Changelog

Merged pull requests:

0.4.1 (2016-04-07)

Full Changelog

Merged pull requests:

0.4.0 (2016-04-04)

Full Changelog

Merged pull requests:

  • All actions now return a readable stream #6 (mvertes)

0.3.5 (2016-04-03)

Full Changelog

Fixed bugs:

  • cartesian incorrect output #4

Merged pull requests:

  • lib/dataset: cartesian.iterate(): fix partition index computation. Fix #4 #5 (mvertes)
  • .travis.yml: set a fixed number of workers (4) #3 (mvertes)

0.3.4 (2016-04-01)

Full Changelog

Fixed bugs:

  • examples/core/parallelize.js fails with 2 workers #2

0.3.3 (2016-03-25)

Full Changelog

0.3.2 (2016-03-23)

Merged pull requests:

* This Change Log was automatically generated by github_changelog_generator