From 0e7591b34be2f133b8017dbcf416cfad557fc2ae Mon Sep 17 00:00:00 2001 From: John Jackson Date: Mon, 3 Aug 2015 17:37:52 -0700 Subject: [PATCH 1/2] Slimmed-down Release Notes. --- .../reference-manual/source/release-notes.rst | 213 ------------------ 1 file changed, 213 deletions(-) diff --git a/cdap-docs/reference-manual/source/release-notes.rst b/cdap-docs/reference-manual/source/release-notes.rst index 4d269570fdea..d489934fd053 100644 --- a/cdap-docs/reference-manual/source/release-notes.rst +++ b/cdap-docs/reference-manual/source/release-notes.rst @@ -251,22 +251,6 @@ New Features Improvements ------------ -- `CDAP-10 `__ - - Remove caching of class loaders in dataset accessor when the dataset framework does not - return conflicting classloaders for any dataset. - -- `CDAP-37 `__ - - Inject hconf in AbstractSparkContext by guice module ConfigModule. - -- `CDAP-362 `__ - - Enabled Spark job listing and job page in Distributed mode. - -- `CDAP-380 `__ - - CDAP Flow to use configurer style API. - -- `CDAP-419 `__ - - Added automation and restructuring of the building of documentation. - - `CDAP-593 `__ - Spark no longer determines the mode through MRConfig.FRAMEWORK_NAME. @@ -282,15 +266,6 @@ Improvements - `CDAP-1132 `__ - Improved the CDAP Doc Search Result Sorting. -- `CDAP-1141 `__ - - Automated running CDAP integration tests using cdap-integration-test. - -- `CDAP-1143 `__ - - Removed the AuthenticatedHttpHandler. - -- `CDAP-1175 `__ - - Resolved Spark classpath issues in distributed mode. - - `CDAP-1416 `__ - Added links to upper level pages on Docs. @@ -300,18 +275,6 @@ Improvements - `CDAP-1583 `__ - Refactored InMemoryWorkerRunner and ServiceProgramRunnner after ServiceWorkers were removed. -- `CDAP-1615 `__ - - Create HBase 1.0 compatibility module. - -- `CDAP-1801 `__ - - Bumped the version of hbase in cdap-data-fabric to use hbase98. - -- `CDAP-1806 `__ - - Added Spark job completion status for empty dummy programs. - -- `CDAP-1912 `__ - - Changed Examples (Purchase, SportResults) to use Writable instead of JSON as mapper output. - - `CDAP-1918 `__ - Switched to using the Spark 1.3.0 release. @@ -321,12 +284,6 @@ Improvements - `CDAP-2007 `__ - CLI output for "call service" is rendered in a copy-pastable manner. -- `CDAP-2058 `__ - - For enabling multiple runs of a Program, instance change of Programs do not change the specification. - -- `CDAP-2288 `__ - - Tested CDAP with the YARN Fair Scheduler. - - `CDAP-2310 `__ - Kafka Source now able to apply a Schema to the Payload received. @@ -336,36 +293,18 @@ Improvements - `CDAP-2422 `__ - Removed redundant catch blocks in AdapterHttpHandler. -- `CDAP-2448 `__ - - Added unit test for `CDAP-2202 `__. - - `CDAP-2455 `__ - Version in CDAP-UI footer is dynamic. -- `CDAP-2472 `__ - - A check for empty strings in addition to null added in ETL Plugins. - -- `CDAP-2479 `__ - - Avoiding duplicate staging repo creation for release builds in nexus. - - `CDAP-2482 `__ - Reduced excessive capitalisation in documentation. -- `CDAP-2492 `__ - - Upgrade to twill 0.6.0-incubating release. - -- `CDAP-2527 `__ - - Removed @beta from Filesets. - - `CDAP-2531 `__ - Adapter details made available through CDAP-UI. - `CDAP-2539 `__ - Added a build identifier (branch, commit) in header of Documentation HTML pages. -- `CDAP-2548 `__ - - Added upgrade for dataset type change for "usage.registry". - - `CDAP-2552 `__ - Documentation Build script now flags errors. @@ -432,57 +371,15 @@ Improvements - `CDAP-2697 `__ - CDAP-UI: added tab names during adapter creation. -- `CDAP-2704 `__ - - Support for Spark ML with CDAP in distributed mode. - - `CDAP-2716 `__ - CDAP-UI: when creating an adapter, the tabbing order shows correctly. - `CDAP-2733 `__ - Implemented a TimeParitionedFileSet source. -- `CDAP-2735 `__ - - As DBSource doesn't use Table Name, removed it from Plugin configuration. - -- `CDAP-2795 `__ - - Dropped support for Hbase-0.94. - -- `CDAP-2797 `__ - - Fixed starting KafkaServer after ZkClientService fails in MapR. - - `CDAP-2811 `__ - Improved Hive version detection. -- `CDAP-2835 `__ - - Upgraded Apache Twill dependencies to handle MapRFS. - -- `CDAP-2840 `__ - - Implemented upgrade for PartitionedFileSetDataset to 3.1.0. - -- `CDAP-2842 `__ - - Closed instances of ProgramClassLoader and PluginClassLoader. - -- `CDAP-2846 `__ - - Persisted artifact contents and metadata. - -- `CDAP-2852 `__ - - Fixed running MapReduce program throwing exception Cannot initialize Cluster. - -- `CDAP-2868 `__ - - Abstracted use of changed HBase client APIs to compat modules. - -- `CDAP-2873 `__ - - Added user stories and design for adding System Service instances restart. - -- `CDAP-2874 `__ - - Modified system/services REST endpoints to handle new requests for restart instances. - -- `CDAP-2875 `__ - - Added in-memory flow for restarting the system services instances. - -- `CDAP-2876 `__ - - Integrated with updated Apache Twill to add support for distributed mode. - - `CDAP-2884 `__ - Treat base paths that start with "/" as absolute in the file system. An absolute base path for a (Partitioned)FileSet was interpreted as relative to the namespace's data @@ -496,44 +393,12 @@ Improvements FileSet is dropped or truncated, and it will not allow adding or deleting files or partitions. In other words, the FileSet is read-only. -- `CDAP-2886 `__ - - Upgrade tool to fix the base path of existing file sets. - -- `CDAP-2887 `__ - - Documented new file set properties. - -- `CDAP-2907 `__ - - Support for CDH HBase 1.0. - -- `CDAP-2914 `__ - - Removed Apache snapshot repo from repository list after Twill release. - -- `CDAP-2918 `__ - - Tested and fixed Explore on insecure and secure clusters with Hive 0.12, 0.13, 0.14, - 1.0, and 1.1. - - `CDAP-2921 `__ - Removed backward-compatibility for pre-2.8 TPFS. -- `CDAP-2931 `__ - - Modified cdap-master startup script to include Spark in classpath. - - `CDAP-2938 `__ - Implemented new ETL application template creation. -- `CDAP-2949 `__ - - Documented how Intellij fails to resolve methods that are imported implicitly while - writing Spark program in Scala. - -- `CDAP-2960 `__ - - Added CDAP integration test for explore. - -- `CDAP-2964 `__ - - Added CDAP integration test for Spark programs. - -- `CDAP-2968 `__ - - Extended TestBase to allow passing config when creating apps. - - `CDAP-2983 `__ - Spark program runner now calls onFailure() of the DatasetOutputCommitter. @@ -546,102 +411,24 @@ Improvements - `CDAP-2989 `__ - Added an example for Spark using FileSets. -- `CDAP-2993 `__ - - Documented Dataset Improvements/features. - -- `CDAP-2994 `__ - - Added example app for PartitionFileSet Partition consumer. - -- `CDAP-3015 `__ - - Added documentation for Workflow Token. - -- `CDAP-3016 `__ - - Tweaked Hadoop and HBase configurations to reduce threads in unit tests. - - `CDAP-3018 `__ - Updated workflow guides for workflow token. - `CDAP-3028 `__ - Improved the system service restart endpoint to handle illegal instance IDs and "service not available". -- `CDAP-3033 `__ - - Fixed integration test failures. - - `CDAP-3053 `__ - Added schema javadocs that explain how to write the schema to JSON. -- `CDAP-3068 `__ - - CDAP-UI: reverted to old design & positioning of Namespaces dropdown. - -- `CDAP-3074 `__ - - Updated startup script Java version checks. - - `CDAP-3077 `__ - Add the ability in TableSink to find schema.row.field case-insensitively. -- `CDAP-3082 `__ - - Updated License Dependency Files. - -- `CDAP-3083 `__ - - Added details on Application Template Plugins. - -- `CDAP-3084 `__ - - Corrected an inconsistency in the documentation on TTL. - -- `CDAP-3093 `__ - - Added cdap-hbase-compat-1.0 dependency for packaging. - -- `CDAP-3094 `__ - - Updated Installation Guide for HBase 1.0. - -- `CDAP-3110 `__ - - Added functional test for Kafka realtime source plugin. - -- `CDAP-3115 `__ - - Detailed documentation for ETL plugins. - -- `CDAP-3134 `__ - - Updated documentation references to required Java versions. - -- `CDAP-3136 `__ - - Updated documentation for restart instance endpoints. - - `CDAP-3144 `__ - Changed CLI command descriptions to use consistent element case. - `CDAP-3152 `__ - Refactored ETLBatch sources and sinks. -- `CDAP-3155 `__ - - Updated installation guide with the supported distros. - -- `CDAP-3156 `__ - - Added detail to File batch source docs. - -- `CDAP-3157 `__ - - Added detail to LogParser transform. - -- `CDAP-3158 `__ - - Added detail to Amazon SQS source docs. - -- `CDAP-3159 `__ - - Added detail to cube docs. - -- `CDAP-3162 `__ - - Documented how to run Explore in a secure cluster. - -- `CDAP-3164 `__ - - Added detail to jms and kafka source docs. - -- `CDAP-3165 `__ - - Added detail to database sink docs. - -- `CDAP-3166 `__ - - Added detail to tpfs sinks docs. - -- `CDAP-3167 `__ - - Added detail to stream sink docs. - Bug Fixes --------- From 1a59acaeaabff58aa88c510835b2368ed60e1c52 Mon Sep 17 00:00:00 2001 From: John Jackson Date: Mon, 3 Aug 2015 17:42:52 -0700 Subject: [PATCH 2/2] Move up and consolidate three issues. --- .../reference-manual/source/release-notes.rst | 28 +++++++++---------- 1 file changed, 13 insertions(+), 15 deletions(-) diff --git a/cdap-docs/reference-manual/source/release-notes.rst b/cdap-docs/reference-manual/source/release-notes.rst index d489934fd053..d0569041fca1 100644 --- a/cdap-docs/reference-manual/source/release-notes.rst +++ b/cdap-docs/reference-manual/source/release-notes.rst @@ -152,8 +152,19 @@ New Features Added ability for custom actions to access the CDAP datasets and services. - `CDAP-2758 `__ - - FileSet now support existing HDFS locations. - + FileSet now support existing HDFS locations. + + Treat base paths that start with "/" as absolute in the file system. An absolute base + path for a (Partitioned)FileSet was interpreted as relative to the namespace's data + directory. Newly created FileSets interpret absolute base paths as absolute in the file + system. + + Introduced a new property for (Partitioned)FileSets name "data.external". If true, the + base path of the FileSet is assumed to be managed by some external process. That is, the + FileSet will not attempt to create the directory, it will not delete any files when the + FileSet is dropped or truncated, and it will not allow adding or deleting files or + partitions. In other words, the FileSet is read-only. + - `CDAP-2784 `__ - Added support to write to PartitionedFileSet Partition metadata from MapReduce. @@ -380,19 +391,6 @@ Improvements - `CDAP-2811 `__ - Improved Hive version detection. -- `CDAP-2884 `__ - - Treat base paths that start with "/" as absolute in the file system. An absolute base - path for a (Partitioned)FileSet was interpreted as relative to the namespace's data - directory. Newly created FileSets interpret absolute base paths as absolute in the file - system. - -- `CDAP-2885 `__ - - Introduced a new property for (Partitioned)FileSets name "data.external". If true, the - base path of the FileSet is assumed to be managed by some external process. That is, the - FileSet will not attempt to create the directory, it will not delete any files when the - FileSet is dropped or truncated, and it will not allow adding or deleting files or - partitions. In other words, the FileSet is read-only. - - `CDAP-2921 `__ - Removed backward-compatibility for pre-2.8 TPFS.