-
Notifications
You must be signed in to change notification settings - Fork 2
/
CHANGES.txt
9328 lines (6629 loc) · 401 KB
/
CHANGES.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Apache Solr Release Notes
Introduction
------------
Apache Solr is an open source enterprise search server based on the Apache Lucene Java
search library, with XML/HTTP and JSON APIs, hit highlighting, faceted search,
caching, replication, and a web administration interface. It runs in a Java
servlet container such as Jetty.
See http://lucene.apache.org/solr for more information.
Getting Started
---------------
You need a Java 1.7 VM or later installed.
In this release, there is an example Solr server including a bundled
servlet container in the directory named "example".
See the tutorial at http://lucene.apache.org/solr/tutorial.html
================== 4.10.4 ==================
Bug Fixes
----------------------
* SOLR-6931: We should do a limited retry when using HttpClient.
(Mark Miller, Hrishikesh Gadre, Gregory Chanan)
* SOLR-6780: Fixed a bug in how default/appends/invariants params were affecting the set
of all "keys" found in the request parameters, resulting in some key=value param pairs
being duplicated. This was noticeably affecting some areas of the code where iteration
was done over the set of all params:
- literal.* in ExtractingRequestHandler
- facet.* in FacetComponent
- spellcheck.[dictionary name].* and spellcheck.collateParam.* in SpellCheckComponent
- olap.* in AnalyticsComponent
(Alexandre Rafalovitch & hossman)
* SOLR-6426: SolrZkClient clean can fail due to a race with children nodes. (Mark Miller)
* SOLR-6457: LBHttpSolrClient: ArrayIndexOutOfBoundsException risk if counter overflows
(longkey via Noble Paul)
* SOLR-6481: CLUSTERSTATUS should check if the node hosting a replica is live when
reporting replica status (Timothy Potter)
* SOLR-6631: DistributedQueue spinning on calling zookeeper getChildren()
(Jessica Cheng Mallet, Mark Miller, Timothy Potter)
* SOLR-6579: SnapPuller Replication blocks clean shutdown of tomcat
(Philip Black-Knight via Noble Paul)
* SOLR-6763: Shard leader elections should not persist across session expiry
(Alan Woodward, Mark Miller)
* SOLR-3881: Avoid OOMs in LanguageIdentifierUpdateProcessor:
- Added langid.maxFieldValueChars and langid.maxTotalChars params to limit
input, by default 10k and 20k chars, respectively.
- Moved input concatenation to Tika implementation; the langdetect
implementation instead appends each input piece via the langdetect API.
(Vitaliy Zhovtyuk, Tomás Fernández Löbbe, Rob Tulloh, Steve Rowe)
* SOLR-6850: AutoAddReplicas makes a call to wait to see live replicas that times
out after 30 milliseconds instead of 30 seconds. (Varun Thacker via Mark Miller)
* SOLR-6839: Direct routing with CloudSolrServer will ignore the Overwrite document option.
(Mark Miller)
* SOLR-7139: Fix SolrContentHandler for TIKA to ignore multiple startDocument events.
(Chris A. Mattmann, Uwe Schindler)
* SOLR-6941: DistributedQueue#containsTaskWithRequestId can fail with NPE. (Mark Miller)
* SOLR-7011: Delete collection returns before collection is actually removed.
(Christine Poerschke via shalin)
* SOLR-6856: Restore ExtractingRequestHandler's ability to capture all HTML tags when
parsing (X)HTML. (hossman, Uwe Schindler, ehatcher, Steve Rowe)
* SOLR-6928: solr.cmd stop works only in english (john.work, Jan Høydahl, Timothy Potter)
* SOLR-7038: Validate the presence of configset before trying to create a collection.
(Anshum Gupta, Mark Miller)
* SOLR-7016: Fix bin\solr.cmd to work in a directory with spaces in the name.
(Timothy Potter, Uwe Schindler)
* SOLR-6693: bin\solr.cmd doesn't support 32-bit JRE/JDK running on Windows due to
parenthesis in JAVA_HOME. (Timothy Potter, Christopher Hewitt, Jan Høydahl)
* SOLR-7067: bin/solr won't run under bash 4.2+. (Steve Rowe)
* SOLR-7033, SOLR-5961: RecoveryStrategy should not publish any state when
closed / cancelled and there should always be a pause between recoveries
even when recoveries are rapidly stopped and started as well as when a
node attempts to become the leader for a shard.
(Mark Miller, Maxim Novikov)
* SOLR-6847: LeaderInitiatedRecoveryThread compares wrong replica's state with lirState.
(shalin)
* SOLR-7128: Two phase distributed search is fetching extra fields in GET_TOP_IDS phase.
(Pablo Queixalos, shalin)
Other Changes
----------------------
* SOLR-7147: Introduce new TrackingShardHandlerFactory for monitoring what requests
are sent to shards during tests. (hossman, shalin)
================== 4.10.3 ==================
Bug Fixes
----------------------
* SOLR-6696: bin/solr start script should not enable autoSoftCommit by default (janhoy)
* SOLR-6704: TrieDateField type drops schema properties in branch 4.10 (Tomás Fernández Löbbe)
* SOLR-6679: Remove suggester component from stock solrconfig.xml since
it caused long startup times on lage indexes even when it wasn't used.
(yonik)
* SOLR-6662: better validation when parsing command-line options that expect a value
(Timothy Potter)
* SOLR-6705: Better strategy for dealing with JVM specific options in the start
scripts; remove -XX:+AggressiveOpts and only set -XX:-UseSuperWord for Java 1.7u40
to u51. (Uwe Schindler, janhoy, hossman, Timothy Potter)
* SOLR-6738: Admin UI - Escape Data on Plugins-View (steffkes)
* SOLR-6085: Suggester crashes when prefixToken is longer than surface form (janhoy)
* SOLR-6323: ReRankingQParserPlugin cleaner paging and fix bug with fuzzy, range and other queries
that need to be re-written. (Adair Kovac, Joel Bernstein)
* SOLR-6684: Fix-up /export JSON. (Joel Bernstein)
* SOLR-6732: Fix handling of leader-initiated recovery state was String in older versions
and is now a JSON map, caused backwards compatibility issues when doing rolling upgrades of
a live cluster while indexing (Timothy Potter)
* SOLR-6781: BBoxField didn't support dynamic fields. (David Smiley)
* SOLR-6784: BBoxField's 'score' mode should have been optional. (David Smiley)
* SOLR-6510: The collapse QParser would throw a NPE when used on a DocValues field on
an empty segment/index. (Christine Poerschke, David Smiley)
* SOLR-2927: Solr does not unregister all mbeans upon exception in constructor
causing memory leaks. (tom liu, Sharath Babu, Cyrille Roy, shalin)
* SOLR-6685: ConcurrentModificationException in Overseer Status API. (shalin)
* SOLR-6706: /update/json/docs throws RuntimeException if a nested structure
contains a non-leaf float field (Noble Paul, shalin)
* SOLR-6610: Slow startup of new clusters because ZkController.publishAndWaitForDownStates
always times out. (Jessica Cheng Mallet, shalin, Noble Paul)
* SOLR-6726: better strategy for selecting the JMX RMI port based on SOLR_PORT in bin/solr
script (Timothy Potter)
* SOLR-6795: distrib.singlePass returns score even though not asked for.
(Per Steffensen via shalin)
* SOLR-6796: distrib.singlePass does not return correct set of fields for multi-fl-parameter
requests. (Per Steffensen via shalin)
* SOLR-6776: Transaction log was not flushed at the end of update requests with softCommit
specified, which could lead to data loss if the server were killed immediately after the
update finished. (Jeffery Yuan via yonik)
Other Changes
----------------------
* SOLR-6661: Adjust all example configurations to allow overriding error-prone
relative paths for solrconfig.xml <lib> references with solr.install.dir
system property; bin/solr scripts will set it appropriately. (ehatcher)
* SOLR-5864: SolrCore.reload(ConfigSet coreConfig, SolrCore prev) deprecated.
Use SolrCore.reload(ConfigSet coreConfig) instead. (Tomás Fernández Löbbe)
* SOLR-6694: Auto-detect JAVA_HOME using the Windows registry if it is not set
(janhoy, Timothy Potter)
* SOLR-6653: bin/solr script should return error code >0 when something fails
(janhoy, Timothy Potter)
* SOLR-6829: Added getter/setter for lastException in DIH's ContextImpl (ehatcher)
================== 4.10.2 ==================
Bug Fixes
----------------------
* SOLR-6509: Solr start scripts interactive mode doesn't honor -z argument (Timothy Potter)
* SOLR-6511: Fencepost error in LeaderInitiatedRecoveryThread (Timothy Potter)
* SOLR-6530: Commits under network partitions can put any node in down state.
(Ramkumar Aiyengar, Alan Woodward, Mark Miller, shalin)
* SOLR-6573: QueryElevationComponent now works with localParams in the query (janhoy)
* SOLR-6524: Collections left in recovery state after node restart because recovery sleep time
increases exponentially between retries. (Mark Miller, shalin)
* SOLR-6587: Misleading exception when creating collections in SolrCloud with bad configuration.
(Tomás Fernández Löbbe)
* SOLR-6452: StatsComponent's stat 'missing' will work on fields with docValues=true and
indexed=false (Xu Zhang via Tomás Fernández Löbbe)
* SOLR-6646: bin/solr start script fails to detect solr on non-default port and then after
30s tails wrong log file (janhoy)
* SOLR-6647: Bad error message when missing resource from ZK when parsing Schema (janhoy)
* SOLR-6545: Query field list with wild card on dynamic field fails.
(Burke Webster, Xu Zhang, shalin)
Other Changes
----------------------
* SOLR-6550: Provide simple mechanism for passing additional metadata / context about a server-side
SolrException back to the client-side (Timothy Potter)
* SOLR-6486: solr start script can have a debug flag option; use -a to set arbitrary options
(Noble Paul, Timothy Potter)
* SOLR-6549: bin/solr script should support a -s option to set the -Dsolr.solr.home property.
(Timothy Potter)
* SOLR-6529: Stop command in the start scripts should only stop the instance that it had started.
(Varun Thacker, Timothy Potter)
================== 4.10.1 ==================
Bug Fixes
----------------------
* SOLR-6425: If using the new global hdfs block cache option, you can end up
reading corrupt files on file name reuse. (Mark Miller, Gregory Chanan)
* SOLR-5814: CoreContainer reports incorrect & misleading path for solrconfig.xml
when there are loading problems (Pradeep via hossman)
* SOLR-6024: Fix StatsComponent when using docValues="true" multiValued="true"
(Vitaliy Zhovtyuk & Tomas Fernandez-Lobbe via hossman)
* SOLR-6493: Fix fq exclusion via "ex" local param in multivalued stats.field (hossman)
* SOLR-6447: bin/solr script needs to pass -DnumShards=1 for boostrapping collection1
when starting Solr in cloud mode. (Timothy Potter)
* SOLR-6501: Binary Response Writer does not return wildcard fields.
(Mike Hugo, Constantin Mitocaru, sarowe, shalin)
Other Changes
---------------------
* SOLR-6503: Removed support for parsing netcdf files in Solr Cell because
of license issues. If you need support for this format, download the parser
JAR yourself (version 4.2) and add it to contrib/extraction/lib folder:
http://www.unidata.ucar.edu/software/thredds/current/netcdf-java/
(Uwe Schindler)
================== 4.10.0 =================
Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release
Versions of Major Components
---------------------
Apache Tika 1.5 (with upgraded Apache POI 3.10.1)
Carrot2 3.9.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.6
Upgrading from Solr 4.9
----------------------
* In Solr 3.6, all primitive field types were changed to omit norms by default when the
schema version is 1.5 or greater (SOLR-3140), but TrieDateField's default was mistakenly
not changed. As of Solr 4.10, TrieDateField omits norms by default (see SOLR-6211).
* Creating a SolrCore via CoreContainer.create() no longer requires an
additional call to CoreContainer.register() to make it available to clients
(see SOLR-6170).
* CoreContainer.remove() has been removed. You should now use CoreContainer.unload() to
delete a SolrCore (see SOLR-6232).
* solr.xml parsing has been improved to better account for the expected data types of
various options. As part of this fix, additional error checking has also been added to
provide errors in the event of duplicated options, or unknown option names that may
indicate a typo. Users who have modified their solr.xml in the past and now upgrade may
get errors on startup if they have typos or unexpected options specified in their solr.xml
file. (See SOLR-5746 for more information.)
Detailed Change List
----------------------
New Features
----------------------
* SOLR-6196: The overseerstatus collection API instruments amILeader and ZK state update calls.
(shalin)
* SOLR-6069: The 'clusterstatus' API should return 'roles' information. (shalin)
* SOLR-6044: The 'clusterstatus' API should return live_nodes as well. (shalin)
* SOLR-5768: Add a distrib.singlePass parameter to make EXECUTE_QUERY phase fetch all fields
and skip GET_FIELDS. (Gregg Donovan, shalin)
* SOLR-6183: New spatial BBoxField for indexing rectangles with search support for most predicates.
It includes extra score relevancy modes in addition to distance: score=overlapRatio|area|area2D.
(David Smiley, Ryan McKinley)
* SOLR-6232: You can now unload/delete cores that have failed to initialize (Alan Woodward)
* SOLR-2245: Improvements to the MailEntityProcessor:
- Support for server-side date filtering if using GMail; requires new
dependency on the Sun Gmail Java mail extensions
- Support for using the last_index_time from the previous run as the
value for the fetchMailsSince filter.
(Peter Sturge, Timothy Potter)
* SOLR-6258: Added onRollback event handler hook to Data Import Handler (DIH).
(ehatcher)
* SOLR-6263: Add DIH handler name to variable resolver as ${dih.handlerName}. (ehatcher)
* SOLR-6216: Better faceting for multiple intervals on DV fields (Tomas Fernandez-Lobbe
via Erick Erickson)
* SOLR-6267: Let user override Interval Faceting key with LocalParams (Tomas Fernandez_Lobbe
via Erick Erickson)
* SOLR-6020: Auto-generate a unique key in schema-less example if data does not have an id field.
The UUIDUpdateProcessor was improved to not require a field name in configuration and generate
a UUID into the unique Key field.
(Vitaliy Zhovtyuk, hossman, Steve Rowe, Erik Hatcher, shalin)
* SOLR-6294: SOLR-6437: Remove the restriction of adding json by only wrapping it in an array in a
new path /update/json/docs (Noble Paul , hossman, Yonik Seeley, Steve Rowe)
* SOLR-6302: UpdateRequestHandlers are registered implicitly /update ,
/update/json, /update/csv , /update/json/docs (Noble Paul)
* SOLR-6318: New "terms" QParser for efficiently filtering documents by a list of values. For
many values, it's more appropriate than a boolean query. (David Smiley)
* SOLR-6283: Add support for Interval Faceting in SolrJ. (Tomás Fernández Löbbe)
* SOLR-6304 : JsonLoader should be able to flatten an input JSON to multiple docs (Noble Paul)
* SOLR-2894: Distributed query support for facet.pivot (Dan Cooper, Erik Hatcher, Chris Russell,
Andrew Muldowney, Brett Lucey, Mark Miller, hossman)
* SOLR-5656: Add autoAddReplicas feature for shared file systems. (Mark Miller, Gregory Chanan)
* SOLR-5244: Exporting Full Sorted Result Sets (Erik Hatcher, Joel Bernstein)
* SOLR-3617: bin/solr and bin/solr.cmd scripts for starting, stopping, and running Solr examples
(Timothy Potter)
* SOLR-6233: Provide basic command line tools for checking Solr status and health.
(Timothy Potter)
Bug Fixes
----------------------
* SOLR-6095 : SolrCloud cluster can end up without an overseer with overseer roles (Noble Paul, Shalin Mangar)
* SOLR-6165: DataImportHandler should write BigInteger and BigDecimal values as strings.
(Anand Sengamalai via shalin)
* SOLR-6189: Avoid publishing the state as down if the node is not live when determining
if a replica should be in leader-initiated recovery. (Timothy Potter)
* SOLR-6197: The MIGRATE collection API doesn't work when legacyCloud=false is set
in cluster properties. (shalin)
* SOLR-6206: The migrate collection API fails on retry if temp collection already exists.
(shalin)
* SOLR-6072: The 'deletereplica' API should remove the data and instance directory by default.
(shalin)
* SOLR-6211: TrieDateField doesn't default to omitNorms=true. (Michael Ryan, Steve Rowe)
* SOLR-6159: A ZooKeeper session expiry during setup can keep LeaderElector from joining elections.
(Steven Bower, shalin)
* SOLR-6223: SearchComponents may throw NPE when using shards.tolerant and there is a failure
in the 'GET_FIELDS/GET_HIGHLIGHTS/GET_DEBUG' phase. (Tomás Fernández Löbbe via shalin)
* SOLR-6180: Callers of ManagedIndexSchema mutators should hold the schemaUpdateLock.
(Gregory Chanan via Steve Rowe)
* SOLR-6229: Make SuggestComponent return 400 instead of 500 for bad dictionary selected in request.
(Tomás Fernández Löbbe via shalin)
* SOLR-6235: Leader initiated recovery should use coreNodeName instead of coreName to avoid marking
all replicas having common core name as down. (shalin)
* SOLR-6208: JettySolrRunner QueuedThreadPool's configuration code is never executed. (dweiss via shalin)
* SOLR-6245: Socket and Connection configuration are ignored in HttpSolrServer when passing in HttpClient.
(Patanachai Tangchaisin, shalin)
* SOLR-6137: Schemaless concurrency improvements:
- Fixed an NPE when reloading a managed schema with no dynamic copy fields
- Moved parsing and schema fields addition to after the distributed phase
- AddSchemaFieldsUpdateProcessor now uses a fixed schema rather than always
retrieving the latest, and holds the schema update lock through the entire
schema swap-out process
(Gregory Chanan via Steve Rowe)
* SOLR-6136: ConcurrentUpdateSolrServer includes a Spin Lock (Brandon Chapman, Timothy Potter)
* SOLR-6257: More than two "!"-s in a doc ID throws an
ArrayIndexOutOfBoundsException when using the composite id router.
(Steve Rowe)
* SOLR-5746: Bugs in solr.xml parsing have been fixed to more correctly deal with the various
datatypes of options people can specify, additional error handling of duplicated/unidentified
options has also been added. (Maciej Zasada, hossman)
* SOLR-5847: Fixed data import abort button in admin UI. (ehatcher)
* SOLR-6264: Distributed commit and optimize are executed serially across all
replicas. (Mark Miller, Timothy Potter)
* SOLR-6163: Correctly decode special characters in managed stopwords and synonym endpoints.
(Vitaliy Zhovtyuk, Timo Schmidt via Timothy Potter)
* SOLR-6336: DistributedQueue can easily create too many ZooKeeper Watches.
(Ramkumar Aiyengar via Mark Miller)
* SOLR-6347: DELETEREPLICA throws a NPE while removing the last Replica in a Custom
sharded collection. (Anshum Gupta)
* SOLR-6062: Fix undesirable edismax query parser effect (introduced in SOLR-2058) in how phrase queries
generated from pf, pf2, and pf3 are merged into the main query. (Michael Dodsworth via ehatcher)
* SOLR-6372: HdfsDirectoryFactory should use supplied Configuration for communicating with secure kerberos.
(Gregory Chanan via Mark Miller)
* SOLR-6284: Fix NPE in OCP when non-existent sliceId is used for a
deleteShard request (Ramkumar Aiyengar via Anshum Gupta)
* SOLR-6380: Added missing context info to log message if IOException occurs in processing tlog
(Steven Bower via hossman)
* SOLR-6383: RegexTransformer returns no results after replaceAll if regex does not match a value.
(Alexander Kingson, shalin)
* SOLR-6387: Add better error messages throughout Solr and supply a work around for
Java bug #8047340 to SystemInfoHandler: On Turkish default locale, some JVMs fail
to fork on MacOSX, BSD, AIX, and Solaris platforms. (hossman, Uwe Schindler)
* SOLR-6338: coreRootDirectory requires trailing slash, or SolrCloud cores are created in wrong location.
(Primož Skale via Erick Erickson)
* SOLR-6314: Facet counts duplicated in the response if specified more than once on the request.
(Vamsee Yarlagadda, Erick Erickson)
* SOLR-6378: Fixed example/example-DIH/ issues with "tika" and "solr" configurations, and tidied up README.txt
(Daniel Shchyokin via ehatcher)
* SOLR-6393: TransactionLog replay performance on HDFS is very poor. (Mark Miller)
* SOLR-6268: HdfsUpdateLog has a race condition that can expose a closed HDFS FileSystem instance and should
close its FileSystem instance if either inherited close method is called. (Mark Miller)
* SOLR-6089: When using the HDFS block cache, when a file is deleted, its underlying data entries in the
block cache are not removed, which is a problem with the global block cache option.
(Mark Miller, Patrick Hunt)
* SOLR-6402: OverseerCollectionProcessor should not exit for ZooKeeper ConnectionLoss.
(Jessica Cheng via Mark Miller)
* SOLR-6405: ZooKeeper calls can easily not be retried enough on ConnectionLoss.
(Jessica Cheng, Mark Miller)
* SOLR-6410: Ensure all Lookup instances are closed via CloseHook
(hossman, Areek Zillur, Ryan Ernst, Dawid Weiss)
Optimizations
---------------------
* LUCENE-5803: Solr's schema now uses DelegatingAnalyzerWrapper. This uses less heap
for cached TokenStreamComponents because it caches per FieldType not per Field, so
indexes with many fields of same type just use one TokenStream per thread.
(Shay Banon, Uwe Schindler, Robert Muir)
* SOLR-6259: Reduce CPU usage by avoiding repeated costly calls to Document.getField inside
DocumentBuilder.toDocument for use-cases with large number of fields and copyFields.
(Steven Bower via shalin)
* SOLR-5968: BinaryResponseWriter fetches unnecessary stored fields when only pseudo-fields
are requested. (Gregg Donovan via shalin)
* SOLR-6261: Run ZooKeeper watch event callbacks in parallel to the ZooKeeper
event thread. (Ramkumar Aiyengar via Mark Miller)
Other Changes
---------------------
* SOLR-6173: Fixed wrong failure message in TestDistributedSearch. (shalin)
* SOLR-5902: Corecontainer level mbeans are not exposed (noble)
* SOLR-6194: Allow access to DataImporter and DIHConfiguration from DataImportHandler.
(Aaron LaBella via shalin)
* SOLR-6170: CoreContainer.preRegisterInZk() and CoreContainer.register() commands
are merged into CoreContainer.create(). (Alan Woodward)
* SOLR-6171: Remove unused SolrCores coreNameToOrig map (Alan Woodward)
* SOLR-5596: Set system property zookeeper.forceSync=no for Solr test cases. (shalin)
* SOLR-2853: Add a unit test for the case when "spellcheck.maxCollationTries=0" (James Dyer)
* SOLR-6240: Removed unused coreName parameter in ZkStateReader.getReplicaProps. (shalin)
* SOLR-6241: Harden the HttpPartitionTest. (shalin)
* SOLR-6228: Fixed bug in TestReplicationHandler.doTestIndexAndConfigReplication. (shalin)
* SOLR-6120: On Windows, when the war is not extracted, the zkcli.bat script
will print a helpful message indicating that the war must be unzipped instead
of a java error about a missing class. (shalin, Shawn Heisey)
* SOLR-6179: Better strategy for handling empty managed data to avoid spurious
warning messages in the logs. (Timothy Potter)
* SOLR-6232: CoreContainer.remove() replaced with CoreContainer.unload(). A call to
unload will also close the core.
* SOLR-3893: DIH should not depend on mail.jar,activation.jar (Timothy Potter, Steve Rowe)
* SOLR-6252: A couple of small improvements to UnInvertedField class.
(Vamsee Yarlagadda, Gregory Chanan, Mark Miller)
* SOLR-3345: BaseDistributedSearchTestCase should always ignore QTime.
(Vamsee Yarlagadda, Benson Margulies via Mark Miller)
* SOLR-6270: Increased timeouts for MultiThreadedOCPTest. (shalin)
* SOLR-6274: UpdateShardHandler should log the params used to configure its
HttpClient. (Ramkumar Aiyengar via Mark Miller)
* SOLR-6194: Opened up "public" access to DataSource, DocBuilder, and EntityProcessorWrapper
in DIH. (Aaron LaBella via ehatcher)
* SOLR-6269: Renamed "rollback" to "error" in DIH internals, including renaming onRollback
to onError introduced in SOLR-6258. (ehatcher)
* SOLR-3622: When using DIH in SolrCloud-mode, rollback will no longer be called when
an error occurs. (ehatcher)
* SOLR-6231: Increased timeouts and hardened the RollingRestartTest. (Noble Paul, shalin)
* SOLR-6290: Harden and speed up CollectionsAPIAsyncDistributedZkTest. (Mark Miller, shalin)
* SOLR-6281: Made PostingsSolrHighlighter more configurable via subclass extension. (David Smiley)
* SOLR-6309: Increase timeouts for AsyncMigrateRouteKeyTest. (shalin)
* SOLR-2168: Added support for facet.missing in /browse field and pivot faceting. (ehatcher)
* SOLR-4702: Added support for multiple spellcheck collations to /browse UI. (ehatcher)
* SOLR-5664: Added support for multi-valued field highlighting in /browse UI. (ehatcher)
* SOLR-6313: Improve SolrCloud cloud-dev scripts. (Mark Miller, Vamsee Yarlagadda)
* SOLR-6360: Remove bogus "Content-Charset" header in HttpSolrServer. (Michael Ryan,
Uwe Schindler)
* SOLR-6362: Fix bug in TestSqlEntityProcessorDelta. (James Dyer)
* SOLR-6388: Force upgrade of Apache POI dependency in Solr Cell to version
3.10.1 to fix CVE-2014-3529 and CVE-2014-3574. (Uwe Schindler)
* SOLR-6391: Improve message for CREATECOLLECTION failure due to missing
numShards (Anshum Gupta)
================== 4.9.0 ==================
Versions of Major Components
---------------------
Apache Tika 1.5
Carrot2 3.9.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.6
Upgrading from Solr 4.8
----------------------
* Support for DiskDocValuesFormat (ie: fieldTypes configured with docValuesFormat="Disk")
has been removed due to poor performance. If you have an existing fieldTypes using
DiskDocValuesFormat please modify your schema.xml to remove the 'docValuesFormat'
attribute, and optimize your index to rewrite it into the default codec, prior to
upgrading to 4.9. See LUCENE-5761 for more details.
Detailed Change List
----------------------
New Features
----------------------
* SOLR-5999: Add checkIntegrityAtMerge support to solrconfig.xml.
(Varun Thacker via Ryan Ernst)
* SOLR-6043: Add ability to set http headers in solr response
(Tomás Fernández Löbbe via Ryan Ernst)
* SOLR-5973: Pluggable Ranking Collectors and Merge Strategies
(Joel Bernstein)
* SOLR-6108: Add support for 'addreplica' Collection API in SolrJ. (shalin)
* SOLR-5468: Allow a client application to request the minium achieved
replication factor for an update request (single or batch) by sending
an optional parameter "min_rf". (Timothy Potter)
* SOLR-6088: Add query re-ranking with the ReRankingQParserPlugin
(Joel Bernstein)
* SOLR-5285: Added a new [child ...] DocTransformer for optionally including
Block-Join descendant documents inline in the results of a search. This works
independent of whether the search itself is a block-join related query and is
supported by he xml, json, and javabin response formats.
(Varun Thacker via hossman)
* SOLR-6150: Add new AnalyticsQuery to support pluggable analytics
(Joel Bernstein)
* SOLR-6125: Allow SolrIndexWriter to close without waiting for merges
(Christine Poerschke via Alan Woodward)
* SOLR-6064: DebugComponent track output should be returned as a JSON
object rather than a list (Christine Poerschke, Alan Woodward)
Bug Fixes
----------------------
* SOLR-5956: Use coreDescriptor.getInstanceDir() instead of getRawInstanceDir()
in the SnapShooter to avoid problems when solr.solr.home is a symbolic link.
(Timothy Potter)
* SOLR-6002: Fix a couple of ugly issues around SolrIndexWriter close and
rollback as well as how SolrIndexWriter manages its ref counted directory
instance. (Mark Miller, Gregory Chanan)
* SOLR-6015: Better way to handle managed synonyms when ignoreCase=true
(Timothy Potter)
* SOLR-6104: The 'addreplica' Collection API does not support 'async' parameter.
(shalin)
* SOLR-6101: Shard splitting doesn't work when legacyCloud=false is set in
cluster properties. (shalin)
* SOLR-6111: The 'deleteshard' collection API should be able to delete a shard
in 'construction' state. (shalin)
* SOLR-6118: 'expand.sort' didn't support function queries. (David Smiley)
* SOLR-6120: zkcli.sh should expand solr.war automatically instead of throwing
ClassNotFoundException. (sebastian badea, shalin)
* SOLR-6149: Specifying the query value without any index value does not work in
Analysis browser. (Aman Tandon, shalin)
* SOLR-6145: Fix Schema API optimistic concurrency by moving it out of
ManagedIndexSchema.add(Copy)Fields() into the consumers of those methods:
CopyFieldCollectionResource, FieldCollectionResource, FieldResource,
and AddSchemaFieldsUpdateProcessorFactory.
(Gregory Chanan, Alexey Serba, Steve Rowe)
* SOLR-6146: Incorrect configuration such as wrong chroot in zk server address can
cause CloudSolrServer to leak resources. (Jessica Cheng, Varun Thacker, shalin)
* SOLR-6158: Relative configSetBase directories were resolved relative to the
container CWD, rather than solr.home. (Simon Endele, Alan Woodward)
* SOLR-5426: Fixed a bug in ReverseWildCardFilter that could cause
InvalidTokenOffsetsException when highlighting. (Uwe Schindler, Arun Kumar, via hossman)
* SOLR-6175: DebugComponent throws NPE on shard exceptions when using shards.tolerant.
(Tomás Fernández Löbbe via shalin)
* SOLR-6129: DateFormatTransformer doesn't resolve dateTimeFormat. (Aaron LaBella via shalin)
* SOLR-6164: Copy Fields Schema additions are not distributed to other nodes.
(Gregory Chanan via Steve Rowe)
* SOLR-6160: An error was sometimes possible if a distributed search included grouping
with group.facet, faceting on facet.field and either facet.range or facet.query.
(David Smiley)
* SOLR-6182: Data stored by the RestManager could not be reloaded after core restart, causing
the core to fail to load; cast the data loaded from storage to the correct data type.
(Timothy Potter)
Other Changes
---------------------
* SOLR-5980: AbstractFullDistribZkTestBase#compareResults always returns false
for shouldFail. (Mark Miller, Gregory Chanan)
* SOLR-5987: Add "collection" to UpdateParams. (Mark Miller, Greg Solovyev)
* SOLR-3862: Add remove" as update option for atomically removing a value
from a multivalued field (Jim Musli, Steven Bower, Alaknantha via Erick Erickson)
* SOLR-5974: Remove ShardDoc.score and use parent's ScoreDoc.score.
(Tomás Fernández Löbbe via Ryan Ernst)
* SOLR-6025: Replace mentions of CommonsHttpSolrServer with HttpSolrServer and
StreamingUpdateSolrServer with ConcurrentUpdateSolrServer. (Ahmet Arslan via shalin)
* SOLR-6013: Fix method visibility of Evaluator, refactor DateFormatEvaluator for
extensibility. (Aaron LaBella via shalin)
* SOLR-6022: Deprecate getAnalyzer() in IndexField and FieldType, and add getIndexAnalyzer().
(Ryan Ernst)
* SOLR-3671: Fix DIHWriter interface usage so users may implement writers that output
documents to a location external to Solr (ex. a NoSql db). (Roman Chyla via James Dyer)
* SOLR-5340: Add support for named snapshots (Varun Thacker via Noble Paul)
* SOLR-5495: Recovery strategy for leader partitioned from replica case. Hardening
recovery scenarios after the leader receives an error trying to forward an
update request to a replica. (Timothy Potter)
* SOLR-6116: Refactor DocRouter.getDocRouter to accept routerName as a String. (shalin)
* SOLR-6026: REQUESTSTATUS Collection API now also checks for submitted tasks which are
yet to begin execution.
* SOLR-6067: Refactor duplicate Collector code in SolrIndexSearcher
(Christine Poerschke via hossman)
* SOLR-5940: post.jar reports back detailed error in case of error responses.
(Sameer Maggon, shalin, Uwe Schindler)
* SOLR-6161: SolrDispatchFilter should throw java.lang.Error back even if wrapped in
another exception. (Miklos Christine via shalin)
* SOLR-6153: ReplicationHandler backup response format should contain backup name.
(Varun Thacker via shalin)
* SOLR-6169: Remove broken handleAlias action in CoreAdminHandler (Alan
Woodward)
* SOLR-6128: Removed deprecated analysis factories and fieldTypes from the example
schema.xml (hossman)
* SOLR-5868: HttpClient should be configured to use ALLOW_ALL_HOSTNAME hostname
verifier to simplify SSL setup. (Steve Davids via Mark Miller)
Optimizations
----------------------
* SOLR-5681: Make the processing of Collection API calls multi-threaded.
(Anshum Gupta, shalin, Noble Paul)
Build
---------------------
* SOLR-6006: Separate test and compile scope dependencies in the Solrj and
Solr contrib ivy.xml files, so that the derived Maven dependencies get
filled out properly in the corresponding POMs. (Steven Scott, Steve Rowe)
* SOLR-6130: Added com.uwyn:jhighlight dependency to, and removed asm:asm
dependency from the extraction contrib - dependencies weren't fully
upgraded with the Tika 1.4->1.5 upgrade (SOLR-5763). (Steve Rowe)
================== 4.8.1 ==================
Bug Fixes
----------------------
* SOLR-5904: ElectionContext can cancel an election when it should not if there
was an exception while trying to register as the leader.
(Mark Miller, Alan Woodward)
* SOLR-5993: ZkController can warn about shard leader conflict even after the conflict
is resolved. (Gregory Chanan via shalin)
* SOLR-6017: Fix SimpleQParser to use query analyzer
(Ryan Ernst)
* SOLR-6029: CollapsingQParserPlugin throws ArrayIndexOutOfBoundsException
if elevated doc has been deleted from a segment. (Greg Harris, Joel Bernstein)
* SOLR-6030: Use System.nanoTime() instead of currentTimeInMills() in LRUCache.warm.
(Tomás Fernández Löbbe via shalin)
* SOLR-6037: Fixed incorrect max/sum/stddev for Date fields in StatsComponent
(Brett Lucey, hossman)
* SOLR-6023: FieldAnalysisRequestHandler throws NPE if no parameters are supplied.
(shalin)
* SOLR-5090: SpellCheckComponent sometimes throws NPE if
"spellcheck.alternativeTermCount" is set to zero (James Dyer).
* SOLR-6039: fixed debug output when no results in response
(Tomás Fernández Löbbe, hossman)
* SOLR-6035: CloudSolrServer directUpdate routing should use getCoreUrl.
(Marvin Justice, Joel Bernstein)
================== 4.8.0 ==================
Versions of Major Components
---------------------
Apache Tika 1.5
Carrot2 3.9.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.6
Upgrading from Solr 4.7
----------------------
* In previous versions of Solr, Terms that exceeded Lucene's MAX_TERM_LENGTH were
silently ignored when indexing documents. Beginning with Solr 4.8, a document
an error will be generated when attempting to index a document with a term
that is too large. If you wish to continue to have large terms ignored,
use "solr.LengthFilterFactory" in all of your Analyzers. See LUCENE-5472 for
more details.
* Solr 4.8 requires Java 7 or greater, Java 8 is verified to be
compatible and may bring some performance improvements. When using
Oracle Java 7 or OpenJDK 7, be sure to not use the GA build 147 or
update versions u40, u45 and u51! We recommend using u55 or later.
An overview of known JVM bugs can be found on
http://wiki.apache.org/lucene-java/JavaBugs
* ZooKeeper is upgraded from 3.4.5 to 3.4.6.
* <fields> and <types> tags have been deprecated. There is no longer any reason to
keep them in the schema file, they may be safely removed. This allows intermixing of
<fieldType>, <field> and <copyField> definitions if desired. Currently, these tags
are supported so either style may be implemented. TBD is whether they'll be
deprecated formally for 5.0
Detailed Change List
----------------------
System Requirements
----------------------
* LUCENE-4747, LUCENE-5514: Move to Java 7 as minimum Java version.
(Robert Muir, Uwe Schindler)
New Features
----------------------
* SOLR-5130: Implement addReplica Collections API (Noble Paul)
* SOLR-5183: JSON updates now support nested child documents using a
"_childDocument_" object key. (Varun Thacker, hossman)
* SOLR-5714: You can now use one pool of memory for for the HDFS block cache
that all collections share. (Mark Miller, Gregory Chanan)
* SOLR-5720: Add ExpandComponent to expand results collapsed by the
CollapsingQParserPlugin. (Joel Bernstein)
* SOLR-3177: Enable tagging and excluding filters in StatsComponent via the
localParams syntax. (Mathias H., Nikolai Luthman, Vitaliy Zhovtyuk, shalin)
* SOLR-1604: Wildcards, ORs etc inside Phrase Queries. (Ahmet Arslan via Erick Erickson)
* SOLR-5477: Async execution of OverseerCollectionProcessor(CollectionsAPI)
tasks. (Anshum Gupta)
* SOLR-5865: Provide a MiniSolrCloudCluster to enable easier testing.
(Greg Chanan via Mark Miller)
* SOLR-5860: Use leaderConflictResolveWait in WaitForState during recovery/startup,
improve logging and force refresh cluster state every 15 seconds.
(Timothy Potter via shalin)
* SOLR-5749: A new Overseer status collection API exposes overseer queue sizes, timing
statistics, success and error counts and last N failures per operation. (shalin)
* SOLR-5858: Add a hl.qparser parameter to allow you to define a queryparser
for hl.q highlight queries. If no queryparser is defined, Solr will use
the overall query's defType. (Alan Woodward)
* SOLR-4478: Allow cores to use configuration from a configsets directory
outside their instance directory. (Alan Woodward, Erick Erickson)
* SOLR-5466: A new List collections and cluster status API which clients can use
to read collection and shard information instead of reading data directly from ZooKeeper.
(Dave Seltzer, Varun Thacker, Vitaliy Zhovtyuk, Erick Erickson, shalin)
* SOLR-5795: New DocExpirationUpdateProcessorFactory supports computing an expiration
date for documents from the "TTL" expression, as well as automatically deleting expired
documents on a periodic basis. (hossman)
* SOLR-5829: Allow ExpandComponent to accept query and filter query parameters
(Joel Bernstein)
* SOLR-5653: Create a RestManager to provide REST API endpoints for
reconfigurable plugins. (Tim Potter, Steve Rowe)
* SOLR-5655: Create a stopword filter factory that is (re)configurable,
and capable of reporting its configuration, via REST API.
(Tim Potter via Steve Rowe)
* SOLR-5654: Create a synonym filter factory that is (re)configurable, and
capable of reporting its configuration, via REST API.
(Tim Potter via Steve Rowe)
* SOLR-5960: Add support for basic authentication in post.jar tool, e.g.:
java -Durl="http://username:password@hostname:8983/solr/update" -jar post.jar sample.xml
(Sameer Maggon via Uwe Schindler)
* SOLR-4864: RegexReplaceProcessorFactory should support pattern capture group
substitution in replacement string.
(Sunil Srinivasan, Jack Krupansky via Steve Rowe)
Bug Fixes
----------------------
* SOLR-5858, SOLR-4812: edismax and dismax query parsers can be used for parsing
highlight queries. (Alan Woodward, Tien Nguyen Manh)
* SOLR-5893: On restarting overseer designate , move itself to front of the queue (Noble Paul)
* SOLR-5915: Attempts to specify the parserImpl for
solr.PreAnalyzedField fieldtype failed. (Mike McCandless)
* SOLR-5943: SolrCmdDistributor does not distribute the openSearcher parameter.
(ludovic Boutros via shalin)
* SOLR-5954: Slower DataImportHandler process caused by not reusing jdbc
connections. (Mark Miller, Paco Garcia, Raja Nagendra Kumar)
* SOLR-5897: Upgraded to jQuery 1.7.2, Solr was previously using 1.4.3, the file was
mistakenly named 1.7.2 (steffkes)
Optimizations
----------------------
* SOLR-1880: Distributed Search skips GET_FIELDS stage if EXECUTE_QUERY
stage gets all fields. Requests with fl=id or fl=id,score are now single-pass.
(Shawn Smith, Vitaliy Zhovtyuk, shalin)
* SOLR-5783: Requests to open a new searcher will now reuse the current registered
searcher (w/o additional warming) if possible in situations where the underlying
index has not changed. This reduces overhead in situations such as deletes that
do not modify the index, and/or redundant commits. (hossman)
* SOLR-5884: When recovery is cancelled, any call to the leader to wait to see
the replica in the right state for recovery should be aborted. (Mark Miller)
Other Changes
---------------------
* SOLR-5909: Upgrade Carrot2 clustering dependency to 3.9.0. (Dawid Weiss)
* SOLR-5764: Fix recently added tests to not use absolute paths to load test-files,
use SolrTestCaseJ4.getFile() and getResource() instead; fix morphlines/map-reduce
to not duplicate test resources and fix dependencies among them.
(Uwe Schindler)
* SOLR-5765: Update to SLF4J 1.7.6. (Mark Miller)
* SOLR-5609: If legacy mode is disabled don't let cores create slices/replicas/collections .
All operations should be performed through collection API (Noble Paul)
* SOLR-5613: Upgrade to commons-codec 1.9 for better BeiderMorseFilter performance.
(Thomas Champagne, Shawn Heisey via shalin)
* SOLR-5771: Add SolrTestCaseJ4.SuppressSSL annotation to disable SSL (instead of static boolean).
(Robert Muir)
* SOLR-5799: When registering as the leader, if an existing ephemeral
registration exists, wait a short time to see if it goes away.
(Mark Miller)