Merge pull request #152 from nicolasembleton/upstream

Update Package.json dependencies, according tests, activate solrVersion, facet.pivot, add helpers for standardization

CONTRIBUTIONS.md

@@ -1,7 +1,17 @@
 # CONTRIBUTIONS
 
-Send a pull request with your code and the tests for it.
+The library lives and prospers thanks to you. If you want to share some of your work, or if you have fixed some of the 
+open issues, send a pull request with your code and the tests for it.
 
-Not mandatory at all but if you don't mind:
+We're currently re-organizing the project organization, and as such we're starting to adopt a stricter commit guideline,
+ very much inspired from that of [AngularJS](https://docs.google.com/document/d/1QrDFcIiPjSLDn3EL15IJygNPiHORgU1_OOAqWjiDU5Y/edit)
 
-* Prepend your commit message with `FIX:` or `NEW:`, make it easy to generate a readable CHANGELOG.
+So we ask you to respect 2 rules: 
+1. Provide the code, the tests, all respecting the style guide
+2. Follow AngularJS commit style-guide that can, at the most minimum, be summarized by a single line like:
+ `feat(solrCloud): Add ability to query pivot per field`
+
+Note: Please restrain from committing code that does other changes than what they are committed for. Avoid changing the 
+style guide of entire files while committing fixes or features.
+
+*Thanks in advance for your support in making the library live.*

README.md

@@ -13,6 +13,19 @@ npm install solr-client
 
 ##Features
 
+###Latest (0.6.0)
+
+- Solr5 Query-Highlighting support through StandardHighlighter (@LukeTavern) (https://github.com/lbdremy/solr-node-client/pull/144)
+- Experimental support for SolrCloud collections administration (@LukeTavern) (https://github.com/lbdremy/solr-node-client/issues/138)
+- Support for large query through POST via configurable automatic switch (@kfitzgerald) (https://github.com/lbdremy/solr-node-client/pull/129) 
+- Set Default Field (query.df()) (@cbarrientos-ias) (https://github.com/lbdremy/solr-node-client/pull/137)
+- Adds support for facet.pivot and facet.pivot.mincount (@nicolasembleton) (https://github.com/lbdremy/solr-node-client/issues/146)
+
+Noticeable change: You can now pass a solrVersion to the Client initialization so it will activate features that  are only supported 
+by your version or higher. Be sure to check the documentation
+
+###0.5.0
+
 - Commands supported: search(select), add, delete, update, commit, rollback, optimize, ping, real-time get, prepare commit, soft commit, arbitrary search handler (i.e: mlt, luke ...)
 - Lucene query / DisMax query
 - Grouping / Field Collapsing. (Apache Solr version must be >= 3.3)
@@ -44,40 +57,54 @@ client.add({ id : 12, title_t : 'Hello' },function(err,obj){
 });
 ```
 
-##Migration between 0.2.x and 0.3.x
+##Migration between 0.5.x and 0.6.x
 
-The only breaking change introduced in `v0.3.0` is about method chaining of the solr `Client`.
-Method chaining as simply been removed because we were actually hidding something really interesting and useful
-the `http.ClientRequest` instance.
+No breaking change has been introduced to this release. However, to take advantage of facet.pivot and facet.pivot.mincount feature,
+you'll need to pass a solrVersion parameter to the createClient() method. 
 
-So, before you could have done this:
+Example:
 
 ```js
-var client = solr.createClient();
+// Will activate features specific to Solr4.0 and above (like facet.pivot and facet.pivot.mincount)
+var client = solr.createClient({
+ solrVersion: '4.0'
+});
+```
 
-client
-	.search('q=hello', function(err, obj){
-		console.log(err, obj);
-	})
-	.search('q=world', function(err, obj){
-		console.log(err, obj);
-	});
+A feature allowing to support large queries by switching to POST when the query size reaches the threshold was introduced. To not become
+a breaking change, it is by default turned OFF. 
+
+To activate it, pass the property `get_max_request_entity_size` to the createClient with the threshold in bytes. Minimum is 1.
+
+Example:
+
+```js
+// Will switch to POST as soon as the query size reaches 1000 bytes (limit in servers is usually 2048 or 8192)  
+// You can set it to 1 so every request will always use POST. 
+var client = solr.createClient({
+ get_max_request_entity_size: 1000
+});
 ```
 
-Now it won't work, but you have now access to the `http.ClientRequest` instead created by `Client#search`:
+##Migration between 0.4.x and 0.5.x
+
+The only breaking change introduced in `0.5.x` is introduced in this commit [3cbc7fc6cf631f019a4626913c0a4b616092133b](https://github.com/lbdremy/solr-node-client/commit/3cbc7fc6cf631f019a4626913c0a4b616092133b) which remove escaping of the Solr special characters in some of the methods of the `Query` class i.e in `Query#rangeFilter`, `Query#matchFilter`, `Query#group`, `Query#facet`, `Query#mlt` if you were relying on this behavior just wrap the arguments you passed to those methods into the [`solr.escapeSpecialChars(arg)`](https://github.com/lbdremy/solr-node-client/blob/master/lib/solr.js#L605) method.
+
+For example, for some weird reason you wanted to escape the special char `*`, don't ask me ;)
 
 ```js
-var client = solr.createClient();
+var query = client.createQuery();
+query.q({ '*' : '*' }).rangeFilter({ field : 'id', start : 100, end : '*'})
+```
 
-var request = client.search('q=hello', function(err, obj){
-	console.log(err, obj);
-});
-request.setTimeout(200, function(){
-	console.log('search timeout');
-});
+You still can:
+
+```js
+var query = client.createQuery();
+query.q({ '*' : '*' }).rangeFilter({ field : 'id', start : 100, end : solr.escapeSpecialChars('*')})
 ```
 
-Post an issue if you have troubles migrating to v0.3.0.
+Post an issue if you have troubles migrating to v0.5.0.
 
 ##Migration between 0.3.x and 0.4.x
 
@@ -97,25 +124,40 @@ client.options.bigint = true;
 
 Post an issue if you have troubles migrating to v0.4.0.
 
-##Migration between 0.4.x and 0.5.x
+##Migration between 0.2.x and 0.3.x
 
-The only breaking change introduced in `0.5.x` is introduced in this commit [3cbc7fc6cf631f019a4626913c0a4b616092133b](https://github.com/lbdremy/solr-node-client/commit/3cbc7fc6cf631f019a4626913c0a4b616092133b) which remove escaping of the Solr special characters in some of the methods of the `Query` class i.e in `Query#rangeFilter`, `Query#matchFilter`, `Query#group`, `Query#facet`, `Query#mlt` if you were relying on this behavior just wrap the arguments you passed to those methods into the [`solr.escapeSpecialChars(arg)`](https://github.com/lbdremy/solr-node-client/blob/master/lib/solr.js#L605) method.
+The only breaking change introduced in `v0.3.0` is about method chaining of the solr `Client`.
+Method chaining as simply been removed because we were actually hidding something really interesting and useful
+the `http.ClientRequest` instance.
 
-For example, for some weird reason you wanted to escape the special char `*`, don't ask me ;)
+So, before you could have done this:
 
 ```js
-var query = client.createQuery();
-query.q({ '*' : '*' }).rangeFilter({ field : 'id', start : 100, end : '*'})
+var client = solr.createClient();
+
+client
+	.search('q=hello', function(err, obj){
+		console.log(err, obj);
+	})
+	.search('q=world', function(err, obj){
+		console.log(err, obj);
+	});
 ```
 
-You still can:
+Now it won't work, but you have now access to the `http.ClientRequest` instead created by `Client#search`:
 
 ```js
-var query = client.createQuery();
-query.q({ '*' : '*' }).rangeFilter({ field : 'id', start : 100, end : solr.escapeSpecialChars('*')})
+var client = solr.createClient();
+
+var request = client.search('q=hello', function(err, obj){
+	console.log(err, obj);
+});
+request.setTimeout(200, function(){
+	console.log('search timeout');
+});
 ```
 
-Post an issue if you have troubles migrating to v0.5.0.
+Post an issue if you have troubles migrating to v0.3.0.
 
 ##Roadmap
 

lib/query.js

@@ -2,7 +2,9 @@
  * Load dependencies
  */
 var querystring = require('querystring'),
-    format = require('./utils/format');
+    format = require('./utils/format'),
+    arrayUtils = require('./utils/array'),
+    versionUtils = require('./utils/version');
 
 /**
  * Expose `Query`
@@ -18,7 +20,8 @@ module.exports = exports = Query;
  * @api private
  */
 
-function Query(){
+function Query(options){
+   this.solrVersion = (options && options.solrVersion) || undefined;
    this.parameters = [];
 }
 
@@ -340,9 +343,7 @@ Query.prototype.group = function(options){
       this.parameters.push('group=true');
    }
    if( options.field ){
-      if(!Array.isArray(options.field)){
-        options.field = [options.field];
-      }
+      options.field = arrayUtils.toArray(options.field);
       options.field.forEach(function(field){
         self.parameters.push('group.field=' + field);
       });
@@ -388,6 +389,8 @@ Query.prototype.group = function(options){
  * @param {Number} [options.mincount=0] - This parameter indicates the minimum counts for facet fields should be included in the response. The solr's default value is 0.
  * @param {Boolean} [options.missing=false] - Set to `true` this param indicates that in addition to the Term based constraints of a facet field, a count of all matching results which have no value for the field should be computed. The solr's default value is false.
  * @param {String} [options.method="fc"] - This parameter indicates what type of algorithm/method to use when faceting a field.The solr's default value is fc (except for BoolField).
+ * @param {String|Array} options.pivot - This parameter allows you to specify a field which should be treated as a facet pivot. It will iterate over each Term in the field. Multiple fields can be defined providing an array instead of a string.
+ * @param {String} [options.pivot.mincount=0] - This parameter indicates the minimum counts for facet pivot fields to be included in the response. The solr's default value is 0.
  *
  * @return {Query}
  * @api public
@@ -403,12 +406,10 @@ Query.prototype.facet = function(options){
       this.parameters.push('facet.query=' + encodeURIComponent(options.query))
    }
    if(options.field){
-      if(!Array.isArray(options.field)){
-         options.field = [options.field];
-      }
-      for (var i = 0; i < options.field.length; i++) {
-         this.parameters.push('facet.field=' + options.field[i])
-      }
+     options.field = arrayUtils.toArray(options.field);
+     options.field.forEach(function(field) {
+       self.parameters.push('facet.field=' + field);
+     });
    }
    if(options.prefix){
       this.parameters.push('facet.prefix=' + encodeURIComponent(options.prefix))
@@ -431,6 +432,20 @@ Query.prototype.facet = function(options){
    if(options.method){
       this.parameters.push('facet.method=' + options.method);
    }
+
+   // Only supported with version 4.0 and above
+   if(this.solrVersion && (versionUtils.version(this.solrVersion) >= versionUtils.Solr4_0)) {
+     if(options.pivot){
+       options.field = arrayUtils.toArray(options.pivot.fields);
+       options.field.forEach(function(field) {
+         self.parameters.push('facet.pivot=' + field);
+       });
+     }
+     if(options.pivot.mincount) {
+       this.parameters.push('facet.pivot.mincount=' + options.pivot.mincount);
+     }
+   }
+
    return self;
 }
 
@@ -743,7 +758,7 @@ Query.prototype.build = function(){
  * @param {Boolean} [options.requireFieldMatch] - If set to True, this parameter will force Solr to highlight terms only if they appear in the specified field. If false, terms are highlighted in all requested fields regardless of which field matches the query.
  * @param {Boolean} [options.usePhraseHighlighter] - If set to True, Solr will use the Lucene SpanScorer class to highlight phrase terms only when they appear within the query phrase in the document.
  * @param {Number} [options.regexSlop] - When using the regex fragmenter, this number specifies the factor by which the fragmenter can stray from the ideal fragment size.
- * @param {String} [options.regexPattern] - This parameter specifies the regulat expression for fragmenting. 
+ * @param {String} [options.regexPattern] - This parameter specifies the regulat expression for fragmenting.
  * @param {Number} [options.regexMaxAnalyzedChars] - This parameters specifies the max number of characters to analyze from a field when using the regex fragmenter.
  * @param {Boolean} [options.preserveMulti] - If True, multi-valued fields will return all values in the order they were saved in the index. If False, only values that match the highlight request will be returned.
  * @param {Boolean} [options.payloads] - If usePhraseHighlighter is True, and the indexed field has payloads but not term vectors, the index payloads will be read into the highlighter's index along with the posting. If you don't want this behavior, you may set this parameter to False and save some memory.

lib/solr.js

@@ -20,7 +20,8 @@ var http = require('http'),
    JSONStream = require('JSONStream'),
    duplexer = require('duplexer'),
    request = require('request'),
-   JSONbig = require('json-bigint');
+   JSONbig = require('json-bigint'),
+   versionUtils = require('./utils/version');
 
 /**
  * Expose `createClient()`.
@@ -39,6 +40,7 @@ exports.createClient = createClient;
  * @param {Boolean} [secure=false] - if true HTTPS will be used instead of HTTP
  * @param {Boolean} [bigint=false] - if true JSONbig serializer/deserializer will be used instead
  *                                    of JSON native serializer/deserializer
+ * @param solrVersion ['3.2', '4.0', '5.0', '5.1'], check lib/utils/version.js for full reference
  *
  * @return {Client}
  * @api public
@@ -85,11 +87,12 @@ function Client(options){
       agent : options.agent,
       secure : options.secure || false,
       bigint : options.bigint || false,
-      get_max_request_entity_size: options.get_max_request_entity_size || false
+      get_max_request_entity_size: options.get_max_request_entity_size || false,
+      solrVersion: options.solrVersion || versionUtils.Solr3_2
    };
 
    // Default paths of all request handlers
-   this.UPDATE_JSON_HANDLER = options.solrVersion >= 4 ? 'update' : 'update/json';
+   this.UPDATE_JSON_HANDLER = (versionUtils.version(this.options.solrVersion) >= versionUtils.Solr4_0) ? 'update' : 'update/json';
    this.UPDATE_HANDLER = 'update';
    this.SELECT_HANDLER = 'select';
    this.COLLECTIONS_HANDLER = 'admin/collections';
@@ -689,13 +692,13 @@ Client.prototype.post = function(handler,query,callback){
 
 /**
  * Create an instance of `Query`
- * 
+ *
  * @return {Query}
  * @api public
  */
 
 Client.prototype.query = function(){
-   return new Query();
+   return new Query(this.options);
 }
 
 /**
@@ -707,7 +710,7 @@ Client.prototype.query = function(){
  */
 
 Client.prototype.createQuery = function(){
-   return new Query();
+   return new Query(this.options);
 }
 
 /**

lib/utils/array.js

@@ -0,0 +1,15 @@
+/**
+ *
+ * @param value
+ * @param defaultIfNull - Will set the value to the default if it is Null or Undefined
+ * @returns {*[]}
+ */
+exports.toArray = function toArray(value, defaultIfNull) {
+  defaultIfNull = defaultIfNull || '';
+
+  function defaultValue(value) {
+    return (value === null || value === undefined) ? defaultIfNull : value;
+  }
+
+  return (Array.isArray(value)) ? value : [defaultValue(value)];
+};

lib/utils/type.js

@@ -0,0 +1,21 @@
+/**
+ *
+ * @param value - The value to check against
+ * @param strict - Pass true if you want to make sure the number is fully and only composed of digits, false to just check if we can extract a number via parseInt(). Default to true.
+ * @returns boolean
+ */
+exports.isNumber = function isNumber(value, strict) {
+  strict = (strict === undefined ? true : strict);
+  var digitRegex = /^\-?\d+$/; // At least 1 digit, possibly a minus sign before
+
+  if (typeof value === 'number') {
+    return true;
+  } else {
+    // String ?
+    if (strict) {
+      return (('' + value).match(digitRegex) !== null);
+    } else {
+      return !isNaN(parseInt(value));
+    }
+  }
+};

lib/utils/version.js

@@ -0,0 +1,37 @@
+/**
+ * The purpose of those helpers is to centralize and standardize the work on detecting current running Solr Version
+ */
+
+var Solr3_2 = 302;
+var Solr4_0 = 400;
+var Solr5_0 = 500;
+var Solr5_1 = 501;
+
+/**
+ * Enum that lists supported versions of Solr. Pass one of the keys from this enum as a solrVersion property
+ *
+ * @type {{3.2: number, 4.0: number, 5.0: number, 5.1: number}}
+ */
+var versionsEnum = {
+  '3.2': Solr3_2,
+  '4.0': Solr4_0,
+  '5.0': Solr5_0,
+  '5.1': Solr5_1
+};
+
+exports.versionsEnum = versionsEnum;
+exports.Solr3_2 = Solr3_2;
+exports.Solr4_0 = Solr4_0;
+exports.Solr5_0 = Solr5_0;
+exports.Solr5_1 = Solr5_1;
+
+/**
+ * solrVersion must match one of enum keys
+ * If a number is passed, it'll be assume a .0 release (3 -> 3.0)
+ * If nothing matches, it will be assumed 3.2
+ *
+ * @param solrVersion
+ */
+exports.version = function(solrVersion) {
+  return (typeof solrVersion === "number") ? (versionsEnum[''+solrVersion+'.0']) : (versionsEnum[solrVersion] ? versionsEnum[solrVersion] : versionsEnum['3.2']);
+};