Merge branch 'release/v3.1.0'

Author: royrusso

AUTHORS.md

@@ -1,6 +1,9 @@
 Roy Russo <royrusso@gmail.com>
+
 Will McGinnis
 
+And other authors: https://github.com/ElasticHQ/elasticsearch-HQ/graphs/contributors
+
 This project also draws upon the work of the elasticsearch-py author:
 
 Honza Král

CONTRIBUTING.md

@@ -0,0 +1,8 @@
+# Contributing
+
+When contributing to this repository, please first discuss the change you wish to make via issue,
+email, or any other method with the owners of this repository before making a change. 
+
+## Pull Request Process
+
+For new features or bug fixes, please make your changes on the ``master`` branch. 

DEVELOPMENT.md

@@ -1,15 +1,3 @@
 # Development Instructions
 
-See README for this version until a GA release.
-
-
-## DEVELOPMENT
-
-1. Use Python 3.4+
-2. We advise that you create a virtual environment for this project.
-
-
-## CHANGELOG
-
-...
-
+See the development section in the docs for details: http://docs.elastichq.org/developer-guide.html

Dockerfile

@@ -3,25 +3,39 @@ FROM python:3.6-alpine3.7
 RUN apk update
 RUN apk add supervisor
 RUN apk add --update py2-pip
-RUN pip install gunicorn
+
+# Upgrade and install basic Python dependencies
+# This block added because of the trouble installing gevent on many systems
+# https://hub.docker.com/r/openwhisk/dockerskeleton/~/dockerfile/
+RUN apk add --no-cache bash \
+ && apk add --no-cache --virtual .build-deps \
+        bzip2-dev \
+        gcc \
+        libc-dev
+#  && pip install --no-cache-dir gevent \
+#  && apk del .build-deps
 
 # reqs layer
 ADD requirements.txt .
 RUN pip3 install -U -r requirements.txt
+RUN pip3 install gunicorn
 
 # Bundle app source
 ADD . /src
 
-# Expose
-EXPOSE  5000
-
 COPY ./deployment/logging.conf /src/logging.conf
 COPY ./deployment/gunicorn.conf /src/gunicorn.conf
 
+EXPOSE 5000
+
 # Setup supervisord
 RUN mkdir -p /var/log/supervisor
 COPY ./deployment/supervisord.conf /etc/supervisor/supervisord.conf
 COPY ./deployment/gunicorn.conf /etc/supervisor/conf.d/gunicorn.conf
 
 # Start processes
-CMD ["supervisord", "-c", "/etc/supervisor/supervisord.conf"]
+CMD ["supervisord", "-c", "/etc/supervisor/supervisord.conf"]
+
+
+#ENTRYPOINT ["python"]
+#CMD ["src/application.py"]

ISSUE_TEMPLATE.md

@@ -0,0 +1,17 @@
+### General information
+
+- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**:
+- **ElasticHQ Version**:
+- **Elasticsearch Version**:
+- **Python version**: 
+- **Browser Vendor and Version (if applicable)**:
+
+### Issue Description
+
+Describe the problem clearly here. Be sure to convey here why it's a bug in ElasticHQ or a feature request.
+
+### Source Code / Logs
+
+Include any logs or source code that would be helpful to diagnose the problem. If including stacktrace, please include the full stacktrace. 
+
+Large logs and files should be attached. Try to provide a reproducible test case or steps that are the bare minimum necessary to generate the problem.

README.md

@@ -10,7 +10,7 @@ Simplified Monitoring and Management for ElasticSearch clusters.
 [![License](https://img.shields.io/badge/license-ASL-blue.svg)](https://opensource.org/licenses/ASL)
 
 
-![alt text](main_dashboard.png)
+![alt text](https://raw.githubusercontent.com/ElasticHQ/elasticsearch-HQ/master/main_dashboard.png)
 
 
 Key Features
@@ -21,6 +21,7 @@ Key Features
 * Create and perform maintenance on Elasticsearch Indices.
 * One-Click access to common ES API endpoints.
 * Well-documented REST API for extensibility.
+* Real-time monitoring charts of important metrics.
 * Active project used by Fortune 100 companies around the world.
 * Free. ;-)
 
@@ -43,19 +44,14 @@ For **full** installation and configuration instructions, see [Getting Started](
 
 For further installation and configuration help, please read the docs: [ElasticHQ Documentation](http://docs.elastichq.org)
 
-Docs + Support
----------------
-* Documentation can be found here: http://docs.elastichq.org
-* Google Group can be found here: https://groups.google.com/d/forum/elastichq
-* If you need sample index + documents, see here: https://github.com/royrusso/elasticsearch-sample-index
-* If you find a bug, **please** create an issue and report it, or fix it and submit a pull request. ;-)
- 
-Legal Stuff
------------
-
-* [License File](LICENSE.md).
-* [TOS and Privacy](TERMS.md).
+For those of you wanting to use Docker: [ElasticHQ on Dockerhub](https://hub.docker.com/r/elastichq/elasticsearch-hq/)
 
+Useful Links
+------------
+* [Documentation](http://docs.elastichq.org)
+* [Official Website](http://www.elastichq.org)
+* [Docker Images](https://hub.docker.com/r/elastichq/elasticsearch-hq/)
+* [Google Group](https://groups.google.com/d/forum/elastichq)
 
 
 

__init__.py

@@ -0,0 +1 @@
+__author__ = 'royrusso'

application.py

@@ -1,14 +1,31 @@
-from elastichq import factory
-import argparse
+# import argparse
+import optparse
+
+from elastichq import create_app
+from elastichq.globals import socketio
+
+default_host = '0.0.0.0'
+default_port = 5000
+default_debug = False
+
+application = create_app()
 
-application = factory.create_app()
-  
 if __name__ == '__main__':
-	parser = argparse.ArgumentParser()
-	parser.add_argument("--port", help="The port to listen to", type=int,default=5000)
-	parser.add_argument("--host", help="local IP address to bind to",default="localhost")
-	args = parser.parse_args()
-	
-	application.run(debug=False, threaded=True, use_reloader=True,host=args.host,port=args.port)
-    #application.run(debug=False, threaded=True, use_reloader=False)
-    #application.run(debug=True, threaded=True, use_reloader=True)
+    # Set up the command-line options
+    parser = optparse.OptionParser()
+    parser.add_option("-H", "--host",
+                      help="Hostname of the Flask app " + \
+                           "[default %s]" % default_host,
+                      default=default_host)
+    parser.add_option("-P", "--port",
+                      help="Port for the Flask app " + \
+                           "[default %s]" % default_port,
+                      default=default_port)
+    parser.add_option("-d", "--debug",
+                      action="store_true", dest="debug", default=default_debug,
+                      help=optparse.SUPPRESS_HELP)
+
+    options, _ = parser.parse_args()
+
+    socketio.run(application, host=options.host, port=options.port, debug=options.debug)
+

deployment/supervisord.conf

@@ -2,5 +2,5 @@
 nodaemon=true
 
 [program:gunicorn]
-command=/usr/local/bin/gunicorn application:application -w 4 --config /src/gunicorn.conf --log-config /src/logging.conf --bind 0.0.0.0:5000
+command=/usr/local/bin/gunicorn application:application -w 1 --worker-class eventlet --config /src/gunicorn.conf --log-config /src/logging.conf --bind 0.0.0.0:5000
 directory=/src

docs/.doctrees/developer-guide.doctree

@@ -131,6 +131,7 @@ <h1>Source code for elastichq.service.ClusterService</h1><div class="highlight">
     <span class="sd">&quot;&quot;&quot;</span>
 <span class="sd">    cc</span>
 <span class="sd">    &quot;&quot;&quot;</span>
+
     <span class="k">def</span> <span class="nf">get_cluster_health</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">cluster_name</span><span class="p">):</span>
         <span class="n">connection</span> <span class="o">=</span> <span class="n">ConnectionService</span><span class="p">()</span><span class="o">.</span><span class="n">get_connection</span><span class="p">(</span><span class="n">cluster_name</span><span class="p">)</span>
         <span class="k">return</span> <span class="n">connection</span><span class="o">.</span><span class="n">cluster</span><span class="o">.</span><span class="n">health</span><span class="p">(</span><span class="n">request_timeout</span><span class="o">=</span><span class="n">REQUEST_TIMEOUT</span><span class="p">)</span>
@@ -160,7 +161,7 @@ <h1>Source code for elastichq.service.ClusterService</h1><div class="highlight">
         <span class="k">return</span> <span class="n">connection</span><span class="o">.</span><span class="n">cluster</span><span class="o">.</span><span class="n">put_settings</span><span class="p">(</span><span class="n">body</span><span class="o">=</span><span class="n">settings</span><span class="p">,</span> <span class="n">request_timeout</span><span class="o">=</span><span class="n">REQUEST_TIMEOUT</span><span class="p">)</span>
 
     <span class="k">def</span> <span class="nf">get_clusters</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">create_if_missing</span><span class="o">=</span><span class="bp">True</span><span class="p">):</span>
-        <span class="n">clusters</span> <span class="o">=</span> <span class="n">ConnectionService</span><span class="p">()</span><span class="o">.</span><span class="n">get_connections</span><span class="p">()</span>
+        <span class="n">clusters</span> <span class="o">=</span> <span class="n">ConnectionService</span><span class="p">()</span><span class="o">.</span><span class="n">get_connections</span><span class="p">(</span><span class="n">create_if_missing</span><span class="o">=</span><span class="n">create_if_missing</span><span class="p">)</span>
         <span class="k">for</span> <span class="n">cluster</span> <span class="ow">in</span> <span class="n">clusters</span><span class="p">:</span>
             <span class="k">if</span> <span class="n">cluster</span><span class="o">.</span><span class="n">cluster_connected</span> <span class="ow">is</span> <span class="bp">True</span><span class="p">:</span>
                 <span class="n">cluster</span><span class="o">.</span><span class="n">cluster_health</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_cluster_health</span><span class="p">(</span><span class="n">cluster_name</span><span class="o">=</span><span class="n">cluster</span><span class="o">.</span><span class="n">cluster_name</span><span class="p">)</span>
@@ -185,7 +186,11 @@ <h1>Source code for elastichq.service.ClusterService</h1><div class="highlight">
         <span class="n">nodes</span> <span class="o">=</span> <span class="n">NodeService</span><span class="p">()</span><span class="o">.</span><span class="n">get_node_summary</span><span class="p">(</span><span class="n">cluster_name</span><span class="p">)</span>
         <span class="n">summary</span><span class="p">[</span><span class="s1">&#39;nodes&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">nodes</span>
 
-        <span class="k">return</span> <span class="n">summary</span></div></div>
+        <span class="k">return</span> <span class="n">summary</span></div>
+
+    <span class="k">def</span> <span class="nf">get_cluster_tasks</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">cluster_name</span><span class="p">):</span>
+        <span class="n">connection</span> <span class="o">=</span> <span class="n">ConnectionService</span><span class="p">()</span><span class="o">.</span><span class="n">get_connection</span><span class="p">(</span><span class="n">cluster_name</span><span class="p">)</span>
+        <span class="k">return</span> <span class="n">connection</span><span class="o">.</span><span class="n">cluster</span><span class="o">.</span><span class="n">pending_tasks</span><span class="p">(</span><span class="n">request_timeout</span><span class="o">=</span><span class="n">REQUEST_TIMEOUT</span><span class="p">)</span></div>
 </pre></div>
 
           </div>

docs/.doctrees/developers.doctree

@@ -113,7 +113,6 @@ <h1>All modules for which code is available</h1>
 <ul><li><a href="elastichq/common/api_response.html">elastichq.common.api_response</a></li>
 <li><a href="elastichq/common/exceptions.html">elastichq.common.exceptions</a></li>
 <li><a href="elastichq/model/ClusterModel.html">elastichq.model.ClusterModel</a></li>
-<li><a href="elastichq/service/ClusterService.html">elastichq.service.ClusterService</a></li>
 </ul>
 
           </div>

docs/.doctrees/environment.pickle

@@ -6,6 +6,11 @@ For Developers
     :depth: 3
     :local:
 
+Contributing
+------------
+
+Please read the `Contributing guidelines <https://github.com/ElasticHQ/elasticsearch-HQ/blob/master/CONTRIBUTING.md>`_ before working on a pull request.
+
 Building Pre-Releases
 ---------------------
 
@@ -65,6 +70,12 @@ Notes
 Building Documentation
 ----------------------
 
+Sphinx requires specific libraries that are not in the default ``requirements.txt``.
+
+First install the sphinx requirements: ``pip install -r sphinx-requirements.txt``
+
+To generate the documentation:
+
 ``./sphinx-build -b html /path/to/docs/source /path/to/docs``
 
 HTTP Responses

docs/.doctrees/faq.doctree

@@ -20,7 +20,7 @@ How do I...
     You can access the REST API under http://localhost:5000/api
 
 Using ElasticHQ with...
---------------------
+-----------------------
 
 X-Pack
    We have added instructions on X-Pack authentication, see :any:`xpack integration`.

docs/.doctrees/index.doctree

@@ -31,6 +31,13 @@ Install ElasticHQ
 
 .. note:: Alternatively, you can start the server with ``python manage.py runserver``
 
+Docker Images
+^^^^^^^^^^^^^
+
+Docker images are offered on the `ElasticHQ Dockerhub <https://hub.docker.com/r/elastichq/elasticsearch-hq/>`_.
+
+The ``latest`` tag deploys the latest stable release. Where ``develop`` is the latest unstable working branch.
+
 Pre-Releases
 ^^^^^^^^^^^^
 
@@ -87,10 +94,21 @@ ElasticHQ ships with SQLLite integration to store clusters you have connected to
 
 .. note:: In the event you want to start with a clean slate, simply delete the ``elastichq.db`` file. ElasticHQ will recreate it at next startup.
 
-Directory Structure
-^^^^^^^^^^^^^^^^^^^
+Startup Parameters
+^^^^^^^^^^^^^^^^^^
+
+The ``application.py`` start script takes parameters passed in as arguments from the command line:
+
+
+
+    ===========  ==============  ====================================================================
+    Arg          Default Value   Definition
+    ===========  ==============  ====================================================================
+    ``--host``   127.0.0.1       Host the HQ server should be reachable on.
+    ``--port``   5000            Port to reach HQ server.
+    ``--debug``  False           If True, exposes debug data to UI and causes reload on code changes.
+    ===========  ==============  ====================================================================
 
-TODO
 
 Upgrading
 ---------
@@ -127,6 +145,24 @@ Upgrading Minor and Patch versions
 3. (Re)Start the server: ``python application.py``
 4. Point your browser to: http://localhost:5000
 
+
+Running in Production
+---------------------
+
+We advise that under any considerable usage/load, this application should be run with a multithreaded server. The current flask implemenation by itself should not be run in production without this, as it is a single-threaded process.
+
+We recommend running this WSGI application with gunicorn. Install gunicorn by either commenting out the line in the ``requirements.txt`` file or simply running ``pip install gunicorn``
+
+In console, run gunicorn with:
+
+``gunicorn -w 1 -b :5000 --worker-class eventlet application:application``
+
+The application will be accessible under http://127.0.0.1:5000
+
+Read the `Gunicorn Docs <http://docs.gunicorn.org/en/stable/configure.html>`_ for further command line options.
+
+.. note:: For the *Metrics* section to broadcast via websocket, you must have gunicorn set to 1 worker.
+
 License
 -------
 

docs/.doctrees/installation.doctree

@@ -52,6 +52,8 @@ div.sphinxsidebar {
     width: 230px;
     margin-left: -100%;
     font-size: 90%;
+    word-wrap: break-word;
+    overflow-wrap : break-word;
 }
 
 div.sphinxsidebar ul {
@@ -83,10 +85,6 @@ div.sphinxsidebar #searchbox input[type="text"] {
     width: 170px;
 }
 
-div.sphinxsidebar #searchbox input[type="submit"] {
-    width: 30px;
-}
-
 img {
     border: 0;
     max-width: 100%;
@@ -187,6 +185,13 @@ div.genindex-jumpbox {
 
 /* -- general body styles --------------------------------------------------- */
 
+div.body p, div.body dd, div.body li, div.body blockquote {
+    -moz-hyphens: auto;
+    -ms-hyphens: auto;
+    -webkit-hyphens: auto;
+    hyphens: auto;
+}
+
 a.headerlink {
     visibility: hidden;
 }
@@ -489,6 +494,13 @@ pre {
     overflow-y: hidden;  /* fixes display issues on Chrome browsers */
 }
 
+span.pre {
+    -moz-hyphens: none;
+    -ms-hyphens: none;
+    -webkit-hyphens: none;
+    hyphens: none;
+}
+
 td.linenos pre {
     padding: 5px 0px;
     border: 0;