-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Add first version of function to compute file size - Add documentation - Add package information - Add project config
- Loading branch information
Showing
5 changed files
with
2,407 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
root = true | ||
|
||
[*] | ||
charset = utf-8 | ||
indent_style = space | ||
indent_size = 2 | ||
end_of_line = lf | ||
insert_final_newline = true | ||
trim_trailing_whitespace = true |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,68 @@ | ||
# algolia-mcm-filesize | ||
Simple script to get per cluster index size while using Algolia MCM feature | ||
# Algolia MCM File Size | ||
|
||
![npm](https://img.shields.io/npm/dm/algolia-mcm-filesize) | ||
![node-lts](https://img.shields.io/node/v-lts/algolia-mcm-filesize) | ||
[![JavaScript Style Guide](https://img.shields.io/badge/code_style-standard-brightgreen.svg)](https://standardjs.com) | ||
|
||
**Important note:** in Algolia's context, **file size** and **index size** | ||
are interchangeable. | ||
The index size represents the space used in RAM, whereas data size is the raw | ||
JSON data stored on disk and used as a backup for e.g. doing a full reindex. | ||
|
||
## What is it? | ||
|
||
This package is a simple script that helps you compute the index size while using Algolia's [MCM](https://www.algolia.com/doc/guides/scaling/managing-multiple-clusters-mcm/) | ||
architecture. | ||
|
||
## How to use it? | ||
|
||
Regular install through npm or yarn: | ||
|
||
```bash | ||
npm i algolia-mcm-filesize | ||
# or | ||
yarn add algolia-mcm-filesize | ||
``` | ||
|
||
It has been designed to be used in a shell environment so it returns JSON | ||
instead of a pure JS object. Here is a quick usage suggestion: | ||
|
||
### index.js | ||
```javascript | ||
const mcmFileSize = require('algolia-mcm-filesize') | ||
|
||
mcmFileSize('APPID', 'ADMIN_APIKEY').then(res => console.log(res)) | ||
``` | ||
|
||
Optionally, you can pass a custom domain (default is `algolia.net`) | ||
|
||
```javascript | ||
const mcmFileSize = require('algolia-mcm-filesize') | ||
|
||
mcmFileSize('APPID', 'ADMIN_APIKEY', 'algolianet.com').then(res => console.log(res)) | ||
``` | ||
|
||
### shell | ||
|
||
Using [jq](https://stedolan.github.io/jq/) for the sake of the example | ||
```bach | ||
node index.js | jq . | ||
``` | ||
|
||
## Technical details | ||
|
||
The only dependency is the [JS API client](https://www.algolia.com/doc/api-client/getting-started/what-is-the-api-client/javascript/) (v4) | ||
|
||
You need to use the [Admin API key](https://www.algolia.com/doc/guides/security/api-keys/#admin-api-key) | ||
(required for all MCM endpoints) | ||
|
||
Under the hood, the package uses the following endpoints: | ||
- [List Clusters](https://www.algolia.com/doc/api-reference/api-methods/list-clusters/) | ||
- [List Indices](https://www.algolia.com/doc/api-reference/api-methods/list-indices/) | ||
|
||
## Disclaimer | ||
|
||
This package is not officially supported by [Algolia](https://www.algolia.com/), | ||
so it cannot be held responsible for any use in production. If you need support, | ||
use GitHub issues or the [community forum](https://discourse.algolia.com/), but | ||
**not** Algolia email support. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,47 @@ | ||
const algoliasearch = require('algoliasearch') | ||
const CallEnum = require('@algolia/transporter').CallEnum | ||
|
||
const bytesToGB = (bytesNumber) => Math.round(bytesNumber / 1000 / 1000 / 1000) | ||
|
||
const getHosts = (clusterName, domain) => { | ||
// https://www.algolia.com/doc/api-client/getting-started/upgrade-guides/javascript/#the-hosts-parameter | ||
return [1, 2, 3].map(n => `${clusterName}-${n}.${domain}`).map(host => { | ||
return { | ||
protocol: 'https', | ||
url: host, | ||
accept: CallEnum.Read // CallEnum.Any or CallEnum.Write | ||
} | ||
}) | ||
} | ||
|
||
const mcmFileSize = (appId, adminKey, domain = 'algolia.net') => { | ||
return algoliasearch(appId, adminKey).listClusters().then(({ clusters }) => { | ||
const clusterNames = clusters.map(cluster => cluster.clusterName) | ||
|
||
const indexLists = clusters.map(cluster => { | ||
const hosts = getHosts(cluster.clusterName, domain) | ||
const client = algoliasearch(appId, adminKey, { hosts }) | ||
return client.listIndices() | ||
}) | ||
|
||
return Promise.all(indexLists) | ||
.then(indices => { | ||
const report = {} | ||
let total = 0 | ||
clusterNames.forEach((clusterName, i) => { | ||
const clusterFileSize = indices[i].items.reduce( | ||
(fileSize, index) => fileSize + bytesToGB(index.fileSize), | ||
0 | ||
) | ||
total += clusterFileSize | ||
report[clusterName] = clusterFileSize | ||
}) | ||
return JSON.stringify({ ...report, total }) | ||
}) | ||
.catch(err => { | ||
throw err | ||
}) | ||
}) | ||
} | ||
|
||
module.exports = mcmFileSize |
Oops, something went wrong.