Skip to content

Commit

Permalink
First version
Browse files Browse the repository at this point in the history
- Add first version of function to compute file size
- Add documentation
- Add package information
- Add project config
  • Loading branch information
julienpa committed Apr 29, 2020
1 parent 723e87b commit 46f7206
Show file tree
Hide file tree
Showing 5 changed files with 2,407 additions and 2 deletions.
9 changes: 9 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
root = true

[*]
charset = utf-8
indent_style = space
indent_size = 2
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true
70 changes: 68 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,68 @@
# algolia-mcm-filesize
Simple script to get per cluster index size while using Algolia MCM feature
# Algolia MCM File Size

![npm](https://img.shields.io/npm/dm/algolia-mcm-filesize)
![node-lts](https://img.shields.io/node/v-lts/algolia-mcm-filesize)
[![JavaScript Style Guide](https://img.shields.io/badge/code_style-standard-brightgreen.svg)](https://standardjs.com)

**Important note:** in Algolia's context, **file size** and **index size**
are interchangeable.
The index size represents the space used in RAM, whereas data size is the raw
JSON data stored on disk and used as a backup for e.g. doing a full reindex.

## What is it?

This package is a simple script that helps you compute the index size while using Algolia's [MCM](https://www.algolia.com/doc/guides/scaling/managing-multiple-clusters-mcm/)
architecture.

## How to use it?

Regular install through npm or yarn:

```bash
npm i algolia-mcm-filesize
# or
yarn add algolia-mcm-filesize
```

It has been designed to be used in a shell environment so it returns JSON
instead of a pure JS object. Here is a quick usage suggestion:

### index.js
```javascript
const mcmFileSize = require('algolia-mcm-filesize')

mcmFileSize('APPID', 'ADMIN_APIKEY').then(res => console.log(res))
```

Optionally, you can pass a custom domain (default is `algolia.net`)

```javascript
const mcmFileSize = require('algolia-mcm-filesize')

mcmFileSize('APPID', 'ADMIN_APIKEY', 'algolianet.com').then(res => console.log(res))
```

### shell

Using [jq](https://stedolan.github.io/jq/) for the sake of the example
```bach
node index.js | jq .
```

## Technical details

The only dependency is the [JS API client](https://www.algolia.com/doc/api-client/getting-started/what-is-the-api-client/javascript/) (v4)

You need to use the [Admin API key](https://www.algolia.com/doc/guides/security/api-keys/#admin-api-key)
(required for all MCM endpoints)

Under the hood, the package uses the following endpoints:
- [List Clusters](https://www.algolia.com/doc/api-reference/api-methods/list-clusters/)
- [List Indices](https://www.algolia.com/doc/api-reference/api-methods/list-indices/)

## Disclaimer

This package is not officially supported by [Algolia](https://www.algolia.com/),
so it cannot be held responsible for any use in production. If you need support,
use GitHub issues or the [community forum](https://discourse.algolia.com/), but
**not** Algolia email support.
47 changes: 47 additions & 0 deletions mcm-filesize.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
const algoliasearch = require('algoliasearch')
const CallEnum = require('@algolia/transporter').CallEnum

const bytesToGB = (bytesNumber) => Math.round(bytesNumber / 1000 / 1000 / 1000)

const getHosts = (clusterName, domain) => {
// https://www.algolia.com/doc/api-client/getting-started/upgrade-guides/javascript/#the-hosts-parameter
return [1, 2, 3].map(n => `${clusterName}-${n}.${domain}`).map(host => {
return {
protocol: 'https',
url: host,
accept: CallEnum.Read // CallEnum.Any or CallEnum.Write
}
})
}

const mcmFileSize = (appId, adminKey, domain = 'algolia.net') => {
return algoliasearch(appId, adminKey).listClusters().then(({ clusters }) => {
const clusterNames = clusters.map(cluster => cluster.clusterName)

const indexLists = clusters.map(cluster => {
const hosts = getHosts(cluster.clusterName, domain)
const client = algoliasearch(appId, adminKey, { hosts })
return client.listIndices()
})

return Promise.all(indexLists)
.then(indices => {
const report = {}
let total = 0
clusterNames.forEach((clusterName, i) => {
const clusterFileSize = indices[i].items.reduce(
(fileSize, index) => fileSize + bytesToGB(index.fileSize),
0
)
total += clusterFileSize
report[clusterName] = clusterFileSize
})
return JSON.stringify({ ...report, total })
})
.catch(err => {
throw err
})
})
}

module.exports = mcmFileSize
Loading

0 comments on commit 46f7206

Please sign in to comment.