Skip to content

Commit

Permalink
Develop (#17)
Browse files Browse the repository at this point in the history 
New options: --folders, --docx
  • Loading branch information
@sonnenkern
sonnenkern authored Jun 7, 2020
1 parent 9eeecb9 commit b00c034
Show file tree
Hide file tree
Showing 11 changed files with 184 additions and 106 deletions.
35 changes: 23 additions & 12 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,26 +1,31 @@
# Quip-Export
Comprehensive full automated export (backup) tool for [Quip](https://quip.com/).

Quip-Export uses official [Quip Automation API](https://quip.com/dev/automation/documentation) and exports all folders and referenced files from an Quip-Account.
The documents are exported as HTML files with original Quip styling.
All referenced files and images in Quip document are saved in 'blobs' folder.
Quip-Export allows optionally to export in a zip-file.
Quip-Export uses official [Quip Automation API](https://quip.com/dev/automation/documentation) and exports all documents and folders from an Quip-Account.

Slides are not supported (due to poor quality of generated PDFs).
Features:

Sometimes the export process goes not smooth and fast enough due to Quip API rate limits (HTTP 503: Over Rate Limit).
* Export in HTML format with original Quip styling
* Export in MS Office format: .docx for documents, .xlsx for spresdsheets _(--docx)_
* Embedded CSS for HTML-export _(--embedded-styles)_
* Embedded images for HTML-export _(--embedded-images)_
* Export of comments and conversations for HTML-export _(--comments)_
* Export of specific folders only _(--folders)_
* Export of referenced files in documents
* Resolving of references between folders and documents to relative paths

Despite Quip-Export is designed as a standalone tool for Node.js environment, it could be also used as Quip export library for any kind of JavaScript applications.
In that case, the Quip-Export project is a good usage example.
Slides are not supported (due to poor quality of generated PDFs). Export in PDF are not supported (due to poor quality of generated PDFs).

Despite Quip-Export is designed as a standalone tool for Node.js environment, it could be also used as Quip export library for any kind of JavaScript applications. In that case, the Quip-Export project is a good usage example.

Quip-Export perfectly works on Windows, Mac OS and Linux/Unix in Node.js or JavaScript (browser) environment.
Quip-Export perfectly works on Windows, Mac OS and Linux/Unix in Node.js or in pure browser environment.

<p align="center">
<img src="https://raw.githubusercontent.com/sonnenkern/quip-export/master/public/example-anim.gif">
</p>

## Online web app and demo
Full featured web app using Quip-Export package with demo mode is available on [www.quip-export.com](https://www.quip-export.com)
Full featured web app using Quip-Export npm package with demo mode is available on [www.quip-export.com](https://www.quip-export.com)

<p align="center">
<img src="https://raw.githubusercontent.com/sonnenkern/quip-export/master/public/demo.gif">
Expand Down Expand Up @@ -78,17 +83,23 @@ npm install quip-export
```
-h, --help Usage guide.
-v, --version Print version info
-t, --token string Quip Access Token.
-d, --destination string Destination folder for export files
-t, --token "string" Quip Access Token.
-d, --destination "string" Destination folder for export files
-z, --zip Zip export files
--embedded-styles Embedded in each document stylesheet
--embedded-images Embedded images
--docx Exports documents in MS-Office format (*.docx , *.xlsx)
--comments Includes comments (messages) for the documents
--folders "string" Comma-separated folder's IDs to export
--debug Extended logging
```

To generate a personal access token, visit the page: [https://quip.com/dev/token](https://quip.com/dev/token)

Be aware, the options --comments, --embedded-images, --embedded-styles don't work together with export in MS-Office format (--docx) and will be ignored.

The easiest way to get to know ID of Quip fodler is just to open the folder in Quip web application in browser and look at adress line. For example the adress "https://quip.com/bGG333444111" points to the folder with ID "bGG333444111".

## Usage examples
Export to folder c:\temp
```
Expand Down
20 changes: 18 additions & 2 deletions __tests__/app.test.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,8 @@ function initApp() {
token: 'TOKEN',
['embedded-styles']: true,
['embedded-images']: true,
['messages']: true
['comments']: true,
['docx']: true
});
QuipService.mockImplementation(() => {
return {
Expand Down Expand Up @@ -165,7 +166,8 @@ describe('main() tests', () => {
documentTemplate,
documentCSS: documentCSS,
embeddedImages: true,
messages: true
comments: true,
docx: true
}
);
expect(app.quipProcessor.setLogger).toHaveBeenCalledWith(app.Logger);
Expand Down Expand Up @@ -213,6 +215,20 @@ describe('main() tests', () => {

expect(console.log).toHaveBeenCalledWith("Zip-file has been saved: ", path.join(app.desinationFolder, 'quip-export.zip'));
});

test('folders option', async () => {
const folders = ['111','222'];
CliArguments.mockReturnValue({
destination: 'c:/temp',
token: 'TOKEN',
['embedded-styles']: false,
['embedded-images']: false,
zip: false,
folders
});
await app.main();
expect(app.quipProcessor.startExport).toHaveBeenCalledWith(folders);
});
});

describe('fileSaver() tests', () => {
Expand Down
21 changes: 16 additions & 5 deletions app.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,6 @@ class App {
//current folder as destination, if not set
this.desinationFolder = (this.cliArguments.destination || process.cwd());


if(this.cliArguments.debug) {
this.Logger = new PinoLogger(PinoLogger.LEVELS.DEBUG, `${this.desinationFolder}/export.log`);
} else {
Expand All @@ -140,6 +139,10 @@ class App {
utils.cliBox(`!!!! A new version of Quip-Export (v${versionInfo.remoteVersion}) is available.`);
}

if(this.cliArguments['comments'] && this.cliArguments['docx']) {
console.log('Docx export: comments option will be ignored.');
}

//Token verification
const quipService = new QuipService(this.cliArguments.token);
quipService.setLogger(this.Logger);
Expand All @@ -161,11 +164,13 @@ class App {
documentTemplate,
documentCSS: this.cliArguments['embedded-styles']? documentCSS : '',
embeddedImages: this.cliArguments['embedded-images'],
comments: this.cliArguments['comments']
comments: this.cliArguments['comments'],
docx: this.cliArguments['docx']
});

this.quipProcessor.setLogger(this.Logger);

if(!this.cliArguments['embedded-styles']) {
if(!this.cliArguments['embedded-styles'] && !this.cliArguments['docx']) {
if(this.cliArguments.zip) {
this.zip.file('document.css', documentCSS);
} else {
Expand All @@ -176,11 +181,17 @@ class App {
let foldersToExport = [
//'FOLDER-1'
//'FOLDER-2'
//'EVZAOAW2e6U'
//'UPWAOAAEpFn' //Test
//'EVZAOAW2e6U',
//'UPWAOAAEpFn', //Test
//'bGGAOAKTL4Y' //Test/folder1
//'EJCAOAdY90Y', // Design patterns
//'NBaAOAhFXJJ' //React
];

if(this.cliArguments['folders']) {
foldersToExport = this.cliArguments['folders'];
}

await this.quipProcessor.startExport(foldersToExport);

this.Logger.debug(this.quipProcessor.quipService.stats);
Expand Down
62 changes: 28 additions & 34 deletions lib/QuipProcessor.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -280,6 +280,20 @@ class QuipProcessor {
return `<div class='messagesBlock'>${html}</div>`;
}

async _processDocumentThreadDocx(quipThread, path) {
const docx = await this.quipService.getDocx(quipThread.thread.id);
if(docx) {
this.saveCallback(docx, sanitizeFilename(`${quipThread.thread.title.trim()}.docx`), 'BLOB', path);
}
}

async _processDocumentThreadXlsx(quipThread, path) {
const xlsx = await this.quipService.getXlsx(quipThread.thread.id);
if(xlsx) {
this.saveCallback(xlsx, sanitizeFilename(`${quipThread.thread.title.trim()}.xlsx`), 'BLOB', path);
}
}

async _processDocumentThread(quipThread, path) {
const pathDeepness = path.split("/").length-1;
let threadHtml = quipThread.html;
Expand Down Expand Up @@ -314,53 +328,32 @@ class QuipProcessor {
this.saveCallback(wrappedHtml, sanitizeFilename(`${quipThread.thread.title.trim()}.html`), 'THREAD', path);
}

async _processSlidesThread(quipThread, path) {
const blob = await this.quipService.getPdf(quipThread.thread.id);
if(blob) {
const fileName = sanitizeFilename(`${quipThread.thread.title.trim()}.pdf`);
this.saveCallback(blob, fileName, "BLOB", `${path}`);
} else {
this.logger.warn("Can't load Slides as PDF, thread.id=" + quipThread.thread.id +
", thread.title=" + quipThread.thread.title +
", thread.type=" + quipThread.thread.type + ", path=" + path);
}
}

async _processSpreadsheetThread(quipThread, path) {
const blob = await this.quipService.getXlsx(quipThread.thread.id);
if(blob) {
const fileName = sanitizeFilename(`${quipThread.thread.title.trim()}.xlsx`);
this.saveCallback(blob, fileName, "BLOB", `${path}`);
} else {
this.logger.warn("Can't load Spreadsheet as PDF, thread.id=" + quipThread.thread.id +
", thread.title=" + quipThread.thread.title +
", thread.type=" + quipThread.thread.type + ", path=" + path);
}
}

async _processThread(quipThread, path) {
this.threadsProcessed++;

if(!quipThread.thread) {
const quipThreadCopy = Object.assign({}, quipThread);
quipThreadCopy.html = '...';
this.logger.error("quipThread.thread is not defined, thread=" + JSON.stringify(quipThreadCopy, null, 2) + ", path=" + path);
return;
}

if(['document', 'spreadsheet'].includes(quipThread.thread.type)) {
await this._processDocumentThread(quipThread, path);
}
/*else if(quipThread.thread.type === 'slides') {
this._processSlidesThread(quipThread, path);
} else if(quipThread.thread.type === 'spreadsheet') {
this._processSpreadsheetThread(quipThread, path);
}*/
else {
if(!['document', 'spreadsheet'].includes(quipThread.thread.type)) {
this.logger.warn("Thread type is not supported, thread.id=" + quipThread.thread.id +
", thread.title=" + quipThread.thread.title +
", thread.type=" + quipThread.thread.type + ", path=" + path);
return;
}

this.threadsProcessed++;
if(this.options.docx) {
if(quipThread.thread.type === 'document') {
await this._processDocumentThreadDocx(quipThread, path);
} else {
await this._processDocumentThreadXlsx(quipThread, path);
}
} else {
await this._processDocumentThread(quipThread, path);
}
}

async _processFile(html, file, path, asImage=false) {
Expand Down Expand Up @@ -513,6 +506,7 @@ class QuipProcessor {
}

for(const index in quipFolders) {
this.foldersTotal++;
await this._countThreadsAndFolders(quipFolders[index], "");
}

Expand Down
6 changes: 6 additions & 0 deletions lib/QuipService.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ class QuipService {
getBlob_count: 0,
getPdf_count: 0,
getXlsx_count: 0,
getDocx_count: 0,
getCurrentUser_count: 0,
getThreadMessages_count: 0,
getUser_count: 0
Expand Down Expand Up @@ -80,6 +81,11 @@ class QuipService {
return this._apiCallBlob(`/threads/${threadId}/export/pdf`);
}

async getDocx(threadId) {
this.stats.getDocx_count++;
return this._apiCallBlob(`/threads/${threadId}/export/docx`);
}

async getXlsx(threadId) {
this.stats.getXlsx_count++;
return this._apiCallBlob(`/threads/${threadId}/export/xlsx`);
Expand Down
Loading

0 comments on commit b00c034

Please sign in to comment.