-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
d0e3caf
commit 9367858
Showing
2 changed files
with
67 additions
and
3 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,68 @@ | ||
# io | ||
- Utilities to load file, save file, zip file | ||
- Export from database to file | ||
- Import from file | ||
## File Reader | ||
- File Stream Reader | ||
- Delimiter (CSV format) File Reader | ||
- Fix Length File Reader | ||
## File Writer | ||
- File Stream Writer | ||
#### Delimiter (CSV format) Transformer | ||
- Transform an object to Delimiter (CSV) format | ||
- Transform an object to Fix Length format | ||
|
||
## Import Data | ||
### Import Flow | ||
![Import flow with data validation](https://cdn-images-1.medium.com/max/800/1*Y4QUN6QnfmJgaKigcNHbQA.png) | ||
|
||
#### Layer Architecture | ||
- Popular for web development | ||
|
||
![Layer Architecture](https://cdn-images-1.medium.com/max/800/1*JDYTlK00yg0IlUjZ9-sp7Q.png) | ||
|
||
#### Hexagonal Architecture | ||
- Suitable for Import Flow | ||
|
||
![Hexagonal Architecture](https://cdn-images-1.medium.com/max/800/1*nMu5_jZJ1omzIB5VK5Lh-w.png) | ||
|
||
#### Based on the flow, there are 4 main components (4 main ports): | ||
- Reader, Validator, Transformer, Writer | ||
##### Reader | ||
Reader Adapter Sample: File Reader. We provide 2 file reader adapters: | ||
- Delimiter (CSV format) File Reader | ||
- Fix Length File Reader | ||
##### Validator | ||
- Validator Adapter Sample: Schema Validator | ||
- We provide the Schema validator based on GOLANG Tags | ||
##### Transformer | ||
We provide 2 transformer adapters | ||
- Delimiter Transformer (CSV) | ||
- Fix Length Transformer | ||
##### Writer | ||
We provide many writer adapters: | ||
- SQL Writer: to insert or update data | ||
- SQL Inserter: to insert data | ||
- SQL Updater: to update data | ||
|
||
- SQL Stream Writer: to insert or update data. When you write data, it keeps the data in the buffer, it does not write data. It just writes data when flush. | ||
- SQL Inserter: to insert data. When you write data, it keeps the data in the buffer, it does not write data. It just writes data when flush. Especially, we build 1 single SQL statement to improve the performance. | ||
- SQL Updater: to update data. When you write data, it keeps the data in the buffer, it does not write data. It just writes data when flush. | ||
|
||
- Mongo Writer: to insert or update data | ||
- Mongo Inserter: to insert data | ||
- Mongo Updater: to update data | ||
|
||
- Mongo Stream Writer: to insert or update data. When you write data, it keeps the data in the buffer, it does not write data. It just writes data when flush. | ||
- Mongo Inserter: to insert data. When you write data, it keeps the data in the buffer, it does not write data. It just writes data when flush. | ||
- Mongo Updater: to update data. When you write data, it keeps the data in the buffer, it does not write data. It just writes data when flush. | ||
|
||
## Installation | ||
Please make sure to initialize a Go module before installing core-go/io: | ||
|
||
```shell | ||
go get -u github.com/core-go/io | ||
``` | ||
|
||
Import: | ||
```go | ||
import "github.com/core-go/io" | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters